Structured output

response_format with a json_schema is honored natively, including the first-call compile that can be slow enough to time out.

response_format: { type: "json_schema", … } is honored and forwarded to the provider's own schema-constrained mode, so the answer is constrained as it is generated rather than coaxed by a prompt.

const completion = await client.chat.completions.create({
  model: "nova-lite",
  messages: [{ role: "user", content: "Extract the city and the temperature." }],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "weather_reading",
      schema: {
        type: "object",
        properties: {
          city: { type: "string" },
          celsius: { type: "number" },
        },
        required: ["city", "celsius"],
        additionalProperties: false,
      },
    },
  },
});
 
const reading = JSON.parse(completion.choices[0]!.message.content!);

json_schema takes { name, schema, description?, strict? }.

The three response_format values

json_object is refused rather than approximated because every approximation is dishonest in a way you couldn't detect. There is no schema-less "any JSON" mode to map it onto: a synthesized { type: 'object' } schema would constrain the answer to something you never asked for, and appending "answer in JSON" to the prompt would change the prompt, change the input tokens billed, and still not guarantee valid JSON, which is precisely the guarantee json_object sells.

strict: false still gets the strong guarantee

The provider constrains the output unconditionally, because it has no non-strict mode. So strict is accepted at either value, and strict: false gets a stronger guarantee than OpenAI promises for it, never a weaker one. Refusing it would break every client that sends OpenAI's own default.

(Don't confuse it with tools[].function.strict, which is a different field and a 400.)

The first call with a new schema can be slow

⚠ The provider compiles the schema's grammar the first time it sees it, documented as taking up to a few minutes, and caches it for 24 hours. That compile happens inside your synchronous call, while a wallet reservation is held, and it can exceed the request budget. When it does:

• the call surfaces as a 502 UPSTREAM_PROVIDER_ERROR;
• the reservation is released and nothing is charged;
• steady-state requests reuse the cached grammar and are fast.

So warm a new schema once, off a user's critical path, before shipping it, and don't treat the first 502 as a broken schema.

What isn't validated locally

The provider accepts a documented subset of JSON Schema Draft 2020-12: no recursive schemas, no external $ref, no numeric or string constraints, and no additionalProperties other than false. Fleeex deliberately does not re-implement that subset check, because it moves on the provider's side, and a false 400 on a schema the provider would have accepted is worse than an accurate refusal upstream.

Consequences:

• A schema outside the subset is a 502 UPSTREAM_PROVIDER_ERROR, with the reservation released and nothing charged.
• Support is per model. A model that doesn't do structured output at all refuses the call the same way. Test the model you intend to ship on.

Validate the body anyway

The provider can stop with a "malformed model output" reason that has no OpenAI equivalent; it maps to finish_reason: "stop" like every other unrecognized stop reason. A dedicated value would have to be a Fleeex extension to a field OpenAI clients switch on, which the wire contract doesn't allow.

That's detectable, since the body won't parse against your schema, so it is not a silent wrong answer. Parse defensively:

const raw = completion.choices[0]?.message.content;
const parsed = raw ? safeParse(schema, raw) : undefined;
if (!parsed) {
  // Retry, or fall back: the model stopped without producing a conforming body.
}

The schema costs input tokens

It's serialized once and forwarded as a string, and those bytes are prompt content: they are counted in the estimate that sizes the wallet reservation, and in the tokens billed. A large schema is paid for on every call that carries it.

Bounds

An undeclared sub-field inside json_schema is a 400 naming it.