Structured LLM Outputs: JSON Schemas and Function Calling
Large language models are conversational by design, which makes them unreliable as data sources. Ask a model to extract contact details from an email, summarize a support ticket into fields, or classify a product inquiry, and you get prose: courteous, variable, and full of markdown fences. Applications that consume model output need contracts, not chatter. Structured outputs close that gap by constraining model generation to a JSON schema you define, turning the model from a chat partner into a dependable data producer.
This article explains how structured outputs work, when to use function calling versus a structured response format, how to design schemas models actually follow, and how to handle failures gracefully.
Why Free-Form Output Breaks Pipelines
Free-form generation optimizes for fluency, not consistency. The same prompt can return "John Doe" one run and "Doe, John" the next, wrap dates in "as of March 3" on some runs and ISO strings on others, and sprinkle conversational filler around the payload. Downstream code that parses regexes or splits on markers breaks silently, and teams respond with retry loops that multiply latency and cost without fixing the root problem.
A schema acts as a data contract between your prompt and your application. When the model must honor it, every run returns the same keys, types, and shapes, so the rest of your pipeline can consume the result directly.
Structured Outputs versus JSON Mode
JSON mode tells the model to emit valid JSON, but it says nothing about what that JSON should contain. A model in JSON mode can still omit required fields, invent keys, or return strings where numbers are expected.
Structured outputs go further. With a JSON Schema and strict mode enabled, the model must produce exactly the keys you declared, with the types you declared, and nothing extra. additionalProperties: false blocks surprise fields; required guarantees presence; enums restrict values to a fixed list. The result is valid JSON that also conforms to your contract.
Structured outputs are available from GPT-4o onward. JSON mode remains useful for older models, but for schema-bound work, structured outputs are the stricter, safer default.
Two Ways to Request Structure
You can request structured output in two places, and the choice depends on intent:
- Function calling (tools) with
strict: true— best when the model's output should trigger something in your application: updating a record, querying a database, or composing a UI card. The model selects a function and fills its arguments according to your schema. response_formatwith typejson_schema— best when the structure belongs to the model's user-facing answer itself: a quiz generator returning questions and answers, or a tutor returning an explanation plus practice problems.
Use function calling when structure feeds an action; use response_format when structure feeds the reader.
Designing Schemas Models Follow
A model-friendly schema is flat, explicit, and restrictive:
- Name keys plainly (
customer_email, note). Titles and descriptions help the model understand intent. - Prefer one level of nesting. Deeply nested objects increase the chance of skipped or malformed fields.
- Use supported JSON Schema types: string, number, boolean, integer, object, array, enum, and anyOf where supported.
- Mark every field required and set
additionalProperties: false. Strict mode treats any missing or extra field as a violation. - Use enum for fixed categories (priority: low, medium, high) and format or pattern for strings such as email, date-time, or uuid.
- Avoid unsupported features such as dependencies, if/then, or unevaluatedProperties.
Example schema for extracting a support contact:
{
"name": "extract_support_contact",
"strict": true,
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Full name of the person contacting support"
},
"email": {
"type": "string",
"format": "email"
},
"issue": {
"type": "string",
"enum": ["billing", "technical", "account", "other"]
},
"urgency": {
"type": "integer",
"minimum": 1,
"maximum": 5
}
},
"required": ["name", "email", "issue", "urgency"],
"additionalProperties": false
}
}
Code Example: Extracting Structured Data
Here is a minimal Python example using the OpenAI client with strict function calling:
from openai import OpenAI
import json
client = OpenAI()
schema = {
"name": "extract_support_contact",
"strict": True,
"parameters": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string", "format": "email"},
"issue": {
"type": "string",
"enum": ["billing", "technical", "account", "other"]
},
"urgency": {
"type": "integer",
"minimum": 1,
"maximum": 5
}
},
"required": ["name", "email", "issue", "urgency"],
"additionalProperties": False
}
}
response = client.chat.completions.create(
model="gpt-4o",
messages=[{
"role": "user",
"content": "Hi, I'm Maria, [email protected], and my billing card keeps declining."
}],
tools=[{"type": "function", "function": schema}],
tool_choice={
"type": "function",
"function": {"name": "extract_support_contact"}
},
parallel_tool_calls=False,
)
arguments = response.choices[0].message.tool_calls[0].function.arguments
data = json.loads(arguments)
Note parallel_tool_calls=False: structured outputs and parallel tool calls do not combine, so disable it. The arguments string is valid JSON by contract, but wrap json.loads in error handling anyway—network issues and refusals still happen.
Refusals, Errors, and Validation
A schema guarantees shape, not truth. A model can return a syntactically perfect object with a hallucinated email or an urgency of 5 for a trivial request. Validate business rules separately: date ranges, referential integrity, whether an email actually belongs to the customer.
Structured outputs do make failures programmatic. Refusals surface as structured refusal messages rather than garbled JSON, and missing tool calls are easier to detect than a broken regex match. Log refusals, surface them to users when appropriate, and never retry a refused request blindly.
Common Mistakes
- Nesting objects four levels deep. Keep schemas shallow.
- Leaving fields optional. In strict mode, omission fails validation; mark what you need.
- Using loose anyOf everywhere. It weakens the contract the model must honor.
- Running structured outputs with parallel tool calls enabled. Disable
parallel_tool_calls. - Confusing JSON mode with structured outputs. JSON mode validates syntax, not shape.
Trade-offs
Structured outputs cost a little more design work up front and reduce the model's freedom to improvise—which is the point for data tasks and the wrong fit for creative writing, brainstorming, or open-ended explanation. Schema changes also need versioning: adding a required field breaks consumers expecting the old shape. Treat the schema as part of your API and evolve it deliberately.
Use evaluations, not vibes, to decide. Run a sample of real inputs through your schema before shipping, and measure refusal rates and field-completion rates.
Production Checklist
- Pick one recurring extraction or classification task first.
- Write a strict schema with required fields and
additionalProperties: false. - Test with real inputs, including edge cases and refusals.
- Disable parallel tool calls when using structured functions.
- Add business-rule validation after parsing.
- Log schema violations and refusals for continuous improvement.
FAQ
Is structured output the same as JSON mode?
No. JSON mode guarantees valid JSON; structured outputs guarantee your schema.
Which models support it?
Structured outputs are available from GPT-4o onward.
Can I combine it with function calling?
Yes. Use tools with strict: true and tool_choice set to a specific function.
Do I still need validation after parsing?
Yes. The schema covers shape, not business logic or truthfulness.
What if the model refuses?
Refusals are detectable and structured. Log them and respond gracefully instead of retrying blindly.