Structured Output

What it is

In the context of Generative AI, Structured Output is the ability of a language model (LLM) to provide responses that follow a predefined and rigorous schema, typically in JSON (JavaScript Object Notation) format. While a normal interaction with ChatGPT results in continuous and conversational text, Structured Output forces the AI to organize information into specific fields, like invisible tables where each piece of data has its correct place.

For a business, this means moving from the 'playing' stage with AI to the 'integrating' stage. It stops being a chat and becomes a software component that communicates directly with billing software, CRM, or inventory without formatting errors.

How it works

Technically, Structured Output works through a technique called 'constrained sampling'. During the process of generating words (tokens), the server running the AI model checks if the next word violates the rules of the schema defined by the developer. If the schema requires a number for the 'price' field, the model is forbidden from generating letters in that location.

In the past, this was done purely through Prompt Engineering, asking the AI: 'Respond only in JSON'. However, models frequently failed, adding extra text or forgetting to close brackets. Today, the main APIs (OpenAI, Anthropic, Google) offer native modes where the model guarantees 100% adherence to the provided schema, making the response fully machine-readable.

When to use

Structured Output is indispensable in any scenario where the AI output needs to be consumed by another automated system. If the idea is just to read the response on the screen, free text serves the purpose. If the goal is for the response to automatically fill out a customer profile, structured output should be used.

Common use cases include:

1. Data Extraction: Converting an order email into an object with 'customer_name', 'quantity', and 'sku_reference'.
2. Classification: Analyzing support tickets and returning only a fixed category (e.g., Suggestion, Complaint, Technical Error).
3. AI Agents: When an agent needs to decide which tool to use (Function Calling), it uses Structured Output to send the correct parameters to that tool.
4. Sentiment Analysis: Transforming product reviews into numerical scores from 1 to 5.

Common errors

The most frequent error is designing schemas that are too complex or ambiguous. If you ask the AI to fill in a field called 'details' without specifying what should be there, the model may hallucinate or include irrelevant information.

Another mistake is forgetting that Structured Output does not prevent the hallucination of facts; it only guarantees formatting. If you ask for the VAT number of a company that the model does not know and require a numerical format, the AI may invent a valid but false number. Finally, companies often neglect error handling: even if the JSON arrives perfectly, business logic must verify if the extracted data makes sense (e.g., if a delivery date is not in the past).

Practical example for an SME

Imagine a Portuguese SME that manages short-term rentals and receives hundreds of monthly messages in several languages through Booking or Airbnb. Without Structured Output, an employee would have to read each message and copy the data into an Excel sheet.

With Structured Output, the company sends the message text to the AI with a schema that defines: check_in_date, check_out_date, number_of_guests, and special_requests.

The AI receives the message: 'Hello, we will arrive on August 15th, we are 3 adults and a baby. Can you provide a crib?'

The Structured Output returns:

{
  "check_in": "2024-08-15",
  "total_people": 4,
  "crib": true,
  "language": "PT"
}

This JSON enters directly into the SME's reservation management system, blocking the dates and alerting the cleaning team to prepare the crib, without anyone having to touch the keyboard.

Frequently Asked Questions

Q: Does Structured Output use more tokens? A: Yes, slightly. The JSON syntax (braces, quotes, and field names) counts as tokens. However, the reliability it brings largely compensates for the extra marginal cost.

Q: Do I need to know how to program to use this? A: For technical implementation in an automated workflow, yes. But no-code automation tools (such as Make or Zapier) already allow configuring structured formats visually.

Q: Can I force the AI to respond in a format other than JSON? A: While JSON is the industry standard, it is possible to force other formats like XML or CSV, although the native support of models to ensure validity is almost always focused on JSON.

Q: What happens if what the user wrote doesn't fit the schema? A: The model will try to adapt the information to the schema as best as possible. If it is impossible (e.g., asking for a date in a text that has no dates), the model may leave the field empty (null) or make an error, so it is important to include a 'notes' or 'status' field for ambiguous cases.

• 01Transform reservation emails into JSON files ready for hotel management software.
• 02Automatically classify digitized invoices by expense category and VAT value.
• 03Extract product names and quantities from handwritten order notes via OCR.
• 04Standardize customer feedback into fixed categories for analysis in BI dashboards.

Want to use Structured Output in your company?

30 minutes, free, no commitment. We map where it fits.

Free AI diagnosis