How to Write Reliable JSON Prompts for LLMs: Schemas, Validation, and Error Handling

Overview

Asking an LLM to “return JSON” is not the same as guaranteeing valid or useful JSON. A model may add an introduction, omit a field, return the wrong data type, include unsupported values, or wrap the object in a Markdown code fence. Reliable AI development treats the prompt as one part of a larger control system: define the expected contract, constrain the response, validate it in application code, and decide what should happen when validation fails.

The most dependable approach combines four layers:

1. A clear output contract: describe the object, required fields, allowed values, and data types.
2. A focused prompt: separate instructions, input data, and output requirements.
3. Machine validation: parse the response and check it against a schema before using it.
4. Recovery and monitoring: handle failures safely and measure changes over time.

Use JSON Schema or an equivalent validation system where possible. A schema turns an informal instruction into an explicit interface that developers, testers, and models can work against. For broader testing guidance, see this LLM prompt evaluation framework and the accompanying LLM evaluation guide.

A practical JSON prompt pattern

A useful prompt states the task, defines the output shape, identifies uncertainty rules, and forbids additional prose. For example:

You extract support-ticket details from the input.

Return one JSON object with exactly these keys:
- category: one of "billing", "technical", "account", or "other"
- urgency: one of "low", "medium", or "high"
- summary: a concise string
- missing_information: an array of strings

If a value cannot be determined, use null where the schema permits it.
Return valid JSON only. Do not add Markdown, commentary, or extra keys.

This is stronger than a vague request because it defines both the positive requirements and the failure behaviour. If your platform supports native structured outputs or tool/function schemas, use those controls in addition to the prompt rather than relying on wording alone.

Checklist by scenario

1. Extracting fields from text

• List every field the application actually needs; remove fields that are merely interesting.
• Specify the type for each field: string, number, boolean, array, object, or null.
• State how missing, ambiguous, or conflicting information should be represented.
• Define whether values should be copied exactly, normalised, or inferred.
• Set limits for free-text fields, such as maximum length or an allowed format.
• Include a stable identifier for the source record outside the model-generated content when possible.

For example, an extraction workflow should distinguish between “the customer requested a refund” and “the refund has been approved”. The first may be a detected intent; the second is a business decision that should not be invented by the model. Prompts for extracting information from PDFs, emails, and tickets benefit from this separation of observed facts and interpretation.

2. Classifying or routing requests

• Use a closed list of labels rather than asking for an open-ended category.
• Explain the boundary between similar labels with short examples.
• Provide an other or unknown option when the input may fall outside the taxonomy.
• Return a confidence or review flag only if your team has defined how it will be used.
• Keep the label values stable so analytics and downstream rules do not break.

Few-shot prompting can help when categories are difficult to distinguish, but examples should demonstrate genuine edge cases rather than add volume for its own sake. Test examples should remain separate from production input and should not accidentally become part of the saved output.

3. Generating records for an API or database

• Mirror the receiving system’s field names and types exactly.
• Define date, time, currency, and identifier formats explicitly.
• Never allow the model to generate trusted permissions, account identifiers, or irreversible commands without independent checks.
• Use application code to add server-owned fields such as timestamps, user IDs, and audit metadata.
• Validate uniqueness, referential integrity, and business rules after JSON parsing.

Valid syntax does not make a record safe or correct. A JSON object can pass schema validation while containing an invalid product code, an impossible date, or an instruction that should not be executed.

4. Producing nested or repeated data

• Show the required nesting with a small example.
• Specify whether arrays may be empty and whether duplicate items are allowed.
• Define ordering only when the consuming system depends on it.
• Set sensible limits on array length and nesting depth.
• Validate each item, not just the outer object.

Deep structures are harder to review and more likely to contain inconsistent values. If a workflow can use several smaller objects instead of one large response, consider prompt chaining: extract, validate, then transform in separate steps.

What to double-check

Schema design

Review required versus optional properties carefully. A field should be required when the application cannot proceed without it, not simply because it would be convenient to have. Decide whether an absent value, an explicit null, and an empty string have different meanings. They often do.

Use enumerations for controlled values and numeric constraints where appropriate. If a string follows a pattern, such as an internal reference, validate that pattern in code. Keep descriptions close to the relevant property so the prompt and schema do not drift apart.

Prompt boundaries

Place untrusted input in clearly marked delimiters and tell the model that the input is data to analyse, not instructions to follow. This is especially important when processing emails, web pages, documents, or support tickets that may contain text such as “ignore the previous instructions”. Treat the content as potentially adversarial even when it appears ordinary.

Keep the system-level requirements concise and stable. Put task-specific input in a separate user message when the API allows it. This structure makes prompt versioning, testing, and troubleshooting easier.

Validation and error handling

Parse the response as JSON before reading any field. Do not repair arbitrary text with a series of string replacements and then trust the result. If parsing fails, record the failure, preserve the original response for debugging under appropriate data-handling rules, and apply a bounded recovery path.

A recovery path might request the same output again with the validation error summarised, use a stricter structured-output mode, or route the item to manual review. Limit retries and avoid sending sensitive content into a different service without an explicit reason. After a successful parse, run schema validation and then business-rule validation.

Useful failure categories include syntax error, missing property, wrong type, invalid enum value, unsupported extra property, and semantic inconsistency. Categorising errors reveals whether the problem belongs in the prompt, schema, model configuration, input cleaning, or application code.

Common mistakes

• Relying on “JSON only”: This instruction helps, but it is not validation. Always parse and validate.
• Changing the schema without changing tests: A new field, renamed enum, or altered nullability can silently break consumers.
• Overloading one prompt: Extraction, judgement, rewriting, and database mutation in one response make failures difficult to isolate.
• Using examples that contradict the schema: Models may copy an example’s shape or value even when the written rules say otherwise.
• Allowing unrestricted extra keys: Unexpected properties can create ambiguity or become unsafe if downstream code acts on them.
• Confusing confidence with truth: A model-generated confidence value is not evidence of factual correctness unless calibrated and tested for the specific task.
• Ignoring empty and unknown cases: Forcing a value when the source does not support one creates false precision.
• Testing only easy inputs: Include incomplete, contradictory, long, multilingual, malformed, and adversarial examples.
• Printing secrets in debugging logs: Store only what your data-handling requirements permit, and redact sensitive fields before logging.

A small, representative evaluation dataset is more useful than a handful of successful demonstrations. Include expected structure, acceptable alternatives, and the conditions that should trigger review. The guide on building a prompt evaluation dataset can help organise this process.

When to revisit

Review JSON prompts and schemas before a seasonal planning cycle, a major workflow change, or any release that changes the model, API settings, input sources, or downstream consumer. Revisit them when a new document type is added, labels are redefined, validation failures increase, or users report that a correctly shaped response is still operationally wrong.

Keep a versioned record of the prompt, schema, model configuration, test set, and validation results. A change should be treated as an interface change when it affects field names, types, required properties, enum values, ordering assumptions, or error handling. Prompt versioning practices make rollbacks and comparisons safer.

Final pre-release checklist

1. Write the expected JSON contract in a schema.
2. Mark required, optional, nullable, and enumerated fields.
3. Separate instructions from untrusted input.
4. Define unknown, missing, and ambiguous-value behaviour.
5. Use native structured-output controls when available.
6. Parse, schema-validate, and business-rule-validate every response.
7. Set bounded retries and a clear manual-review path.
8. Test normal, edge, adversarial, and oversized inputs.
9. Log failure categories without exposing unnecessary sensitive data.
10. Version the prompt, schema, tests, and configuration together.

For practical development work, a JSON formatter and validator can help inspect examples, but it should complement—not replace—automated validation in the application. Reliable structured output comes from the complete workflow: precise prompt engineering, explicit schemas, defensive code, and regular evaluation.