Structured Outputs

Contents

Structured Outputs#

Materialized’s Batch API allows you to specify a JSON schema for the outputs of your inference job. This can be useful if you want to ensure that the outputs of your inference job are in a specific format, or if you want to extract specific information from the outputs.

To specify a JSON schema for the outputs of your inference job, you can pass the json_schema parameter to the API. The JSON schema must follow the json-schema.org specification.

Example#

import requests
import json

url = "https://api.materialized.dev/batch-inference"

json_schema = {
    "type": "object",
    "properties": {
        "is_aviation_related": {
            "type": "boolean"
        },
        "answer_justification": {
            "type": "string"
        }
    },
    "required": ["is_aviation_related", "answer_justification"]
}

texts = ['The airplane is flying at an altitude of 30,000 feet.', 'The capital of France is Paris.', 'The best way to cook a steak is on a grill.']

prompts = ["Is the following sentence aviation related? Please provide a justification for your answer. " + text for text in texts]

params = {
    "model": "llama-3.1-8b",
    "inputs": prompts,
    "json_schema": json_schema
}
headers = {
    "Authorization": "Key <YOUR_API_KEY>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=params, headers=headers)
results = response.json()

print(results)
Contents