Execution & dependencies

The default execution="auto" batches independent fields. When a field declares depends_on, it follows the dependency graph.

Within a batch or a DAG layer, strings and choices are each generated in one batched request, numeric fields decode in lockstep with one batched request per digit, and thinking runs for every field in one batched request.

Batch

Each field reads the original context without seeing the other fields’ answers. Here, quantity, total and paid can all be extracted independently. With no dependencies, the default auto mode behaves the same way.

result = client.generate(
    context="Bought 3 notebooks for £12.50. Paid in full.",
    execution="batch",
    questions={
        "quantity": {
            "type": "integer",
            "instructions": "How many notebooks were bought?",
        },
        "total": {
            "type": "number",
            "instructions": "Extract the total amount in GBP.",
        },
        "paid": {
            "type": "boolean",
            "instructions": "Was the purchase paid in full?",
        },
    },
)
print(result)

{"quantity": 3, "total": 12.5, "paid": True}

Sequential

Fields run in declaration order. In this example, next_action sees the selected department, then summary sees both earlier results. Set execution="sequential"; no depends_on is needed.

result = client.generate(
    context="A customer reports two charges for the same order.",
    execution="sequential",
    questions={
        "department": {
            "type": "string",
            "enum": ["billing", "technical"],
            "instructions": "Which department should handle this?",
        },
        "next_action": {
            "type": "string",
            "maxLength": 120,
            "instructions": "Using the selected department, suggest one action.",
        },
        "summary": {
            "type": "string",
            "maxLength": 160,
            "instructions": "Summarize the selected department and next_action in one sentence.",
        },
    },
)
print(result)

{
    "department": "billing",
    "next_action": "Check the duplicate charge and refund it if confirmed.",
    "summary": "Billing should check the duplicate charge and refund it if confirmed.",
}

Dependency graph

Use depends_on when only specific earlier results should be visible, rather than the full sequential history.

Dependency rules

• Supported execution modes are auto, batch, sequential and dag. You can override the client mode on a generate call.
• Declaring any depends_on, including an empty list, makes auto mode select DAG execution. Explicit batch or sequential mode combined with dependencies raises SchemaError.
• In DAG mode, a missing dependency list or depends_on=[] creates a root field. It sees only the original context.
• Parents may be declared later in the input. Unknown names, duplicate dependencies, self-dependencies and cycles are rejected before inference.
• Fields in the same topological layer run independently. Children receive their direct and transitive ancestors’ results, not unrelated siblings.
• Dependencies do not interpolate instructions, change enum candidates or conditionally skip fields.

result = client.generate(
    context="A customer reports two charges for the same order.",
    questions={
        "department": {
            "type": "string",
            "enum": ["billing", "technical"],
            "instructions": "Which department should handle this?",
        },
        "next_action": {
            "type": "string",
            "maxLength": 120,
            "depends_on": ["department"],
            "instructions": "Suggest one action for the selected department.",
        },
    },
)

Dependencies determine execution order; returned keys keep their original declaration order. A failed layer stops downstream execution. For multiple parents, TypeLLM reuses one parent prefix and supplies all dependency values. SGLang manages the resulting KV cache; KV tensors from separate branches are not merged.

Permutation averaging

Enum fields can independently set permutations in any execution mode. Variants share the same visible context and are scored in a batch. Their probabilities are averaged before selecting one value. Sequential successors and DAG children receive that value in the original option representation; other fields execute normally.

See the per-question API and example.