The workflow model

We need a model that runs the workflow.

Many useful AI features are workflows. A support ticket is classified, checked for urgency, assessed for a refund amount if it is a billing issue, and answered using the category and urgency. Some decisions can run together; others need earlier answers.

Chat models return messages. Decision models return decisions. To connect those answers into a process, developers usually write the workflow in code around the model.

TypeLLM makes the workflow the unit of inference. You declare the decisions, their types and how they depend on each other. TypeLLM compiles and executes the graph, returning typed results in one API call. TypeLLM runs the graph; the language model works within each step.

SQL for workflows

With SQL, you declare the result you want. The database chooses how to execute the query. You do not write the join loops or schedule each read yourself.

A workflow model is the same split, for AI workflows. Here is the ticket workflow, declared:

{
  "category": {"type": "string", "enum": ["billing", "technical", "account"]},
  "urgent":   {"type": "boolean"},
  "refund":   {"type": "number", "when": {"category": "billing"},
               "instructions": "Refund in dollars."},
  "reply":    {"type": "string", "depends_on": ["category", "urgent"],
               "instructions": "One sentence to the customer."}
}

Category and urgency can run together. The refund is generated only for billing tickets. The reply reads category and urgency; it does not wait for the refund.

Your application still checks payment records, approves any refund and sends the reply. The graph handles the inference that prepares those actions.

For the ticket, "I upgraded to the Pro plan on the 3rd and my card was charged twice, $49 each time. I can't find a way to get the second charge back in the billing page. Please help, this is my company card," the call returned:

{
  "category": "billing",
  "urgent": true,
  "refund": 49,
  "reply": "I am sorry to hear about the duplicate charge on your company card, and I have flagged this as urgent for our billing team to process your refund immediately."
}

Four steps, one API call, every answer in its type.

What knowing the whole graph changes

Beyond one call, TypeLLM sees the dependencies before it runs a step:

The context is computed once. Every step reuses it; dependent steps extend it with the earlier answers they need; a branch that when skips is never generated. You do not assemble a new prompt for each step.

Types are enforced during generation, so there is no parse-and-retry loop in your application.

Probabilities come from the model itself. The probability of each option is read from the model as it decides. These are probabilities over the declared choices, rather than confidence scores the model was asked to write down.

Parallel and sequential steps are handled automatically. Steps that need nothing from each other run in parallel; dependent steps run in order once their inputs are ready. No client round trips between steps, and no ordering logic of your own.

A first measurement

In an initial 15-run measurement of the ticket workflow above, one TypeLLM call was compared with four serial calls, with earlier answers pasted into later prompts:

In that test, the serial version used 2.1× the reported input tokens and took about 1.9× as long.

What it takes

An answer can become a dependent step's input. If it comes back as "Billing" instead of "billing", or "$49" instead of 49, the steps that consume it receive the wrong input. In a single call, a malformed answer is an error; in a workflow, it spreads.

So the foundation is type safety, by construction: every answer is held to its type as it is generated, so dependent steps receive values that satisfy the declared contract. That contract is about structure. A refund of 49 has the right type; it can still be the wrong amount. A reply can be a valid string and still make an unsupported claim, like the reply above, which says it flagged the ticket. Instructions, evaluation and application checks determine whether those answers are useful.

On that foundation, a workflow model has to handle different kinds of step:

Generate the types a workflow needs. Free text for a reply, a number for a refund, an integer, a boolean, an object for a record, an array as long as the input says. Each one in its type, every time.

Make decisions, with probability and confidence. Pick one of a set of options, and return the probability of each and a confidence for the choice, so the workflow can act on how sure it is: continue when it is sure, and route the rest for review.

Think when it is hard. Reason before answering when a step needs it, like adding up the lines before saying whether an invoice adds up. With "thinking": "auto", the model judges each step on each call and thinks only as much as it needs: easy steps answer at once, and hard ones get more computation.

Where agents fit

An agent is useful when the next action has to be discovered at run time: investigate a problem, choose a tool, inspect its result, and decide what to try next.

A declared workflow is useful when the process is already known. You know which checks an invoice must pass and which decisions precede a refund review. You want the model to make those decisions within a process you can read and test.

The two can work together. An agent can call a declared workflow as one operation, and an application can place tool calls between inference graphs.

Where it goes

Today: branch on answers, and on how sure they are. refund runs when category is billing. It can also run only when that decision has a confidence of at least 0.8, and send the rest to a person. The graph follows the model's certainty, not just its answer.

Next: the workflow as a probabilistic graph. Today each step passes one answer to its dependents. We want downstream decisions to account for uncertainty in their inputs, rather than treating every selected answer as certain. A reply built on a category that was 60% billing should retain that uncertainty. Passing distributions over categorical choices is one path we are exploring.

Then: decisions under uncertainty. A workflow could use those distributions to choose among declared next steps, weighing the cost of a check against the cost of being wrong. Take fault diagnosis: rank possible causes of a failed service, select the check expected to narrow them down most, and conclude when the evidence meets a stopping rule. You would declare the available checks and stopping rules; inference would select among them.

This is what we are building. The goal is a model that runs your workflow and tells you where it might be wrong.

What you might ask

Isn't this just server-side orchestration? Moving a loop to a server is not enough: orchestration becomes part of the inference interface, with typed decoding, shared context and model-derived probabilities.

Isn't this just structured outputs? Structured outputs define an answer's shape. TypeLLM also defines execution: what each field reads, when it runs, and when it is skipped.

What about steps that call my own tools? One call holds the steps the model decides: reading, classifying, extracting, writing. A step that needs your database or an API sits between calls, and the answers it needs come back typed.

Declare the process once. Get its typed results in one call. Try a ticket workflow in the playground, or start with the quick start.