TypeLLM

Your type-safe workflow in one API call

September 24, 2026

A customer writes: “I was charged twice. My renewal is tomorrow. Please help.” To recommend what happens next, a support agent needs two things: the issue category and whether it is urgent.

TypeSafe's Jev returns typed decisions like these, but only for questions that are conditionally independent given the input. Its documentation states that “every question is evaluated in parallel and in isolation against the same state.” Category and urgency can each be read from the ticket, so they fit. A recommendation that should follow from both does not: it has to be a second request, with your code passing the first two answers along.

With TypeLLM, you define all three as typed questions and connect them with depends_on. One generate() call runs the workflow and returns the results.

Define the dependencies

The support ticket feeds issue category, urgency, and handling advice; category and urgency also feed handling advice.

category and urgent independently read the ticket. Once both finish, handling_advice uses their values and the original context to recommend next steps. The intermediate decisions remain available for filtering tickets, setting priorities, and reviewing the recommendation.

Run the workflow

With a compatible model running on SGLang:

from typellm import TypeLLMClient

client = TypeLLMClient(
    "http://127.0.0.1:30000",
    model="Qwen/Qwen3.8-27B",
)

result = client.generate(
    context=(
        "Customer: I was charged twice. My renewal is tomorrow. "
        "Policy: route issues to the matching support team. "
        "Duplicate charges before renewal are urgent."
    ),
    questions={
        "category": {
            "type": "string",
            "enum": ["billing", "technical", "account"],
            "instructions": "Classify the issue.",
        },
        "urgent": {
            "type": "boolean",
            "instructions": "Is this urgent under the policy?",
        },
        "handling_advice": {
            "type": "string",
            "instructions": (
                "Suggest next steps using the category, urgency, and policy. "
                "Do not claim actions were already taken."
            ),
            "depends_on": ["category", "urgent"],
        },
    },
)

Example output:

{
  "category": "billing",
  "urgent": true,
  "handling_advice": "Prioritize for billing support. Check whether both charges settled, review the upcoming renewal, and determine the appropriate correction before confirming it to the customer."
}

The application receives an enum value, a Boolean, and a string. The recommendation is generated whether urgent is true or false; the value affects its content, not whether the step runs.

What TypeLLM handles

  • Execution order. The default execution="auto" detects dependencies and runs the graph in layers. Independent enum and Boolean questions within a layer can be scored together.
  • Context. Each question receives the original input and the selected values of its direct and transitive dependencies.
  • Validation. Unknown dependencies, self-dependencies, duplicates, and cycles are rejected before model execution.
  • Results. Returned keys follow declaration order, even when execution order differs.

Reuse shared prefixes

TypeLLM builds each prompt by extending an earlier one, so SGLang can reuse the cached KV state for the shared part instead of computing it again.

  • Independent questions share the input. category and urgent both start from the same ticket. The ticket is prefilled once; each question adds only its own instructions.
  • A dependent question continues its longest parent. handling_advice has two parents. TypeLLM picks the parent whose prompt is longest (the ticket, that parent's question, and its answer) and reuses it as the prefix. It then appends one new turn with every dependency value as JSON, followed by the question's instructions.

If category has the longer prompt, handling_advice looks like this:

reused    [ticket] [category question] [answer: "billing"]
appended  Dependency results (JSON): {"category": "billing", "urgent": true}
          Suggest next steps using the category, urgency, and policy. ...

Only one branch can be reused. The KV state for urgent was computed after a different preceding text, so it cannot be attached to the category branch. Its value still reaches handling_advice through the appended JSON. The ticket and the longer branch are already cached, so only the appended turn needs new prefill. How much this saves depends on the serving configuration and whether the prefix is still in the cache.

Define the outputs, declare their dependencies, and let TypeLLM handle the execution. See the execution docs for details, or explore TypeLLM on GitHub.

← all posts