LLMs with type-safe generation

Build with a coding agent

Paste this into Claude Code, Cursor, or any agent that can read a URL.

Read https://typellm.ai/SKILL.md and follow it to set up TypeLLM in this project.

Manual setup

1. Serve a model

Start a compatible model with SGLang and enable prefix caching. This example uses Qwen3.8-27B; follow its SGLang deployment guide and use the same model ID in the client.

2. Run TypeLLM

Install

pip install -U typellm

Connect to your SGLang HTTP endpoint:

Create a client

from typellm import TypeLLMClient

client = TypeLLMClient(
    "http://127.0.0.1:30000",
    model="Qwen/Qwen3.8-27B",
)

Example request:

Python

result = client.generate(
    context="""
    Receipt from Hilton London
    Total: £324.50
    Employee travelled to London for a client meeting.
    """,
    questions={
        "merchant": {
            "type": "string",
            "instructions": "Return only the merchant name.",
        },
        "total": {
            "type": "number",
            "instructions": "Extract the total amount in GBP.",
        },
        "expense_type": {
            "type": "string",
            "enum": ["meal", "travel", "equipment"],
            "instructions": "What type of expense is this?",
        },
        "reimbursable": {
            "type": "boolean",
            "instructions": "Should this expense be reimbursed?",
        },
        "confidence": {
            "type": "number",
            "enum": [0.0, 0.25, 0.5, 0.75, 1.0],
            "instructions": "How confident are you?",
        },
    },
)

print(result)

Example return:

Python dictionary

{
    "merchant": "Hilton London",
    "total": 324.5,
    "expense_type": "travel",
    "reimbursable": True,
    "confidence": 0.75,
}