Client reference — Docs

The Python client, TypeLLMClient. The hosted API's endpoints, fields, limits and HTTP errors are in the API reference.

Client settings

Setting	Default	Purpose
`api_key`	`TYPELLM_API_KEY`	Your key from Dashboard, API keys. It makes the client call the hosted API; without it, the TYPELLM_API_KEY environment variable is used.
`base_url`	`"https://api.typellm.ai"`	The API's address.
`model`	`None`	The public model; `typellm-latest` when left out.
`temperature`	`0`	0 gives each field its most likely answer; above 0 samples at that temperature.
`seed`	`None`	Client random-number seed.
`timeout`	`None`	Seconds for each call; the service uses 60 when unset and allows at most 90.
`max_retries`	`2`	Retries after HTTP 429, a 5xx other than 504 or a connection error, waiting as the service asks (Retry-After) or backing off from half a second. 0 turns retries off.

generate arguments

Supply exactly one of context or its alias state. The value must be a string.
Pass questions, a mapping of names to field definitions; see the question definition.
Optionally pass images, a list of file paths, data: URIs, bytes or PIL images. They precede the context. URLs are not fetched; pass the file instead. See Images.
temperature and seed may override client settings for one request.
timeout caps the whole call in seconds and raises GenerationTimeout.
Every declared field is evaluated, unless a when skips it; required does not make other fields optional.

After a call

generate() returns what the HTTP API does: the typed answers in .result, in declaration order, each thinking field's reasoning in .thinking, the level each "auto" field got in .thinking_effort, the fields a when skipped in .skipped, and the call's tokens in .usage. One client can be shared by many threads.

.usage holds what the call is billed for: input_tokens, what you sent counted once (the context, the questions as JSON, and the images), and thinking_tokens, the reasoning. Answers are free; see Pricing.

.status is "completed", or "needs_input" when the call stopped for fields answered outside the model, listed in .needs_input. client.resume(response, answers) goes on with your answers and returns the call's answers so far, the earlier ones included; it takes a timeout too. See Tools and human input.

Limits

The service's limits are in the API reference.

Errors

SGLangError: an HTTP error from the API, with the status in .status, such as 400 for an invalid question or a limit exceeded; see Errors. The client retries a 429, a 5xx other than 504 and a connection error first (max_retries).
GenerationTimeout: the call passed its timeout (HTTP 504).
ValueError: an invalid client setting or argument, before anything is sent.

To run TypeLLM on your own model, see self-hosting on GitHub.