API reference — Docs

The hosted API has one endpoint that does the work, POST /v1/generate. The Python client calls it for you; any language can call it directly. Two more endpoints take other APIs' requests; see Compatible endpoints.

Authentication

The base URL is https://api.typellm.ai. Send your key from Dashboard, API keys as a bearer token on every request:

curl

curl https://api.typellm.ai/v1/models \
  -H "Authorization: Bearer $TYPELLM_API_KEY"

POST /v1/generate

Answers every question about the context in one call.

Request

curl https://api.typellm.ai/v1/generate \
  -H "Authorization: Bearer $TYPELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "Receipt from Hilton London. Total: £324.50",
    "questions": {
      "total": {"type": "number", "instructions": "Total in GBP."},
      "expense_type": {"type": "string", "enum": ["meal", "travel", "equipment"]},
      "policy_ok": {"type": "boolean", "instructions": "Within the travel policy?",
                    "thinking": true, "thinking_budget": 1024}
    },
    "options": {"temperature": 0, "seed": 42},
    "timeout": 60
  }'

Request body

Field	Type	Description
`context`	string, required	The text to answer from.
`questions`	object, required	Field names mapped to question definitions (below). At most 64.
`images`	array of strings	Base64 `data:image/...` URIs; no URLs. At most 8, each up to 10 MB. See Images.
`model`	string	A public model; `typellm-latest` when left out.
`options.temperature`	number ≥ 0	0 (the default) gives each question its most likely answer; above 0 samples at that temperature.
`options.seed`	integer	Fixes the service's random choices. Results can still vary slightly with batching.
`timeout`	number	Seconds for the call: 60 by default, at most 90.
`resume_state`	string	Goes on with a call that stopped for answers from outside the model: its `resume_state`, sent with `answers` and in place of `context`, `questions` and `images`. See Tools and human input.
`answers`	object	With `resume_state`: an answer for each question in the call's `needs_input`.

Question definition

Each question is a JSON Schema field with a few TypeLLM keys. maxLength, minLength, pattern, format, minimum and maximum are not accepted; string answers stop at 128 tokens.

Key	Type	Description
`type`	string	`string`, `number`, `integer`, `boolean`, `object` or `array`; `["string", "null"]` allows no answer. See Output types.
`enum`	array	The only answers allowed.
`choices`	array	An enum whose values carry descriptions: `[{"value", "description"}]`. See Choices.
`levels`	array	On a `number`: an ordered scale, `[{"label", "description"}]`, lowest first; the answer is the expected level. See Scores.
`properties`	object	On an `object`: its fields, each a question definition. See Objects.
`items`	object	On an `array`: the schema of each item; `minItems` and `maxItems` bound it, at most 200 for an array of objects and 50 otherwise. See Arrays.
`continue_from`	array	On an `array`: the items so far, from an earlier call on another part of the input. See Long inputs.
`instructions`	string	What to answer; `description` is read when it is missing.
`thinking`	boolean or "auto"	Reason before answering; `"auto"` chooses the effort on each call. See Thinking.
`thinking_budget`	integer	Cap on that reasoning: 1,024 tokens by default, at most 4,096.
`thinking_effort`	string	`none`, `low`, `medium` or `high`: no thinking, or up to 512, 2,048 or 4,096 tokens. Not with `thinking_budget`. See Effort.
`return_probabilities`	boolean	On enum, choices, boolean and score questions: answer with `{"value", "probabilities", "confidence"}`. See Probabilities.
`permutations`	boolean	On enum, choices and boolean questions: average probabilities over a balanced set of option orders. On by default with `return_probabilities`; `false` reads the options once, as listed. Not on scores.
`source`	string	`"user"`: your code answers this question, from a tool or a person, not the model; the call stops for it. A top-level string, number, integer or boolean, without `thinking`, `return_probabilities`, `permutations` or `levels`. See Tools and human input.
`depends_on`	array of strings	Questions answered first and shown to this one. See Workflows.
`when`	object	Run this question only if other questions' answers meet these conditions, such as `{"amount": {"gte": 1000}}`, or on how sure an answer is, such as `{"category": {"confidence": {"gte": 0.8}}}`; those questions become dependencies. See Conditional fields.

Response

200 OK

{
  "id": "gen_...",
  "model": "typellm-latest",
  "result": {"total": 324.5, "expense_type": "travel", "policy_ok": true},
  "thinking": {"policy_ok": "The hotel is in London, where the cap is..."},
  "usage": {"input_tokens": 410, "thinking_tokens": 204},
  "elapsed": 3.1
}

Field	Type	Description
`id`	string	The call's id; quote it when you contact us.
`model`	string	The public model that answered.
`result`	object	One typed answer per question.
`thinking`	object	The reasoning of each question that thought.
`thinking_effort`	object	The level each `"auto"` question got.
`skipped`	array of strings	Questions a `when` skipped. They have no key in `result`.
`status`	string	`completed`, or `needs_input` when the call stopped for questions answered outside the model.
`needs_input`	object	With `needs_input`: each question to ask, with its `type`, `instructions` and `enum`.
`resume_state`	string	With `needs_input`: send it back with the answers to go on.
`usage.input_tokens`	integer	What you sent, counted once: the context, the questions as JSON, and the images. Billed.
`usage.thinking_tokens`	integer	Reasoning tokens. Billed. Answers are free.
`elapsed`	number	Seconds the service spent, queueing included.

GET /v1/models

The public models your key can use: {"models": ["typellm-latest"]}.

Compatible endpoints

Code written for these APIs runs on TypeLLM with its base URL and API key changed. Each request is answered, checked and billed as a /v1/generate call.

Endpoint	For	Description
`POST /v1/decisions`	OpenAI	OpenAI's Decisions API: predicate, choice and score questions. Base URL `https://api.typellm.ai/v1`. See OpenAI Decisions.
`POST /v1/systemone`	Jev	Jev's System One API: noul, choice and score questions. Base URL `https://api.typellm.ai`. See Jev System One.

Limits

What	Limit
Questions per call	64
Input	128K tokens: the context and the questions
Request body	128 MB
Images per call	8, each at most 10 MB; large ones are scaled to about 2,048 tokens
String answers	128 tokens; maxLength is not accepted
thinking_budget	1,024 by default, at most 4,096 per field
thinking_effort	none, low (512), medium (2,048) or high (4,096)
timeout	60 s by default, at most 90 s
resume_state	24 hours, with the keys of the account that made the call
Requests per minute	Per account, shared by all its keys: 200; 300 once your top-ups add up to $10; 500 from $100. Contact us for a higher limit.
Calls in flight	Per account, shared by all its keys: 4; 8 from $10 in top-ups; 16 from $100. More get 429

Errors

Errors come with an HTTP status and a body like:

Error

{"error": {"type": "insufficient_balance", "message": "the balance has run out; ..."}}

Status	Type	When
`400`	invalid_request, limit_exceeded, invalid_image, context_length_exceeded	A malformed body or question, a limit exceeded, an unreadable image, an input over 128K tokens, or a resume_state that is invalid, expired or answered wrongly.
`401`	unauthorized	A missing, unknown or revoked key.
`402`	insufficient_balance	The balance has run out. Add credits in Dashboard, Billing.
`404`	unknown_model	No such public model.
`413`	http_error	The body is over 128 MB.
`429`	rate_limit_exceeded, concurrency_limit, overloaded	Over the account's requests a minute (200; 300 once top-ups reach $10 and 500 from $100; yours is on the dashboard), too many calls in flight, or the service is full. Retry after Retry-After. Contact us for a higher limit.
`502`	upstream_error	The model server failed; retry.
`503`	unavailable	The model is temporarily unavailable.
`504`	timeout	The call ran past its timeout.

The Python client retries a 429, a 5xx other than 504 and a connection error up to twice (max_retries), then raises these as SGLangError with the status in .status, and a 504 as GenerationTimeout.

Pricing

$0.05 per million input tokens and $0.50 per million thinking tokens; answers are free. See Pricing.