LLMs with type-safe generation

Setting	Default	Purpose
`execution`	`"auto"`	Batch independent fields or follow dependencies.
`thinking`	`False`	Enable native model reasoning.
`thinking_budget`	`None`	Optional thinking-token cap per field.
`text_max_tokens`	`512`	Text generation budget per field.
`numeric_max_digits`	`32`	Digit limit for open numeric fields.
`mode`	`"argmax"`	Argmax or sample decoding.
`base_url`	`"http://127.0.0.1:30000"`	SGLang server URL.
`model`	`None`	Model ID; discovered from the server when omitted.
`tokenizer`	`None`	Matching tokenizer ID or local path; discovered from the server when omitted.
`temperature`	`1.0`	Final-answer sampling temperature; finite and positive in sample mode.
`seed`	`None`	Client random-number seed.
`timeout`	`120.0`	HTTP request timeout in seconds.

Advanced settings

label_pool customizes internal candidate labels. numeric_cache_dir chooses a directory for numeric-token tables. Leave both unset for normal use.

For debugging, set print_final_prompt=True or inspect client.last_prompts. Sequential execution also populates client.last_prompt.

generate arguments

Supply exactly one of context or its alias state. The value must be a string.
Optionally pass images, a list of file paths, http(s) URLs, data: URIs, bytes or PIL images, for a vision-language model. They precede the context and are sent with every request in the call.
Supply exactly one of questions or schema. Questions map names to field definitions; a JSON Schema uses a root object with nonempty properties.
mode, execution and temperature may override client settings for one request.
Returns a dictionary in field declaration order. Every declared field is evaluated; required does not make other fields optional.

Per-field permutation setting

permutations belongs inside each question or schema property, not the client constructor or the top-level generate arguments. It defaults to 1 and accepts a positive integer or "all", only with an explicit enum. Each field may evaluate at most 720 orderings after capping the budget at the number of possible permutations.

See permutation averaging for sampling, output, and validation rules.

Errors

SchemaError: invalid candidates, dependency definitions or unsupported field options.
ValueError: invalid client settings or a generated numeric value outside its declared bounds.
NotImplementedError: unsupported output schema types such as arrays or nested objects.
SGLangError: backend failures, incompatible chat templates or invalid/incomplete generated text or reasoning.

For the full API and implementation, see the README and source code.