Thinking
Let a field reason before its constrained answer. Thinking usually makes answers more accurate, but the call takes longer to return. It is off by default; turn it on for the fields that need it. The other fields answer at once, and the fields that think reason side by side.
With "thinking": "auto", TypeLLM chooses how much each field thinks on each call. response.thinking_effort reports the level each "auto" field got.
response = client.generate(context="Counting letters.", questions={
"hello": {"type": "integer", "thinking": "auto",
"instructions": 'How many times does the letter "r" appear in "hello"?'},
"strawberry": {"type": "integer", "thinking": "auto",
"instructions": 'How many times does the letter "r" appear in "strawberry"?'},
})
print(response.result) # {"hello": 0, "strawberry": 3}
print(response.thinking_effort) # {"hello": "none", "strawberry": "low"}
print(response.thinking) # {"strawberry": "... s t r a w b e r r y ... => 3 ..."}
print(response.usage.thinking_tokens) # 68An "auto" field runs after its dependencies, with one quick step first. thinking_effort and thinking_budget cannot be combined with "auto" or with each other.
Budget and effortAdvanced
To set the amount yourself, turn thinking on with "thinking": True and cap it. A field's thinking_budget caps its reasoning and must be a positive integer.
"strawberry": {"type": "integer", "thinking": True, "thinking_budget": 1024,
"instructions": 'How many times does the letter "r" appear in "strawberry"?'}thinking_effort sets the budget by level instead: "none" does not think, "low" thinks up to 512 tokens, "medium" up to 2,048 and "high" up to 4,096.
"strawberry": {"type": "integer", "thinking_effort": "low",
"instructions": 'How many times does the letter "r" appear in "strawberry"?'}- The budget may be at most 4,096 tokens; a larger one is refused with
limit_exceeded. - A field that thinks without a budget gets 1,024.
- Reasoning tokens are billed, like input; answers are free. The response carries each field's reasoning.