Thinking

Enable reasoning before constrained decoding. Thinking is off by default for models with a switchable thinking mode.

client = TypeLLMClient(
    "http://127.0.0.1:30000",
    model="Qwen/Qwen3.8-27B",
    thinking=True,
    thinking_budget=2048,
)

Budget and model behavior

• thinking_budget applies per field and must be a positive integer. The default None uses available context while reserving room for the answer.
• At a length limit or recognized early end-of-turn, TypeLLM closes nonempty partial reasoning and continues typed decoding. Empty unfinished reasoning, unknown stops and server errors still fail.
• Ling requires thinking off; Ring always reasons. Tested Qwen and MiniCPM5 models support both modes.
• Final-answer sampling settings do not change reasoning sampling.