Probabilities

Add return_probabilities=True to an enum, choices, boolean or score field. That field returns its value, the probability of every candidate, and a confidence from 0 to 1; other fields still return plain values.

response = client.generate(
    context=ticket,
    questions={
        "category": {
            "type": "string",
            "enum": ["billing", "bug", "account", "feature_request"],
            "return_probabilities": True,
        },
    },
)
answer = response.result["category"]

{
    "value": "billing",
    "probabilities": {"billing": 0.81, "bug": 0.02, "account": 0.15, "feature_request": 0.02},
    "confidence": 0.75,
}

Probabilities describe how the model weighs the candidates you gave it, not the chance that an answer is factually correct.

Return format

• return_probabilities defaults to false; open number and text fields do not take it.
• Probability-map keys retain enum candidate values. Boolean probability keys are Python True and False.
• Probabilities sum to one over the allowed candidates.
• With temperature 0, the default, each field gets its most likely answer; above 0 it samples one from the probabilities. See Client reference for temperature and seed.

Confidence

confidence is how sure the answer is: 0 when the model has no preference, 1 when all the probability is on one answer.

• For an enum or a boolean, it is how far the top probability is above an even split: (p_max − 1/n) / (1 − 1/n) for n candidates. Above, (0.81 − 0.25) / 0.75 = 0.75. For a boolean this is |2p − 1|.
• For a score, it also weighs how far apart the likely levels are: 1 − Σ p_i |i − m| / d, where m is the score and d is the same sum for an even split, floored at 0. Torn between neighbouring levels costs less than torn between the ends.

Act on it

A common pattern accepts sure answers automatically and sends the rest to a person, or to a second pass with thinking turned on.

if answer["confidence"] >= 0.8:
    route(ticket, answer["value"])   # sure: act on it
else:
    send_to_review(ticket, answer)   # unsure: let a person decide

Or route inside the call: a field whose when tests confidence runs only when the answer is sure enough, so the next step happens in the same request. See Conditions.

response = client.generate(context=ticket, questions={
    "category": {"type": "string", "enum": ["billing", "technical", "account"],
                 "return_probabilities": True},
    "refund": {"type": "number", "instructions": "Refund in dollars.",
               "when": {"category": {"in": ["billing"], "confidence": {"gte": 0.8}}}},
})

• “I was charged twice for my Pro upgrade ($49 each). Please refund one.” returns billing at confidence 1.0, and refund 49.0.
• “Since I changed my email I can't log in, and my last invoice looks wrong too.” returns account at confidence 0.34, and skips refund.

Run it in the playground from the Refund only when sure example.

• When several fields must all be right, use the lowest confidence among them.
• Set each field's threshold on a labelled sample of your own inputs: the lowest one that still gives the accuracy you need.

response = client.generate(
    context="A single roll of a fair die.",
    questions={"roll": {
        "type": "string",
        "enum": ["one", "two", "three", "four", "five", "six"],
        "instructions": "What number will come up on this roll?",
        "return_probabilities": True,  # averaged over option orders by default
    }},
)
print(response.result["roll"])