Audit an expense claim
Open in playground →Read every line of a claim, then judge the claim from those lines, in one call. Later fields depend on the typed list: one checks the total, one lists the rules broken, and the decision and the note to the employee follow from them.
Challenges
line_itemsis an array of typed records, each with a category from fixedchoices. The fields after it see the list, not the raw text, so they check numbers that were already read.adds_upandviolationsthink before they answer: the dinner for two is $148, $74 a person, under the $75 cap, so it is not a violation, while the $289 hotel night and the wine are.approvedepends on the check and the violations and returns its probabilities and confidence: a claim that breaks a rule is turned down, andnotesays what to fix.
import os
from typellm import TypeLLMClient
client = TypeLLMClient(api_key=os.environ["TYPELLM_API_KEY"])
response = client.generate(
context="Expense claim: Priya Shah, client visit to Boston, 14-15 September.\n\nPolicy: hotels up to $250 a night; meals up to $75 per person; no alcohol; economy fares only.\n\nSept 14 Round-trip flight to Boston, economy $412.00\nSept 14 Harbor View Hotel, 1 night $289.00\nSept 14 Client dinner for 2, incl. 2 glasses of wine $148.00\nSept 15 Taxi to Logan Airport $42.50\n\nTotal claimed: $891.50",
questions={
"line_items": {
"type": "array",
"instructions": "Every expense line, as listed.",
"items": {
"type": "object",
"properties": {
"date": {
"type": "string"
},
"description": {
"type": "string"
},
"amount": {
"type": "number"
},
"category": {
"type": "string",
"choices": [
{
"value": "travel",
"description": "Flights and trains."
},
{
"value": "lodging",
"description": "Hotels."
},
{
"value": "meals",
"description": "Food and drink."
},
{
"value": "ground",
"description": "Taxis, rideshare and parking."
}
]
}
}
}
},
"claimed_total": {
"type": "number",
"instructions": "The total claimed, in dollars."
},
"adds_up": {
"type": "boolean",
"depends_on": [
"line_items",
"claimed_total"
],
"thinking_effort": "low",
"instructions": "Do the line items add up to the claimed total?"
},
"violations": {
"type": "array",
"depends_on": [
"line_items"
],
"instructions": "Every policy rule the claim breaks; none if it follows the policy.",
"items": {
"type": "string",
"choices": [
{
"value": "hotel_over_cap",
"description": "A hotel night above $250."
},
{
"value": "meal_over_cap",
"description": "A meal above $75 per person."
},
{
"value": "alcohol",
"description": "Alcohol on a bill."
},
{
"value": "premium_fare",
"description": "A fare above economy."
}
],
"thinking_effort": "low"
}
},
"approve": {
"type": "boolean",
"depends_on": [
"adds_up",
"violations"
],
"return_probabilities": True,
"instructions": "Approve the claim as submitted? Only if it adds up and breaks no rule."
},
"note": {
"type": "string",
"depends_on": [
"violations"
],
"instructions": "One sentence to the employee on what to fix before resubmitting."
}
},
)
print(response)Generation(
result={
'line_items': [
{
'date': 'Sept 14',
'description': 'Round-trip flight to Boston, economy',
'amount': 412,
'category': 'travel',
},
{
'date': 'Sept 14',
'description': 'Harbor View Hotel, 1 night',
'amount': 289,
'category': 'lodging',
},
{
'date': 'Sept 14',
'description': 'Client dinner for 2, incl. 2 glasses of wine',
'amount': 148,
'category': 'meals',
},
{
'date': 'Sept 15',
'description': 'Taxi to Logan Airport',
'amount': 42.5,
'category': 'ground',
},
],
'claimed_total': 891.5,
'adds_up': True,
'violations': [
'hotel_over_cap',
'alcohol',
],
'approve': {
'value': False,
'probabilities': {
'true': 0.002,
'false': 0.998,
},
'confidence': 0.997,
},
'note': 'Please resubmit the claim with the hotel charge reduced to the $250 cap and the alcohol portion removed from the client dinner.',
},
thinking={
'adds_up': 'The user wants me to verify if the line items add up to the claimed total.\n\nLine items:\n- $412.00\n- $289.00\n- $148.00\n- $42.50\n\nSum: 412.00 + 289.00 + 148.00 + 42.50 = 891.50\n\nClaimed total: $891.50\n\n891.50 = 891.50, so yes, the line items add up to the claimed total.\n\nThe answer is true, which corresponds to label "A".',
'violations[0]': 'We need answer user\'s request. Need produce final JSON {"label": ...}. Need determine next item of violations array. Claim: hotel 1 night $289 > $250 => hotel_over_cap. Meal client dinner for 2 incl 2 glasses wine $148. Meals up to $75 per person: $148/2 = $74, so not over cap if total includes wine? But alcohol present => alcohol violation. Also if meal over cap? $148 for 2 = $74 per person, under $75. But alcohol on bill. Flight economy, no premium. Taxi irrelevant. Current array empty. Need give one new item distinct. Which one? The instruction: "Give the next item of the array above: one new item, distinct from the items already in the current array." We need output one item. There are multiple violations: hotel_over_cap, alcohol. Maybe choose first? They ask one item. Need likely A: hotel_over_cap. But could be C. Need infer expected from sequential array? Current array [] and need next item. It says every policy rule claim breaks; none if follows. Each item one of... One item of array. We need next item. If multiple, any? Usually these tasks ask to generate array items one by one; choose one valid violation not already present. Since current empty, choose a violation. Which one is most salient? Hotel over cap definitely. Also alcohol definitely. Meal over cap? Let\'s examine: meals up to $75 per person. Client dinner for 2, incl. 2 glasses of wine $148.00. If alcohol not allowed, the bill includes alcohol. Does meal cap apply to total meal amount or per person excluding alcohol? It says meals up to $75 per person; no alcohol. The meal amount $148 for 2 = $74 per person, so no meal_over_cap. But if alcohol is disallowed, perhaps the entire meal is not over cap but alcohol violation. So violations: hotel_over_cap, alcohol. Need one. The prompt: "Give the next item of the array above: one new item, distinct from the items already in the current array. Answer null instead if the current array already satisfies the specification, with no distinct item left that would materially improve it." Since current [] doesn\'t satisfy because missing violations. Need choose one. Usually expected to output first in some canonical order? Choices listed\n\nI will now give the final answer.',
'violations[1]': 'We need answer user\'s request. Need produce final JSON. Let\'s reason carefully.\n\nUser: Expense claim with policy. Need array violations. Current array ["hotel_over_cap"]. Ask: Give next item of array above: one new item, distinct from items already in current array. Answer null if current array already satisfies spec, with no distinct item left that would materially improve it.\nField item, type choice. Choices A hotel_over_cap, B meal_over_cap, C alcohol, D premium_fare, E null.\n\nWe need determine all violations:\n- Flight economy: no premium fare.\n- Hotel 1 night $289 > $250 => hotel_over_cap.\n- Client dinner for 2 incl 2 glasses wine $148. Meal cap up to $75 per person. For 2, cap $150 total. $148 <= $150, so no meal_over_cap. Alcohol on bill: yes, 2 glasses wine => alcohol violation.\n- Taxi no policy issue.\nSo violations: hotel_over_cap, alcohol. Current has hotel_over_cap. Need next distinct item: alcohol. Label C.\n\nNeed final {"label":"C"}.',
},
usage=Usage(input_tokens=579, thinking_tokens=1431),
)