Read a photographed receipt

Open in playground →

A crumpled receipt goes in; typed strings, integers, numbers and booleans come out, and a discount percentage is computed from two of them.

Image input
Typed numbers
depends_on

Challenges

The only input is a photo of a crumpled receipt: there is no text to copy from.
Amounts are printed as 194,000. Each must come back as the number 194000, not a string, and not 194.
The discount percentage is not printed anywhere. It has to be worked out from the subtotal and the discount, so it is asked after them with depends_on.

The photographed receipt sent with this call — Receipt from CORD v2, CC BY 4.0.

import os
from typellm import TypeLLMClient

client = TypeLLMClient(api_key=os.environ["TYPELLM_API_KEY"])
response = client.generate(
    context="Read the attached photo of a restaurant receipt.",
    questions={
      "first_item": {
        "type": "string",
        "instructions": "Name of the first line item, exactly as printed."
      },
      "item_lines": {
        "type": "integer",
        "instructions": "How many line items are listed?"
      },
      "subtotal": {
        "type": "number",
        "instructions": "Subtotal in rupiah as a plain number; the receipt uses commas as thousands separators."
      },
      "discount": {
        "type": "number",
        "instructions": "Discount in rupiah as a plain number."
      },
      "total": {
        "type": "number",
        "instructions": "Total in rupiah as a plain number."
      },
      "paid_in_cash": {
        "type": "boolean",
        "instructions": "Was the bill paid in cash?"
      },
      "discount_percent": {
        "type": "number",
        "depends_on": [
          "subtotal",
          "discount"
        ],
        "instructions": "Discount as a percentage of the subtotal."
      }
    },
    images=["receipt.jpg"],
)
print(response)

Time 2.89 s · Cost $0.000063

Generation(
    result={
        'first_item': 'ICE BLACKCOFFEE',
        'item_lines': 3,
        'subtotal': 194000,
        'discount': 19400,
        'total': 174600,
        'paid_in_cash': True,
        'discount_percent': 10,
    },
    thinking={},
    usage=Usage(input_tokens=1270, thinking_tokens=0),
)