TypeLLM
Examples

Answer a question about a photo, and box where the answer is

Open in playground →
  • Image input
  • Box
  • depends_on
  • when
  • Probabilities

Task

Say whether any bear in the photo has caught a fish, and box the bear and the fish that the answer is based on.

Challenges

  • Five bears stand in the falls and only one has a fish in its mouth: the answer rests on that one bear.
  • The box depends_on the answer, so it is drawn around what the answer rests on, the bear and its salmon, and not around just any bear.
  • With when, the box is asked only if the answer is true: when no bear has caught one, there is nothing to point at.
The image sent with this callwhere
Photo by Joseph C Boone, CC BY-SA 4.0, via Wikimedia Commons.

Laying out…

Time 2.70 s · Cost $0.000052

Generation(
    result={
        'caught_fish': {
            'value': True,
            'probabilities': {
                'true': 1,
                'false': 0,
            },
            'confidence': 0.999,
        },
        'where': {
            'top': 0.575,
            'bottom': 0.812,
            'left': 0.457,
            'right': 0.712,
        },
    },
    thinking={},
    usage=Usage(input_tokens=1037, thinking_tokens=0),
)