TypeLLM

Classification with new category discovery

Open in Colab →View on GitHub

Invoices keep coming in, one after another, and you want to sort each one into a category. But you can't know all the categories ahead of time. So let the categories grow as the invoices arrive: each new invoice either joins a category you already have or starts a new one.

With TypeLLM, sorting an invoice and creating a new category happen in one API call, with two fields. category chooses from the categories so far, plus new. new_category runs only when the answer is new, and names the category from what the invoice is.

The single call is in the Discover a new category example. This recipe runs it as invoices keep coming in, and keeps the list of categories growing.

Set up

Install the library:

pip install "typellm>=0.6.7"

Get an API key from your dashboard:

from typellm import TypeLLMClient

client = TypeLLMClient(api_key="YOUR_API_KEY")

INVOICES is a list of 50 one-line invoices; see the notebook for details.

Start with two categories

They show the model how broad you want a category to be.

categories = ["travel", "legal"]

Classify and create categories in one API call

Every category so far is a choice, plus new. new_category is only asked when the answer is new. TypeLLM allows you to classify an invoice and create a new category in one API call with when. That is faster and cheaper, since TypeLLM only needs to read each invoice once.

def classify(invoice):
    questions = {
        "category": {
            "type": "string", "enum": categories + ["new"], "thinking": "auto",
            "instructions": "Which category does this invoice belong to? Choose new unless one of them clearly fits.",
        },
        "new_category": {
            "type": "string", "when": {"category": "new"},
            "instructions": "A broad snake_case category name of one or two words, like travel or utilities, "
                            "that many other invoices could share.",
        },
    }
    result = client.generate(context=f"Invoice: {invoice}", questions=questions).result
    return result["category"], result.get("new_category")

A new category joins the list right away, so the next invoice can choose it:

assigned = {}  # invoice -> category
for invoice in INVOICES:
    category, new_name = classify(invoice)
    if category == "new":
        if new_name not in categories:
            categories.append(new_name)
        assigned[invoice] = new_name
    else:
        assigned[invoice] = category

Over 50 invoices, 16 new categories join the first two. Some of what came back:

NEW  software    <- Slack, Pro plan
NEW  furniture   <- Herman Miller, 4 Aeron chairs
     travel      <- Delta Air Lines, BOS to SFO
NEW  marketing   <- Google Ads, search campaign
     marketing   <- Meta Platforms, Facebook ads
     software    <- GitHub, Team plan
     legal       <- USPTO, trademark application
NEW  shipping    <- FedEx, overnight shipping
     shipping    <- UPS, laptops to remote hires

Final result

travel                 5: Delta, Hilton, Uber, Marriott, United
legal                  2: Morgan & Lee LLP, USPTO
cloud                  2: AWS, Namecheap
software               8: Slack, GitHub, Figma, Zoom, Datadog, Gusto and 2 more
furniture              2: Herman Miller, Steelcase
office_supplies        3: Staples, Blue Bottle Coffee, Office Depot
meals                  3: Nopa Restaurant, DoorDash, Sweetgreen
professional_services  4: Grant Thornton, Ana Ruiz Design and 2 more
marketing              3: Google Ads, Meta Platforms, LinkedIn
utilities              4: Eversource, Comcast, Verizon, City Water Department
rent                   1: Seaport Properties
conferences            3: SaaStr Annual, Printful, Cavallo Point
insurance              2: Hiscox, Blue Shield of California
equipment              3: Apple, Dell, Logitech
shipping               2: FedEx, UPS
taxes                  1: Delaware Division of Corporations
bank_fees              1: Chase
donations              1: Red Cross

The notebook runs all of it with your key: open it in Colab.

← all recipes