ContextGem 1.0: the right model for each question about a document

ContextGem is our open-source framework for extracting structured data from documents. You describe in plain language what to extract, and it writes the prompts, validates what the model returns into typed values and traces each extracted fact to the paragraph or sentence it came from. Version 1.0 is out, and it adds decision models beside LLMs.

This post is the short version: what ContextGem does, what 1.0 brings, why it matters, and how to use it. The details are in its documentation. The examples share one invented application: an agent for a property manager, which reads the reports that the caretakers file, one for each defect they find, and opens a work order for each.

What ContextGem does

What to extract is described as aspects, the parts of a document on a topic, and concepts, typed values such as a yes or no, a label, a number, a date or a text. One call to a model extracts them. For the property manager's agent, the first thing to extract is the defect itself, with the sentence of the report that it was taken from:

from contextgem import (
    Document,
    DocumentLLM,
    StringConcept,
)

doc = Document(raw_text=report)
defect = StringConcept(
    name="Defect",
    description="What is broken",
    add_references=True,
    reference_depth="sentences",
)
doc.add_concepts([defect])

llm = DocumentLLM(
    model="openai/gpt-6-luna",
)
doc = llm.extract_all(doc)

Each item that comes back has its value, the defect as the report states it, and in reference_sentences the sentences of the report it was found in. Whoever checks it reads a sentence, not the report.

There is no search step that picks the passages that look relevant, as retrieval-augmented generation (RAG) has: the model reads every paragraph, all at once or in batches of a size you set. In an agent, ContextGem is the part that reads, one document at a time.

Decision models beside LLMs

A decision model takes a text and closed questions about it (yes or no, one of your labels, a level on a scale) and returns the probability of each possible answer. It writes no text. The kind is new: TypeSafe introduced it with Jev in September 2026, OpenAI (Decisions API) and Microsoft (Microsoft-Decision-1) have followed, and Ollama and LM Studio run them locally.

In 1.0, a DocumentDecisionModel extracts aspects and Boolean, label and rating concepts, each item with the probabilities of all its answers, and you set the threshold from which a yes counts as a yes. A model group sends each aspect and concept to the model of its role, the closed questions to a decision model and the open values to an LLM, and any model can fall back on others.

The rest of the release is a long round of fixes, all in the changelog. The documentation has a new address, contextgem.shcherbak.ai, with llms.txt for coding agents.

Upgrading from 0.x? Read the migration guide first, or point a coding agent at llms.txt: some names, defaults and the format of serialized data have changed.

Why it matters

Extraction from documents goes wrong more often by leaving a fact out than by making one up. A 2026 systematic review of 27 studies of LLMs that extracted data for systematic reviews found omissions to be 60 to 74% of the errors, and hallucination rates of 0.08 to 6%. A search step that picks the passages is one more place to miss a fact. In FinanceBench, a 2023 benchmark of questions about company filings, GPT-4-Turbo answered 50% of the questions correctly when a vector store picked passages from the one filing, and 79% with the filing in its prompt. So ContextGem reads every paragraph.

What is extracted still has to be checked, and the source makes the check short. In a preprint from CSIRO, Australia's national science agency, people who judged extracted answers against a few passages of the article were about as accurate as those who had the whole article (83.1% against 85.7%), in 28% less time.

Asked how sure it is, an LLM tends to say too much: on SimpleQA, OpenAI's benchmark of short factual questions, "models consistently overstate their confidence" (Wei et al., 2024). For a closed question, a decision model returns the probability of every answer instead, so you decide where the line falls: above it an agent acts, below it a person looks. It is also priced for volume: OpenAI's Decisions API charges for input tokens only. But the kind is a month old, and an early independent test found Jev's probabilities close to how often it was right on one dataset and far from it on another (a calibration error of 0.034 against 0.29), so ContextGem leaves the model and the threshold to you, to set on your own documents.

How to use it

Install it with uv:

uv add contextgem

Or with pip:

pip install -U contextgem

Closed questions to a decision model. The property manager's agent asks three more things of each caretaker's report: whether the defect puts people at risk, which trade fixes it, and how soon. Each is a closed question, and goes to the decision model by its role:

from contextgem import (
    BooleanConcept,
    DocumentDecisionModel,
    DocumentModelGroup,
    LabelConcept,
    RatingConcept,
)

safety = BooleanConcept(
    name="Safety risk",
    description="Can hurt people",
    model_role="decide",
    decision_threshold=0.3,
)
trade = LabelConcept(
    name="Trade",
    description="Who fixes it",
    labels=[
        "Carpentry",
        "Plumbing",
        "Electrical",
        "Other",
    ],
    model_role="decide",
)
urgency = RatingConcept(
    name="Urgency",
    description="How soon to fix",
    rating_scale=(1, 5),
    model_role="decide",
)
doc.add_concepts(
    [safety, trade, urgency],
)

A model group of the two takes the place of the first example's call: the defect goes to the LLM, which has no role, and the three questions to the decision model:

dm = DocumentDecisionModel(
    model="typesafe/jev-1.13.0",
    role="decide",
)
group = DocumentModelGroup(
    models=[llm, dm],
)
doc = group.extract_all(doc)

A threshold that fits the question. Each item that the decision model extracts has the probabilities of all its answers (the numbers below are invented):

risk = doc.get_concept_by_name(
    "Safety risk",
)
item = risk.extracted_items[0]
item.value  # True
item.probabilities
# {True: 0.41, False: 0.59}

A yes counts as a yes from a probability of 0.5, unless you set another threshold. For a safety risk, a yes that is missed costs more than one too many, so the example's agent takes a yes from 0.3, and this report is marked as a risk.

Before you rely on it. Compare the decision model with an LLM on a sample of your own documents whose answers you know, and set the threshold on that sample (when to use a decision model): for the example's agent, caretakers' reports that the property manager has already sorted. Ask for a version of the model and not an alias, as the example does (jev-1.13.0): jev-latest moves to each new release, and a threshold set on one version is not checked on the next.

Questions to ask

Whether the extraction is ContextGem, another library or a model provider's own API, these questions show whether you can rely on what it extracts.

1. Which of the values you extract are closed questions, and which are open?
2. Can each extracted value be traced to the sentence it came from?
3. Does a search step decide which passages the model reads?
4. How sure is the model of each answer, and who sees the answers it is unsure of?
5. Was the threshold set on your own documents, and is it checked again when the model changes?
6. What does one document cost to extract, and how much of that is generated text?

We only publish tools we use ourselves and can recommend to others. Each reflects how we build, and ContextGem reflects one word of it: verified. We build AI agents to fit your enterprise, each verified, constrained, and governed.