A8 model card

At a glance

§1A call, and a correction

A8 answers the chat completions request you already send to a generative model. Only the base URL and the model name change.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.u22a8.ai/eval/v1",
    api_key="<your key>",
)

resp = client.chat.completions.create(
    model="a8",
    messages=[
        {"role": "system", "content": "Does the reply answer it?"},  # 1
        {"role": "user", "content": "Your parcel left today."},  # 2
    ],
    extra_body={"min_accuracy": 95},  # 3
)

resp.choices[0].message.content  # the verdict, or empty

# 4: the last verdict was wrong
client.chat.completions.create(
    model="a8",
    messages=[...],
    extra_body={"expected": False},
)

1. The system message is the criterion: what you are judging.
2. The last user message is the subject: the thing under judgment.
3. min_accuracy is the floor. At 95, at least 95 answered verdicts in a hundred are correct. Where the evidence can’t carry a case that far, A8 abstains.
4. expected is a correction. False says the verdict just returned was wrong. Send it with the same messages, and your organization’s model learns from it.

An optional json_schema names the fields the verdict comes back in.

§2Scored, not sampled

A generative model asked to judge writes its verdict the way it writes anything else: it samples it. Nothing in the answer says how likely it is to be right, finding out takes labeled data, and asking again can change it.

A8 scores the subject against what it has learned about the criterion, and answers only where that evidence clears the accuracy you set. No text is generated on the way, so the same request returns the same verdict.

When you need text back, such as an explanation, a rewrite or a summary, a generative model is the right tool. Use it to write, and A8 to check.

In a regression suite, a changed verdict means the input changed or the model moved.

§3How it works

Encoders place the criterion and the subject in one space. For each criterion it has learned, the eval model holds a fitted direction in that space and the evidence around it. The verdict is where the subject falls along that direction.

Fitting runs in the background, and answering does not wait for it. Each fit mints a new horizon, and the response names the one that answered.

§4Abstention and the accuracy floor

min_accuracy is a promise about answered verdicts: an integer from 0 to 99, default 90. At 95, at least 95 in a hundred are correct. The cases the evidence does not cover leave the answered set instead of entering it as guesses, which is what keeps the rate true.

Every answer carries the level it is promised at: the level asked for, or the nearest earned one above it. 0 asks for no promise. A subject unlike anything the model learned from abstains at any level above that.

It is a floor the request sets, not a belief the model reports. Nothing in the response says a particular answer is 95% likely to be right. The number is the accuracy the evidence had to show before the model returned a verdict.

§5Learning from corrections

A correction is an ordinary request that carries expected. There is no dataset to upload, no training job to schedule or poll, and no tuned model to host.

The correction is banked immediately. A fit follows once your organization has been quiet for about a minute, so a hundred corrections sent in a loop become one fit. When it completes, a new horizon starts serving, and the one before it stays pinnable.

What your organization teaches trains your organization’s model. It never reaches another organization.

§6Evidence

qed-bench measured the approach A8 is built on: scoring in an encoder’s space. On essay quality (ASAP 2.0), its rank correlation with trained human raters was 0.815. The strongest LLM judge on the panel, claude opus 4.7, reached 0.813.

§7What to expect

§8Compared with the other two ways

A judgment is automated today with a trained classifier or a generative model acting as judge. This is how the three differ.