How Unee works, and how it compares.

At a glance

Three decision models, side by side.

Jev and Laya facts come from their public documentation and model cards. Laya’s context figure is for its typed-decisions checkpoint.

How it works

Four ideas, explained simply.

1. A decision is one quick look, not an essay.

Big chatbots answer by writing word after word. Unee answers a decision the way you tick a box: it reads your text, your question and the options once, then gives every option a probability. That is why it is fast, and why it can never answer something that is not on your list.

1. Your text“I was charged twice for order #4821.”
2. Your question and optionsWhich team? Billing, technical or shipping.
3. Unee reads it onceOne pass through the model, about a tenth of a second on a laptop graphics card.
4. A probability for every optionBilling 98%, every other option under 1%. Your app acts when Unee is sure and asks a person when it is not.

2. It practised on checked examples.

A larger open model wrote and checked the practice examples, and Unee learned from them. No benchmark’s test questions were used.

1. Practice examplesDecisions across 40 kinds of task in 45 languages, written and checked by Qwen3.5-9B.
2. Tricky pairsNear-twins where one small change flips the right answer.
3. Best of several runsPractice runs are averaged into one model, which smooths out each run’s mistakes.

3. A tiny date helper does the calendar maths.

Small models are bad at counting days in one glance, and so is Jev. Before Unee reads your text, a simple program works out the gaps between any dates it finds, in English and 18 other languages, and adds them as facts.

1. Your text“Delivered on 1 May 2026. Return asked for on 5 June 2026.”
2. The date helper adds“5 June 2026 is 35 days after 1 May 2026.”
3. Unee just compares“Is it within 30 days?” becomes an easy question. This alone added about 3 points of accuracy.

4. It answers from your own documents.

For a chat assistant, Unee finds the right part of your help pages and answers from it. If the answer is not there, it says so instead of making something up.

1. Your documentsFAQ, policies, product pages, prices.
2. Short passagesSplit into bite-sized pieces, in any script.
3. Best matchesThe pieces closest to the question are picked, in milliseconds.
4. A streamed answerWord by word, in the user’s language, or “I don’t know” when the documents do not say.

Accuracy and cost

On the frontier: nothing cheaper is more accurate.

DecideBench gives 400 tricky decisions to every model and records how often it is right and what a million decisions would cost. The dashed line joins the models that nothing beats on both at once. Both Unee sizes sit on it.

Accuracy and size

More right answers per gigabyte.

Smaller models are cheaper to run and fit on more devices. Among models whose size is published, nothing smaller than Unee is more accurate, and every model that beats Unee 2B is at least twice its size.

Where each model wins

Strong at intent and triage, still learning agent safety.

The 400 decisions come in eight kinds. Unee matches Jev on understanding what a customer wants, and is weakest on reviewing AI agent actions and moderation.

An independent check

On a benchmark Unee never trained on.

JevBench is a separate benchmark with 1,682 checked decisions. Unee 0.1 ran it once, zero-shot, and clearly beat Laya, with Jev clearly ahead. Later versions were not rerun: some of their practice data now overlaps JevBench’s checked questions, so their score would not be independent.

JevBench leaderboard, 232-row gold sample (not the same rows as Unee's 1,682). Unee 0.1 was run on all 1682 gold decisions, zero-shot, once. After this run, new practice tasks were added for the kinds of question JevBench showed as weak (long option lists, two-step reasoning, arithmetic), so a JevBench score for later versions would no longer be fully independent. Later versions are checked on S1MB and MASSIVE instead.

Beyond English

The same requests, in 51 languages.

MASSIVE is a public set of everyday requests to a voice assistant, such as “wake me up at seven” or “will it rain tomorrow”, written by native speakers in 51 languages. We asked each model to pick which of 18 parts of the assistant should handle 60 of them, the same requests in every language, so the languages can be compared fairly. Jev and Laya have not been run on these requests, so the comparison is with the model Unee is built on.

Answers from your documents

Does it stick to your pages?

We wrote small help centres for made-up companies Unee never saw, some in other languages, and asked 116 questions. Some are answered in the pages and some are not. A larger model judged every reply, and strictly: an answer that leaves out part of a policy counts as wrong.

Every model gets the same pages through the same built-in search. Unee says so when your pages do not have the answer, where the model it is built on mostly guessed. The judge is Qwen3.5-9B, the model Unee learned from, so treat these as a guide rather than a final verdict.

Strict mode checks the answer before it is sent.

In strict mode Unee writes the whole answer, then checks it sentence by sentence against the pages and removes what it cannot support. The same answers were judged with and without that check.

Strict mode costs a few points of correct answers, and it lowers made-up facts without ruling them out: the checker is the same small model. It checks at a threshold of 0.5 here, and the threshold is yours to set.

A third benchmark

S1MB: over a hundred public tests in one score.

S1MB is a community benchmark that gathers public datasets for the three kinds of decision Unee makes: pick one, yes or no, and rate on a scale. Each test is scored so that 0 means no better than a trivial guess, such as always giving the most common answer, and 100 means perfect. The headline is the average of the three kinds.

S1MB leaderboard, full English suite, snapshot of 2026-09-30. Unee was run by us with S1MB’s own evaluator on 137 scored tests. S1MB’s test rows were never used in training, but Unee 0.3 did learn from the public training splits of the datasets S1MB draws on. Checking every S1MB test input against all of Unee’s training text finds 6.9% that resemble it, almost all in tests generated from templates, with 48 exact copies. Leaving out every test with any overlap (101 remain), the task average is 45.69 for 2B and 36.74 for 0.8B, no lower than the full scores, so they do not rest on the overlap. The few long-document tests (contracts, research papers) did not fit the four-at-once setup, so they were run again one at a time with a longer context window. Per-kind scores are shown for Unee only, as the leaderboard snapshot lists the average.

General ability

What it keeps from its base.

Unee is trained for decisions and answers from your documents, not general knowledge. These widely used tests show how much general ability it keeps. They were run with EleutherAI’s evaluation harness (lm-eval 0.4.13) on every item, with the model’s thinking switched off.

Qwen publishes none of these for the base models except IFEval: 52.1% for 0.8B and 61.2% for 2B, sampled with some randomness and without saying which IFEval score it is. Unee’s per-instruction score is close to those and its whole-prompt score is lower, so read that comparison as a rough guide. We did not run the base models ourselves.

Honest confidence

Sure answers you can trust, unsure ones you can spot.

A probability is only useful if it is honest. Most of Unee’s answers come with high confidence, and there it is honest: answers given at about 96% are right about 95% of the time. Its less confident answers are right less often than they claim, which is exactly why your app should act on its own above a threshold and hand the rest to a person.

Speed on real hardware

Fast on a graphics card, usable on an old laptop.

Times for one decision with worked examples in the prompt (the benchmark’s setup), measured on a recent laptop (Intel Core Ultra 9, RTX 4070). Without examples, decisions are up to about twice as fast. Older, slower processors will take longer.

Median time per decision, one request at a time, llama.cpp with 4-bit weights. For comparison, DecideBench lists Jev at 639 ms through its hosted API and Laya at 97 ms on its own server, measured on different hardware.

Honest limits

What Unee is not good at yet.

• Large hosted models are more accurate. Jev and the biggest models still get more decisions right. Unee’s advantage is that it is small, free to run and runs anywhere.
• One quick look has limits. Decisions that need several steps of reasoning, sums or very long lists of options are harder for it.
• It is not a general knowledge assistant. On 60 open-ended questions, a larger model judging blind preferred the answers of the plain Qwen model Unee is built on. Unee’s answers are shorter and, like any small model, it gets facts wrong. Its chat is meant for answering from your own documents.
• The explanation is the model’s own account. When Unee explains a decision, that is useful, but it is not proof of how the decision was made.
• Not for high-stakes calls on its own. Medical, legal, credit or hiring decisions need a person in the loop.