Baseten's joey-1: A 2B Open-Weight "Decision Model" That Answers in 20 ms for a Penny per Million Tokens
Sources: Baseten's official blog post "Decisions for a penny: introducing joey-1, our fastest decision model" (published Oct 29, 2026) and the official baselabs/joey-1 model card on Hugging Face (checked Oct 30, 2026)
Oct 30, 2026: Not every AI task needs a chatbot. Routing a support ticket, flagging a message or approving a request only needs an answer and a confidence score. On Oct 29, 2026, Baseten's research lab Base Labs released joey-1, a 2-billion-parameter "decision model" built for exactly that. The weights are open on Hugging Face under the MIT license.
Baseten calls it its "fastest, cheapest model."
What a "decision model" actually does
A normal LLM answers a yes/no question by generating text, which your app then has to parse. According to Baseten, a decision model instead "return[s] the answer directly as a single token and a calibrated probability score."
With joey-1, you send a state (text, optionally with images) plus one or more typed questions. The model returns a probability for every option, and it answers all the questions from one shared-prefix prefill. The model card lists three question types:
- Yes/no (returns P(yes))
- Choice between named options (for example billing vs. technical vs. fraud)
- Score on an ordered scale (for example "Can wait" to "Blocking the customer now")
The card adds that requests longer than the trained limits are "refused, not truncated."
The numbers (all Baseten-reported)
- Size: 2B parameters, post-trained from Qwen3.5-2B
- Latency: a P50 end-to-end latency of 20 ms on an average 2,000-token request. Baseten says that's about 2.5x faster than its 9B elephant-1 (49 ms) and "5x faster than comparable decision model models."
- Price on Baseten Model APIs: $0.01 per 1M input tokens, which Baseten says is a tenth of its 27B mammoth-1
- Context: 16k on Model APIs, and the full 262k on dedicated deployments, per the blog (the model card lists a 16,384-token max sequence length)
- Privacy: zero data retention on by default on Model APIs
Benchmarks (self-reported, from the model card): - 63.30 on the Jev Decision Index 0.4 public split. The card calls it the "best model under 4B," ranked #37 of 230 models. - 62.9 on Baseten's own replica of the Artificial Analysis Decision Index
Baseten's cost example: classifying 10 million support tickets a month at about 500 tokens each (5 billion tokens) would cost about $50/month on joey-1, against $500 on mammoth-1.
The family: joey, elephant, mammoth
joey-1 joins two models Baseten released about a week earlier:
| Model | Size | Baseten's positioning |
|---|---|---|
| mammoth-1 | 27B | Highest accuracy |
| elephant-1 | 9B | "Balanced default" |
| joey-1 | 2B | High-volume, latency-sensitive calls |
Baseten says all three share the same API and training data, so "moving between them is a one-line change." It suggests using joey-1 as the first line and escalating to a bigger model "when joey-1 is uncertain."
Self-hosting
The model card's self-hosting instructions need one CUDA GPU and a pinned pytorch/pytorch:2.8.0-cuda12.8-cudnn9-devel image. A setup.sh script installs the serving runtime. The card says training used 6.21M decision examples on 2 nodes × 8 H100s.
Why it matters (our take)
For builders, the pitch is cost and latency. A lot of "AI features" are really just classification, and paying for a full LLM to write a sentence that you then parse is wasteful. A 2B model that fits on one GPU makes always-on decisions cheap.
For game developers and community teams, the obvious uses are things like chat or report triage, ticket routing and content moderation queues. These are our examples, not Baseten's. Calibrated probabilities matter here, because you can set a confidence threshold and send unsure cases to a human or a bigger model.
What's confirmed / not confirmed
Confirmed (Baseten blog, Oct 29, 2026, and the official model card): - joey-1 is a 2B decision model post-trained from Qwen3.5-2B, with open weights on Hugging Face (MIT license per the Hugging Face model card metadata) and availability on Baseten Model APIs. - The listed price is $0.01 per 1M input tokens, with zero data retention by default. - It accepts text and images and returns calibrated probabilities for yes/no, choice and score questions.
Not confirmed / self-reported: - The 20 ms P50 latency, the "5x faster" comparison and all benchmark scores are Baseten's own figures. Baseten says the full internal testing details are "coming soon." - The Artificial Analysis score comes from Baseten's replica of that index, not an official Artificial Analysis listing. - Real-world accuracy on your own data. The card also shows an unverified, externally submitted evaluation (safeagents judge bench v0, marked "verified: false" on Hugging Face) with a balanced accuracy of 0.502 on that judging task, so test it before relying on it.
Related
Up: AI
