We cannot yet name the most accurate or fastest option: the available provider examples use different tests. A high confidence score can still be wrong. Try each model on examples from your own work before letting it approve an action.
Do you need a choice, a value or a location?
Choosing the support team is different from reading an invoice total or finding a damaged part in a photo. Liquid d1 2B and d1 Omni 2B can return values and image boxes as well as choices and scores. For example, ask for the invoice date and total, then check whether either was actually found before using it. A missing value must not become zero.
These extra answer formats require the Transformers software setup. Their llama.cpp setup supports only yes/no, choice and score questions. For object locations, supply exactly one picture. Choose d1 Omni 2B when you also need audio clips or spoken questions.
Test the decisions you will actually act on
Give each model the same records, questions and answer choices, with known correct answers. Count wrong approvals, cases that should have gone to a person, and unanswered questions. Measure response time and cost too. Check confidence scores on separate examples: when a model says 90% confident, is it actually right about nine times out of ten? Record how many examples you tested; a small test leaves more uncertainty.
For example, a support-ticket test can ask for the destination team, urgency and whether the record contains enough information. Record when the model is confidently wrong and what a retry costs. Extra questions and images can add input charges. Liquid’s hosted decision models bill each question as its own prompt, including its images; multiply the billed prompt by the question count before using the budget estimate.
When does reusing a document save money?
GPT-6 Luna Decisions can reuse an identical shared document across requests once it reaches 1,024 tokens. The questions themselves are not cached. Writing the reusable copy costs more than ordinary input, while reading it costs less. At standard short-input rates, writing a 10,000-token document once costs $0.00125; reading it twice adds $0.0002. That totals $0.00145, versus $0.003 for three uncached copies. Question tokens and any regional charges are extra; savings require actual cache hits.
Baseten uses a different rule within one request: shared input is counted once, then each question adds its own tokens. Current default input prices per million billed tokens are $0.01 for Joey, $0.035 for Elephant and $0.10 for Mammoth. For 100 million billed input tokens, that is $1, $3.50 or $10 before other fees. This compares the same billed volume, not equal accuracy: a cheaper model may need more retries or human review.
Should you group questions together?
Perplexity Decider v1.2 reads related questions together, which can save repeated input charges. But the questions influence each other: asking about urgency alongside the destination team may change both answers. Score independent candidates separately. Flash answers every question separately and bills the shared information again each time. For three questions about the same 10,000-token record, Flash charges roughly $0.0003 for the repeated record; regular Decider charges roughly $0.0002 if all three fit in one group. Question tokens add to both bills. Keep grouping fixed when comparing answer quality.
Will an old model name still run the old model?
Not always. Perplexity now sends native requests for Decider v1 and v1.1 to v1.2, even though the response repeats the name you requested. Recheck your confidence thresholds after this change. The older downloadable weights and Gateway’s unlisted v1.1 offer need their own version check.
Did the model you requested actually answer?
Sqwish can automatically switch models when the requested one is unavailable or busy. That helps keep an application running, but can make a comparison misleading. For a controlled test, turn automatic fallback off and save the model name and version returned with every answer. In production, decide whether a substituted answer is acceptable or needs review. A fixed request name alone is not enough.
What happens when the model is unsure?
You can send uncertain answers to another model or a person. AI Gateway supports confidence rules that call a second model, but both calls are billed. Choose the threshold using examples with known answers: a confident mistake will not be caught just by raising the cutoff. How Gateway’s fallback rules work ↗
What does a low token price mean for your workload?
Liquid d1.1 now costs $0.02 per million billed input tokens: $2 for 100 million. Output is free, and cached input has the same rate. Each question is charged for its complete text and images, so one request with several questions can cost more than reading the document once. Use the billed total from a realistic sample in the calculator above.
For simple local routing, Lumma 0.1B v2 offers a roughly 300 MB download that can run on a CPU. Its text and question must fit in 2,048 tokens. The publisher reports weaker results on difficult judgment tasks, so test the cases that matter to you before replacing a larger model.
Can retries change the bill?
Yes. A connection failure does not always mean the service did no work. Sqwish’s optional zero data retention mode does not keep answers for reuse: sending the same request again runs it again and costs more, even when your software supplies the same retry key. Automatic retries count too. Check your privacy setting and measure billed requests, not just successful answers.
How much does a picture add to the bill?
Sqwish’s image-capable Kettei models now charge for the image size they process: 64–2,048 input tokens per unique picture. Its example 640 × 480 photo adds 300 tokens; a 1920 × 1080 screenshot adds 2,040. Include those tokens in the estimate above. Sending the same picture once and referring to it again does not add a second image charge.
Will adding questions make the request too large?
Drex Max v1.1 and Max Finance fit the shared document, images and every question into one 262,144-token budget. Original Drex Max checks the document against each question separately and can shorten long text. All three allow up to 512 questions, but a large question set may reach a size or timeout limit first. Keep the model version and question grouping fixed in your comparison.
Will your information fit?
The same model name can come with different limits online and on your own hardware. For example, Mammoth’s own API now allows 16,384 tokens per question’s complete prompt, matching the listed capacity of its downloadable model and Gateway offer. Gateway lists text input; native Baseten also supports images. Choose the service first, then check its limits.
Drex v2.0 now accepts up to 131,072 tokens of shared information, or 262,144 including the longest question. Longer requests are refused; they are no longer handed to v1.5. The whole request must also fit its 4 MB size limit. Running the downloadable v2.0 yourself has a separate limit of up to 131,072 tokens.
Jev on AI Gateway allows 32,000 tokens for the information shared across questions plus the longest question, and 64,000 tokens across the whole request. Its maximum number of questions is not confirmed. The exact input quote is $0.042 per million, with zero output-token price.
Laya on AI Gateway lists an 8,192-token limit; its maximum number of questions is not confirmed. The free promotion has ended. Gateway now charges $0.0105 per million billed input tokens, or $1.05 for 100 million, with no output-token charge.
The current Perplexity v1.2 API requires the shared information, images and each question to fit below 256K input tokens, and accepts up to 128 questions. Regular Decider splits oversized groups and bills the shared information again; Flash repeats it for every question. Gateway lists 262,144 context tokens. The older v1.1 local download accepts only 8,192 tokens per decision.
Clef documents 65,536 tokens, Flash 24,576 and Omni 64,000, each with up to 64 questions. Media and questions share the available space. Some background text can be cut off to make everything fit. A larger limit does not guarantee that the model will use every detail correctly. Clef and Flash model descriptions mention video, but their hosted API instructions explicitly document images. Omni’s API also documents audio and video. Check the service you will actually use.
A downloadable model still needs hardware, electricity, software and maintenance. Include those costs when comparing it with an online service.
Track verified releases and access changes ↗
AI Agent Store