Part 1 of 2 · Concepts, architectures and model landscape. In Part 2, we test decision models on practical hospitality data.

Not every AI task needs another generated answer. Sometimes our software needs a bounded judgement: does a guest's message attempt to manipulate an assistant, and does the assistant's reply stay within its boundaries?

Those are two different decisions about the same conversation. Asking a model twice means sending much of the same information twice. In a voice or messaging workflow, the extra requests can add latency and consume tokens without providing any additional context.

At CASABLANCA hotelsoftware, our work on evaluating AI pipelines and building safer agents keeps bringing us back to this distinction. A conversational assistant powered by a large language model (LLM) needs to generate language. Its surrounding application often needs a category, a score or a reason to ask a person for review.

Decision models specialise in that second requirement: evaluating bounded options defined by the application, rather than composing a free-form answer. They do not make classification new. What makes them interesting is the flexibility to describe different questions and answer sets at request time, without training a separate classifier for each task.

In this article, we explain how decision models work, what makes them different and where their limits matter. Part 2 looks at how they behave in practice. We use hospitality conversations to test shared context, latency, stability and disagreements. We also compare the results with Luna, a reasoning model we already use at CASABLANCA.

One request, one bounded decision

Imagine a guest writes: “I would like a double room for two nights.”

An assistant could respond conversationally. A routing component needs a different result: does this message describe a new `booking`, a `change` to an existing reservation, or `other`?

The Microsoft Decision-1 application programming interface (API) uses a shared `state` and named `questions`. Here is a simplified request in JavaScript Object Notation (JSON):

{
  "state": "I would like a double room for two nights.",
  "questions": {
    "intent": {
      "type": "choice",
      "instructions": "Classify the guest's intent.",
      "criteria": {
        "booking": "New reservation or availability request",
        "change": "Modify or cancel an existing reservation",
        "other": "Any other request"
      }
    }
  }
}

‍

The application supplies the option names and their meanings. A different request could ask about security risk or which tool to call, without training a new classifier for every rubric. That flexibility does not mean every new task will be accurate: each workflow still needs testing.

Many decision-oriented APIs support three related question types:

Exact payloads and confidence formulas differ between providers. Multiple questions can also refer to the same state. This does not mean that the model executes them as an explicit reasoning chain: the application still owns dependencies, permissions and business rules.

LLM vs. classifier vs. decision model: what's actually different?

"Isn't this just a classifier - or an LLM returning JSON?" Both objections are fair: neither classification nor structured output is new. The difference is the combination of generalisation across decision tasks, direct scoring instead of text generation, and probabilities designed to support calibrated decisions.

1. New questions, classes and tasks without retraining

A traditional specialist classifier is usually trained for a particular task with a fixed label set. A generalist decision model can take new questions, criteria and class descriptions at runtime - and can answer multiple different classification or scoring questions about the same state in one request, rather than requiring one fine-tuned classifier per task. LLMs and some zero-shot classifiers can also handle changing classes; the difficult part is learning to generalise reliably across many unrelated decisions.

An early catalyst was TypeSafe AI's Jev, one of the first general-purpose decision models offered through a dedicated API, which helped popularise this approach and has remained competitive on the Artificial Analysis Decision Index.

In Sebastian Raschka's blog post comparing Jev with a fine-tuned ModernBERT classifier, the benefit becomes concrete: a classifier trained on movie reviews would need new training for an unrelated task such as Tetris, while Jev can be given a different task through its question and options. Raschka also explains in Section 4 that adding a Jev-like API - an interface for scoring caller-defined options - is relatively straightforward; making it generalise reliably across unfamiliar tasks is the difficult part.

2. Faster, cheaper decisions without generating an answer

A generative LLM can return the same JSON result, but ordinarily generates its response token by token. Many decision models instead compute scores for the allowed options and return a probability distribution without an autoregressive answer-generation loop.

How much faster and cheaper? In one of our own conversation-safety evaluations, Microsoft Decision-1 was roughly 10–12× faster than our Luna reasoning-model deployment. At published GPT-6 Luna standard list prices, the recorded token usage also suggests around 4–5× lower model costs for that workload. These are specific deployment measurements and a conditional pricing estimate, not a universal speedup, verified billing saving or proof of equal accuracy; we examine the results in Part 2. An independent study of Jev versus LLM rubric judges likewise reports substantial performance and cost differences, although its results depend on how the judges were called.

The gains vary substantially by model, task, context length and hosting. Decision-1 is neither automatically the fastest nor the cheapest option: the independent Artificial Analysis Decision Index compares decision models on both latency and cost per task, alongside decision quality.

The architectural path also differs, although there is no single decision-model architecture. Cloudflare Clef runs a Qwen backbone to obtain input representations, then a Joint Schema Head uses option-to-input and cross-question attention to score all permitted answers. Base Labs' Mammoth-1 and Elephant-1 instead score option-label logits using the existing language-model (LM) head, with modified attention masks and a reusable shared-context cache across question suffixes. The innovation is not a new softmax layer: it is how the model learns, processes and scores decisions without generating an entire textual response.

3. Training and confidence calibration

Decision models are trained on structured decision examples such as state + question + options + target distribution. Their scoring path produces one logit per option and typically applies softmax to obtain probabilities. In many systems, `confidence` is then a deterministic calculation from these probabilities, not the output of a separate confidence head.

TypeSafe Jev documents its formula for a `choice` question: with n allowed options and p_max as the highest option probability, it normalises that maximum relative to a uniform distribution:

For Figure 1's three options with probabilities 0.80 / 0.15 / 0.05, the selected answer has probability 0.80, but the normalised confidence is 0.70. A confidence of 0.70 is therefore not a separate claim of 70% accuracy. Different providers and question types may calculate the reported field differently.

A generative LLM also has next-token probabilities, but asking it to "state your confidence" normally makes it generate a number as text, not produce a tested probability distribution over the actual decision options. Training matters: cross-entropy rewards probability on the reference answer, while Brier loss penalises inaccurate probability distributions, including overconfident mistakes.

Where generative LLM post-training often uses Reinforcement Learning from Human Feedback (RLHF) to align generated responses with preferences, TypeSafe Jev and Cloudflare Clef describe Reinforcement Learning for Calibrated Decisions (RLCD) to optimise decision quality and probability calibration. These objectives are not mutually exclusive, and RLCD is not mandatory: Base Labs documents cross-entropy plus Brier loss without a described RLCD phase. Most importantly, only testing against representative, independently labelled examples can show whether predicted probabilities and human-in-the-loop thresholds are reliable in your application.

In short: a specialist classifier may be best for one stable, high-volume task; an LLM is useful when you need generated language or open-ended reasoning; and a generalist decision model is worth considering when you need flexible decisions, low-latency scoring and usable probability estimates.

Different models, different approaches

Decision models are a useful category of systems, not one standardised architecture or training recipe. The following is a short guide to what is publicly known; the linked technical write-ups contain the implementation details.

One distinction is worth remembering: diffusion describes a modelling approach, contrastive learning an objective for learning representations, RLCD a reinforcement-learning strategy, and a schema head an architectural component. These terms describe different layers of a system, not competing model families. For example, Inception's generative Mercury research explains iterative token refinement, but we cannot assume that Mercury Decide uses that exact denoising loop. Likewise, Liquid AI's published LFM2.5 hybrid uses gated short convolutions and attention; this does not establish a pure state-space backbone for d1-ultra.

This is not a benchmark ranking. For an independent and time-dependent view of model quality, latency, calibration and cost, see the Artificial Analysis Decision Index. We have not benchmarked most of the models listed here.

Where decision models are useful - and where they are not enough

Decision models are useful wherever software needs to choose between options or assign a score. Examples include booking-intent routing, support triage, security and guardrail checks, code-review findings, and tool selection for coding agents.

One promising pattern is human-in-the-loop review. First, test the model on independently labelled examples. Then choose a confidence threshold for each model and task, below which decisions go to a person for review. Low-risk decisions above the tested threshold may be automated, while high-impact actions may always require approval. This is related to selective classification.

Confidence alone does not guarantee correctness. A model may confidently select the wrong option, particularly on unfamiliar data. The same reported confidence can correspond to different error rates across models, so review thresholds are not directly transferable. Before automating decisions, test candidate thresholds against actual errors and the number of cases sent for review. Repeat this validation when switching models. Longer context windows and consistent answers are not substitutes for accuracy tests.

In Part 2, we move from the concepts to experiments with a practical hospitality conversation dataset. We test how decision models handle shared context, additional questions and longer inputs. We also look at the stability of their judgements and where they disagree. Alongside the two decision APIs, Microsoft Decision-1 and Inception Mercury Decide, we compare results with Luna, a reasoning model already used extensively at CASABLANCA. The aim is not to declare a winner. We want to understand how these models behave on relevant tasks and what still needs validation before using their decisions in production.

References