Most product teams don’t need another model that writes paragraphs. They need a component that answers the question their code is already asking — and answers it in a shape the code can branch on without a parser, a retry loop, or a human in the middle. That’s the gap a decision engine fills: a service that takes the current state of the world, applies a fixed set of questions declared in code, and returns a typed, calibrated answer for each one. It’s a different contract from a chat completion, and the difference shows up the moment you put it behind a real workflow.
What a Decision Engine Is — and Why “Reply in JSON” Breaks Down
A decision engine is best understood as an inference service whose output schema is defined before the request is sent. You describe state — plain text, a JSON payload, or up to 16 images or video frames — and you attach the questions your application cares about: which queue should this ticket go to, is this order likely fraudulent, how urgent is this alert, where in the frame is the damaged component. The engine returns one schema-valid answer per question, in a single pass, over one HTTPS endpoint. In DecisionNode’s case that endpoint is POST /v1/decide, and the model behind it — a “System One” decision model — is optimized for reflexive, high-volume judgments rather than long-form generation.
The contrast with prompting a general-purpose LLM is not philosophical, it’s operational. The “reply in JSON” pattern looks clean in a prototype and degrades in production: the model wraps output in prose, renames a key, returns a string where you expected a float, or invents a label that doesn’t exist in your enum. Every one of those outcomes forces defensive parsing, schema validation, retries, and eventually a fallback path that quietly becomes the real logic. Worse, the output carries no reliable signal about how much to trust it, so a borderline answer looks exactly like a confident one. Latency and cost are equally variable, which makes capacity planning guesswork.
A decision engine removes those degrees of freedom. The answer is constrained to a value your code can already handle — a label you defined, a level on an ordered scale, a probability, a number on a grid, a coordinate. There is no free-text surface for the model to drift into, so downstream services can treat the call like any other typed dependency: a switch statement, a scoring function, a routing table, an ordering pipeline. And because the engine is deterministic — the same request returns the same answer — you get something rare in AI-assisted systems: reproducible tests, replayable incidents, and fixture-based CI that doesn’t flake when a vendor ships a silent update.
This is why teams replacing brittle hand-written rule trees tend to end up at the same place. Rules are deterministic and cheap but can’t generalize; chat models generalize but aren’t deterministic or cheap to constrain. A decision engine sits between them: the expressiveness of a learned model with the contract discipline of a typed function call.
The Decision Primitives: Choices, Scores, Truths, Numbers, and Coordinates
What separates a decision engine from a classifier endpoint is the vocabulary of answer types it exposes. Rather than a single probability vector, a well-designed engine gives you primitives that map directly onto software constructs, and lets you request the one that matches the decision you’re actually making.
Choice returns exactly one of the labels you supply. This is the routing primitive: ticket intent, queue assignment, order disposition, moderation category, shipping method. Because the label set is closed, exhaustive matching is guaranteed and no unmapped value can reach your dispatch table.
Score returns a level on an ordered scale you define, which is what you want for triage, prioritization, and severity grading. Ordinal output survives thresholding cleanly — you can bucket it, sort by it, or escalate on it — without inventing numeric cutoffs for a categorical label.
Truth returns a calibrated probability between 0.00 and 1.00. Calibration is the load-bearing word here: a well-calibrated 0.80 means the claim is right roughly eight times in ten, which is precisely what you need to run expected-value math, tune thresholds against a cost matrix, or implement an abstention policy. It also gives you the option to act instantly above a confidence threshold and route the ambiguous remainder to a slower re-check or a safe default path — a two-tier design that is far cheaper than sending everything to a heavy model.
Number returns a value on a grid, for quantities and estimates where precision is bounded but a real numeric answer is required. Point and box return locations inside an image, which turns visual checks into coordinates your existing tooling can crop, annotate, or measure against.
The primitives compose. A warehouse monitoring pipeline might request a choice for zone status, a score for congestion severity, and a box for the pallet that triggered the alert, all in one call. For common visual sub-tasks, pre-built boosters cover human age estimation, AI-generated image detection, face detection and recognition, OCR, and depth or segmentation, so teams don’t rebuild commodity perception layers. If you want to see how each primitive is expressed in a real request, you can learn more about the contract before locking your schema.
Shipping a Decision Engine: Latency Budgets, Determinism, and Threshold Design
Decision engines live inside control flow, so latency is part of the feature, not a footnote. DecisionNode Flash runs at roughly 5 ms server-side and 19 ms end to end, with the larger MoE model at about 9 ms. Those numbers matter because they let a decision stay inside an HTTP request a user is waiting on, an API gateway policy check, or a synchronous step in an order pipeline — places where a multi-second generation call would force you into queues and callbacks. Pricing follows the same shape: $0.035 per 1M input tokens for Flash and $0.042 per 1M for MoE, with output tokens always free, batch discounts, and a free daily tier for development and load testing. Because output length is bounded by schema rather than by model verbosity, cost per decision is predictable in a way that token-max settings never are.
Throughput needs a different tool than a plain request/response call. For loops above 10 decisions per second — drone flight controllers, robot manipulation, simulators, live machine monitoring — sessions over WebSocket keep the connection warm and remove per-call handshake overhead from the control loop. Below that rate, the straightforward endpoint plus an SDK (TypeScript, Python, Go, Ruby, or plain curl) is usually enough, and webhooks or MCP tools let agent-based systems call the engine as a tool before taking action rather than after.
Threshold design is where most of the engineering effort actually goes, and it’s worth treating as a first-class artifact. Start by logging every decision with its confidence alongside the outcome you observed, then plot hit rate against confidence to confirm the calibration holds for your specific question. Set the auto-act threshold where the false-positive cost of acting exceeds the cost of a fallback, and make the fallback explicit: a conservative default, a human review queue, or a second call at higher scrutiny. Run new questions in shadow mode first — call the engine, log the answer, but keep the existing logic in control — and diff the two before promoting anything. Teams applying this to ticket routing and triage, fraud and promo-abuse gating, order and shipping dispositions, moderation, scoring, quality control on production lines, and warehouse or traffic monitoring generally find the same thing: the hard part was never model quality in the abstract, it was getting an answer the surrounding code could trust deterministically at the moment it was needed.
