Judgment

The Judgment block asks TypeSafe's Jev model a typed question about text or JSON and returns an answer with probabilities, never free text. Use it to classify, decide yes or no, or score content, then branch on the answer, a probability, or the confidence with a Condition block.

Operations

Choose One

Pick one option for the state.

• Options: one row per option with a slug and a description. Slugs must match ^[a-z0-9_]+$ because they become reference path segments.
• Include none of the above: adds an option with the fixed slug none for when no listed option applies.

Outputs answer (the chosen slug), probabilities keyed by slug, and confidence.

Yes or No

Decide whether a condition holds.

• Yes threshold: the yes probability at or above which answer is true. Default 0.5.

Outputs answer as a boolean and probabilities.yes and probabilities.no. There is no confidence output; a two-way probability already carries it.

Score

Place the state on an ordered scale.

• Levels: two to ten level descriptions in order, lowest first. Each level should describe a concrete situation that stands on its own.

Outputs answer (the description of the most likely level), level (its index, starting at 0), score (the probability-weighted position, which may fall between levels), probabilities keyed by level index, legend, and confidence.

Ask Several

Ask several questions about one State in a single Jev request. Use it instead of chaining Judgment blocks that all judge the same text: one round trip, one cost line, and every answer arrives together.

• Questions: a builder with one card per question. Give each an Id (typed text is coerced to the slug rule ^[a-z0-9_]+$ because ids become reference path segments), pick a Type of Choose one, Yes or no, or Score, write the Instructions, and fill in that type's criteria: option slugs and descriptions with an optional "none of the above", a yes threshold, or ordered levels. Problems are named under the field they belong to, and the field label counts what is left to fix.

The copilot and the API write the same questions as a JSON object keyed by id, which the builder loads as cards:

{
  "team": {
    "type": "choose_one",
    "instructions": "Which team should handle this message?",
    "options": { "billing": "Invoices or refunds", "technical": "Bugs or outages" },
    "includeNone": true
  },
  "refund": { "type": "yes_no", "instructions": "Is a refund requested?", "threshold": 0.7 },
  "mood": {
    "type": "score",
    "instructions": "How upset is the sender?",
    "levels": ["Calm and polite", "Annoyed but civil", "Angry"]
  }
}

Try it. The flask button in the block header opens a Try it workbench in the bottom panel, beside Logs. It is not part of the block's configuration: a run never sees anything in it. The left column holds named sample messages. Suggest asks the workspace's wand model for a set that covers every option, level, and both yes/no outcomes, shaped like the State field (billed as wand usage); it runs only when clicked, and samples otherwise start blank. Judge sends every question against every sample in one request per sample using the workspace's TypeSafe key (its tooltip shows the request count and a rough cost first). Answers fill a grid of samples by questions, an inspector shows the probabilities behind any cell, and a yes/no meter marks every sample so moving the threshold shows which ones flip without re-judging. Each question card in the sidebar also shows a chip with the selected sample's answer. Results are marked stale when a question changes and are never saved. Samples are saved on the block so a team shares one calibration set; they stay out of the form and the canvas, but appear in execution logs like every other block input. Removing a sample can be undone for a few seconds, and a running Judge or Suggest can be cancelled.

Outputs answers, with one entry per question id shaped like the matching single-question operation: <judgment.answers.team.answer>, <judgment.answers.team.probabilities.billing>, <judgment.answers.refund.probabilities.yes>, <judgment.answers.mood.level>, and so on. model, tokens, and cost cover the whole request. A block accepts up to 20 questions.

Configuration Options

State

The text or JSON to judge. Reference a block output such as <agent.content> or enter it directly. JSON text is sent as structured state. State is limited to about 32,000 tokens; larger requests fail with a clear error.

Instructions

The question to answer about the state, as literal text. A reference tag such as <agent.content> inside Instructions, an option description, or a level description fails the run before anything is sent. Put dynamic content in State only, so untrusted text can never rewrite the question.

Outputs

Every operation also outputs model, tokens, and cost, in the same shape the Evaluator block uses, so run cost summaries and per-model breakdowns include Jev calls under the provider typesafe.

Ask Several nests the same fields under <judgment.answers.<id>.…>, one entry per question, and has no top-level answer.

Example Use Cases

Triage in one call

Trigger → Judgment (Ask Several: team, refund, mood) → Condition (<judgment.answers.mood.level> >= 2) → Human in the Loop or Route

Route by category with a human fallback

Agent (Summarize) → Judgment (Choose One) → Condition (<judgment.probabilities.billing> > 0.8) → Billing or Human in the Loop

Gate an automated reply

Trigger → Judgment (Yes or No: "Is this a repeat contact?") → Condition (<judgment.answer> === true) → Reply or Escalate

Escalate by severity

Agent (Draft) → Judgment (Score: calm, frustrated, angry) → Condition (<judgment.level> >= 2) → Escalate

Best Practices

• Branch on probabilities, not only the answer: <judgment.probabilities.spam> > 0.8 is stricter than <judgment.answer> === 'spam'. Send the rest to a Human in the Loop block.
• Do not gate irreversible actions on the answer alone when State is untrusted: Jev always returns a typed answer and does not treat instructions inside State as hostile. Combine a probability threshold with a Human in the Loop step before sending mail, creating tickets, or calling webhooks.
• Keep State small and factual: Jev does not count, compare numbers or dates, or reason step by step. Give it the facts the question needs and let code do the arithmetic.
• Write standalone level descriptions: for Score, each level should describe the situation, not just "low" or "high".
• Pin what you branch on: the block pins the Jev model version and records it in model, so saved Conditions on probabilities do not drift under a vendor release.
• Ask several questions about the same State in one block: three Judgment blocks on one message cost three round trips; Ask Several costs one.

Data and Privacy

State and Instructions are sent to TypeSafe, which is US-hosted, and are stored in SteelEngine execution logs like other AI block inputs. An organization-level key enables this for every workspace in the organization. TypeSafe states it does not train on customer requests; see the TypeSafe privacy policy for retention terms.