Siftline
Guide

Label real Records

Feed siftline label a file of Records and get one Decision line per Record.

A Record is one piece of text with an id. Its state is what the model reads, and it can be a string, an object or an array, so the subject and sender can stay as fields:

records.jsonl
{"id":"msg-1","state":{"subject":"Charged twice","sender":"anna@example.com","text":"You charged my card twice this month. I want the second charge refunded now, and I want to talk to an actual person, not a bot."}}
{"id":"msg-2","state":{"subject":"Export question","sender":"ben@example.com","text":"Hi, how do I export my projects to CSV? I found the export button but it only gives me JSON."}}
{"id":"msg-3","state":{"subject":"Re: invoice","sender":"dan@example.com","text":"Is the invoice from March going to be corrected or should I just pay it? Somebody promised a call back last week."}}

siftline label judges each one and writes a Decision line to stdout:

npx siftline label recipe.json records.jsonl > decisions.jsonl
decisions.jsonl
{"format":1,"id":"0192f3c2-7b1e-7c4a-9f0e-000000000001","recordId":"msg-1","recipe":{"name":"support-inbox","version":1},"model":"jev-1.13.0","judgedAt":"2026-09-22T09:00:00.000Z","trimmed":false,"answers":{"category":"complaint","wants_human":true},"questions":{"category":{"confidence":1,"probabilities":{"complaint":1,"question":0,"other":0}},"wants_human":{"probability":0.99,"confidence":0.98}},"confidence":0.98,"review":false,"rule":null,"action":null,"usage":{"inputTokens":412,"outputTokens":60}}
{"format":1,"id":"0192f3c2-7b1e-7c4a-9f0e-000000000002","recordId":"msg-2","recipe":{"name":"support-inbox","version":1},"model":"jev-1.13.0","judgedAt":"2026-09-22T09:00:00.000Z","trimmed":false,"answers":{"category":"question","wants_human":false},"questions":{"category":{"confidence":1,"probabilities":{"complaint":0,"question":1,"other":0}},"wants_human":{"probability":0.02,"confidence":0.96}},"confidence":0.96,"review":false,"rule":null,"action":null,"usage":{"inputTokens":404,"outputTokens":59}}
{"format":1,"id":"0192f3c2-7b1e-7c4a-9f0e-000000000003","recordId":"msg-3","recipe":{"name":"support-inbox","version":1},"model":"jev-1.13.0","judgedAt":"2026-09-22T09:00:00.000Z","trimmed":false,"answers":{"category":"question","wants_human":false},"questions":{"category":{"confidence":0.58,"probabilities":{"complaint":0.27,"question":0.72,"other":0.01}},"wants_human":{"probability":0.25,"confidence":0.5}},"confidence":0.5,"review":true,"rule":null,"action":null,"usage":{"inputTokens":406,"outputTokens":59}}

Decisions come out in input order, whatever order the model answers in. Every line is parsed before the first call, so a bad Record costs nothing. Records are read from a file, or from stdin when the second argument is - or missing, so label sits at the end of a pipe.

Reading a Decision

  • answers are the plain values, one per Question. This is what Rules and receivers read.
  • questions is the evidence: the confidence behind each answer, the probability of every label in Recipe order, and for a Score the model's expected value across the levels.
  • confidence is the lowest of those, and review is whether it fell under the Recipe's threshold.
  • recipe and model say which questions were answered and by what.
  • rule and action are null here. Rules fill them.

Unsure Decisions

Look at msg-3. The model leaned towards question at only 0.58, and was no surer whether the sender wanted a person. A Decision's confidence is the lowest across its Questions, 0.5 here, which is under the Recipe's threshold of 0.7, so the Decision carries review: true. It is still written, with its answers, because a low-confidence answer is still information. What is different is what happens next: an Unsure Decision selects no Rule and no Action, however you configure them.

The hosted Siftline app sends these to a Review queue where a person answers instead. The toolkit has no queue, so filtering on review is your job: send those lines to a person, and let the rest flow on.

Failures

A Record the model refuses is named on stderr and skipped; the run continues and exits 1 at the end. A rate limit or an outage that outlives the retries aborts the run, since a partial file with no marker is worse than none. If the model that answered differs from the one the Recipe pins, one warning names both.

Sending

label never sends. With --rules each Decision names the Action its Rule selected, and that is all: the CLI holds no webhook URL and no secret. Rules come next. To send, pipe the Decisions into a short script that keeps its secrets in the environment:

export SLACK_WEBHOOK_URL=... TICKETS_WEBHOOK_URL=... TICKETS_WEBHOOK_SECRET=...
npx siftline label recipe.json records.jsonl --rules rules.json | node send.ts recipe.json
send.ts
// siftline label recipe.json records.jsonl --rules rules.json | node send.ts recipe.json
import { readFileSync } from "node:fs";
import { createInterface } from "node:readline";

import { defineActions, dispatch } from "@siftline/actions";
import { parseDecision, parseRecipe } from "@siftline/core";

const recipe = parseRecipe(readFileSync(process.argv[2] ?? "recipe.json", "utf8"));

// The ids rules.json selects. URLs and the secret come from the environment, never from a file;
// an unset one fails here, before anything is sent.
const actions = defineActions({
  escalations: {
    kind: "slack_incoming_webhook",
    config: { url: process.env.SLACK_WEBHOOK_URL ?? "" },
  },
  "linear-tickets": {
    kind: "webhook",
    config: {
      url: process.env.TICKETS_WEBHOOK_URL ?? "",
      secret: process.env.TICKETS_WEBHOOK_SECRET ?? "",
    },
  },
});

for await (const line of createInterface({ input: process.stdin })) {
  if (line.trim() === "") continue;

  // `null` when the Decision went to Review or selected no Action: nothing to send.
  const sent = await dispatch(parseDecision(line), actions, recipe);

  if (sent !== null) console.log(`${sent.request.idempotencyKey} ${sent.response.status}`);
}

It parses each line with parseDecision and calls dispatch once. A Decision that went to Review, such as msg-3, or whose Rule selected no Action, sends nothing. The Action ids MUST be the ones rules.json names; an id the Rules select but the script does not define throws. Node runs the .ts file as is from 22.18 on the 22 line, and from 23.6. On 22.14 to 22.17, add --experimental-strip-types. Send an Action covers defineActions, dispatch and their errors.

A failed send throws and stops the pipe. The lines before it were sent, and each request carries an Idempotency-Key, so a receiver can drop the repeats when you run it again.

Next: turn Answers into an Action with Rules.

Measure it with Fixtures

Label a handful of real Records, then let siftline test grade the Recipe.

Route with Rules

An ordered list of conditions that maps Answers to an Action.

On this page

Reading a DecisionUnsure DecisionsFailuresSending