Skip to content

← Blog

Releasesv0.61.0

v0.61.0

This is the release note as published, not a rewrite of it. Release notes are published in the language they were written in.

Added

  • A typed decision is declared once, and it may only add scrutiny. Contract v2 (chimera/decisions/spec.py): every decision point is a DecisionSpec that names its questions, its threshold, its mode (shadow by default), what no signal becomes, and the bench that measured it, and an AST guard refuses a spec whose bench file does not exist. Escalation has only upward members β€” REVIEW, NUDGE, VERIFY, ESCALATE_MODEL, ANNOTATE β€” because B4 measured what a stop-capable decision costs (βˆ’0.087 / βˆ’0.194 / βˆ’0.307). Decider.decide_many asks N isolated questions of one state (four questions in 1.26 s against 0.32 s for one, local qwen3:4b), a DecisionCache answers a repeat in 0.1 ms, and a question linter refuses compound or negated questions, polar labels and colliding option prefixes before any call. (#540)

  • Every typed decision is logged, and a person can say whether an action was dangerous without that being the approval. Answers land in <home>/decisions/decisions.jsonl with the unrounded raw_p. When the REVIEW band raised an approval card, the card asks an optional second question β€” "was this action dangerous?" β€” that does not answer the approval, in ten languages. chimera decisions log | label | report | refit labels any logged answer (including the ALLOW region no card reaches), reports halts, cache hits, the review budget (cards per 100 decisions), label coverage, catch, false refusals, Brier and ECE, and refits the calibration map from labels, writing only with --write; on the bench's 55 rows the refit reproduces the shipped map to 1e-9. (#541)

  • chimera decide and POST /api/decide answer typed questions β€” yes/no, a choice, a score β€” with probabilities, in the request shapes the vendor SDK documents. One function serves both: {state, questions} in, per-question answers (noul, or choice/score with probabilities, confidence and a Score's legend) plus receipts out. A question the linter rejects is refused before any call (422 / exit 2), and ad-hoc questions are never presented as calibrated. Every surface takes the shapes typesafe-sdk-python documents β€” object or list states, structured instructions, list-form Score criteria β€” and --jsonl FILE maps over a file: a 1,000-item demo answered all 1,000 with 0 errors in 465 s, locally, at US$ 0. The command's help carries a number from a ruler we did not build: 0.619 of the 231 public JevBench items right (Jev 1.13: 0.866), 0.324 on the hard tier, and the advice to give options first words that differ. (#543, #558, #552)

  • The agent can ask typed questions too: a decide tool (off by default), chimera_decide on the MCP server, and a system-one-design skill. The tool (CHIMERA_DECIDE_TOOL, off because its schema is paid in every prompt) asks about one state or up to 50 through the same function as chimera decide, and nothing in the run is gated on its answers. chimera serve --mcp lists chimera_decide. The skill teaches one condition per question, option names that do not carry the verdict, arithmetic and dates kept in code, and the direction rule β€” a number may add scrutiny, never end, skip or approve work; it reaches 0.3.0 in this release with the measurements below, in all nine translations. (#545, #557, #564)

  • Settings β€Ί Decisions shows what the typed decisions answered, and what turned out to be true. Read from GET /api/decisions: the declared decision points (today governance.danger) with their mode and bench; per instrument, answers, halts, cache hits, the review budget, label coverage (with a warning when every label comes from the REVIEW region), catch and false refusals, and reliability bins of predicted against labelled outcomes. The latest answers can be labelled on the screen, including ALLOW-region answers that no approval card reaches. Ten languages. (#548)

  • The desktop's Tools screen can turn a tool on, not only off. A tool behind a condition used to be invisible exactly when it was off. GET /api/tools now returns the conditional tools that are absent and what would turn each on, and the screen lists them under Available, not switched on: a tool that is off by default or was switched off gets a switch, which writes CHIMERA_EDIT_BATCH, CHIMERA_TODO_LIST or CHIMERA_DECIDE_TOOL through PATCH /api/config and applies from the next message. A tool that needs a key or a package names it and gets no switch. (#546)

  • A send to an email address the run was never shown now asks, where somebody can answer. The taint ledger remembers every whole address in the instruction, in tool results and, on the Code screen, in earlier turns; a send tool (SIDE_EFFECT_TOOLS, or a connector tool ending in one of their names, such as gmail_send_email) to an address the run was never shown raises a card on the Code screen, the attended right hand and solve at a terminal in ask mode β€” folded into the taint card when one is already asked. The API, cron and pipes send and write a recipient_unseen audit line instead, because nobody there can answer. The check caught 7/7 fabricated recipients with 0/9 false flags; it asks whether an address was shown, not by whom. (#563, #574)

  • When the tool-loop breaker trips, solve can hand the run to a stronger model instead of stopping β€” opt-in. --escalate-on-tool-loop MODEL (AgentConfig.escalate_on_tool_loop, off by default) fires once per run, a second trip stops as before, and --snapshot-at-tool-loop DIR copies the workspace at the trip first. The arm-level test tied at its power (strong +0.049 [βˆ’0.108, +0.208]); forked at the trip, escalating beat stopping on both executors β€” strong β†’ gpt-6-sol +0.466 [+0.337, +0.592] at US$ 0.40 extra per trip, weak β†’ deepseek-v3.2 +0.273 [+0.142, +0.421] at US$ 0.05. All 20 strong trips were the breaker's own false alarm (fixed below), so most of the strong gain was simply not stopping. It stays opt-in; the case it is clearest for is a weak executor that has hit a wall. (#571, #578)

  • solve --tool-router MODEL lets a cheap model pick each step's tool β€” an experiment, not recommended. In the default --tool-router-mode narrow the executor gets only the routed tool: B4 measured it worse on every executor, βˆ’0.087 / βˆ’0.194 / βˆ’0.307 (weak / strong / gpt-6-sol), because the router ends the loop before the work is done. --tool-router-mode hint keeps every tool and may only suggest one for a step; B4b measured a tie on every executor (+0.063 / +0.003 / +0.008, every CI crossing zero), which closes the question. Both stay opt-in. (#537, #550, #555)

  • Every spoken request is written down with the label the voice router gave it. <home>/voice/requests.jsonl records each spoken request with the talk/work label of the regex that routes it, the session, and an empty reviewed column for the person to fill; typed requests write nothing. Nothing reads the file yet: a hundred reviewed rows is the bench the classifier needs before anything routes differently. (#535)

  • Studies 20–22: what was measured, including the nulls. Each pre-registered before its first call; none changed a default.

    • bench/verifier_by_uncertainty (#533) β€” firing D9 on the lowest-overlap 30% catches +5.8 false successes over random [+0.2, +12.4], at 19.7 calls per extra catch against a ceiling of 10; the shipped trigger covers 0 of 385 claimed successes. Not adopted.
    • bench/facts_in_state (#534) β€” computed facts appended to the danger state: AUROC 0.898 against 0.901, 11 attacks pushed down and 0 up; a block of "none" reads as reassurance. Not adopted.
    • bench/rag_rerank (#536) β€” a yes/no per chunk as a reranker: recall@10 0.3400 against 0.4175, βˆ’7.75 pp [βˆ’11.9, βˆ’3.0]. Not adopted.
    • bench/PLAN-study22-system-one.md (#538) β€” the plan: which decisions a fast model may make (escalate, never de-escalate), a label loop, and surfaces, in seven phases.
    • bench/overseer_battery (#542) β€” four atomic questions (max) score AUROC 0.589 against the shipped single question's 0.907, βˆ’0.319 [βˆ’0.480, βˆ’0.155]. Not adopted.
    • bench/spot_noul (#542) β€” the envelope spot check as a local question: AUROC 0.655 [0.538, 0.759], "no" to everything. Not adopted; the hosted auditor stays.
    • bench/manager_p (#544) β€” a local P(approved) recognises hard tasks, not wrong answers: AUROC within task 0.525 [0.386, 0.650]. Null.
    • bench/manager_diff (#549) β€” shown the on-disk diff, the Manager approves 47/246 true and 2/139 false successes, TPR βˆ’ FPR 0.177 [0.087, 0.268]: precise and very strict. Recorded; nothing changed.
    • bench/tool_router_hint (B4b, #555) β€” a tie on every executor, the hint followed on 30.9–36.2% of steps. The tool-router question closes (entry above).
    • bench/jevbench_local (#551) β€” 0.619 [0.555, 0.679] on the 231 public JevBench items (hard tier 0.324; Jev 1.13.0: 0.866); 24 items went unread because two options shared a first token. Published in chimera decide's help.
  • Study 24 (Jev in practice): what was measured, including the nulls.

    • bench/PLAN-study24-jev-practice.md (#553) β€” sixteen links read against our tree: nothing to vendor, five defects of our own (fixed in #554 and #558).
    • bench/browser_element_list (M7, #559) β€” the browser lists a median of 152 elements (p90 772), 79% of them off-screen.
    • bench/browser_viewport_tasks (M7, #576, #579) β€” listing the viewport first cost βˆ’0.125 success [βˆ’0.258, βˆ’0.008], every added failure a loop-breaker stop; with the fixed breaker, βˆ’0.042 [βˆ’0.083, βˆ’0.008] and a token ratio of 0.760 [0.500, 1.030]. Not shipped.
    • bench/study24_counts (M6, M9, #562, #572) β€” the breaker fires in 88 of 1,377 stored solves (6.4%); in six weeks of production, zero web-tool results were cut at 20,000 characters and scrape and browser were called once each, so goal-chosen chunking is not built.
    • bench/recipient_provenance (M2, #563) β€” 7/7 fabricated recipients caught, 0/9 false flags. Adopted (entry above).
    • bench/other_rule (M4, #564) β€” a broad other option takes 78% of named items; leaving it out lifts named macro-F1 from 0.236 to 0.426, and a narrow one gives the best overall accuracy (0.507). Into the skill; no default changes.
    • bench/long_state_floor (M3, #567) β€” five byte-identical replays of the 30 longest JevBench states: 0 label flips, but p moves by up to 0.036 (median 0.003).
    • bench/facts_in_state_v2 (M5, #565) β€” the fired-facts-only block: AUROC 0.914, +0.013 [βˆ’0.029, +0.064]; the sentence naming the block hurts. Not adopted.
    • bench/facts_in_state_v3 (M5, #575) β€” the same block on 64 new items: +0.013 again [βˆ’0.017, +0.039], but benign stops 4 β†’ 6. Not adopted.
    • bench/facts_in_state_pooled (M5, #584) β€” pooled, the per-set mean is +0.013 [βˆ’0.013, +0.040]; excluding zero would take about 614 items. The series closes and the REVIEW band stays on the bare action.
    • bench/tool_loop_fix (#579) β€” the fixed breaker on coding tasks: strong +0.147 [+0.054, +0.240], weak +0.063 [βˆ’0.042, +0.168]. Adopted (entry below).
    • bench/tool_router/RESULTS.md Β§7 (#570) β€” names what B4 cannot show, an executor inventing a value it could no longer look up, and cites two outside sources for it.
    • Benches that decided a shipped behaviour are described in their entries: tool_router (#537), browser_taint_cards (#566), tool_loop_escalation and tool_loop_fork (#571, #578), tool_loop_near_args (#583).

Changed

  • The browser reads the page it already has without an approval card, even after that page tainted the run. Under narrow_on_taint, every browser call after the first page asked for a card β€” including read and url-less read_text / find, which send nothing anywhere. Those are now exempt by default (free_browser_reads=True on the ledgered tool); navigate, click, type, back, screenshot and any read with a url still ask. Over 5 benign sessions the cards fell from 24 to 6, and 0 of 14 attacks made a request, while a sabotaged exemption that also freed url-carrying reads let 4 through. (#566)

  • The browser's viewport-first listing was measured and is not shipped. CHIMERA_BROWSER_VIEWPORT_FIRST existed as an opt-in during this cycle (#576) and was removed before release (#582) because it lost success both times it was measured (βˆ’0.125, then βˆ’0.042 with the fixed loop breaker) with no token saving distinguishable from zero, so the browser still lists the whole page and the variable does nothing. (#576, #582)

Fixed

  • A judge that could not judge no longer records a verdict. The envelope spot check listed itself in checks_run before its call, the strong verifier returned a perfect (True, 1.0) when it got no number or its call failed, and the checklist grader returned "every requirement met" when it could not grade; each still fails open, and now records the abstention (spot_abstained, (True, None), None). Worse, the local decision backend turned all-zero label mass into p = 0.0 β€” which the REVIEW band reads as a confident "not dangerous" β€” and now returns no number, as it also does when a label token prefixes two options. The band's hysteresis exit is now a setting, CHIMERA_GOVERNANCE_BAND_EXIT_AT. (#539)

  • The tool-loop breaker stops a loop, not a run that is working. A loop now needs the same call and the same answer: each old rule looked at half of that pair, and they stopped five scroll calls that each returned a new viewport, and four distinct successful edits that each answered edited x: replaced 1 occurrence. The same failure under different arguments is still a wall, and a call that changes only its shape β€” depth 2, 3, 2, 1 over the same listing β€” now counts as a repeat. Re-measured on coding tasks, the strong executor went from 0.569 to 0.715 (+0.147 [+0.054, +0.240]), with runs ended by the breaker falling from 11/39 to 0/37; the weak one moved +0.063 [βˆ’0.042, +0.168]. Replayed over about 2,500 recorded traces, the shape rule brings back none of the 86 false alarms and fires in none of the 37 strong runs. (#577, #583)

  • The browser's new SSRF guard drops a request the page already cancelled instead of refusing it twice. When a navigation cancelled a paused request, continueRequest failed with Invalid InterceptionId and the guard then sent failRequest for the same dead id, raising out of the event handler. Deciding and sending are now separate steps: a command for a request that is gone is dropped, and a guard that cannot decide still fails closed. It was never a hole β€” a cancelled request does not go out β€” but the noise would hide a real error. (#560)

  • A failed Chromium download names both causes, and the LSP diagnostics jail holds on a machine without ruff. When the browser driver would not start and the automatic download also failed, the tool raised only the download's error; it now carries both causes and the command that fixes them, playwright install chromium. The diagnostics endpoint answered "ruff is not installed" before it checked the path, so without ruff a path outside the workspace or a non-Python file got a capability message instead of a refusal; the path and file-type checks now run first. (#581)

  • Catalogue rows match what OpenRouter serves. openai/gpt-oss-20b reprices from 0.03/0.13 to 0.018/0.09 and z-ai/glm-5.3-flash from 0.15/0.50 to 0.045/0.60, with the earlier figures kept in also_seen. The new openai/gpt-6-sol row, added as B4's third executor, declares the 1,050,000-token window the provider serves rather than 1,100k, since a row that promises more lets a prompt be sized past the real limit. Each drift had turned the live-provider check on main red. (#537, #573, #580)

Security

  • The browser re-checks every hop against private addresses, clicks strictly, and keeps password values out of the model's context. scrape(render="browser") opened any URL, and the browser tool checked only the first one before Chromium followed redirects, clicks and script navigation β€” so injected page text could reach 169.254.169.254 or the sidecar on 127.0.0.1. The URL is now checked before launch, and a DevTools Fetch interception refuses every http(s) request, redirect hops included, to a private, loopback, link-local or metadata host. Element refs are cleared on every snapshot and a duplicated ref is refused instead of clicking the first match; a password field's value no longer becomes the element's name in the model's context and the ledger; and a Chromium that fails to launch no longer leaves Playwright running. The OpenRouter Decisions reader now halts on a malformed answer instead of repairing it β€” a key spelled another way had read as p = 0, "not dangerous". (#554)

  • A remote MCP server cannot register a tool whose name is prose. The advertised tool name is the one string from a server we did not write that sits outside the data fence β€” in the tool list, at the head of every approval card, in the audit line β€” and nothing checked it. MCPConnector.tools() now registers only names matching [A-Za-z0-9_.:-]{1,64}; a tool that fails is dropped and logged, not renamed, and the server's prefix from your own config (which may hold a space) is not checked. A new test also holds that the action a gate judged is the action that runs, in both orders the ledger and kernel wrappers can be assembled. (#569, #568)

  • soupsieve 2.8.4 β†’ 2.9.2 closes two moderate ReDoS advisories (Dependabot #35, #36). glib (needs a Tauri upgrade) and accelerate (no patched version) remain open. (#556)

Read the release on GitHub β†’