chimera measure

chimera measure rag

Recall@k of each retriever over a real folder — lexical, and vector when an embedder is set. This is the measurement `chimera/rag/__init__.py` names when it says the retriever's existence is not a claim that it helps. That sentence pointed at a module you could not run: `rag_bench` had no caller outside its own test and was not exported from `chimera.eval`. No embedder is passed, so the vector and hybrid figures come back as None rather than zero — an embedder that was never called did not fail, and printing 0.0 invites the wrong conclusion.

Аргументы

ROOTpathобязательный
Folder to index and probe.

Параметры

--kintпо умолчанию: 10
Retrieve this many chunks per probe.
--max-probesintпо умолчанию: 200
Cap the probe count; each one is a query.

chimera measure reranker

Leave-one-out AUC of the success reranker — does it discriminate, or is it noise? `chimera/evolution/reranker.py` says to measure with this BEFORE putting the reranker in a hot path. It was prose pointing at an unreachable module. AUC of 0.5 is a coin flip. A reranker at 0.5 is not a weak reranker, it is not a reranker.

Аргументы

CORPUSpathобязательный
JSONL of {query, text, success} records.

Параметры

--kintпо умолчанию: 5
Rank cut-off for the leave-one-out scoring.

Подкоманды

chimera measure rag

Аргументы

Параметры

chimera measure reranker

Аргументы

Параметры