agentic-rubric-runner

Overview

agentic-rubric-runner ingests a task brief and source PDF, produces a structured report, scores it against a rubric, and records every tool invocation for audit.

Use cases: due diligence, compliance review, proposal QA, and any workflow that requires document-grounded evaluation.

Install from PyPI

pip install agentic-rubric-runner

export DEEPSEEK_API_KEY="sk-your-key"
agentic-rubric --help

Install agentic-rubric-runner[web] only when you need the local Streamlit UI.

CLI

agentic-rubric phase1 \
  --query fixtures/query.txt \
  --pdf fixtures/attachment.pdf \
  --out outputs/demo

agentic-rubric run \
  --query fixtures/query.txt \
  --pdf fixtures/attachment.pdf \
  --rubrics fixtures/rubrics.json

agentic-rubric bench \
  --manifest fixtures/benchmarks/agent_cases.json \
  --out outputs/bench

Deploy Console

上传材料、运行 Phase 1 / Phase 2，下载 PDF、JSON 与审计轨迹。

agentic-rubric-runner.streamlit.app

在线评审 — 页面输入 API Key，结果缓存于当前会话。部署说明见 Streamlit Cloud guide。

Stack

Python
Function Calling
PyMuPDF
ReportLab
Pydantic
Typer
Streamlit

Audit

agent_trace.jsonl — step / timestamp / duration
Phase 1 path whitelist
run_meta.json — input SHA256
agent_eval.json — Agent run score
agent_benchmark_result.json — suite summary
Structured error codes E001–E007