DwarfStar / GB10 matched comparison
Primary: GLM Q2 versus Qwen Q2. Secondary: Qwen Q4.
Experiments remain in separate directories. Thinking modes and clarification diagnostics never replace original scores. See datasets.json and each frozen protocol for exact counts and controls.
All initial scored trials and pilot answers retained.
Scoring is automated checking plus Codex-assisted review, not expert grading.
Raw requests, responses, model-generated reasoning and SSE events are preserved. Model-generated reasoning is evaluation data and may contain errors.
Operational logs remain private; their hashes are supplied. Aggregate shared-memory and I/O metrics are in audit-results.json.
Startup failure before inference is documented separately and is not a quality failure.
Code answers were tested only in isolated Docker with fixed tests; never execute untrusted answers on the host.
