SciResearch-Eval: Evaluating Scientific Judgment in Autonomous Experimental Agents
We introduce SciResearch-Eval, an interactive benchmark of scientific judgment built on a standard laboratory problem: measuring a material's thermal properties by heating it and fitting a heat-conduction simulator. An agent designs its own experiments, uses a supplied simulator that may be under-resolved or physically wrong, and reports each property with a 95% interval. To our knowledge it is…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
Source: arxiv.org