Weekly signal

This week saw concrete, practical progress toward agentic systems that can (a) read and interpret raw experimental evidence in wet labs, (b) expose how model training data affects discovery behavior, and (c) provide cross‑domain perceptual tests needed for real-world scientific observation — plus community coordination and funding windows to push those directions. The developments sharpen the immediate engineering checklist for teams that want to put agents into scientific workflows.

What changed

1. OpenAI released BioBasicsBench (Nov 6), a 100‑task benchmark (gels, colony plates, microscopy, instrument readouts, cloning workflows) built with enzyme‑design partner Ligo to measure how well agents interpret raw wet‑lab evidence and spot experimental errors; results show large but incomplete gains (GPT‑6 Astra ~66.7% mean reward at high reasoning effort vs ~43.8% for earlier models). Use of raw, messy evidence (not paper figures) and attention to control checks are explicit design choices.

2. A control‑style arXiv study (Nov 2) traced LLMs’ scientific‑discovery capacity to the presence of “monofacts” (facts seen only once in training data). The authors manipulate monofact rates and report consistent increases in discovery outputs in materials science and regulatory genomics, linking discovery signal to rare factual exposure rather than simple dataset diversity. That points to a concrete lever (training‑data composition) that influences what agents will propose as novel hypotheses.

3. ScienceVision (arXiv, early Nov) introduced a cross‑domain benchmark for scientific 3D observation and spatial reasoning: 250 expert‑curated questions across five domains (biology, chemistry, medical imaging, CAD, architecture). Results show models struggle on native 3D artifacts (best mean ~0.53) and that agent harnesses with measurement/inspection tools materially change outcomes. This highlights the perceptual gap for agents interacting with real scientific instruments and files.

4. The Stanford Data‑Centric AI Workshop (Nov 3) foregrounded agentic systems for biology and data‑centric methods; several talks and posters explicitly targeted agent orchestration, verifiable records, and lab‑validation pipelines — indicating the academic community is shifting from toy demos to reproducible, auditable agent workflows.

5. Amazon Research Awards / AWS posted an Agentic AI call for proposals (Fall 2026 CFP open through Nov 11) with modest unrestricted awards plus cloud credits — a near‑term funding path for projects that combine multi‑agent orchestration, verification, and domain safety for science applications.

What to do with it

• If you use agents near experiments: adopt BioBasicsBench as a pre‑deployment check and require blinded replay/simulation trials plus manual audit of control recognition.
• For model builders: treat training‑data composition (monofact rates) as a tunable: increasing rare, high‑quality domain facts can boost discovery signals but requires provenance controls to avoid memorization and spurious claims. Instrument these changes and run placebo/memorization tests.
• For perception + instrument teams: add native‑artifact inspection tools and MCP‑style measurement endpoints to agent harnesses (ScienceVision shows these shift results).
• Funders and program leads: prioritize projects that pair multi‑agent orchestration with auditable evidence trails and safety gates; consider applying to the AWS Agentic AI CFP before Nov 11.
• Attend or watch Stanford workshop sessions and poster recordings to align on evaluation practices and reproducibility expectations.