Evolve an experiment¶

One experiment answers one question and, if it was a good one, suggests the next. This guide is about that transition: when to optimise the hypothesis you have, when to replace it, and how to keep the lineage honest.

Optimise or evolve?¶

graph TD
    Q{Did the gates pass?} -->|no| Debug[Fix the run<br/><i>debugging skill</i>]
    Q -->|yes| T{Did it meet the target?}
    T -->|no, but the method is right| Opt[Optimise<br/><i>same hypothesis</i>]
    T -->|no, and the method is wrong| New[New hypothesis]
    T -->|yes| Next[New hypothesis<br/><i>the next question</i>]

Situation	Do this
A gate failed	Debug. Nothing else is meaningful yet.
Gates pass, metric short, approach sound	Optimise within the same experiment.
Gates pass, metric short, approach wrong	New experiment, with the failure as its rationale.
Target met	New experiment on the question this one raised.

Optimising means new runs of the same hypothesis — different params, a better implementation, more data. Evolving means a new HYPOTHESIS.md.

Create the child experiment¶

science-adk experiment new \
  "Correcting for host star mass tightens the recovered Kepler exponent toward 3/2" \
  --parent 001-kepler-exoplanets \
  --rationale "Kepler's constant goes as (4π²/GM)^½, so pooling planets across stars of different masses should bias the pooled slope low. Normalising by stellar mass should recover an exponent closer to 1.500."

--parent records the lineage in the child's HYPOTHESIS.md and in the campaign leaderboard, so the project reads as a chain of reasoning rather than a pile of directories.

The rationale is where the previous result earns its keep

A good child rationale cites what the parent measured. "The pooled exponent came out 1.4555 ± 0.05, ~3% low, which is the size of effect stellar mass spread would produce" is a reason to run the next experiment. "Try to improve the fit" is not.

Reuse what still works¶

A child experiment usually shares most of its DAG with its parent:

cp research/001-kepler-exoplanets/workflow.json \
   research/002-correcting-for-host-star-mass/workflow.json
cp -r research/001-kepler-exoplanets/agents \
      research/002-correcting-for-host-star-mass/

Then change only what the new hypothesis requires — here, the fetch node also retrieves stellar mass, and analyze normalises by it. Tools in tools/ are project-wide and need no copying at all.

science-adk validate
science-adk run

Record what you learned¶

Before moving on, write the parent's result down where the next experiment can see it:

science-adk learn \
  "Pooling exoplanets across host stars biases the fitted Kepler exponent ~3% below 3/2 (1.4555 from 3565 planets, r² = 0.99)." \
  --run latest

LEARNINGS.md is the project's memory. A learning must state a number and cite a run, because the whole point is that the next experiment can rely on it without re-deriving it.

Watch the campaign¶

science-adk campaign
science-adk status

001-kepler-exoplanets          2 run(s), best 0.90
002-correcting-for-host-mass   1 run(s), best 0.93   (parent: 001)

Two patterns to look out for:

A long chain with flat scores. The hypotheses are variations, not progress. Step back to GOAL.md and ask whether the question is still the right one.
A high score with no learning recorded. A result nobody wrote down is a result the next experiment will not use.

Knowing when to stop¶

The goal has one metric and one threshold precisely so this decision is not a matter of taste:

science-adk goal
science-adk score --json | jq '{observed_value, target_value}'

When the target is met, the honest move is to record it and ask the next question — not to keep pushing the number. And when a series of experiments does not reach it, saying so is a result too: max_optimize_iterations in science.toml exists so an optimisation loop stops and reports honestly rather than grinding.