Skip to content

Your first research project

This page walks the complete loop on a question of your own. Every command is something you type; everything between the commands is either a file you write or code your coding agent writes for you.

The worked question here is aqueous solubility prediction, but the shape is the same for any computational investigation.

1. Create the project and state the question

science-adk init my-research
cd my-research
science-adk goal \
  --set-question "Does a GNN embedding predict aqueous solubility better than Morgan fingerprints?" \
  --metric rmse \
  --target 0.8

This writes GOAL.md:

GOAL.md
---
question: Does a GNN embedding predict aqueous solubility better than Morgan fingerprints?
target_metric: rmse
target_value: 0.8
---

# Research Goal

## Question
...
## Background
...
## Success
...
## Constraints
- ...

Open it and fill in Background and Constraints. The frontmatter is the part the framework reads; the prose is what makes the project reviewable by another scientist six months later.

A good question is falsifiable and bounded

"Can a GNN beat a fingerprint baseline on ESOL?" can be settled by an experiment. "Investigate ML for chemistry" cannot.

2. Write your tools

Drop plain Python functions into tools/. Every public top-level function is discovered automatically — the docstring becomes the description and the type hints become the schema.

tools/descriptors.py
"""Molecular descriptor tools."""

from __future__ import annotations


def morgan_fingerprint(smiles: str, radius: int = 2, bits: int = 2048) -> list:
    """Compute a Morgan (ECFP) fingerprint as a bit vector.

    Returns a list of `bits` zeros and ones. Raises ValueError if the SMILES
    string cannot be parsed, rather than returning an all-zero vector that
    would silently pollute the training set.
    """
    ...

Check what the registry found:

science-adk tools

Tools can also be MCP servers — see tools and MCP.

3. Create an experiment

science-adk experiment new \
  "GNN embeddings outperform Morgan fingerprints on ESOL solubility prediction" \
  --rationale "Learned 3D representations should capture solvation geometry that topological fingerprints discard."

This creates research/001-gnn-embeddings-outperform-morgan-fingerprints/HYPOTHESIS.md.

A hypothesis is not a goal restated. It is a claim that the run could contradict.

4. Define the DAG

Write workflow.json inside the experiment directory. Nodes declare typed ports; edges wire an output port to an input port of the same (or compatible) type.

research/001-.../workflow.json
{
  "name": "GNN vs Morgan on ESOL",
  "description": "Train both representations on the same split and compare RMSE.",
  "nodes": [
    {
      "id": "config",
      "name": "Experiment parameters",
      "kind": "config",
      "ports": {
        "inputs": [],
        "outputs": [{ "name": "settings", "type": "json" }]
      }
    },
    {
      "id": "load",
      "name": "Load ESOL",
      "kind": "data",
      "ports": {
        "inputs": [{ "name": "settings", "type": "json" }],
        "outputs": [{ "name": "dataset", "type": "dataframe" }]
      },
      "tools": ["load_esol"]
    },
    {
      "id": "train",
      "name": "Fit both models",
      "kind": "training",
      "ports": {
        "inputs": [{ "name": "dataset", "type": "dataframe" }],
        "outputs": [{ "name": "predictions", "type": "dataframe" }]
      },
      "tools": ["morgan_fingerprint"]
    },
    {
      "id": "evaluate",
      "name": "Compare RMSE",
      "kind": "evaluation",
      "ports": {
        "inputs": [{ "name": "predictions", "type": "dataframe" }],
        "outputs": [
          { "name": "metric", "type": "str" },
          { "name": "value", "type": "float" },
          { "name": "detail", "type": "json", "required": false }
        ]
      }
    }
  ],
  "edges": [
    { "source": "config", "source_port": "settings", "target": "load", "target_port": "settings" },
    { "source": "load", "source_port": "dataset", "target": "train", "target_port": "dataset" },
    { "source": "train", "source_port": "predictions", "target": "evaluate", "target_port": "predictions" }
  ]
}

The evaluation node is mandatory

The result_measured gate requires at least one node of kind evaluation that outputs a value. Without it the run cannot be scored.

5. Write one agent per node

Each node needs agents/<node id>.py containing exactly one Agent subclass whose ports match the node's declaration.

research/001-.../agents/train.py
"""Fit both representations on an identical, seeded split."""

from science_adk import Port, Ports, TrainingAgent


class TrainBoth(TrainingAgent):
    """Train a GNN and a Morgan-fingerprint baseline on the same split."""

    ports: Ports = Ports(
        inputs=[Port("dataset", "dataframe")],
        outputs=[Port("predictions", "dataframe")],
    )

    async def execute(self):
        dataset = await self.input("dataset")
        train, test = self.split(dataset, test_fraction=0.2)   # (1)!
        await self.log(f"{len(train)} train / {len(test)} test")

        fingerprints = [
            await self.call_tool("morgan_fingerprint", smiles=s)
            for s in train["smiles"]
        ]
        ...
        return {"predictions": predictions}
  1. split() is seeded and deterministic, which is what keeps the comparison honest across re-runs.

See write an agent node for the full programming model, and agent primitives for what each base class adds.

6. Validate

science-adk validate

Static, instant, and worth running constantly. It checks the DAG for cycles, dangling edges and port-type mismatches, and the agent source for missing execute, mismatched ports and integrity hazards such as a bare except: that would swallow a real failure.

error  edge load.dataset -> train.dataset: type 'json' cannot feed 'dataframe'
       fix: Change one of the port types so they match.

7. Run

science-adk run

The runner executes nodes in topological order, passes outputs along the edges, and writes the whole thing to research/001-.../runs/<timestamp>/trace.json — inputs, outputs, logs, timings, fingerprints and the provenance ledger.

science-adk run --reuse            # re-run only what changed
science-adk run --only train evaluate

8. Score

science-adk score

  [pass] execution_completed: All nodes ran to completion.
  [pass] all_nodes_ran: 4 node(s) produced output.
  [pass] result_measured: evaluate reported a measurement.
  [pass] metric_is_finite: evaluate reported 0.72.
  [pass] produced_output: 7 output value(s) recorded.
  [pass] tool_use_verified: Declared tool use matches the record.

Any failure means 0.00. Fix the cause, not the gate.

9. Audit and record

Gates prove the number is real. The audit says whether it matters.

science-adk audit \
  --scientific-value 0.85 \
  --method-fidelity 0.90 \
  --implementation-quality 0.95 \
  --rationale "GNN embedding reaches RMSE 0.72 on held-out ESOL against 0.95 for Morgan fingerprints on the same seeded split."

science-adk report
science-adk learn "GNN embeddings cut solubility RMSE by 24% versus Morgan fingerprints on ESOL." --run latest

10. Evolve

science-adk experiment new \
  "Adding solvent-accessible surface area to GNN features improves solubility prediction" \
  --parent 001-gnn-embeddings-outperform-morgan-fingerprints \
  --rationale "SASA captures explicit solvent exposure the GNN may not learn from structure alone."

science-adk status
science-adk campaign

--parent records lineage, so the campaign leaderboard shows not just which experiment won but which question led to which.

Next