Skip to content

Provenance and datasets

The provenance ledger is the mechanism that makes a Science ADK result checkable rather than merely plausible. It is recorded by the runtime, in a structure the agent's own code does not write, and it is what the tool_use_verified gate reads.

What gets recorded

Every node's trace entry carries a provenance object with four lists:

Key Written by Each entry
tool_calls await self.call_tool(...) {"tool": "power_law_fit", "ok": true}
inputs_read await self.input(...) {"port": "planets", "found": true}
urls_fetched await self.fetch(...) {"url": "https://...", "ok": true}
files_written self.artifact(...) {"file": "residuals.png"} 
"provenance": {
  "tool_calls":   [{"tool": "power_law_fit", "ok": true}],
  "inputs_read":  [{"port": "planets", "found": true}],
  "urls_fetched": [],
  "files_written": []
}

Failures are recorded too. If a tool raises, the entry is written with "ok": false before the exception propagates — an agent cannot fail invisibly and then claim the tool never ran.

Why the agent cannot forge it

The four recording calls are the only way to reach the outside world through the supported API, and each appends to the ledger as a side effect of doing its real work:

async def call_tool(self, name, **arguments):
    try:
        result = await self._services.call_tool(name, arguments)
    except Exception:
        self._provenance["tool_calls"].append({"tool": name, "ok": False})
        raise
    self._provenance["tool_calls"].append({"tool": name, "ok": True})
    return result

There is no supported way to append a successful tool call without a tool having actually succeeded. The only way to make the ledger say a fit happened is to perform the fit.

The gate that reads it

For every node that declares tools in workflow.json and finished in state passed or cached, the runtime requires at least one tool call with "ok": true. Otherwise:

[FAIL] tool_use_verified: Node(s) declared tools but made no successful tool
       call: analyze. Either the data was not really fetched, or the
       declaration is wrong.

The error deliberately offers both readings. Sometimes the declaration is stale and the fix is to remove it; sometimes the computation was quietly replaced by a constant. The gate cannot tell the difference, so it makes a human look.

The CI fabrication test

On every push, CI runs the Kepler example, then rewrites analyze.py to replace the call_tool line with a hard-coded perfect answer (exponent = 1.5, r_squared = 1.0) and runs it again. Every structural gate still passes. tool_use_verified fails and the run scores 0.00. If that assertion ever stops holding, the build breaks.

Datasets

Large values would make a trace unreadable and mutable-by-accident, so they spill to disk and travel as pointers.

Any value with more than 200 elements is externalised into the run's data/ directory:

Value Written as Pointer records
pandas.DataFrame <name>.parquet (falls back to .csv) shape, columns
numpy.ndarray <name>.npy shape, dtype
Anything else JSON-able <name>.json length or keys

The trace then holds a reference rather than the data:

"outputs": {
  "planets": {
    "$dataset": "data/planets.parquet",
    "format": "parquet",
    "bytes": 184320,
    "shape": [3565, 4],
    "columns": ["planet_name", "period_days", "semi_major_axis_au", "host_star"]
  }
}

Downstream, await self.input("planets") hydrates the pointer transparently — the agent receives the dataframe, not the dictionary. Small values are inlined in the trace directly, so a scalar metric stays readable.

Two consequences worth knowing:

  1. Traces stay diffable. A git diff on trace.json shows what changed in the result, not three megabytes of re-serialised floats.
  2. Data cannot be mutated out of band. The value a downstream node reads is the file the upstream node wrote, with its size recorded.

Reading provenance back

Inside an agent:

await self.log(f"tool calls so far: {self.provenance['tool_calls']}")

From the shell:

jq '.nodes[] | {node_id, provenance}' research/*/runs/*/trace.json

Reference