name: arbor

description: Autonomously improve a real artifact (code, training recipe, agent harness, data pipeline, prompt) against an objective and an evaluator, using Hypothesis Tree Refinement (HTR) from the Arbor paper. Use this whenever someone wants to iteratively optimize something over many experiments without overfitting — e.g. "get my model's eval score up", "improve this agent/harness", "tune this pipeline", "beat the baseline on this benchmark", "run a search over approaches and keep the best", "do an MLE-bench / Kaggle-style optimization", or any long-horizon "make this artifact better and don't just memorize the dev set" task. Trigger it even when the user doesn't say "Arbor" or "hypothesis tree" but describes repeated experiment-and-evaluate loops, branching exploration of competing ideas, or worries about a dev/test gap. Runs Claude itself as the coordinator with subagent executors in isolated git worktrees; for the standalone arbor CLI tool see references/arbor-upstream.md.

allowed-tools: Read Write Edit Bash Agent

license: MIT license

metadata: {"version": "1.0", "skill-author": "K-Dense Inc."}

Arbor — Autonomous Optimization via Hypothesis Tree Refinement

Overview

This skill runs an Autonomous Optimization (AO) loop: starting from an existing artifact and a measurable objective, improve it through many rounds of experiment and evaluation — without step-by-step human supervision and without overfitting to the feedback signal. It's the right tool when the bottleneck isn't writing one good change, but organizing dozens of trials so that lessons accumulate instead of evaporating.

It implements Hypothesis Tree Refinement (HTR) from Arbor (Jin et al., 2026). The key idea: keep the research state in a persistent hypothesis tree rather than in conversation history. Each node binds a hypothesis, the distilled insight it produced, and a pointer to the artifact version that realizes it. You play the long-lived coordinator that owns this tree and decides where to search; short-lived executor subagents test one hypothesis each in isolated git worktrees and report back. A held-out merge gate admits a change only when it improves on a test evaluator the search never optimized against. This is what turns trial-and-error into cumulative, auditable research.

Use the scripts/tree.py state manager for all the bookkeeping (creating nodes, writing evidence, propagating insights, pruning, the merge gate, the Observe projection). It keeps the state consistent and frees you to spend judgment on what the evidence means.

When to use this skill

Reach for Arbor when the task is iterative improvement of a concrete artifact under an evaluator:

• Model training: optimizer/architecture/recipe changes to lower loss or hit a target in fewer steps.
• Harness/agent engineering: raising pass rate or accuracy of an agent loop, search harness, or tool-use scaffold.
• Data synthesis: improving a generation/filtering pipeline judged by downstream model behavior.
• Benchmark optimization: MLE-bench / Kaggle-style "improve the submission" tasks.
• Prompt/system optimization where you can score outputs automatically.

The distinguishing signals: there's an artifact you can modify, an objective, a way to score candidates, and you expect to run many experiments. If the user only wants a single fix or a one-shot answer, this is overkill — just do the work directly. If they want open-ended ideation with no evaluator, use hypothesis-generation or scientific-brainstorming instead.

The AO setup — pin this down first

Before any experiments, establish the task tuple (M_0, O, E_dev, E_test). Getting this right matters more than any later decision, so confirm it explicitly:

• M_0 — initial material: the artifact to improve (a repo, a script, a config, a prompt). Make sure it's under git and currently runs.
• O — objective: the natural-language goal and the metric direction (maximize accuracy? minimize loss/steps?).
• E_dev — development evaluator: a command you can run freely during search to score a candidate. Fast, repeatable.
• E_test — held-out test evaluator: a separate evaluator (different seeds, different split, or a larger run) used only at the merge gate. It must not be used as a search oracle — that's the whole point.

If the user hasn't given you a clean dev/test split, construct one and say so. The dev/test separation is the mechanism that catches overfitting: a candidate that wins on dev but not on test isn't a success, it's a warning that you're exploiting the feedback signal. Without it, autonomous search reliably overfits.

Initialize the run:

python scripts/tree.py init \
  --objective "Improve BrowseComp answer accuracy on the search harness" \
  --dev-eval "python eval.py --split dev --n 50" \
  --test-eval "python eval.py --split test --n 300" \
  --material "." --metric-direction max --branching 3 --m