Arbor

--- name: arbor description: Autonomously improve a real artifact (code, training recipe,agent harness, data pipeline,prompt ) against an objective and an evaluator, using Hypothesis Tree Refinement (HTR) from the Arbor paper. Use this whenever someone wants to iteratively optimize something across many experiments without overfitting:e.g., “get my model’s evaluation score up,” “improve thisagent /harness,” “tune this pipeline,” “beat the baseline on this benchmark,” “run a search across approaches and keep the best,” “perform an MLE-bench or Kaggle-style optimization,” or any long-term task aimed at “improving this artifact without simply memorizing the dev set.” Trigger it even when the user doesn’t say “Arbor” or “hypothesis tree” but describes repeated experiment-and-evaluate loops, branching exploration of competing ideas, or concerns about a dev/test gap. RunsClaude itself as the coordinator with subagent executors in isolated Git worktrees; for the standalonearbor CLI tool, see references/arbor-upstream.md . allowed-tools: Read Write Edit Bash Agent license: MIT license metadata: {"version": "1.0", "skill -author": "K-Dense Inc."} --- # Arbor : Autonomous Optimization via Hypothesis Tree Refinement ## Overview Thisskill runs an Autonomous Optimization (AO) loop: starting from an existing artifact and a measurable objective, it improves it through many rounds of experimentation and evaluation:without step-by-step human supervision and without overfitting to the feedback signal. It’s the right tool when the bottleneck isn’t writing a single good change, but organizing dozens of trials so that lessons accumulate rather than being lost. It implements Hypothesis Tree Refinement (HTR) from Arbor (Jin et al., 2026). The key idea: store the research state in a persistent hypothesis tree rather than in conversation history. Each node contains a hypothesis, the distilled insight it produced, and a pointer to the artifact version that implements it. You play the role of the long-lived coordinator who owns this tree and decides where to search; short-lived executor subagents each test one hypothesis in isolated Git worktrees and report back. A held-out merge gate admits a change only when it improves upon a test evaluator that the search never optimized against. This is what transforms trial-and-error into cumulative, auditable research. Use thescripts/tree.py state manager for all the bookkeeping (creating nodes, writing evidence, propagating insights, pruning, the merge gate, the Observe projection). It keeps the state consistent and frees you to focus on what the evidence means. ## When to use thisskill Turn to Arbor when the task involves iterative improvement of a concrete artifact under an evaluator: - Model training: changes to the optimizer, architecture, or training recipe to reduce loss or reach a target in fewer steps. - Harness/agent engineering: increasing the pass rate or accuracy of anagent loop, search harness, or tool-use scaffold. - Data synthesis: improving a generation/filtering pipeline evaluated based on downstream model behavior. - Benchmark optimization: MLE-bench / Kaggle-style “improve the submission” tasks. - Prompt/system optimization where you can automatically score outputs. The distinguishing features: there is an artifact you can modify, an objective, a way to score candidates, and you expect to run many experiments. If the user only wants a single fix or a one-shot answer, this is overkill:just do the work directly. If they want open-ended ideation without an evaluator, usehypothesis-generation orscientific-brainstorming instead. ## The AO setup : pin this down first Before any experiments, establish the task tuple(M_0, O, E_dev, E_test) . Getting this right matters more than any later decision, so confirm it explicitly: - M_0 : initial material: the artifact to improve (repo , script, config, orprompt ). Make sure it’s under Git and currently runs. - O : objective: the natural-language goal and the metric direction (maximize accuracy? minimize loss/steps?). - E_dev : development evaluator: a command you can run freely during the search to score a candidate. Fast, repeatable. - E_test : held-out test