Autoresearch Agent

Features

• Autonomous experiment loop for file optimization
• Automated editing, evaluation, committing, and reverting
• Support for various domains (engineering, marketing, content, prompts)
• Configurable evaluation commands, metrics, and directions
• Git integration for versioning and rollback

Use Cases

• Use when you want to optimize code speed, reduce bundle/image size, or improve test pass rates.
• Use when you need to improve content quality such as headlines, copy, or click-through rates.
• Use when optimizing prompts for LLM interactions or agent instructions.
• Use when running any measurable improvement loop that can be automated.

Non-Goals

• Modifying the evaluation script or external dependencies.
• Performing optimization without a clear, measurable metric.
• Handling the initial setup of the target project's build or testing environment.

Workflow

1. User runs `/ar:setup` to configure experiment parameters (target file, evaluation command, metric, direction).
2. Script creates experiment directory, config files, and a git branch.
3. User (or AI agent) calls `python scripts/run_experiment.py --single`.
4. Script edits the target file based on AI's instruction (or a generated change).
5. Script runs the evaluation command and parses the metric.
6. Script compares the new metric to the best metric found so far.
7. If improved, the change is committed; otherwise, the repo is reset.
8. Result (keep/discard/crash) is logged to `results.tsv`.
9. This loop continues until interrupted or a goal is met.