[Pixel Post] Toward an Automatic Research Machine

Lately I’ve been circling around a question: what if computers didn’t just train models, but actually did the research itself?

Humans stumbled on ideas like attention and Adam—simple equations that changed the trajectory of machine learning. But these were artifacts of our own creativity, not inevitabilities. Could a machine, armed only with raw compute and a way to score usefulness, rediscover such breakthroughs—or even invent new ones we never imagined?

The sketch is simple:

1. Representation. Everything must be written as math: symbolic expressions, LaTeX-style equations, or graph-based wiring of blocks. This keeps the space interpretable and bounded.

2. Generation. Start with nonsense. Let a generative model or evolutionary search propose candidate equations or architectures.

3. Evaluation. Throw these candidates into automated experiments. Most will collapse, diverge, or be useless. A few will train faster, generalize better, or show unexpected structure.

4. Selection. Keep what works, simplify it, archive it. Rinse and repeat.

The beauty is that machines don’t get bored. With the right loop, they become tireless junior researchers—exploring, failing, and occasionally producing sparks of insight.

This isn’t a polished roadmap. It’s an anchor: a reminder that automatic discovery might be the next frontier. Instead of hand-designing the next optimizer or architecture, we might focus on building the research machines that invent them for us.

That’s the horizon I want to keep in sight.