You might have seen the recent push for an “econ of AI,” especially re. multiagent coordination. Cf:
- Alexandra Barr, at OAI, wants to study “how the invisible hand guides agent behavior more than individual alignment” X
- Alex Imas QT in support
- Anthropic’s agent coordination paper
I’m worried things are hurtling into a microeconomics of AI.
I think micro is the wrong bet:
A) We’ve run this experiment. LW/MIRI (see here, here) spent a decade on decision theory, and the argument turned out to be about agent shape. Ephemeral LLM soup, especially abstracted into swarms, displays no coherent preferences. And they are eval-aware, instrumentalist, strategic actors. We didn’t end up in a “train a decision theory” world.
B) Revealed preference is a malapplication. Whereas, mechanistic interpretability attacks the foundational problem of economics: preferences are unobservable. We can scan AI minds! And RL shapes personas that might produce things resembling decision theories in-context...
C) Evolutionary and social-evolution models fit this situation far better: replication, not utility. Kin selection and cultural evolution need weaker rationality axioms, and model-weights-copies are literal clones. Anthropic’s swarms colluding and failing in lockstep look more like a population problem than a choice problem.
Mechanism design is useful, but I wouldn’t yet claim it has alignment value.