
Optimizing AI Agents: Prompts vs. Weights (GEPA, GRPO)
Michael Ryan, a Stanford CS PhD student behind several widely used prompt-optimization algorithms, asks a practical question for anyone building AI agents: when should you optimize the prompt versus fine-tune the model's weights? This CS329Z: Engineering AI Agents lecture surveys context optimization methods including bootstrapped few-shot prompting, OPRO, MIPROv2, TextGrad, and GEPA, then turns to weight optimization approaches such as supervised fine-tuning on agent rollouts, PPO versus GRPO, and on-policy distillation. Ryan also covers agents that optimize themselves, citing the Darwin Godel Machine and autoresearch systems, before laying out a practical decision framework for choosing between prompt and weight optimization in real deployments. Ryan's own work underpins the DSPy open-source project and prompt-optimization tools used at Meta, Google, Dropbox, and Amazon, giving the lecture an industry-grounded perspective on a fast-moving area of machine learning engineering.