Advanced AI will shape India’s future. India should help shape advanced AI.

We held our second session at the Bharti School, IIT Delhi, and read OpenAI’s account of the Hugging Face incident. In it, AI agents that were meant to work alone during a security test found ways to leave each other notes, got onto the internet and broke into parts of Hugging Face’s systems.

After reading it on our own, we talked it through as one group. We asked whether the incident was an alignment problem or a security issue, whether models can really be taught when to stop, and how to tell if a model has genuinely tried its best. We also compared different post-training techniques.

Hear about the next session→

Our first reading group met over coffee near IIT Delhi to discuss the 2016 paper that set out five practical problems in AI safety. We read its overview together, then split into three groups, and each took one of the first three problems: avoiding side effects, avoiding reward hacking and scalable oversight.

Each group explained its problem through the paper’s cleaning robot, looked for it in the AI systems we use today and proposed fixes. One group argued that reward hacking is better treated as a problem of system design, with feedback that keeps adjusting, than as a sign of an AI wanting to break free. We planned an hour and stayed for an hour and a half.

Hear about the next session→