What If We’re Doing AI All Wrong?

In March 2024 a group of scientists joined a panel to discuss a seven-year-old research paper named after a Beatles song. The event took place at the San Jose McEnery Convention Center, where the line to get in snaked around the second floor. About 2,200 people made it into the auditorium or watched from an overflow room, while an additional 10,000 streamed the event online. Those inside the main room snapped photos on their phones; a few even sneaked up to the stage to get better shots. It was about as close as a tech conference can come to Beatlemania.

The paper, “Attention Is All You Need,” describes a novel architecture for artificial intelligence systems known as a transformer. Widely regarded as one of the most influential scientific publications of the 21st century, it’s earned more than 191,000 citations in other papers since its publication almost a decade ago. It serves as the technical foundation for the modern AI boom and is therefore arguably responsible for increasing the value of a handful of tech giants by trillions of dollars, creating some of the most valuable startups of all time and inspiring an eye-watering global construction boom in data centers. At the very least, the transformer has put the “T” in ChatGPT.