HC2024-K1: Predictable Scaling and Infrastructure

Keynote 1, Hot Chips 2024, Monday, August 26, 2024
Trevor Cai, OpenAI
In this keynote talk, Trevor gives a brief history of large language models (LLMs) like ChatGPT and describes how compute hardware has been increasing exponentially in order to keep up with the size and complexity of these rapidly developing models ... and will need to continue increasing in size and complexity for the foreseeable future, because currently there is no end in sight as these models continue to improve.