Site menu:
OPT2024
We welcome you to participate in the 16th International OPT Workshop on Optimization for Machine Learning, to be held as a part of the NeurIPS 2024 conference. This year we particularly encourage (but not limit) submissions with a focus on "scaling up optimization".
We are looking forward to an exciting OPT!
The Workshop
Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops. We aim to foster discussion, discovery, and dissemination of state-of-the-art research in optimization relevant to ML.
The focus of OPT 2024 is on "Scaling up optimization". The advent of large language models (LLMs) has changed our perceptions of the landscape of optimization and is resulting in the emergence of new interesting questions related to scaling. For instance, we can view optimization as a sequence of problems parameterized by the size of the model. Questions naturally arise around scaling and optimization. Are there natural model size dependent learning rates that allow extrapolation from smaller models to large ones, and therefore facilitating fine-tuning? Or given a fixed compute budget, how should one choose the hyper-parameters of the model (e.g., width size, depth size, architecture, batch) so as to minimize the loss function? How dependent are these scaling laws on the optimization algorithm? Answers to these questions would have a huge impact in AI – saving time and millions of dollars in training, plus helping reduce AI’s environmental impact through reducing energy costs. The new area of scaling laws and its deep ties to the optimization community warrants a necessary discussion.
Please note that this (in-person) workshop will take place at the Vancouver Convention Center.
The OPT2024 Organizers (preferred contact email: optmlworkshop@googlegroups.com)
- Jelena Diakonikolas (University of Wisconsin)
- Dan Garber (Technion)
- Cristóbal Guzmán (chair) (Pontificia Universidad Católica de Chile)
- Courtney Paquette (McGill University)
- Sebastian Stich (CISPA Helmholtz Center for Information Security)
Registration
For registration and pricing see NeurIPS.cc.
Important Dates
- Deadline for submission of papers: September 27, 2024 (AoE)
- Notification of acceptance: October 9, 2024 (AoE)
- Camera-ready papers upload to openreview: Dec 1, 2024 (OPT2024 style file required)
- Workshop date: December 15, 2024