Site menu:
OPT2022
We welcome you to participate in the 14th International OPT Workshop on Optimization for Machine Learning, to be held as a part of the NeurIPS 2022 conference. This year we particularly encourage (but not limit) submissions in the area of Reliable Optimization Methods for ML.
We are looking forward to an exciting in-person OPT!
Accepted Papers
Accepted Papers (oral)
- On the Complexity of Finding Small Subgradients in Nonsmooth Optimization — Guy Kornowski, Ohad Shamir
- Differentially Private Adaptive Optimization with Delayed Preconditioners — Tian Li, Manzil Zaheer, Ken Liu, Sashank J. Reddi, Hugh Brendan McMahan, Virginia Smith
- Parameter Free Dual Averaging: Optimizing Lipschitz Functions in a Single Pass — Aaron Defazio, Konstantin Mishchenko
- Sufficient conditions for non-asymptotic convergence of Riemannian optimization methods — Vishwak Srinivasan, Ashia Camage Wilson
- Nonsmooth Composite Nonconvex-Concave Minimax Optimization — Jiajin Li, Linglingzhi Zhu, Anthony Man-Cho So
- Strong Lottery Ticket Hypothesis with epsilon-perturbation — Fangshuo Liao, Zheyang Xiong, Anastasios Kyrillidis
- How Does Sharpness-Aware Minimization Minimizes Sharpness? — Kaiyue Wen, Tengyu Ma, Zhiyuan Li
Accepted Papers (poster)
- Private Stochastic Optimization With Large Worst-Case Lipschitz Parameter: Optimal Rates for (Non-Smooth) Convex Losses & Extension to Non-Convex — Andrew Lowy, Meisam Razaviyayn
- Completing the Model Optimization Process by Correcting Patterns of Failure in Regression Tasks — Thomas Bonnier
- Quantization based Optimization: Alternative Stochastic Approximation of Global Optimization — Jinwuk Seok, Changsik Cho
- Optimizing the Performative Risk under Weak Convexity Assumptions — Yulai Zhao
- On Convergence of Average-Reward Off-Policy Control Algorithms in Weakly Communicating MDPs — Yi Wan, Richard S. Sutton
- BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach — Mao Ye, Bo Liu, Stephen Wright, Peter Stone, Qiang Liu
- Solving a Special Type of Optimal Transport Problem by a Modified Hungarian Algorithm — Yiling Xie, Yiling Luo, Xiaoming Huo
- Random initialisations performing above chance and how to find them — Frederik Benzing, Simon Schug, Robert Meier, Johannes Von Oswald, Yassir Akram, Nicolas Zucchet, Laurence Aitchison, Angelika Steger
- Adaptive Methods for Nonconvex Continual Learning — Seungyub Han, Yeongmo Kim, Tae Hyun Cho, Jungwoo Lee
- An Accuracy Guaranteed Online Solver for Learning in Dynamic Feature Space — Diyang Li, Bin Gu
- Optimization using Parallel Gradient Evaluations on Multiple Parameters — Yash Chandak, Shiv Shankar, Venkata Gandikota, Philip S. Thomas, Arya Mazumdar
- Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods — Aleksandr Beznosikov, Eduard Gorbunov, Hugo berard, Nicolas Loizou
- A Unified Framework to Understand Decentralized and Federated Optimization Algorithms: A Multi-Rate Feedback Control Perspective — Xinwei Zhang, Nicola Elia, Mingyi Hong
- RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updates — Laurent Condat, Peter Richtárik
- Boosting as Frank-Wolfe — Ryotaro Mitsuboshi, Kohei Hatano, Eiji Takimoto
- Gradient Descent: Robustness to Adversarial Corruption — Fu-Chieh Chang, Farhang Nabiei, Pei-Yuan Wu, Alexandru Cioba, Sattar Vakili, Alberto Bernacchia
- Accelerated Riemannian Optimization: Handling Constraints to Bound Geometric Penalties — David Martínez-Rubio, Sebastian Pokutta
- Accelerated Algorithms for Monotone Inclusion and Constrained Nonconvex-Nonconcave Min-Max Optimization — Yang Cai, Argyris Oikonomou, Weiqiang Zheng
- Accelerated Single-Call Methods for Constrained Min-Max Optimization — Yang Cai, Weiqiang Zheng
- Counterfactual Explanations Using Optimization With Constraint Learning — Donato Maragno, Tabea Elina Röber, Ilker Birbil
- A Stochastic Prox-Linear Method for CVaR Minimization — Si Yi Meng, Vasileios Charisopoulos, Robert M. Gower
- Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence — Diyuan Wu, Vyacheslav Kungurtsev, Marco Mondelli
- Solving Constrained Variational Inequalities via a First-order Interior Point-based Method — Tong Yang, Michael Jordan, Tatjana Chavdarova
- Understanding Curriculum Learning in Policy Optimization for Online Combinatorial Optimization — Runlong Zhou, Yuandong Tian, Yi Wu, Simon Shaolei Du
- Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability — Alex Damian, Eshaan Nichani, Jason D. Lee
- Learning deep neural networks by iterative linearisation — Adrian Goldwaser, Hong Ge
- Adaptive Inexact Sequential Quadratic Programming via Iterative Randomized Sketching — Ilgee Hong, Sen Na, Mladen Kolar
- Differentially Private Federated Learning with Normalized Updates — Rudrajit Das, Abolfazl Hashemi, Sujay Sanghavi, Inderjit S Dhillon
- Fast Convergence of Random Reshuffling under Interpolation and the Polyak-L ojasiewicz Condition — Chen Fan, Christos Thrampoulidis, Mark Schmidt
- Reducing Communication in Nonconvex Federated Learning with a Novel Single-Loop Variance Reduction Method — Kazusato Oko, Shunta Akiyama, Tomoya Murata, Taiji Suzuki
- Improved Deep Neural Network Generalization Using m-Sharpness-Aware Minimization — Kayhan Behdin, Qingquan Song, Aman Gupta, David Durfee, Ayan Acharya, Sathiya Keerthi, Rahul Mazumder
- Online Min-max Optimization: Nonconvexity, Nonstationarity, and Dynamic Regret — Yu Huang, Yuan Cheng, Yingbin Liang, Longbo Huang
- A Second-order Regression Model Shows Edge of Stability Behavior — Fabian Pedregosa, Atish Agarwala, Jeffrey Pennington
- Near-optimal decentralized algorithms for network dynamic optimization — Judy Gan, Yashodhan Kanoria, Xuan Zhang
- Stochastic Gradient Estimator for Differentiable NAS — Libin Hou, Linyuan Wang, Qi Peng, Bin Yan
- Neural DAG Scheduling via One-Shot Priority Sampling — Wonseok Jeon, Mukul Gagrani, Burak Bartan, Weiliang Will Zeng, Harris Teague, Piero Zappi, Christopher Lott
- Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization — Kaiwen Zhou, Anthony Man-Cho So, James Cheng
- PyEPO: A PyTorch-based End-to-End Predict-then-Optimize Library with Linear Objective Function — Bo Tang, Elias Boutros Khalil
- adaStar: A Method for Adapting to Interpolation — Gary Cheng, John Duchi
- A Light-speed Linear Program Solver for Personalized Recommendation with Diversity Constraints — Haoyue Wang, Miao Cheng, Kinjal Basu, Aman Gupta, Sathiya Keerthi, Rahul Mazumder
- Clairvoyant Regret Minimization: Equivalence with Nemirovski’s Conceptual Prox Method and Extension to General Convex Games — Gabriele Farina, Christian Kroer, Chung-Wei Lee, Haipeng Luo
- Annealed Training for Combinatorial Optimization on Graphs — Haoran Sun, Etash Kumar Guha, Hanjun Dai
- A Neural Tangent Kernel Perspective on Function-Space Regularization in Neural Networks — Zonghao Chen, Xupeng Shi, Tim G. J. Rudner, Qixuan Feng, Weizhong Zhang, Tong Zhang
- Conditional gradient-based method for bilevel optimization with convex lower-level problem — Ruichen Jiang, Nazanin Abolfazli, Aryan Mokhtari, Erfan Yazdandoost Hamedani
- The Solution Path of the Group Lasso — Aaron Mishkin, Mert Pilanci
- Linear Convergence Analysis of Neural Collapse with Unconstrained Features — Peng Wang, Huikang Liu, Can Yaras, Laura Balzano, Qing Qu
- Exact Gradient Computation for Spiking Neural Networks — Jane Lee, Saeid Haghighatshoar, Amin Karbasi
- A Variable-Coefficient Nuclear Norm Penalty for Low Rank Inference — Nathan Wycoff, Ali Arab, Lisa Singh
- Gradient dynamics of single-neuron autoencoders on orthogonal data — Nikhil Ghosh, Spencer Frei, Wooseok Ha, Bin Yu
- On Generalization of Decentralized Learning with Separable Data — Hossein Taheri, Christos Thrampoulidis
- Semi-Random Sparse Recovery in Nearly-Linear Time — Jonathan Kelner, Jerry Li, Allen Liu, Aaron Sidford, Kevin Tian
- Uniform Convergence and Generalization for Nonconvex Stochastic Minimax Problems — Siqi Zhang, Yifan Hu, Liang Zhang, Niao He
- Escaping from Moderately Constrained Saddles — Dmitrii Avdiukhin, Grigory Yaroslavtsev
- Decentralized Stochastic Optimization with Client Sampling — Ziwei Liu, Anastasia Koloskova, Martin Jaggi, Tao Lin
- A Novel Stochastic Gradient Descent Algorithm for LearningPrincipal Subspaces — Charline Le Lan, Joshua Greaves, Jesse Farebrother, Mark Rowland, Fabian Pedregosa, Rishabh Agarwal, Marc G Bellemare
- Network Pruning at Scale: A Discrete Optimization Approach — Wenyu Chen, Riade Benbaki, Xiang Meng, Rahul Mazumder
- The Importance of Temperature in Multi-Task Optimization — David Mueller, Mark Dredze, Nicholas Andrews
- Relating Regularization and Generalization through the Intrinsic Dimension of Activations — Bradley CA Brown, Jordan Juravsky, Anthony L. Caterini, Gabriel Loaiza-Ganem
- Policy gradient finds global optimum of nearly linear-quadratic control systems — Yinbin Han, Meisam Razaviyayn, Renyuan Xu
- A Better Way to Decay: Proximal Gradient Training Algorithms for Neural Nets — Liu Yang, Jifan Zhang, Joseph Shenouda, Dimitris Papailiopoulos, Kangwook Lee, Robert D Nowak
- NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learning — Buyun Liang, Tim Mitchell, Ju Sun
- Quadratic minimization: from conjugate gradients to an adaptive heavy-ball method with Polyak step-sizes — Baptiste Goujaud, Adrien Taylor, Aymeric Dieuleveut
- Fast Convergence of Greedy 2-Coordinate Updates for Optimizing with an Equality Constraint — Amrutha Varshini Ramesh, Aaron Mishkin, Mark Schmidt
- Dimension-Reduced Adaptive Gradient Method — Jingyang Li, Pan Zhou, Kuangyu Ding, Kim-Chuan Toh, Yinyu Ye
- ProxSkip for Stochastic Variational Inequalities: A Federated Learning Algorithm for Provable Communication Acceleration — Siqi Zhang, Nicolas Loizou
- Stochastic Adaptive Regularization Method with Cubics: A High Probability Complexity Bound — Katya Scheinberg, Miaolan Xie
- Momentum Extragradient is Optimal for Games with Cross-Shaped Spectrum — Junhyung Lyle Kim, Gauthier Gidel, Anastasios Kyrillidis, Fabian Pedregosa
- Why (and When) does Local SGD Generalize Better than SGD? — Xinran Gu, Kaifeng Lyu, Longbo Huang, Sanjeev Arora
- Target-based Surrogates for Stochastic Optimization — Jonathan Wilder Lavington, Sharan Vaswani, Reza Babanezhad Harikandeh, Mark Schmidt, Nicolas Le Roux
- Optimal Complexity in Non-Convex Decentralized Learning over Time-Varying Networks — Xinmeng Huang, Kun Yuan
- On Convexity and Linear Mode Connectivity in Neural Networks — David Yunis, Kumar Kshitij Patel, Pedro Henrique Pamplona Savarese, Gal Vardi, Jonathan Frankle, Matthew Walter, Karen Livescu, Michael Maire
- Distributed Online and Bandit Convex Optimization — Kumar Kshitij Patel, Aadirupa Saha, Lingxiao Wang, Nathan Srebro
- Nesterov Meets Optimism: Rate-Optimal Optimistic-Gradient-Based Method for Stochastic Bilinearly-Coupled Minimax Optimization — Chris Junchi Li, Angela Yuan, Gauthier Gidel, Michael Jordan
- Neural Networks Efficiently Learn Low-Dimensional Representations with SGD — Alireza Mousavi-Hosseini, Sejun Park, Manuela Girotti, Ioannis Mitliagkas, Murat A Erdogdu
- Optimization for Robustness Evaluation beyond â„“p Metrics — Hengyue Liang, Buyun Liang, Ying Cui, Tim Mitchell, Ju Sun
- Bidirectional Adaptive Communication for Heterogeneous Distributed Learning — Dmitrii Avdiukhin, Vladimir Braverman, Nikita Ivkin, Sebastian U Stich
- Toward Understanding Why Adam Converges Faster Than SGD for Transformers — Yan Pan, Yuanzhi Li
- TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization — Xiang Li, Junchi Yang, Niao He
- Rethinking Sharpness-Aware Minimization as Variational Inference — Szilvia Ujváry, Zsigmond Telek, Anna Kerekes, Anna Mészáros, Ferenc Huszár
- On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data — Tina Behnia, Ganesh Ramachandra Kini, Vala Vakilian, Christos Thrampoulidis
- TorchOpt: An Efficient Library for Differentiable Optimization — Jie Ren, Xidong Feng, Bo Liu, Xuehai Pan, Yao Fu, Luo Mai, Yaodong Yang
- Rieoptax: Riemannian Optimization in JAX — Saiteja Utpala, Andi Han, Pratik Jawanpuria, Bamdev Mishra
- Data-heterogeneity-aware Mixing for Decentralized Learning — Yatin Dandi, Anastasia Koloskova, Martin Jaggi, Sebastian U Stich
- A Finite-Particle Convergence Rate for Stein Variational Gradient Descent — Jiaxin Shi, Lester Mackey