Neural Information Processing Systems (NeurIPS) 2024

Apple is presenting new research at the annual conference on Neural Information Processing Systems (NeurIPS), which takes place in person in Vancouver, Canada, from December 10 – 15. We are proud to again sponsor the multi-track interdisciplinary conference, which brings together the scientific and industrial research communities surrounding Machine Learning. Below is an overview of Apple’s participation at NeurIPS 2024.

Demos

MLX

MLX is a flexible array framework that is optimized for Apple silicon and brought to you by Apple machine learning research. It enables training and inference of arbitrarily complex models on Apple silicon powered devices with great brevity and flexibility. The demo presents an example of fine-tuning of a 7B parameter LLM on an iPhone, image generation using a large diffusion model on an iPad and text generation using a number of large language models on an M2 Ultra. This demo will be hosted during exhibition booth hours Tuesday through Thursday. Learn more about MLX here.

MobileCLIP: Real-Time Image-Text Models

MobileCLIP is a family of mobile-friendly image-text models with hybrid CNN/Transformer architectures. In combination, these models attain the best accuracy-latency tradeoff. MobileCLIP-B obtains state-of-the-art results. This demo will be hosted during exhibition booth hours Tuesday through Thursday. Learn more about MobileCLIP here.

All conference attendees are invited to visit our booth to experience these demos in person.

Accepted Papers

Links to papers with ◊ will be added after the conference, as they become available

4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities

Roman Bachmann, Oguzhan Kar, David Mizrahi, Ali Garjani, Mingfei Gao, David Griffiths, Jimmy Hu, Afshin Dehghan, Amir Zamir

Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP

Chen Huang, Skyler Seto, Samira Abnar, David Grangier, Navdeep Jaitly, Josh Susskind

DataComp-LM: In search of the Next Generation of Training Sets for Language Models

Jeffrey Li, Alex Fang, Georgios Smyrnis, Matt Jordan, Maor Igvi, Hadi Pour Ansari, Fartash Faghri, Alaaeldin Mohamed Elnouby Ali, Alexander Toshev, Alex Dimakis, et al.

Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum

Hadi Pour Ansari, Chun-Liang Li, Rick Chang, Pavan Kumar Anasosalu Vasu, Cem Koc, Vaishaal Shankar, Oncel Tuzel

GENOT: Entropic (Gromov) Wasserstein Flow Matching with Applications to Single-Cell Genomics

Dominik Klein, Theo Uscidda, Fabian Theis, Marco Cuturi Cameto

Faster Algorithms for User-Level Private Stochastic Convex Optimization
Hilal Asi, Daogao Liu, Andrew Lowy

Grounding of Multimodal Large Language Models in Action Spaces ◊

Andrew Szot, Bogdan Mazoure, Harsh Agrawal, Devon Hjelm, Zsolt Kira, Alexander Toshev

How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad

Emmanuel Abbe, Samy Bengio, Aryo Lotfi, Colin Sandon, Omid Saremi

How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks

Etai Littwin, Omid Saremi, Madhu Advani, Chen Huang, Preetum Nakkiran, Josh Susskind, Vimal Thilak

Instance Optimal Private Density Estimation in the Wasserstein Distance

Vitaly Feldman, Audra McMillan, Satchit Sivakumar, Kunal Talwar

Kaleido Diffusion: Improving Conditional Diffusion Models with Auto-Regressive Latent Modeling

Jiatao Gu, Ying Shen, Shuangfei Zhai, Yizhe Zhang, Navdeep Jaitly, Josh Susskind

Learning Spatially Aligned Text and Audio Representations ◊

Bhavika Devnani, Skyler Seto, Zak Aldeneh, Alessandro Toso, Elena Menyaylenko, Barry Theobald, Jonathan Sheaffer, Miguel Sarabia del Castillo

Learning Elastic Costs to Shape Monge Displacements

Michal Klein, Aram Alexandre Pooladian, Pierre Ablin, Eugene Ndiaye, Jonathan Niles Weed, Marco Cuturi

ODGEN: Domain-Specific Object Detection Data Generation with Diffusion Models

JingYuan Zhu, Shiyu Li, Andy Liu, Ping Huang, Jiulong Shan, Huimin Ma, Jian Yuan

PFL-Research: Simulation Framework for Accelerating Research in Private Federated Learning

Filip Granqvist, Congzheng Song, Aine Cahill, Rogier van Dalen, Martin Pelikan, Yi Sheng Chan, Xiaojun Feng, Natarajan Krishnaswami, Vojta Jina, Mona Chitnis

Private and Personalized Frequency Estimation in a Federated Setting

Amrith Setlur, Vitaly Feldman, Kunal Talwar

Private Online Learning via Lazy Algorithms

Hilal Asi, Daogao Liu, Tomer Koren, Kunal Talwar

Private Stochastic Convex Optimization with Heavy Tails

Hilal Asi, Daogao Liu, Kevin Tian

Progressive Entropic Optimal Transport Solvers

Parnian Kassraie, Aram Alexandre Pooladian, Michal Klein, James Thornton, Jonathan Niles-Weed, Marco Cuturi

Strategic Linear Contextual Bandits

Aadi Saha, Thomas Kleine Buening, Christos Dimitrakakis, Haifeng Xu

Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization ◊

Omar Montasser, Han Shao, Emmanuel Abbe

When is Multicalibration Post-Processing Necessary?

Dutch Hansen, Siddartha Devic, Preetum Nakkiran, Vatsal Sharan

Accepted Workshop Papers

Links to workshop papers with ◊ will be added after the conference, as they become available

AdEMAMix: Leveraging the Surprising Relevance of Very Old Gradients ◊

Matteo Pagliardini, Pierre Ablin, David Grangier

Classifier-Free Guidance is a Predictor-Corrector

Arwen Bradley, Preetum Nakkiran

Computational Bottlenecks of Training Small-Scale Large Language Models

Saleh Ashkboos, Iman Mirzadeh, Keivan Alizadeh, Mohammad Hossein Sekhavat, Moin Nabi, Mehrdad Farajtabar, Fartash Faghri

Device-Directed Speech Detection for Follow-up Conversations Using Large Language Models

Oggi Rudovic, Pranay Dighe, Yi Su, Vineet Garg, Sameer Dharur, Xiaochuan Niu, Ahmed Hussen Abdelaziz, Saurabh Adya, Ahmed Tewfik

Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications

Scott Hoang, Minsik Cho, Thomas Merth, Atlas Wang, Mohammad Rastegari, Devang Naik

Do LLMs Estimate Uncertainty Well in Instruction-Following?

Juyeon Heo, Miao Xiong, Christina Heinze-Deml, Jaya Narain

Do LLMs Internally “Know” When They Follow Instructions?

Juyeon Heo, Christina Heinze-Deml, Shirley Ren, Oussama Elachqar, Udhay Nallasamy, Andy Miller, Jaya Narain

Dueling in the Dark: An Efficient and Optimal O(√T) Mirror Descent Approach for Competing against Adversarial Preferences ◊

Aadi Saha, Yonathan Efroni, Barry Theobald

Duo-LLMs: A Framework for Studying Adaptive Computation in Large Language Models

Keivan Alizadeh Vahid, Iman Mirzadeh, Mohammad Sekhavat, Minsik Cho, Dmitry Belenko, Frank Sun, Hooman Shahrokhi, Moin Nabi, Mehrdad Farajtabar

Efficient and Effective Uncertainty Quantification in LLMs ◊

Miao Xiong, Andrea Santilli, Michael Kirchhof, Adam Golinski, Sinead Williamson

Enhancing JEPAs with Spatial Conditioning: Robust and Efficient Representation Learning

Etai Littwin, Vimal Thilak, Anand Gopalakrishnan

Evaluating Gender Bias Transfer Between Pretrained and Prompt Adapted Language Models ◊

Niv Sivakumar, Natalie Mackraz, Samira Khorshidi, Krishna Patel, Barry Theobald, Luca Zappella, Nick Apostoloff

Fairness Dynamics During Training ◊

Krishna Patel, Niv Sivakumar, Barry Theobald, Luca Zappella, Nick Apostoloff

Learning Functions on Symmetric Matrices and Point Clouds via Lightweight Invariant Features ◊

Ben Blum-Smith, Teresa Huang, Marco Cuturi, Soledad Villar

Leveraging Periodicity for Robustness with Multi-Modal Mood Pattern Models

Jaya Narain, Jenny Sun, Oussama Elachqar, Haraldur Hallgrimsson, Feng Zhu, Shirley Ren

Memory Retaining Finetuning via Distillation

Zitong Yang, Aonan Zhang, Sam Wiseman, Xiang Kong, Ke Ye, Dong Yin

Momentum Approximation in Asynchronous Private Federated Learning ◊

Tao Yu, Congzheng Song, Jianyu Wang, Mona Chitnis

On a Spurious Interaction Between Uncertainty Scores and Answer Evaluation Metrics in Generative QA Tasks ◊

Andrea Santilli, Miao Xiong, Michael Kirchhof, Pau Rodriguez Lopez, Federico Danieli, Xavier Suau Cuadros, Luca Zappella, Sinead Williamson, Adam Golinski

Promoting Cross-Modal Representations to Improve Multimodal Foundation Models for Physiological Signals

Ching Fang, Chris Sandino, Behrooz Mahasseni, Juri Minxha, Hadi Pour Ansari, Erdrin Azemi, Ali Moin, Ellen Zippi

SALSA: Soup-Based Alignment Learning for Stronger Adaptation in RLHF ◊

Atoosa Malemir Chegini, Hamid Kazemi, Iman Mirzadeh, Dong Yin, Max Horton, Moin Nabi, Mehrdad Farajtabar, Keivan Alizadeh Vahid

Scaling Smart: Accelerating Large Language Model Pre-Training with Small Model Initialization

Mohammad Samragh Razlighi, Iman Mirzadeh, Keivan Alizadeh Vahid, Fartash Faghri, Minsik Cho, Moin Nabi, Devang Naik, Mehrdad Farajtabar

TiC-LM: A Multi-Year Benchmark for Continual Pretraining of Language Models ◊

Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pour Ansari, Fartash Faghri

Towards Time-Series Reasoning with LLMs

Winnie Chow, Lauren Gardiner, Haraldur Hallgrimsson, Maxwell A. Xu, Shirley Ren

Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparison

Judy Hanwen Shen, Archit Sharma, Jun Qin

Towards Low-Bit Communication for Tensor Parallel LLM Inference

Harry Dong, Tyler Johnson, Minsik Cho, Emad Soroush

Understanding Compute-Parameter Trade-offs in Sparse Mixture-of-Expert Language Models ◊

Harshay Shah, Samira Abnar, Vimal Thilak, Dan Busbridge, Alaaeldin Mohamed Elnouby Ali, Josh Susskind

Acknowledgements

Samy Bengio is a Board Member.

Kunal Talwar, Marco Cuturi, Pierre Ablin, Samy Bengio, and Sinead Williamson are Senior Area Chairs.

Aadirupa Saha, Byeongjoo Ahn, Natalie Schluter, Navdeep Jaitly, Oncel Tuzel, Pau Rodriguez Lopez, Preetum Nakkiran, Shams Azam, Tatiana Likhomanenko, and Yizhe Zhang are Area Chairs.

Audra McMillan is an Ethics Reviewer.

Arno Blaas, Dapeng Hu, Enrico Fini, Harsh Sharma, Josh Gardner, Louis Béthune, Maartje ter Hoeve, Miguel Sarabia, Mohammad Sekhavat, Niv Sivakumar, Pau Rodriguez Lopez, Ramprasaath Ramasamy Selvaraju, Richard Bai, TT Guo, Vimal Thilak and Yuyang Wang are Conference Reviewers.

Antoine Wehenkel, Arno Blaas, Pau Rodriguez Lopez, Rin Metcalf Susa, and Xavier Suau Cuadros are Workshop Organizers.

Samira Abnar and Vimal Thilak are Workshop Reviewers.