Goutham Rajendran
Email: gouthamrdn [at] gmail [dot] com
Google Scholar
Hi, I'm Goutham.
About me
I am currently a Senior Research Scientist on the Gemini team at Google DeepMind.
Previously, I was at Meta Superintelligence Labs, where I worked on large-scale distributed training of LLMs. My work focused on how to enable native multimodality during pre-training (especially on scaling recipes involving architecture, data, evals and infra), as well as on the science of scaling test-time compute to improve math/code/visual reasoning.
Before that, I spent a fantastic time as a Research Associate in the Machine Learning Department at Carnegie Mellon University, working with Pradeep Ravikumar. I worked on representation learning and generative models, researching how to learn representations of data that are interpretable and controllable. I graduated with a PhD in Computer Science from the University of Chicago, where I was extremely fortunate to have been advised by Madhur Tulsiani (Toyota Technological Institute at Chicago) and Aaron Potechin (University of Chicago).
Publications
Machine Learning:
2024
-
Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
Goutham Rajendran*, Simon Buchholz*, Bryon Aragam, Bernhard Schölkopf, Pradeep Ravikumar
NeurIPS 2024
[Conference version] -
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam
NeurIPS 2024
Also at workshops:
i. Mechanistic Interpretability
ii. Theoretical Foundations of Foundation Models
at ICML 2024
-
On the Origins of Linear Representations in Large Language Models
Goutham Rajendran*, Yibo Jiang*, Pradeep Ravikumar, Bryon Aragam, Victor Veitch
ICML 2024 -
An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysis
Goutham Rajendran*, Patrik Reizinger*, Wieland Brendel, Pradeep Ravikumar
CLeaR 2024 (Oral)
[Slides]
2023
-
Learning Linear Causal Representations from Interventions under General Nonlinear Mixing
Goutham Rajendran*, Simon Buchholz*, Elan Rosenfeld, Bryon Aragam, Bernhard Schölkopf, Pradeep Ravikumar
NeurIPS 2023 (Oral, top 0.6%)
Also at workshops:
i. Structured Probabilistic Inference and Generative Modeling
ii. Spurious Correlations, Invariance, and Stability
at ICML 2023
[Slides, Poster]
2022
-
Identifiability of deep generative models without auxiliary information
Goutham Rajendran*, Bohdan Kivva*, Pradeep Ravikumar, Bryon Aragam
NeurIPS 2022 (Oral/Spotlight, top 1.8%)
[Poster] -
Sub-exponential time Sum-of-Squares lower bounds for Principal Components Analysis
Goutham Rajendran*, Aaron Potechin*
NeurIPS 2022
[Slides] -
Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition
Goutham Rajendran*, Wei Zou*
Manuscript 2022
2021
-
Structure learning in polynomial time: Greedy algorithms, Bregman information and exponential families
Goutham Rajendran, Bohdan Kivva, Ming Gao, Bryon Aragam
NeurIPS 2021
[Slides, Poster] -
Learning latent causal graphs via mixture oracles
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam
NeurIPS 2021
[Slides]
Theoretical Computer Science:
(** indicates alphabetical author list, as is customary)2024
-
Efficient Certificates of Anti-Concentration Beyond Gaussians
Ainesh Bakshi, Pravesh Kothari, Goutham Rajendran**, Madhur Tulsiani, Aravindan Vijayaraghavan
FOCS 2024
2023
-
Sum-of-Squares Lower Bounds for Densest k-Subgraph
Chris Jones, Aaron Potechin, Goutham Rajendran**, Jeff Xu
STOC 2023
-
Concentration of polynomial random matrices via Efron-Stein inequalities
Goutham Rajendran**, Madhur Tulsiani
SODA 2023
2022
-
Nonlinear Random Matrices and Applications to the Sum of Squares Hierarchy
Goutham Rajendran
PhD Dissertation, 2022, University of Chicago
[Slides]
2021
-
Sum-of-Squares Lower Bounds for Sparse Independent Set
Chris Jones, Aaron Potechin, Goutham Rajendran**, Madhur Tulsiani, Jeff Xu
FOCS 2021
2020
-
Sum-of-Squares Lower Bounds for Sherrington-Kirkpatrick via Planted Affine Planes
Mrinalkanti Ghosh, Fernando Granha Jeronimo, Chris Jones, Aaron Potechin, Goutham Rajendran**
FOCS 2020
-
Machinery for Proving Sum-of-Squares Lower Bounds on Certification Problems
Aaron Potechin, Goutham Rajendran**
Manuscript 2020
2018
-
Combinatorial Optimization via the Sum of Squares Hierarchy
Goutham Rajendran
Master's thesis, 2018, University of Chicago [Slides]
Fun stuff
I used to be a competitive programmer, including competing in ICPC. My handle in online judges is xorfire: Codeforces, Topcoder, Codechef.
In another life, I would have been a professional footballer (read: soccer) but in this one, my career has been plagued with injuries :)
Website theme based on minimal by orderedlist