Goutham Rajendran
Email: gouthamrdn [at] gmail [dot] com
Google Scholar
Hi, I'm Goutham.
About me
I am currently on the LLaMA team at Meta GenAI, where I work on large-scale distributed training of LLMs, specifically focusing on multimodality and reasoning capabilities.
Previously, I spent a fantastic time as a Research Associate in the Machine Learning Department at Carnegie Mellon University, working with Pradeep Ravikumar. I worked on representation learning and generative models, researching how to learn representations of data that are interpretable and controllable. This lets us improve pre-trained generative models, such as LLMs and diffusion models, to not only be accurate and robust, but also be highly customizable and aligned.
Before that, I graduated with a PhD in Computer Science from the University of Chicago, where I was extremely fortunate to have been advised by Madhur Tulsiani (Toyota Technological Institute at Chicago) and Aaron Potechin (University of Chicago). My thesis investigated the applicability of modern convex optimization techniques for various problems in machine learning and robust statistics.
Publications
Machine Learning:
2024
-
Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
Goutham Rajendran*, Simon Buchholz*, Bryon Aragam, Bernhard Schölkopf, Pradeep Ravikumar
NeurIPS 2024 -
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam
NeurIPS 2024
Also at workshops:
i. Mechanistic Interpretability
ii. Theoretical Foundations of Foundation Models
at ICML 2024
-
On the Origins of Linear Representations in Large Language Models
Goutham Rajendran*, Yibo Jiang*, Pradeep Ravikumar, Bryon Aragam, Victor Veitch
ICML 2024 -
An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysis
Goutham Rajendran*, Patrik Reizinger*, Wieland Brendel, Pradeep Ravikumar
CLeaR 2024 (Oral)
[Slides]
2023
-
Learning Linear Causal Representations from Interventions under General Nonlinear Mixing
Goutham Rajendran*, Simon Buchholz*, Elan Rosenfeld, Bryon Aragam, Bernhard Schölkopf, Pradeep Ravikumar
NeurIPS 2023 (Oral, top 0.6%)
Also at workshops:
i. Structured Probabilistic Inference and Generative Modeling
ii. Spurious Correlations, Invariance, and Stability
at ICML 2023
[Slides, Poster]
2022
-
Identifiability of deep generative models without auxiliary information
Goutham Rajendran*, Bohdan Kivva*, Pradeep Ravikumar, Bryon Aragam
NeurIPS 2022 (Oral/Spotlight, top 1.8%)
[Poster] -
Sub-exponential time Sum-of-Squares lower bounds for Principal Components Analysis
Goutham Rajendran*, Aaron Potechin*
NeurIPS 2022
[Slides] -
Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition
Goutham Rajendran*, Wei Zou*
Manuscript 2022
2021
-
Structure learning in polynomial time: Greedy algorithms, Bregman information and exponential families
Goutham Rajendran, Bohdan Kivva, Ming Gao, Bryon Aragam
NeurIPS 2021
[Slides, Poster] -
Learning latent causal graphs via mixture oracles
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam
NeurIPS 2021
[Slides]
Theoretical Computer Science:
(** indicates alphabetical author list, as is customary)2024
-
Efficient Certificates of Anti-Concentration Beyond Gaussians
Ainesh Bakshi, Pravesh Kothari, Goutham Rajendran**, Madhur Tulsiani, Aravindan Vijayaraghavan
FOCS 2024
2023
-
Sum-of-Squares Lower Bounds for Densest k-Subgraph
Chris Jones, Aaron Potechin, Goutham Rajendran**, Jeff Xu
STOC 2023
-
Concentration of polynomial random matrices via Efron-Stein inequalities
Goutham Rajendran**, Madhur Tulsiani
SODA 2023
2022
-
Nonlinear Random Matrices and Applications to the Sum of Squares Hierarchy
Goutham Rajendran
PhD Dissertation, 2022, University of Chicago
[Slides]
2021
-
Sum-of-Squares Lower Bounds for Sparse Independent Set
Chris Jones, Aaron Potechin, Goutham Rajendran**, Madhur Tulsiani, Jeff Xu
FOCS 2021
2020
-
Sum-of-Squares Lower Bounds for Sherrington-Kirkpatrick via Planted Affine Planes
Mrinalkanti Ghosh, Fernando Granha Jeronimo, Chris Jones, Aaron Potechin, Goutham Rajendran**
FOCS 2020
-
Machinery for Proving Sum-of-Squares Lower Bounds on Certification Problems
Aaron Potechin, Goutham Rajendran**
Manuscript 2020
2018
-
Combinatorial Optimization via the Sum of Squares Hierarchy
Goutham Rajendran
Master's thesis, 2018, University of Chicago [Slides]
Fun stuff
I used to be a competitive programmer, including competing in ICPC. My handle in online judges is xorfire: Codeforces, Topcoder, Codechef.
In another life, I would have been a professional footballer (read: soccer) but in this one, my career has been plagued with injuries :)
Website theme based on minimal by orderedlist