$$ \Huge \textbf {Qizhen Zhang (Irene)} \\ $$
I am a PhD student at the University of Oxford. I’m interested in large language model capability research across all stages of training.
Previously, I interned as a research scientist in Meta’s core Llama team and FAIR. I was also a member of technical staff / researcher at Cohere doing research (and building LLM pretraining frameworks for some of the first >100B models ever in early 2022!).
I wrote my Master's thesis on cooperative multi-agent reinforcement learning at the University of Toronto and the Vector Institute.
<aside> <img src="/icons/graduate_blue.svg" alt="/icons/graduate_blue.svg" width="40px" /> Google Scholar
</aside>
<aside> <img src="/icons/mail_blue.svg" alt="/icons/mail_blue.svg" width="40px" /> Email
</aside>
<aside> <img src="/icons/duck_blue.svg" alt="/icons/duck_blue.svg" width="40px" /> Twitter
</aside>
<aside> <img src="/icons/sharing_blue.svg" alt="/icons/sharing_blue.svg" width="40px" /> Linkedin
</aside>
(***** indicates equal contribution)
Pretraining data & training dynamics
An Empirical Study on Noisy Data and LLM Pretraining Loss Divergence paper Qizhen Zhang, Ankush Garg, Jakob Foerster, Niladri Chatterji*, Kshitiz Malik*****, Mike Lewis*** Preprint 2026**
Pretraining model surgery
BTS: Harmonizing Specialized Experts into a Generalist LLM paper, tweet, talk Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan*****, Suchin Gururangan*****, Mike Lewis*** EMNLP 2025 Main**
BAM: Simple and Efficient Parameter Upcycling for Mixture of Experts **paper, tweet, talk Qizhen Zhang**, Nikolas Gritsch, Dwaraknath Gnaneshwar, Simon Guo, David Cairuz, Bharat Venkitesh, Jakob Foerster, Phil Blunsom, Sebastian Ruder, Ahmet Üstün*****, Acyr Locatelli*** NeurIPS 2024** Also at workshops ES-FoMo, NGSM @ ICML 2024 (Spotlight Talk)
Multi-agent RL
Analysing the Sample Complexity of Opponent Shaping [paper](https://arxiv.org/abs/2402.05782#:~:text=Learning in general-sum games,group performances in many settings.)** Kitty Fung, Qizhen Zhang, Chris Lu, Jia Wan, Timon Willi, Jakob Foerster AAMAS 2024 (Oral)
Centralized Model and Exploration Policy for Multi-Agent RL paper, tweet, talk Qizhen Zhang, Chris Lu, Animesh Garg, Jakob Foerster AAMAS 2022 (Oral)
Research Scientist Intern | Meta FAIR 2025 (Menlo Park, USA) Llama pretraining team / FAIR / MSL
Research Scientist Intern | Meta GenAI 2024 - 2025 (New York City, USA) Llama post-training team
Research Scientist Intern (Part-time)| **Cohere** 2023 - 2024 (London, United Kingdom) Pretraining team