$$ \Huge \textbf {Qizhen Zhang (Irene)} \\ $$
I am a PhD student at the University of Oxford. I’m interested in large language model capability research across all stages of training. Currently, I’m interning as a research scientist in Meta’s Llama pretraining team.
Previously, I was a member of technical staff / researcher at Cohere doing research and building LLM pretraining frameworks for O(100B) models.
I wrote my Master's thesis on cooperative multi-agent reinforcement learning at the University of Toronto and the Vector Institute.
<aside> <img src="/icons/graduate_blue.svg" alt="/icons/graduate_blue.svg" width="40px" /> Google Scholar
</aside>
<aside> <img src="/icons/mail_blue.svg" alt="/icons/mail_blue.svg" width="40px" /> Email
</aside>
<aside> <img src="/icons/duck_blue.svg" alt="/icons/duck_blue.svg" width="40px" /> Twitter
</aside>
<aside> <img src="/icons/sharing_blue.svg" alt="/icons/sharing_blue.svg" width="40px" /> Linkedin
</aside>
(***** indicates equal contribution)
LLM (Continual) Pre-training
BTS: Harmonizing Specialized Experts into a Generalist LLM paper, tweet Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan*****, Suchin Gururangan*****, Mike Lewis*** EMNLP 2025 Main**
BAM: Simple and Efficient Parameter Upcycling for Mixture of Experts **paper, tweet, talk Qizhen Zhang**, Nikolas Gritsch, Dwaraknath Gnaneshwar, Simon Guo, David Cairuz, Bharat Venkitesh, Jakob Foerster, Phil Blunsom, Sebastian Ruder, Ahmet Üstün*****, Acyr Locatelli*** NeurIPS 2024** Also at workshops ES-FoMo, NGSM @ ICML 2024 (Spotlight Talk)
Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts paper, tweet Nikolas Gritsch, Qizhen Zhang, Acyr Locatelli, Sara Hooker, Ahmet Üstün EMNLP 2025 Findings Also at the AFM workshop @ NeurIPS 2024
LLM Safety
PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition **paper, tweet** Ziyang Zhang, Qizhen Zhang, Jakob Foerster ICML 2024
Multi-Agent RL
Analysing the Sample Complexity of Opponent Shaping [paper](https://arxiv.org/abs/2402.05782#:~:text=Learning in general-sum games,group performances in many settings.)** Kitty Fung, Qizhen Zhang, Chris Lu, Jia Wan, Timon Willi, Jakob Foerster AAMAS 2024 (Oral)
Centralized Model and Exploration Policy for Multi-Agent RL paper, tweet, talk Qizhen Zhang, Chris Lu, Animesh Garg, Jakob Foerster AAMAS 2022 (Oral)
Research Scientist Intern | Meta GenAI 2025 (Menlo Park, USA) Llama pretraining team
Research Scientist Intern | Meta GenAI 2024 - 2025 (New York City, USA) Llama post-training team
Research Scientist Intern (Part-time)| **Cohere** 2023 - 2024 (London, United Kingdom) Pretraining team
Member of Technical Staff | **Cohere** 2022 (Toronto, Canada) Built LLM pretraining frameworks using JAX & TPUs. Co-owned pretraining runs of O(100 Billion) parameters.