$$ \Huge \textbf {Qizhen Zhang (Irene)} \\ $$

I am a PhD student at the University of Oxford, I work on large language models capability research.

Previously, I was interning as a research scientist with the Llama team at Meta GenAI. I was also a member of technical staff / researcher at Cohere doing research and building LLM pretraining frameworks for O(100B) models.

I wrote my Master's thesis on cooperative multi-agent reinforcement learning at the University of Toronto and the Vector Institute.

<aside> <img src="/icons/graduate_blue.svg" alt="/icons/graduate_blue.svg" width="40px" /> Google Scholar

</aside>

<aside> <img src="/icons/mail_blue.svg" alt="/icons/mail_blue.svg" width="40px" /> Email

</aside>

<aside> <img src="/icons/duck_blue.svg" alt="/icons/duck_blue.svg" width="40px" /> Twitter

</aside>

<aside> <img src="/icons/sharing_blue.svg" alt="/icons/sharing_blue.svg" width="40px" /> Linkedin

</aside>

Research

***** indicates equal contribution

BTS: Harmonizing Specialized Experts into a Generalist LLM paper, tweet Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan*****, Suchin Gururangan*****, Mike Lewis*** Under Submission**

BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts paper, tweet, talk Qizhen Zhang, Nikolas Gritsch, Dwaraknath Gnaneshwar, Simon Guo, David Cairuz, Bharat Venkitesh, Jakob Foerster, Phil Blunsom, Sebastian Ruder, Ahmet Üstün*****, Acyr Locatelli*** NeurIPS 2024** Also at workshops ES-FoMo, NGSM @ ICML 2024 (Spotlight Talk)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts paper, tweet Nikolas Gritsch, Qizhen Zhang, Acyr Locatelli, Sara Hooker, Ahmet Üstün Under submission Also at the AFM workshop @ NeurIPS 2024

PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition **paper, tweet** Ziyang Zhang, Qizhen Zhang, Jakob Foerster ICML 2024

Analysing the Sample Complexity of Opponent Shaping [paper](https://arxiv.org/abs/2402.05782#:~:text=Learning in general-sum games,group performances in many settings.)** Kitty Fung, Qizhen Zhang, Chris Lu, Jia Wan, Timon Willi, Jakob Foerster AAMAS 2024 (Oral)

Centralized Model and Exploration Policy for Multi-Agent RL paper, tweet, talk Qizhen Zhang, Chris Lu, Animesh Garg, Jakob Foerster AAMAS 2022 (Oral)