I research large language models (LLMs) as a PhD student at Stanford. I’m fortunate to be collaborating with Contextual AI & Ai2. I did my Bachelor’s at Peking University.

If you want to work/research with me, I recommend contributing to MTEB. We are a community building the go-to place for everything related to embeddings with 200K active monthly users on our leaderboard and regular publications you can co-author.

If you have questions about any papers I’ve co-authored, please open a GitHub issue on the respective repository :)

Research

I research making LLMs more useful, e.g., by improving their capabilities. Below are some areas I’ve been working on with amazing collaborators & some papers I’ve had the chance to be involved in (see Google Scholar for all papers):

• LLM pretraining: I want to better understand how models scale & help us build more capable base models
  • Scaling Data-Constrained Language Models (NeurIPS 2023, Oral & Outstanding Paper Runner-Up Award)
  • OLMoE: Open Mixture-of-Experts Language Models & OLMo (ACL 2024, Best Theme Paper Award)
• LLM alignment/instruction tuning: Models need to precisely follow instructions to be truly useful
  • Crosslingual generalization through multitask finetuning (ACL 2022, Most influential ACL paper #2)
  • Octopack: Instruction tuning code large language models (ICLR 2024, Spotlight)
• Embeddings/Retrieval: For LLMs to be useful, they often need to rely on search/retrieval, usually powered by embeddings
  • MTEB: Massive Text Embedding Benchmark (EACL 2022, >2M software downloads)
  • Generative Representational Instruction Tuning (ICLR 2024 AGI Workshop, Best Paper Award)

Selected honors

• 2024: Stanford School of Engineering Fellowship; ACL Best Theme Paper Award, Best Paper Award, Best Resource Paper Award; ICLR AGI Workshop Best Paper Award
• 2023: NeurIPS Outstanding Paper Runner-Up Award
• 2020: Meta AI Hateful Memes Challenge, 2nd place/3300+
• 2019: JLPTN1
• 2018-2022: Peking University Full Scholarship

Misc unsorted

• Apart from AI, I’m very interested in health (🏊🎾🏃🌸)
• I’ve worked in Chinese, English, Japanese, French & German, but I don’t have enough parameters, so I forgot much of the last two 😅
• As a kid I worked as a voice-over artist dubbing German voices for Peter Pan (Disney), Pokemon, Game Of Thrones (HBO), Dracula (NBC) & others (sample) 😁