I research large language models (LLMs) as a 1st year PhD at Stanford. I’m fortunate to be collaborating with Contextual AI & Ai2. I did my Bachelor’s at Peking University.
Feel free to reach out, especially for doing research together. I’m also happy to advise self-motivated people on a research project - I have many ideas I think could be impactful & we can work on together :)
Research
I research making LLMs more useful, e.g. by improving their capabilities. Below are some areas I’ve been working on with amazing collaborators & some papers I’ve had the chance to be involved in (see Google Scholar for all papers):
- LLM pretraining: I want to better understand how models scale & help us build more capable base models
- Scaling Data-Constrained Language Models (NeurIPS 2023, Oral & Outstanding Paper Runner-Up Award)
- OLMoE: Open Mixture-of-Experts Language Models & OLMo (ACL 2024, Best Theme Paper Award)
- LLM alignment/instruction tuning: Models need to precisely follow instructions to be truly useful
- Crosslingual generalization through multitask finetuning (ACL 2022, Most influential ACL paper #2)
- Octopack: Instruction tuning code large language models (ICLR 2024, Spotlight)
- Embeddings/Retrieval: For LLMs to be useful, they often need to rely on search/retrieval, usually powered by embeddings
- MTEB: Massive Text Embedding Benchmark (EACL 2022, >1M software downloads)
- Generative Representational Instruction Tuning (ICLR 2024 AGI Workshop, Best Paper Award)
Selected honors
- 2024: Stanford School of Engineering Fellowship; ACL Best Theme Paper Award, Best Paper Award, Best Resource Paper Award; ICLR AGI Workshop Best Paper Award
- 2023: NeurIPS Outstanding Paper Runner-Up Award
- 2020: Meta AI Hateful Memes Challenge, 2nd place/3300+
- 2019: JLPTN1
- 2018-2022: Peking University Full Scholarship
Misc unsorted
- Apart from AI, I’m very interested in health (🏊🎾🏃🌸)
- I’ve worked in Chinese, English, Japanese, French & German, but I don’t have enough parameters so I forgot much of the last two 😅
- As a kid I worked as a voice-over artist dubbing German voices for Peter Pan (Disney), Pokemon (Pokemon Company), Game Of Thrones (HBO), Dracula (NBC) & others (sample) 😁