Ben (Quan) Shi

Recent Princeton University '24 CS grad!

prof_pic.jpg

Hi, my name is Ben, and I’m a recent graduate of Princeton University where I received my undergraduate degree in computer science. I’m originally from Changsha, China, and moved to West Virginia in the 5th grade. I am fortunate to be advised by Professor Karthik Narasimhan, as well as Dr. Shunyu Yao. Previous I was an intern at Meta.

Nowadays, I’m particularly interested in building agent tools and frameworks for Human-AI collaboration in reasoning environments, briding the gap between human intent and model capability. Optimistic about tools built on reasoning-based retrieval. Evals are also a big part of this: there needs to be more utility-based evals that return transferrable signals to downstream tasks.

My undergraduate thesis was on analyzing and improving LLM performance on competitive programming, where I received the Sigma Xi Book award for outstanding undergraduate research and publication record.

In my free time I love making and critiquing music. So much so that I built a generative AI pop music composition tool. I also love to produce music + sing.

Feel free to contact me at [x]@princeton.edu, where [x] = benshi.

news

Jul 09, 2024 Our work on competitive programming was accepted into COLM, landing in the top 1% of review scores!
May 01, 2024 Our new preprint on LLM performance on Olympiad Programming is out here!
Nov 03, 2002 Started Existing

selected publications

  1. llmcomp_image.png
    Can Language Models Solve Olympiad Programming?
    Quan Shi ,  Michael Tang ,  Karthik Narasimhan ,  and  Shunyu Yao
    2024
  2. bright_image.png
    BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
    Hongjin Su ,  Howard Yen ,  Mengzhou Xia ,  Weijia Shi ,  Niklas Muennighoff ,  Han-yu Wang ,  Haisu Liu ,  Quan Shi ,  Zachary S. Siegel ,  Michael Tang ,  Ruoxi Sun ,  Jinsung Yoon ,  Sercan O. Arik ,  Danqi Chen ,  and  Tao Yu
    2024