Ben (Quan) Shi

Recent Princeton University '24 CS grad!

prof_pic.jpg

Hi, my name is Ben, and I’m a recent graduate of Princeton University where I received my undergraduate degree in computer science. I’m originally from Changsha, China, and moved to West Virginia in the 5th grade. I am fortunate to be advised by Professor Karthik Narasimhan, as well as Dr. Shunyu Yao. Previous I was an intern at Meta.

Nowadays, I’m particularly interested in Human-AI collaboration in reasoning environments, briding the gap between human intent and model capability. This will require better alignment of human and model representations of the world to enable data efficient generalization and seamless communication. Super excited to explore this in future work. Evals are also a big part of this: there needs to be more utility-based evals that return transferrable signals to downstream tasks. See my past work.

My undergraduate thesis was on analyzing and improving LLM performance on competitive programming, where I received the Sigma Xi Book award for outstanding undergraduate research and publication record.

In my free time I love to do coding contests and to make music. So much so that I built a generative AI pop music composition tool. I also love to produce music + sing. I frequently perform at local coffee shops!

Feel free to contact me at [x]@princeton.edu, where [x] = benshi.

Sneak peek as to what’s next. Email for a demo.

news

Dec 15, 2024 Just finished a new preprint investigating the effectiveness of reasoning-based retrieval! See a preview here.
Jul 09, 2024 Our work on competitive programming was accepted into COLM, landing in the top 1% of review scores!
May 01, 2024 Our new preprint on LLM performance on Olympiad Programming is out here!
Nov 03, 2002 Started Existing

selected publications

  1. Teaser4.png
    When Does Retrieval Benefit Reasoning-Intense Tasks?
    Quan Shi ,  Howard Yen ,  Mengzhou Xia ,  Shunyu Yao ,  Danqi Chen ,  and  Karthik Narasimhan
    2024
  2. llmcomp_image.png
    Can Language Models Solve Olympiad Programming?
    Quan Shi ,  Michael Tang ,  Karthik Narasimhan ,  and  Shunyu Yao
    2024
  3. bright_image.png
    BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
    Hongjin Su ,  Howard Yen ,  Mengzhou Xia ,  Weijia Shi ,  Niklas Muennighoff ,  Han-yu Wang ,  Haisu Liu ,  Quan Shi ,  Zachary S. Siegel ,  Michael Tang ,  Ruoxi Sun ,  Jinsung Yoon ,  Sercan O. Arik ,  Danqi Chen ,  and  Tao Yu
    2024