Hi! I’m Nitin, a CS PhD Student at The University of Texas at Austin, where I’m advised by Prof. Aditya Akella. My research interests lie at the intersection of computer systems and machine learning.

Before UT, I was a Pre-Doctoral Research Fellow in the AI Infrastructure group at Microsoft Research, India, where I worked with Dr. Ramachandrandran Ramjee, Dr. Jayashree Mohan, Dr. Ashish Panwar and Dr. Nipun Kwatra. My research focus was on optimizing Large Language Model (LLM) inference there.

Previously, I was a Senior Software Engineer at Zeta, a unicorn startup modernizing legacy banking systems with a cloud-native stack. During my three years there, I scaled their multi-tenant web application platform and developed Zeta’s API Playground.

I graduated with a B.Tech. in Computer Science and Engineering from IIT Guwahati in 2020. I have fond memories of competing in programming contests at Codeforces. See this repo for fast plug-n-play data structures and algorithm implementations in C++.

Publications

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, and Ramachandran Ramjee
OSDI’24
PDF / Code

Vidur: A Large Scale Simulation Framework For LLM Inference
Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, and Alexey Tumanov
MLSys’24
PDF / Code

On Evaluating Performance of LLM Inference Serving Systems
Amey Agrawal, Nitin Kedia, Anmol Agarwal, Jayashree Mohan, Nipun Kwatra, Souvik Kundu, Ramachandran Ramjee, and Alexey Tumanov
Preprint
PDF / Code