My role at DeepSeek-AI is dedicated to help building inference infrastructure for the next generation of DeepSeek models. A key part of my work is optimizing our large-scale internal software systems to ensure peak performance on various hardware platforms.

I work with the most excellent peers and engineers to build incredible inference system for SOTA large language models like DeepSeek-R1. Our system deployed for WeChat serves a billion of users.

We also work closely with open-source communities, such as SGLang and vLLM, to contribute our industrial insight and optimizations for extreme large deployment.

I worked on the Application Defined Network project under the supervision of Xiangfeng Zhu and Ratul Mahajan. I independently developed a compiler backend that compiles our AppNet program into Envoy Native Filter.

See also: UW Syslab.

I worked closely with Jianyu under the guidance of Ting Cao. I participated in and contributed some preliminary results to ISCA'2024 Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference.