Adaptive LLM evaluation
Drift detection
Internal Regeneron project. The illustrations use synthetic data, and many technical details are withheld.
MLOps & LLM evaluation
Building a scalable monitoring platform for AI/ML models
Regeneron Pharmaceuticals · AI/ML Engineering Intern · Summer 2026
I built a platform that monitors 30+ production AI models and evaluates LLMs. It detects when live data drifts from training data distributions or when model performance unexpectedly degrades, and helps teams drill down to find the cause. Its LLM benchmarks help AI engineers make informed decisions about the best models for their use case.
- Built the platform on AWS with a React dashboard, designed from 13 stakeholder interviews and adopted by 3 pilot teams.
- Built drift detection that alerts teams when live data changes, using a Random Forest two-sample test alongside metrics like PSI and Jensen-Shannon divergence.
- Tracked how data subgroups shift over time using Gaussian mixture models, BIC selection, and cost matching.
- Developed a Python SDK that lets teams send data and run model analysis from any environment.