Blog

A mix of things: deep dives into deep learning, technical write-ups on applications and things I build, and sometimes just personal posts. Expect toy problems, gentle math, random unfunny jokes, and things you can click on.

• October 23, 2026

  Play Around with Distillation, Part 1: Forward vs. Reverse KL, On vs. Off Policy, and Hard vs. Soft!

  What do forward and reverse KL, on- and off-policy, and hard and soft targets actually do? A casual, hands-on tour with toy problems, gentle proofs, and a demo where you can watch the student learn.

  Knowledge DistillationKL DivergenceOn-PolicyToy Problems
  Read PDF