👋 Hi, I’m Nhi

In Part 2 of the Diffusion Models series, I mentioned that Improved DDPM can match DDPM’s sample quality using only 100 steps instead of 1000, thanks to learned variance. That’s a 10x speedup and already a significant improvement. But if you’ve sat through a full 1000-step DDPM sampling loop, even 100 steps still feels slow. Denoising Diffusion Implicit Models (DDIM; Song et al., 2021) takes a more radical approach to the speed problem. Instead of making the existing sampling loop more efficient, it asks: does the sampling loop even need to be Markovian in the first place? ...

In Part 1 of the Diffusion Models series, I covered the theory behind DDPM, the most basic diffusion model, which consists of: a forward process that gradually corrupts an image with Gaussian noise, a reverse process where a neural network U-Net learns to denoise step by step. If you got through all the math needed to understand DDPM, were not scared by it, and are in fact even more fascinated about diffusion models, the next (and hopefully easier to digest) paper is: ...

One of the places my curiosity took me to recently is Diffusion Models. Let’s start with Denoising Diffusion Probabilistic Models (DDPM; Ho et al. 2020). The GitHub repo for my PyTorch implementation of DDPM with instructions on how to train and generate images can be found here: halannhile/ddpm. Table of Contents Section 1: Theory Overview of DDPM The forward process The reverse process The training objective Training & sampling algorithms Section 2: Code ...

Multimodal AI has always been an area I am personally very fascinated by, because I believe our understanding of the world comes from more than just language - it’s also vision and audio (and touch and smell, of course, but that might be outside the scope of what I’m capable of exploring in my own time). Since working at Adobe, I’ve grown an even deeper appreciation for how hard it is to teach machines to reason about the visual world, and beyond that, the physical world. ...

Most of my professional experience so far has been focused on language and 2D vision. But lately, I’ve been getting very curious about spatial intelligence - the capability to perceive, reason about, and generate in 3D space, not just flat pixels and tokens. If you’ve been following the news, you’ll notice a surge of interest in world models (Yann LeCun’s AMI Labs, Fei-Fei Li’s World Labs) and embodied agents - systems that don’t just process text, but understand and act in physical space. Spatial intelligence is highly relevant here, although it’s more directly related to embodied agents (which need spatial intelligence directly since they have to navigate and act in physical 3D space) than world models (they are about learning an internal model of how the world works - spatial understanding is important for this, but world models also cover physics, causality, and dynamics, not just 3D geometry). This series is my first step into understanding more about spatial intelligence, starting with NeRF. ...