Tinkering more with Ray Tracing In One Weekend, I managed to render a dolly zoom. The Dolly Zoom Wikipedia page has an equation for calculating distance as a function of FOV, which made this straightforward. I solved the equation for FOV instead, and then rendered a bunch of frames while moving the camera closer to the center ball.