Hm. I partly answered my own question. If I use `esri.tiledMapLayer` instead of `esri.dynamicMapLayer` with the exact same URL, I see the tiled images.
So I guess my real question is, when displaying data from a map service, when should you use dynamicMapLayer vs tiledMapLayer?