Eric, when you say you are removing the original images, you aren't messing with the cache folders under arcgisserver directory, are you? Also, I don't see anyplace that you've mention the version you are using. my suggestions might vary depending on the version.
Have you already checked to make sure that the cache is completely built? If not, and you go to a location/scale there is no cache, and you have crate cache in demand, it will try to go back to the original imagery.
Also, depending on where/how you are using the service and cache will make a difference, for example, ArcMap will interpolate at scales in between the cache scales, but if you specify scales in a web app that are different then the cache scale, it may not interpolate and may not show any image if it cant find the original source.
So, a few more details on how you are using the cache, and exactly the situation when you do not have imagery....if/when you move the source files...would be helpful.