NEED ADVICE/IDEA TO BACKUP CACHE FOLDERS

[MuryadiOey]

Hi All,
I work in a company where we have cache folders in different locations (Mount Points). the cache folders size around 20 GB - 100 GB and you know they contain small png files (~8 kb). Our backup software (BackupExec 12) always fail whenever trying to backup cache folders due to those small files that create busy read/write to tape. I am hoping someone has successful backing up cache folders to give me advice/idea on how they backup cache folders. I really appreciate any advice/idea since I have no luck to find best solution for this.

Thank you so much,
Murry

[AlexeyTereshenkov]

Hi Murry,

It can indeed be a difficult task to transfer the cache data. I have a couple of considerations for you:

1) Have you thought to migrate to the compact cache which was introduced in ArcGIS 10? You can do this by using the GP tool Server toolbox > Caching > Convert Map Server Cache Storage Server (or just by caching the map service to the compact cache). Compact cache files are much larger (and there will be fewer of them than those .png files you have for sure) thus it goes *much* faster to transfer them.

2) Another thing I have seen a couple of times - it took a bit longer time to copy massive data in Windows when files have longer names. It may be not directly relevant to your case, but still worth to check: http://support.microsoft.com/kb/130694.

3) There is a KB article: http://support.esri.com/en/knowledgebase/techarticles/detail/33234 where you can read on ways to copy the cache data.

4) Maybe you can archive the cache data and then copy the archive files? This should definitely boost the performance. Then just exclude the ArcGIS Server cache folder in your backup software, so it will not choke when copying cache tiles.

[MuryadiOey]

Hi Murry,

It can indeed be a difficult task to transfer the cache data. I have a couple of considerations for you:

1) Have you thought to migrate to the compact cache which was introduced in ArcGIS 10? You can do this by using the GP tool Server toolbox > Caching > Convert Map Server Cache Storage Server (or just by caching the map service to the compact cache). Compact cache files are much larger (and there will be fewer of them than those .png files you have for sure) thus it goes *much* faster to transfer them.

2) Another thing I have seen a couple of times - it took a bit longer time to copy massive data in Windows when files have longer names. It may be not directly relevant to your case, but still worth to check: http://support.microsoft.com/kb/130694.

3) There is a KB article: http://support.esri.com/en/knowledgebase/techarticles/detail/33234 where you can read on ways to copy the cache data.

4) Maybe you can archive the cache data and then copy the archive files? This should definitely boost the performance. Then just exclude the ArcGIS Server cache folder in your backup software, so it will not choke when copying cache tiles.

Thank you so much for your advices. I have question with number 1, in your experience, how long will it take to compact cache files that have size 20GB?
With number 4. what do you mean with archive? windows archive? 

Thank you,
Murry

[AlexeyTereshenkov]

Murry,

Yeah, I meant archiving with Windows, just packing in .zip or .7z. Regarding converting big caches, it all depends on the hardware resources available (i.e., how many instances of the service you can allocate). In the staging environment, I've converted a couple of caches (ca 10GB) and it took roughly 14 hours. I had 4 cores and 8 GB RAM, SATA hard drive. But I had some load on the staging server at the same so it is quite hard to estimate the exact time that was required.