Suggestions on calculating Slope on a 5Tb Raster Dataset?

[deleted-user-xAqOOPuoS824]

I have a 5Tb Raster Dataset in SDE that had no problems calculating Hillshade, now I want to calculate Slope and it pops back in 5 seconds with a 99999 error Table not Registered.  ESRI Support suggesteed that the file is simply too big. Anyone out there playing with mega files that can offer suggestions.  I can dice it up into smaller rasters, but what about the edge effects?

[JayantaPoddar]

Testing the suggestions might take a lot of time.

1. Disable Background Processing. Then try running the Slope tool.

2. Try Running the Slope Tool from (standalone) ArcCatalog.

3. If the Raster doesn't have a Projected Coordinate System, try Projecting the Raster to an appropriate Projected Coordinate System. The Units should be linear (Meters, Feet, etc.). Try running Slope Tool using the Projected Elevation Raster.

[deleted-user-xAqOOPuoS824]

Jayanta,

Thanks for the tips. I tried running with background off and it still dies in 5 seconds. Seems that ArcGIS isn’t ready for big rasters. However, I did get a clue from Dan Patterson who said try it in ArcPro and it seems to be working. It takes about 3 days to see results so I will find out Monday.

-John

[DanPatterson_Retired]

I agree with tech support and if you had any chance, running it in Pro would be it.

what is the file storage type (mosaic dataset? other?)

What is your resolution? doubling the cell size (resample) would reduce your file size to about a 1/4.

[deleted-user-xAqOOPuoS824]

Dan,

Looks like ArcPro is doing the trick. It takes about 3 days to get results, so I will know Monday. The file is a 5Tb, 0.5 cell Raster Dataset in SDE.

Thanks for the tip.

-John

[DanPatterson_Retired]

part of the bottleneck is undoubtedly SDE... but at least you are moving forward

[VinceAngelo]

There shouldn't be much difference between running this as a 64-bit Python script (64-bit Background Geoprocessing) and running it within Pro.

Given the size of the raster, and the fact that the ArcSDE API reads in tiles, not rows, constructing a cache for the tile height might have caused memory issues in 32-bit clients. Re-loading the raster with a W*2 x H/2 tile size might help for borderline cases, depending on the original tile Width & Height. Changing the tile size to be small enough to reside in-row might also result in significant performance improvement, as would using file-based storage (using rasters in databases was novel at ArcGIS 9.0, but is most of the way toward deprecated now).

In the future, please include image and tile dimensions in the question, since storage doesn't provide an indication of width, height, depth, or compression algorithm.

- V