area solar radiation performance

[robertkalasek]

i am working on a intel core i9-13900k machine with 24 cores, 128gb ram and one of the fastests nvme ssds currently existing. and i am about to do some experiments with a test region of about 4000 x 4500 cells 10x10m each. 

the expected reuslult set to "Whole year", all the other settings are left default exept for the "Diffusse proportion" - changed from 0.3 to 0.32 and "Transmittivity" from 0.5 to 0.48.

that calculation takes quite a long time and the most annoying thing is, that the taskmanager shows just one single core doing at least something - at an overall cpu workload of 7 - 9% and a rate of about 1,5 to 5.2 GHz. 

but on top of that astonishingly poor performance one older machine with lower performance specs in any dimension is outperforming the modern machine by about 30% (!).

[GBacon]

No, but I wasn't getting anywhere with the old tool after running for days. Going to run this again on a new, much more powerful machine when I get it up and running. The whole county ~400 sq. miles at 5-ft DSM resolution took ~72 hours.

[RyanDeBruyn]

Hi @robertkalasek , thanks for the post. 

It is important to note that the original Solar radiation tools are not multi-core / multithread and will run on a single core.  So having  more (24) cores will not be an advantage here.  At what portion during the analysis you are seeing the performance metrics I am not sure. Not sure what the comparison to the older machine is or what the difference is there.

With the new Solar Radiation tools available in ArcGIS Pro 3.2, these tools will use all available cores and if you have an available GPU will benefit significantly. 

Even running CPU only mode one would benefit from the new implementation of the tool compared to the original.

Little additional details.  In CPU mode only  all cores will be occupied, but when in GPU mode only 2-3 cores will be used to handle the processing that is required.  So with a good GPU card and the machine above you should have much improved processing performance and scalability.

We are additionally working on performance improvements for the new tools for the next release 3.3 for CPU and GPU processing.

[GBacon]

@RyanDeBruyn  I ran this analysis again against a GPU with 48 GB of dedicated memory without downgrading resolution and it completed in 50 hours as opposed to 72 hours at coarser resolution. I've published image services of the resulting rasters and am very happy with the results.

One question I have though: why does the output of a mean summary statistic from the multidimensional toolbar produce a .crf if different dates are no longer present? The sum statistic does not produce a .crf.

[RyanDeBruyn]

Hi @GBacon glad you are having some success. 

"Why does the output of a mean summary statistic from the multidimensional toolbar produce a .crf if different dates are no longer present?"

The result of the summary statistics is an aggregated output and we maintain the consistency of the output format as a multidimensional raster, even though as you mentioned the data has only one result (mean values) from all time slices.  This does also help to make clear the properties of the data from the analysis. The result .crf should not be any different or affect any subsequent use.   

Note: You can run similar results using the GP tool Aggregate Multidimensional Raster (also found in the MD ribbon) where the result may differ in the number (count) of output slices.

Hope this helps.  

[robertkalasek]

@GBacon: i would have expected a more impressive advantage by using a GPU. 72 compared to 50 hours obviously is less than 50% increase in performance. so even without using the gpu this increase should be much higher by using up to 32 cores on modern machines compared to just one core with the previous version.

[GBacon]

Yeah, well. Who knows. I've also encountered a GPU related error along the way even though the processing finished. I've updated my NVIDIA driver to see if that helps.