5 Replies Latest reply on Mar 29, 2011 9:46 AM by csny490

    Feature to raster Python very slow in ArcGIS 10

    kvolleberg
      I've got an ArcGIS 9.3 script that is also to be used in ArcGIS 10 (but not exclusively).
      Under ArcGIS 9.3, this script runs quite fast (<1min per raster), but in ArcGIS 10, running time is dramatically longer (>10min per raster).
      This is, obviously, not acceptable.

      The resulting raster has 7349 columns and 6110 rows.

      Can anyone tell the reason behind this huge differnce in processing-time (is it because I'm using arcgisscriping instead of arcpy?).


      import os, sys, arcgisscripting
      gp = arcgisscripting.create(9.3)
      
      inShape = gp.GetParameterAsText(0)
      outRas = gp.GetParameterAsText(1)
      field = gp.GetParameterAsText(2)
      cellSize = gp.GetParameterAsText(3)
      snapRaster = gp.getParameter(4)
      
      #Storing old snapping parameter
      oldSnap = gp.snapRaster
      #Define snap raster 
      #define extent
      gp.SnapRaster = snapRaster
      oldExt = gp.extent
      rasDesc = gp.Describe(snapRaster)
      newExt = rasDesc.extent
      gp.extent = newExt
      
      #Conversion
      gp.FeatureToRaster_conversion(inShape, field, outRas, cellSize)
      gp.snapRaster = oldSnap
      gp.extent = oldExt
      del oldSnap, oldExt
        • Re: Feature to raster Python very slow in ArcGIS 10
          csny490
          I'm guessing this is related to your issue, but a while back I reported a bug to ESRI:

          When you set a snaperaster in a Python script, the SA tools all perform significantly slower (compared to if you don't set it).

          The ESRI fellow noted the slowdown, but didn't seem to understand why this was a big deal, and wanted to close the "incident" right away ( incident #807205 BTW)... Maybe they get incentive pay for "number of closed incidents"? Hmm...

          Anyway, in ancient times (pre v9.2), before the gp.snapraster function worked (at all), I wrote a function that adjusted a given gp.extent to the cell alignment of a given raster (see code below). Basically this is what gp.snapraster is doing behind the scenes... Upon finding the aforementioned bug in v9.3, I resurrected my code, and happily use it to this day. Probably more verbose than necessary, but it works great... Might require some small re-writes for v10.

          def snapExtentToRaster(inputExtent, snapRaster):
              "Returns a xMin, yMin, xMax, yMax extent snapped to the cell allignment of a raster"
              xMinInput = float(inputExtent.split(" ")[0])
              yMinInput = float(inputExtent.split(" ")[1])
              xMaxInput = float(inputExtent.split(" ")[2])
              yMaxInput = float(inputExtent.split(" ")[3])
              dscSnapRaster = gp.describe(snapRaster)
              cellSize = float(dscSnapRaster.meancellheight)
              xMinSnap = float(dscSnapRaster.extent.xmin)
              yMinSnap = float(dscSnapRaster.extent.ymin)
              xMaxSnap = float(dscSnapRaster.extent.xmax)
              yMaxSnap = float(dscSnapRaster.extent.ymax)
              #Calculates a modified xMinInput
              if xMinSnap < xMinInput:
                  if divmod(abs(xMinInput - xMinSnap), cellSize)[1] / cellSize < .5:
                      xMinOutput = xMinSnap + divmod(abs(xMinInput - xMinSnap), cellSize)[0] * cellSize
                  else:
                      xMinOutput = xMinSnap + cellSize + divmod(abs(xMinInput - xMinSnap), cellSize)[0] * cellSize
              else:
                  if divmod(abs(xMinInput - xMinSnap), cellSize)[1] / cellSize < .5:
                      xMinOutput = xMinSnap - divmod(abs(xMinInput - xMinSnap), cellSize)[0] * cellSize
                  else:
                      xMinOutput = xMinSnap - cellSize - divmod(abs(xMinInput - xMinSnap), cellSize)[0] * cellSize      
              #Calculates a modified yMinInput
              if yMinSnap < yMinInput:
                  if divmod(abs(yMinInput - yMinSnap), cellSize)[1] / cellSize < .5:
                      yMinOutput = yMinSnap + divmod(abs(yMinInput - yMinSnap), cellSize)[0] * cellSize
                  else:
                      yMinOutput = yMinSnap + cellSize + divmod(abs(yMinInput - yMinSnap), cellSize)[0] * cellSize
              else:
                  if divmod(abs(yMinInput - yMinSnap), cellSize)[1] / cellSize < .5:
                      yMinOutput = yMinSnap - divmod(abs(yMinInput - yMinSnap), cellSize)[0] * cellSize
                  else:
                      yMinOutput = yMinSnap - cellSize - divmod(abs(yMinInput - yMinSnap), cellSize)[0] * cellSize     
              #Calculates a modified xMaxInput
              if xMaxSnap < xMaxInput:
                  if divmod(abs(xMaxInput - xMaxSnap), cellSize)[1] / cellSize < .5:
                      xMaxOutput = xMaxSnap + divmod(abs(xMaxInput - xMaxSnap), cellSize)[0] * cellSize
                  else:
                      xMaxOutput = xMaxSnap + cellSize + divmod(abs(xMaxInput - xMaxSnap), cellSize)[0] * cellSize
              else:
                  if divmod(abs(xMaxInput - xMaxSnap), cellSize)[1] / cellSize < .5:
                      xMaxOutput = xMaxSnap - divmod(abs(xMaxInput - xMaxSnap), cellSize)[0] * cellSize
                  else:
                      xMaxOutput = xMaxSnap - cellSize - divmod(abs(xMaxInput - xMaxSnap), cellSize)[0] * cellSize
              #Calculates a modified yMaxInput
              if yMaxSnap < yMaxInput:
                  if divmod(abs(yMaxInput - yMaxSnap), cellSize)[1] / cellSize < .5:
                      yMaxOutput = yMaxSnap + divmod(abs(yMaxInput - yMaxSnap), cellSize)[0] * cellSize
                  else:
                      yMaxOutput = yMaxSnap + cellSize + divmod(abs(yMaxInput - yMaxSnap), cellSize)[0] * cellSize
              else:
                  if divmod(abs(yMaxInput - yMaxSnap), cellSize)[1] / cellSize < .5:
                      yMaxOutput = yMaxSnap - divmod(abs(yMaxInput - yMaxSnap), cellSize)[0] * cellSize
                  else:
                      yMaxOutput = yMaxSnap - cellSize - divmod(abs(yMaxInput - yMaxSnap), cellSize)[0] * cellSize     
              #Returns the entire modified extent
              return str(xMinOutput) + " " + str(yMinOutput) + " " + str(xMaxOutput) + " " + str(yMaxOutput)
          
          
          #Example of use.... adjuest the extent of oesfBufferFC so it aligns with gnnReadGrd 
          oesfBufferFC = fgdbPath + "\\oesf_buffer"
          gnnReadGrd = r"\\dnrutoly05\d$\csny490\gnn_20100303\mr200_sppsz_2006\gnn_20100303"
          gp.extent = oesfBufferFC
          gp.extent = snapExtentToRaster(gp.extent, gnnReadGrd)
          • Re: Feature to raster Python very slow in ArcGIS 10
            kvolleberg
            Hi Chris,
            thanks for the suggestion. I implemented it (indeed I had to modify a bit), but without success. I'm going to try if it has to do with arcgisscripting instead of arcpy in ArcGIS 10.
            If I find a solution, I'll post it here.

            EDIT: arcpy does not work either. Seems like something in FeaturetoRaster coversion. Maybe ESRI has a solution?
            • Re: Feature to raster Python very slow in ArcGIS 10
              csny490
              Is it faster if you comment out the gp.snapraster line?

              Also, I notice you are setting the extent of the new raster to be the same as the snapRaster... Why not set the gp.extent = to that of the inputShapefile and then set gp.snapraster = snapRaster? By setting the gp.extent = the extent of snapRaster, you don't need to then set gp.snapraster = snapRaster.

              Could it be that the extent of the snapRaster is much more expansive than the extent of the inputShapefile (and therefore has to take a lot of time to produce a bunch of extra NODATA cells on the periphery)?
              • Re: Feature to raster Python very slow in ArcGIS 10
                mzcoyle
                In my experience, when migrating scripts from the arcgisscripting module to arcpy, a lot of issues can come up.

                When running an arcgisscripting module script in an Arc10 environment, there was a noticeable loss of performance in terms of speed of processing. Compared to running the same scripts under an Arc9.3 environment.

                When testing using both arcgisscripting and arcpy modules in an Arc10 environment in the same script, there were severe performance impacts and instability.

                Conclusion I found: It was be easier to write a separate script for Arc10 and Arc9.x than to try to make the same script compatible under both environments.
                • Re: Feature to raster Python very slow in ArcGIS 10
                  csny490
                  True - My forays into v10 arcpy land have not been very encouraging.

                  ArcGIS v9.3.1 remains my geoprocessing workhorse...