Cost Distance Tool Failure

[MaryM]

I have a cost distance tool setup for a batch process.  To take a bunch of inputs from a folder and then use the same cost raster and file saving conventions.  My tool will run but it never runs the process of the cost distance.  It just sits there and does not run.  I tried making it with arcpy the regular way and tried with gp arcpy.  Either way it does the same thing.  Anyone know what I can do to make this run?

#Cost distance for all hubs #by rangermry, 7-11-2013 using Python 2.6.5  import arcpy, os from arcpy.sa import * from arcpy import env  #User inputs #Input Rasters Location Folder inputFolder=arcpy.GetParameterAsText(0) #Cost Raster a.k.a. Impedence Layer inCostRaster=arcpy.GetParameterAsText(1)  #Derived inputs #Make output folder #Take input folder name as variable. rootFolName=str(inputFolder.split("\\")[-1]) #Take input folder name off of root folder path. newRoot=inputFolder.replace(rootFolName,"") outputFolder=os.path.join(newRoot,"Cost_Distance") if not os.path.isdir(outputFolder):     os.makedirs(outputFolder)     arcpy.AddMessage("-Folder made for output : "+outputFolder+'\n') #Environmental Workspace env.workspace=inputFolder #Get all Raster Items from inputFolder as workspace rasterList=arcpy.ListRasters() arcpy.AddMessage("-Obtaining all raster inputs\n.")  try:     for rast in rasterList:         if not rast=="imped":             fullpath = os.path.join(inputFolder, rast)             arcpy.AddMessage("-Processing file : "+rast+'\n')             #Set Local Variables             inSourceData=fullpath             outBkLinkRaster=outputFolder+"\\BkLk_"+rast.replace("value","")             outDistRaster=outputFolder+"\\CD_"+rast.replace("value","")             #Execute CostDistance             arcpy.AddMessage("Proceeding to cost distance.\n")             #Check out Spatial Analyst Licesne             arcpy.CheckOutExtension("spatial")             arcpy.gp.CostDistance_sa(inSourceData,inCostRaster,outDistRaster,"",outBkLinkRaster)             arcpy.AddMessage("-Processed file: "+rast+"Saving file...")             arcpy.AddMessage("-Saved file! \n ") except IOError:     #Print traceback error.     tb=sys.exc_info()[2]     tbinfo=traceback.format_tb(tb)[0]     arcpy.AddMessage("PYTHON ERRORS:\nTraceback info: "+tbinfo+"\nError Info:"+str(sys.exc_info()[1]))      

[MaryM]

It makes the files for the first value and then takes all day and does not finish.  If I do one value manually, it takes 2-3 hours.  Does anyone know any kind of monitoring I can build in this or any ideas of why this script seems to stall out?

[curtvprice · Accepted solution]

It makes the files for the first value and then takes all day and does not finish.  If I do one value manually, it takes 2-3 hours.

I'm wondering if your workspace and scratch workspace are set the same when you run it interactively.

Does anyone know any kind of monitoring I can build in this or any ideas of why this script seems to stall out?

Once a script starts a GP tool there is no way to monitor it except to check the folder in Windows Explorer to see if anything is happening and Task Manager to see if the process is using CPU or memory.

When debugging a script like this, from my own sad experience I highly recommend testing with small datasets - it can save you a LOT of time!

I took the liberty of re-writing your script for style, you may find my changes helpful to improve readability and debugging. Note I set the scratch and current workspace to the same value (x = y = 1 is equivalent to x=1;y=1), this should help cut down on unnecessary file copying. I also ran the Cost Distance using map algebra (this is why you include "from arcpy.sa import *"), and used format() to process your strings instead of adding them together with + -- this improves readability.

I also explicitly set the raster processing environment to match the source raster. You may have to change what I set it to, but when running cost distance you want to be explicit so you don't process cells you don't have to -- it takes a long time to run as the processing is involved and requires a lot of I/O and floating point processing.

I never skimp on spaces in Python, the style guide frowns on this and when I do so I find I can't easily debug my own code.

import os import arcpy from arcpy.sa import * from arcpy import env  # Check out Spatial Analyst License first (once) arcpy.CheckOutExtension("spatial")  # User inputs # Parameter 0: Input Rasters Location Folder inputFolder = arcpy.GetParameterAsText(0) # Parameter 2: Cost Raster a.k.a. Impedance Layer inCostRaster = arcpy.GetParameterAsText(1)  # Derived inputs # Make output folder # Take input folder name as variable. rootFolName = os.path.basename(inputFolder) # Take input folder name off of root folder path. newRoot = os.path.dirname(inputFolder) outputFolder = os.path.join(newRoot,"Cost_Distance") if not os.path.isdir(outputFolder):     os.makedirs(outputFolder)     arcpy.AddMessage("-Folder made for output : {}\n".format(outputFolder)) # Environmental Workspaces env.workspace = env.scratchWorkspace = inputFolder  #Get all Raster Items from inputFolder as workspace rasterList = arcpy.ListRasters() arcpy.AddMessage("-Obtaining all raster inputs\n.") env.workspace = env.scratchWorkspace = outputFolder  try:     for rast in rasterList:         if rast !="imped":             fullpath = os.path.join(inputFolder, rast)             arcpy.AddMessage("-Processing file : {}\n".format(rast))             #Set Local Variables             inSourceData = fullpath             outBkLinkRaster = "BkLk_{}".format(rast.replace("value",""))             outDistRaster = "CD_{}".format(rast.replace("value",""))             #Execute CostDistance             arcpy.AddMessage("Proceeding to cost distance.\n")             # set raster environment             env.extent = inSourceRaster             env.snapRaster = inSourceRaster             env.cellSize = inSourceRaster             ras = CostDistance(inSourceData,inCostRaster,"",outBkLinkRaster)             arcpy.AddMessage("-Processed file: {}\n".format(rast))             ras.save(outDistRaster)             arcpy.AddMessage("-Saved file! \n ") except IOError:     #Print traceback error.     tb = sys.exc_info()[2]     tbinfo = traceback.format_tb(tb)[0]     arcpy.AddMessage(         "PYTHON ERRORS:\nTraceback info:\n{0}\nError Info:{1}".format(         (tbinfo,sys.exc_info()[1])))

[MaryM]

Cool beans!

Your solution worked, all I did was add the scratch workspace part to mine and everything ran successfully and with great timing!  Thanks for the writing of the code so cleanly, it is very clear, I will try to write in that manner.  I seem to get to so excited when I write code and cannot think of the explainations so clearly.