7 Replies Latest reply on Oct 24, 2012 7:45 AM by curtvprice

    Overwrite of output not working

    jaxelrodmfa
      Hi,
      I have a script that writes .dbf files to a folder. The first time the script runs it does what it is supposed to. The second time I run it however, it says the first .dbf to be created already exists and stops. It's not overwriting the files.
      I have arcpy.env.overwriteOutput = True and had it successfully working to overwrite at one point in the development of the script. I'm not sure what made it stop working. I'm pretty sure I added the field mapping and the listing of the files in the log file after it was running smoothly but, have commented them out and it still gives me the error message. Any suggestions would be appreciated....I have no idea what is causing the problem.



      # Import arcpy module
      import arcpy, sys, traceback, time, datetime, os

      # Set overwrite option
      arcpy.env.overwriteOutput = True

      # INPUT NEEDED: Set Read me file workspace
      CURDATE = datetime.date.today()
      logpath = "X:\\Script\Out\ReadMe"
      logfile1 = logpath + str (" ") + str(CURDATE) + ".txt"


      if arcpy.Exists(logfile1):
          arcpy.Delete_management(logfile1)

      log1 = open(logfile1, "a")

      print "Generate Readme file " + str(CURDATE)
      print >> log1, time.strftime("%c")
      print >> log1, "End Processing "



      #Set the current workspace for input file geodatabase
      arcpy.env.workspace = "X:\\Script\Input.gdb"

      #Set Local variables specifyting the output table folder:
      outTable = "X:\\Script\Out"

      print "Start Processing"

      #Set a Search Cursor on the Polygons
      srows = arcpy.SearchCursor("Polygons", "","", "WATERSHED", "" )
      srow = srows.next()

      print "Get Polygon"

      mapFields = ["ID", "STUDY", "NAME"]
      fieldMappings = arcpy.FieldMappings()
      for mapField in mapFields:
          fieldMap = arcpy.FieldMap()
          fieldMap.addInputField("Points", mapField)
          fieldMappings.addFieldMap(fieldMap)


      while srow:

          pid = srow.ObjectID
          poly_lyr = arcpy.MakeFeatureLayer_management("Polygons",'pgon_select',""" "ObjectID" = """+ str(pid))
         
          print "Watershed is " + str(srow.WATERSHED)
        
          arcpy.MakeFeatureLayer_management("Points", "Points_lyr", "", "",  )
          print "Make point feature layer"

        
          arcpy.SelectLayerByLocation_management("Points_lyr", "INTERSECT", "pgon_select", "", "NEW_SELECTION")
          print "Select points by " + str(srow.WATERSHED) + " watershed"

          arcpy.TableToTable_conversion("Points_lyr", outTable , str(srow.WATERSHED)+ ".dbf","")
          print "Make table " + str(srow.WATERSHED)
         
          srow = srows.next()
          print "Next Polygon"



      print "Write .dbf tables to log file"
      print >> log1, "Files generated = "
      file_list = os.listdir(outTable)
      for file in file_list:
          if file.endswith (".dbf"):
              log1.write(file + "\n")


      print "End Processing"


      log1.close()

        • Re: Overwrite of output not working
          mdenil
          Honor the Zen of Python

          The second line of the Zen of Python doxology is:
          Explicit is better than implicit.

          It is better to test for and remove files you want to replace (Explicit )
          than to rely on an Arc environment setting (implicit).

          Although python can import arcpy, python is not itself arcpy.
          It does not necessarily honor an Arc environment.

          You can get the Zen of Python by typing
          import this
          on the Python command line.
          • Re: Overwrite of output not working
            curtvprice
            Hi,
            I have a script that writes .dbf files to a folder. The first time the script runs it does what it is supposed to. The second time I run it however, it says the first .dbf to be created already exists and stops. It's not overwriting the files.
            I have arcpy.env.overwriteOutput = True and had it successfully working to overwrite at one point in the development of the script. I'm not sure what made it stop working. I'm pretty sure I added the field mapping and the listing of the files in the log file after it was running smoothly but, have commented them out and it still gives me the error message. Any suggestions would be appreciated....I have no idea what is causing the problem.

            # Import arcpy module import arcpy, sys, traceback, time, datetime, os  # Set overwrite option arcpy.env.overwriteOutput = True  # INPUT NEEDED: Set Read me file workspace CURDATE = datetime.date.today() logpath = "X:\\Script\Out\ReadMe" logfile1 = logpath + str (" ") + str(CURDATE) + ".txt"   if arcpy.Exists(logfile1):     arcpy.Delete_management(logfile1)  log1 = open(logfile1, "a")  print "Generate Readme file " + str(CURDATE) print >> log1, time.strftime("%c") print >> log1, "End Processing "    #Set the current workspace for input file geodatabase arcpy.env.workspace = "X:\\Script\Input.gdb"  #Set Local variables specifyting the output table folder: outTable = "X:\\Script\Out"  print "Start Processing"  #Set a Search Cursor on the Polygons srows = arcpy.SearchCursor("Polygons", "","", "WATERSHED", "" ) srow = srows.next()  print "Get Polygon"  mapFields = ["ID", "STUDY", "NAME"] fieldMappings = arcpy.FieldMappings() for mapField in mapFields:     fieldMap = arcpy.FieldMap()     fieldMap.addInputField("Points", mapField)     fieldMappings.addFieldMap(fieldMap)   while srow:      pid = srow.ObjectID     poly_lyr = arcpy.MakeFeatureLayer_management("Polygons",'pgon_select',""" "ObjectID" = """+ str(pid))          print "Watershed is " + str(srow.WATERSHED)         arcpy.MakeFeatureLayer_management("Points", "Points_lyr", "", "",  )     print "Make point feature layer"          arcpy.SelectLayerByLocation_management("Points_lyr", "INTERSECT", "pgon_select", "", "NEW_SELECTION")     print "Select points by " + str(srow.WATERSHED) + " watershed"      arcpy.TableToTable_conversion("Points_lyr", outTable , str(srow.WATERSHED)+ ".dbf","")     print "Make table " + str(srow.WATERSHED)          srow = srows.next()     print "Next Polygon"    print "Write .dbf tables to log file" print >> log1, "Files generated = " file_list = os.listdir(outTable) for file in file_list:     if file.endswith (".dbf"):         log1.write(file + "\n")   print "End Processing"   log1.close() 


            You have neglected to close the cursor when you are done. You must do this, otherwise the cursor stays open -- and you may run into file locking on the next run.

            del srows


            Also, using a layer object ("Polygons") with a tool that is attached to an open cursor (outside of directly accessing the geometry or actions like that) is generally bad practice as you are accessing the dataset from two objects at the same time.

            I think it is much safer to create a list of unique watershed ids first, closing the cursor and then looping on that list.

            Note the "for" loop construct below. This is the best way to loop through cursors in arcpy (until 10.1 when you should use arcpy.da).

            #Set a Search Cursor on the Polygons srows = arcpy.SearchCursor("Polygons", "","", "WATERSHED", "" ) sheds= list() for srow in srows:   sheds.append(srow.WATERSHED) del row,srows # close/delete cursor variables  for shed in sheds:   ...    arcpy.MakeFeaturelayer_management('"WATERSHED" = \'' + shed + "'"...)   ...


            Hope this helps.
            • Re: Overwrite of output not working
              jaxelrodmfa
              mdenil,
              Although the Zen is enlightening, the overwrite function still won't work as needed. Is there a specific piece of code you could suggest? I'm a python novice.
              • Re: Overwrite of output not working
                curtvprice
                mdenil,
                Although the Zen is enlightening, the overwrite function still won't work as needed. Is there a specific piece of code you could suggest? I'm a python novice.


                I think what Mark is getting at is inserting something like this before the TableToTable call to explicitly delete the table if it's already there:

                tabName = srow.WATERSHED + ".dbf")
                tabPath = os.path.join(outTable,tabName) 
                if arcpy.exists(tabPath): arcpy.Delete_management(tabPath)
                arcpy.TableToTable_conversion("Points_lyr", outTable , tabName)
                


                However, my experience is that arcpy.env.overwriteOutput seems to work fine as long as there are no outstanding file locks (ie active layers pointing to it, etc).
                • Re: Overwrite of output not working
                  danagerous
                  Hi. I ended up here because i was having the same problem. The suggestions here make sense as well. The thing is i don't want to put an
                  if arcpy.Exists() 
                  for every layer.

                  I was having the same problem until i put my section of code between the
                  try: 
                     <code> 
                  except: 
                     <code>
                  and it worked. Why that makes all the difference is beyond me but it might be worth trying because it worked for me.
                  • Re: Overwrite of output not working
                    laurelkish
                    I don't know if it's a python issue or an ESRI/arcpy issue and I don't know why, but this last tidbit of info (try:except) solved a problem I've been having off and on for months.  Halle-freaking-lujah ! Thank you.
                    • Re: Overwrite of output not working
                      curtvprice
                      I don't know if it's a python issue or an ESRI/arcpy issue and I don't know why, but this last tidbit of info (try:except) solved a problem I've been having off and on for months.  Halle-freaking-lujah ! Thank you.


                      Even better is a try - except - finally, to make sure things are cleaned up no matter what happens. (Layers, table views, open files, cursors, etc.)

                      For example:

                      try:
                           lyr1 = arcpy.MakeFeatureLayer("myfile.shp","myLyr")
                           # ... do stuff with lyr 1
                      except:
                          # ... do stuff you want to do if it fails
                      finally:
                          # do stuff you want to do no matter what, for example, try to delete the layer:
                          try:
                              arcpy.Delete_management(lyr1)
                          except:
                              pass