Better errors porting python

578
4
11-18-2022 04:24 PM
JohnZastrow
New Contributor III

I'm trying to get an old python script from 10.1 days to run in ArcGISPro 3. I ran it through 2to3 and while it runs, it emits useless warnings (File cannot be processed)  for each layer and has blank output after the layer name. I'm hoping to a) get help getting it to run in its fully glory (maybe switching in some newer functions) and/or b) figure out how to get more informative errors. I just started using ArcGIS Pro and chose to use Notebook until I can figure out running from an IDE. 

 

PS, I thought it might be some outdated metadata, but I created a new layer in ArcGISPro and it still shows the same warnings. So far it's ran on Shapefiles, FileGDB vectors, and .tifs. Same problems.

Here's the top 10% of it.

 

 

# Version: ArcGIS 10.1 and 10.2
# Purpose: This script will iterate through each GIS file in a folder (and all recursive folders) and report information
# about each layers metadata.  The script is intended to run from a script tool inside an ArcGIS Toolbox.  The script requires three
# user input values:
#   1. An input folder
#   2. An output Excel File
#   3. An output folder to store temporary files
# Revised: John Zastrow, November 18, 2022, porting to work with Python 3 and ArcGIS Pro 3.0

try:
    # Setup environment and import modules
    import xlwt, os, arcpy
    from xml.etree.ElementTree import ElementTree
    from xml.etree.ElementTree import Element, SubElement

    arcpy.env.overwriteOutput = True

    # Get user input parameters, please note that the inputFolder cannot contain any GIS files, or they will not be read
    # inputFolder = arcpy.GetParameterAsText(0)
   # outputFile = arcpy.GetParameterAsText(1)
    # tempOutputFolder = arcpy.GetParameterAsText(2)
    # Input parameters below are for testing only
     inputFolder = r"C:\data\workingData\dataMiner\subfolder"
    outputFile = r"C:\dataMiner\version_2.0\output\output.xls"
    tempOutputFolder = r"C:\dataMiner\version_2.0\temp"

    # Create workbook with one sheet named Output
    wb = xlwt.Workbook()
    ws = wb.add_sheet("Output")

    # Create list of column names in the workbook
    headerNamesList = [
        "ID",
        "Feature Name",
        "Feature Path",
        "Feature Type",
        "Object Type",
        "Feature Count",
        "Title",
        "Keywords",
        "Abstract",
        "Purpose",
        "Contact",
        "Distribution",
        "Coordinate System Type",
        "Geographic Coordinate System",
        "Projected Coordinate System",
        "Horizontal Units",
        "North Bounding Coordinate",
        "East Bounding Coordinate",
        "South Bounding Coordinate",
        "West Bounding Coordinate",
        "Created Date",
        "Modified Date",
        "MetaData Version",
    ]
    headerColumn = 0
    for name in headerNamesList:
        ws.write(0, headerColumn, name)
        headerColumn = headerColumn + 1

    # Create empty global lists and other global variables
    datasetList = []
    fcList = []
    tableList = []
    workspaceList = []
    rasterList = []
    listOfEnvWorkspaces = []
    excelRow = 1

    # Fucntion to test if metadata tags have values in XML file
    def verifyMetadata(tag‌‌
        global tagReturn
        if tag is not None:
            tagReturn = tag.text
            return tagReturn
        else:
            tagReturn = "METADATA VALUE IS EMPTY"
            return tagReturn

    # Searches through all folders for GIS files
    for dir, folderNames, fileNames in os.walk(inputFolder‌‌
        listOfEnvWorkspaces.append(dir)
    for subFolderName in listOfEnvWorkspaces:
        arcpy.env.workspace = subFolderName
        print("Processing Folder: " + subFolderName)
        arcpy.AddMessage("Processing Folder: " + subFolderName)

        datasetList = arcpy.ListDatasets()
        fcList = arcpy.ListFeatureClasses()
        tableList = arcpy.ListTables()
        workspaceList = arcpy.ListWorkspaces("", "Access")
        rasterList = arcpy.ListRasters()

        # Checks for feature classes in file geodatabases feature classes, coverages, and CAD files
        for dataset in datasetList:
            for fc in arcpy.ListFeatureClasses("", "", dataset‌‌
                try:
                    # Creates a temporary empty XML file to store each files XML data
                    fcXML = tempOutputFolder + "/" + fc + ".xml"
                    tempXMLFile = open(fcXML, "w")
                    tempXMLFile.write("<metadata>\n")
                    tempXMLFile.write("</metadata>\n")
                    tempXMLFile.close()
                    # Copies metadata from file into temporary XML file
                    arcpy.MetadataImporter_conversion(fc, fcXML)

                    # Makes an ElementTree object and reads XML into ElementTree
                    tree = ElementTree()
                    tree.parse(fcXML)
                    # Checks on XML version (9.x vs. 10.x) and gathers information on metadata
                    if tree.find("Esri/ArcGISFormat") is not None:
                        print(
                            "     Processing File: "
                            + os.path.join(dataset, fc)
                            + " (Version 10.x XML)"
                        )
                        arcpy.AddMessage(
                            "     Processing File: "
                            + os.path.join(dataset, fc)
                            + " (Version 10.x XML)"
                        )
                        title = tree.find("dataIdInfo/idCitation/resTitle")
                        keywords = tree.find("dataIdInfo/searchKeys/keyword")
                        abstract = tree.find("dataIdInfo/idAbs")
                        purpose = tree.find("dataIdInfo/idPurp")
                        contact = tree.find("mdContact/rpIndName")
                        distribution = tree.find("distInfo/distFormat/formatName")
                        fcMetaDataVersion = "10.x"
                    else:
                        print(
                            "     Processing File: "
                            + os.path.join(dataset, fc)
                            + " (Version 9.x XML)"
                        )
                        arcpy.AddMessage(
                            "     Processing File: "
                            + os.path.join(dataset, fc)
                            + " (Version 9.x XML)"
                        )
                        title = tree.find("idinfo/citation/citeinfo/title")
                        keywords = tree.find("idinfo/keywords/theme/themekey")
                        abstract = tree.find("idinfo/descript/abstract")
                        purpose = tree.find("idinfo/descript/purpose")
                        contact = tree.find("idinfo/ptcontac/cntinfo/cntperp/cntper")
                        distribution = tree.find("distinfo/resdesc")
                        fcMetaDataVersion = "9.x"
                    created = tree.find("Esri/CreaDate")
                    modified = tree.find("Esri/ModDate")

 



 

0 Kudos
4 Replies
DanPatterson
MVP Esteemed Contributor

could you edit your post so it is formatted with line numbers

Code formatting ... the Community Version - Esri Community


... sort of retired...
by Anonymous User
Not applicable

There are some syntax errors in this pasted code- for example:

 

for fc in arcpy.ListFeatureClasses("", "", dataset‌‌

 

its missing the last ). 

Edit to add- I think those were the 😞 that were converted to emojis before changing it a formatted code.

0 Kudos
by Anonymous User
Not applicable

The MetadataImporter_conversion line, doesn't look like it has the right arguments. From the Pro docs (since we are working in Pro now), it looks like it expects a template for arg 1, and arg 2 is expecting a featureclass, or string 'ALWAYS'?  This is kind of odd that the docs do not provide the typical arg, explanation, expected type format so I am not sure what they expect since the two uses of the MetadataImporter method in that doc have different types in the args.

# Copies metadata from file into temporary XML file
fcXML = tempOutputFolder + "/" + fc + ".xml"
arcpy.MetadataImporter_conversion(fc, fcXML)

 

Maybe you need the saveAsXML, which matches your code comment better:

# Save an exact copy of the item's ArcGIS metadata
output_copy = "C:/data/roads_copy.xml"
src_item_md.saveAsXML(output_copy)
JohnZastrow
New Contributor III

Thanks. My computer melted down shortly after posting this and I'm still getting it recovered. I'll start work on this again soon hopefully.

0 Kudos