blake.terhune

Rebuild Geocode Service

Discussion created by blake.terhune on Mar 10, 2017
Latest reply on Jul 5, 2017 by Stefan.Chapkanski

Our organization hosts several geocode services from both regular and composite address locators. As we all should know, it is important to rebuild an address locator as changes are made to the reference data. Since the locator gets copied to the server, rebuilding the original locator is not sufficient so I've found the entire service needs to also be "rebuilt" (overwritten). We will create scheduled tasks that run a Python script to automate this process.

 

Having experienced the nightmare that occurs when an address locator gets "corrupted" and cannot be rebuilt, I have made the choice to simply recreate the locator each time instead of rebuilding it. The advantage of this method is that you aren't dead in the water should something tragic happen to the original locator because the specifications for creating it are in the script. Alternatively, you could just ensure the specifications of the locator are well documented in the script and just do a rebuild. Either way, the geocode service still needs to get republished with the new address locator reference data.

 

Here is a simplified process I use to rebuild a geocode service.

  1. Create temporary directory where the address locator and service definition files will be staged. This is cleaned up automatically when the script ends.
  2. Create address locator
  3. Create and analyze service definition draft file for sharing the address locator
  4. Modify Service Definition Draft file to overwrite the existing service.
  5. Convert Service Definition Draft to a Service Definition file
  6. Upload and publish service definition file as a service

 

To clarify, this was all tested on 10.2.2.

 

import arcpy
from contextlib import contextmanager
import datetime
import os
import shutil
import tempfile
import xml.dom.minidom as DOM  ## sddTypeReplacement()

def main():
    # Local variables
    agsconn = r"C:\test\MyAdminArcGISServerConnection.ags"
    sdeconn = r"C:\test\MySDEConnection.sde"
    my_fancy_source_data = os.path.join(sdeconn, "my_fancy_source_data")
    locator_name = "MY_FANCY_LOCATOR"

    with makeTempDir() as temp_dir:
        # Create address locator
        temp_locator = os.path.join(temp_dir, locator_name)
        field_info=(
            "'Feature ID' OBJECTID VISIBLE NONE;"
            "'*House Number' ADDR_NUM VISIBLE NONE;"
            "Side <None> VISIBLE NONE;"
            "'Prefix Direction' PRE_DIR VISIBLE NONE;"
            "'Prefix Type' <None> VISIBLE NONE;"
            "'*Street Name' NAME VISIBLE NONE;"
            "'Suffix Type' STREET_TYPE VISIBLE NONE;"
            "'Suffix Direction' SUF_DIR VISIBLE NONE;"
            "'City or Place' <None> VISIBLE NONE;"
            "'ZIP Code' <None> VISIBLE NONE;"
            "State <None> VISIBLE NONE;"
            "Longitude <None> VISIBLE NONE;"
            "Latitude <None> VISIBLE NONE;"
            "'Street ID' <None> VISIBLE NONE;"
            "'Min X value for extent' <None> VISIBLE NONE;"
            "'Max X value for extent' <None> VISIBLE NONE;"
            "'Min Y value for extent' <None> VISIBLE NONE;"
            "'Max Y value for extent' <None> VISIBLE NONE;"
            "'Additional Field' <None> VISIBLE NONE;"
            "'Altname JoinID' <None> VISIBLE NONE"
        )
        arcpy.CreateAddressLocator_geocoding(
            "US Address - Single House",  ## in_address_locator_style
            "'{}' 'Primary Table'".format(my_fancy_source_data),  ## in_reference_data
            field_info,  ## in_field_map
            temp_locator
        )

        # Create and analyze service definition draft file
        sddraft = os.path.join(temp_dir, locator_name+".sddraft")
        sdd_analyze_result = arcpy.CreateGeocodeSDDraft(
            temp_locator,  ## in_address_locator
            sddraft,  ## out_sddraft
            locator_name,  ## service_name
            "FROM_CONNECTION_FILE",  ##  server_type
            agsconn,  ## connection_file_path
            folder_name="Geocoders",
            summary="Created by Python script {}".format(
                datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
            )
        )

        # Inspect analyze result
        """result object of CreateGeocodeSDDraft is a dictionary of dictionaries structured like
            {
                'errors': {},
                'messages': {},
                'warnings': {
                    (u'Warning message', 9999): [],
                    (u'Warning message', 9999): []
                }
            }
        """

        ## Check for error messages to create the issues list.
        ## If there are no analyze errors, analyze_issues will be empty list []
        analyze_issues = sdd_analyze_result.get("errors").keys()
        ## Check for warning messages and add them to the issues list.
        ## Only include those that aren't in the ignore list.
        warn_ignore = [
            ('Missing Summary in Item Description', 97),
            ('Missing Tags in Item Description', 98),
            ('Locator will be copied to the server', 24044)
        ]
        analyze_issues += [
            warn for warn in sdd_analyze_result.get("warnings").keys()
            if warn not in warn_ignore
        ]

        if analyze_issues:
            raise Exception(
                "Analyzing the service definition draft revealed an issue.\n"
                "{}".format(analyze_issues)
            )
        else:
            # Modify Service Definition Draft file to overwrite the existing service
            sddTypeReplacement(sddraft)

            # Convert Service Definition Draft to a Service Definition file.
            ## Staging compiles all the necessary information needed to publish
            ## the GIS resource. If your data is not registered with the server,
            ## the data will be added when the Service Definition Draft file is staged.
            service_definition = os.path.join(temp_dir, locator_name+".sd")
            arcpy.StageService_server(sddraft, service_definition)

            # Upload and publish service definition file as a service
            arcpy.server.UploadServiceDefinition(service_definition, agsconn)
            return arcpy.GetMessages()
        ## end if analyze_issues
    ## end with temp_dir


def sddTypeReplacement(service_definition_draft):
    """Modifies a service definition draft file to overwrite an existing service
        instead of the defaut to create a new service.
        This function based on Esri example code for arcpy.UploadServiceDefinition_server()

        Requires module xml.dom.minidom as DOM
        Takes string input parameter of the path to a service definition draft file.

        I did test overwriting a service without this function and it seems to
        work fine. However, I kept it because Esri took the effort to post the
        code example in the documentation and there must be a good reason.
        Futher investigation may be helpful in understanding this.
    """

    xml_doc = DOM.parse(service_definition_draft)
    descriptions = xml_doc.getElementsByTagName("Type")
    for desc in descriptions:
        if desc.parentNode.tagName == "SVCManifest":
            if desc.hasChildNodes():
                desc.firstChild.data = "esriServiceDefinitionType_Replacement"
    with open(service_definition_draft, 'w') as f:
        xml_doc.writexml(f)


@contextmanager
def makeTempDir():
    """Creates a temporary folder and returns the full path name.
    Use in with statement to delete the folder and all contents on exit.
    Requires contextlib contextmanager, shutil, and tempfile modules.
    """

    temp_dir = tempfile.mkdtemp()
    try:
        yield temp_dir
    finally:
        shutil.rmtree(temp_dir)


# Script start
if __name__ == '__main__':
    main()

 

I will be also adapting this same process for rebuilding geocode services with a composite locator. As always, suggestions are welcome!

 

P.S.

When I got this vague error, I found it was caused by incorrect field names in the field info for the in_field_map parameter of arcpy.CreateAddressLocator_geocoding()

ERROR 000042: Failed to create the address locator.
Underlying DBMS error [ORA-00923: FROM keyword not found where expected]

Outcomes