Repetitive task: model builder or python

4217
26
Jump to solution
05-26-2017 12:36 PM
CelesteWalz
Occasional Contributor

I have many gpx files that I need to get into a mapping project.

The workflow looks like this:

1.  Open gpx file in ascii editor to fix the last elevation. Every file from this vendor has the last elevation as -10000000 which is of course incorrect.

2.  Use the 'gpx to points' tool to convert to a feature class

3.  Use the 'points to line' tool to convert to line

I've only played a bit in Model Builder, but this looks like something that could be done. Not sure how model builder works with just gpx files as input - my first tries haven't been successful (but then again, I'm kinda shooting in the dark here).  I don't know python, but I have done snippets of codes for other projects, so maybe that is the way to go?

Any suggestions, or a good place to start?

I'll be going back through the model builder help to see what I may have missed but any other input would be most appreciated.

EDIT: I managed to get this to work; I've attached a zip file of the toolbox. If anybody is interested, these files were from an app that I use to record my travels; the resulting lines went in to building this StoryMap: http://arcg.is/2sbFG27  I used the model/tool to process 50+ 'treks'.

0 Kudos
1 Solution

Accepted Solutions
curtvprice
MVP Esteemed Contributor

Assuming all your files look exactly like that one, I tweaked the Calculate Value script to process your files, when applied to your test file, the python edit of the file seems to work fine. This avoids the step of the ascii editing. Once you get it working for one, add an iterator to your model and you should be good to go!

View solution in original post

26 Replies
curtvprice
MVP Esteemed Contributor

The editing tasks will require some Python, but the rest would work fine.  I think this is a great example of the kind of thing Model Builder does well, a simple batch operation.

The editing could be done using Calculate Value with something like waht is below. Pipe the Calcualte Value output as input to the next two tools in the processing chain (GPX to Feature, Point To Line) and see if you can get it work. Once you get that going, add the iterator (Iterate Files), and don't forget to use variables to name the output so it won't just overwrite one output dataset over and over again.

Expression

fix(r"%GPX file%")‍‍‍‍‍‍‍‍

Code Block

import os
import arcpy
import os
import arcpy
tmpout = os.path.join(arcpy.env.scratchFolder, "xxtemp.gpx")
def fix(gpx_file):
  with open(gpx_file, "r") as f:
    data = f.readlines()
  data = data[:-7] + data[-3:] # delete last data point
  with open(tmpout, "w") as f:
    f.writelines(data)
  return tmpout‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Output data type

File

UPDATE - I took a look at your GPX file, which is an XML document. This would complicate the Python a bit -- it may be easiest to skip the Calculate Value python bit, and simply import the fake -1000000 point, and use the Select tool in your processing chaing to remove your closing point from your dataset.

UPDATE 2 - I tried the data file and it won't import with the fake last point, so I updated the script above to delete the 7th from the last line to the 4th from the last line:

        <ele>-8.924316406</ele>
        <time>2017-05-02T12:44:10UTC</time>
      </trkpt>
      <trkpt lat="57.70792238770994" lon="11.970914599637863"> ## -7
        <ele>-1000000.0</ele>                                  ## -6
        <time>2017-05-02T12:44:15UTC</time>                    ## -5
      </trkpt>                                                 ## -4  
    </trkseg>
  </trk>
</gpx>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
curtvprice
MVP Esteemed Contributor

Assuming all your files look exactly like that one, I tweaked the Calculate Value script to process your files, when applied to your test file, the python edit of the file seems to work fine. This avoids the step of the ascii editing. Once you get it working for one, add an iterator to your model and you should be good to go!

ChrisDonohue__GISP
MVP Alum

In general it is possible to do this with Modelbuilder.  I say in general as I'm not familiar with GPX files and "gpx to points" tool you mention. 

To help in providing a more defined answer, can you describe which software platform you are using?  ArcGIS Desktop, ArcGIS Pro, ArcGIS Online, other?  Also, what version?  Are third-party tools being employed or is the current process all with ESRI software?

Some ideas:

Note that if the "gpx to points" tool that is mentioned is from a non-ESRI program, it still may be possible to run it using Modelbuilder.  The following link has an example using "R", but other programs can be run ("called") in a similar matter.

Integrating external programs within ModelBuilder—Help | ArcGIS for Desktop 

In terms of automating the processing of the full dataset, at least one Iterator will be needed:

A quick tour of using iterators—Help | ArcGIS for Desktop 

In the project you have described, it may actually need two Iterators - one to loop through (pick) a GPX file and a second to loop through the selected GPX files records.  This is a case where one typically would take a Submodel approach - one model to pick the GPX to work on from your collection, and another model to process the GPX file.  This submodel approach will need to be done as it doesn't work out to have more than one Iterator in a model.

Integrating a model within a model—Help | ArcGIS for Desktop 

Chris Donohue, GISP

Note - edited for improved clarity

CelesteWalz
Occasional Contributor

Thank you!

Answers to your questions:

I'm using ArcGIS for Desktop 10.4.1. No third-party tools. 'GPX to Points' is in a toolbox in ArcMap.

I'll have a look at the links you sent along - I'll probably have more questions next week, but this gives more lots of info to start with. Thanks again! 

0 Kudos
curtvprice
MVP Esteemed Contributor

The tool is GPX To Features.

GPX To Features—Help | ArcGIS Desktop 

CelesteWalz
Occasional Contributor

Yep - I used the wrong name. 

0 Kudos
CelesteWalz
Occasional Contributor

I got it to work: the python script you provided, the iterator and the processes. A very successful model for my use. Thanks again!

DanPatterson_Retired
MVP Emeritus

You might want to mark Curtis' answer correct so that resolution is noted on this thread

0 Kudos
curtvprice
MVP Esteemed Contributor

Hey Celeste, could you post this model (tbx file) to this thread?  I'm curious how you put it together. Sometimes Calculate Value and iterators can be tricky.

0 Kudos