8 Replies Latest reply on Sep 18, 2012 5:37 AM by KG22

    Anyone using the new Python Toolboxes at 10.1?

    KG22
      Just wondering if anyone has used the new Python toolboxes at 10.1?
      The .pyt
      I am experimenting with one and could use some tips.
      fyi, if you want to edit that pyt file in your favorite python editor you may have to go into the options of your editor and set it up to explicitly recognize pyt files. Mine (Wing, pyScripter) did not do that by default and there was no coloring or intellisense until i modified the program under options to read that pyt.

      I want to use pyt because at 10.0 we had such a hassle with deploying script tools and toolboxes because you always need arcatalog to setup the dang params. And they seem to get out of whack on their own over time.

      I'm hoping these python-based toolboxes don't work like that.
      One drawback is that you only get the one pyt file - so if you want to cram 20 custom tools in there you are looking at a huge file.
        • Re: Anyone using the new Python Toolboxes at 10.1?
          dwynne-esristaff
          Hi Kevin,
          If you have a few minutes, perhaps check out the training seminar on Python toolboxes that we did last week: it will walk through some of these questions you have:
          http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseID=2523

          If you're looking for additional samples just to see, check out http://esriurl.com/pyt1, http://esriurl.com/pyt2, http://esriurl.com/pyt3

          -Dave
          • Re: Anyone using the new Python Toolboxes at 10.1?
            KG22
            Thanks for those tips. I will look into the seminar and samples.
            My greatest concern now is that these examples are designed with one tool in one toolbox.
            I have over 30 tools that i would like to present as services for my applications.

            Running each one of those in ArcMap and publishing the results as a service is just not an option.
            It is too much work.
            I know the code works because i have been using the tools at 10.0
            I just want a painless way to expose many py scripts as services.
            • Re: Anyone using the new Python Toolboxes at 10.1?
              lpinner
              I'm still using 10.0 so can't actually test this, but have you tried using the standard python package structure?

              Something like:

              #  \--SomeDir
              #     |  toolbox.pyt
              #     \--toolpackage
              #        |  __init__.py
              #        |  script_a.py
              #        |  script_b.py
              
              
              #----------------------------
              #The .pyt file
              #----------------------------
              
              import arcpy
              from toolpackage.script_a import Tool1
              from toolpackage.script_a import Tool2
              from toolpackage.script_b import Tool3
              
              class Toolbox(object):
                  def __init__(self):
                      """Define the toolbox (the name of the toolbox is the name of the
                      .pyt file)."""
                      self.label = "MultiTool_Toolbox"
                      self.alias = "Toolbox with Multiple Tools"
              
                      # List of tool classes associated with this toolbox
                      self.tools = [Tool1, Tool2, Tool3]
              
              
              • Re: Anyone using the new Python Toolboxes at 10.1?
                KG22
                I think i get what you are saying.
                In my master pyt python toolbox file i just have the definitions of the tools and the params for each.
                Each tool has an execute method and in that method i will just place a link to the actual py script file that i already have from pre-10.1 days. I think that will work, but i may have to add a path to the environ variable so python knows where to look for my scripts.

                Hopefully this will let me keep my 10.0 py scripts intact and i can better manage the params using the 10.1 pyt structure.

                One thing about 10.1 - we are spending alot of time figuring out where ags puts things during a publish and then trying to rig things to work better than that.

                thanks for the tip.
                • Re: Anyone using the new Python Toolboxes at 10.1?
                  lpinner
                  If you didn't want to modify your existing script tools to convert them to classes, then you could do that (calling your scripts from the execute method of each Tool class). I was thinking more of just converting the scripts to classes though, especially if you're writing stubs for those classes anyway.

                  How you could do it depends on how you've written your scripts.  If your existing 10.0 scripts are basic scripts where all the code is in the top-level scope, then in the execute method of the new Tool class you could use the subprocess module to call python and pass it the path to the 10.0 script.  If your 10.0 scripts are organised into functions, you might be able to do something like:

                  #----------------
                  # 10.0 script tool
                  # script_10_0.py
                  #----------------
                  import arcpy
                  def do_some_stuff(arg1,arg2):
                      print arg1,arg2
                  
                  if __name__ == '__main__':
                      arg1=arcpy.GetParameterAsText(0)
                      arg2=arcpy.GetParameterAsText(1)
                      do_some_stuff(arg1,arg2)
                  
                  #----------------------
                  # 10.1 Python Toolbox
                  #----------------------
                  import arcpy
                  import script_10_0 #import the old 10.0 script tool so you can call its functions.
                  
                  class Toolbox(object):
                      def __init__(self):
                          """Define the toolbox (the name of the toolbox is the name of the
                          .pyt file)."""
                          self.label = "10_0_Tool_Toolbox"
                          self.alias = "Toolbox with 10.0 Tools"
                  
                          # List of tool classes associated with this toolbox
                          self.tools = [Tool1]
                  
                  class Tool1(object):
                      #...
                      def execute(self, parameters, messages):
                          arg1=parameters[0].valueAsText
                          arg2=parameters[1].valueAsText
                          script_10_0.do_some_stuff(arg1,arg2)
                  
                  • Re: Anyone using the new Python Toolboxes at 10.1?
                    KG22
                    that is pretty much what i did. For each individual py script i had i changed the signature and made a main def.
                    then in the pyt i import each of those, then call the script by name.main.
                    i like this because each of my scripts was pulling in a params json file. now i can pull that params file into the pyt once and dish it off to each side script.

                    All that remains is for me to find a way around esri wanting me to run each script in arcmap and publish the results.
                    if i could find a way to just publish the pyt once then throw the extra scripts in there and have them recognized that would be the bomb
                    • Re: Anyone using the new Python Toolboxes at 10.1?
                      lpinner
                      if i could find a way to just publish the pyt once then throw the extra scripts in there and have them recognized that would be the bomb


                      Something like the following might work:
                      #  \--SomeDir
                      #     |  toolbox.pyt
                      #     |  some_script.py
                      #     |  another_script.py
                      #     |  ...
                      #     |  last_script.py
                      #
                      # Each *.py contains a class called Tool that's just a stub for your "main" def.
                      
                      #----------------------------
                      #The .pyt file (all Tools are dynamically imported not hardcoded)
                      #----------------------------
                      
                      import arcpy
                      import os,sys,glob
                      
                      class Toolbox(object):
                          def __init__(self):
                              """Define the toolbox (the name of the toolbox is the name of the .pyt file)."""
                              self.label = "MultiTool_Toolbox"
                              self.alias = "Toolbox with Multiple Dynamically Imported Tools"
                      
                              # Dynamic list of tool classes associated with this toolbox
                              path=os.path.dirname(__file__)
                              self.tools=[]
                              for py in glob.glob(os.path.join(path,'*.py')):
                                  if not __file__ in py:
                                      module=os.path.basename(py)[:-3]
                                      #Note: "Tool" class must have same 
                                      #name as the script "Tool" classes
                                      self.tools.append(__import__(module).Tool)
                      
                      #----------------------------
                      #The .py script files
                      #----------------------------
                      
                      #"Tool" class can be called anything but must be 
                      # the same in each script and in the calling toolbox
                      
                      class Tool(object):
                      
                          # tool/parameter setup methods etc...
                      
                          def execute(self,*args,**kwargs):
                              main(*args,**kwargs)
                      
                      def main(*args,**kwargs):
                          #do stuff
                          return
                      
                      if __name__=='__main__':
                          #Run script
                          main(sys.argv[1:])
                      
                      • Re: Anyone using the new Python Toolboxes at 10.1?
                        KG22
                        Still not where i want to be on this one unfortunately.
                        I can get all the py and pyt files in order but that still doesn't get me around having to use ArcMap to create the sd file.
                        And when that sd file gets published all of the sudden there are lots of new files and folders and my code is being modified by esri hand-holding.

                        When i publish the sd esri's convoluted directory structure makes this:

                        -GPServiceName.GPServer
                          -extracted
                          .GPServiceName.sd

                        then in the extracted folder we have:
                        -esriinfo
                        -v101
                        .manifest.xml
                        .serviceconfiguration.json
                        tilingservice.xml

                        under esriinfo we have more:
                        -metadata
                        -thumbnail
                        .iteminfo.xml

                        thumbnail is just an image, the metadata folder has a file named metadata.xml

                        under v101 we have:
                        afoldernamedafterwhereyouputyourscripts
                        GPServiceName.tbx
                        GPServiceName.rlt

                        the tbx and rlt are binary
                        under the folder that esri creates based on your script folder you have:
                        all of your py files, pyc versions for each, a param text file that i was using and esri dragged in, along with any 'helper' py files
                        the pyt file for your original python toolbox
                        a pyt.xml for the toolbox
                        a Tool.pyt.xml for the toolbox
                        pyt.xml files for each Tool in the toolbox

                        Python toolboxes provide the opportunity to take advantage of your Python skills and create tools entirely and easily out of Python


                        perhaps. but getting them published is a different story.

                        Say i want to add a Tool to my currently published GPService? Not fun at all. I have to open my pyt, edit it, open it in ArcMap, run EVERY OTHER TOOL including my new tool and then publish?

                        Certainly i'm missing some very important in this workflow.

                        At ArcGIS 9.0, the script tool framework was first introduced and is geared towards creating Python-based tools for new users, but it is an inefficient process for more experienced users. In the script tool framework:
                        You define parameters through the wizard
                        You create validation code that lives in the toolbox
                        Plus you create and maintain the source script separately
                        All of these parts are segregated and more difficult to manage collectively.


                        That workflow is starting to look a lot better in hindsight.