AddJoin failed

[ElaineKuo]

System: Vista, ArcGIS 9.3

Hello,

I wrote a python code to AddJoin a grid shapefile (File A) to a bird range shapefile (File B).
They have a field in common (GID).

However, the error message is
ERROR 000732: Input Features: Dataset H:/temp_D/testt does not exist or is not supported
Failed to execute (MakeFeatureLayer).

Please kindly advise how to modify the code.
Thank you.
(The file location is
"H:/temp_D/testt/grid.shp"
"H:/temp_D/test/birdrange.shp")

#Import standard library modules import arcgisscripting import os  #Create the Geoprocessor object gp = arcgisscripting.create(9.3)  #Get a list of the featureclasses in the input folder source_fc = "H:/temp_D/testt" Entity = "H:/temp_D/test" outputfolder = "H:/temp_D"  gp.Toolbox = "Data Management"      # convert a featureclass to a layer  gp.MakeFeatureLayer_management (source_fc, "source_lyr")  # have Layers for AddJoin gp.AddJoin_management("source_lyr", "GID", Entity, "GID", "KEEP_ALL")  # convert a layer to a featureclass  gp.CopyFeatures_management ("source_lyr", source_fc) gp.FeatureClassToShapefile_conversion(source_fc, outputfolder)  # clear memory of layers gp.Delete("source_lyr")      gp.AddMessage(gp.GetMessages()) print gp.GetMessages()

[RichardFairhurst]

System: Vista, ArcGIS 9.3

Hello,

I wrote a python code to AddJoin a grid shapefile (File A) to a bird range shapefile (File B).
They have a field in common (GID).

However, the error message is
ERROR 000732: Input Features: Dataset H:/temp_D/testt does not exist or is not supported
Failed to execute (MakeFeatureLayer).

Please kindly advise how to modify the code.
Thank you.
(The file location is
"H:/temp_D/testt/grid.shp"
"H:/temp_D/test/birdrange.shp")

#Import standard library modules
import arcgisscripting
import os

#Create the Geoprocessor object
gp = arcgisscripting.create(9.3)

#Get a list of the featureclasses in the input folder
source_fc = "H:/temp_D/testt"
Entity = "H:/temp_D/test"
outputfolder = "H:/temp_D"

gp.Toolbox = "Data Management"
    
# convert a featureclass to a layer 
gp.MakeFeatureLayer_management (source_fc, "source_lyr")

# have Layers for AddJoin
gp.AddJoin_management("source_lyr", "GID", Entity, "GID", "KEEP_ALL")

# convert a layer to a featureclass 
gp.CopyFeatures_management ("source_lyr", source_fc)
gp.FeatureClassToShapefile_conversion(source_fc, outputfolder)

# clear memory of layers
gp.Delete("source_lyr")
    
gp.AddMessage(gp.GetMessages())
print gp.GetMessages()

The error makes it fairly apparent what the problem is.  You did not provide any actual input feature classes in your code, nor did you actually read the directories to get the feature classes (so your comment in the code is wrong).  You also must feed in paths as either raw strings or with double slashes "//".  Change this part this way:

#Provide the input shapefiles and output shapefiles
source_fc = r"H:/temp_D/testt/grid.shp"
Entity = r"H:/temp_D/test/birdrange.shp"
output_fc = r"H:/temp_D/test/grid_rng.shp"

If you get the same error still it is because these are not the actual paths or file names and you should verify that you have written them out correctly.

Your code has other errors within it.  It will also break where you try to copy a feature class over itself in Copy Features line and you don't need to run the conversion to a shape file if you output the CopyFeatures to a new shapefile in the first place.  So use the output_fc line I changed above and change the Copy Features part from:

# convert a layer to a featureclass
gp.CopyFeatures_management ("source_lyr", source_fc)
gp.FeatureClassToShapefile_conversion(source_fc, outputfolder)

to

# Copy features to a Shape File
gp.CopyFeatures_management ("source_lyr", output_fc)

You may not like the result using a shapefile, however, since joins will corrupt the field names when they truncate to 10 characters and just keep the table names.  I hate shapefiles for that reason and only use Feature Classes in file geodatabases in my models.  But with layer settings and aliases you might be able to prevent that corruption of your data despite using a shapefile.

I question the use of the Delete tool at the end, since I am not sure it actually clears memory of the layer only and won't delete the data.  It may be the correct way to do it, but I have not tried it.  You should also add a Remove Join tool in the script prior to trying to delete the layer.

Since everything in this script except the last few lines can be set up in ModelBuilder, I recommend recreating the whole model in ModelBuilder from scratch.  Complete each tool input correctly in ModelBuilder, and run the model to verify it actually works.  Make sure the output deals with field names the way you want.  Then if this tool is supposed to be rerun regularly, set it up to run a second time by adding a Delete tool to delete the output shapefile of the Copy Feature tool, or it will throw an error saying the output files already exists.  After the tool runs successfully in ModelBuilder the second time, output it as a Python script and add the last few lines.  Then it is ready to run as a batch script on a regular schedule.

[RichardFairhurst]

A few other thought on your model.  It only works if the grids match just one bird range.  It they match two or more bird ranges, a join won't handle that.  In that case you would have to convert your grid and bird range shapefiles into two feature classes within a single File geodatabase, and then use the Make Query Table tool instead to deal with the one to many or many to many relationship.  Or else consider using the Spatial Join tool instead with the One to Many option and a small negative tolerance or the One to One option and some merge rules to join up multiple bird range values into summary fields or list fields.

The ModelBuilder to script approach does not create the prettiest, compact code, but I almost always create gp tools in ModelBuilder and export them as python scripts to make sure I get the syntax and paths correct, rather than write them out directly in a python editor.  Troubleshooting a model is a lot easier than troubleshooting a python script for me.  And frankly it is a good way to learn Python if you are not that experienced in writing python code.

[ElaineKuo]

Hello,

Thanks for the clear explanation for a beginner like me.
Please kindly help again with the questions following your advice.

You also must feed in paths as either raw strings or with double slashes "//". 
=> Please kindly give an example of double slashes and explain why not using double slashes in the path below.

Change this part this way:

#Provide the input shapefiles and output shapefiles
source_fc = r"H:/temp_D/testt/grid.shp"
Entity = r"H:/temp_D/test/birdrange.shp"
output_fc = r"H:/temp_D/test/grid_rng.shp"

If you get the same error still it is because these are not the actual paths or file names.
=> The message says "ERROR 000732: Join Table: Dataset H:/temp_D/test/birdrange.shp does not exist."
Please kindly advise how to name the result shapefile if not preexist yet.


You should also add a Remove Join tool in the script prior to trying to delete the layer.
=> Thank you for reminding.

Complete each tool input correctly in ModelBuilder, and run the model to verify it actually works.  Make sure the output deals with field names the way you want.  Then if this tool is supposed to be rerun regularly, set it up to run a second time by adding a Delete tool to delete the output shapefile of the Copy Feature tool, or it will throw an error saying the output files already exists.  After the tool runs successfully in ModelBuilder the second time, output it as a Python script and add the last few lines. 
=> Please kindly advise how to output a modelbuilder as a Python script.
    Totally agreed with you on the idea of ModelBuilder.
    I used to do the batch addjoin using ModelBuilder (one grid to 100 bird ranges.)
    However, this time the working load increased (one grid to 3000 bird ranges)
    That's why I want to try pyton.

[ElaineKuo]

It they match two or more bird ranges, a join won't handle that.  In that case you would have to convert your grid and bird range shapefiles into two feature classes within a single File geodatabase, and then use the Make Query Table tool instead to deal with the one to many or many to many relationship.  Or else consider using the Spatial Join tool instead with the One to Many option and a small negative tolerance or the One to One option and some merge rules to join up multiple bird range values into summary fields or list fields.

Thank you for the important advice on File geodatabase.
It is new to me.

Please kindly
1.introduce any references on File geodatabase
2. how to run "AddJoin" for a grid and multiple bird range shapefiles in File geodatabase.
I tried Spatial Join tool. It took 13 seconds to complete a join of a grid and a bird range. (it only took one second to use AddJoin.)
Thank you.

[RichardFairhurst]

Here are my responses:

=> Please kindly give an example of double slashes and explain why not using double slashes
=> in the path below.
The slash is an escape character in python and is not read as a text character unless the entire string is preceded by an r to make it a raw string interpreted without handling escape characters or the slash is repeated to make it a literal text slash.  So only these two versions work:
source_fc = r"H:/temp_D/testt/grid.shp"
or
source_fc = "H://temp_D//testt//grid.shp"

=> The message says "ERROR 000732: Join Table: Dataset H:/temp_D/test/birdrange.shp does
=> not exist."
Probably the path is wrong and actually matches the path of the grid.  To verify the path use Windows Explorer and find the file and its path and copy that path into the code.  I expect you will find it is actually:

Entity = r"H:/temp_D/testt/birdrange.shp"

=> Please kindly advise how to output a modelbuilder as a Python script.
From this help (10.1 but it should be the same with 9.3)
Steps:
From the menu in ModelBuilder, point to and click Model > Export > To Python Script.
Click the Save in drop-down arrow and navigate to the location where you want to save your script.
Type a file name for the script.
Click Save.

=>    Totally agreed with you on the idea of ModelBuilder.
=>    I used to do the batch addjoin using ModelBuilder (one grid to 100 bird ranges.)
=>    However, this time the working load increased (one grid to 3000 bird ranges)
=>    That's why I want to try python.
If all you are doing is a normal Join and Copy Feature using gp tools, it is the same thing to use Model Builder as it is to python.  Python would only make a difference if you were using python specific code, such as cursors, loops, or python functions.  So just build it in Model Builder.  Even if it turns out you need a loop, build a single process that works first and then worry about putting it inside a loop.

If you are actually doing a join of one grid to 100 bird ranges then it sounds like maybe you are doing something other than what I am reading in your code.  Were you using an iterator in Model Builder before to process one grid against 100 bird range joins involving 100 shape files?  Right now you code only does one join, which will only join one grid to one bird range.  So what do you really intend to do here?

[ElaineKuo]

Right now you code only does one join, which will only join one grid to one bird range. So what do you really intend to do here?
=> a quick response.
just as you said, I made a try of a grid to one bird range to see if the code is basically right.
The loop did not start yet.

By the way, for model builder, I copied the whole process of AddJoin for 10 times and ran it for 10 joins simultaneously.

Thank you always for the in-time and clear explanation.

[RichardFairhurst]

Right now you code only does one join, which will only join one grid to one bird range. So what do you really intend to do here?
=> a quick response.
just as you said, I made a try of a grid to one bird range to see if the code is basically right.
The loop did not start yet.

By the way, for model builder, I copied the whole process of AddJoin for 10 times and ran it for 10 joins simultaneously.

Thank you always for the in-time and clear explanation.

OK.  Then once you get a single join going you will need a loop to read the bird ranges and do the joins before doing the copy features.  You may have to do it in chunks, since I am not sure how many simultaneous joins are allowed before you run out of memory or hit some other limit.

The Spatial Join idea was for the case where all of your bird ranges were appended together in one feature class.  Then it makes sense to do that.  With each bird range in a separate file Add Join makes sense and the Spatial join doesn't help.  Also, that make more sense why you want to try python, since a loop through the bird range directory can do the joins faster than setting up each in a string in Model Builder.  Model Builder could maybe do it with an iterator, but iterators don't export to python.  So once your code works for a single instance then come back and show it to get help building a python file reading loop.

[ElaineKuo]

Thank you for the help.
The AddJoin code below worked well.
As you said, I want to add a loop to the AddJoin of a grid and multiple bird range.
Please kindly advise how to code it.
Thank you.

################################################## ########################
##AddJoin feature layers to a feature layer
##Elaine Kuo
##30 June 2013
################################################## #######################

#Import standard library modules
import arcgisscripting
import os

#Create the Geoprocessor object
gp = arcgisscripting.create(9.3)
gp.QualifiedFieldNames = "UNQUALIFIED"

#Get a list of the featureclasses in the input folder
source_fc = r"H:/temp_D/testt/A_grid.shp"
Entity = r"H:/temp_D/test/geoc0283.shp"
output_fc = r"H:/temp_D/test/grid_rng.shp"

gp.Toolbox = "Data Management"
    
# convert a featureclass to a layer 
gp.MakeFeatureLayer_management (source_fc, "source_lyr")

# have Layers for AddJoin
gp.AddJoin_management("source_lyr", "GID", Entity, "GID", "KEEP_ALL")

# convert a layer to a featureclass 
gp.CopyFeatures_management ("source_lyr", output_fc)

# delete Field
gp.deletefield (output_fc, "FID_1; GID_1")

# Remove the join
gp.RemoveJoin("source_lyr", "geoc0283")
    
# clear memory of layers
gp.Delete("source_lyr")
    
gp.AddMessage(gp.GetMessages())
print gp.GetMessages()

[RDHarles]

Just for the record, a single forward slash ("/") works fine, it's the backslash ("\") that's illegal.

The following techniques will all work in a path:

1.) forward slashes = /
2.) double backslashes = \\
3.) raw string with backslashes = r"H:\InfoRequest"