Trouble with Change Version (Data Management) to Historical Marker in Python Script

3557
4
09-19-2013 11:20 AM
AndrewStauffer1
New Contributor II
The problem: I'm having some problems with the Change Version (Data Management) tool.  I'm trying to change the version of my SDE database to a Historical Marker/Timestamp within a python script.  However, the version only changes from sde.DEFAULT to the current time today, instead of the input date and time.  I'm able to successfully change the TRANSACTIONAL version to what I specify in the script, but the HISTORICAL option seems to be a bit buggy.

The gp tool works fine for both TRANSACTIONAL and HISTORICAL inputs, but I need to embed this tool into my custom script.  This leads me to believe that I might be doing something incorrect in my python code.

Has anyone had any troubles with this before or know of a solution to the problem I'm having?  Any help would be much appreciated!

My Computer Setup: I'm running ArcInfo 10.1 on Windows 7.  My SDE is a PostgreSQL Enterprise Geodatabase.  I'm currently working with a test dataset of 100 points that I created two days ago.  This morning, I deleted several of the points in the dataset.

My Code Snipit:
import arcpy, datetime
indata = arcpy.GetParameter(0)   #A feature layer input from the sde.DEFAULT
indate = arcpy.GetParameter(1)   #A date type input parameter (type = datetime)

arcpy.AddMessage(type(indate))#returns datetime.datetime

arcpy.AddMessage("Current Data Count: "+str(arcpy.GetCount_management(indata))) #Correctly returns 75

arcpy.ChangeVersion_management(indata, "HISTORICAL", "", indate)
arcpy.AddMessage("Old Data Count: "+str(arcpy.GetCount_management(indata)))#Returns 75, but should be returning 100
0 Kudos
4 Replies
AndrewStauffer1
New Contributor II
I'm still finding this problem somewhat buggy, but I found a rough work around solution that I would like to share.

The first thing, I've noticed that the my validation using the Get Count tool doesn't seem to work with SDE versions.  The only result I get back is the count from the sde.DEFAULT version.  I've implemented a small workaround to this in my code using a SearchCursor:

import arcpy
indata = arcpy.GetParameter(0)#Some versioned data in ArcMap (say referencing my point dataset with 100 points).

arcpy.AddMessage("getCount gp tool count: "+str(arcpy.GetCount_management(indata)))
     #This returns a value of 75 (the number of points in sde.DEFAULT)

i = 0
with arcpy.da.SearchCursor(indata, ["OBJECTID"]) as cursor:
    for row in cursor:
        i+=1
arcpy.AddMessage("search cursor count: "+str(i))
     #This returns a value of 100 (the number of points in the versioned dataset)

This work around is a bit clunky, but allows me to validate the data properly.  In the below code, I will state that I'm doing the GetCount_management tool, but in fact I'm using the search cursor.  This will save a few lines.

Solution 1: Using the changeVersion_management tool.
So, as a refresher, my original object was to input the sde.DEFAULT dataset and a date into my script tool.  I then wanted to change the version to a Historical Marker of the input date and return the number of features.  However, whenever this would happen, the SDE version in ArcMap would change from sde.DEFAULT to today's date and time instead of what was input.

The solution I've discovered was that the changeVersion_management tool was able to successfully change between Transactional Versions.  Thus, I "reversed" the order of the script and I was able to get appropriate results.

import arcpy
indata = arcpy.GetParameter(0)#A historical SDE version from two days ago

arcpy.AddMessage("Versioned data count: "+str(arcpy.GetCount_management(indata)))
     #Correctly returns 100 points

arcpy.ChangeVersion_management(indata, "TRANSACTIONAL", "sde.DEFAULT", '#')

arcpy.AddMessage("Default version data count: "+str(arcpy.GetCount_management(indata)))
     #Correctly returns 75 points

By having the user input the versioned data first, then changing the version to DEFAULT seems to work.  However, this is very clunky and not user friendly, especially if the users isn't familiar with changing SDE versions.

Solution 2: The other solution I've found involves manipulating the Historical Archive Layer (in ArcCatalog, right click on the layer, select properties, on the general tab, see the Archiving box at the bottom).  This layer contains 2 time stamped fields that are updated when the data is added, deleted, or edited.  I'm not 100% familiar yet with how this table works, when time stamps are updated, or when new records in the table are added.  I'm also not sure if the Historical Markers query this table or not.  By querying the date fields, I can access what I'm interested in (for now).  This allows me to input the sde.DEFAULT data layer and a date, as I intended to do in the first place.

import arcpy
arcpy.env.workspace = "blablabla"#my sde connection file location

indata = arcpy.GetParameter(0)#My sde.DEFAULT data layer
indate = arcpy.GetParameterAsText(1)#Using the date data type input (format: MM/DD/YYYY HH:MM:SS AM)

sde_olddata = indata.datasetName+"_H" #access the archive layer.  does it always end with "_H"?  Mine appear to...
if arcpy.Exists(sde_olddata):#Make sure it exists
    arcpy.AddMessage("the archive version exists!")
else:
    arcpy.AddMessage("your data isn't archive enabled!")

#Convert the date to the correct format for SQL queries: (YYYY-MM-DD)
if " " in indate:
    year = int(indate[indate.rfind("/")+1:indate.find(" ")])
else:
    year = int(indate[indate.rfind("/")+1:])
day = int(indate[indate.find("/")+1:indate.rfind("/")])
month = int(indate[:indate.find("/")])
tstamp = str(year)+"-"+str(month)+"-"+str(day)

#Query the archived layer based on the input date
olddata = "in_memory\\oldData"
arcpy.Select_analysis(sde_olddata, olddata, "gdb_to_date >= timestamp '"+tstamp+"' AND gdb_from_date <= timestamp '"+tstamp+"'")

#Count up the number of rows
arcpy.AddMessage("Versioned data count: "+str(arcpy.GetCount_management(olddata)))
     #Correctly returns 100 points

arcpy.AddMessage("Default version data count: "+str(arcpy.GetCount_management(indata)))
     #Correctly returns 75 points


Summary: So, this second method seems to solve my initial problem in the best way.  It is a clunky workaround (a far cry from the one-line command to change the SDE version), but gets the job done.  For now this works, but may have to be refined in the future pending several things:
1. How is the archived data layer maintained when adds/deletes/updates are performed?
2. Do Historical Markers truly work by querying the timestamp fields of this layer?
3. Do all archived layers end with "_H"
4. Is the functionality of Historical Markers improved in ArcGIS 10.2?

I hope this helps other who may be in my same situation.  Also, if you have any other feedback/information on this problem, please let me know!!
0 Kudos
MaximeDemers
Occasional Contributor III

Hi Andrew,

Thanks for your sharings with this issue. Do you encounter the same buggy behaviour with ArcGIS 10.3 and above?

Also, do you have an idea how to create a Historical Marker with arcpy?

Thank you and have a good day!

0 Kudos
ColinLang1
New Contributor III

I have just found the same behaviour in 10.2  I don't have 10.3 or higher to test with.  One thing I did notice - If I select the tool in the ArcToolbox -> Data Management Tools -> Versions -> Change Version, and I enter all the information for the historical version, it changes correctly.  Then I go into the Geoprocessing results, and I right click on the successful action, and Copy as Python Snippet, and I paste that into the python window, and run it again, I immediately get the incorrect behaviour of it changing to the current date and time rather than the specified date and time.  So the Python version of the command fails, but the GUI version works.  This is extremely frustrating when I am trying to write code to create a transmittal file that documents all the daily or monthly data changes and sends it to our data partners.

0 Kudos
ColinLang1
New Contributor III

Tested today in 10.5 and this is still a bug.

0 Kudos