Alternatives to using SDE command line tools - Blog Discussion

58517
131
10-21-2013 07:40 AM
ChetDobbins
Esri Contributor
With the release of 10.2 and plans to deprecate the ArcSDE command line tools, you may be wondering how current tasks that use these tools can be completed elsewhere. This blog provides some workflows that have alternate user interface tools in ArcCatalog/ArcMap that will make transitioning as seamless as possible.

http://blogs.esri.com/esri/supportcenter/2013/10/04/do-this-not-that-alternatives-to-using-sde-comma...

We are very interested in hearing feedback from everyone who uses the ArcSDE commands, including questions, concerns, and ideas for making this successful. You can also contact Esri Support Services for specific ArcSDE commands that do not have comparable replacements.

This document outlines the planned changes in platform and functionality in the ArcGIS 10.2 release and
includes a reference to ArcGIS 10.1 deprecation notes.
http://downloads2.esri.com/support/TechArticles/W26496_W25918_DEPRECATION_PLAN_FOR_ARCGIS_101_and_10...
131 Replies
MelissaJarman
Esri Contributor

Regarding the .sdx SDEexport files - Below is an enhancement for an equivalent arcpy function or tool that will will be considered for a future release. If you contact support services with a good example of why these export files perform more optimally than other data transfer methods (XML workspace documents, etc) we can take a look at these cases and append them to these enhancements for justification. We will need to have a good working example (clear steps to reproduce) of how the SDEEXPORT files are more efficient other methods of data transfer.

NIM095122 - SDEEXPORT/SDEIMPORT: Create geoprocessing tools or arcpy functions for operations performed when using the 'sdeexport/sdeimport' commands.

0 Kudos
Maartende_Jong
New Contributor II
There are currently two ideas about creating spatial views on ideas.arcgis.com:

Create SDE spatial view from ArcGIS Desktop
http://ideas.arcgis.com/ideaView?id=08730000000884aAAA

Create Views In ArcCatalog Like In SDE Command Line
http://ideas.arcgis.com/ideaView?id=087E00000005A9tIAE

The first one is marked as implemented by the Create Database View tool, which I object against, because (as other contributors above have mentioned) the result is not nearly the same as the sdetable -o create_view.

The second one is the request for a replacement for sdetable -o create_view. Please promote the idea if this is something you also run into.

A lot of improvements on the geodatabase side have been made in the lastest ArcGIS releases like renaming and modifying columns and making the sde command line tools available via tools. The possibility of creating proper spatial views however, still needs attention before deprecating the current tools.
DanMcCoy
Occasional Contributor III
0 Kudos
MarcoBoeringa
MVP Regular Contributor
A lot of improvements on the geodatabase side have been made in the lastest ArcGIS releases like renaming and modifying columns and making the sde command line tools available via tools. The possibility of creating proper spatial views however, still needs attention before deprecating the current tools.


It would be more correct to replace the highlighted text with:

"A lot of improvements on the geodatabase side have been made in the lastest ArcGIS releases like renaming and modifying columns and adding (geoprocessing) tools that can substitute functionality of the sde command line tools."

The new tools do not rely on the sde command line tools for their implementation in ArcGIS.
0 Kudos
JoshuaBixby
MVP Esteemed Contributor
'sdemon' certainly did require SDE user access rights, it just passed those access
rights to any connection on the port (which could be construed as a security hole).

- V


I am not sure playing the security card solicits the intended response for most people with this issue.  I know our IT security folks won't feel better knowing Esri viewed the functionality of SDEMON for listing users as weak security, possibly a security hole, and yet never addressed the issue over years and many versions of the software.  I felt better about it when I thought it was just incomplete functionality.
0 Kudos
BoGuo
by
New Contributor II
There are three important advantages that the SDE-command-line tools offer, beyond the much talked about rich functional capabilities they have:

1) Easily scriptable for automation
2) Run on server with only SDE binary is loaded and without consuming a license (free)
3) Run all platforms

SDE tools are mainly used by admins, most of whom, myself being one, would much rather script for consistency and performance than use GUI for the same tasks.

In perspective, a Windows user (not a programmer) can do most* any tasks and much more using Windows GUI, and yet DOS shell is supported and used by many.
 
Please, please, please do not ever dump SDE command line tools until equally-capable tools that Admins could use are available.
0 Kudos
by Anonymous User
Not applicable
sdelayer -o si_stats is indispensible for maintaining some feature classes, and no equivalent is available in desktop at the current version. The GDBT spatial index examination tool at 10.0 sort of works, but command line works better and faster.

Today I had to use sdelayer -o si_stats to examine and redesign the spatial indexes for statewide parcels. Before change, took four minutes to draw at a reasonably large scale. After change, 6 seconds or less. The before-change spatial index was the one "estimated" automatically by ArcGIS desktop when the data were loaded. Take away this command line tool and our ability to provide well-performing data will be crippled.
BetsySchenck-Gardner
Occasional Contributor
We specifically use the sde command-line scripting to handle loading data into sde from non-Windows platforms such as Linux systems. All in all this move away from sde command line is not very helpful for those of us that are just trying to automate loading of shapefiles into SDE/geodatabase in a shell scripting environment on a Linux platform.
0 Kudos
ForrestJones
Esri Contributor
We specifically use the sde command-line scripting to handle loading data into sde from non-Windows platforms such as Linux systems. All in all this move away from sde command line is not very helpful for those of us that are just trying to automate loading of shapefiles into SDE/geodatabase in a shell scripting environment on a Linux platform.


Hi Betsy,

What about using python and Engine or Server on Linux?

This is from the geprocessing tools help:

FeatureClassToGeodatabase example 2 (stand-alone script)
The following stand-alone script demonstrates how to use the FeatureClassToGeodatabase function.


# Name: FeatureClassToGeodatabase_Example2.py
# Description: Use FeatureClassToGeodatabase to copy feature classes
#  to geodatabase format
 
# Import system modules
import arcpy
from arcpy import env
 
# Set environment settings
env.workspace = "C:/data"
 
# Set local variables
inFeatures = ["climate.shp", "majorrds.shp"]
outLocation = "C:/output/output.gdb"
 
# Execute TableToGeodatabase
arcpy.FeatureClassToGeodatabase_conversion(inFeatures, outLocation)



To take this a step further here's an example (pseudo code) that uses a folder location to load data to an enterprise geodatabase.

Pseudo code:

# Import system modules
import arcpy
from arcpy import env

# Process: Create Database Connection File...
# Usage:  out_file_location, out_file_name, DBMS_TYPE, instance, database, account_authentication, username, password, save_username_password(must be true)

arcpy.CreateDatabaseConnection_management("/net/shared/location/connections/",
                                          "zion.sde",
                                          "ORACLE",
                                          "zionserver/ORCL")

# Set environment settings
# Set the workspace to the specified data location
env.workspace = "/net/sharedata/location/data"

outLocation = "Database Connections\zion.sde"

# Get all the feature classes in the environment
# list of fc's should be similar to this: ["accident.shp", "veg.shp"]
inFeatures = arcpy.ListFeatureClasses()

# Process: Load Data (Use the "Feature Class To Geodatabase" Tool...)
try:
   # Execute FeatureClassToGeodatabase
   print "Loading data...\n"
   arcpy.FeatureClassToGeodatabase_conversion(inFeatures, outLocation)

except:
   for i in range(arcpy.GetMessageCount()):
       arcpy.AddReturnMessage(i)



Finally to take this yet another step further, there is a script available that works on Linux and Windows and uses the above concepts to create a database connection file and then load data from a specified directory to the enterprise geodatabase.

It also uses some code from the Upgrade Geodatabase (Data Management) help documentation for parsing the sde connection parameters (similar feel to the sde command line tools).

Full script Usage:
Examples:

(Linux)
/tmp>loaddata_sde_params.py --DBMS ORACLE -i myserver/orcl -u user1 -p user1 --dataloc /net/sharedata/location/data

(Windows)
c:\tmp>loaddata_sde_params.py --DBMS ORACLE -i myserver/orcl -u user1 -p user1 --dataloc \\sharedata\location\data


>loaddata_sde_params.py --help

Usage: loaddata_sde_params.py [Options]

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --DBMS=DATABASE_TYPE  Type of enterprise DBMS: SQLSERVER, ORACLE, or
                        POSTGRESQL.
  -i INSTANCE           DBMS instance name
  --auth=ACCOUNT_AUTHENTICATION
                        Authentication type options (case-sensitive):
                        DATABASE_AUTH, OPERATING_SYSTEM_AUTH.
                        Default=DATABASE_AUTH
  -u USER               user name
  -p PASSWORD           password
  --dataloc=DATALOCATION
                        Path to the data (either a geodatabase or a directory
                        with shapefiles
  -D DATABASE           Database name (Not required for Oracle)



The full working sample script and code is available here

Hope this helps.
0 Kudos
NickHarvey
Occasional Contributor II
This one has my attention (list item #2).  We have a local partnering agency that dutifully provides/supports us with a 24-hr update of critical data, however it is provided in the form of sde export files (.exp) because they're fast & efficient and the process was well established.  The partnering agency may not move away from 10.1 for some time (3rd party apps), and I'd rather not ask for a process change until they upgrade.  Is there any Arcpy functionality for importing .exp's?

2)            Sdeexport / sdeimport:  should we continue to use this as a way to transfer data?

Nick Harvey, GISP
GIS Analyst - MSD of Buncombe County, NC
nharvey@msdbc.org
0 Kudos