71 Replies Latest reply on Mar 26, 2015 12:44 PM by kyle969

    Why would I install ArcSDE with 10.1?

    alantoms
      Hello all,
      Why would I install ArcSDE with 10.1?  It seems I can us ArcCatalog to create an enterprise geodatabase without SDE then have all my users use direct connections.  What am I missing?

      Thank you
      Alan
        • Re: Why would I install ArcSDE with 10.1?
          jskinner-esristaff
          Hi Alan,

          With ArcGIS 10.1 you can connect to an enterprise database that does not have an SDE geodatabase repository and create feature classes with a native spatial type.  For example, you could connect to a SQL Server database and import/create feature classes with the GEOMETRY or GEOGRAPHY spatial type.  However, you can only view this feature class.  You cannot perform any edits, or have the feature class participate in any geodatabase functionality (such as replication, topology, geometric networks, relationship classes, network datasets, etc).  Essentially, you are getting a 'view' only feature class w/o SDE.
          • Re: Why would I install ArcSDE with 10.1?
            rbrennan-esristaff
            Alan,

            To further add on to what Jake said...

            Prior to 10.1 (10.0, 9.x, 8.x) if you wanted to use a Geodatabase with all of the cool Geodatabase functionality (versioning, archiving, topology, relationship classes, terrains, geometric networks, etc) on a DBMS (Oracle, SQL Server, Postgresql, Informix, DB2) you had to perform a few steps.

            First, you needed to install ArcSDE onto your machine, usually this was your DBMS server. This install included the files needed to run ArcSDE command line utilities, the application server, as well as the ArcSDE post installer.

            After installing this onto your machine you would need to run the ArcSDE post installer. The main purpose of this post install was to install the Geodatabase into your enterprise database. This includes all the stored procedures, functions, privileges, and schema needed to provide the functionality I mentioned earlier. The post install could also be used to set up your application server.

            The application server can be used to connect from a client machine to the DBMS/Geodatabase. It is used more or less to manage the connection requests coming in from clients and provide a way for the clients to 'talk' to the DBMS. For a while this was the only way to connect to a Geodatabase. At some point (I think 9.0) we added the ability to make direct connections (2-tier: client-DBMS) to the DBMS. This made the use of the application server (3-tier: client-app server-dbms) optional. For some people this meant they stopped setting up the application server and started using direct connections. Others continued using the application server.

            The ArcSDE command line utilities are a method for the Geodatabase administrator to manage data, users, the application server service.

            Fast forward to 10.1 - We have tried to allow you to manage your Geodatabase completely within ArcGIS applications (ArcCatalog, ArcMap, ArcGIS Server, etc). This is done through the use of dialogues in ArcMap/Catalog and the use of geoprocessing. The first thing we did was to break out the installation of the geodatabase schema tasks into geoprocessing tools. If you want to create an enterprise geodatabase there are now two options. Option 1, you can use the 'Create Enterprise Geodatabase' geoprocessing tool. This tool will create a new empty geodatabase in an existing instance. The second option is to use the 'Enable Enterprise Geodatabase' tool will allow you to install the Geodatabase schema in an already existing instance. The new functionality that Jake mentioned that allows you to now connect to a enterprise database (not a geodatabase) is what allow ArcGIS to then enable geodatabase behavior in your enterprise geodatabase. This second option would be used where you have already set up a database, have user permissions assigned and maybe have loaded some data (essentially converting your database to a geodatabase). The first option would be used if you are starting from nothing.

            Esri recommends using direct connections for making connections to your geodatabase, it is not mandatory that you install the application server.

            Much of the commonly used functionality found in the ArcSDE command line utilities is now available either through ArcGIS applications mentioned earlier or through geoprocessing (disconnecting users, identifying locks, loading data, investigating data, etc). For most users the install of these utilities should not be necessary.

            If you determine that you really need the application server or the command line utilities they are available as a separate install.

            So, this has been a pretty long winded answer to a pretty straightforward question. Answer is, it's not mandatory to install the application server or command line utilities. If you want to take advantage of Geodatabase behavior you do need to run one of the geoprocessing tools to install the Geodatabase.
            2 of 2 people found this helpful
            • Re: Why would I install ArcSDE with 10.1?
              alantoms
              Thank you very much for the education.

              Alan
              • Re: Why would I install ArcSDE with 10.1?
                sshields-esristaff
                Hi Alan,

                at 10.1 there is no need to install the ArcSDE software unless you need to run an ArcSDE service. If all of your users are making Direct Connections to the geodatabase then the ArcSDE installation is not necessary. As well, most of the functionality offered by ArcSDE commands is now available in ArcGIS Desktop & through GP tools.

                -Shannon
                1 of 1 people found this helpful
                • Re: Why would I install ArcSDE with 10.1?
                  lucasimone
                  the answer from russellb is really clear, but I have another little question:

                  what about editing?

                  I have some services that I can edit on the web with ArcGIS Server 10.1, should I still use the SDE connection or it is ok to use a direct Connection?
                  Is the new ArcGIS Server able to 'replace' or acts as the SDE layer?
                  • Re: Why would I install ArcSDE with 10.1?
                    vangelo-esristaff
                    Direct Connect *is* an ArcSDE connection.  This thread is about application server install,
                    but the ArcSDE functionality still exists -- It's ArcSDE that provides the basis for versioned
                    geodatabases.  With Direct Connect, it's just a multi-threaded solution in the ArcGIS client
                    instead of a multi-processing solution on the database server.

                    - V
                    • Re: Why would I install ArcSDE with 10.1?
                      61179
                      You cannot perform any edits, or have the feature class participate in any geodatabase functionality (such as replication, topology, geometric networks, relationship classes, network datasets, etc). Essentially, you are getting a 'view' only feature class w/o SDE.


                      Is the qouted text true if I don't install ArcSDE 10.1?
                      • Re: Why would I install ArcSDE with 10.1?
                        mboeringa2010
                        Is the qouted text true if I don't install ArcSDE 10.1?


                        No, it's true, like Russell Brennan wrote, if you don't Create or Enable Enterprise Geodatabase functionality on your database using the new geoprocessing tools in ArcGIS 10.1.

                        It has nothing to do with whether or not ArcSDE is installed on your database server. As Vince says, even if you don't use or install it on your database server, ArcSDE is still part of your client ArcGIS application, there are by default DLLs installed on your own computer with ArcGIS that handle the geodatabase SQL logic ---> That is essentially what ArcSDE is!

                        E.g., have a look at your "C:\Program Files (x86)\ArcGIS\Desktop10.X\Bin" folder on your local computer. You will see DLLs like "sde.dll" and "sdesqlsrvr100.dll" etc.

                        As soon as you attempt to connect to an ESRI Enterprise Geodatabase, these DLLs will be in use by your client ArcGIS application. There is no way around this when connecting to an ESRI geodatabase - at least for full functionality including editing - ArcSDE is just the component ESRI devised to handle the connection to the database and SQL stuff needed to allow advanced geodatabase functionality like versioning.

                        There is nothing special about ArcSDE or these DLLs in this respect, other vendors like Bentley or Autodesk have similar software components in their software to handle connections and SQL stuff related to geospatial databases, they just call it differently.
                        • Re: Why would I install ArcSDE with 10.1?
                          61179
                          Just to make sure that I understood you correctly.
                          What you are saying is I do not need to install ArcSDE on the database server because the client contains the SDE functionality.
                          • Re: Why would I install ArcSDE with 10.1?
                            61179
                            Another way of sying it:

                            - I could use direct connection without ArcSDE installed for two-tier topology.
                            - I could use ArcSDE to hide or conceal the database in a three-tier topology.

                            am I right?
                            • Re: Why would I install ArcSDE with 10.1?
                              mboeringa2010
                              Just to make sure that I understood you correctly.
                              What you are saying is I do not need to install ArcSDE on the database server because the client contains the SDE functionality.


                              Yes, this is true, but in a 2-tier Direct Connect situation, be aware you may need to install other additional software like Oracle Client on the client ArcGIS computer, to get the connection to your database up and running. This is not necessary with a separate server side 3-tier installation of ArcSDE.

                              Another way of saying it:

                              - I could use direct connection without ArcSDE installed for two-tier topology.
                              - I could use ArcSDE to hide or conceal the database in a three-tier topology.

                              am I right?


                              Yes, that is about the story, although "hide" is probably a bit overstated, you will always have some database related stuff, like database login with username and password, that is there on the client side. And again be aware that for Oracle, you need to install Oracle Client in a two-tier topology, and that it needs to be a 32-bit version, as ArcGIS is still 32-bit.

                              Also, rephrasing the first sentence to:

                              - I could use direct connection without ArcSDE installed on a server for two-tier topology.

                              is probably a bit more accurate, as, as you now understand, ArcSDE is always installed on the client side as part of the ArcGIS installation.

                              ArcSDE is the engine of the "car" called "geodatabase"... Take away the engine entirely, and you will grind to a halt.
                              • Re: Why would I install ArcSDE with 10.1?
                                fleith
                                Hi,
                                I'm trying to wrap my head around this, as I setup my test server for our upgrade.

                                I have set up my server with Windows Server 2008 R2 sp1 and SQL Server 2012 standard edition.
                                I have installed ArcGIS desktop 10.1 + sp1
                                I ran through the ArcGIS for Server 10.1 sp1 installer but did not create a ArcGIS Server site. (essentially I needed the authorization file for the create enterprise geodatabase tool )

                                I then ran the tool > 'Create Enterprise Geodatabase'
                                I loaded some data into the geodatabase, registered as versioned.
                                When exploring the schema all the SDE and GDB tables are present.

                                Am I missing something? Do I need to install ArcSDE 10.1 to get the multi user/versioning/history capabilties of ArcSDE, or as this discussion indicates this is now inherent in the desktop.

                                I still am not 100% clear why I would need to install the ArcSDE application server or command line tools.
                                If I am managing data via desktop (ArcMap & ArcCatalog) I do not need to install anything further
                                If I am managing data via web editing tools I do need to install the ArcSDE application server.

                                Thank you for any clarification you can provide.
                                Fiona
                                • Re: Why would I install ArcSDE with 10.1?
                                  alantoms
                                  Yeah it took me a bit to get it as well.  The ArcSDE Command Line tools do not need to be installed unless you want to run stuff with a command line or need to do some major troubleshoot. 

                                  Hi,
                                  I'm trying to wrap my head around this, as I setup my test server for our upgrade.

                                  I have set up my server with Windows Server 2008 R2 sp1 and SQL Server 2012 standard edition.
                                  I have installed ArcGIS desktop 10.1 + sp1
                                  I ran through the ArcGIS for Server 10.1 sp1 installer but did not create a ArcGIS Server site. (essentially I needed the authorization file for the create enterprise geodatabase tool )

                                  I then ran the tool > 'Create Enterprise Geodatabase'
                                  I loaded some data into the geodatabase, registered as versioned.
                                  When exploring the schema all the SDE and GDB tables are present.

                                  Am I missing something? Do I need to install ArcSDE 10.1 to get the multi user/versioning/history capabilties of ArcSDE, or as this discussion indicates this is now inherent in the desktop.

                                  I still am not 100% clear why I would need to install the ArcSDE application server or command line tools.
                                  If I am managing data via desktop (ArcMap & ArcCatalog) I do not need to install anything further
                                  If I am managing data via web editing tools I do need to install the ArcSDE application server.

                                  Thank you for any clarification you can provide.
                                  Fiona
                                  • Re: Why would I install ArcSDE with 10.1?
                                    mboeringa2010
                                    I then ran the tool > 'Create Enterprise Geodatabase'
                                    I loaded some data into the geodatabase, registered as versioned.
                                    When exploring the schema all the SDE and GDB tables are present.


                                    Hi Fiona,

                                    Good, this means you are most likely "up-and-running", and can start using ArcGIS to fill and use your ESRI geodatabase

                                    Am I missing something? Do I need to install ArcSDE 10.1 to get the multi user/versioning/history capabilities of ArcSDE, or as this discussion indicates this is now inherent in the desktop.


                                    I still am not 100% clear why I would need to install the ArcSDE application server or command line tools.


                                    No, you do not need to install an ArcSDE 10.1 application server to get full geodatabase functionality, including multi user/versioning/history.

                                    As I and others have attempted to explain in this thread, ArcSDE's software components in the form of DLLs (Dynamic Link Libraries) form an inherent - and VITAL - part of any "client" ArcGIS application, most prominently "ArcGIS for Desktop" and "ArcGIS for Server" (Server is after all a "client" of the DBMS too, e.g. Oracle, Microsoft SQL Server). They are installed by default as part of the respective software's installation.

                                    You can't access or run an ESRI geodatabase without these DLLs that incorporate all the functionality and logic of ArcSDE to handle ESRI geodatabases (at least not by re-doing years long software development by ESRI and at great risk of corrupting the database in case you want to edit something).

                                    Like I wrote in my last post:

                                    *************************************
                                    ArcSDE is the "engine" of the "car" called "geodatabase"... Take away the engine entirely, and you will grind to a halt.
                                    *************************************

                                    However, if this sentence still leaves you confused, think of the ArcSDE application server running on a remote server being "public transport", and the ArcSDE DLLs on your local computer as part of "ArcGIS for Desktop" as your "private car". Both share an "engine" (THIS IS ARCSDE!), but they are independent and both get you from A to B (allow you to access an ESRI geodatabase with all of it's functionality).

                                    It is up to you to decide if you want to travel by "public transport" or use your "private car". One mode of transport may be faster than the other (or the other way around), depending on the conditions in your local "area"...

                                    To also recap some of the very good comments by the ESRI staff in this thread (Russell Brennan in this case):

                                    "Fast forward to 10.1 - We have tried to allow you to manage your Geodatabase completely within ArcGIS applications (ArcCatalog, ArcMap, ArcGIS Server, etc). This is done through the use of dialogues in ArcMap/Catalog and the use of geoprocessing."

                                    "If you determine that you really need the application server or the command line utilities they are available as a separate install."


                                    These comments mean that only in some exceptional cases, you might need one of the command line tools. Until you run into some serious trouble that really can't be dealt with using ArcGIS's new tools, you should be fine using Direct Connect and leaving your setup as it is now.
                                    • Re: Why would I install ArcSDE with 10.1?
                                      BrianK
                                      HI All,  Please help me understand this for the personal ArcSDE.  For the previous item...

                                      No, you do not need to install an ArcSDE 10.1 application server to get full geodatabase functionality, including multi user/versioning/history.


                                      So can I still use a personal ArcSDE with Desktop without ArcGIS Server, say on a laptop?  If so, do I need to through through the same procedure for creating or enabling a enterprise geodatabase?  If, so how.  The create geodatabase requires an authorization code for ArcGIS Server.

                                      Thanks

                                      Brian Kaplan
                                      • Re: Why would I install ArcSDE with 10.1?
                                        mboeringa2010
                                        HI All,  Please help me understand this for the personal ArcSDE.  For the previous item...

                                        No, you do not need to install an ArcSDE 10.1 application server to get full geodatabase functionality, including multi user/versioning/history.


                                        So can I still use a personal ArcSDE with Desktop without ArcGIS Server, say on a laptop?  If so, do I need to through through the same procedure for creating or enabling a enterprise geodatabase?  If, so how.  The create geodatabase requires an authorization code for ArcGIS Server.

                                        Thanks

                                        Brian Kaplan


                                        Brian, there is no "personal ArcSDE", there is just the ArcSDE DLL's that incorporate all the functionality to allow a Direct Connect connection on your local personal PC to a (remote) enterprise geodatabase, but that isn't a product called "personal ArcSDE".

                                        Additionally, if you want to create an enterprise geodatabase (so no file geodatabase), you will need an ArcGIS for Server commercial licence, as ArcSDE / ArcGIS for Server is a sale / non-free product of ESRI, and using an enterprise geodatabase requires this licence, even if you only start using it for testing purposes. That is why you are being asked for an authorization code.

                                        You may be able to get a temporary trial licence for free for testing purposes, IDK, but as soon as you start deploying it, you will surely need a true paid licence. For that, there are two possible licencing levels: "ArcGIS for Server Enterprise" and "ArcGIS for Server Workgroup", see this page:

                                        http://www.esri.com/software/arcgis/arcgisserver/features/enterprise-workgroup
                                        • Re: Why would I install ArcSDE with 10.1?
                                          BrianK
                                          Brian, there is no "personal ArcSDE", there is just the ArcSDE DLL's that incorporate all the functionality to allow a Direct Connect connection on your local personal PC to a (remote) enterprise geodatabase, but that isn't a product called "personal ArcSDE".

                                          Additionally, if you want to create an enterprise geodatabase (so no file geodatabase), you will need an ArcGIS for Server commercial licence, as ArcSDE / ArcGIS for Server is a sale / non-free product of ESRI, and using an enterprise geodatabase requires this licence, even if you only start using it for testing purposes. That is why you are being asked for an authorization code.

                                          You may be able to get a temporary trial licence for free for testing purposes, IDK, but as soon as you start deploying it, you will surely need a true paid licence. For that, there are two possible licencing levels: "ArcGIS for Server Enterprise" and "ArcGIS for Server Workgroup", see this page:

                                          http://www.esri.com/software/arcgis/arcgisserver/features/enterprise-workgroup



                                          Hi Marco,

                                          Thank you for your reply.  I must have used the wrong terminology.  The install for ArcGIS Desktop includes an option to enable geodatabase storage on SQL Server Express.  Isn't this not an enterprise geodatabase?
                                          • Re: Why would I install ArcSDE with 10.1?
                                            vangelo-esristaff
                                            Enterprise geodatabases, which run in many different flavors of RDBMS software, require an
                                            ArcGIS for Server Enterprise license.  Desktop (formerly known as "Personal") and Workgroup
                                            multiuser geodatabases (which only run in SQL-Server Express, and are subject to restrictions
                                            in user count, storage, RAM, and CPU cores) are available at different licensing levels. 

                                            The Multiuser Geodatabase page breaks down the options.

                                            - V
                                            • Re: Why would I install ArcSDE with 10.1?
                                              BrianK
                                              Enterprise geodatabases, which run in many different flavors of RDBMS software, require an
                                              ArcGIS for Server Enterprise license.  Desktop (formerly known as "Personal") and Workgroup
                                              multiuser geodatabases (which only run in SQL-Server Express, and are subject to restrictions
                                              in user count, storage, RAM, and CPU cores) are available at different licensing levels. 

                                              The Multiuser Geodatabase page breaks down the options.

                                              - V


                                              Thank you.  I found that the ArcGIS desktop menu included the ArcSDE install and the ability to take an existing SQL Server Express database and convert it to a geodatabase.  Brian.
                                              • Re: Why would I install ArcSDE with 10.1?
                                                vangelo-esristaff
                                                Desktop multi-user geodatabases are to Enterprise ArcSDE as "Physics for Social Science Majors"
                                                is to "Intermediate Electricity and Magnetism"  -- the flavor is there, but a good deal of the rigor
                                                is missing.

                                                - V
                                                • Re: Why would I install ArcSDE with 10.1?
                                                  ArielLo
                                                  I have a similar question, regarding Oracle.

                                                  Some of you here said that if I do not install ArcSDE application on the server, and use only ArcGIS for Desktop, I'll need to install an Oracle Client (32bit ofcourse) on each desktop that will direct connect to the geodatabase via ArcCatalog/ArcMap.

                                                  However, I need to be able to connect to the geodatabase (direct connection or application connection) via a desktop without an Oracle client installed. For that, I'll need to install the ArcSDE application on the server, correct?

                                                  Also, will I need to use application connection or a direct connection?

                                                  Thanks,

                                                  Ariel.
                                                  • Re: Why would I install ArcSDE with 10.1?
                                                    mboeringa2010
                                                    Some of you here said that if I do not install ArcSDE application on the server, and use only ArcGIS for Desktop, I'll need to install an Oracle Client (32bit ofcourse) on each desktop that will direct connect to the geodatabase via ArcCatalog/ArcMap.

                                                    However, I need to be able to connect to the geodatabase (direct connection or application connection) via a desktop without an Oracle client installed. For that, I'll need to install the ArcSDE application on the server, correct?

                                                    Also, will I need to use application connection or a direct connection?


                                                    Ariel, if you need the full geodatabase functionality but can't install the Oracle Client, than yes, your only option is to install and run an ArcSDE Application Server (which isn't a bad thing or so ;)).

                                                    In this scenario, the connection is called an Application Server connection, so you won't be using Direct Connect.

                                                    Please note you need to add the following line to your Windows services file of your client PC running ArcGIS for Desktop per installation instructions for the ArcSDE Application Server. Please note this has to be done on each machine connecting to your Application Server!:

                                                    esri_sde      5151/tcp

                                                    The Windows services file is simply called "services" without a file extension and in Windows 7 located in:

                                                    C:\Windows\System32\drivers\etc
                                                    • Re: Why would I install ArcSDE with 10.1?
                                                      rlwatson

                                                      Please note you need to add the following line to your Windows services file of your client PC running ArcGIS for Desktop per installation instructions for the ArcSDE Application Server. Please note this has to be done on each machine connecting to your Application Server!:

                                                      esri_sde      5151/tcp

                                                      The Windows services file is simply called "services" without a file extension and in Windows 7 located in:
                                                      C:\Windows\System32\drivers\etc


                                                      Is this really necessary on client machines?  I ask because I have never done this and used the SDE Application Server from Windows extensively.  My understanding, which could be wrong, is that this needs to be on machine on which the SDE Application Server is installed.
                                                      • Re: Why would I install ArcSDE with 10.1?
                                                        vangelo-esristaff
                                                        The services file on the application server must be modified because the service
                                                        is started by name.  It's good practice to distribute a modified services file among
                                                        clients, but it's only necessary if those clients will also lookup by name (if you
                                                        specify the port number, then it isn't necessary).

                                                        - V
                                                        • Re: Why would I install ArcSDE with 10.1?
                                                          ArielLo
                                                          Ariel, if you need the full geodatabase functionality but can't install the Oracle Client, than yes, your only option is to install and run an ArcSDE Application Server (which isn't a bad thing or so ;)).

                                                          In this scenario, the connection is called an Application Server connection, so you won't be using Direct Connect.

                                                          Please note you need to add the following line to your Windows services file of your client PC running ArcGIS for Desktop per installation instructions for the ArcSDE Application Server. Please note this has to be done on each machine connecting to your Application Server!:

                                                          esri_sde      5151/tcp

                                                          The Windows services file is simply called "services" without a file extension and in Windows 7 located in:

                                                          C:\Windows\System32\drivers\etc



                                                          Thanks!

                                                          This is exactly what I did, works perfectly.

                                                          I have a question though - I've heared a rumor that ESRI will probably eventually stop supporting Application Server Connection, is that true?

                                                          What is the "recommended" method of connection by ESRI?

                                                          Thanks again,

                                                          Ariel.
                                                          • Re: Why would I install ArcSDE with 10.1?
                                                            mboeringa2010
                                                            I have a question though - I've heared a rumor that ESRI will probably eventually stop supporting Application Server Connection, is that true?

                                                            What is the "recommended" method of connection by ESRI?


                                                            Rumours are rumours, and since I don't work for ESRI, I can say nothing sensible about it. From a technical point of view, I see little reason why ESRI would decide to abandon the Application Server option completely. As much of the code base for the Application Server executables is the same as for the Direct Connect DLL drivers (see Derek Law's slightly older but still relevant Enterprise Geodatabase 101 here on the ESRI website), there isn't a whole lot to win by abandoning the Application Server option.

                                                            From a "support" point of view, there may be reasons, as it seems the Application Server option seems to be a bit more difficult to get up-and-running for some users, and causing more questions and confusion requiring ESRI intervention.

                                                            As for "recommended", ESRI made Direct Connect default, so I guess this would qualify as the "recommended" option. I doubt though, Vince would give you such an answer, as he probably justly and more precisely will tell you it all depends on the configuration of your specific client/server hard- and software LAN network environment.

                                                            As I said before in this thread:

                                                            "It is up to you to decide if you want to travel by "public transport" or use your "private car". One mode of transport may be faster than the other (or the other way around), depending on the conditions in your local "area".."
                                                            • Re: Why would I install ArcSDE with 10.1?
                                                              mboeringa2010
                                                              Also, in terms of Application Server versus Direct Connect, and why ESRI may have decided to make Direct Connect default, it may have to do with a number of (or probably many) clients of ESRI having seen a significant rise in the number of concurrent users using enterprise geodatabases (and not just through cached webservices).

                                                              Where in the past for most organizations it used to be that only a few highly active editors / viewers concurrently accessed the database (e.g. maximum a dozen) through ArcMap, now for some shops maybe dozens or even hundreds might access it.

                                                              In that scenario, Direct Connect is probably the best option.

                                                              However, if you still are within a mid-size organization, with maybe a (few) dozen maximum active users at any point in time, and have a beefy modern 8+ core database server / Application Server with a tens or hundreds of Gigabytes big enterprise geodatabase, than Application Server connections may still be very much a valid choice. In fact, the server with it's specialized high performance hardware and processors (e.g. Xeon), may outperform your desktop in processing the GIS data, especially in cases of network bandwidth constraints.

                                                              Also, this remark by D.E.Wright on the GIS StackExchange Forum may be of interest:

                                                              �??There are some very good reasons to use an ArcSDE Server Engine (Edit: Should be ArcSDE Application Server in official ESRI terminology), the first being the load. When you utilize a ArcSDE (Edit: Application) Server Service you are taking the bulk of that data load off the database server and queuing it versus relying on just your local machine to store all that temp data.
                                                              One thing you will see especially with a MSSQL server when you make your initial database connection in a MXD is that ArcGIS does a 'SELECT *' (as seen in your query analyzer and logs on your DB Server) against that table/feature-class. Now, this can be a huge impact if you have very large datasets; the ArcSDE Service/Process helps in this by aiding in the request of the appropriate data scope.
                                                              Now as we have all gotten bigger machines, with more RAM its much easier to just load everything into the current session and run with it; but don't just discount the idea of using the service just because the ArcGIS docs say you 'don't need it' anymore, versus when you probably could/should use it.�?�
                                                              • Re: Why would I install ArcSDE with 10.1?
                                                                stacemaples
                                                                I think the length and complexity of this thread make a VERY good case for someone overhauling BOTH the ArcSDE and ArcGIS Server installation documentation.  This represents a significant departure from previous architectures and not all of us have the luxury of teams of devs to help us sort out the new 'paradigm.'  I just wasted about 2 days trying to figure out how to work with the new iteration of SDE, only to find that I no longer need it from a forum posting, rather than ESRI documentation.  This is for a directed study with a student interested in learning to manage these types of architectures.  He is quickly learning that, perhaps, the greatest takeaway will be forum mining skills.

                                                                In F, L & T,
                                                                Mr. Stacey D. Maples
                                                                GIS Specialist & Instruction Coordinator

                                                                Twitter = MapNinja
                                                                AIM = yalemapninja
                                                                GTalk = yale.map.ninja
                                                                Yahoo IM = stacemaples
                                                                ICQ = 57891396

                                                                Yale University Map Department
                                                                Sterling Memorial Library 7th Floor
                                                                130 Wall Street /P.O. Box 208240
                                                                New Haven, CT 06520-8240
                                                                P:203-432-8269 F:203-432-8240
                                                                The Yale University GIS Support Page [http://guides.library.yale.edu/GIS]
                                                                The Yale University Map Department [http://www.library.yale.edu/maps]
                                                                Yale GIS-L Mailing List Subscription [http://mailman.yale.edu/mailman/listinfo/gis-l]
                                                                Search for scanned maps and other Digital Objects from Yale's Library [http://discover.odai.yale.edu/ydc/]
                                                                Yale GIS Workshops Calendar [http://bit.ly/xqPvdS]


                                                                "I have a map of the United States... actual size.
                                                                                It says, "Scale: 1 mile = 1 mile."
                                                                                                I spent last summer folding it."
                                                                                                                                -Steven Wright-
                                                                • Re: Why would I install ArcSDE with 10.1?
                                                                  hawatson
                                                                  I have to admit, as someone who has been managing SDE installs since 9.1, I didn't pick up on this paradigm shift until a few days ago when I did a test 10.1 on Oracle install, and realized that with Toad and the Python interpreter in ArcCatalog I could pretty much do everything I needed to get the enterprise geodatabase created, a user-schema geodatabase created, and the .sde connection files to access them created.  I think that is a good step forward for basic installs.  If I can take the SDE Application server out of the picture, that is a big time saver.  The toolbox tools are a little rough, but with arcpy you can get the job done.

                                                                  It would be great to have access to some more of the sde app server command line tools available via the python arcpy module.  I'm thinking especially of the tools like sdeconfig and sdedbtune, which still seem necessary to me.  I hope that is in the works.  If I'm missing something on that front, I would love to be educated.

                                                                  Looking at it now, the progression makes sense and I'm glad for the new options.
                                                                  • Re: Why would I install ArcSDE with 10.1?
                                                                    xyknewtry
                                                                    I haven't had much experience with ArcSDE or ArcGIS server in general, and I have one question. While I was reading through the tutorial on ArcGIS API for Silverlight (I believe that was before 10.1 was released) it was mentioned somewhere that to be able to edit a feature class using a web application, the feature class must be stored in an SDE database and published as a feature service. Is the part regarding SDE still true at 10.1?

                                                                    Our office is finally getting an ArcGIS sever (Standard Edition at the Workgroup level) so we can play with it and look into web map application development. The system admin is going to set it up soon, and the ArcGIS Server 10.1 SP1 installer is all we've got (no Web Adaptor or ArcSDE installer/setup). Is the following scenario for testing purposes possible? If not, what else do we need to install/configure, or what is the alternative?

                                                                    Install ArcGIS for Sever on sever A;
                                                                    Using ArcGIS for Desktop on a workstation, create a file geodatabase on either sever A or some other server B on the same office LAN, and create a feature class in the FGDB and publish it as a feature service;
                                                                    Create a web mapping application and host it on server A (separate IIS set up necessary?);
                                                                    Display, query, and edit the feature class through the web application over the internet.

                                                                    If this scenario is possible, does ArcGIS Server create a copy of the feature class and serve the copy when I publish the FC as a service? If that is true, will I be able to access/edit this copy in ArcGIS for Desktop on the workstation?
                                                                    • Re: Why would I install ArcSDE with 10.1?
                                                                      jskinner-esristaff
                                                                      Hi Yukon,

                                                                      You will need the feature class stored in SDE in order to create a feature service.  The ArcGIS for Server Workgroup (Standard Edition) comes with SQL Server Express.  You should be able to download the installation files from the customer care portal:

                                                                      http://resources.arcgis.com/en/help/main/10.1/index.html#//018t00000010000000

                                                                      After you install SQL Server Express, you will just need to enable SQL Server Express to store geodatabases:

                                                                      http://resources.arcgis.com/en/help/main/10.1/index.html#/Enabling_SQL_Server_Express_to_store_geodatabases/018t0000000w000000/

                                                                      Additional helpful links:

                                                                      Setting up a connection:
                                                                      http://resources.arcgis.com/en/help/main/10.1/index.html#/Setting_up_a_connection_to_SQL_Server/018t0000002z000000/

                                                                      Adding a database server to Desktop:
                                                                      http://resources.arcgis.com/en/help/main/10.1/index.html#/Adding_a_database_server_to_ArcGIS_Desktop/018t0000001s000000/

                                                                      Creating a geodatabase on a database server:
                                                                      http://resources.arcgis.com/en/help/main/10.1/index.html#/Creating_a_geodatabase_on_a_database_server/018t0000000m000000/
                                                                      • Re: Why would I install ArcSDE with 10.1?
                                                                        xyknewtry
                                                                        You should be able to download the installation files from the customer care portal:

                                                                        http://resources.arcgis.com/en/help/main/10.1/index.html#//018t00000010000000



                                                                        Thank you for the replay Jake, very helpful information. If we have the ArcGIS Sever 10.1 SP1 installer, does it typically include the installation files you mentioned for SQL express (We haven't started installing yet)? I'm hoping it's the case, as due to the structure of our organization, our office is not directly involved in software purchasing/management. I logged into my customer portal with our customer ID, but there is no order history, and I don't see anything for download.
                                                                        • Re: Why would I install ArcSDE with 10.1?
                                                                          ktareen
                                                                            You will need the feature class stored in SDE in order to create a feature service. 


                                                                          To confirm, a direct connect is sufficient for a feature service - as long as the database supporting the feature service has the SDE tables within it, correct?
                                                                          • Re: Why would I install ArcSDE with 10.1?
                                                                            mboeringa2010
                                                                            To confirm, a direct connect is sufficient for a feature service - as long as the database supporting the feature service has the SDE tables within it, correct?


                                                                            Karyn,

                                                                            ArcGIS for Server is, just like ArcGIS for Desktop, fully capable of using both an ArcSDE Application Server, or Direct Connect, for connecting to an enterprise geodatabase.

                                                                            From a functionality point of view, there is absolutely no difference between using an Application Server or Direct Connect. So yes, "Direct Connect" is sufficient for ArcGIS for Server to create (editable) Feature Services.
                                                                            • Re: Why would I install ArcSDE with 10.1?
                                                                              mboeringa2010
                                                                              I think the length and complexity of this thread make a VERY good case for someone overhauling BOTH the ArcSDE and ArcGIS Server installation documentation.


                                                                              Well, it isn't an overhaul, but I now published a document describing "The ESRI Geodatabase Framework" (as I call it and see it) in another forum thread. It may be a good read for people interested in the contents of this thread and help out with a better understanding of 2-tier Direct Connect and 3-tier Application Server connections to geodatabases:

                                                                              "The ESRI Geodatabase Framework" PDF

                                                                              Please post any comments to the document in that thread and not here.

                                                                              Marco
                                                                              • Re: Why would I install ArcSDE with 10.1?
                                                                                rlwatson
                                                                                http://forums.arcgis.com/threads/85096-Deprecation-plans-for-10.1-and-10.2

                                                                                "ArcGIS 10.2 will be the last major release to include the ArcSDE application server. Esri
                                                                                encourages those that have deployed an ArcSDE application server to move to new
                                                                                database connections introduced in 10.1 as part of their next upgrade."
                                                                                • Re: Why would I install ArcSDE with 10.1?
                                                                                  mboeringa2010
                                                                                  http://forums.arcgis.com/threads/85096-Deprecation-plans-for-10.1-and-10.2

                                                                                  "ArcGIS 10.2 will be the last major release to include the ArcSDE application server. Esri
                                                                                  encourages those that have deployed an ArcSDE application server to move to new
                                                                                  database connections introduced in 10.1 as part of their next upgrade."


                                                                                  Yes, I saw this too. Interesting, I had not yet seen this updated deprecation plan of the 7th of May including the changes expected for 10.2 and beyond and the now officially announced deprecation of the ArcSDE Application Server, ArcSDE Command Line tools and ArcSDE C/Java SDKs after 10.2.

                                                                                  Not unexpected, but still, I wonder if there might be future extension for this deprecation... To many people may still depend on it and not yet be in a situation to make all the necessary changes in time. Just look at VBA... extended to 10.2 while initially announced deprecated at 10.1...

                                                                                  Anyway, I am in the process of creating two updated Geodatabase Framework diagrams that will incorporate and show this deprecation to make it more understandable. One diagram will depict the situation at 10.2, the other the situation beyond (well, at least as far as my "crystal ball" in the form of this deprecation plan allows me...).
                                                                                  • Re: Why would I install ArcSDE with 10.1?
                                                                                    Wolseley
                                                                                    I have found it easier to do the following using SQL Server Management Studio 2008 R2:

                                                                                    In assigning users, roles, and permissions inside ArcCatalog, you get a dialog box, but no list of choices for the types of roles/permissions you can assign. In SQL Server Management Studio, you do this and their dialog box provides you a list of check boxes. You choose which box(es) to check for roles/permissions. It also verifies that you have the correct syntax and username does in fact exist on the network. Using ArcCatalog, you would never know this as it does not verify the user name to see if you typed it correctly or exists.

                                                                                    I found myself having to use a little of both together to manage our Geodatabases: SQL Server Management Studio and ArcCatalog Geodatabase Administration.

                                                                                    I only use ArcCatalog Geodatabase Administration for:

                                                                                    1) Creating my schema (using X-Ray Geodatabase, a great tool from ESRI!) - anyone who is managing Enterprise Geodatabase should use X-Ray Geodatabase!

                                                                                    2) Editing my Geodatabase (Includes Post and Reconcile operations)

                                                                                    3) Setting up Versioning

                                                                                    4) Creating/Managing Replicas 

                                                                                    5) Archiving

                                                                                    5) Seeing who is connected, and identifying Locks



                                                                                    I only use SQL Server Management Studio to:

                                                                                    1) Set up my Instance

                                                                                    2) Create Database (name only, not the schema)

                                                                                    3) Assign Users/Roles/Permissions


                                                                                    I use SDE, but this runs in the background allowing me to do what I need to do.

                                                                                    Personally, I found the SDE Command Line to be obnoxious, exceedingly difficult, and cumbersome. We are a two person, very overworked GIS shop and do not have the time to learn the commands and syntax. I say use the dialog and check boxes it will make your life much easier.
                                                                                    • Re: Why would I install ArcSDE with 10.1?
                                                                                      vopgis
                                                                                      So when upgrading to 10.1 from 9.3.1, and not wanting to install the 10.1 ArcSDE Server Application, do I have to uninstall ArcSDE 9.3.1 from my existing Geodatabase? Then simply run the "Upgrade Geodatabase" tool in ArcCatalog to get it to 10.1?
                                                                                      • Re: Why would I install ArcSDE with 10.1?
                                                                                        mboeringa2010
                                                                                        So when upgrading to 10.1 from 9.3.1, and not wanting to install the 10.1 ArcSDE Server Application, do I have to uninstall ArcSDE 9.3.1 from my existing Geodatabase ?


                                                                                        I guess you meant to say:

                                                                                        "... do I have to uninstall the ArcSDE 9.3.1 Application Server from my database server..." ?

                                                                                        If so: yes. Of course, the geodatabase itself with the entire ArcSDE Repository and the ArcSDE and geodatabase system tables therein, should be left untouched, otherwise nothing is left to upgrade...

                                                                                        So when upgrading to 10.1 from 9.3.1, and not wanting to install the 10.1 ArcSDE Server Application, do I have to uninstall ArcSDE 9.3.1 from my existing Geodatabase? Then simply run the "Upgrade Geodatabase" tool in ArcCatalog to get it to 10.1?


                                                                                        Don't forget you need to install an appropriate database client. Without client, you won't be able to connect any more once your Application Server is down!

                                                                                        There may also be an extra issue if you decided to migrate to a new server and in the process go from a 32 bit to 64 bit environment... There are descriptions for that in the Help, but it isn't an easy task.
                                                                                        • Re: Why would I install ArcSDE with 10.1?
                                                                                          vopgis
                                                                                          Yes, my apologies, I meant the server. We are currently at 64-bit so no need to upgrade there. Regarding the database client I'm not sure what you mean? We're currently connecting via ArcReader and a few web apps which are already migrated to the Direct Connect method. Wouldn't the database client simply be installed with ArcGIS Desktop or ArcReader? Or is there something else I should be looking at? My main concern was what to uninstall. But, looking more clearly, I would have to uninstall ArcSDE anyway because we have a desktop install on our server as well which of course would also need an upgrade. Thank you for the clarification.
                                                                                          • Re: Why would I install ArcSDE with 10.1?
                                                                                            mboeringa2010
                                                                                            We're currently connecting via ArcReader and a few web apps which are already migrated to the Direct Connect method.


                                                                                            If you can already use Direct Connect in ArcReader on your local PC, than you must have the required database client installed. So you shouldn´t have to worry about it.

                                                                                            More info on this Help page:
                                                                                            http://resources.arcgis.com/en/help/main/10.1/index.html#//019v00000009000000

                                                                                            Again, note the database client is only required in Direct Connect, Application Server connections do not use the database client. But since ESRI plans to phase out the Application Server, the database client can better be assumed compulsory.

                                                                                            Wouldn't the database client simply be installed with ArcGIS Desktop or ArcReader?


                                                                                            The database client is a separate install, see the first Help page linked above. You can download it either from the database vendor, or the ESRI Customer Care portal.

                                                                                            But your IT department is likely to have already installed such a client in their default drive-images in case database connections are used by other non-GIS desktop applications in your organization.
                                                                                            • Re: Why would I install ArcSDE with 10.1?
                                                                                              mcwhitehead
                                                                                              I guess you meant to say:

                                                                                              "... do I have to uninstall the ArcSDE 9.3.1 Application Server from my database server..." ?

                                                                                              If so: yes. Of course, the geodatabase itself with the entire ArcSDE Repository and the ArcSDE and geodatabase system tables therein, should be left untouched, otherwise nothing is left to upgrade...



                                                                                              Don't forget you need to install an appropriate database client. Without client, you won't be able to connect any more once your Application Server is down!

                                                                                              There may also be an extra issue if you decided to migrate to a new server and in the process go from a 32 bit to 64 bit environment... There are descriptions for that in the Help, but it isn't an easy task.


                                                                                              I am upgrading from 9.3.1 to 10.1 and just want to make sure I have my ducks in a row. Are these the general steps I need to take:

                                                                                              1. Make backups, check permissions, etc, etc
                                                                                              2. Uninstall ArGIS Server 9.3.1 including ArcSDE from the server(and delete ArsSDE services)
                                                                                              3. Install ArcGIS for Server 10.1 on the server(?)
                                                                                              4. Upgrade from client via ArcCatalog

                                                                                              I guess my confusion is when reading through the steps on http://resources.arcgis.com/en/help/main/10.1/index.html#/Preparing_to_upgrade_a_geodatabase_in_SQL_Server/002q000000m6000000/
                                                                                              It says that I need to install the current release of ArcGIS client on a computer that can directly connect. I've got that and checked the connection. Do I need ArcGIS for Server 10.1 installed on my server? I assume the answer is yes but I'm not reading that explicitly in those steps. I just see to get rid of the old 9.3.1 install.
                                                                                              • Re: Why would I install ArcSDE with 10.1?
                                                                                                mboeringa2010
                                                                                                - You don't need to install an ArcSDE Application Server (Also called "ArcSDE service" in ESRI documentation) if you want to connect to your geodatabase, use Direct Connect instead.

                                                                                                The ArcSDE Application Server will no longer be available after ArcGIS 10.2, so it is highly recommended to switch to Direct Connect at this point in time.

                                                                                                - You may need an ArcGIS for Server installation if you use it to create webservices (ESRI Map, Image, Feature Services or WMS, WFS or any of the other service types it supports). If you're not interested in serving your own webservices, you don't need ArcGIS for Server.

                                                                                                - Use the new geoprocessing tools in the Geodatabase Administration toolset in ArcGIS for Desktop for management and creation of your ESRI enterprise geodatabase. See for example the Blog article by ESRI's Melissa J linked below, and the second link to the Geodatabase Administration toolset:

                                                                                                Do This, Not That! �?? Alternatives to using SDE command line tools:
                                                                                                http://blogs.esri.com/esri/supportcenter/2013/10/04/do-this-not-that-alternatives-to-using-sde-command-line-tools/#sthash.YiYFmz5c.dpuf

                                                                                                Geodatabase Administration toolset
                                                                                                An overview of the Geodatabase Administration toolset

                                                                                                You may find it helpful to read these two PDF documents I created:

                                                                                                The ESRI Geodatabase Framework
                                                                                                http://forums.arcgis.com/threads/83644-quot-The-ESRI-Geodatabase-Framework-quot-PDF?p=295462&viewfull=1#post295462
                                                                                                The ESRI Geodatabase Framework - Future developments at ArcGIS 10.2 and 11
                                                                                                http://forums.arcgis.com/threads/83644-quot-The-ESRI-Geodatabase-Framework-quot-PDF?p=303021&viewfull=1#post303021
                                                                                                • Re: Why would I install ArcSDE with 10.1?
                                                                                                  vangelo-esristaff
                                                                                                  Do I need ArcGIS for Server 10.1 installed on my server? I assume the answer is yes but I'm not reading that explicitly in those steps. I just see to get rid of the old 9.3.1 install.


                                                                                                  It's certainly not required (which is the main point of this thread).  Whether you "need" it depends
                                                                                                  on how bound you are to using an application server.  It won't be an option after 10.2, so at some
                                                                                                  point you'll need to switch over to Direct Connect exclusively.

                                                                                                  - V
                                                                                                  • Re: Why would I install ArcSDE with 10.1?
                                                                                                    mcwhitehead
                                                                                                    - You don't need to install an ArcSDE Application Server (Also called "ArcSDE service" in ESRI documentation) if you want to connect to your geodatabase, use Direct Connect instead.

                                                                                                    The ArcSDE Application Server will no longer be available after ArcGIS 10.2, so it is highly recommended to switch to Direct Connect at this point in time.

                                                                                                    - You may need an ArcGIS for Server installation if you use it to create webservices (ESRI Map, Image, Feature Services or WMS, WFS or any of the other service types it supports). If you're not interested in serving your own webservices, you don't need ArcGIS for Server.

                                                                                                    - Use the new geoprocessing tools in the Geodatabase Administration toolset in ArcGIS for Desktop for management and creation of your ESRI enterprise geodatabase. See for example the Blog article by ESRI's Melissa J linked below, and the second link to the Geodatabase Administration toolset:

                                                                                                    Do This, Not That! �?? Alternatives to using SDE command line tools:
                                                                                                    http://blogs.esri.com/esri/supportcenter/2013/10/04/do-this-not-that-alternatives-to-using-sde-command-line-tools/#sthash.YiYFmz5c.dpuf

                                                                                                    Geodatabase Administration toolset
                                                                                                    An overview of the Geodatabase Administration toolset

                                                                                                    You may find it helpful to read these two PDF documents I created:

                                                                                                    The ESRI Geodatabase Framework
                                                                                                    http://forums.arcgis.com/threads/83644-quot-The-ESRI-Geodatabase-Framework-quot-PDF?p=295462&viewfull=1#post295462
                                                                                                    The ESRI Geodatabase Framework - Future developments at ArcGIS 10.2 and 11
                                                                                                    http://forums.arcgis.com/threads/83644-quot-The-ESRI-Geodatabase-Framework-quot-PDF?p=303021&viewfull=1#post303021



                                                                                                    Thanks you. We use direct connections so no need for the application server. I have upgraded my 9.3.1 database to 10.1 successfully. I currently have server 9.3.1 installed on the same server the DBMS and geodatabases are stored on. I will likely retroactively upgrade to server 10.2 (as well as upgrading client apps to 10.2 and subsequently gdbs to 10.2) in the near future.
                                                                                                    • Re: Why would I install ArcSDE with 10.1?
                                                                                                      stsnider
                                                                                                      I wanted to chime in incase this point had not been made clear.  There's a difference between an enterprise geodatabase and ArcSDE application service.  You can have an enterprise geodatabase (Example: SQL Server or Oracle) and also have ArcSDE installed for an application service.  Or you can have an enterprise geodatabase and not have ArcSDE installed.  Both options require an ArcGIS Server Basic license. 

                                                                                                      At ArcGIS Desktop 10.0, the default connection type to an enterprise geodatabase assumed the application service was installed - although ESRI recommended using a direct connect at "best practices".  At 10.1 and above, ArcGIS Desktop uses the direct connection by default.  You still have the option of connecting using an application service, but the setup is hidden in the ArcToolbox.  After 10.2, ArcSDE will no longer be available...so you will have to use a direct connect to get to your enterprise geodatabase.  Hope this helps.
                                                                                                      • Re: Why would I install ArcSDE with 10.1?
                                                                                                        KG22
                                                                                                        I would like to add my 2 cents to this post based on my experiences of the past few days.
                                                                                                        In the past i relied heavily on sde views - take a spatial table, create a view on it, register that with sde using the command line tools and then modify the sql of the view to join in several other, non-spatial, tables.
                                                                                                        The resulting 'sde view' is seen by esri in the same way a feature class would be.
                                                                                                        This is a very effective tool and we use it extensively.
                                                                                                        At 10.1, the idea was that sde command line tools are being mothballed. 'Most' everything you need to do will be available in Catalog, ArcMap and/or python.
                                                                                                        Well that is not entirely true in the case of spatial views.

                                                                                                        At 10.1 you can make a spatial view in ArcCatalog and it works great. So we did that, and we based ags services on map layers based on those views. Then a few days ago we started seeing issues. The services failed. ags error logs stated that ags could not recognize the geometry type of the layers in those services.
                                                                                                        The issue is that our data expires over time and we remove it from the view via a sql where clause. When that view is empty ags/sde no longer recognize it as being a view - it is just a table. So our view sql may say the value in some field must be less than 4 hours old. If no data is added to that table over 4 hours it will be empty. The service will fail and our app throws errors. Not good.

                                                                                                        The answer we are getting now is to use the command line tool to create the views and make sure they are registered with sde.
                                                                                                        Apparently with the new style views the geometry metadata is gathered on the fly from the first record in the table.
                                                                                                        That makes sense to me. But what about when there are no records? Seems like a better system would have been to get the metadata from the only spatial column in the view - the one in the base table - which is a feature class. A feature class can be empty and sde/ags still knows it is a feature class.

                                                                                                        Anyway, sde command line is not ready for the closet yet.
                                                                                                        • Re: Why would I install ArcSDE with 10.1?
                                                                                                          vangelo-esristaff
                                                                                                          In fact, you have been encouraged to "register your view with ArcSDE".  This is significantly
                                                                                                          different from "use the command line tool to create the views."  I strongly discourage anyone
                                                                                                          from using 'sdetable -o create_view' ever again, particularly with native or ST_GEOMETRY
                                                                                                          spatial columns.

                                                                                                          In your particular use, registration of the existing view is necessary (it is not possible for a query
                                                                                                          cursor to mine the database query plan to extract the information you expect it to have), but that
                                                                                                          only requires 'sdelayer -o register', not recreation of the existing views (be sure to use 'sdelayer
                                                                                                          -o describe_long' to extract the SRID of the base table for the -R flag in the register options).

                                                                                                          Hopefully, there will be a ArcPy tool that will register views with geodatabase metadata once
                                                                                                          that which was once ArcSDE is gone.

                                                                                                          - V
                                                                                                          • Re: Why would I install ArcSDE with 10.1?
                                                                                                            mboeringa2010
                                                                                                            Hopefully, there will be a ArcPy tool that will register views with geodatabase metadata once
                                                                                                            that which was once ArcSDE is gone.


                                                                                                            "Hopefully, there will be a ArcPy tool that will register views with geodatabase metadata once that which was once considered ArcSDE is gone."

                                                                                                            or

                                                                                                            "Hopefully, there will be a ArcPy tool that will register views with geodatabase metadata once that which was once the ArcSDE command line toolset is gone."

                                                                                                            might be more appropriate?... or are the ArcSDE system tables due for retirement too?
                                                                                                            • Re: Why would I install ArcSDE with 10.1?
                                                                                                              vangelo-esristaff
                                                                                                              The dichotomy of the the two different interfaces has always been the issue.  If there's no command
                                                                                                              line and no API to make a command line, then the geodatabase takes over full ownership of all tables
                                                                                                              owned by the SDE user.  At that point, your suggested change is a distinction without a difference.
                                                                                                              The "SDEish" metadata tables have always been geodatabase metadata tables in fact, so I don't see
                                                                                                              any major change.

                                                                                                              - V
                                                                                                              • Re: Why would I install ArcSDE with 10.1?
                                                                                                                mboeringa2010
                                                                                                                The dichotomy of the the two different interfaces has always been the issue. If there's no command
                                                                                                                line and no API to make a command line, then the geodatabase takes over full ownership of all tables owned by the SDE user. At that point, your suggested change is a distinction without a difference.
                                                                                                                The "SDEish" metadata tables have always been geodatabase metadata tables in fact, so I don't see
                                                                                                                any major change
                                                                                                                .


                                                                                                                X2, this was how I viewed it, but I was just wondering if any new major changes to ArcSDE (/ Geodatabase) Repository were due. I guess any future changes will be pretty much inconsequential for anyone sticking to official APIs like ArcPy, ArcObjects, just like the change to XML based tables for the geodatabase system tables at 10.x, as the APIs pretty much hide what is "behind the scene"...
                                                                                                                • Re: Why would I install ArcSDE with 10.1?
                                                                                                                  vangelo-esristaff
                                                                                                                  If there were major changes under way, I wouldn't be the one making the announcement,
                                                                                                                  fifty posts deep in the Forums.

                                                                                                                  - V
                                                                                                                  • Re: Why would I install ArcSDE with 10.1?
                                                                                                                    KG22
                                                                                                                    One could argue that the collection of metadata on the (potential) first record in a view as opposed to the true metadata in the (singleton) spatial column of the base table that was the progenitor of said view is synonymous with proprietary software manufacturer laziness.

                                                                                                                    -K
                                                                                                                    • Re: Why would I install ArcSDE with 10.1?
                                                                                                                      Wolseley
                                                                                                                      My goodness what a long and confusing thread! do you need to install ArcSDE? well, yes, no, and maybe.

                                                                                                                      Here is the skinny I got out of all this mumbo-jumbo:

                                                                                                                      IF you are at 10.2/10.2.1 YOU do not need to install ArcSDE and you will NOT be given an option to install ArcSDE because it will not be there. ArcSDE is gone! kaput! no more!

                                                                                                                      At 10.2/10.2.1 you must use Direct Connect. This is your only option so use it.

                                                                                                                      For those of you NOT at 10.2.1 I say make it so.
                                                                                                                      • Re: Why would I install ArcSDE with 10.1?
                                                                                                                        vangelo-esristaff
                                                                                                                        and you will NOT be given an option to install ArcSDE because it will not be there.
                                                                                                                        ArcSDE is gone! kaput! no more!


                                                                                                                           Incorrect.  It is still possible to run the application server at 10.2.x, though it would
                                                                                                                        certainly be wise to move to a Direct Connect model now, before the next major
                                                                                                                        release does take the decision away.

                                                                                                                        - V
                                                                                                                        • Re: Why would I install ArcSDE with 10.1?
                                                                                                                          mboeringa2010
                                                                                                                          My goodness what a long and confusing thread! do you need to install ArcSDE? well, yes, no, and maybe.

                                                                                                                          Here is the skinny I got out of all this mumbo-jumbo:

                                                                                                                          IF you are at 10.2/10.2.1 YOU do not need to install ArcSDE and you will NOT be given an option to install ArcSDE because it will not be there. ArcSDE is gone! kaput! no more!


                                                                                                                          As an addition to what Vince said: it would also be wise to refer to the ArcSDE Application Server, and the ArcSDE Command Line Tools instead of just generic "ArcSDE" in case you wish to refer to the components of the ESRI Geodatabase Framework that will no longer be available in the next major release.

                                                                                                                          ArcSDE is far from gone as I tried to illustrate in the PDF document I referenced for you in the other thread you started, and Direct Connect implements all of the geodatabase functionality accessible through the ArcSDE Application Server, while the new Python tools and GUI components for geodatabase management in ArcGIS for Desktop attempt, at least partly, to cover the ArcSDE Command Line Tools functionality. Where they don't cover it yet, ESRI seems to actively work on this and seeking user input.
                                                                                                                          • Re: Why would I install ArcSDE with 10.1?
                                                                                                                            Wolseley
                                                                                                                            Here is my situation:

                                                                                                                            I have been using ArcGIS Server 10.1.1 with SQL Server Express 2008 R2 for the past year. I have several Enterprise (Multi-User)
                                                                                                                            Geodatabases in SQL Server. I am using Versioning, Replicas, Post/Reconcile, and Archiving. All is working great. No problems.

                                                                                                                            I upgraded to 10.2.1 a few days ago. Again, all is working good.

                                                                                                                            Here are the facts:

                                                                                                                            - ArcGIS Documentation says there are 3 types of Geodatabases: Personal, File, and SDE. To me it seems to get muddled as to whether you are required to use ArcSDE or not. If ESRI says there is no need to use ArcSDE then they should say something like this under the SDE Geodatabase Help: "If you do not want to use ArcSDE, click here to use Direct Connect to a Multi-User Geodatabase".

                                                                                                                            - When I created my Enterprise Geodatabases I used the Geodatabase Tools in ArcToolbox combined with SQL Server Management Studio. I did not use the ArcSDE Command Line and have never used it.

                                                                                                                            - The default filenames of the Multi-User (Enterprise) Geodatabases I created in ArcCatalog end in .sde.

                                                                                                                            - I have ArcSDE 10.1.1 installed and never installed ArcSDE 10.2.1.

                                                                                                                            Question:

                                                                                                                            I find it very difficult to determine if I am using ArcSDE or Direct Connect? If ESRI says there is no need to use ArcSDE then should there not be a 4th Geodatabase Type maybe referred to as "Direct Connect Geodatabase"?

                                                                                                                            I could uninstall ArcSDE 10.1.1 and see what happens. If my Multi-User Geodatabases still connect to SQL Server then I know I am using Direct Connect and Not ArcSDE. What is the best way to determine if I am actually using ArcSDE?
                                                                                                                            • Re: Why would I install ArcSDE with 10.1?
                                                                                                                              vangelo-esristaff
                                                                                                                              This is not an "either-or" situation. *EVERY* Direct Connect is *ALWAYS* an
                                                                                                                              ArcSDE connection.

                                                                                                                              As stated previously, the name "ArcSDE" applies to the technology used to
                                                                                                                              connect to enterprise geodatabases, the database components used to manifest
                                                                                                                              enterprise geodatabases, the 3-tier application server, and the 'C'/Java API.
                                                                                                                              The only part that is optional when working with enterprise geodatabases is
                                                                                                                              the application server.

                                                                                                                              Direct Connect still uses the 'C' API to interface with the RDBMS. Post-10.2,
                                                                                                                              Esri will no longer distribute the API SDK, but they will continue to use it, so it
                                                                                                                              really won't go away, just get pushed deeper into the software stack.

                                                                                                                              The application server component was always the thinnest and least significant
                                                                                                                              part to wear an "ArcSDE" label, but for whatever reason, became the most
                                                                                                                              famous -- this may be fueling the confusion (and the outrageous length of
                                                                                                                              this thread).

                                                                                                                              -V
                                                                                                                              • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                Wolseley
                                                                                                                                Does this mean I can uninstall ArcSDE 10.1.1 from my Server Machine. Based on how I created my Enterprise Geodatabases (using the "Create Enterprise Geodatabase" Tool in ArcToolbox in 10.1.1) I am not using the ArcSDE "Application Server" (3-Tier) I am already using Direct Connect (2-Tier)?

                                                                                                                                If I understand correctly, the ArcSDE download option from the Licensing/Provisioning Site is actually the "ArcSDE Application Server" which is only used for the 3-Tier Model. However, there is another "component" of ArcSDE, that IS NOT the same as "ArcSDE Application Server" that runs inside/behind-the-scenes/deep inside the "stack" with the 2-Tier (Direct Connect) Model AND this "component" is automatically installed when ArcGIS for Server is installed?
                                                                                                                                • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                  vangelo-esristaff
                                                                                                                                  You can uninstall the 10.1 SP1 ArcSDE application server software if you are not using it.
                                                                                                                                  Just because the 10.x installer uses Direct Connect, and some clients may be using Direct
                                                                                                                                  Connect, doesn't mean that all clients are using Direct Connect.  If you never configured
                                                                                                                                  your application server, then no one could use it, but if you did (e.g. there exists a running
                                                                                                                                  service), then it would be wise to stop the service before removing it, just to see if anything
                                                                                                                                  was using it.

                                                                                                                                  ArcGIS Server is an ArcSDE client application.  That's all it ever takes for any application
                                                                                                                                  to claim that it has a "component of ArcSDE" installed.  All ArcSDE clients have the ability
                                                                                                                                  to make a Direct Connect connection to an enterprise geodatabase, including the Esri
                                                                                                                                  command-line tools (distributed with and without the application server components)
                                                                                                                                  and custom applications available elsewhere.  All it takes to make that connection is
                                                                                                                                  the Direct Connect DLL appropriate to the database release and the database DLLs
                                                                                                                                  necessary for the Direct Connect DLL to connect to the database (both are included
                                                                                                                                  within the ArcGIS Server install, though you need to unpack some database client
                                                                                                                                  libraries according to the instructions in order to use them).

                                                                                                                                  - V
                                                                                                                                  • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                    Wolseley
                                                                                                                                    You can uninstall the 10.1 SP1 ArcSDE application server software if you are not using it.
                                                                                                                                    Just because the 10.x installer uses Direct Connect, and some clients may be using Direct
                                                                                                                                    Connect, doesn't mean that all clients are using Direct Connect.  If you never configured
                                                                                                                                    your application server, then no one could use it, but if you did (e.g. there exists a running
                                                                                                                                    service), then it would be wise to stop the service before removing it, just to see if anything
                                                                                                                                    was using it.

                                                                                                                                    ArcGIS Server is an ArcSDE client application.  That's all it ever takes for any application
                                                                                                                                    to claim that it has a "component of ArcSDE" installed.  All ArcSDE clients have the ability
                                                                                                                                    to make a Direct Connect connection to an enterprise geodatabase, including the Esri
                                                                                                                                    command-line tools (distributed with and without the application server components)
                                                                                                                                    and custom applications available elsewhere.  All it takes to make that connection is
                                                                                                                                    the Direct Connect DLL appropriate to the database release and the database DLLs
                                                                                                                                    necessary for the Direct Connect DLL to connect to the database (both are included
                                                                                                                                    within the ArcGIS Server install, though you need to unpack some database client
                                                                                                                                    libraries according to the instructions in order to use them).

                                                                                                                                    - V



                                                                                                                                    Thank you Mr. Angelo! I uninstalled ArcSDE 10.1.1 my SQL Server Enterprise Geodatabase connections still work. All is well. I FINALLY understand now. I get it. Turns out I was already using 2-Tier (Direct Connect).
                                                                                                                                    • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                      mboeringa2010
                                                                                                                                      All is well. I FINALLY understand now. I get it. Turns out I was already using 2-Tier (Direct Connect).


                                                                                                                                      Good, and I really hope you had a good look at the PDFs I created and pointed out to you in the other thread, as they graphically show what Vince has written in his last posts here (and many before in other threads).

                                                                                                                                      Direct Connect still uses the 'C' API to interface with the RDBMS. Post-10.2,
                                                                                                                                      Esri will no longer distribute the API SDK, but they will continue to use it, so it
                                                                                                                                      really won't go away, just get pushed deeper into the software stack.


                                                                                                                                      Not unexpected, but another nice insight you are sharing here for people to learn from...

                                                                                                                                      The application server component was always the thinnest and least significant
                                                                                                                                      part to wear an "ArcSDE" label, but for whatever reason, became the most
                                                                                                                                      famous
                                                                                                                                      -- this may be fueling the confusion (and the outrageous length of
                                                                                                                                      this thread)
                                                                                                                                      .


                                                                                                                                      So true...

                                                                                                                                      Best,

                                                                                                                                      Marco
                                                                                                                                        • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                          powerplanner

                                                                                                                                          So I am a single user 10.1, and don't share my files often or work in groups.

                                                                                                                                          Are  there any reason why I would use SDE for my work?  I do use large databases, i.e. several gigabytes.  Is there any speed enhancements for queries using SDE?

                                                                                                                                            • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                              vangelo-esristaff

                                                                                                                                              First of all, you need to understand that there is no longer any such thing as "SDE".  The technology was renamed "ArcSDE" at ArcGIS 8.0, and even "ArcSDE" isn't available as a product anymore (as of ArcGIS 9.2).  The current term of art is "enterprise geodatabase,"  which encompasses all RDBMS, Object-Relational, and NoSQL storage of spatial data, as accessed by ArcGIS through a Database Connection.

                                                                                                                                               

                                                                                                                                              There may or may not be performance benefits from storing spatial data in a enterprise geodatabase.  Back in the infancy of SDE, some marketing reps were selling it as a "spatial data accelerator",  but that practice has been stopped.  There are benefits, but there are additional costs as well, starting with database installation and maintenance.  Shapefile and File Geodatabase formats will still generally outperform an RDBMS engine on a "draw all features" query, precisely because of the principal benefits of a transactional RDBMS (Atomicity, Consistency, Isolation, & Durability).  Enterprise databases usually allow for more sophisticated queries (because of the SQL language, or lack thereof) and more exotic indexing options, which often result in better performance with small query results from large tables.

                                                                                                                                               

                                                                                                                                              "Several gigabytes" of data might not reach the conventional definition of "large" if you have a large number of wide string columns.  The definition is also somewhat elastic, since 200,000 complex polygons might be large while 500,000 points might not.  Certainly tens of millions of features would reach "large" or even "very large".  Mostly it's a matter of how you need to manage the data to do something useful with it.

                                                                                                                                               

                                                                                                                                              In order to help you decide whether an enterprise geodatabase would be of a benefit to you, you'd need to provide quite a bit more information on the numbers and kinds of tables, with row counts and width indicators, and the kinds of operations which are commonly performed against your dataset.

                                                                                                                                               

                                                                                                                                              - V

                                                                                                                                          • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                            filipkral

                                                                                                                                            Hi, I am glad to see somebody brought this up.

                                                                                                                                             

                                                                                                                                            I was so pleased when Esri announced that ArcSDE will be part of both Desktop and Server products since 9.2 but to this day I haven't been able to use it the way I expected on Desktop. Vince Angelo, could you please explain how or to what extent is ArcSDE technology available in ArcGIS for Desktop?

                                                                                                                                             

                                                                                                                                            I thought I would be able to, for example, turn my PostGIS database on localhost into an "enterprise geodatabase" and use it fully in ArcGIS while I could still benefit from full SQL (SELECT) flexibility in pgAdmin or through ArcSDESQLExecute tool.

                                                                                                                                             

                                                                                                                                            However, whenever I try to set up a geodatabase on a proper RDBMS, it turns out that actually I cannot do it with ArcGIS for Desktop alone and that I need ArcGIS for Server to do that. For example Enable Enterprise Geodatabase tool requires some kind of authorization file from ArcGIS for Server Enterprise.

                                                                                                                                             

                                                                                                                                            So is it possible to create a geodatabase in a proper RDBMS with ArcGIS for Desktop only or not?

                                                                                                                                            Regards,

                                                                                                                                            Filip.

                                                                                                                                            • Re: Why would I install ArcSDE with 10.1?
                                                                                                                                              kyle969

                                                                                                                                              I wouldn't, we ditched ArcSDE, direct connections are much faster.