Enterprise (ArcGIS?) Geodatabase on Azure SQL

6487
12
08-02-2016 02:45 PM
JeffGarland_AGOL
New Contributor II

Trying to get some clarification.  Can you have an enterprise geodatabase on an Azure SQL DB.  (Note this is not a SQL Server instance running on an Azure VM (IaaS), but MS Azure SQL (PaaSP.)   This documentation seems to indicate the answer is "no".   (See first bullet under the header "Database requirements\limitations". I assume "ArcGIS Geodatabase" translates to Enterprise Geodatabase.)  This documentation seems to indicate "yes".  Note the text in the gray box, "...These 10.4.1 geodatabases are in a beta state, and are for use only with your GIS server or web GIS deployment on Azure.

What does the latter part mean?  Can I edit the feature classes in an EGDB on Azure SQL with Desktop?  Is versioning supported?  Are roles supported?  Or is it just a "managed database" for ArcGIS Server.  Any clarification from an Esri product member would be greatly appreciated.

12 Replies
ChetDobbins
Esri Contributor

Currently Microsoft Azure SQL Database have not been certified for full geodatabase functionality. We are in the process of extending geodatabase support to this platform and hope to do so in the next major release. You can still create and work with geodatabase functionality in a test-only, or beta capacity but they are not yet supported in a full production environment.

I would also like to mention that due to some limitations with Azure SQL Database v11, enterprise geodatabases will only be certified on v12 instances.

JeffGarland_AGOL
New Contributor II

Thanks for the reply.  To extend the initial question a bit further.  Is the design intent to be able to access a geodatabase hosted on Azure SQL DB with an on premised ArcGIS Desktop client? Reading through the documentation for ArcGIS Server hosted on AWS it is quite clear that the geodatabase needs to replicated from on premise to the cloud.  Will that be the case for Azure SQL DB or can one create a hybrid scenario as described above (production geodatabase in Azure SQL with access and management by on premise ArcGIS Deskop client)?

MicahCallough2
New Contributor

Any answers on this one?

ZacharyHart
Occasional Contributor III

I'd like to hear more about this scenario too!

EllenNodwell_IntegraShare
Occasional Contributor

We set-up the whole ArcGIS Enterprise 10.5 in Azure on VM's - two separate environments - for Production and for Staging.  Open the image in a new tab to see this better. 

In Azure, we have two SQL-Server VM's, two ArcGIS Server VM's, and two VM's for Portal.  We also have two multi-user VM's to support ArcGIS Desktop.  The servers' OS 2012 R2, the RDP's are 2016.  We are running SQL Server 13.0.4446.0.

The file shares are mixed on-premise and Azure storage shares - all networked in the organization's WAN. We have our main GIS data share located on one of the ArcGIS Server VM's where extra storage was added.  It functions like any other share drive, but is faster in the Azure environment.  On premise non-Azure workstations utilize that also, as well as connect to SDE, but latency makes it slow so most users of Desktop use the RDP's.

The integration with non-Azure networked servers using ADFS with SSO is developing - we are in the middle of this conversion.  In our staging environment, this went smoothly. 

The Azure experience is very much the same feel and function as working with on-premise servers, when managed through the multi-user VM used for ArcGIS Desktop apps - these are desktops and have everything there - Microsoft Office, FME, Adobe, etc. essentially, ArcGIS plus all the support tools. 

Because it is all connected inside the firewall, for the user, and the administrators, it is just understanding the addressing and permissions settings.  Sometimes the addressing just has to be a certain way for making connections. 

Portal being the control mechanism for users takes adjustment - some aspects of the user-set up process had to be adjusted. 

We begin with setting up a new user, assigning them to their appropriate user-groups using Active Directory;  The user is assigned to a given user-group to give them the correct permissions.  We set up the user first in the Active Directory GIS user-group, then we pick them off the list that is generated in Portal, to add them to Portal, which gives them the appropriate read-write permissions.  We have limited "Level 2" users - most are read-only. 

We set up the databases in SQL Server first.  We set-up several SDEs (enabled geodatabases) of those databases in SQL-Server using the tools in the ArcGIS Desktop interface.

I have one running in my own shop that is not Azure, and see little difference in the management using ArcGIS Desktop comparing non-Azure to Azure.

The upgrade path, has been craggy on the 10.5 to 10.5.1, as when we used Cloudbuilder back in January, initially in Azure, we had issues and this required manual intervention with Esri Support on the line.  We did a teardown and reinstall two more times. 

The key is getting Azure setup optimally ahead of deployment. 

We hear that Cloudbuilder has been improved in its latest iteration.  Because we had a "broken" Cloudbuilder implementation, we were told by Esri Support that we cannot use Cloudbuilder for the upgrades.  We have a lot of content out there that we now must preserve.

The joy of Azure is the capability to do a lot of testing, then wadding up and throwing in the trash what does not work and rebuilding the improved version, leveraging what you learned the first time, and the second...and maybe the third! 

As with ArcGIS architecture strategy in general, there are many variations. 

Recommend starting with doing your system architecture diagram "on paper" (or whiteboard) (or Visio or some sort of visual tool) and use design strategies in the WIKI for the latest greatest info.

As of this writing, we have a functional enterprise ArcGIS 10.5 running in our production environment.  We have an upgraded one (10.5.1) in our staging environment.  That was painful, that upgrade.  Web adaptor settings, security certificates, and other issues arose for our sysadmin who is not a GIS guy, but now has a lot more experience than he did when we began this adventure.

Our GIS users love all of this functionality, and are leveraging it more and more, especially the Portal.  We do SDE updates and the Portal maps are always kept up to date.  Pretty much run of the mill data updating going on at present, but, more integration with data warehouse and other "fancier" streams - all coming ahead.

After visiting with a couple of the Azure Team in San Diego, I got the impression that we were pushing the envelope on some of what we are doing.

Good luck with your Azure ArcGIS adventures!

LourdesSuman
New Contributor II

Good morning Ellen.  Thanks for your post is very interesting your experience.

I'm trying to configure the ArcGIS desktop on-premise to connect to Azure SQL but we confront a lot of latancy.  Do you resolve the latancy problem?  

My architecture is: 1 ArcGIS server 10.5 in azure with all tools installed, sql 2016 instance in azure and ArcGIS map desktop on-premise, approximately 62 users with the desktop.  Is so hard try to work with the desktop, one map that take 3 or 5 minutes to be created is taking approximately 15 minutes.   Could you share your experience with the solution of the connection problem with me?

Regards,

0 Kudos
EllenNodwell_IntegraShare
Occasional Contributor

Lourdes,

We set up two multi user desktops in Azure using a configuration that enables users to login through remote access and effectively work there, all in Azure, to resolve the latency issues. We heard that Esri were working on an optimized Azure environment for Desktop, but we couldn’t wait and tried this set-up. I will post more if you are interested in the details. 

For one user the Azure configuration we used could be expensive, but for > 2, up to 5, we think that it works. We are still optimizing for Pro. Pro uses GPU that is pretty high dollar for average use, but would be required for rendering a lot of points - we just have not yet determined the tipping point - we are still working on this. 

All in all, however, if all work is done in Azure, it all works well.  If the hybrid approach of data in Azure and tools are on PC’s on premises then bandwidth makes it unpredictable. Web services do work pretty well, so we do serve data that way to on premises machine users successfully. Our users like Portal maps and apps, so that is another option for them which is on the up trend.

Keeping in mind that your Azure options are location dependent, this also may have an impact. Our IT crew has been working with Microsoft to locate their Azure services to a closer farm. You might speak with your Microsoft folks about latency, as well. 

Let me know if you need more info. 

LourdesSuman
New Contributor II

Hi Ellen, thanks a lot for your reply.   Could you share what kind of configuration you did to enable multi-user remote desktop logins on VM and let me know what kind of ArcGIS license we need to use.

Regards,

0 Kudos
EllenNodwell_IntegraShare
Occasional Contributor

Hi Lourdes, 

Each of the workstation VM's are set up on the NC6 Standard (Azure package name). Each has 6 cores, 56 GB memory, 8 data disks, 8X500 Max IOPS, 380 GB local SSD, has load balancing, and for now we have a K80 Graphics Card.  We have tried the upgrade, but it made no difference to the ArcGIS Pro rendering issue that took up all the GPU and threw an error, making the map look kind of weird with as much data as we were trying to show - which IMHO was not that much data.  That being said, the users are not yet ready to switch to ArcGIS Pro because the applications and extensions are not even ready for using 10.5, in some cases - but, that is another discussion.

If you click on the graphic in my original post, you should be able to see the image better, which on the right side, in that box, has the configuration of the workstation VM's as stated above. 

We have two of them set up to support up to 5 users each - however, as I may have said, if one "heavy" user takes up a lot of memory, doing processing, this could affect other users; most of the time, with relatively normal use of ArcMap, doing mapping, we don't have a problem.  We have two workstations set up due to the limitation of ArcGIS Administrator in enabling the user to switch license levels once the user is logged on and opens ArcMap - because it is "one machine" to the Administrator, it does not enable the second user, or even the first user to switch levels.  We spoke with Esri about this, and they are aware, but told us that a fix was not on their update list at that time.  We therefore have one workstation VM set for Standard users, and one set for Advanced users. 

For their storage, we have a share drive set up that originates from the "prod" ArcGIS Server VM, which works well and we suffer little latency, using that arrangement. 

Again, the users have the option of using their own on premise workstation if they have a FGDB that they are using, on an on premise file share, or on their own machine, but most prefer to go "all cloud" and just use remote access to gain access to work in 100% Azure.  It's quick and they have a lot of resources "up there".

As an administrator, using Azure is noticeably not that different from managing the "on premise" environment.  You just get used to it after awhile!

We run our SDE on SQL Server.  The specs are in my original post.  We have 4 TB storage on the ArcGIS Server VM's.  

0 Kudos