Migrate Content for ArcGIS Enterprise/ArcGIS Online

[#1]

It's always a good practice to have a DEV/STAGING environment along with a PRODUCTION environment for ArcGIS Enterprise. I frequently get asked how to migrate content from one to the other.  Hosted services can be straight forward, and can easily be scripted (expect an update later to this toolset for this functionality), but referenced services can be rather difficult. For example, there are numerous prerequisites that are required for a service to be published from an Enterprise Geodatabase, such as:

• does the ArcGIS Service Account have necessary privileges
• is the geodatabase registered with the ArcGIS Server instance
• does the feature class in fact exist within the new geodatabase
• etc

These tools pick up once your services have been published to the other environment and you would like to begin migrating web maps, apps, and dashboards. Take a look at the video below on how to execute these tools.

These tools should only be used to migrate web maps/apps where the environments are the same version (i.e. 11.1).  Migrating content between two ArcGIS Online organizations is also supported.

Currently, these tools have only been tested in ArcGIS Enterprise 11.1 and ArcGIS Online. If there are any issues, please report in the comment section below.

 

Updates

1/16/24:  Added tools to copy hosted feature services and file based items.

1/29/24:  Added option to run when portal has Windows Authentication enabled.  Also, updated the Copy Dashboard tool to specify stand-alone layers.

2/12/24:  Web Map item IDs now update

3/28/24:  Added tool to copy Story Maps

Known Issues

• If hosted feature services exist in Enterprise, they will not successfully overwrite when executing the Copy Hosted Feature Services tool.  Will successfully overwrite in AGOL.  Possible bug with ArcGIS API for Python
• After Story Map is copied to Enterprise, the Story Map must first be opened in Edit mode and published before it can be successfully viewed.  This is not the case for AGOL.  Possible bug with ArcGIS API for Python

[DarinaTchountcheva]

Hello Jake,

Thank you so much for this solution! You are a life saver! 

I was just going to tackle the task of deploying dashboards from dev to prod and your post popped up. It saved me so much effort and time. Thank you!

Here are my 2 cents:

I have tried the Copy Web Map and Copy Dashboard tools. 

We use Integrated Windows Authentication and an error popped up while filling out the parameters that connection to the portal cannot be made. 

I changed the validation code to not provide user and password as arguments, as they are assumed from the user currently logged in and it worked: 

source = GIS(self.params[0].value)

target = GIS(self.params[6].value)

I made the same changes in all .py scripts:

source = GIS(sourcePortal)

target = GIS(targetPortal)

And I was able to copy a web map and a dashboard that has a map widget. 

A lot of our dashboards don't have a map widget, and the tool cannot be used, but I was able to use your script to get the JSON of the original dashboard, change the item IDs of the source to the target ones and then use the second part of the script to copy the dashboard to the target portal. And it worked.

Thank you again for your priceless contribution!

Darina

[JakeSkinner]

@DarinaTchountcheva thank you for this valuable feedback.  I've updated the tools to include options for when Windows Authentication is enabled for Portal, and also updated the Copy Dashboard tool to include updating the item IDs of stand-alone layers.

[DarinaTchountcheva]

@Jake you are truly amazing! 

Thank you so much for taking the time to make the modifications. I and a lot of other users will benefit greatly from it and will be more efficient. 

I will let you know how it worked as soon as I get a chance to test it. 

Best regards,

Darina

[Eliot_Reid]

@JakeSkinner 

I'm trying to use these tools with an IWA integrated portal and an AGOL. I'm getting the following error when I use my portal in the Source URL and check 'Source Portal Windows Authentication Enabled'. I've tried with my Pro signed into my portal and out of my portal, as well as with different Python environments. My user does have admin rights to the Portal. Do you have any thoughts? Thanks!

Error 032659 updateParameters Error: Traceback (most recent call last):
  File "Q:\admin\Template\expressions\ESRI_ScriptsToolboxs\Migrate Content\Migrate Content.atbx\CopyWebMap.tool\tool.script.validate.py", line 203, in <module>
  File "Q:\admin\Template\expressions\ESRI_ScriptsToolboxs\Migrate Content\Migrate Content.atbx\CopyWebMap.tool\tool.script.validate.py", line 109, in updateParameters
  File "Q:\admin\Template\expressions\ESRI_ScriptsToolboxs\Migrate Content\Migrate Content.atbx\CopyWebMap.tool\tool.script.validate.py", line 40, in usersDropDown
  File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\__init__.py", line 4943, in search
    users = self._portal.get_org_users(
  File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 1244, in get_org_users
    resp = self._org_users_page(
  File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 2922, in _org_users_page
    return self.con.post("portals/self/users", postdata)
  File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1524, in post
    return self._handle_response(
  File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1000, in _handle_response
    self._handle_json_error(data["error"], errorcode)
  File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1023, in _handle_json_error
    raise Exception(errormessage)
Exception: You do not have permissions to access this resource or perform this operation.
(Error Code: 403)

[JakeSkinner]

@Eliot_Reid 

Open the python console, or new Notebook in Pro by going to Analysis tab > Python dropdown:

Enter the following code, replacing https://gis.esri.com/portal with your URL to portal:

from arcgis.gis import GIS
gis = GIS('https://gis.esri.com/portal')
source_users = gis.users.search(max_users=10000)
source_users

 Ex:

Does this execute successfully?

[Eliot_Reid]

It does not, same error. 

I just double checked and our Portal security actually says SAML is set up, not IWA. I assumed they were the same thing. I'm not sure if they work the same.

[JakeSkinner]

@Eliot_Reid I had a feeling this may be the case.  SAML is not supported.  If you can, create a built-in Administrator account and use that instead.