Unable to upgrade cloned environment at 3.2

[SFM_TravisBott]

I upgraded to 3.2 in order to access the upgrade arcgis module. However I can't seem to upgrade my previous environment. Is it necessary to create a new clone?

> "C:\Program Files\ArcGIS\Pro\bin\Python\Scripts\conda-proup.exe" --prefix "C:\Users\tbott\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone" --force  --backup "C:\Users\tbott\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone.tar.bz2"


Collecting packages...
Upgrading environment at 'C:\\Users\\tbott\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone' with backup at C:\Users\tbott\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone.tar.bz2
Detected packages added to environment:
  conda: tabulate geopandas
Traceback (most recent call last):
  File "conda_proup\core.py", line 580, in backup
  File "conda_pack\core.py", line 561, in pack
  File "conda_pack\core.py", line 396, in pack
  File "conda_pack\core.py", line 1095, in add
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\tbott\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone\\Scripts/normalizer.exe'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "conda_proup\cli.py", line 64, in main
  File "conda_proup\core.py", line 1082, in upgrade
  File "conda_proup\core.py", line 997, in upgrade
  File "conda_proup\core.py", line 583, in backup
Exception: Unable to backup original environment, stopping upgrade.
Backing up environment to C:\Users\tbott\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone.tar.bz2
Collecting packages...
Packing environment at 'C:\\Users\\tbott\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone' to 'C:\\Users\\tbott\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone.tar.bz2

I'm getting a 'Permission denied.' I don't have admin rights on my machine but this should be an issue. I was able to clone the environment previously and manipulate packages as necessary. 

Attempting to do so at the command line doesn't seem to like the syntax of the provided code. 

[Brian_Wilson]

Well, it's saying it cannot write this file, so starting there,

C:\\Users\\tbott\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone.tar.bz2

The first thing I would do is make sure there is not already a file there by that name and if there is, delete it. If you can't delete it, try to rename it. Make sure you can write into that folder too (e.g. create a doc and delete it)

The proup tool does a tar backup of an existing env, gets a list of packages installed in it, nukes the existing env and then builds a new one with the same name and adds your packages into it.

I am not comfortable letting Esri's closed source tool nuke environments that I built. It's slow, and I can't see what it's doing (except when it makes a mistake and crashes.) (Too bad they took away the open source version.) (Is there a "prodown" tool to run if the proup tool leaves you with no working env??)

My thought is you can very easily build a new clean environment, test it, and if it works, and you need space, then nuke the old one once you are sure the new one works and you don't need the old one any more.

conda create -c esri -c forge -n pro32 python=3.9 arcpy arcgis
conda activate pro32

Add more tools, for example

conda install geopandas

Of course if there are other packages you installed yourself such as geopandas so you'd need to do that. I keep conda environments with each of the projects I use them with and document what packages I need to add.

[SFM_TravisBott]

@Brian_Wilson Thanks for the reply. 

I did check to see if that file already exists (I promise I read my error message before posting this!). None existed. I can also write to this location (which is a concern, as I do not have full administrative rights on my machine). 

I created a new clone with the exact same name/location as the environment I can't update using the Pro tool, wondering if it would overwrite it. The cloning seemed to work fine and I can now interact with that environment using Pro's GUI. However, the crux package that I needed updated (arcgis, to 2.2) doesn't appear to be updated. 

[Brian_Wilson]

Ah interesting. I picked an old environment and tried "conda update arcgis" and it said

The following packages will be UPDATED:

  arcgis                                    2.1.0.2-py39_14 --> 2.1.0.3-py39_10
  ca-certificates                      2023.7.22-h56e8100_0 --> 2023.11.17-h56e8100_0
  certifi                            2023.7.22-pyhd8ed1ab_0 --> 2023.11.17-pyhd8ed1ab_0

That's disappointing.  Also the fact that it wants to downgrade the Python package. I tried this,

conda install  'arcgis>=2.2'

it gave it its best shot but failed. I think what happens is that in a given environment, python is pinned. That means it can't be updated to a newer version. I tried the version that installed with my "create" command.

conda install python=3.9.18

This caused it to complain about openssl versions, so I thought maybe I can remove the errant package with "conda remove openssl", that triggered downgrades to things like arcpy 1.6, I pressed on in the interest of science, and there were too many other errors and incompatibilities. It could not go on.

Just build a fresh environment, it's easy!

I am learning why the tools that Esri provides have such a hard time doing what we want. They'd often be better off just writing docs for the standard conda tools than trying to hide the complexity inside blackboxes.

[SFM_TravisBott]

Indeed. Something else screwy is going on and I lack the savvy to easily get at it. 

I created a new environment which has a near-identical path to the old one. When I go to select it as my new python interpreter in VS code I'm warned that it's 'identified as a Conda environment even though Conda not found.' But earlier in the initialization it finds its way to 

~\.conda\Scripts\conda.exe info --json

just fine. 

[SFM_TravisBott]

Adding another one....

So I nuked the old environment and created a new one. All is well. However, now in VS code I can't get it to recognize the new environment. 

I've been attempting to follow the steps outlined by you (Brian) here but I can't get I can't get the command prompt to recognize conda.

[Brian_Wilson]

I've succeeded in breaking my setup while I look at this. 🙂 Hmmm...

[Jen_Zumbado-Hannibal]

I'm having the same issues. I cloned my environment and then proceeded to 'Update All' but it didn't work. 

Thus, I had to do the updates one by one. Yet, there are a few with a 'Lock' symbol (e.g., arcgis, matplotlib, notebook, etc.) that refuse to update.

 

For those of you running Python scripts (ArcPro 3.2.1) on Task Scheduler, don't forget to clone your new environment so that your tasks can run as desired. 

Here are examples of the updates: 

- azure-core
The following packages will be downloaded:
package | build
---------------------------|-----------------
azure-core-1.29.5 | py39haa95532_0 197 KB
brotli-python-1.0.9 | py39hd77b12b_7 309 KB
cffi-1.16.0 | py39h2bbff1b_0 242 KB
cryptography-41.0.7 | py39h89fc84f_0 1.1 MB
typing-extensions-4.7.1 | py39haa95532_0 10 KB
typing_extensions-4.7.1 | py39haa95532_0 57 KB
urllib3-1.26.18 | py39haa95532_0 198 KB
------------------------------------------------------------
Total: 2.1 MB
The following NEW packages will be INSTALLED:
brotli-python pkgs/main/win-64::brotli-python-1.0.9-py39hd77b12b_7
The following packages will be UPDATED:
azure-core esri/noarch::azure-core-1.12.0-py_0 --> pkgs/main/win-64::azure-core-1.29.5-py39haa95532_0
cffi 1.15.1-py39h2bbff1b_3 --> 1.16.0-py39h2bbff1b_0
cryptography esri::cryptography-41.0.3-py39_2 --> pkgs/main::cryptography-41.0.7-py39h89fc84f_0
six esri::six-1.16.0-py_0 --> pkgs/main::six-1.16.0-pyhd3eb1b0_1
typing-extensions 4.3.0-py39haa95532_0 --> 4.7.1-py39haa95532_0
typing_extensions 4.3.0-py39haa95532_0 --> 4.7.1-py39haa95532_0
urllib3 1.26.16-py39haa95532_0 --> 1.26.18-py39haa95532_0
The following packages will be SUPERSEDED by a higher-priority channel:
win_inet_pton esri::win_inet_pton-1.1.0-py39_0 --> pkgs/main::win_inet_pton-1.1.0-py39haa95532_0
Downloading and Extracting Packages
urllib3-1.26.18 | 198 KB | | 0%
urllib3-1.26.18 | 198 KB | 8 | 8%
urllib3-1.26.18 | 198 KB | ########## | 100%
urllib3-1.26.18 | 198 KB | ########## | 100%
brotli-python-1.0.9 | 309 KB | | 0%
brotli-python-1.0.9 | 309 KB | ########## | 100%
brotli-python-1.0.9 | 309 KB | ########## | 100%
typing_extensions-4. | 57 KB | | 0%
typing_extensions-4. | 57 KB | ########## | 100%
typing-extensions-4. | 10 KB | | 0%
typing-extensions-4. | 10 KB | ########## | 100%
cffi-1.16.0 | 242 KB | | 0%
cffi-1.16.0 | 242 KB | ########## | 100%
cffi-1.16.0 | 242 KB | ########## | 100%
azure-core-1.29.5 | 197 KB | | 0%
azure-core-1.29.5 | 197 KB | 8 | 8%
azure-core-1.29.5 | 197 KB | ########## | 100%
azure-core-1.29.5 | 197 KB | ########## | 100%
cryptography-41.0.7 | 1.1 MB | | 0%
cryptography-41.0.7 | 1.1 MB | ########## | 100%
cryptography-41.0.7 | 1.1 MB | ########## | 100%
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Retrieving notices: ...working... done

****************************
- azure-storage-blob
The following packages will be downloaded:
package | build
---------------------------|-----------------
azure-storage-blob-12.19.0 | py39haa95532_0 406 KB
blinker-1.6.2 | py39haa95532_0 29 KB
isodate-0.6.1 | py39haa95532_0 50 KB
msrest-0.7.1 | py39haa95532_0 106 KB
oauthlib-3.2.2 | py39haa95532_0 209 KB
pyjwt-2.4.0 | py39haa95532_0 38 KB
------------------------------------------------------------
Total: 839 KB
The following packages will be UPDATED:
azure-storage-blob esri/noarch::azure-storage-blob-12.8.~ --> pkgs/main/win-64::azure-storage-blob-12.19.0-py39haa95532_0
blinker 1.4-py39haa95532_0 --> 1.6.2-py39haa95532_0
isodate esri/noarch::isodate-0.6.0-py_0 --> pkgs/main/win-64::isodate-0.6.1-py39haa95532_0
msrest esri/noarch::msrest-0.6.21-py_0 --> pkgs/main/win-64::msrest-0.7.1-py39haa95532_0
oauthlib esri::oauthlib-3.2.0-py39_0 --> pkgs/main::oauthlib-3.2.2-py39haa95532_0
The following packages will be SUPERSEDED by a higher-priority channel:
pyjwt esri/noarch::pyjwt-2.4.0-py_1 --> pkgs/main/win-64::pyjwt-2.4.0-py39haa95532_0
Downloading and Extracting Packages
msrest-0.7.1 | 106 KB | | 0%
msrest-0.7.1 | 106 KB | #5 | 15%
msrest-0.7.1 | 106 KB | ########## | 100%
msrest-0.7.1 | 106 KB | ########## | 100%
isodate-0.6.1 | 50 KB | | 0%
isodate-0.6.1 | 50 KB | ###1 | 32%
isodate-0.6.1 | 50 KB | ########## | 100%
oauthlib-3.2.2 | 209 KB | | 0%
oauthlib-3.2.2 | 209 KB | 7 | 8%
oauthlib-3.2.2 | 209 KB | ####5 | 46%
oauthlib-3.2.2 | 209 KB | ########## | 100%
oauthlib-3.2.2 | 209 KB | ########## | 100%
pyjwt-2.4.0 | 38 KB | | 0%
pyjwt-2.4.0 | 38 KB | ########## | 100%
azure-storage-blob-1 | 406 KB | | 0%
azure-storage-blob-1 | 406 KB | 3 | 4%
azure-storage-blob-1 | 406 KB | ###9 | 39%
azure-storage-blob-1 | 406 KB | ######3 | 63%
azure-storage-blob-1 | 406 KB | ########2 | 83%
azure-storage-blob-1 | 406 KB | ########## | 100%
azure-storage-blob-1 | 406 KB | ########## | 100%
blinker-1.6.2 | 29 KB | | 0%
blinker-1.6.2 | 29 KB | ########## | 100%
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Retrieving notices: ...working... done

****************************
- black
The following packages will be downloaded:
package | build
---------------------------|-----------------
backcall-0.2.0 | pyhd3eb1b0_0 13 KB
black-23.11.0 | py39haa95532_0 289 KB
click-8.1.7 | py39haa95532_0 164 KB
exceptiongroup-1.0.4 | py39haa95532_0 29 KB
ipython-8.15.0 | py39haa95532_0 1.1 MB
jedi-0.18.1 | py39haa95532_1 982 KB
mypy_extensions-1.0.0 | py39haa95532_0 12 KB
packaging-23.1 | py39haa95532_0 77 KB
pathspec-0.10.3 | py39haa95532_0 49 KB
platformdirs-3.10.0 | py39haa95532_0 35 KB
prompt-toolkit-3.0.36 | py39haa95532_0 565 KB
prompt_toolkit-3.0.36 | hd3eb1b0_0 5 KB
pygments-2.15.1 | py39haa95532_1 1.7 MB
regex-2023.10.3 | py39h2bbff1b_0 326 KB
stack_data-0.2.0 | pyhd3eb1b0_0 22 KB
traitlets-5.7.1 | py39haa95532_0 205 KB
------------------------------------------------------------
Total: 5.5 MB
The following NEW packages will be INSTALLED:
prompt-toolkit pkgs/main/win-64::prompt-toolkit-3.0.36-py39haa95532_0
The following packages will be UPDATED:
black esri::black-22.1.0-py39_3 --> pkgs/main::black-23.11.0-py39haa95532_0
click 8.0.4-py39haa95532_0 --> 8.1.7-py39haa95532_0
exceptiongroup esri/noarch::exceptiongroup-1.0.0rc9-~ --> pkgs/main/win-64::exceptiongroup-1.0.4-py39haa95532_0
ipython esri/noarch::ipython-7.34.0-pyhbfea03~ --> pkgs/main/win-64::ipython-8.15.0-py39haa95532_0
mypy_extensions 0.4.3-py39haa95532_1 --> 1.0.0-py39haa95532_0
packaging pkgs/main/noarch::packaging-21.3-pyhd~ --> pkgs/main/win-64::packaging-23.1-py39haa95532_0
pathspec esri/noarch::pathspec-0.9.0-py_0 --> pkgs/main/win-64::pathspec-0.10.3-py39haa95532_0
platformdirs 2.5.2-py39haa95532_0 --> 3.10.0-py39haa95532_0
prompt_toolkit esri::prompt_toolkit-3.0.5-py_0 --> pkgs/main::prompt_toolkit-3.0.36-hd3eb1b0_0
pygments esri/noarch::pygments-2.14.0-py_0 --> pkgs/main/win-64::pygments-2.15.1-py39haa95532_1
regex 2022.7.9-py39h2bbff1b_0 --> 2023.10.3-py39h2bbff1b_0
traitlets esri/noarch::traitlets-5.5.0-py_0 --> pkgs/main/win-64::traitlets-5.7.1-py39haa95532_0
The following packages will be SUPERSEDED by a higher-priority channel:
backcall esri --> pkgs/main
jedi esri/noarch::jedi-0.18.2-py_1 --> pkgs/main/win-64::jedi-0.18.1-py39haa95532_1
stack_data esri::stack_data-0.5.1-py_0 --> pkgs/main::stack_data-0.2.0-pyhd3eb1b0_0
Downloading and Extracting Packages
jedi-0.18.1 | 982 KB | | 0%
jedi-0.18.1 | 982 KB | 1 | 2%
jedi-0.18.1 | 982 KB | ########## | 100%
jedi-0.18.1 | 982 KB | ########## | 100%
packaging-23.1 | 77 KB | | 0%
packaging-23.1 | 77 KB | ########## | 100%
packaging-23.1 | 77 KB | ########## | 100%
pygments-2.15.1 | 1.7 MB | | 0%
pygments-2.15.1 | 1.7 MB | ########## | 100%
pygments-2.15.1 | 1.7 MB | ########## | 100%
pathspec-0.10.3 | 49 KB | | 0%
pathspec-0.10.3 | 49 KB | ########## | 100%
pathspec-0.10.3 | 49 KB | ########## | 100%
backcall-0.2.0 | 13 KB | | 0%
backcall-0.2.0 | 13 KB | ########## | 100%
backcall-0.2.0 | 13 KB | ########## | 100%
stack_data-0.2.0 | 22 KB | | 0%
stack_data-0.2.0 | 22 KB | ########## | 100%
ipython-8.15.0 | 1.1 MB | | 0%
ipython-8.15.0 | 1.1 MB | ########## | 100%
ipython-8.15.0 | 1.1 MB | ########## | 100%
platformdirs-3.10.0 | 35 KB | | 0%
platformdirs-3.10.0 | 35 KB | ########## | 100%
mypy_extensions-1.0. | 12 KB | | 0%
mypy_extensions-1.0. | 12 KB | ########## | 100%
mypy_extensions-1.0. | 12 KB | ########## | 100%
prompt-toolkit-3.0.3 | 565 KB | | 0%
prompt-toolkit-3.0.3 | 565 KB | ########## | 100%
prompt-toolkit-3.0.3 | 565 KB | ########## | 100%
click-8.1.7 | 164 KB | | 0%
click-8.1.7 | 164 KB | ########## | 100%
click-8.1.7 | 164 KB | ########## | 100%
regex-2023.10.3 | 326 KB | | 0%
regex-2023.10.3 | 326 KB | ########## | 100%
black-23.11.0 | 289 KB | | 0%
black-23.11.0 | 289 KB | ########## | 100%
black-23.11.0 | 289 KB | ########## | 100%
prompt_toolkit-3.0.3 | 5 KB | | 0%
prompt_toolkit-3.0.3 | 5 KB | ########## | 100%
traitlets-5.7.1 | 205 KB | | 0%
traitlets-5.7.1 | 205 KB | ########## | 100%
traitlets-5.7.1 | 205 KB | ########## | 100%
exceptiongroup-1.0.4 | 29 KB | | 0%
exceptiongroup-1.0.4 | 29 KB | ########## | 100%
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Retrieving notices: ...working... done

[SFM_TravisBott]

I had to upgrade ArcGIS Pro and then create a new clone. It wouldn't let me upgrade the cloned environment afterwards. I stopped investigating as this fixed it and I could move on.