Conda cloned environment has problems with GDAL

986
2
Jump to solution
11-02-2023 05:42 AM
Dumbatz
New Contributor II

Hello all,

I have a problem with a clone of the standard 'arcgispro-py3' environment.

I am working with ArcGIS Pro 3.0.5. As I am employed the device is owned and managed by the company, which leaves no way of changing the ArcGIS installation.

However, to gain some freedome in the use of 'arcpy', I do create clones of the standard environment. Therefore, I go to the directory that contains the 'conda'-related scripts and clone the environment in the C:\Program\ArcGIS\Pro\bin\Python\envs
to
C:\Users\*\AppData\*\envs.
I do create a clone in order to install the 'esda' package which conveniently comes with 'libpysal', that I also need. Now, to prevent any major hiccups and knowing the issue that comes up, I do pin the ‘gdal’ to 3.4.0.
I perform all installations of packages in cloned environments through the command line and the ‘conda’-related scripts.

I have a .pyt based Toolbox, that uses subscripts. These should employ ‘gdal’, ‘esda’, and ‘libpysal’.

What I tested:

  1. Clone environment, NOT installing ‘esda’ and ‘libpysal’, disabling the sections in the script that need these, and disabling the ‘import’ of said packages.
    ‘from osgeo import gdal’ is used.
    Then, said toolbox and script can be run over and over, within one session of ArcGIS Pro, or closing Pro and opening Pro again.
    The cloned environment and all package dependencies are stable.
  2. Installing ‘esda’ and ‘libpysal’ (while ArcGIS Pro can be opened or closed).
    Note:

 

gdal                           3.4.0-arcgispro_py39_17119 --> 3.4.0-arcgispro_py39_17150​

This happens along with the otherwise flawless installation.

I enable the sections of the script that use ‘esda’ and ‘libpysal’ functionality, ‘import’ the packages.
Also, ‘from osgeo import gdal’ is used.
Then I (start ArcGIS Pro and) open the toolbox, the syntax check is successful, NO errors.
The toolbox can be used again and again.

But, if I now close ArcGIS Pro (the day might be over), and then come back to it and want to use the Toolbox, I get an error:

 

 

 

Traceback (most recent call last):
  File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 40, in swig_import_helper
    return importlib.import_module(mname)
  File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 565, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1173, in create_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
ImportError: DLL load failed while importing _gdal: Die angegebene Prozedur wurde nicht gefunden.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 17, in <module>
  File "C:\Users\schwenk_m\Documents\ArcGIS\Projects\Geo_gen\RasterSurfaceAnalysis.py", line 23, in <module>
    from osgeo import gdal
  File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 56, in <module>
    _gdal = swig_import_helper()
  File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 52, in swig_import_helper
    raise ImportError(traceback_string + '\n' + msg)
ImportError: Traceback (most recent call last):
  File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 40, in swig_import_helper
    return importlib.import_module(mname)
  File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 565, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1173, in create_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
ImportError: DLL load failed while importing _gdal: Die angegebene Prozedur wurde nicht gefunden.

On Windows, with Python >= 3.8, DLLs are no longer imported from the PATH.
If gdalXXX.dll is in the PATH, then set the USE_PATH_FOR_GDAL_PYTHON=YES environment variable
to feed the PATH into os.add_dll_directory().

 

 

My questions is, what happend to the environment through the installation of the new packages, and by closing and opening ArcGIS Pro?

For all that I tryed, I could not, as suggested, set the option 'USE_PATH_FOR_GDAL_PYTHON=YES'.
I did not grasp how to exactly use the command

 

 

conda env config vars list
USE_PATH_FOR_GDAL_PYTHON=YES

 

 


that was suggested here. However, I did add the line to the '.condarc' file in my Home directory, which does not help.

Furthermore, upon inspection all the .py scripts related to GDAL in the OSGEO folder are still there.

I am thankful for any pointers.

Tags (3)
0 Kudos
1 Solution

Accepted Solutions
Dumbatz
New Contributor II

Currently, the problem is solved. The 'gdal'-update that happened, is not performed any more.
Since 'gdal' is stable, the environment is stable.

But this could change again, over time.

View solution in original post

0 Kudos
2 Replies
Dumbatz
New Contributor II

True. Thanks for the suggestion.
This is what I do, although I do not work with VS Code.

The problem is, that the enviornment is 'distroyed' by the ArcGIS Pro program itself. After using the toolbox in one instance, I cannot use it again.
If I activate the environment on a command prompt, and call Python, the "from osgeo import gdal" is no longer functional.

The cloned environment is kaputt.

0 Kudos
Dumbatz
New Contributor II

Currently, the problem is solved. The 'gdal'-update that happened, is not performed any more.
Since 'gdal' is stable, the environment is stable.

But this could change again, over time.

0 Kudos