Bingo! Thanks Daryl. That fixed it...I did have to also do a pip uninstall
pyproj to get conda to properly install it. If I understand I had pip handling
pyproj and when I performed the update some of my packages got out of sync??
To answer your other question, I am firing of the python processes via cron.
GOES restitch runs in pqact and files away the imagery. The python processs in
cron fires and checks to make sure all proper imagery is present. If so, it
creates an image, if not it will wait for it for a time. If it arrives, then it
creates the image, if all tiles fail to arrive properly, It exits until the
next cron time executes. It also keeps track of what has been created to avoid
duplicates.
That does bring me to one other side question if anyone cares to chime in. What
other software or routines do others use to display GOES imagery? I am still
actively trying to learn what all is out there.
Thanks again for your help, the users in this group have been very helpful in
assisting new comers.
- Jim
-----Original Message-----
From: Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>
Sent: Thursday, May 19, 2022 9:15 AM
To: Weber, Jim <Jim.Weber@xxxxxxx>; python-users@xxxxxxxxxxxxxxxx
Subject: Re: miniconda error message
Greetings,
Thanks. That pyproj from pypi looks to be trouble. Can we try this?
conda install --force-reinstall pyproj
Also, how are you running these python processes? From cron? From pqact ?
daryl
________________________________________
From: Weber, Jim <Jim.Weber@xxxxxxx>
Sent: Wednesday, May 18, 2022 6:30 PM
To: Herzmann, Daryl E [AGRON]; python-users@xxxxxxxxxxxxxxxx
Subject: RE: miniconda error message
Hi, it is a long list, but here it is: (Thanks!)
# packages in environment at /home/ldm/miniconda3:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
brotli 1.0.9 h166bdaf_7 conda-forge
brotli-bin 1.0.9 h166bdaf_7 conda-forge
brotlipy 0.7.0 py39hb9d737c_1004 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.18.1 h7f98852_0 conda-forge
ca-certificates 2021.10.8 ha878542_0 conda-forge
cartopy 0.20.2 py39h0ddd56c_4 conda-forge
certifi 2021.10.8 py39hf3d152e_2 conda-forge
cffi 1.15.0 py39h4bc2ebd_0 conda-forge
cftime 1.6.0 py39hd257fcd_1 conda-forge
charset-normalizer 2.0.12 pyhd8ed1ab_0 conda-forge
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
conda 4.12.0 py39hf3d152e_0 conda-forge
conda-package-handling 1.8.1 py39hb9d737c_1 conda-forge
cryptography 36.0.2 py39hd97740a_1 conda-forge
curl 7.83.1 h7bff187_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
esmf 8.2.0 nompi_h61edca3_0 conda-forge
expat 2.4.8 h27087fc_0 conda-forge
fonttools 4.33.3 py39hb9d737c_0 conda-forge
freetype 2.10.4 h0708190_1 conda-forge
geos 3.10.2 h9c3ff4c_0 conda-forge
giflib 5.2.1 h36c2ea0_2 conda-forge
gsl 2.7 he838d99_0 conda-forge
hdf4 4.2.15 h10796ff_3 conda-forge
hdf5 1.12.1 nompi_h2386368_104 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
importlib-metadata 4.11.3 py39hf3d152e_1 conda-forge
importlib_metadata 4.11.3 hd8ed1ab_1 conda-forge
importlib_resources 5.7.1 pyhd8ed1ab_1 conda-forge
jbig 2.1 h7f98852_2003 conda-forge
jpeg 9e h166bdaf_1 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.2 py39hf939315_1 conda-forge
krb5 1.19.3 h3790be6_0 conda-forge
lcms2 2.12 hddcbb42_0 conda-forge
ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge
lerc 3.0 h9c3ff4c_0 conda-forge
libblas 3.9.0 14_linux64_openblas conda-forge
libbrotlicommon 1.0.9 h166bdaf_7 conda-forge
libbrotlidec 1.0.9 h166bdaf_7 conda-forge
libbrotlienc 1.0.9 h166bdaf_7 conda-forge
libcblas 3.9.0 14_linux64_openblas conda-forge
libcurl 7.83.1 h7bff187_0 conda-forge
libdeflate 1.10 h7f98852_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 12.1.0 h8d9b700_16 conda-forge
libgfortran-ng 12.1.0 h69a702a_16 conda-forge
libgfortran5 12.1.0 hdcd56e2_16 conda-forge
libgomp 12.1.0 h8d9b700_16 conda-forge
liblapack 3.9.0 14_linux64_openblas conda-forge
libnetcdf 4.8.1 nompi_h329d8a1_102 conda-forge
libnghttp2 1.47.0 h727a467_0 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libopenblas 0.3.20 pthreads_h78a6416_0 conda-forge
libpng 1.6.37 h21135ba_2 conda-forge
libssh2 1.10.0 ha56f1ee_2 conda-forge
libstdcxx-ng 12.1.0 ha89aaad_16 conda-forge
libtiff 4.3.0 h542a066_3 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libwebp 1.2.2 h3452ae3_0 conda-forge
libwebp-base 1.2.2 h7f98852_1 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libzip 1.8.0 h4de3113_1 conda-forge
libzlib 1.2.11 h166bdaf_1014 conda-forge
lz4-c 1.9.3 h9c3ff4c_1 conda-forge
matplotlib 3.4.3 pypi_0 pypi
matplotlib-base 3.5.2 py39h700656a_0 conda-forge
metpy 1.3.0 pyhd8ed1ab_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
nco 5.0.3 hcfc2ecc_0 conda-forge
ncurses 6.3 h27087fc_1 conda-forge
netcdf-fortran 4.5.4 nompi_h2b6e579_100 conda-forge
netcdf4 1.5.8 nompi_py39h64b754b_101 conda-forge
numpy 1.22.3 py39hc58783e_2 conda-forge
openjpeg 2.4.0 hb52868f_1 conda-forge
openssl 1.1.1o h166bdaf_0 conda-forge
packaging 21.3 pyhd8ed1ab_0 conda-forge
pandas 1.3.3 pypi_0 pypi
pillow 8.3.2 pypi_0 pypi
pint 0.19.2 pyhd8ed1ab_0 conda-forge
pip 22.1 pyhd8ed1ab_0 conda-forge
pooch 1.6.0 pyhd8ed1ab_0 conda-forge
proj 9.0.0 h93bde94_1 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
pycosat 0.6.3 py39hb9d737c_1010 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pyproj 3.2.1 pypi_0 pypi
pyshp 2.3.0 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 py39hf3d152e_5 conda-forge
python 3.9.12 h9a8a25e_1_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.9 2_cp39 conda-forge
pytz 2022.1 pyhd8ed1ab_0 conda-forge
readline 8.1 h46c0cb4_0 conda-forge
requests 2.27.1 pyhd8ed1ab_0 conda-forge
ruamel_yaml 0.15.80 py39h3811e60_1006 conda-forge
scipy 1.8.0 py39hee8e79c_1 conda-forge
setuptools 62.3.1 py39hf3d152e_0 conda-forge
shapely 1.8.2 py39h73b9895_1 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sqlite 3.38.5 h4ff8645_0 conda-forge
tempest-remap 2.1.3 h1a2fe09_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
tqdm 4.64.0 pyhd8ed1ab_0 conda-forge
traitlets 5.2.1.post0 pyhd8ed1ab_0 conda-forge
typing_extensions 4.2.0 pyha770c72_1 conda-forge
tzdata 2022a h191b570_0 conda-forge
udunits2 2.2.28 hc3e0081_0 conda-forge
unicodedata2 14.0.0 py39hb9d737c_1 conda-forge
urllib3 1.26.9 pyhd8ed1ab_0 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
xarray 2022.3.0 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xz 5.2.5 h516909a_1 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
zipp 3.8.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.11 h166bdaf_1014 conda-forge
zstd 1.5.2 h8a70e8d_1 conda-forge
Jim Weber
-----Original Message-----
From: Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>
Sent: Wednesday, May 18, 2022 4:11 PM
To: Weber, Jim <Jim.Weber@xxxxxxx>; python-users@xxxxxxxxxxxxxxxx
Subject: Re: miniconda error message
Greetings,
It may be good to list out your environment and see which packages are coming
from conda and which are coming from pip/pypi
conda list
daryl
________________________________________
From: Weber, Jim <Jim.Weber@xxxxxxx>
Sent: Wednesday, May 18, 2022 2:38 PM
To: Herzmann, Daryl E [AGRON]; python-users@xxxxxxxxxxxxxxxx
Subject: RE: miniconda error message
Daryl,
Miniconda is installed and started in the .bashrc so all path variables are
set then. It seems to be tied to when I did the metpy update, it installed many
other updates at that time also. One of them seems to have caused a small issue
to get things out of sync. Admittedly, I am not very proficent in the conda
enviroment as this is still new to me.
I did realize there was one more part of the line that I did not include in my
original email. The entire error message is below when I run a python script.
Specifically I am using a reciepe created by Brian Blaylock called
https://urldefense.proofpoint.com/v2/url?u=http-3A__mapping-5FGOES16-5FTrueColor.py&d=DwIFAg&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=KKIRMcmZ6BcKOjHM4171gGDDLEx31ZuiUy6Gn81J_cs&m=GK5ToT1IGxzCtZQ4TGNSte6MpAce6NleCh1_Oyl6KH4zpjRbsgDs9uSaaGrpC1OV&s=HzdaanOdDVeq5SmtFckKKa13UW6KEaJv5ESyxg_fzV0&e=
where I have modified it for my local use.
/home/ldm/miniconda3/lib/python3.9/site-packages/pyproj/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwIFAg&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=KKIRMcmZ6BcKOjHM4171gGDDLEx31ZuiUy6Gn81J_cs&m=GK5ToT1IGxzCtZQ4TGNSte6MpAce6NleCh1_Oyl6KH4zpjRbsgDs9uSaaGrpC1OV&s=_ZlMxA_xtyV5Rn7Gb1lYxiIQDK_VOIujYyTlBWfxF6U&e=
:89: UserWarning: pyproj unable to set database path.
_pyproj_global_context_initialize()
It seems to be tied to the import metpy in the beginning as that specific error
goes away if I comment it out (of course other parts do not work).
When I do a printenv, I do see "PROJ_LIB=/home/ldm/miniconda3/share/proj"
listed so the variable is set.
Looking back through my notes of when I originally installed everything I see
that I had reached out regarding an error message with pandas. A user gave me a
quick fixed of typing "pip uninstall pandas" followed by "pip install -U
pandas" but then warned me about a mixed pip-conda environment getting out of
sync. Could this be what is going on that I did a metpy update but something
got out of sync because one other part was installed with pip? Not to get too
far off topic, my notes suggest my first command during the installed was "pip
install metpy" followed by "conda install -c conda-forge metpy"
I guess I am also asking is my install correct or am I on a wrong tangent with
my install that future updates will cause me headache.
Sorry for the confusion and rambling, this is still a learning process for me.
Jim
-----Original Message-----
From: Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>
Sent: Tuesday, May 17, 2022 11:46 PM
To: Weber, Jim <Jim.Weber@xxxxxxx>; python-users@xxxxxxxxxxxxxxxx
Subject: Re: miniconda error message
Greetings,
How are you instantiating miniconda? If you are just manually setting a $PATH
variable, then your shell is missing out on other variables that miniconda may
be wishing to set at activation time. In this case PROJ_LIB. see:
etc/conda/activate.d/https://urldefense.proofpoint.com/v2/url?u=http-3A__proj4-2Dactivate.sh&d=DwIFAg&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=KKIRMcmZ6BcKOjHM4171gGDDLEx31ZuiUy6Gn81J_cs&m=GK5ToT1IGxzCtZQ4TGNSte6MpAce6NleCh1_Oyl6KH4zpjRbsgDs9uSaaGrpC1OV&s=HkVKv1y68hqUOUGwuU95_PnesHJmrIUuqo_Tf1Gx0g4&e=
Additionally, a best practice is to install packages into a dedicated
environment and not the base environment.
Now for LDM, having each shell invocation run a conda environment activation
can create horrible latency for some workflows. In such case, just add the
needed PROJ_LIB setting when you set your PATH. For your example I am setting
export PROJ_LIB=/home/ldm/miniconda3/share/proj
daryl
________________________________________
From: python-users <python-users-bounces@xxxxxxxxxxxxxxxx> on behalf of Weber,
Jim <Jim.Weber@xxxxxxx>
Sent: Tuesday, May 17, 2022 2:15 PM
To: python-users@xxxxxxxxxxxxxxxx
Subject: [python-users] miniconda error message
Hi,
I upgraded to metpy 1.3.0 last month and notice a little after I was getting a
warning message. It does not seem to affect the end result of anything that I
run, but it throws it out for anything that I run. Any idea of how I can
resolve the issue for this error:
/home/ldm/miniconda3/lib/python3.9/site-packages/pyproj/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwIFAg&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=KKIRMcmZ6BcKOjHM4171gGDDLEx31ZuiUy6Gn81J_cs&m=BMlerE2UKcRw0Su-YOnb-GIQ3I318UBjkUf-J0Z77Y2bWkl_YC86KWmjfx_kpi2W&s=kSytVwiagj1W9KzuBn7HlnxnOmKIOwean1FjmjUTwIw&e=
:89: UserWarning: pyproj unable to set database path.
Thanks for any help!
Jim Weber
This message and its attachments may contain legally privileged or confidential
information. It is intended solely for the named addressee(s). If you are not
an addressee indicated in this message (or responsible for delivery of the
message to an addressee), you may not copy or deliver this message or its
attachments to anyone. Rather, you should permanently delete this message and
its attachments and kindly notify the sender by reply e-mail. Any content of
this message and its attachments that does not relate to the official business
of Fox Corporation, or its subsidiaries must be taken not to have been sent or
endorsed by any of them. No representation is made that this email or its
attachments are without defect.
This message and its attachments may contain legally privileged or confidential
information. It is intended solely for the named addressee(s). If you are not
an addressee indicated in this message (or responsible for delivery of the
message to an addressee), you may not copy or deliver this message or its
attachments to anyone. Rather, you should permanently delete this message and
its attachments and kindly notify the sender by reply e-mail. Any content of
this message and its attachments that does not relate to the official business
of Fox Corporation, or its subsidiaries must be taken not to have been sent or
endorsed by any of them. No representation is made that this email or its
attachments are without defect.
This message and its attachments may contain legally privileged or confidential
information. It is intended solely for the named addressee(s). If you are not
an addressee indicated in this message (or responsible for delivery of the
message to an addressee), you may not copy or deliver this message or its
attachments to anyone. Rather, you should permanently delete this message and
its attachments and kindly notify the sender by reply e-mail. Any content of
this message and its attachments that does not relate to the official business
of Fox Corporation, or its subsidiaries must be taken not to have been sent or
endorsed by any of them. No representation is made that this email or its
attachments are without defect.
This message and its attachments may contain legally privileged or confidential
information. It is intended solely for the named addressee(s). If you are not
an addressee indicated in this message (or responsible for delivery of the
message to an addressee), you may not copy or deliver this message or its
attachments to anyone. Rather, you should permanently delete this message and
its attachments and kindly notify the sender by reply e-mail. Any content of
this message and its attachments that does not relate to the official business
of Fox Corporation, or its subsidiaries must be taken not to have been sent or
endorsed by any of them. No representation is made that this email or its
attachments are without defect.
