Cannot install netCDF4 by pip on windows
Question:
I just installed ‘vanilla’ python on windows 10 and tried to install netCDF4, only to find it gives an error saying that:
“ValueError: did not find HDF5 headers”
This is Python 3.7 and versions are as follows:
py -m pip list
Package Version
------------------ -------
attrs 19.1.0
backcall 0.1.0
bleach 3.1.0
bokeh 1.3.4
colorama 0.4.1
cycler 0.10.0
Cython 0.29.13
decorator 4.4.0
defusedxml 0.6.0
entrypoints 0.3
h5py 2.9.0
ipykernel 5.1.2
ipython 7.8.0
ipython-genutils 0.2.0
ipywidgets 7.5.1
jedi 0.15.1
Jinja2 2.10.1
jsonschema 3.0.2
jupyter 1.0.0
jupyter-client 5.3.1
jupyter-console 6.0.0
jupyter-core 4.5.0
kiwisolver 1.1.0
MarkupSafe 1.1.1
matplotlib 3.1.1
mistune 0.8.4
nbconvert 5.6.0
nbformat 4.4.0
notebook 6.0.1
numpy 1.17.1
packaging 19.1
pandas 0.25.1
pandocfilters 1.4.2
parso 0.5.1
pickleshare 0.7.5
Pillow 6.1.0
pip 19.2.3
pip-review 1.0
pivottablejs 0.9.0
prometheus-client 0.7.1
prompt-toolkit 2.0.9
Pygments 2.4.2
pyparsing 2.4.2
pypl 40.10
pyrsistent 0.15.4
python-dateutil 2.8.0
pytz 2019.2
pywinpty 0.5.5
PyYAML 5.1.2
pyzmq 18.1.0
qtconsole 4.5.5
scipy 1.3.1
seaborn 0.9.0
Send2Trash 1.5.0
setuptools 41.2.0
six 1.12.0
svgwrite 1.3.1
terminado 0.8.2
testpath 0.4.2
tornado 6.0.3
traitlets 4.3.2
virtualenv 16.7.5
wcwidth 0.1.7
webencodings 0.5.1
widgetsnbextension 3.5.1
xarray 0.12.3
When I try to install netCDF with pip, it gives an error.
The whole error message is:
>py -m pip install netCDF4
Collecting netCDF4
Using cached
https://files.pythonhosted.org/packages/64/2e/abc0bce095ab5a3b8374f052ace2509a031fd7633b23917e557487067225/netCDF4-1.5.2.tar.gz
ERROR: Command errored out with exit status 1:
command: 'c:python37-32python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\***\AppData\Local\Temp\pip-install-374v_ecc\netCDF4\setup.py'"'"'; __file__='"'"'C:\Users\***\AppData\Local\Temp\pip-install-374v_ecc\netCDF4\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
cwd: C:Users***AppDataLocalTemppip-install-374v_eccnetCDF4
Complete output (16 lines):
reading from setup.cfg...
HDF5_DIR environment variable not set, checking some standard locations ..
checking C:Users***...
checking /usr/local ...
checking /sw ...
checking /opt ...
checking /opt/local ...
checking /usr ...
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:Users***AppDataLocalTemppip-install-374v_eccnetCDF4setup.py", line 397, in <module>
_populate_hdf5_info(dirstosearch, inc_dirs, libs, lib_dirs)
File "C:Users***AppDataLocalTemppip-install-374v_eccnetCDF4setup.py", line 345, in _populate_hdf5_info
raise ValueError('did not find HDF5 headers')
ValueError: did not find HDF5 headers
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Please tell me what to do or what information you need.
Thank you.
Answers:
I resolved the problem by myself.
I needed to install C++ compiler manually since it is not included in vanilla python package (of course).
I downloaded and installed ‘build tools for visual studio’ from the site below:
https://visualstudio.microsoft.com/downloads/
and this solved the problem.
TL;DR: Check if there’s an issue with the present h5py wheels if you run into this problem
I’ve just run into almost the exact same problem as the OP this afternoon and spent the last hour banging my head against a wall.
My story:
- I am using Python 3.11.1
- Found this thread. I had seemingly the same issue as the OP
- I considered installing the build tools for visual studio but noticed it was going to be ~5 GB…
- Decided to investigate further
- Found this GitHub issue demonstrating it’s an issue with the h5py wheel for 3.11.1
- Installed netCDF4 on a different computer with Python 3.8.3 without any issues
- I will be using Python<=3.10 until h5py wheel is sorted
So was the C++ compiler the real solution to the OP’s issue? Well I checked and it turns out, there was another issue open for wheels not being prepared correctly for h5py back in September 2019 which was closed around the time of the OP’s first post. Though, there seems to be a ~23 hour gap in the timeline (I think I’ve accounted for timezones correctly). Perhaps the C++ compiler was able to resolve the issue introduced by the faulty wheels?
I have a new conda environment with python 3.7 installed and I get a similar error on a Windows 10 machine. It basically can’t find the headers for h5py. (I don’t have enough points to comment, but here is the same problem persisting in 2023.)
Collecting netCDF4
Using cached netCDF4-1.6.3.tar.gz (777 kB)
Installing build dependencies … done
Getting requirements to build wheel … error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [37 lines of output]
reading from setup.cfg…
HDF5_DIR environment variable not set, checking some standard locations ..
checking C:UsersrringuetAnaconda3envspip_testinclude ...
hdf5 headers not found in C:UsersrringuetAnaconda3envspip_testinclude
checking C:UsersrringuetAnaconda3envspip_testLibraryinclude ...
hdf5 headers not found in C:UsersrringuetAnaconda3envspip_testLibraryinclude
checking C:Usersrringuetinclude ...
hdf5 headers not found in C:Usersrringuetinclude
checking /usr/localinclude ...
hdf5 headers not found in /usr/localinclude
checking /swinclude ...
hdf5 headers not found in /swinclude
checking /optinclude ...
hdf5 headers not found in /optinclude
checking /opt/localinclude ...
hdf5 headers not found in /opt/localinclude
checking /opt/homebrewinclude ...
hdf5 headers not found in /opt/homebrewinclude
checking /usrinclude ...
hdf5 headers not found in /usrinclude
Traceback (most recent call last):
File "C:UsersrringuetAnaconda3envspip_testlibsite-packagespip_vendorpep517in_process_in_process.py", line 351, in <module>
main()
File "C:UsersrringuetAnaconda3envspip_testlibsite-packagespip_vendorpep517in_process_in_process.py", line 333, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:UsersrringuetAnaconda3envspip_testlibsite-packagespip_vendorpep517in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "C:UsersrringuetAppDataLocalTemp1pip-build-env-9ynl9nxtoverlayLibsite-packagessetuptoolsbuild_meta.py", line 338, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "C:UsersrringuetAppDataLocalTemp1pip-build-env-9ynl9nxtoverlayLibsite-packagessetuptoolsbuild_meta.py", line 320, in _get_build_requires
self.run_setup()
File "C:UsersrringuetAppDataLocalTemp1pip-build-env-9ynl9nxtoverlayLibsite-packagessetuptoolsbuild_meta.py", line 335, in run_setup
exec(code, locals())
File "<string>", line 453, in <module>
File "<string>", line 394, in _populate_hdf5_info
ValueError: did not find HDF5 headers
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
I just installed ‘vanilla’ python on windows 10 and tried to install netCDF4, only to find it gives an error saying that:
“ValueError: did not find HDF5 headers”
This is Python 3.7 and versions are as follows:
py -m pip list
Package Version
------------------ -------
attrs 19.1.0
backcall 0.1.0
bleach 3.1.0
bokeh 1.3.4
colorama 0.4.1
cycler 0.10.0
Cython 0.29.13
decorator 4.4.0
defusedxml 0.6.0
entrypoints 0.3
h5py 2.9.0
ipykernel 5.1.2
ipython 7.8.0
ipython-genutils 0.2.0
ipywidgets 7.5.1
jedi 0.15.1
Jinja2 2.10.1
jsonschema 3.0.2
jupyter 1.0.0
jupyter-client 5.3.1
jupyter-console 6.0.0
jupyter-core 4.5.0
kiwisolver 1.1.0
MarkupSafe 1.1.1
matplotlib 3.1.1
mistune 0.8.4
nbconvert 5.6.0
nbformat 4.4.0
notebook 6.0.1
numpy 1.17.1
packaging 19.1
pandas 0.25.1
pandocfilters 1.4.2
parso 0.5.1
pickleshare 0.7.5
Pillow 6.1.0
pip 19.2.3
pip-review 1.0
pivottablejs 0.9.0
prometheus-client 0.7.1
prompt-toolkit 2.0.9
Pygments 2.4.2
pyparsing 2.4.2
pypl 40.10
pyrsistent 0.15.4
python-dateutil 2.8.0
pytz 2019.2
pywinpty 0.5.5
PyYAML 5.1.2
pyzmq 18.1.0
qtconsole 4.5.5
scipy 1.3.1
seaborn 0.9.0
Send2Trash 1.5.0
setuptools 41.2.0
six 1.12.0
svgwrite 1.3.1
terminado 0.8.2
testpath 0.4.2
tornado 6.0.3
traitlets 4.3.2
virtualenv 16.7.5
wcwidth 0.1.7
webencodings 0.5.1
widgetsnbextension 3.5.1
xarray 0.12.3
When I try to install netCDF with pip, it gives an error.
The whole error message is:
>py -m pip install netCDF4
Collecting netCDF4
Using cached
https://files.pythonhosted.org/packages/64/2e/abc0bce095ab5a3b8374f052ace2509a031fd7633b23917e557487067225/netCDF4-1.5.2.tar.gz
ERROR: Command errored out with exit status 1:
command: 'c:python37-32python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\***\AppData\Local\Temp\pip-install-374v_ecc\netCDF4\setup.py'"'"'; __file__='"'"'C:\Users\***\AppData\Local\Temp\pip-install-374v_ecc\netCDF4\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
cwd: C:Users***AppDataLocalTemppip-install-374v_eccnetCDF4
Complete output (16 lines):
reading from setup.cfg...
HDF5_DIR environment variable not set, checking some standard locations ..
checking C:Users***...
checking /usr/local ...
checking /sw ...
checking /opt ...
checking /opt/local ...
checking /usr ...
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:Users***AppDataLocalTemppip-install-374v_eccnetCDF4setup.py", line 397, in <module>
_populate_hdf5_info(dirstosearch, inc_dirs, libs, lib_dirs)
File "C:Users***AppDataLocalTemppip-install-374v_eccnetCDF4setup.py", line 345, in _populate_hdf5_info
raise ValueError('did not find HDF5 headers')
ValueError: did not find HDF5 headers
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Please tell me what to do or what information you need.
Thank you.
I resolved the problem by myself.
I needed to install C++ compiler manually since it is not included in vanilla python package (of course).
I downloaded and installed ‘build tools for visual studio’ from the site below:
https://visualstudio.microsoft.com/downloads/
and this solved the problem.
TL;DR: Check if there’s an issue with the present h5py wheels if you run into this problem
I’ve just run into almost the exact same problem as the OP this afternoon and spent the last hour banging my head against a wall.
My story:
- I am using Python 3.11.1
- Found this thread. I had seemingly the same issue as the OP
- I considered installing the build tools for visual studio but noticed it was going to be ~5 GB…
- Decided to investigate further
- Found this GitHub issue demonstrating it’s an issue with the h5py wheel for 3.11.1
- Installed netCDF4 on a different computer with Python 3.8.3 without any issues
- I will be using Python<=3.10 until h5py wheel is sorted
So was the C++ compiler the real solution to the OP’s issue? Well I checked and it turns out, there was another issue open for wheels not being prepared correctly for h5py back in September 2019 which was closed around the time of the OP’s first post. Though, there seems to be a ~23 hour gap in the timeline (I think I’ve accounted for timezones correctly). Perhaps the C++ compiler was able to resolve the issue introduced by the faulty wheels?
I have a new conda environment with python 3.7 installed and I get a similar error on a Windows 10 machine. It basically can’t find the headers for h5py. (I don’t have enough points to comment, but here is the same problem persisting in 2023.)
Collecting netCDF4
Using cached netCDF4-1.6.3.tar.gz (777 kB)
Installing build dependencies … done
Getting requirements to build wheel … error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [37 lines of output]
reading from setup.cfg…
HDF5_DIR environment variable not set, checking some standard locations ..
checking C:UsersrringuetAnaconda3envspip_testinclude ...
hdf5 headers not found in C:UsersrringuetAnaconda3envspip_testinclude
checking C:UsersrringuetAnaconda3envspip_testLibraryinclude ...
hdf5 headers not found in C:UsersrringuetAnaconda3envspip_testLibraryinclude
checking C:Usersrringuetinclude ...
hdf5 headers not found in C:Usersrringuetinclude
checking /usr/localinclude ...
hdf5 headers not found in /usr/localinclude
checking /swinclude ...
hdf5 headers not found in /swinclude
checking /optinclude ...
hdf5 headers not found in /optinclude
checking /opt/localinclude ...
hdf5 headers not found in /opt/localinclude
checking /opt/homebrewinclude ...
hdf5 headers not found in /opt/homebrewinclude
checking /usrinclude ...
hdf5 headers not found in /usrinclude
Traceback (most recent call last):
File "C:UsersrringuetAnaconda3envspip_testlibsite-packagespip_vendorpep517in_process_in_process.py", line 351, in <module>
main()
File "C:UsersrringuetAnaconda3envspip_testlibsite-packagespip_vendorpep517in_process_in_process.py", line 333, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:UsersrringuetAnaconda3envspip_testlibsite-packagespip_vendorpep517in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "C:UsersrringuetAppDataLocalTemp1pip-build-env-9ynl9nxtoverlayLibsite-packagessetuptoolsbuild_meta.py", line 338, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "C:UsersrringuetAppDataLocalTemp1pip-build-env-9ynl9nxtoverlayLibsite-packagessetuptoolsbuild_meta.py", line 320, in _get_build_requires
self.run_setup()
File "C:UsersrringuetAppDataLocalTemp1pip-build-env-9ynl9nxtoverlayLibsite-packagessetuptoolsbuild_meta.py", line 335, in run_setup
exec(code, locals())
File "<string>", line 453, in <module>
File "<string>", line 394, in _populate_hdf5_info
ValueError: did not find HDF5 headers
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.