[gpaw-users] installation of gpaw
Nichols A. Romero
naromero at alcf.anl.gov
Thu Nov 17 14:42:11 CET 2011
Marcin,
I tried linking with -lmkl_def and -lmkl_mc and it still failed. Only
do LD_PRELOAD work. I found the suggestion at the Intel website.
When I get to the office, I will send the link to the list.
Nichols A. Romero, Ph.D.
Argonne Leadership Computing Facility
Argonne National Laboratory
Building 240 Room 2-127
9700 South Cass Avenue
Argonne, IL 60490
(630) 252-3441
----- Original Message -----
From: "Marcin Dulak" <Marcin.Dulak at fysik.dtu.dk>
To: "Marcin Dulak" <Marcin.Dulak at fysik.dtu.dk>
Cc: "Nichols A. Romero" <naromero at alcf.anl.gov>, "gpaw-users" <gpaw-users at listserv.fysik.dtu.dk>
Sent: Thursday, November 17, 2011 3:36:16 AM
Subject: Re: [gpaw-users] installation of gpaw
Marcin Dulak wrote:
> Hi,
>
> Nichols A. Romero wrote:
>
>> Hi,
>>
>> I am replying to this very old post because I just ran into Boris problem
>> and was able to solve it.
>>
>> If I run the regression test on the _gpaw.so (effectively, the serial binary),
>> you get the following error when running numpy_zdotc_graphite.py
>>
>> *** libmkl_mc.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_mc.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>> *** libmkl_def.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_def.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>> MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so
>>
>> The correct link line is provided by this handy Java script:
>> http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/
>>
>>
what about linking to explicitly mkl_def (or mkl_mc; or both)?
As this post suggests
(http://mail.scipy.org/pipermail/numpy-discussion/2009-October/046020.html)
sometimes only static linking works for mkl.
Expect the way you link to change with every mkl release.
> in my experience what you get from this tool does not always work.
> Try to search for the required symbols under
> /soft/intel/mkl/10.2.5.035/lib/em64t/
> using nm or strings, make a guess which *.so (many can contain those
> symbols) is more relevant,
> and link to it explicitly. I can't guarantee this works, but worked many
>
> times for me.
>
Just for reference this is how you can search for symbols with nm:
find . -name "*.so" | xargs -I file sh -c "echo file && nm -a file |
grep 'T mkl_dft_commit_descriptor_s_c2c_md_omp' "
For mkl 10.2.4.032 these symbols are in:
libmkl_intel_thread.so
libmkl_sequential.so
libmkl_pgi_thread.so
libmkl_gnu_thread.so
Marcin
> Marcin
>
>> For dynamic linking, all that is needed is:
>> libraries += ['mkl_intel_lp64',
>> 'mkl_sequential',
>> 'mkl_core']
>>
>> which is what Boris was using. Thie static linking line is different and
>> not relevant for _gpaw.so.
>>
>> Setting this environment variable solved the problem:
>> LD_PRELOAD = /soft/mkl/10.2.2.025/lib/em64t/libmkl_core.so:/soft/mkl/10.2.2.025/
>> lib/em64t/libmkl_sequential.so
>>
>> I don't fully comprehend why LD_PRELOAD is needed here, but it seems to solve
>> the problem. Maybe someone on the list that has in-depth knowledge of the linker
>> can comment.
>>
>> Nichols A. Romero, Ph.D.
>> Argonne Leadership Computing Facility
>> Argonne National Laboratory
>> Building 240 Room 2-127
>> 9700 South Cass Avenue
>> Argonne, IL 60490
>> (630) 252-3441
>>
>>
>> ----- Original Message -----
>> From: "Marcin Dulak" <Marcin.Dulak at fysik.dtu.dk>
>> To: "Boris Averkiev" <averkiev75 at yahoo.com>
>> Cc: "gpaw-users at listserv.fysik.dtu.dk" <gpaw-users at servfys.fysik.dtu.dk>
>> Sent: Friday, May 21, 2010 2:39:19 AM
>> Subject: Re: [gpaw-users] installation of gpaw
>>
>> Hi,
>>
>> let's continue on gpaw-users. The information is useful to the others.
>>
>> Boris Averkiev wrote:
>>
>>
>>> Dear Marcin,
>>>
>>> Thank you very much for your reply.
>>> Of course I have forgotten to uncomment settings in customize.py, so now they worked and now I didn't change config.py (I started everything from the very beginning).
>>> Now I used gcc x86_64-suse-linux ver 4.1.2
>>> Open MPI 1.4.1 and the same mkl - mkl/10.2.5.035
>>>
>>> I didn't use any blas/lapack to build numpy - it compiled without them.
>>> So, should I reinstall NumPy with MKL?
>>>
>>>
>>>
>> i would say that is tricky to have both numpy and gpaw-python linked to
>> non-standard blas/lapack.
>> As a first try it's USUALLY better to have numpy built with it's
>> internal defaults.
>> You may consider downloading the file
>> https://svn.fysik.dtu.dk/projects/gpaw/trunk/gpaw/test/numpy_zdotc_graphite.py
>> and run it with gpaw-python.
>> If it crashes you have some problems with blas/lapack.
>>
>>
>>
>>> I also set scalapack = False in customize.py, as you said.
>>> Now after command "python setup.py build_ext 2>&1 | tee build_ext.log" I've got
>>> * Scientific is not installed.
>>> You will not be able to write and read wave functions
>>> in the netCDF format.
>>> * Using standard lapack
>>> * Architecture: linux-x86_64
>>> * Building a custom interpreter
>>> So, it doesn't say anything about "Only a serial version of gpaw was built!"
>>> But gpaw-test still fails with the same output.
>>>
>>>
>>>
>> lines such:
>>
>> MKL ERROR: Parameter 5 was incorrect on entry to ZHEEV
>> 0.036 OK
>>
>> suggest that you may have a mismatch in numpy's and gpaw-python
>> libraries, however:
>>
>> *** libmkl_mc.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_mc.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>> *** libmkl_def.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_def.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>> MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so
>>
>> shows that linking is not correct. I think you should follow the intel
>> link advisor to find the correct linking.
>> Start with the minimum recommended, if you use gcc:
>> "-L$MKLPATH $MKLPATH/libmkl_solver_lp64_sequential.a -Wl,--start-group
>> -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread"
>> which translates probably eventually to (strangely): ['mkl_intel_lp64',
>> 'mkl_sequential', 'mkl_core', 'guide', 'mkl_lapack', 'mkl_def']
>>
>>
>>
>>> May be I should also install SciPy?
>>>
>>>
>>>
>> you can do it later if you find that you need it (some new gpaw
>> functionality like vdw, response(? i think) needs it).
>>
>>
>>> Also during building it has messages like
>>> c/libxc/src/work_lda.c: In function âwork_ldaâ:
>>> c/libxc/src/lda_c_pw.c:125: warning: âvcpâ may be used uninitialized in this function
>>> c/libxc/src/lda_c_pw.c:125: warning: âdalphaâ may be used uninitialized in this function
>>> c/libxc/src/lda_c_pw.c:125: warning: âd2alphaâ may be used uninitialized in this function
>>>
>>>
>>> c/libxc/src/work_gga_x.c:46: warning: âldfdxâ may be used uninitialized in this function
>>>
>>> c/libxc/src/gga_xc_hcth.c:203: warning: initialization from incompatible pointer type
>>>
>>>
>>>
>> for the moment ignore this.
>>
>> Marcin
>>
>>
>>> Thank you
>>>
>>> Boris
>>>
>>>
>>>
>>> --- On Tue, 5/18/10, Marcin Dulak <Marcin.Dulak at fysik.dtu.dk> wrote:
>>>
>>>
>>>
>>>
>>>> From: Marcin Dulak <Marcin.Dulak at fysik.dtu.dk>
>>>> Subject: Re: [gpaw-users] installation of gpaw
>>>> To: "Boris Averkiev" <averkiev75 at yahoo.com>
>>>> Cc: gpaw-users at listserv.fysik.dtu.dk
>>>> Date: Tuesday, May 18, 2010, 11:25 PM
>>>> Hi,
>>>>
>>>> Boris Averkiev wrote:
>>>>
>>>>
>>>>
>>>>> I am trying to install gpaw on cluster in my home
>>>>>
>>>>>
>>>>>
>>>> directory (I am just user, not administrator). As it is
>>>> written in manual, I first did Custom installation
>>>>
>>>>
>>>>
>>>>> I installed NumPy - numpy-1.3.0 in my home
>>>>>
>>>>>
>>>>>
>>>> directory
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>> what blas/lapack did you use to build numpy?
>>>>
>>>>
>>>>
>>>>> Also I installed ASE - python-ase-3.3.1.1390
>>>>> We have blas and lapac libraries in MKL, so I
>>>>>
>>>>>
>>>>>
>>>> specified the path to these libraries in customize.py
>>>>
>>>>
>>>>
>>>>> #library_dirs =
>>>>>
>>>>>
>>>>>
>>>> ['/soft/intel/mkl/10.2.5.035/lib/em64t']
>>>>
>>>>
>>>>
>>>>> And libraries themselves
>>>>> #libraries = ['mkl_lapack', 'mkl_intel_thread',
>>>>>
>>>>>
>>>>>
>>>> 'mkl_def', 'mkl-mc', 'mkl_core', 'guide', 'mkl_intel_ilp64',
>>>> 'mkl_blas95_ilp64']
>>>>
>>>>
>>>>
>>>>> However, it didn’t work. When I run “python
>>>>>
>>>>>
>>>>>
>>>> setup.py build_ext 2>&1 | tee build_ext.log”
>>>> command, I got error “cannot find -lblas”, so I changed
>>>> all “blas” to “mkl_blas95_ilp64” and all
>>>> “lapack” to “mkl_lapack” in config.py. After that it
>>>> compiled, however it also gave the following message:
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>> there is no need to change config.py (it should not be
>>>> changed), are you sure that your settings in customize.py
>>>> are uncommented (I see # in front of the variables)?
>>>> Every time you compile please run `python setup.py clean`
>>>> (or just remove the build directory).
>>>> Linking with mkl is tricky: i would start with a guess at
>>>> http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/,
>>>> and then search for missing symbols using nm (for
>>>> undefined reference to symbol_ use: nm
>>>> /soft/intel/mkl/10.2.5.035/lib/em64t/lib*.so | grep "T
>>>> symbol_")
>>>> Please also set scalapack = False in customize.py first.
>>>>
>>>>
>>>>
>>>>> * Scientific is not installed.
>>>>> You will not be able to write and
>>>>>
>>>>>
>>>>>
>>>> read wave functions
>>>>
>>>>
>>>>
>>>>> in the netCDF format.
>>>>> * Using standard mkl_lapack
>>>>> * Compiling with ScaLapack
>>>>> * Architecture: linux-x86_64
>>>>> * Only a serial version of gpaw was built!
>>>>>
>>>>> We have MPI - intel mpi, but I don't know how should I
>>>>>
>>>>>
>>>>>
>>>> specify it.
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>> let's get the serial version working first.
>>>>
>>>> Best regards,
>>>>
>>>> Marcin
>>>>
>>>>
>>>>
>>>>> When I run test gpaw-test 2>&1 | tee test.log
>>>>> it fails. I have attached the test.log file
>>>>> Information about cluster - SGI Altix XE 1300 Linux
>>>>>
>>>>>
>>>>>
>>>> Cluster
>>>>
>>>>
>>>>
>>>>> Could you help me to resolve those problems?
>>>>> Thank you
>>>>> Boris
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>> _______________________________________________
>>>>> gpaw-users mailing list
>>>>> gpaw-users at listserv.fysik.dtu.dk
>>>>> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>
>
>
--
***********************************
Marcin Dulak
Technical University of Denmark
Department of Physics
Building 307, Room 229
DK-2800 Kongens Lyngby
Denmark
Tel.: (+45) 4525 3157
Fax.: (+45) 4593 2399
email: Marcin.Dulak at fysik.dtu.dk
***********************************
More information about the gpaw-users
mailing list