[gpaw-users] installation of gpaw
Marcin Dulak
Marcin.Dulak at fysik.dtu.dk
Thu Nov 17 14:55:56 CET 2011
Yes, i have seen that too:
http://software.intel.com/en-us/forums/showthread.php?t=60460
Marcin
Nichols A. Romero wrote:
> Marcin,
>
> I tried linking with -lmkl_def and -lmkl_mc and it still failed. Only
> do LD_PRELOAD work. I found the suggestion at the Intel website.
> When I get to the office, I will send the link to the list.
>
> Nichols A. Romero, Ph.D.
> Argonne Leadership Computing Facility
> Argonne National Laboratory
> Building 240 Room 2-127
> 9700 South Cass Avenue
> Argonne, IL 60490
> (630) 252-3441
>
>
> ----- Original Message -----
> From: "Marcin Dulak" <Marcin.Dulak at fysik.dtu.dk>
> To: "Marcin Dulak" <Marcin.Dulak at fysik.dtu.dk>
> Cc: "Nichols A. Romero" <naromero at alcf.anl.gov>, "gpaw-users" <gpaw-users at listserv.fysik.dtu.dk>
> Sent: Thursday, November 17, 2011 3:36:16 AM
> Subject: Re: [gpaw-users] installation of gpaw
>
> Marcin Dulak wrote:
>
>> Hi,
>>
>> Nichols A. Romero wrote:
>>
>>
>>> Hi,
>>>
>>> I am replying to this very old post because I just ran into Boris problem
>>> and was able to solve it.
>>>
>>> If I run the regression test on the _gpaw.so (effectively, the serial binary),
>>> you get the following error when running numpy_zdotc_graphite.py
>>>
>>> *** libmkl_mc.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_mc.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>>> *** libmkl_def.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_def.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>>> MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so
>>>
>>> The correct link line is provided by this handy Java script:
>>> http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/
>>>
>>>
>>>
> what about linking to explicitly mkl_def (or mkl_mc; or both)?
> As this post suggests
> (http://mail.scipy.org/pipermail/numpy-discussion/2009-October/046020.html)
> sometimes only static linking works for mkl.
> Expect the way you link to change with every mkl release.
>
>> in my experience what you get from this tool does not always work.
>> Try to search for the required symbols under
>> /soft/intel/mkl/10.2.5.035/lib/em64t/
>> using nm or strings, make a guess which *.so (many can contain those
>> symbols) is more relevant,
>> and link to it explicitly. I can't guarantee this works, but worked many
>>
>> times for me.
>>
>>
> Just for reference this is how you can search for symbols with nm:
> find . -name "*.so" | xargs -I file sh -c "echo file && nm -a file |
> grep 'T mkl_dft_commit_descriptor_s_c2c_md_omp' "
> For mkl 10.2.4.032 these symbols are in:
> libmkl_intel_thread.so
> libmkl_sequential.so
> libmkl_pgi_thread.so
> libmkl_gnu_thread.so
>
> Marcin
>
>> Marcin
>>
>>
>>> For dynamic linking, all that is needed is:
>>> libraries += ['mkl_intel_lp64',
>>> 'mkl_sequential',
>>> 'mkl_core']
>>>
>>> which is what Boris was using. Thie static linking line is different and
>>> not relevant for _gpaw.so.
>>>
>>> Setting this environment variable solved the problem:
>>> LD_PRELOAD = /soft/mkl/10.2.2.025/lib/em64t/libmkl_core.so:/soft/mkl/10.2.2.025/
>>> lib/em64t/libmkl_sequential.so
>>>
>>> I don't fully comprehend why LD_PRELOAD is needed here, but it seems to solve
>>> the problem. Maybe someone on the list that has in-depth knowledge of the linker
>>> can comment.
>>>
>>> Nichols A. Romero, Ph.D.
>>> Argonne Leadership Computing Facility
>>> Argonne National Laboratory
>>> Building 240 Room 2-127
>>> 9700 South Cass Avenue
>>> Argonne, IL 60490
>>> (630) 252-3441
>>>
>>>
>>> ----- Original Message -----
>>> From: "Marcin Dulak" <Marcin.Dulak at fysik.dtu.dk>
>>> To: "Boris Averkiev" <averkiev75 at yahoo.com>
>>> Cc: "gpaw-users at listserv.fysik.dtu.dk" <gpaw-users at servfys.fysik.dtu.dk>
>>> Sent: Friday, May 21, 2010 2:39:19 AM
>>> Subject: Re: [gpaw-users] installation of gpaw
>>>
>>> Hi,
>>>
>>> let's continue on gpaw-users. The information is useful to the others.
>>>
>>> Boris Averkiev wrote:
>>>
>>>
>>>
>>>> Dear Marcin,
>>>>
>>>> Thank you very much for your reply.
>>>> Of course I have forgotten to uncomment settings in customize.py, so now they worked and now I didn't change config.py (I started everything from the very beginning).
>>>> Now I used gcc x86_64-suse-linux ver 4.1.2
>>>> Open MPI 1.4.1 and the same mkl - mkl/10.2.5.035
>>>>
>>>> I didn't use any blas/lapack to build numpy - it compiled without them.
>>>> So, should I reinstall NumPy with MKL?
>>>>
>>>>
>>>>
>>>>
>>> i would say that is tricky to have both numpy and gpaw-python linked to
>>> non-standard blas/lapack.
>>> As a first try it's USUALLY better to have numpy built with it's
>>> internal defaults.
>>> You may consider downloading the file
>>> https://svn.fysik.dtu.dk/projects/gpaw/trunk/gpaw/test/numpy_zdotc_graphite.py
>>> and run it with gpaw-python.
>>> If it crashes you have some problems with blas/lapack.
>>>
>>>
>>>
>>>
>>>> I also set scalapack = False in customize.py, as you said.
>>>> Now after command "python setup.py build_ext 2>&1 | tee build_ext.log" I've got
>>>> * Scientific is not installed.
>>>> You will not be able to write and read wave functions
>>>> in the netCDF format.
>>>> * Using standard lapack
>>>> * Architecture: linux-x86_64
>>>> * Building a custom interpreter
>>>> So, it doesn't say anything about "Only a serial version of gpaw was built!"
>>>> But gpaw-test still fails with the same output.
>>>>
>>>>
>>>>
>>>>
>>> lines such:
>>>
>>> MKL ERROR: Parameter 5 was incorrect on entry to ZHEEV
>>> 0.036 OK
>>>
>>> suggest that you may have a mismatch in numpy's and gpaw-python
>>> libraries, however:
>>>
>>> *** libmkl_mc.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_mc.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>>> *** libmkl_def.so *** failed with error : /soft/intel/mkl/10.2.5.035/lib/em64t/libmkl_def.so: undefined symbol: mkl_dft_commit_descriptor_s_c2c_md_omp
>>> MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so
>>>
>>> shows that linking is not correct. I think you should follow the intel
>>> link advisor to find the correct linking.
>>> Start with the minimum recommended, if you use gcc:
>>> "-L$MKLPATH $MKLPATH/libmkl_solver_lp64_sequential.a -Wl,--start-group
>>> -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread"
>>> which translates probably eventually to (strangely): ['mkl_intel_lp64',
>>> 'mkl_sequential', 'mkl_core', 'guide', 'mkl_lapack', 'mkl_def']
>>>
>>>
>>>
>>>
>>>> May be I should also install SciPy?
>>>>
>>>>
>>>>
>>>>
>>> you can do it later if you find that you need it (some new gpaw
>>> functionality like vdw, response(? i think) needs it).
>>>
>>>
>>>
>>>> Also during building it has messages like
>>>> c/libxc/src/work_lda.c: In function âwork_ldaâ:
>>>> c/libxc/src/lda_c_pw.c:125: warning: âvcpâ may be used uninitialized in this function
>>>> c/libxc/src/lda_c_pw.c:125: warning: âdalphaâ may be used uninitialized in this function
>>>> c/libxc/src/lda_c_pw.c:125: warning: âd2alphaâ may be used uninitialized in this function
>>>>
>>>>
>>>> c/libxc/src/work_gga_x.c:46: warning: âldfdxâ may be used uninitialized in this function
>>>>
>>>> c/libxc/src/gga_xc_hcth.c:203: warning: initialization from incompatible pointer type
>>>>
>>>>
>>>>
>>>>
>>> for the moment ignore this.
>>>
>>> Marcin
>>>
>>>
>>>
>>>> Thank you
>>>>
>>>> Boris
>>>>
>>>>
>>>>
>>>> --- On Tue, 5/18/10, Marcin Dulak <Marcin.Dulak at fysik.dtu.dk> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> From: Marcin Dulak <Marcin.Dulak at fysik.dtu.dk>
>>>>> Subject: Re: [gpaw-users] installation of gpaw
>>>>> To: "Boris Averkiev" <averkiev75 at yahoo.com>
>>>>> Cc: gpaw-users at listserv.fysik.dtu.dk
>>>>> Date: Tuesday, May 18, 2010, 11:25 PM
>>>>> Hi,
>>>>>
>>>>> Boris Averkiev wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I am trying to install gpaw on cluster in my home
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> directory (I am just user, not administrator). As it is
>>>>> written in manual, I first did Custom installation
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I installed NumPy - numpy-1.3.0 in my home
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> directory
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>> what blas/lapack did you use to build numpy?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Also I installed ASE - python-ase-3.3.1.1390
>>>>>> We have blas and lapac libraries in MKL, so I
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> specified the path to these libraries in customize.py
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> #library_dirs =
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> ['/soft/intel/mkl/10.2.5.035/lib/em64t']
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> And libraries themselves
>>>>>> #libraries = ['mkl_lapack', 'mkl_intel_thread',
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> 'mkl_def', 'mkl-mc', 'mkl_core', 'guide', 'mkl_intel_ilp64',
>>>>> 'mkl_blas95_ilp64']
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> However, it didn’t work. When I run “python
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> setup.py build_ext 2>&1 | tee build_ext.log”
>>>>> command, I got error “cannot find -lblas”, so I changed
>>>>> all “blas” to “mkl_blas95_ilp64” and all
>>>>> “lapack” to “mkl_lapack” in config.py. After that it
>>>>> compiled, however it also gave the following message:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>> there is no need to change config.py (it should not be
>>>>> changed), are you sure that your settings in customize.py
>>>>> are uncommented (I see # in front of the variables)?
>>>>> Every time you compile please run `python setup.py clean`
>>>>> (or just remove the build directory).
>>>>> Linking with mkl is tricky: i would start with a guess at
>>>>> http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/,
>>>>> and then search for missing symbols using nm (for
>>>>> undefined reference to symbol_ use: nm
>>>>> /soft/intel/mkl/10.2.5.035/lib/em64t/lib*.so | grep "T
>>>>> symbol_")
>>>>> Please also set scalapack = False in customize.py first.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> * Scientific is not installed.
>>>>>> You will not be able to write and
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> read wave functions
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> in the netCDF format.
>>>>>> * Using standard mkl_lapack
>>>>>> * Compiling with ScaLapack
>>>>>> * Architecture: linux-x86_64
>>>>>> * Only a serial version of gpaw was built!
>>>>>>
>>>>>> We have MPI - intel mpi, but I don't know how should I
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> specify it.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>> let's get the serial version working first.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Marcin
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> When I run test gpaw-test 2>&1 | tee test.log
>>>>>> it fails. I have attached the test.log file
>>>>>> Information about cluster - SGI Altix XE 1300 Linux
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> Cluster
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Could you help me to resolve those problems?
>>>>>> Thank you
>>>>>> Boris
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> _______________________________________________
>>>>>> gpaw-users mailing list
>>>>>> gpaw-users at listserv.fysik.dtu.dk
>>>>>> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
>
More information about the gpaw-users
mailing list