[gpaw-users] libmpifort.so: undefined reference compiler issue

Jussi Enkovaara jussi.enkovaara at csc.fi
Thu Feb 9 08:33:41 CET 2017


Hi Michael,
it seems that you are mixing Open MPI and Intel MPI (i.e. using Intel 
MPI's wrapper mpiicc but having Open MPI directories in your library 
paths).

You could try use only Intel MPI by removing all openmpi related stuff 
from "library_dirs" and LD_LIBRARY_PATH (replace also 
mkl_blacs_openmpi_lp64 with mkl_blacs_intelmpi_lp64 in
"libraries"). Other option could be to use Open MPI wrapper with 
something like:

compiler = 'icc'
mpicompiler = 'OMPI_CC=icc mpicc'
mpilinker = 'OMPI_CC=icc mpicc'

(You might need to add Open MPI's bin directory into your PATH in this 
case).

Best regards,
Jussi


On 2017-02-08 18:06, Groves, Michael via gpaw-users wrote:
> Hello gpaw-users,
>
> I’m working to install GPAW onto my home cluster and when compiling gpaw-python I’ve run into this issue:
>
>
> mpiicc -o build/bin.linux-x86_64-2.7//gpaw-python build/temp.linux-x86_64-2.7/c/_gpaw.o build/temp.linux-x86_64-2.7/c/bc.o build/temp.linux-x86_64-2.7/c/blacs.o build/temp.linux-x86_64-2.7/c/blas.o build/temp.linux-x86_64-2.7/c/bmgs/bmgs.o build/temp.linux-x86_64-2.7/c/cerf.o build/temp.linux-x86_64-2.7/c/fftw.o build/temp.linux-x86_64-2.7/c/hdf5.o build/temp.linux-x86_64-2.7/c/lapack.o build/temp.linux-x86_64-2.7/c/lcao.o build/temp.linux-x86_64-2.7/c/lfc.o build/temp.linux-x86_64-2.7/c/lfc2.o build/temp.linux-x86_64-2.7/c/localized_functions.o build/temp.linux-x86_64-2.7/c/mlsqr.o build/temp.linux-x86_64-2.7/c/mpi.o build/temp.linux-x86_64-2.7/c/operators.o build/temp.linux-x86_64-2.7/c/plane_wave.o build/temp.linux-x86_64-2.7/c/plt.o build/temp.linux-x86_64-2.7/c/point_charges.o build/temp.linux-x86_64-2.7/c/spline.o build/temp.linux-x86_64-2.7/c/symmetry.o build/temp.linux-x86_64-2.7/c/transformers.o build/temp.linux-x86_64-2.7/c/utilities.o build/temp.linux-x86_64-2.7/c/wigner_seitz.o build/temp.linux-x86_64-2.7/c/woperators.o build/temp.linux-x86_64-2.7/c/xc/ensemble_gga.o build/temp.linux-x86_64-2.7/c/xc/libvdwxc.o build/temp.linux-x86_64-2.7/c/xc/libxc.o build/temp.linux-x86_64-2.7/c/xc/m06l.o build/temp.linux-x86_64-2.7/c/xc/pbe.o build/temp.linux-x86_64-2.7/c/xc/pw91.o build/temp.linux-x86_64-2.7/c/xc/revtpss.o build/temp.linux-x86_64-2.7/c/xc/revtpss_c_pbe.o build/temp.linux-x86_64-2.7/c/xc/rpbe.o build/temp.linux-x86_64-2.7/c/xc/tpss.o build/temp.linux-x86_64-2.7/c/xc/vdw.o build/temp.linux-x86_64-2.7/c/xc/xc.o build/temp.linux-x86_64-2.7/c/xc/xc_mgga.o  -L/share/apps/openmpi/1.10.5/lib -L/share/apps/intel/composer_xe_2015.0.090/mkl/lib/intel64 -L/home/mgroves/DFT/libxc-3.0.0/install/lib -L/home/mgroves/DFT/python/2.7/anaconda2/lib/python2.7/config -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_lapack95_lp64 -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_blas95_lp64 -lpthread -lmkl_rt -lm -lxc -lpython2.7 -lpthread -ldl  -lutil -lm  -L/home/mgroves/DFT/python/2.7/anaconda2/lib -Wl,-rpath=/h
ome/mgroves/DFT/python/2.7/anaconda2/lib,--no-as-needed -Xlinker -export-dynamic
> /home/mgroves/DFT/python/2.7/anaconda2/lib/python2.7/config/libpython2.7.a(posixmodule.o): In function `posix_tmpnam':
> /home/ilan/minonda/conda-bld/work/Python-2.7.12/./Modules/posixmodule.c:7631: warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
> /home/mgroves/DFT/python/2.7/anaconda2/lib/python2.7/config/libpython2.7.a(posixmodule.o): In function `posix_tempnam':
> /home/ilan/minonda/conda-bld/work/Python-2.7.12/./Modules/posixmodule.c:7578: warning: the use of `tempnam' is dangerous, better use `mkstemp'
> build/temp.linux-x86_64-2.7/c/_gpaw.o: In function `main':
> c/_gpaw.c:(.text+0x3d): undefined reference to `__kmpc_begin'
> c/_gpaw.c:(.text+0x97): undefined reference to `__kmpc_end'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `PMPIX_Comm_group_failed'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Datatype_builtin'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Type_contiguous'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Type_free_impl'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Grequest_set_lang_f77'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Wtime_todouble'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_CommGetAttr_fort'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_WinGetAttr'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIU_Handle_get_ptr_indirect'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Keyval_set_proxy'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Type_commit'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Wtick'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPI_WEIGHTS_EMPTY'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_TypeSetAttr'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Err_create_code'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Err_preOrPostInit'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_TypeGetAttr'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Datatype_direct'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_WinSetAttr'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_b_use_gettimeofday'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_CommSetAttr'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Datatype_mem'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Process'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPI_UNWEIGHTED'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `PMPIX_Comm_reenable_anysource'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPID_Datatype_set_contents'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Add_finalize'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `PMPIX_Comm_remote_group_failed'
> /share/apps/intel//impi/5.0.1.035/intel64/lib/libmpifort.so: undefined reference to `MPIR_Err_return_comm'
> running build_scripts
> error: can't copy 'build/bin.linux-x86_64-2.7/gpaw-python': doesn't exist or not a regular file
>
>
> I’m compiling this with openmpi and intel compilers and I’m at a bit of a loss as to how to get this to work.  Here is my customize.py file:
>
> compiler = 'icc'
> mpicompiler = 'mpiicc'
> mpilinker = 'mpiicc'
>
>
> scalapack = True
>
> library_dirs += ['/share/apps/openmpi/1.10.5/lib','/share/apps/intel/composer_xe_2015.0.090/mkl/lib/intel64']
> libraries = ['mkl_intel_lp64' ,'mkl_sequential' ,'mkl_core',
>              'mkl_lapack95_lp64',
>              'mkl_scalapack_lp64', 'mkl_blacs_openmpi_lp64',
>              'mkl_blas95_lp64',
>              'pthread',
>     'mkl_rt','m'
>              ]
>
> libraries += ['xc']
> # change this to your installation directory
> LIBXCDIR='/home/mgroves/DFT/libxc-3.0.0/install/'
> library_dirs += [LIBXCDIR + 'lib']
> include_dirs += [LIBXCDIR + 'include']
>
> extra_compile_args = ['-O3','-fPIC','-std=c99']
>
> extra_compile_args += ['-qopenmp']
>
> define_macros += [('GPAW_NO_UNDERSCORE_CBLACS', '1')]
> define_macros += [('GPAW_NO_UNDERSCORE_CSCALAPACK', '1')]
> define_macros += [('PARALLEL', '1')]
> define_macros += [("GPAW_ASYNC",1)]
> define_macros += [("GPAW_MPI2",1)]
>
>
> Here is my PATH variable
>
> [mgroves at kepler gpaw-1.1.0]$ echo $PATH
> /share/apps/intel//impi/5.0.1.035/intel64/bin:/share/apps/intel//impi/5.0.1.035/intel64/bin:/share/apps/intel/composer_xe_2015.0.090/bin/intel64:/share/apps/intel/composer_xe_2015.0.090/mpirt/bin/intel64:/share/apps/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/bin:/share/apps/intel/bin:/share/apps/intel/advisor_xe_2015.1.0.367266/bin64:/share/apps/intel/vtune_amplifier_xe_2015.1.0.367959/bin64:/share/apps/intel/inspector_xe_2015.1.0.366509/bin64:/share/apps/intel//impi/5.0.1.035/intel64/bin:/share/apps/intel//itac/9.0.1.033/intel64/bin:/share/apps/intel/composer_xe_2015.0.090/bin/intel64:/share/apps/intel/composer_xe_2015.0.090/mpirt/bin/intel64:/share/apps/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/bin:/home/mgroves/DFT/ase/tools:/home/mgroves/DFT/python/2.7/anaconda2/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/ibutils/bin:/usr/java/latest/bin:/opt/maven/bin:/opt/maui/bin:/opt/torque/bin:/opt/torque/sbin:/opt/pdsh/bin:/opt/rocks/bin:/opt/rocks/sbin:/home/mgroves/bin
>
>
>
> And here is my LD_LIBRARY_PATH
>
> [mgroves at kepler gpaw-1.1.0]$ echo $LD_LIBRARY_PATH
> /share/apps/openmpi/1.10.5/lib:/home/mgroves/DFT/libxc-3.0.0/install/lib:/share/apps/intel//impi/5.0.1.035/intel64/lib:/share/apps/intel//impi/5.0.1.035/intel64/lib:/share/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/mpirt/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/ipp/../compiler/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/ipp/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/mkl/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4:/share/apps/intel//impi/5.0.1.035/intel64/lib:/share/apps/intel//itac/9.0.1.033/mic/slib:/share/apps/intel//itac/9.0.1.033/intel64/slib:/share/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/mpirt/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/ipp/../compiler/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/ipp/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/mkl/lib/intel64:/share/apps/intel/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4::/opt/python/lib
>
>
>
> Does anyone know as to what I’m missing so that gpaw-python finds these undefined references?
>
> Thanks!
>
> Michael
>
>
> Michael Groves, PhD
> Assistant Professor
> Department of Chemistry and Biochemistry
> California State University, Fullerton
> mgroves at fullerton.edu<mailto:mgroves at fullerton.edu>
> (657)278-7018
>
>
>
> _______________________________________________
> gpaw-users mailing list
> gpaw-users at listserv.fysik.dtu.dk
> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
>




More information about the gpaw-users mailing list