[gpaw-users] Trouble Compiling GPAW
Tomlinson, Warren (CDR)
wwtomlin at nps.edu
Fri Mar 18 21:19:48 CET 2016
Ask-
I will let you know. Another question. I installed GPAW on a Cray cluster (no errors) and when I did the tests I got a problem I can’t solve. The test fd_ops/laplace.py fails for "Illegal instruction.” I took a look at the program and everything imports fine, but during the very first iteration of the loop the command lap.apply(a_g, b_g) causes the Illegal instruction error. Any ideas on what could be happening?
Thanks
Warren
> On Mar 18, 2016, at 12:37 PM, Ask Hjorth Larsen <asklarsen at gmail.com> wrote:
>
> If it works, I would also like to know it :)
>
> 2016-03-18 19:19 GMT+01:00 Tomlinson, Warren (CDR) <wwtomlin at nps.edu>:
>> Ask-
>> Thanks for the help. I’ll give it a try and let you know if I have any
>> problems.
>> Thanks
>> Warren
>>
>> On Mar 18, 2016, at 3:52 AM, Ask Hjorth Larsen <asklarsen at gmail.com> wrote:
>>
>> Hello
>>
>> 2016-03-17 19:57 GMT+01:00 Tomlinson, Warren (CDR) <wwtomlin at nps.edu>:
>>
>> All-
>> Please ignore my installation problem form my previous email (below).
>> Problem has been solved.
>>
>> On an unrelated note I have a question about a large system I’m
>> working with (656 atoms). It is the unit cell for the UiO-67 MOF (periodic
>> in all three directions). I’m trying to relax the structure and I’ve gotten
>> to a point where the energy is not changing much at all between SCF steps,
>> but the relaxation keeps running. The system seemed to settle relatively
>> quickly to -4163.05 eV (about 200 or so SCF steps), but since then there
>> have been 1063 SCF steps and the energy went from -4163.026398 eV to
>> -4163.069115 eV. The lowest energy over those 1063 SCF steps was
>> -4163.074407. It seems to me that the system is just shifting around a
>> minimum, but never quite settling there. Do you have any tips for
>> addressing this?
>>
>> Below is my python file:
>>
>> from ase import Atoms
>> from ase.io import read, write
>> from ase.ga.relax_attaches import VariansBreak
>> from gpaw import GPAW, Mixer, FermiDirac
>> from ase.optimize import QuasiNewton
>> from gpaw.poisson import PoissonSolver
>> import numpy as np
>>
>> cell = read('Full_N_Pd_szp_ptl.pdb')
>> cell.set_cell([26.52,26.52,26.52])
>> cell.set_pbc(1)
>>
>> calc_LCAO = GPAW(mode='lcao',
>> gpts=(144,144,144),
>> xc='PBE',
>> poissonsolver=PoissonSolver(relax='GS', eps=1e-10),
>> parallel={'band':2,'sl_default':(24,24,64)},
>> basis = 'szp',
>> mixer=Mixer(0.1, 5, weight=100.0),
>>
>>
>> Try lowering the mixer parameter from 0.1 to 0.05. Larger systems
>> tend to require more conservative mixing.
>>
>> If it does not work, please attach the logfile.
>>
>> It looks like you are using a very large number of cores (576). On
>> normal architectures this should probably run on 24-48 cores, or even
>> less given that it is only szp. Adding more cores beyond that may
>> make it faster, but it is very unlikely that it strong-scales well to
>> 576.
>>
>> Best regards
>> Ask
>>
>> occupations=FermiDirac(width=0.1),
>> txt='Full_N_Pd_szp.out',
>> maxiter=1000
>> )
>>
>> cell.set_calculator(calc_LCAO)
>> opt = QuasiNewton(cell)
>> vb = VariansBreak(cell, opt)
>> opt.attach(vb.write)
>> opt.run()
>>
>>
>> Thanks,
>> Warren
>>
>>
>> On Mar 15, 2016, at 12:16 PM, Tomlinson, Warren (CDR) <wwtomlin at nps.edu>
>> wrote:
>>
>> Hello-
>> I’m having an issue compiling GPAW on a cluster and I’m wondering if someone
>> can help.
>>
>> The system is a Cray XC30 cluster (called Lightning)
>> I am attempting to compile in the GNU environment (although I get the same
>> error when trying with Intel)
>>
>> Relevant modules:
>> gcc/5.1.0
>> cray-mpich/7.2.6
>> cray-libsci/13.2.0
>> python/gnu/2.7.9
>> numpy/gnu/1.9.2
>> spicy/gnu/0.15.1
>>
>>
>> Relevant excerpts from system user’s manual:
>> 5.1.1. Message Passing Interface (MPI)
>>
>> This release of MPI-2 derives from Argonne National Laboratory MPICH-2 and
>> implements the MPI-2.2 standard except for spawn support, as documented by
>> the MPI Forum in "MPI: A Message Passing Interface Standard, Version 2.2."
>>
>> The Message Passing Interface (MPI) is part of the software support for
>> parallel programming across a network of computer systems through a
>> technique known as message passing. MPI establishes a practical, portable,
>> efficient, and flexible standard for message passing that makes use of the
>> most attractive features of a number of existing message-passing systems,
>> rather than selecting one of them and adopting it as the standard. See "man
>> intro_mpi" for additional information.
>>
>> When creating an MPI program on Lightning, ensure the following:
>>
>> • That the default MPI module (cray-mpich) has been loaded. To check this,
>> run the "module list" command. If cray-mpich is not listed and a different
>> MPI module is listed, use the following command:
>> module swap other_mpi_module cray-mpich
>>
>> If no MPI module is loaded, load the cray-mpich module.
>>
>> module load cray-mpich
>>
>> • That the source code includes one of the following lines:
>> INCLUDE "mpif.h" ## for Fortran, or
>> #include <mpi.h> ## for C/C++
>>
>> To compile an MPI program, use the following examples:
>>
>> ftn -o pi_program mpi_program.f
>> ## for Fortran, or
>> cc -o mpi_program mpi_program.c ## for C/C++
>>
>> ———————————————————————
>>
>> 5.1.3. Open Multi-Processing (OpenMP)
>>
>> OpenMP is a portable, scalable model that gives programmers a simple and
>> flexible interface for developing parallel applications. It supports
>> shared-memory multiprocessing programming in C, C++ and Fortran, and
>> consists of a set of compiler directives, library routines, and environment
>> variables that influence compilation and run-time behavior.
>>
>> When creating an OpenMP program on Lightning, ensure the following:
>>
>> • That the default MPI module (cray-mpich) has been loaded. To check this,
>> run the "module list" command. If cray-mpich is not listed and a different
>> MPI module is listed, use the following command:
>> module swap other_mpi_module cray-mpich
>>
>> If no MPI module is loaded, load the cray-mpich module.
>>
>> module load cray-mpich
>>
>> • That if using OpenMP functions (for example, omp_get_wtime), the source
>> code includes one of the following lines:
>>
>> INCLUDE 'omp.h' ## for Fortran, or
>> #include <omp.h> ## for C/C++
>>
>> Or, if the code is written in Fortran 90 or later, the following line may be
>> used instead:
>>
>> USE omp_lib
>>
>> • That the compile command includes an option to reference the OpenMP
>> library. The PGI, Cray, Intel, and GNU compilers support OpenMP, and each
>> one uses a different option.
>>
>> To compile an OpenMP program, use the following examples:
>>
>> For C/C++ codes:
>>
>> cc -o OpenMP_program -mp=nonuma OpenMP_program.c ## PGI
>> cc -o OpenMP_program -h omp OpenMP_program.c ## Cray
>> cc -o OpenMP_program -openmp OpenMP_program.c ## Intel
>> cc -o OpenMP_program -fopenmp OpenMP_program.c ## GNU
>>
>> ———————————————————————
>>
>> 5.2. Available Compilers
>>
>> Lightning has four programming environment suites.
>>
>> • Portland Group (PGI)
>> • Cray Fortran and C/C++
>> • Intel
>> • GNU
>> On Lightning, different sets of compilers are used to compile codes for
>> serial vs. parallel execution.
>>
>> Compiling for the Compute Nodes
>>
>> Codes compiled to run on the compute nodes may be serial or parallel. The
>> x86-64 instruction set for Intel Ivy Bridge E5-2697 processors has
>> extensions for the Floating Point Unit (FPU) that require the module
>> craype-ivybridge to be loaded. This module is loaded for you by default. To
>> compile codes for execution on the compute nodes, the same compile commands
>> are available in all programming environment suites as shown in the
>> following table:
>>
>> Compute Node Compiler Commands
>> Language PGI
>> Cray Intel
>> GNU Serial/Parallel
>> C cc
>> cc cc
>> cc Serial/Parallel
>> C++ CC
>> CC CC
>> CC Serial/Parallel
>> Fortran 77 f77
>> f77 f77
>> f77 Serial/Parallel
>> Fortran 90 ftn
>> ftn ftn
>> ftn Serial/Parallel
>> ——————————————————————————————
>>
>>
>> Building the serial version of GPAW gives me no trouble. I have compiled it
>> and run all the tests and they all pass. When trying to build the custom
>> interpreter, the compiling goes fine, but the linking fails.
>>
>> Below is the link line causing the issue:
>> cc -o build/bin.linux-x86_64-2.7//gpaw-python
>> build/temp.linux-x86_64-2.7/c/woperators.o
>> build/temp.linux-x86_64-2.7/c/plt.o build/temp.linux-x86_64-2.7/c/lapack.o
>> build/temp.linux-x86_64-2.7/c/symmetry.o
>> build/temp.linux-x86_64-2.7/c/plane_wave.o
>> build/temp.linux-x86_64-2.7/c/operators.o
>> build/temp.linux-x86_64-2.7/c/mlsqr.o
>> build/temp.linux-x86_64-2.7/c/transformers.o
>> build/temp.linux-x86_64-2.7/c/utilities.o
>> build/temp.linux-x86_64-2.7/c/spline.o build/temp.linux-x86_64-2.7/c/lfc2.o
>> build/temp.linux-x86_64-2.7/c/localized_functions.o
>> build/temp.linux-x86_64-2.7/c/wigner_seitz.o
>> build/temp.linux-x86_64-2.7/c/mpi.o build/temp.linux-x86_64-2.7/c/lfc.o
>> build/temp.linux-x86_64-2.7/c/bc.o build/temp.linux-x86_64-2.7/c/hdf5.o
>> build/temp.linux-x86_64-2.7/c/blas.o build/temp.linux-x86_64-2.7/c/fftw.o
>> build/temp.linux-x86_64-2.7/c/lcao.o
>> build/temp.linux-x86_64-2.7/c/point_charges.o
>> build/temp.linux-x86_64-2.7/c/_gpaw.o build/temp.linux-x86_64-2.7/c/cerf.o
>> build/temp.linux-x86_64-2.7/c/blacs.o
>> build/temp.linux-x86_64-2.7/c/bmgs/bmgs.o
>> build/temp.linux-x86_64-2.7/c/xc/rpbe.o
>> build/temp.linux-x86_64-2.7/c/xc/tpss.o
>> build/temp.linux-x86_64-2.7/c/xc/xc.o
>> build/temp.linux-x86_64-2.7/c/xc/revtpss_c_pbe.o
>> build/temp.linux-x86_64-2.7/c/xc/pbe.o
>> build/temp.linux-x86_64-2.7/c/xc/libxc.o
>> build/temp.linux-x86_64-2.7/c/xc/m06l.o
>> build/temp.linux-x86_64-2.7/c/xc/pw91.o
>> build/temp.linux-x86_64-2.7/c/xc/revtpss.o
>> build/temp.linux-x86_64-2.7/c/xc/ensemble_gga.o
>> build/temp.linux-x86_64-2.7/c/xc/xc_mgga.o
>> build/temp.linux-x86_64-2.7/c/xc/vdw.o -L/home/wwtomlin/xc/lib
>> -L/opt/gcc/5.1.0/snos/lib64 -L/opt/cray/libsci/13.2.0/GNU/5.1/x86_64/lib
>> -L/app/COST/python/2.7.9/gnu/lib -L/opt/cray/mpt/7.2.6/gni/mpich-gnu/51/lib
>> -L/app/COST/python/2.7.9/gnu/lib/python2.7/config -lxc -lpython2.7 -lpthread
>> -ldl -lutil -lm -pg -L. -L/app/COST/bzip2/1.0.6/gnu//lib
>> -L/app/COST/tcltk/8.6.4/gnu//lib
>> -L/app/COST/dependencies/sqlite/3081101/gnu//lib
>> -L/app/COST/dependencies/readline/6.3/gnu//lib -Xlinker -export-dynamic
>>
>>
>> Below are the warnings and error I get:
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(dynload_shlib.o): In function
>> `_PyImport_GetDynLoadFunc':
>> /app/COST/source/Python-2.7.9/Python/dynload_shlib.c:130: warning: Using
>> 'dlopen' in statically linked applications requires at runtime the shared
>> libraries from the glibc version used for linking
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(posixmodule.o): In function
>> `posix_tmpnam':
>> /app/COST/source/Python-2.7.9/./Modules/posixmodule.c:7575: warning: the use
>> of `tmpnam_r' is dangerous, better use `mkstemp'
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(posixmodule.o): In function
>> `posix_tempnam':
>> /app/COST/source/Python-2.7.9/./Modules/posixmodule.c:7522: warning: the use
>> of `tempnam' is dangerous, better use `mkstemp'
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(posixmodule.o): In function
>> `posix_initgroups':
>> /app/COST/source/Python-2.7.9/./Modules/posixmodule.c:4161: warning: Using
>> 'initgroups' in statically linked applications requires at runtime the
>> shared libraries from the glibc version used for linking
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function
>> `pwd_getpwall':
>> /app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:165: warning: Using
>> 'getpwent' in statically linked applications requires at runtime the shared
>> libraries from the glibc version used for linking
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function
>> `pwd_getpwnam':
>> /app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:139: warning: Using
>> 'getpwnam' in statically linked applications requires at runtime the shared
>> libraries from the glibc version used for linking
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function
>> `pwd_getpwuid':
>> /app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:114: warning: Using
>> 'getpwuid' in statically linked applications requires at runtime the shared
>> libraries from the glibc version used for linking
>> /app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function
>> `pwd_getpwall':
>> /app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:164: warning: Using
>> 'setpwent' in statically linked applications requires at runtime the shared
>> libraries from the glibc version used for linking
>> /app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:176: warning: Using
>> 'endpwent' in statically linked applications requires at runtime the shared
>> libraries from the glibc version used for linking
>> /usr/bin/ld: dynamic STT_GNU_IFUNC symbol `strcmp' with pointer equality in
>> `/usr/lib/../lib64/libc.a(strcmp.o)' can not be used when making an
>> executable; recompile with -fPIE and relink with -pie
>> collect2: error: ld returned 1 exit status
>>
>>
>> Finally, here are the lines from my customize.py file:
>> compiler = 'cc'
>> define_macros += [('PARALLEL', '1')]
>> mpicompiler = 'cc'
>> mpilinker = mpicompiler
>>
>> libraries = ['xc']
>> scalapack = True
>>
>> mpi_library_dirs += ['/opt/cray/mpt/7.2.6/gni/mpich-gnu/51/lib']
>> library_dirs += ['/home/wwtomlin/xc/lib']
>> library_dirs += ['/opt/gcc/5.1.0/snos/lib64']
>> library_dirs += ['/opt/cray/libsci/13.2.0/GNU/5.1/x86_64/lib']
>> library_dirs += ['/app/COST/python/2.7.9/gnu/lib']
>>
>> mpi_include_dirs += ['/opt/cray/mpt/7.2.6/gni/mpich-gnu/51/include']
>> include_dirs += ['/home/wwtomlin/xc/include']
>> include_dirs += ['/opt/gcc/5.1.0/snos/include']
>> include_dirs += ['/opt/cray/libsci/13.2.0/GNU/5.1/x86_64/include']
>> include_dirs += ['/app/COST/python/2.7.9/gnu/include']
>>
>> define_macros += [('GPAW_NO_UNDERSCORE_CBLACS', '1')]
>> define_macros += [('GPAW_NO_UNDERSCORE_CSCALAPACK', '1’)]
>>
>>
>> My limited experience in this area tells me that there might be a problem
>> with the way libc.a was compiled, but I’m note sure. I can’t do anything
>> about that directly, but I could bring a problem to the attention of the
>> system administrators and they’re usually pretty helpful getting things
>> updated. Is that what I need to do?
>>
>> Thank you for any help,
>> Warren
>> PhD Student
>> Naval Postgraduate School
>>
>> _______________________________________________
>> gpaw-users mailing list
>> gpaw-users at listserv.fysik.dtu.dk
>> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
>>
>>
>>
>> _______________________________________________
>> gpaw-users mailing list
>> gpaw-users at listserv.fysik.dtu.dk
>> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
>>
>>
More information about the gpaw-users
mailing list