[gpaw-users] Trouble Compiling GPAW
Tomlinson, Warren (CDR)
wwtomlin at nps.edu
Fri Mar 18 19:19:58 CET 2016
Ask-
Thanks for the help. I’ll give it a try and let you know if I have any problems.
Thanks
Warren
On Mar 18, 2016, at 3:52 AM, Ask Hjorth Larsen <asklarsen at gmail.com<mailto:asklarsen at gmail.com>> wrote:
Hello
2016-03-17 19:57 GMT+01:00 Tomlinson, Warren (CDR) <wwtomlin at nps.edu<mailto:wwtomlin at nps.edu>>:
All-
Please ignore my installation problem form my previous email (below). Problem has been solved.
On an unrelated note I have a question about a large system I’m working with (656 atoms). It is the unit cell for the UiO-67 MOF (periodic in all three directions). I’m trying to relax the structure and I’ve gotten to a point where the energy is not changing much at all between SCF steps, but the relaxation keeps running. The system seemed to settle relatively quickly to -4163.05 eV (about 200 or so SCF steps), but since then there have been 1063 SCF steps and the energy went from -4163.026398 eV to -4163.069115 eV. The lowest energy over those 1063 SCF steps was -4163.074407. It seems to me that the system is just shifting around a minimum, but never quite settling there. Do you have any tips for addressing this?
Below is my python file:
from ase import Atoms
from ase.io<http://ase.io> import read, write
from ase.ga.relax_attaches import VariansBreak
from gpaw import GPAW, Mixer, FermiDirac
from ase.optimize import QuasiNewton
from gpaw.poisson import PoissonSolver
import numpy as np
cell = read('Full_N_Pd_szp_ptl.pdb')
cell.set_cell([26.52,26.52,26.52])
cell.set_pbc(1)
calc_LCAO = GPAW(mode='lcao',
gpts=(144,144,144),
xc='PBE',
poissonsolver=PoissonSolver(relax='GS', eps=1e-10),
parallel={'band':2,'sl_default':(24,24,64)},
basis = 'szp',
mixer=Mixer(0.1, 5, weight=100.0),
Try lowering the mixer parameter from 0.1 to 0.05. Larger systems
tend to require more conservative mixing.
If it does not work, please attach the logfile.
It looks like you are using a very large number of cores (576). On
normal architectures this should probably run on 24-48 cores, or even
less given that it is only szp. Adding more cores beyond that may
make it faster, but it is very unlikely that it strong-scales well to
576.
Best regards
Ask
occupations=FermiDirac(width=0.1),
txt='Full_N_Pd_szp.out',
maxiter=1000
)
cell.set_calculator(calc_LCAO)
opt = QuasiNewton(cell)
vb = VariansBreak(cell, opt)
opt.attach(vb.write)
opt.run()
Thanks,
Warren
On Mar 15, 2016, at 12:16 PM, Tomlinson, Warren (CDR) <wwtomlin at nps.edu<mailto:wwtomlin at nps.edu>> wrote:
Hello-
I’m having an issue compiling GPAW on a cluster and I’m wondering if someone can help.
The system is a Cray XC30 cluster (called Lightning)
I am attempting to compile in the GNU environment (although I get the same error when trying with Intel)
Relevant modules:
gcc/5.1.0
cray-mpich/7.2.6
cray-libsci/13.2.0
python/gnu/2.7.9
numpy/gnu/1.9.2
spicy/gnu/0.15.1
Relevant excerpts from system user’s manual:
5.1.1. Message Passing Interface (MPI)
This release of MPI-2 derives from Argonne National Laboratory MPICH-2 and implements the MPI-2.2 standard except for spawn support, as documented by the MPI Forum in "MPI: A Message Passing Interface Standard, Version 2.2."
The Message Passing Interface (MPI) is part of the software support for parallel programming across a network of computer systems through a technique known as message passing. MPI establishes a practical, portable, efficient, and flexible standard for message passing that makes use of the most attractive features of a number of existing message-passing systems, rather than selecting one of them and adopting it as the standard. See "man intro_mpi" for additional information.
When creating an MPI program on Lightning, ensure the following:
• That the default MPI module (cray-mpich) has been loaded. To check this, run the "module list" command. If cray-mpich is not listed and a different MPI module is listed, use the following command:
module swap other_mpi_module cray-mpich
If no MPI module is loaded, load the cray-mpich module.
module load cray-mpich
• That the source code includes one of the following lines:
INCLUDE "mpif.h" ## for Fortran, or
#include <mpi.h> ## for C/C++
To compile an MPI program, use the following examples:
ftn -o pi_program mpi_program.f
## for Fortran, or
cc -o mpi_program mpi_program.c ## for C/C++
———————————————————————
5.1.3. Open Multi-Processing (OpenMP)
OpenMP is a portable, scalable model that gives programmers a simple and flexible interface for developing parallel applications. It supports shared-memory multiprocessing programming in C, C++ and Fortran, and consists of a set of compiler directives, library routines, and environment variables that influence compilation and run-time behavior.
When creating an OpenMP program on Lightning, ensure the following:
• That the default MPI module (cray-mpich) has been loaded. To check this, run the "module list" command. If cray-mpich is not listed and a different MPI module is listed, use the following command:
module swap other_mpi_module cray-mpich
If no MPI module is loaded, load the cray-mpich module.
module load cray-mpich
• That if using OpenMP functions (for example, omp_get_wtime), the source code includes one of the following lines:
INCLUDE 'omp.h' ## for Fortran, or
#include <omp.h> ## for C/C++
Or, if the code is written in Fortran 90 or later, the following line may be used instead:
USE omp_lib
• That the compile command includes an option to reference the OpenMP library. The PGI, Cray, Intel, and GNU compilers support OpenMP, and each one uses a different option.
To compile an OpenMP program, use the following examples:
For C/C++ codes:
cc -o OpenMP_program -mp=nonuma OpenMP_program.c ## PGI
cc -o OpenMP_program -h omp OpenMP_program.c ## Cray
cc -o OpenMP_program -openmp OpenMP_program.c ## Intel
cc -o OpenMP_program -fopenmp OpenMP_program.c ## GNU
———————————————————————
5.2. Available Compilers
Lightning has four programming environment suites.
• Portland Group (PGI)
• Cray Fortran and C/C++
• Intel
• GNU
On Lightning, different sets of compilers are used to compile codes for serial vs. parallel execution.
Compiling for the Compute Nodes
Codes compiled to run on the compute nodes may be serial or parallel. The x86-64 instruction set for Intel Ivy Bridge E5-2697 processors has extensions for the Floating Point Unit (FPU) that require the module craype-ivybridge to be loaded. This module is loaded for you by default. To compile codes for execution on the compute nodes, the same compile commands are available in all programming environment suites as shown in the following table:
Compute Node Compiler Commands
Language PGI
Cray Intel
GNU Serial/Parallel
C cc
cc cc
cc Serial/Parallel
C++ CC
CC CC
CC Serial/Parallel
Fortran 77 f77
f77 f77
f77 Serial/Parallel
Fortran 90 ftn
ftn ftn
ftn Serial/Parallel
——————————————————————————————
Building the serial version of GPAW gives me no trouble. I have compiled it and run all the tests and they all pass. When trying to build the custom interpreter, the compiling goes fine, but the linking fails.
Below is the link line causing the issue:
cc -o build/bin.linux-x86_64-2.7//gpaw-python build/temp.linux-x86_64-2.7/c/woperators.o build/temp.linux-x86_64-2.7/c/plt.o build/temp.linux-x86_64-2.7/c/lapack.o build/temp.linux-x86_64-2.7/c/symmetry.o build/temp.linux-x86_64-2.7/c/plane_wave.o build/temp.linux-x86_64-2.7/c/operators.o build/temp.linux-x86_64-2.7/c/mlsqr.o build/temp.linux-x86_64-2.7/c/transformers.o build/temp.linux-x86_64-2.7/c/utilities.o build/temp.linux-x86_64-2.7/c/spline.o build/temp.linux-x86_64-2.7/c/lfc2.o build/temp.linux-x86_64-2.7/c/localized_functions.o build/temp.linux-x86_64-2.7/c/wigner_seitz.o build/temp.linux-x86_64-2.7/c/mpi.o build/temp.linux-x86_64-2.7/c/lfc.o build/temp.linux-x86_64-2.7/c/bc.o build/temp.linux-x86_64-2.7/c/hdf5.o build/temp.linux-x86_64-2.7/c/blas.o build/temp.linux-x86_64-2.7/c/fftw.o build/temp.linux-x86_64-2.7/c/lcao.o build/temp.linux-x86_64-2.7/c/point_charges.o build/temp.linux-x86_64-2.7/c/_gpaw.o build/temp.linux-x86_64-2.7/c/cerf.o build/temp.linux-x86_64-2.7/c/blacs.o build/temp.linux-x86_64-2.7/c/bmgs/bmgs.o build/temp.linux-x86_64-2.7/c/xc/rpbe.o build/temp.linux-x86_64-2.7/c/xc/tpss.o build/temp.linux-x86_64-2.7/c/xc/xc.o build/temp.linux-x86_64-2.7/c/xc/revtpss_c_pbe.o build/temp.linux-x86_64-2.7/c/xc/pbe.o build/temp.linux-x86_64-2.7/c/xc/libxc.o build/temp.linux-x86_64-2.7/c/xc/m06l.o build/temp.linux-x86_64-2.7/c/xc/pw91.o build/temp.linux-x86_64-2.7/c/xc/revtpss.o build/temp.linux-x86_64-2.7/c/xc/ensemble_gga.o build/temp.linux-x86_64-2.7/c/xc/xc_mgga.o build/temp.linux-x86_64-2.7/c/xc/vdw.o -L/home/wwtomlin/xc/lib -L/opt/gcc/5.1.0/snos/lib64 -L/opt/cray/libsci/13.2.0/GNU/5.1/x86_64/lib -L/app/COST/python/2.7.9/gnu/lib -L/opt/cray/mpt/7.2.6/gni/mpich-gnu/51/lib -L/app/COST/python/2.7.9/gnu/lib/python2.7/config -lxc -lpython2.7 -lpthread -ldl -lutil -lm -pg -L. -L/app/COST/bzip2/1.0.6/gnu//lib -L/app/COST/tcltk/8.6.4/gnu//lib -L/app/COST/dependencies/sqlite/3081101/gnu//lib -L/app/COST/dependencies/readline/6.3/gnu//lib -Xlinker -export-dynamic
Below are the warnings and error I get:
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(dynload_shlib.o): In function `_PyImport_GetDynLoadFunc':
/app/COST/source/Python-2.7.9/Python/dynload_shlib.c:130: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(posixmodule.o): In function `posix_tmpnam':
/app/COST/source/Python-2.7.9/./Modules/posixmodule.c:7575: warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(posixmodule.o): In function `posix_tempnam':
/app/COST/source/Python-2.7.9/./Modules/posixmodule.c:7522: warning: the use of `tempnam' is dangerous, better use `mkstemp'
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(posixmodule.o): In function `posix_initgroups':
/app/COST/source/Python-2.7.9/./Modules/posixmodule.c:4161: warning: Using 'initgroups' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function `pwd_getpwall':
/app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:165: warning: Using 'getpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function `pwd_getpwnam':
/app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:139: warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function `pwd_getpwuid':
/app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:114: warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/app/COST/python/2.7.9/gnu/lib/libpython2.7.a(pwdmodule.o): In function `pwd_getpwall':
/app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:164: warning: Using 'setpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/app/COST/source/Python-2.7.9/./Modules/pwdmodule.c:176: warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: dynamic STT_GNU_IFUNC symbol `strcmp' with pointer equality in `/usr/lib/../lib64/libc.a(strcmp.o)' can not be used when making an executable; recompile with -fPIE and relink with -pie
collect2: error: ld returned 1 exit status
Finally, here are the lines from my customize.py file:
compiler = 'cc'
define_macros += [('PARALLEL', '1')]
mpicompiler = 'cc'
mpilinker = mpicompiler
libraries = ['xc']
scalapack = True
mpi_library_dirs += ['/opt/cray/mpt/7.2.6/gni/mpich-gnu/51/lib']
library_dirs += ['/home/wwtomlin/xc/lib']
library_dirs += ['/opt/gcc/5.1.0/snos/lib64']
library_dirs += ['/opt/cray/libsci/13.2.0/GNU/5.1/x86_64/lib']
library_dirs += ['/app/COST/python/2.7.9/gnu/lib']
mpi_include_dirs += ['/opt/cray/mpt/7.2.6/gni/mpich-gnu/51/include']
include_dirs += ['/home/wwtomlin/xc/include']
include_dirs += ['/opt/gcc/5.1.0/snos/include']
include_dirs += ['/opt/cray/libsci/13.2.0/GNU/5.1/x86_64/include']
include_dirs += ['/app/COST/python/2.7.9/gnu/include']
define_macros += [('GPAW_NO_UNDERSCORE_CBLACS', '1')]
define_macros += [('GPAW_NO_UNDERSCORE_CSCALAPACK', '1’)]
My limited experience in this area tells me that there might be a problem with the way libc.a was compiled, but I’m note sure. I can’t do anything about that directly, but I could bring a problem to the attention of the system administrators and they’re usually pretty helpful getting things updated. Is that what I need to do?
Thank you for any help,
Warren
PhD Student
Naval Postgraduate School
_______________________________________________
gpaw-users mailing list
gpaw-users at listserv.fysik.dtu.dk<mailto:gpaw-users at listserv.fysik.dtu.dk>
https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
_______________________________________________
gpaw-users mailing list
gpaw-users at listserv.fysik.dtu.dk<mailto:gpaw-users at listserv.fysik.dtu.dk>
https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.fysik.dtu.dk/pipermail/gpaw-users/attachments/20160318/eb5af734/attachment-0001.html>
More information about the gpaw-users
mailing list