[gpaw-users] MPI-Scalapack library for Python

[Ask Hjorth Larsen]

Hi,

2017-09-21 14:50 GMT+02:00 marc via gpaw-users
<gpaw-users at listserv.fysik.dtu.dk>:
> Hi Mikael,
>
> nice to see that you are interested in writing a Scalapack wrapper. I think
> it is really missing in python.
> I found one, name scalapy. But it is not maintained anymore and the
> installation failed. We could try to write something similar using the code
> from GPAW.

It would be very useful to have a well-designed stack of Python
binding for BLAS, LAPACK, MPI, and ScaLAPACK.

The sane thing to do is to rely on numpy for array support and mpi4py
for the MPI interface, and then write the remaining parts of the
interface (BLAS, LAPACK, ScaLAPACK).  Since C/Fortran interfaces
involve a lot of tedious boiler plate code (Fortran/C interfaces,
headers, ...), I imagine that a lot of interface code could be
generated or use macros.

GPAW has an object oriented interface for setting up parallel
ScaLAPACK layouts and calling a few ScaLAPACK functions (I wrote the
OO part of the interface).  Its only limitation is that it depends on
GPAW's own MPI interface rather than mpi4py, and therefore is less
general than a complete interface.  One would need to port it to
mpi4py to have independent MPI support.

GPAW is GPL3+, to be exact.

Best regards
Ask

>
> Best regards,
> Marc
>
>
> On 09/21/2017 01:42 PM, Kuisma, Mikael wrote:
>
> Hi!
>
> I added some Scalapack/blacs routines while implementing the LCAO-TDDFT for
> GPAW and it was somewhat cumbersome. Also, the API we have in GPAW is
> somewhat tedious to use at the moment: we have a lot of code to handle
> ScaLAPACK and LAPACK calls as different cases. So I would vote for writing a
> general Python-Scalapack library (instead that you take code from GPAW to
> your Python/C code).  We could make this library transparent, so it would
> work also without ScaLAPACK performing serial linear algebra..
>
> ScaPACK does not parallize as well as I would like it to parallelize.  With
> a library wrapper, one could also try other libraries (for eigensolvers,
> there are at least more parallel solvers than ScaLAPACK).
>
> One more remark regarding GPAW: I always resist adding external libraries to
> GPAW, so if one does a python with c-bindings library, I think it would make
> sense to embed that Python and C-code directly to GPAW repository. The end
> users do not like install new dependencies every time the version number
> increases by 0.1. (I wish that we should also provide basic working
> installation of libxc and libvdwxc as well in our repository, with option to
> link to external libraries).
>
> I do not know about licencing issues that well, but as far as I think, it
> should be ok to take code from GPAW (GPL), if your code is also GPL and make
> your code available.
>
> BR,
> Mikael Kuisma
>
> Academy of Finland Post Doc
>
> Nanoscience Center
> Department of Chemistry
> University of Jyväskylä
> P.O. Box 35
> FIN-40014 University of Jyväskylä
> Finland
>
> E-mail: mikael.j.kuisma (at) jyu.fi
> Phone: +358 40 077 4114
>
> https://scholar.google.se/citations?user=w0eP9esAAAAJ
>
> Visiting address: NSC, 3rd floor, YN320
>
> ________________________________________
> Lähettäjä: gpaw-users-bounces at listserv.fysik.dtu.dk
> [gpaw-users-bounces at listserv.fysik.dtu.dk] k&#228;ytt&#228;j&#228;n marc via
> gpaw-users [gpaw-users at listserv.fysik.dtu.dk] puolesta
> Lähetetty: 21. syyskuuta 2017 14:10
> Vastaanottaja: gpaw-users at listserv.fysik.dtu.dk
> Kopio: Peter Koval
> Aihe: [gpaw-users] MPI-Scalapack library for Python
>
> Dear Gpaw developers,
>
> We are actually developing a python code to perform efficient TDDFT
> calculation with LCAO. Our program can read DFT data from different code
> such as Siesta, OpenMX, Pyscf and GPAW (with some limitations for the
> moment).
> We would like to parallelize our code using MPI and Scalapack but we
> could not find any library in python that wrap both MPI and Scalapack.
> Looking to the GPAW code, we have seen that you wrote your own wrapper
> for MPI-Saclapack. My question is the following, would it be difficult
> to modify your MPI wrapper to use it in our code? Is it feasible from a
> license point of view?
> Do you think it could be feasible to write a general python wrapper for
> MPI and Scalapack in an other library using your implementation?
>
> Best regards,
> Marc Barbry
> _______________________________________________
> gpaw-users mailing list
> gpaw-users at listserv.fysik.dtu.dk
> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
>
>
>
> _______________________________________________
> gpaw-users mailing list
> gpaw-users at listserv.fysik.dtu.dk
> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users