[gpaw-users] Different MPI worlds ASE vs. GPAW: small fix, big fix or migrating to mpi4py?

[Ask Hjorth Larsen]

Hi,

2018-08-09 2:56 GMT-05:00 Jens Jørgen Mortensen <jjmo at dtu.dk>:
> On 08/09/2018 05:46 AM, Ask Hjorth Larsen via gpaw-users wrote:
>> 2018-08-08 13:27 GMT-05:00 Gaël Donval <G.Donval at bath.ac.uk>:
>>>> 2018-08-08 9:59 GMT-05:00 Gaël Donval via gpaw-users
(...)
>> When ASE is imported, it does not know whether the user is later going
>> to import GPAW or ASAP or mpi4py, and hence whether to get a
>> communicator from there.  But a ton of mechanisms need to know whether
>> things will occur in parallel, and the way they are written, they
>> cannot change dynamically.  They could be rewritten to change
>> dynamically, but I think it would be very prone to regressions.
>
>
> It would be a bit of work, but doable and worth it in my opinion. In the
> long run, I'd like to get rid of gpaw-python and just have _gpaw.so.
>
> We could start by getting rid of all "from ase.parallel import rank" and
> "from gpaw.mpi import rank" and maybe add a deprecation warning (don't know
> how that can be done).  Then we could create a new ase.parallel.world object
> that can be modified at run time.
>
> I think switching to mpi4py would be difficult and, as Ask mentioned, not so
> nice for users.  And our C-extension still needs to call MPI functions.
>
> Jens Jørgen

I would love to kill the global size/rank numbers and only have a
communicator - I have cursed those variables many times.  Classes
cannot implement getattr AFAIK (I think that becomes possible in 3.7
or so?) so I don't know how to easily we can get a deprecation
warning.

Having our own interpreter is good for two things: 1) we control
command-line arguments (see gpaw-python --help) and 2) we can control
startup/closing code like parallel imports and stack trace formatting.
When gpaw is used by a normal Python interpreter, I don't like doing
those things automatically because it 'corrupts' the user's runtime.
However in principle these things can all be replaced by the user
running 'gpaw <command>' - then we "own" the runtime and can do these
things with a clean conscience.  Only when we are being used as a
library must we be humble.

Best regards
Ask