[ase-users] "Calculator" with client/server connectivity across machines

[Ask Hjorth Larsen]

Dear Andrew,

Am Mi., 13. Mai 2020 um 15:29 Uhr schrieb Andrew Logsdail via
ase-users <ase-users at listserv.fysik.dtu.dk>:
>
> Dear ASE-users,
>
> Recently, we have been encountering issues with system administrators when jobs using ASE have extensive serial compute components compare with the parallel executions (i.e. longer Pythonic part rather than QM energy calculations). The most prevalent example is when coupling with CatLearn, as we find that optimisation on the ML landscape can in some cases become much longer than QM energy calculations used to define the landscape (ML optimisation cost increases with number of data points).
>
> There are several possible ways this could be tackled, and one that came forward in our discussions recently was about running the ASE jobs on a local computer, and defining a calculator object capable of connecting to a remote client through ssh that could then submitting, wait for and analyse jobs, returning necessary data to the host machine for continuation of the scripts (as per the standard ASE calculation). I thought perhaps something like this would already exist but I cannot immediately see functionality that matches the description (the closest I think is the i-Pi sockets, but I think this is designed for the same machine?). Is anyone able to advise if such functionality does exist somewhere? And if not, is there a deliberate reason to avoid this kind of inter-computer connectivity? It’d be something we’d be interesting in investigating further if deemed appropriate.

The socketio with i-PI protocol works remotely, but it's trickier to
set up because the initial input file must exist on the compute node
since the protocol itself only supports changing positions etc.  Feel
free to tinker with that.

You can perhaps set it up so ASE talks to another ASE (the socketio
/client/) remotely, so the remotely running part can generate input
files.  Then you'd have one remote ASE-ASE socket connection, and one
local ASE-<computational code> socket connection inside the compute
node.

Requires tinkering.

However the sysadmins probably won't be much happier: The compute
nodes will still sit still while waiting for the new geometry.  A more
likely solution is to parallelize or otherwise optimize the learning
step, as this seems to be the actual bottleneck.

Best regards
Ask

>
> Thanks in advance,
>
> Andy
>
> —
>
> Dr. Andrew Logsdail
>
> School of Chemistry,
> Cardiff University,
> Main Building,
> Park Pl,
> Cardiff CF10 3AT
>
> Yr Ysgol Cemeg,
> Prifysgol Caerdydd,
> Y Prif Adeilad,
> Plas-y-Parc,
> Caerdydd,
> CF10 3AT
>
> T/F: +44 2922 510 162
> E: LogsdailA at cardiff.ac.uk
> W: http://logsdail.github.io
> @: http://www.twitter.com/a_logsdail
>
> The University welcomes correspondence in Welsh or English.
> Mae'r Brifysgol yn croesawu gohebiaeth yn Gymraeg neu'n Saesneg.
>
> Please note: I sometimes work irregular hours. I don’t expect you to read, action or respond out of your normal work hours.
>
>
> _______________________________________________
> ase-users mailing list
> ase-users at listserv.fysik.dtu.dk
> https://listserv.fysik.dtu.dk/mailman/listinfo/ase-users