[ase-users] [Fwd: Re: Opinion about potentially adding Cython as a dependency]

[Gaël Donval]

Really no reason to go off-list except my clicking on the wrong
button...

On Thu, 2017-10-12 at 16:33 +0200, Jens Jørgen Mortensen wrote:
> On 10/12/2017 04:01 PM, Gaël Donval wrote:
> > > If it can be done on a wholly opt-in basis then I think it is
> > > great.
> > > The question of C extensions and dependencies on compiled
> > > libraries
> > > have come up several times because they have legitimate
> > > applications,
> > > but anything that makes installation trickier will also affect
> > > users
> > > who don't need those features.  And then there's the question of
> > > release testing (platform dependence).
> > 
> > I suspect releasing wheels will solve the problem for most of us
> > anyway. Apparently, people usually take advantage CI systems to
> > generate theirs but I have not tried it myself.
> > 
> > So my take out of this it: it is fine to use Cython provided that
> > its
> > absence (or the absence of working compiler) doesn't affect people
> > who
> > are not using the functionality. So I guess, at best, fall-back
> > function should be provided or, if really impractical, no failure
> > should occur as long as the specific functionalities are not used.
> > 
> > > Can we establish a working minimal example where ASE builds and
> > > uses
> > > Cython to perform some trivial action, and see how it affects the
> > > build requiements?
> > 
> > I don't have time to do that just now but I can implement existing
> > functions in, say, ase/geometry/distance.py in cython this week-
> > end. It
> > would be small and decoupled enough not to be too invasive and yet
> > provide a useful sandbox.
> 
> You haven't told us what it is you need C-code for.  If you want to 
> calculate distances efficiently then maybe take a look at the 
> scipy.spatial module.  Maybe there is some help there and adding
> scipy 
> as an ASE dependency would be fine for me (it sort of already is).

Oh, sorry. Just fast periodic distances and accessible surface areas
for the time being. I am currently extensively using cKDTree (indeed
from scipy.spatial) for that purpose. Surface area is an interesting
property to know about if you do catalysis or adsorption experiments
for instance but not only... The same code could also trivially be used
to know where to add atoms on a surface or to get a first approximation
of whether a given window is considered opened for instance...

It's working fast enough but the surface area part is very brittle at
the moment: it works but I used an ugly hack I really want to get rid
of. 

I think I am about to find a fast non-cython solution in this case. But
if I implement something like He pore volume or pore size distribution,
I'm on my own. I guess providing a supported way to bind to
C/C++/Fortran in ASE would also prove to be useful in other cases but I
don't want to speculate.

Gaël

> 
> Jens Jørgen
> 
> > Cheers,
> > Gaël
> > 
> > 
> > > Best regards
> > > Ask
> > > 
> > > 2017-10-11 6:47 GMT+02:00 Jens Jørgen Mortensen <jjmo at dtu.dk>:
> > > > Den 10-10-2017 kl. 16:32 skrev Gaël Donval via ase-users:
> > > > 
> > > > Dear Ask,
> > > > 
> > > > Dear Gaël,
> > > > 
> > > > 2017-10-10 12:52 GMT+02:00 Gaël Donval via ase-users
> > > > <ase-users at listserv.fysik.dtu.dk>:
> > > > 
> > > > Dear list,
> > > > 
> > > > I am currently working on some geometrical code that I will
> > > > ultimately
> > > > propose adding to ASE. I am currently using Cython to speed up
> > > > what
> > > > needs speeding up and, to a more limited extent, I also use it
> > > > to
> > > > very
> > > > conveniently generate bindings to C.
> > > > 
> > > > The question is: would anyone see using Cython as a problem?
> > > > 
> > > > I know that the fewer dependencies, the better. However Cython
> > > > is
> > > > almost as ubiquitous as Scipy nowadays in Python's scientific
> > > > stack.
> > > > 
> > > > The only real drawbacks I can think of (and I can read about on
> > > > the
> > > > internet) are pain points on Windows and incompatibility with
> > > > different
> > > > Python implementations (e.g. PyPy). However, with pip wheels,
> > > > Windows
> > > > is a non problem (most Python distributions on Windows now also
> > > > include
> > > > Cython by default so manual compilation is far from the pain it
> > > > used to
> > > > be) and ASE/GPAW do not support PyPy anyway because of all the
> > > > C
> > > > dependencies.
> > > > 
> > > > Note that I am *not* proposing to convert any existing code to
> > > > Cython
> > > > nor to use Cython by default. I am talking about specific cases
> > > > which
> > > > would currently be handled by writing pure C code.
> > > > 
> > > > There are already a few cases where performance becomes an
> > > > issue,
> > > > so
> > > > this is worth thinking about.  Can Cython be an optional
> > > > dependency?
> > > > 
> > > > Yes it can and at two different levels:
> > > > 
> > > >      1) *Provide fallback functions* Nothing would really
> > > > prevent us
> > > > from
> > > >      providing two different versions of hot functions and
> > > > degrade
> > > > to
> > > >      pure Python ones upon import if needed. However this would
> > > > require
> > > >      to keep function APIs in sync in two different places.
> > > > 
> > > >      On the bright side, the slow and hopefully correct Python
> > > > function
> > > >      could be used as a reference in unit tests: these would
> > > > rarely
> > > > be
> > > >      changed and most of the optimisation would occur in the
> > > > Cython
> > > > ones
> > > >      instead.
> > > > 
> > > >      2) *Provide C/C++ files* Ultimately, Cython generates a
> > > > bunch
> > > > of C
> > > >      files containing Cython-defined code + everything needed
> > > > to
> > > > create a
> > > >      native Python module. Directly using those C files (thus
> > > > not
> > > >      requiring Cython itself) is just a matter of tracking them
> > > > in
> > > > git!
> > > >      :) That kind of optional Cython support can be reflected
> > > > in
> > > > setup.py
> > > >      directly (look for USE_CYTHON    ):
> > > > 
> > > > 
> > > > https://gist.github.com/ctokheim/6c34dc1d672afca0676a#setuppy-a
> > > > nd-d
> > > > isutils
> > > > 
> > > > (note however that a C compiler is still needed).
> > > > 
> > > > 
> > > > I would like "pip install ase" to always just work, and for
> > > > that I
> > > > think we
> > > > would need 1).  Alternatively, we should provide wheels for all
> > > > architectures (manylinux1, win32, macosx, ...), but I don't
> > > > think
> > > > we have
> > > > the resources to do that.
> > > > 
> > > > PS: You can always start with the slow Python version.
> > > > 
> > > > Jens Jørgen
> > > > 
> > > > 
> > > > Regards,
> > > > Gaël
> > > > 
> > > > Best regards
> > > > Ask
> > > > 
> > > > Best regards,
> > > > Gaël
> > > > 
> > > > 
> > > > _______________________________________________
> > > > ase-users mailing list
> > > > ase-users at listserv.fysik.dtu.dk
> > > > https://listserv.fysik.dtu.dk/mailman/listinfo/ase-users
> > > > 
> > > > 
> > > > 
> > > > _______________________________________________
> > > > ase-users mailing list
> > > > ase-users at listserv.fysik.dtu.dk
> > > > https://listserv.fysik.dtu.dk/mailman/listinfo/ase-users
> > > > 
> > > > 
> 
>