[gpaw-users] About temporal scales of merge requests

[Ask Hjorth Larsen]

Hi Mikael,

2017-11-28 19:44 GMT+01:00 Kuisma, Mikael via gpaw-users
<gpaw-users at listserv.fysik.dtu.dk>:
> Hello!
>
> At the moment, there are ongoing development processes with time constants
> deviating by an order of magnitude. The scientific developments are in
> timescale of months or even few years and the restructuring, streamlining
> and maintenance the classes takes place in from hours to weeks.

Right, JJ and I have been working on a lot of changes these days, so
first of all sorry if anything has been broken.  Note that below, I
can only speak for myself.

The recent merge of the 'matrices' branch (written almost exclusively
by JJ) has actually been underway for 6+ months, I think, so these
changes are not made from one day to another.  We made an effort just
now to fix any remaining problems and merge it, so as to avoid
diverging even more from master.  This is not just code maintenance
but also adds spinors.

But it gets much worse: The idea for the next big batch of changes is
to make the whole state of the GPAW object simpler and less prone to
bugs (the infamous calc.set() comes to mind).  I am afraid this will
also grossly break compatibility with any branch that is not public.
These are quite large changes that hopefully shouldn't be necessary
very often.

Anyway my personal feeling/opinion is that I am somewhat liable for
causing problems with existing merge requests, but not at all for
private branches.

>
> The fell that development of new features has difficulies, as the code bases
> of the different temporal scale developments are to large extent offsync.
> Currently, there is a tendency only to accept documented, well defined and
> finished features to the trunk. I appreciate and agree to this to some
> extent, but this is apt to suppress the testing and adaptation of new
> features in the current development framework. It can easily happen, that a
> new feature is being developed for a year in a custom branch without a
> single merge request, or test, making it very fragile. This means that any

Well, it would be wise not to do things like that at all, particularly
the thing about not even having a test :(.  Better put it on Gitlab as
a WIP merge request, and initiate a discussion.  It is very reasonable
to request that a larger feature can be developed over the course of
an extended period within master, so as long as it is properly tested.
Keeping compatibility of the whole API to avoid breaking possible
private branches is much too restrictive - then the code will be
completely locked down.

> rapid merge request potentially breaks all or some of the ongoing
> developments, and it is left up to the all of the
> non-software-developer-expert developers to change their codes.
>
> I hope that we could make the development version more open to merge
> requests. Here are few suggestions and rules to remedy the situation, I hope
> you would consider:
>
> 1. One establishes two main branches, a development and  main branch.
> 2. The core of GPAW is to be maintained identically in the two branches
> (practically meaning the gpaw main folder and few other relevant
> subfolders).
> 3. Upon addition of new features, the core of GPAW should be changed
> minimally.
> 4. If a new feature of GPAW requires changes of GPAW core, a simultaneous
> merge request is done to both branches for the core changes. These should be
> carefully audited and discussed.
> 5. Otherwise, the merge requests of features should be done (daily, weekly)
> to the development branch.
> 6. All merge requests are automatically accepted to the development branch.
> 7. Tests are run nightly in the development branch. Test breaking commits
> can be rolled back.
> 8. Changes in the core of GPAW should be made to both branches. This way,
> the core feature changes breaking 10 different feature developments can be
> detected early before too much offsync has happened.
> 9. After the feature is finished, the file/subfolder of the feature is
> merged to the main branch (and possible residual GPAW core changes should be
> merged also) .
> 10. The new GPAW tagged versions can be released from the main branch, so it
> only contains finished and documented features.
>
> I gave some thought to this outline, as I have experience of tens of
> breakdowns of my off-main-branch code due to new updates. The most important
> thing are the tests, and this would bring the tests also for the unfinished
> development features. In addition, it would make the community more aware of
> the ongoing developments, as they would all be ongoing in a common branch. I
> can also see some scientific benefit from the more rapid exchange of
> development intentions. Also, I have a feeling that there might be easier
> solutions also to this problem (do not get me wrong, I don't feel that this
> is  a big problem, but a bit annoying I have to say). For example, one could
> somehow make the merge requests to be accepted more easily.

If it breaks down for you all the time, it must be because you have
many changes in many places.  Maybe it should have been merged already
then.

So in conclusion I think the best solution is to develop major
features within master, and make sure they never diverge too much.
Obviously it requires convincing someone to merge it in the first
place, but that is a question of communication.

Some features are just 'add-ons' that don't make changes within the
main modules.  They don't need such frequent merges because there
won't be git conflicts, and thus it should not be too difficult to fix
them if some variables are renamed in master.  I don't know what
changes you refer to though, so it is difficult for me to comment very
intelligently.

Best regards
Ask

>
> Best Regards.
> Mikael
>
>
> _______________________________________________
> gpaw-users mailing list
> gpaw-users at listserv.fysik.dtu.dk
> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users