[gpaw-users] About temporal scales of merge requests

[Jens Jørgen Mortensen]

Den 28-11-2017 kl. 23:34 skrev Ask Hjorth Larsen via gpaw-users:
> Hi Mikael,
>
> 2017-11-28 19:44 GMT+01:00 Kuisma, Mikael via gpaw-users
> <gpaw-users at listserv.fysik.dtu.dk>:
>> Hello!
>>
>> At the moment, there are ongoing development processes with time constants
>> deviating by an order of magnitude. The scientific developments are in
>> timescale of months or even few years and the restructuring, streamlining
>> and maintenance the classes takes place in from hours to weeks.
> Right, JJ and I have been working on a lot of changes these days, so
> first of all sorry if anything has been broken.  Note that below, I
> can only speak for myself.
>
> The recent merge of the 'matrices' branch (written almost exclusively
> by JJ) has actually been underway for 6+ months, I think, so these
> changes are not made from one day to another.  We made an effort just
> now to fix any remaining problems and merge it, so as to avoid
> diverging even more from master.  This is not just code maintenance
> but also adds spinors.
>
> But it gets much worse: The idea for the next big batch of changes is
> to make the whole state of the GPAW object simpler and less prone to
> bugs (the infamous calc.set() comes to mind).  I am afraid this will
> also grossly break compatibility with any branch that is not public.
> These are quite large changes that hopefully shouldn't be necessary
> very often.
>
> Anyway my personal feeling/opinion is that I am somewhat liable for
> causing problems with existing merge requests, but not at all for
> private branches.
>
>> The fell that development of new features has difficulies, as the code bases
>> of the different temporal scale developments are to large extent offsync.
>> Currently, there is a tendency only to accept documented, well defined and
>> finished features to the trunk. I appreciate and agree to this to some
>> extent, but this is apt to suppress the testing and adaptation of new
>> features in the current development framework. It can easily happen, that a
>> new feature is being developed for a year in a custom branch without a
>> single merge request, or test, making it very fragile. This means that any
> Well, it would be wise not to do things like that at all, particularly
> the thing about not even having a test :(.  Better put it on Gitlab as
> a WIP merge request, and initiate a discussion.  It is very reasonable
> to request that a larger feature can be developed over the course of
> an extended period within master, so as long as it is properly tested.
> Keeping compatibility of the whole API to avoid breaking possible
> private branches is much too restrictive - then the code will be
> completely locked down.
>
>> rapid merge request potentially breaks all or some of the ongoing
>> developments, and it is left up to the all of the
>> non-software-developer-expert developers to change their codes.
>>
>> I hope that we could make the development version more open to merge
>> requests. Here are few suggestions and rules to remedy the situation, I hope
>> you would consider:
>>
>> 1. One establishes two main branches, a development and  main branch.
>> 2. The core of GPAW is to be maintained identically in the two branches
>> (practically meaning the gpaw main folder and few other relevant
>> subfolders).
>> 3. Upon addition of new features, the core of GPAW should be changed
>> minimally.
>> 4. If a new feature of GPAW requires changes of GPAW core, a simultaneous
>> merge request is done to both branches for the core changes. These should be
>> carefully audited and discussed.
>> 5. Otherwise, the merge requests of features should be done (daily, weekly)
>> to the development branch.
>> 6. All merge requests are automatically accepted to the development branch.
>> 7. Tests are run nightly in the development branch. Test breaking commits
>> can be rolled back.
>> 8. Changes in the core of GPAW should be made to both branches. This way,
>> the core feature changes breaking 10 different feature developments can be
>> detected early before too much offsync has happened.
>> 9. After the feature is finished, the file/subfolder of the feature is
>> merged to the main branch (and possible residual GPAW core changes should be
>> merged also) .
>> 10. The new GPAW tagged versions can be released from the main branch, so it
>> only contains finished and documented features.
>>
>> I gave some thought to this outline, as I have experience of tens of
>> breakdowns of my off-main-branch code due to new updates. The most important
>> thing are the tests, and this would bring the tests also for the unfinished
>> development features. In addition, it would make the community more aware of
>> the ongoing developments, as they would all be ongoing in a common branch. I
>> can also see some scientific benefit from the more rapid exchange of
>> development intentions. Also, I have a feeling that there might be easier
>> solutions also to this problem (do not get me wrong, I don't feel that this
>> is  a big problem, but a bit annoying I have to say). For example, one could
>> somehow make the merge requests to be accepted more easily.
> If it breaks down for you all the time, it must be because you have
> many changes in many places.  Maybe it should have been merged already
> then.
>
> So in conclusion I think the best solution is to develop major
> features within master, and make sure they never diverge too much.
> Obviously it requires convincing someone to merge it in the first
> place, but that is a question of communication.

I also think the main problem here is lack of communication.  We should 
all become better at talking to each other.

I think the only way to develop code in parallel is to merge often 
(either to master or from master) and communicate.  A good way to 
communicate is to actually visit each other.  If anyone would like to 
come to DTU for a longer stay then he or she is very welcome.

Jens Jørgen

>
> Some features are just 'add-ons' that don't make changes within the
> main modules.  They don't need such frequent merges because there
> won't be git conflicts, and thus it should not be too difficult to fix
> them if some variables are renamed in master.  I don't know what
> changes you refer to though, so it is difficult for me to comment very
> intelligently.
>
> Best regards
> Ask
>
>> Best Regards.
>> Mikael
>>
>>
>> _______________________________________________
>> gpaw-users mailing list
>> gpaw-users at listserv.fysik.dtu.dk
>> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
> _______________________________________________
> gpaw-users mailing list
> gpaw-users at listserv.fysik.dtu.dk
> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users