• Evilham's avatar
    Fix locking by using flock instead of tombstone files. · 8b277c4e
    Evilham authored
    Necessary changes to orchestrate.sh:
    
    The way I understand things, there are three processes (A,B,C)
    A. amprolla_update is run with orchestrate.sh (very often!):
    1. there is always a consistent working set in merged which usually points to
       merged-production
    2. before amprolla_update, merged switches to merged-staging (why here and not
       after amprolla_update?)
    3. amprolla_update works against -volatile, during this process that directory
       is not necessarily fully consistent
    4. after amprolla_update, merged-volatile is synchronised to merged-production
    5. merged switches to merged-production
    6. merged-volatile is synchronised to merged-staging
    7. merged-production is synchronised to pkgmaster
    
    B. Sometimes amprolla_merge is run
    C. Sometimes amprolla_merge_contents + amprolla_merge are run
    
    The *intent* of the implemented locks appears to be that only one process out
    of A,B,C is active at a time, but the reality is that this is not formally
    provided, timing just happens to match, but shouldn't be relied upon:
    Existence of an active lock does prevent A, B or C from starting.
    But in A the lock is only active while point A.3. is executed, everything else
    runs without an active lock, so if e.g. A.4 to A.7 take long enough that A.4 is
    running again while A.7 is happening, bogus data will be synchronised to
    pkgmaster.
    Since A.7 is a network operation and A.5 and A.6 are disk operations, delays
    *could* happen and they could lead to this scenario (maybe it has happened at
    some point).
    
    That's why I added the support for the --no-lock-I-am-sure argument to
    amprolla_update, and instead obtain the lock as a step A.0 that is valid
    throughout all process A in the orchestrate.sh script.
    
    With that, I think there is now only need for 2 directories: -volatile and
    -production, with A being redefined as:
    
    A. amprolla_update is run with orchestrate.sh (very often!):
    0. obtain amprolla lock and exit if it can't be obtained
    1. there is always a consistent working set in merged which usually points to
       merged-production
    2. amprolla_update --no-lock-I-am-sure works against -volatile, during this
       process that directory is not necessarily fully consistent
    3. merged switches to merged-volatile
    4. merged-volatile is synchronised to merged-production
    5. merged switches to merged-production
    7. merged-production is synchronised to pkgmaster
    
    I have adapted orchestrate.sh and therefore merged-staging is not used with
    these patches and could be removed in theory.
    8b277c4e
Name
Last commit
Last update
contrib Loading commit data...
doc Loading commit data...
gnupg Loading commit data...
lib Loading commit data...
.gitignore Loading commit data...
LICENSE Loading commit data...
README.md Loading commit data...
TODO.md Loading commit data...
amprolla_init.py Loading commit data...
amprolla_merge.py Loading commit data...
amprolla_merge_contents.py Loading commit data...
amprolla_update.py Loading commit data...
orchestrate.sh Loading commit data...