Keeping URIs encoded in the acquire system depends on having them
encoded in the first place. While many other places got the encoding
2 out of 3 ArchiveURI implementations were missed which are in practice
responsible for nearly all of the URI building, just that index filename
do not contain characters to escape and the Filename fields in Packages
files usually aren't. Usually. Except if you happen to have e.g. an epoch
featuring package with the colon encoded in the filename. On the upside,
in most repositories the epoch isn't part of the filename.
Reported-By: Johannes 'josch' Schauer on IRC
If a package is not installed yet, we do need to apply
phasing as we otherwise get into weird situations when
In the launchpad bug below, ubuntu-release-upgrader-core
was installed, and hence the phasing for the upgrade to it
was applied. However, ubuntu-release-upgrader-gtk was about
to be installed - and hence the phasing did not apply, causing
a version mismatch, because ubuntu-release-upgrader-gtk from
-updates was used, but -core from release pocket. Sigh.
An alternative approach to dealing with this issue could be to
apply phasing to all packages within the same source package,
which would work in most cases. However, there might be unforeseen
side effects and it is of course possible to have = depends between
source packages, such as -signed packages on the unsigned ones for
This problem does not occur in the update-manager implementation
of phased updates as update-manager only deals with upgrading packages,
but does not install new packages and thus does not see that issue. APT
however, has to apply phasing more broadly, as you can and often do
install additional packages during upgrade, or upgrade packages during
install commands, as both accept package list arguments and have the
same code in the backend.
(cherry picked from commit 81eb944c76)
The JSON encoder only looked at the top state, but did not
pop it, so if we nested objects, we got stuck in whatever
the last state we pushed aside was, so in our example, we
wrongly get a comma inserted _after_ key "b":
(cherry picked from commit 09e3c0c855)
This allows us to correctly encode strings containing quotation
marks, escape characters and control characters.
The test case is a bit nasty because it embeds private-cachefile.cc
for linkage reasons.
(cherry picked from commit 57b727af4e)
Repositories without Size information for packages are not
proper and need fixing. This ensures people see an error in
CI, and get notifications and hence the ability to fix it.
It can be turned off by setting Acquire::AllowUnsizedPackages
(cherry picked from commit 1412cf5140)
Remove inversed comment for AllowUnsizedPackages
It defaults to false, like the other options there do.
(cherry picked from commit 9d95c64be8)
The maximum request size is accidentally set to any sized file,
so if an unsized file is present, and it turns out to be larger
than the maximum size we set, we'd error out when checking if
its size is smaller than the maximum request size.
(cherry picked from commit 27a4fb3a9a)
MaybeAddAuth() here tells us that it refused to use the credentials
for an http source; but that caused the test suite to fail at a later
stage because we checked if there were any errors/warning. Strangely,
this is only triggered with LTO enabled.
Actually check that the warning is being set and then reject it.
(cherry picked from commit 8613225f31)
If a transaction is doomed we want to gracefully shutdown our zoo of
worker processes. As explained in the referenced commit we do this by
stopping the main process from handing out new work and ignoring the
replies it gets from the workers, so that they eventually run out of
We tested this previously by checking if a rred worker was given work
items at all, but depending on how lucky the stars of the machine
working on this are the worker would have already gotten work before the
transaction was aborted – so we tried this 25 times a row (f35601e5d2).
No machine can be this lucky, right?
Turns out the autopkgtest armhf machine is very lucky.
I feel a bit sorry for feeding grep such a long "line" to work with, but
it seems to work out. Porterbox amdahl (who is considerably less lucky;
had to turn down to 1 try to get it to fail sometimes) is now happily
running the test in an endless loop.
Of course, I could have broken the test now, but its still a rather
generic grep (in some ways more generic even) and the main part of the
testcase – the update process finishes and fails – is untouched.
References: 38f8704e41Closes: #984966
(cherry picked from commit 0d25ce3d46)
The mirror method can distribute requests for files based on various
metadata bits, but some – the main index files – weren't actually
passing those on to the methods as advertised in the manpage.
This is hidden both by mirror usually falling back to other sources
which will eventually hit the right one and that if the repository does
not support by-hash apt will automatically stick to the mirror which was
used for the Release file.
(cherry picked from commit 2a81f98b12)
Especially in small sections of an archive it can happen that an index
returns to a previous state (e.g. if a package was first added and then
removed with no other changes happening in between). The result is that
we have multiple patches which start from the same hash which if we
perform clientside merging is no problem although not ideal as we
perform needless work.
For serverside merging it would not matter, but due to rred previously
refusing to merge zero-size patches but dak ignoring failure letting it
carry these size-zero patches until they naturally expire we run into a
problem as these broken patches won't do and force us to fall back to
downloading the entire index. By always starting from the last patch
instead of the first with the starter hash we can avoid this problem
and behave optimally in clientside merge cases, too.
(cherry picked from commit 59933938f5)
The rred method expects the patches to have a certain name, which we
have to rename the file to before calling the method, but by delaying
the rename we ensure that if the download of one of them fails and a
successful fallback occurs they are all properly cleaned up as no longer
useful while in the error case the next apt run can potentially pick
them up as already downloaded.
Our test-pdiff-usage test was encountering this every other run, but did
not fail as the check for unaccounted files in partial/ was wrapped
in a subshell so that the failure produced failing output, but did not
change the exit code.
(cherry picked from commit 246f66561e)
There isn't a lot of sense in working on empty patches as they change
nothing (quite literally), but they can be the result of merging
multiple patches and so to not require our users to specifically detect
and remove them, we can be nice and just ignore them instead of erroring
(cherry picked from commit 9bd27033c4)
2 years ago
29 changed files with 420 additions and 84 deletions