You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

skip-acquire-same-repository-multiple-times 2.9 KiB

support arch:all data e.g. in separate Packages file Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
5 years ago
support arch:all data e.g. in separate Packages file Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
5 years ago
support arch:all data e.g. in separate Packages file Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
5 years ago
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485
  1. #!/bin/sh
  2. set -e
  3. TESTDIR="$(readlink -f "$(dirname "$0")")"
  4. . "$TESTDIR/framework"
  5. setupenvironment
  6. configarchitecture 'amd64'
  7. TESTFILE="$TESTDIR/framework"
  8. cp "$TESTFILE" aptarchive/foo
  9. APTARCHIVE="$(readlink -f ./aptarchive)"
  10. getcodenamefromsuite() { echo "jessie"; }
  11. buildsimplenativepackage 'foo' 'all' '1.0' 'stable'
  12. setupaptarchive --no-update
  13. ln -s "${APTARCHIVE}/dists/stable" "${APTARCHIVE}/dists/jessie"
  14. for FILE in rootdir/etc/apt/sources.list.d/*-stable-* ; do
  15. sed 's#stable#jessie#g' $FILE > $(echo "$FILE" | sed 's#stable#jessie#g')
  16. done
  17. # install a slowed down file: otherwise its to fast to reproduce combining
  18. NEWMETHODS="$(readlink -f rootdir)/usr/lib/apt/methods"
  19. OLDMETHODS="$(readlink -f rootdir/usr/lib/apt/methods)"
  20. rm "$NEWMETHODS"
  21. mkdir "$NEWMETHODS"
  22. backupIFS="$IFS"
  23. IFS="$(printf "\n\b")"
  24. for METH in $(find "$OLDMETHODS" ! -type d); do
  25. ln -s "$OLDMETHODS/$(basename "$METH")" "$NEWMETHODS"
  26. done
  27. IFS="$backupIFS"
  28. rm "${NEWMETHODS}/file" "${NEWMETHODS}/http"
  29. cat >"${NEWMETHODS}/file" <<EOF
  30. #!/bin/sh
  31. while read line; do
  32. echo "\$line"
  33. if [ -z "\$line" ]; then
  34. sleep 0.5
  35. fi
  36. done | '$OLDMETHODS/'"\${0##*/}"
  37. EOF
  38. chmod +x "${NEWMETHODS}/file"
  39. ln -s "${NEWMETHODS}/file" "${NEWMETHODS}/http"
  40. tworepos() {
  41. msgtest "Downloading the same repository twice over $1" "$3"
  42. testsuccess --nomsg aptget update -o Debug::pkgAcquire::Worker=1
  43. cp rootdir/tmp/testsuccess.output download.log
  44. #cat download.log
  45. aptget indextargets --format '$(FILENAME)' --no-release-info | sort > file.lst
  46. testequal "$(find "$(readlink -f ./rootdir/var/lib/apt/lists)" -name '*_dists_*' \( ! -name '*InRelease' \) -type f | sort)" cat file.lst
  47. testsuccess aptcache policy
  48. testequal "foo:
  49. Installed: (none)
  50. Candidate: 1.0
  51. Version table:
  52. 1.0 500
  53. 500 $1:$2 jessie/main all Packages
  54. 500 $1:$2 stable/main all Packages" aptcache policy foo
  55. testfailure aptcache show foo/unstable
  56. testsuccess aptcache show foo/stable
  57. testsuccess aptcache show foo/jessie
  58. }
  59. tworepos 'file' "$APTARCHIVE" 'no partial'
  60. testequal '14' grep -c '200%20URI%20Start' ./download.log
  61. testequal '14' grep -c '201%20URI%20Done' ./download.log
  62. testequal '8' grep -c '^ @ Queue: Action combined' ./download.log
  63. tworepos 'file' "$APTARCHIVE" 'hit'
  64. testequal '6' grep -c '200%20URI%20Start' ./download.log
  65. testequal '6' grep -c '201%20URI%20Done' ./download.log
  66. testequal '0' grep -c '^ @ Queue: Action combined' ./download.log
  67. rm -rf rootdir/var/lib/apt/lists
  68. changetowebserver
  69. tworepos 'http' "//localhost:${APTHTTPPORT}" 'no partial'
  70. testequal '12' grep -c '200%20URI%20Start' ./download.log
  71. testequal '12' grep -c '201%20URI%20Done' ./download.log
  72. testequal '8' grep -c '^ @ Queue: Action combined' ./download.log
  73. tworepos 'http' "//localhost:${APTHTTPPORT}" 'hit'
  74. testequal '2' grep -c '200%20URI%20Start' ./download.log
  75. testequal '4' grep -c '201%20URI%20Done' ./download.log
  76. testequal '0' grep -c '^ @ Queue: Action combined' ./download.log
  77. rm -rf rootdir/var/lib/apt/lists