You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

300 lines
14 KiB

  1. # Acquire additional files in 'update' operations
  2. The download and verification of data from multiple sources in different
  3. compression formats, with partial downloads and patches is an involved
  4. process which is hard to implement correctly and securely.
  5. APT frontends share the code and binaries to make this happen in libapt
  6. with the Acquire system, supported by helpers shipped in the apt package
  7. itself and additional transports in individual packages like
  8. apt-transport-https.
  9. For its own operation libapt needs or can make use of Packages, Sources
  10. and Translation-* files, which it will acquire by default, but
  11. a repository might contain more data files (e.g. Contents) a frontend
  12. (e.g. apt-file) might want to use and would therefore need to be
  13. downloaded as well.
  14. This file describes the configuration scheme such a frontend can use to
  15. instruct the Acquire system to download those additional files.
  16. # The Configuration Stanza
  17. The Acquire system uses the same configuration settings to implement the
  18. files it downloads by default. These settings are the default, but if
  19. they would be written in a configuration file the configuration
  20. instructing the Acquire system to download the Packages files would look
  21. like this (see also apt.conf(5) manpage for configuration file syntax):
  22. Acquire::IndexTargets::deb::Packages {
  23. MetaKey "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages";
  24. ShortDescription "Packages";
  25. Description "$(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages";
  26. flatMetaKey "Packages";
  27. flatDescription "$(RELEASE) Packages";
  28. Optional "no";
  29. };
  30. All files which should be downloaded (nicknamed 'Targets') are mentioned
  31. below the Acquire::IndexTargets scope. 'deb' is here the type of the
  32. sources.list entry the file should be acquired for. The only other
  33. supported value is hence 'deb-src'. Beware: You can't specify multiple
  34. types here and you can't download the same (evaluated) MetaKey from
  35. multiple types!
  36. After the type you can pick any valid and unique string which preferable
  37. refers to the file it downloads (In the example we picked 'Packages').
  38. This string is used as identifier for the target class and accessible as
  39. 'Created-By' e.g. in the "apt-get indextargets" output as detailed
  40. below. It is also used to allow user to enable/disable targets per
  41. sources.list entry.
  42. All targets have three main properties you can define:
  43. * MetaKey: The identifier of the file to be downloaded as used in the
  44. Release file. It is also the relative location of the file from the
  45. Release file. You can neither download from a different server
  46. entirely (absolute URI) nor access directories above the Release file
  47. (e.g. "../../").
  48. * ShortDescription: Very short string intended to be displayed to the
  49. user e.g. while reporting progress. apt will e.g. use this string in
  50. the last line to indicate progress of e.g. the download of a specific
  51. item.
  52. * Description: A preferable human understandable and readable identifier
  53. of which file is acquired exactly. Mainly used for progress reporting
  54. and error messages. apt will e.g. use this string in the Get/Hit/Err
  55. progress lines.
  56. An identifier of the site accessed as seen in the sources.list (e.g.
  57. "" or "file:/path/to/a/repository") is
  58. automatically prefixed for this property.
  59. Additional optional properties:
  60. * DefaultEnabled: The default value is 'yes' which means that apt will
  61. try to acquire this target from all sources. If set to 'no' the user
  62. has to explicitly enable this target in the sources.list file with the
  63. Targets option(s) – or override this value in a config file.
  64. * Optional: The default value is 'yes' and should be kept at this value.
  65. If enabled the acquire system will skip the download if the file isn't
  66. mentioned in the Release file. Otherwise this is treated as a hard
  67. error and the update process fails. Note that failures while
  68. downloading (e.g. 404 or hash verification errors) are failures,
  69. regardless of this setting.
  70. * KeepCompressed: The default is the value of Acquire::GzipIndexes,
  71. which defaults to false. If true, the acquire system will keep the
  72. file compressed on disk rather than extract it. If your frontend can't
  73. deal with compressed files transparently you have to explicitly set
  74. this option to false to avoid problems with users setting the option
  75. globally. On the other hand, if you set it to true or don't set it you
  76. have to ensure your frontend can deal with all compressed fileformats
  77. supported by apt (libapt users can e.g. use FileFd).
  78. * flat{MetaKey,Description}: APT supports two types of repositories:
  79. dists-style repositories which are the default and by far the most
  80. common which are named after the fact that the files are in an
  81. elaborated directory structure. In contrast a flat-style repository
  82. lumps all files together in one directory. Support for these flat
  83. repositories exists mainly for legacy purposes only. It is therefore
  84. recommend to not set these values.
  85. The acquire system will automatically choose to download a compressed
  86. file if it is available and uncompress it for you, just as it will also
  87. use PDiff patching if provided by the repository and enabled by the
  88. user. You only have to ensure that the Release file contains the
  89. information about the compressed files/PDiffs to make this happen.
  90. *NO* properties have to be set to enable this!
  91. More properties exist, but these should *NOT* be set by frontends
  92. requesting files. They exist for internal and end-user usage only.
  93. Some of these are – which are documented here only to ensure that they
  94. aren't accidentally used by frontends:
  95. * PDiffs: controls if apt will try to use PDiffs for this target.
  96. Defaults to the value of Acquire::PDiffs which is true by default.
  97. Can be overridden per-source by the sources.list option of the same
  98. name. See the documentation for both of these for details.
  99. * By-Hash: controls if apt will try to use an URI constructed from
  100. a hashsum of the file to download. See the documentation for config
  101. option Acquire::By-Hash and sources.list option By-Hash for details.
  102. * CompressionTypes: The default value is a space separated list of
  103. compression types supported by apt (see Acquire::CompressionTypes).
  104. You can set this option to prevent apt from downloading a compression
  105. type a frontend can't open transparently. This should always be
  106. a temporary workaround through and a bug should be reported against
  107. the frontend in question.
  108. # More examples
  109. The stanzas for Translation-* files as well as for Sources files would
  110. look like this:
  111. Acquire::IndexTargets {
  112. deb::Translations {
  113. MetaKey "$(COMPONENT)/i18n/Translation-$(LANGUAGE)";
  114. ShortDescription "Translation-$(LANGUAGE)";
  115. Description "$(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)";
  116. flatMetaKey "$(LANGUAGE)";
  117. flatDescription "$(RELEASE) Translation-$(LANGUAGE)";
  118. };
  119. deb-src::Sources {
  120. MetaKey "$(COMPONENT)/source/Sources";
  121. ShortDescription "Sources";
  122. Description "$(RELEASE)/$(COMPONENT) Sources";
  123. flatMetaKey "Sources";
  124. flatDescription "$(RELEASE) Sources";
  125. Optional "no";
  126. };
  127. };
  128. # Substitution variables
  129. As seen in the examples, properties can contain placeholders filled in
  130. by the acquire system. The following variables are known; note that
  131. unknown variables have no default value nor are they touched: They are
  132. printed as-is.
  133. * $(RELEASE): This is usually an archive- or codename, e.g. "stable" or
  134. "stretch". Note that flat-style repositories do not have an archive-
  135. or codename per-se, so the value might very well be just "/" or so.
  136. * $(COMPONENT): as given in the sources.list, e.g. "main", "non-free" or
  137. "universe". Note that flat-style repositories again do not really
  138. have a meaningful value here.
  139. * $(LANGUAGE): Values are all entries (expect "none") of configuration
  140. option Acquire::Languages, e.g. "en", "de" or "de_AT".
  141. * $(ARCHITECTURE): Values are all entries of configuration option
  142. APT::Architectures (potentially modified by sources.list options),
  143. e.g. "amd64", "i386" or "armel" for the 'deb' type. In type 'deb-src'
  144. this variable has the value "source".
  145. * $(NATIVE_ARCHITECTURE): The architecture apt treats as the native
  146. architecture for this system configured as APT::Architecture
  147. defaulting to the architecture apt itself was built for.
  148. Note that while more variables might exist in the implementation, these
  149. are to be considered undefined and their usage strongly discouraged. If
  150. you have a need for other variables contact us.
  151. # Accessing files
  152. Do NOT hardcode specific file locations, names or compression types in
  153. your application! You will notice that the configuration options give
  154. you no choice over where the downloaded files will be stored. This is by
  155. design so multiple applications can download and use the same file
  156. rather than each and every one of them potentially downloads and uses
  157. its own copy somewhere on disk.
  158. "apt-get indextargets" can be used to get the location as well as other
  159. information about all files downloaded (aka: you will see Packages,
  160. Sources and Translation-* files here as well). Provide a line of the
  161. default output format as parameter to filter out all entries which do
  162. not have such a line. With --format, you can further more define your
  163. own output style. The variables are what you see in the output, just all
  164. uppercase and wrapped in $(), as in the configuration file.
  165. To get all the filenames of all Translation-en files you can e.g. call:
  166. apt-get indextargets --format '$(FILENAME)' "Created-By: Translations" "Language: en"
  167. The line-based filtering and the formating is rather crude and feature-
  168. less by design: The default format is Debians standard format deb822 (in
  169. particular: Field names are case-insensitive and the order of fields in
  170. the stanza is undefined), so instead of apt reimplementing powerful
  171. filters and formating for this command, it is recommend to use piping
  172. and dedicated tools like 'grep-dctrl' if you need more than the basics
  173. provided.
  174. Accessing this information via libapt is done by reading the
  175. sources.lists (pkgSourceList), iterating over the metaIndex objects this
  176. creates and calling GetIndexTargets() on them. See the source code of
  177. "apt-get indextargets" for a complete example.
  178. Note that by default targets are not listed if they weren't downloaded.
  179. If you want to see all targets, you can use the --no-release-info, which
  180. also removes the Codename, Suite, Version, Origin, Label and Trusted
  181. fields from the output as these also display data which needs to be
  182. downloaded first and could hence be inaccurate [on the pro-side: This
  183. mode is faster as it doesn't require a valid binary cache to operate].
  184. The most notable difference perhaps is in the Filename field through: By
  185. default it indicates an existing file, potentially compressed (Hint:
  186. libapt users can use FileFd to open compressed files transparently). In
  187. the --no-release-info mode the indicated file doesn't need to exist and
  188. it will always refer to an uncompressed file, even if the index would be
  189. (or is) stored compressed.
  190. Remarks on fields only available in (default) --release-info mode:
  191. * Trusted: Denotes with a 'yes' or 'no' if the data in this file is
  192. authenticated by a trust chain rooted in a trusted gpg key. You should
  193. be careful with untrusted data and warn the user if you use it.
  194. * Codename, Suite, Version, Origin and Label are fields from the Release
  195. file, are only present if they are present in the Release file and
  196. contain the same data.
  197. Remarks on other available fields:
  198. * MetaKey, ShortDesc, Description, Site, Release: as defined
  199. by the configuration and described further above.
  200. * Created-By: configuration entity responsible for this target
  201. * Target-Of: type of the sources.list entry
  202. * URI, Repo-URI: avoid using. Contains potentially username/password.
  203. Prefer 'Site', especially for display.
  204. * Optional, DefaultEnabled, KeepCompressed: Decode the options of the
  205. same name from the configuration.
  206. * Language, Architecture, Component: as defined further above, but with
  207. the catch that they might be missing if they don't effect the target
  208. (aka: They weren't used while evaluating the MetaKey template).
  209. Again, additional fields might be visible in certain implementations,
  210. but you should avoid using them and instead talk to us about a portable
  211. implementation.
  212. # Multiple applications requiring the same files
  213. It is highly encouraged that applications talk to each other and to us
  214. about which files they require. It is usually best to have a common
  215. package ship the configuration needed to get the files, but specific
  216. needs might require specific solutions. Again: talk to us.
  217. Bad things will happen if multiple frontends request the same file(s)
  218. via different targets, which is another reason why coordination is very
  219. important!
  220. # Acquiring files not mentioned in the Release file
  221. You can't. This is by design as these files couldn't be verified to not
  222. be modified in transit, corrupted by the download process or simple if
  223. they are present at all on the server, which would require apt to probe
  224. for them. APT did this in the past for legacy reasons, we do not intend
  225. to go back to these dark times.
  226. This is also why you can't request files from a different server. It
  227. would have the additional problem that this server might not even be
  228. accessible (e.g. proxy settings) or that local sources (file:/, cdrom:/)
  229. start requesting online files…
  230. In other words: We would be opening Pandora's box.
  231. # Acquiring files to a specific location on disk
  232. You can't by design to avoid multiple frontends requesting the same file
  233. to be downloaded to multiple different places on (different) disks
  234. (among other reasons). See the next point for a solution if you really
  235. have to force a specific location by creating symlinks.
  236. # Post processing the acquired files
  237. You can't modify the files apt has downloaded as apt keeps state with
  238. e.g. the modification times of the files and advanced features like
  239. PDiffs break.
  240. You can however install an APT::Update::Post-Invoke{-Success,} hook
  241. script and use them to copy (modified) files to a different location.
  242. Use 'apt-get indextargets' (or similar) to get the filenames – do not
  243. look into /var/lib/apt/lists directly!
  244. Please avoid time consuming calculations in the scripts and instead just
  245. trigger a background task as there is little to no feedback for the user
  246. while hook scripts run.