You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

writer.h 5.1 KiB

support arch:all data e.g. in separate Packages file Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
5 years ago
15 years ago
support arch:all data e.g. in separate Packages file Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
5 years ago
support arch:all data e.g. in separate Packages file Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
5 years ago
support arch:all data e.g. in separate Packages file Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
5 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208
  1. // -*- mode: cpp; mode: fold -*-
  2. // Description /*{{{*/
  3. /* ######################################################################
  4. Writer
  5. The file writer classes. These write various types of output, sources,
  6. packages and contents.
  7. ##################################################################### */
  8. /*}}}*/
  9. #ifndef WRITER_H
  10. #define WRITER_H
  11. #include <apt-pkg/hashes.h>
  12. #include <iostream>
  13. #include <map>
  14. #include <set>
  15. #include <string>
  16. #include <utility>
  17. #include <vector>
  18. #include <stdio.h>
  19. #include <stdlib.h>
  20. #include <sys/types.h>
  21. #include "apt-ftparchive.h"
  22. #include "cachedb.h"
  23. #include "contents.h"
  24. #include "override.h"
  25. using std::string;
  26. using std::cout;
  27. using std::endl;
  28. using std::vector;
  29. using std::map;
  30. class FTWScanner
  31. {
  32. protected:
  33. vector<string> Patterns;
  34. vector<std::pair<string, bool>> FilesToProcess;
  35. string Arch;
  36. bool IncludeArchAll;
  37. const char *OriginalPath;
  38. bool ErrorPrinted;
  39. // Stuff for the delinker
  40. bool NoLinkAct;
  41. static FTWScanner *Owner;
  42. static int ScannerFTW(const char *File,const struct stat *sb,int Flag);
  43. static int ScannerFile(const char *const File, bool const ReadLink);
  44. static int ProcessFile(const char *const File, bool const ReadLink);
  45. bool Delink(string &FileName,const char *OriginalPath,
  46. unsigned long long &Bytes,unsigned long long const &FileSize);
  47. inline void NewLine(unsigned const &Priority)
  48. {
  49. if (ErrorPrinted == false && Quiet <= Priority)
  50. {
  51. c1out << endl;
  52. ErrorPrinted = true;
  53. }
  54. }
  55. public:
  56. FileFd *Output;
  57. bool OwnsOutput;
  58. unsigned int DoHashes;
  59. unsigned long DeLinkLimit;
  60. string InternalPrefix;
  61. virtual bool DoPackage(string FileName) = 0;
  62. bool RecursiveScan(string const &Dir);
  63. bool LoadFileList(string const &BaseDir,string const &File);
  64. void ClearPatterns() { Patterns.clear(); };
  65. void AddPattern(string const &Pattern) { Patterns.push_back(Pattern); };
  66. void AddPattern(char const *Pattern) { Patterns.push_back(Pattern); };
  67. void AddPatterns(std::vector<std::string> const &patterns) { Patterns.insert(Patterns.end(), patterns.begin(), patterns.end()); };
  68. bool SetExts(string const &Vals);
  69. FTWScanner(FileFd * const Output, string const &Arch = string(), bool const IncludeArchAll = true);
  70. virtual ~FTWScanner();
  71. };
  72. class MultiCompress;
  73. class TranslationWriter
  74. {
  75. MultiCompress *Comp;
  76. std::set<string> Included;
  77. FileFd *Output;
  78. public:
  79. bool DoPackage(string const &Pkg, string const &Desc, string const &MD5);
  80. TranslationWriter(string const &File, string const &TransCompress, mode_t const &Permissions);
  81. ~TranslationWriter();
  82. };
  83. class PackagesWriter : public FTWScanner
  84. {
  85. Override Over;
  86. CacheDB Db;
  87. public:
  88. // Some flags
  89. bool DoAlwaysStat;
  90. bool NoOverride;
  91. bool DoContents;
  92. bool LongDescription;
  93. // General options
  94. string PathPrefix;
  95. string DirStrip;
  96. struct CacheDB::Stats &Stats;
  97. TranslationWriter * const TransWriter;
  98. inline bool ReadOverride(string const &File) {return Over.ReadOverride(File);};
  99. inline bool ReadExtraOverride(string const &File)
  100. {return Over.ReadExtraOverride(File);};
  101. virtual bool DoPackage(string FileName) APT_OVERRIDE;
  102. PackagesWriter(FileFd * const Output, TranslationWriter * const TransWriter, string const &DB,
  103. string const &Overrides,
  104. string const &ExtOverrides = "",
  105. string const &Arch = "",
  106. bool const IncludeArchAll = true);
  107. virtual ~PackagesWriter();
  108. };
  109. class ContentsWriter : public FTWScanner
  110. {
  111. CacheDB Db;
  112. GenContents Gen;
  113. public:
  114. // General options
  115. struct CacheDB::Stats &Stats;
  116. string Prefix;
  117. bool DoPackage(string FileName,string Package);
  118. virtual bool DoPackage(string FileName) APT_OVERRIDE
  119. {return DoPackage(FileName,string());};
  120. bool ReadFromPkgs(string const &PkgFile,string const &PkgCompress);
  121. void Finish() {Gen.Print(*Output);};
  122. inline bool ReadyDB(string const &DB) {return Db.ReadyDB(DB);};
  123. ContentsWriter(FileFd * const Output, string const &DB, string const &Arch = string(),
  124. bool const IncludeArchAll = true);
  125. virtual ~ContentsWriter() {};
  126. };
  127. class SourcesWriter : public FTWScanner
  128. {
  129. CacheDB Db;
  130. Override BOver;
  131. Override SOver;
  132. char *Buffer;
  133. unsigned long long BufSize;
  134. public:
  135. bool NoOverride;
  136. bool DoAlwaysStat;
  137. // General options
  138. string PathPrefix;
  139. string DirStrip;
  140. struct CacheDB::Stats &Stats;
  141. virtual bool DoPackage(string FileName) APT_OVERRIDE;
  142. SourcesWriter(FileFd * const Output, string const &DB,string const &BOverrides,string const &SOverrides,
  143. string const &ExtOverrides=string());
  144. virtual ~SourcesWriter() {free(Buffer);};
  145. };
  146. class ReleaseWriter : public FTWScanner
  147. {
  148. public:
  149. ReleaseWriter(FileFd * const Output, string const &DB);
  150. virtual bool DoPackage(string FileName) APT_OVERRIDE;
  151. void Finish();
  152. // General options
  153. string PathPrefix;
  154. string DirStrip;
  155. struct CheckSum
  156. {
  157. HashStringList Hashes;
  158. // Limited by FileFd::Size()
  159. unsigned long long size;
  160. ~CheckSum() {};
  161. };
  162. protected:
  163. map<string,struct CheckSum> CheckSums;
  164. };
  165. #endif