Commit 24f1e4ab authored by Mark Hindley's avatar Mark Hindley

Publish updated Packaging Guide.

parent bfa68c0f
# Git structure for devuan packages
* [Introduction](#introduction)
* [Packages coming from debian](#packages-coming-from-debian)
* [debian packages using pristine-tar format for origin upstream](#debian-packages-using-pristine-tar-format-for-origin-upstream)
* [debian packages with sources as git branches](#debian-packages-with-sources-as-git-branches)
* [Packages coming from third party upstream](#packages-coming-directly-from-third-party-upstream)
* [packages using pristine-tar format for origin upstream](#packages-using-pristine-tar-format-for-origin-upstream)
* [packages with sources as git branches](#packages-with-sources-as-git-branches)
* [Devuan native packages](#devuan-native-packages)
* [Autobuild of devuan packages](#autobuild-of-devuan-packages)
## Introduction
Once granted access to our gitlab, you can start hosting sources for devuan packages. Packages can be maintained both in a gitlab group for collaborative maintainence, or in your user namespace if you prefer, at least initially, and then moved to a group only when the package is ready to be pushed in our autobuild infrastructure and then in the official devuan repository once accepted for inclusion.
I assume you already know how to mark the package you are working on as maintained on the issues of the [Devuan Maintainers Project](/devuan/devuan-maintainers/)
The git repository for devuan packages follows a standardized schema that depends on where the original upstream source comes from and on the tools you prefer to use.
In any case, there are some conventions about branches and tags names to be used on devuan git repos:
* branches:
* **master**: the main developement branch. It should be used for package developement efforts, and it's not assured it will compile cleanly as it is a work in progress. master is synced with the "next unstable"
* **suites/experimental**: complete source tree for package [autobuild](#autobuild-of-devuan-packages) in experimental suite
* **suites/unstable**: complete source tree for package [autobuild](#autobuild-of-devuan-packages) in unstable suite
* **suites/[codename]**: complete source tree for package [autobuild](#autobuild-of-devuan-packages) in [codename], both for stable and testing
* **suites/[codename]-updates**: complete source tree for package [autobuild](#autobuild-of-devuan-packages) in [codename] after [codename] is released as stable and there are important updates
* **suites/[codename]-proposed-updates**: complete source tree for package [autobuild](#autobuild-of-devuan-packages) in [codename] after is released as stable and there are important updates not yet accepted
* **suites/[codename]-backports**: complete source tree for backports to [codename] after is released as stable
* **pristine-tar**: only for pristine-tar diff files is used
* **upstream/[version]**: complete source tree of original source package of [version] from upstream
* **debian/[debian_version]**: complete source tree of original source package of [debian_version] from debian
* tags
* **upstream/[version]**: tagged upstream version
Not all branches/tags are needed for every package, it depends, for example, if you use pristine-tar for upstream source tarball re-generation, or if the package source is coming from debian or it's native.
The mandatory branches are master and suites/*, the others will vary package from package.
Any other branch can be used for developement purpose as you prefer.
## Packages coming from debian
As we claim to be debian 1:1 compatible at least for jessie (devuan 1.0 - debian 8), we will try to maintain 1:1 version match with debian packages.
In future versions of devuan maybe 1:1 compatibility will be broken, but it's better to try to maintain it as long as we can to allow for easy migration of debian users to devuan.
For those reasons, many of our packages that are also in debian and that can be easily derived from it without major modifications will be derived directly from debian sources instead of upstream sources.
It's up to the devuan maintainer to monitor debian package versions and to maintain our packages in sync when they are updated on debian, a must for devuan/debian jessie, optional but strongly suggested for testing and unstable too.
All debian packages will have in common, indipendently from the method used to import sources, those branches:
* **master** the main developement branch, where the dirty work will be done, with the version of package equivalent to the most recent one present in other branches ( experimental if exists, or unstable )
* **suites/experimental** optional but suggested for package testing before to push it in unstable
* **suites/unstable** unstable package in devuan
* **suites/[suitename]** unstable package in [suitename]
* **suites/[suitename]-[updates|proposed-updates|backports]** optional only if they are needed
* **debian/[version]** at least 1 branch for every suite of debian ( stable/testing/unstable ) of equivalent versions
The first step for push debian package sources in Devuan repository is to get source packages from debian.
At the moment i'm writing, for Devuan there is no equivalent of "testing" in debian, as jessie in debian is testing, so as a starting point we will update jessie packages to both our "stable" aka jessie and to our "testing" aka ascii.
### debian packages using pristine-tar format for origin upstream
work in progress...
### debian packages with sources as git branches
work in progress...
## Packages coming directly from third party upstream
Some packages in devuan are not in debian or there is a need to re-package them from scratch cause debian version is too patched and/or badly patched for devuan inclusion without major modifications.
Those packages will have origin upstream sources coming directly from original upstream instead of debian sources. In case the same package is present in debian, it's suggested to maintain the same package name as debian has, and also it is strongly suggested, but not mandatory, to try to have the same version in devuan that debian has.
### packages using pristine-tar format for origin upstream
work in progress...
### packages with sources as git branches
work in progress...
## Devuan native packages
There are packages born in devuan for devuan, or born in devuan for wide usage. Those packages don't have an external upstream as their source origins are in the devuan git repositories. An example of those packages are loginkit, or devuan-baseconf. For those packages upstream branching isn't needed, nor is pristine-tar support, and they usually just use upstream tagging to create/re-create source tarballs.
work in progress...
## Autobuild of Devuan packages
work in progress...
[Jenkins autobuild](JenkinsAutobuild)
# Jenkins Autobuild infrastructure in Devuan
Autobuild in devuan works as easy as to open an issue.
Autobuild in Devuan works as easy as to open an issue.
There is a little python bot, devuan-releasebot, that every 5 mins in a cron list all issues assigned to him and then, parsing labels and checking permissions, start a jenkins job to build the package and to push it in the repository.
There is a little python bot, [devuan-releasebot](https://git.devuan.org/devuan-infrastructure/devuan-releasebot), that every 5 mins in a cron list all issues assigned to him and then, parsing labels and checking permissions, start a jenkins job to build the package and to push it in the repository.
## Requisites to start a build
Before to start a build with the autobuilder, you need to be sure to match some pre-requisites:
* having a buildable package source in devuan gitlab, with at least a branch names suites/<something> where <something> is the name of the suite to build ( at the moment i write this doc, jessie, ascii, unstable and experimental are accepted suites name).
* having a buildable package source in devuan gitlab, with at least a branch names suites/<something> where <something> is the name of the suite to build (at the moment i write this doc, jessie, ascii, beowulf, unstable and experimental are accepted suites name).
* having at least master or owner access to the repository on gitlab for the package to be built
* the package should be already configured in jenkins ( ask the release team for that )
* the package should be already configured in jenkins (ask the release team for that)
* having labels for the suites name in the gitlab project issues
* add the "autobuild" member to the project or group, "guest" access is enough, so you can assign an issue to him.
* add the "releasebot" member to the project or group, "guest" access is enough, so you can assign an issue to him.
## How to start a build
If you have all requisites to start a build, you can start it as easy as to open an issue with title "build" on the project you want to build, assign the issue to "autobuild" gitlab member, and then add labels with the name of the suite you want to build to.
If you have all requisites to start a build, you can start it as easy as to open an issue with title "build" on the project you want to build, assign the issue to "releasebot" gitlab member, and then add labels with the name of the suite you want to build to.
At the first run of the releasebot ( every 5 minutes ) it will comment the issue with infos regarding if the build is queued or if there is any error in the build request, and then it will close the issue.
At the first run of the releasebot (every 5 minutes) it will comment the issue with infos regarding if the build is queued or if there is any error in the build request, and then it will close the issue.
You can then look at jenkins ( https://ci.devuan.org, no login needed ) to see if your package is being built and pushed i the repository without errors.
You can then look at [jenkins](https://jenkins.devuan.dev) (no login needed) to see if your package is being built and pushed to the repository without errors.
## which packages are already configured i jenkins?
Jenkins also sends notifications to the #devuan-ci channel on irc.freenode.net.
You can see a list of packages already configured in [jenkins source jobs](https://ci.devuan.org/view/Only_sources/)
## which packages are already configured in jenkins?
Enjoy!
\ No newline at end of file
You can see a list of packages already configured in [jenkins source jobs](https://jenkins.devuan.dev/view/Only_sources/)
Enjoy!
# Packaging Guide for Devuan
## Introduction
This file documents Devuan's packaging infrastructure, guidelines and typical
workflow.
Devuan is a derivative of Debian and much of Debian's documentation also
provides useful and relevant resources. In particular, the following are good
starting points:-
- https://wiki.debian.org/Packaging
- https://www.debian.org/doc/manuals/maint-guide/
- https://wiki.debian.org/PackagingWithGit
Devuan's source code is maintained in [git](https://git.devuan.org), builds are
performed using [jenkins](https://jenkins.devuan.dev), package repositories are
maintained with [dak](https://git.devuan.org/devuan-infrastructure/dak) and
those repositories are merged with Debian's to produce the distribution by
[amprolla](https://git.devuan.org/devuan-infrastructure/amprolla3).
The majority of Devuan's packages are used directly from Debian with no
recompilation. Only packages that require modification are forked and rebuilt.
Packaging help can be found by asking on the #devuan-dev channel on
irc.freenode.net.
## Infrastructure
Devuan's source code is managed in a git repository (GitLab as of 2020, but with
planned migration to gitea). The server is accessed at https://git.devuan.org.
Users can create their own accounts to work on packages and create merge
requests. Official Devuan packages reside in the [devuan-packages
group](https://git.devuan.org/devuan-packages) which provides the source for
Jenkins when builds are triggered.
If a user has sufficient permissions, builds can be triggered by opening a new
issue against the relevant devuan-packages/${SOURCE_PACKAGE} with the title
'build' and assigning it to releasebot. Further details can be found in
[JenkinsAutobuild](./JenkinsAutobuild.md).
## Package Categories
There are four categories of packages in Devuan:-
1. Devuan only, not present in Debian.
2. Forked from Debian and modified.
3. Present in Debian but Devuan is the packaging upstream (currently only
elogind and eudev).
4. Used directly from Debian unchanged.
Packages in the last category will usually not be present in Devuan's git or
jenkins. However, packages can change category over time and so remnants of
previous incarnations may remain.
## Git
1. Clone a local copy of the relevant repository from
[git.devuan.org/devuan-packages](https://git.devuan.org/devuan-packages].
2. Add Debian's SCM as another remote. The URL can be found through [Debian
Package Tracker](https://tracker.debian.org). This is optional, but makes
merging much easier and is encouraged.
3. Create a new branch from your chosen starting point. This is usually the
version in Debian unstable. rmadison(1) from the devscripts package is a
convenient way of determining current Debian versions.
Usually packages are initially built for unstable or experimental so the
branch should be named suites/unstable or suites/experimental accordingly.
Sometimes packages are built to update suites that have already been
released. In that case the branch will be suites/${CODENAME} where CODENAME
is the release suite (jessie, ascii, beowulf, chimaera, daedelus...). The
release codenames may optionally have -proposed appended.
Once the branch is created and checked-out, add ```debian-branch =
suites/unstable``` (or whatever codename is appropriate) to the file
debian/gbp.conf. Create the file if necessary.
## Build Environment
The jenkins build pipeline uses git-buildpackage, pbuilder and cowbuilder. This
ensures more flexible and reproducible builds in a cleaner environment than
other build approaches. To avoid unexpected build failures in jenkins, you
should replicate this approach when testing. This approach also means that you
can build for suites, architectures and configurations that are incompatible
with your host sytem without conflicts.
To setup a local build environment
1. Install the required packages.
```apt install git-buildpackage pbuilder cowbuilder```
Some packages may require other debhelper modules to be installed (e.g.
dh-golang). If these are required builds will fail and the build log will
explicitly detail what is required.
2. Create a cowbuilder chroot.
There are many ways to do this. One of the simplest is to run
```gbp buildpackage --git-pbuilder --git-dist=ceres create --mirror http://pkgmaster.devuan.org/merged ```
from within the git repository.
## Packaging Conventions
### debian/control
- Update Maintainers and Uploaders appropriately.
- Ensure that the Vcs-* fields contain the official Devuan URL
(https://git.devuan.org/devuan-packages/${SOURCE_PACKAGE}).
- Add ```Origin: Devuan``` to the source stanza.
- Remove any ```Built-Using:``` field.
As the Devuan archive (managed by DAK) is only partial, Built-Using
dependencies are not necessarily satisfied.
### debian/gbp.conf
This file is useful for setting defaults for git-buildpackage. See gbp.conf(5).
Besides ```debian-branch``` already mentioned above, particularly useful settings include:-
- upstream-tag=upstream/v%(version)s
- debian-tag = devuan/%(version)s
- pristine-tar=False
Note that Devuan avoids the use of pristine-tar wherever possible as it fills
the repository with binary blobs. If Debian's package repository has a
pristine-tar branch you will need to prevent git-buildpackage from using it.
## Packaging Workflow
1. Review bug reports at https://bugs.devuan.org and also any issues that might
have been opened at https://git.devuan.org/devuan-packages/${SOURCE_PACKAGE}
and plan to address/incorporate fixes as required.
For any bugreports that you close, add (Closes: #<bugnumber>) at the end of
the debian/changelog line. This will automatically close the bug when the
package is built and enters the archive.
2. Make the changes required to the source code.
When doing this, remember that for non-native packages any changes outside
the debian/ directory are required to be implemented as quilt patches in
debian/patches/. This restriction does not apply to native packages.
3. Update debian/changelog.
The general convention for package numbering is to use the Debian version
with +devuanN appended (where N is an incremental integer).
4. Local build.
```gbp buildpackage --git-pbuilder --git-dist=ceres```
5. Lintian
Install devuan-lintian-profile.
```lintian *.changes```
6. Local install.
7. Repeat from 2 until you are finished.
8. Tag the release.
If you are working on a new package use the format
devuan/${VERSION}. However, if there is already a convention in use that
particular package you might prefer to continue with the existing one.
9. Publish.
Push your branch and the tag to https://git.devuan.org. If you are working on a
non-native package you should also push the upstream tag.
10. Merge.
Create a merge request to https://git.devuan.org/devuan-packages or contact
a developer with access to review and merge your changes.
11. Build for release.
Details of triggering the Jenkins Build Pipeline can be found in
[JenkinsAutobuild](./JenkinsAutobuild.md).
## Tips
### Updating Build Chroots
Cowbuilder chroots will need updating periodically, more often for the testing
and unstable distributions where packages change frequently. The indication that
this is required is usually build failures with 404 errors for build
dependencies, for example:
```
E: Failed to fetch http://mirrors.dotsrc.org/debian/pool/main/d/doxygen/doxygen_1.8.16-2_amd64.deb:
404 Not Found [IP: 130.225.254.116 80]
```
There are many ways of updating the chroots. The easiest is by running
```gbp buildpackage --git-pbuilder --git-dist=<distribution> update```
from the root of the git repository.
You may also find a pbuilder hook useful.
1. Configure a hook dir in ~/.pbuilderrc. For example:
```HOOKDIR=$HOME/.pbuilder/hooks```
2. Add a executable script in that directory to update the chroot package lists at the
beginning of the build:
The file should be ~/.pbuilder/hooks/D05apt-update
containing
```
#!/bin/sh
apt update
```
For further information see pbuilder(8).
# Devuan Maintainers Repository
# Contributing to Devuan
This project was created to circumvent a limitation of Gitlab groups
that can't receive issues directly, and thus address the need for
volunteers to step up and take responsibility for the maintenance of a
Devuan package.
There is no formal procedure to become a Devuan contributor. A good starting
place is to join the conversations on the #devuan-dev channel at
irc.freenode.net and subscribe to the [Devuan Developers
list](https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/devuan-dev). The
community will be interested to hear your ideas, experience and the ways in
which you wish to contribute. There is also a weekly meeting (currently Thursday
at 2030UTC) which is open to everybody.
## Becoming a Package Maintainer
# Becoming a Package Maintainer
Currently the packages are listed in the [devuan-packages](/wikis/../../../../groups/devuan-packages) group. If you're interested in maintaining a package, simply [open an issue](../../issues/new) with the package name. Everything is manual, so bear it in mind if you experience delays. Once your application has been accepted, you will be granted master privilege for the package.
Sources for current (and historical) packages are kept in the
[devuan-packages](https://git.devuan.org/devuan-packages) group of our [git
repository](https://git.devuan.org). If you are looking to adopt a package,
those that are maintained by
[devuan-dev@lists.dyne.org](https://bugs.devuan.org/cgi/pkgreport.cgi?maint=devuan-dev%40lists.dyne.org)
generally have no specific maintainer and could be taken over.
Also, take care to read:
* Guide for setting up [git repos in devuan](/wikis/../../wikis/GitPackages)
* Guide for [jenkins autobuild infrastructure in devuan](/wikis/../../wikis/JenkinsAutobuild)
# [Adopt a package](/wikis/../../wikis/home)
You can use the package's issue to help the community keeping track of the state of maintenance of this package. Especially, you can use labels to ease transitioning maintainers, etc. Here are a few shortcuts:
Prospective Package Maintainers will also find the following useful:
+ [Maintained Packages](/issues/../../issues?label_name=maintained)
+ [Orphan Packages](/issues/../../issues?label_name=orphan)
+ [Maintainer wanted](/issues/../../issues?label_name=maintainer+wanted)
If a package is missing, you may [create an issue](/issues/../../issues/new) for it.
\ No newline at end of file
* [Devuan Packaging Guide](./PackagingGuide.md)
* [Jenkins Autobuild Infrastructure](./JenkinsAutobuild.md)
Devuan packages can be "adopted": **missing** and **claimed** packages are indicated below.
## Adopting a Package
A **missing** package is up for adoption: adopting a package means you're responsible for maintaining it up-to-date. You're also responsible for the relation with upstream.
Each package should have an issue open in its name in the [devuan-maintainers issue tracker](/devuan/devuan-maintainers/issues). The packages themselves belong to the [packages-base](/groups/packages-base) group.
1. Verify that the package was not already adopted, or is pending adoption. If an issue already exists, talk with the people there.
2. Claim a **missing** package: add or change the corresponding line in the table, and put your Gitlab username.
3. Please [open an issue named after the package](/devuan/devuan-maintainers/issues/new) to adopt it.
| Package Name | State | Maintainer |
|------------------------------------|------------|-------------------------|
| sample-missing-package | missing | |
| sample-claimed-package | claimed | @hellekin |
Note: once you have a package project in [packages-base](/groups/packages-base), you can update the state to **maintained** and link the package name to its Gitlab page.
Related issues: devuan/devuan-project#10 and devuan/devuan-project#11.
* Guide for setting up [git repos in devuan](GitPackages)
* Guide for [jenkins autobuild infrastructure in devuan](JenkinsAutobuild)
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment