Browse Source

Resynced to upstream

tags/devuan/0.15.2+devuan4
nextime 5 years ago
parent
commit
1648670555
20 changed files with 661 additions and 204 deletions
  1. +22
    -0
      debian/NEWS
  2. +12
    -6
      debian/changelog
  3. +46
    -89
      debian/control
  4. +8
    -3
      debian/copyright
  5. +0
    -1
      debian/jenkins-debian-glue-buildenv-lintian.install
  6. +0
    -2
      debian/jenkins-debian-glue-buildenv-piuparts.install
  7. +0
    -5
      debian/jenkins-debian-glue-buildenv-taptools.install
  8. +9
    -0
      debian/jenkins-debian-glue.install
  9. +32
    -13
      examples/debian_glue
  10. +34
    -16
      pbuilder-hookdir/B20autopkgtest
  11. +1
    -0
      pbuilder-hookdir/C10shell
  12. +1
    -5
      puppet/jenkins_debian_glue.pp
  13. +223
    -22
      scripts/build-and-provide-package
  14. +54
    -26
      scripts/generate-git-snapshot
  15. +29
    -3
      scripts/generate-reprepro-codename
  16. +9
    -2
      scripts/generate-svn-snapshot
  17. +26
    -2
      scripts/remove-reprepro-codename
  18. +23
    -1
      scripts/repository_checker
  19. +1
    -1
      tap/pep8_tap
  20. +131
    -7
      tap/tap_tool_dispatcher

+ 22
- 0
debian/NEWS View File

@@ -1,3 +1,25 @@
jenkins-debian-glue (0.14.0) unstable; urgency=medium

To reduce the amount of Debian binary packages we decided to
minimize the number of packages we ship via jenkins-debian-glue.
This means that the packages named
jenkins-debian-glue-buildenv-git,
jenkins-debian-glue-buildenv-svn,
jenkins-debian-glue-buildenv-slave,
jenkins-debian-glue-buildenv-lintian,
jenkins-debian-glue-buildenv-taptools and
jenkins-debian-glue-buildenv-piuparts are deprecated now.
To get all the scripts just the jenkins-debian-glue package is
needed. To install all the Debian packages that might be needed
to be useful with all the scripts the
jenkins-debian-glue-buildenv package can be used. If you don't
want to install all the packages which are listed in
jenkins-debian-glue-buildenv's Depends/Recommends you're
encouraged to create your own metapackage or use the
configuration management system of your choice.

-- Michael Prokop <mika@grml.org> Sat, 22 Aug 2015 12:26:59 +0200

jenkins-debian-glue (0.6.0) unstable; urgency=low

The timestamp format used in automatically generated version


+ 12
- 6
debian/changelog View File

@@ -1,3 +1,9 @@
jenkins-debian-glue (0.15.2+devuan1) unstable; urgency=medium

* Import latest changes from upstream

-- Franco (nextime) Lanza <nextime@devuan.org> Tue, 01 Dec 2015 07:19:10 +0100

jenkins-debian-glue (0.14.1+devuan6) unstable; urgency=low

* copy updated B20autopkgtest from pbuilder/examples
@@ -112,7 +118,7 @@ jenkins-debian-glue (0.11.0) unstable; urgency=medium
[ Mathieu Parent ]
* [2934b08] Ensure correct version comparison

-- Michael Prokop <prokop@grml-solutions.com> Mon, 20 Oct 2014 10:13:18 +0200
-- Michael Prokop <mika@debian.org> Mon, 20 Oct 2014 10:13:18 +0200

jenkins-debian-glue (0.10.0) unstable; urgency=medium

@@ -142,7 +148,7 @@ jenkins-debian-glue (0.10.0) unstable; urgency=medium
* [f9a139d] tap_tool_dispatcher: run all tools in parallel
* [5d33d21] pep8_tap: don't re-enable default-disabled checkers

-- Michael Prokop <prokop@grml-solutions.com> Wed, 04 Jun 2014 16:56:34 +0200
-- Michael Prokop <mika@debian.org> Wed, 04 Jun 2014 16:56:34 +0200

jenkins-debian-glue (0.9.0) unstable; urgency=medium

@@ -184,7 +190,7 @@ jenkins-debian-glue (0.9.0) unstable; urgency=medium
[ Andrey Elmanov ]
* [0a4191c] Allow adding extra repositories for resolving dependencies

-- Michael Prokop <prokop@grml-solutions.com> Fri, 16 May 2014 12:56:48 +0200
-- Michael Prokop <mika@debian.org> Fri, 16 May 2014 12:56:48 +0200

jenkins-debian-glue (0.8.1) unstable; urgency=medium

@@ -192,7 +198,7 @@ jenkins-debian-glue (0.8.1) unstable; urgency=medium
that shouldn't include sources" [Closes: #88] Thanks to Christoph Berg
<christoph.berg@credativ.de> for feedback

-- Michael Prokop <prokop@grml-solutions.com> Fri, 14 Feb 2014 10:58:19 +0100
-- Michael Prokop <mika@debian.org> Fri, 14 Feb 2014 10:58:19 +0100

jenkins-debian-glue (0.8.0) unstable; urgency=medium

@@ -238,7 +244,7 @@ jenkins-debian-glue (0.8.0) unstable; urgency=medium
* [b1e42c1] Make it possible to verify our own reprepro repositories by
providing a pubkey

-- Michael Prokop <prokop@grml-solutions.com> Mon, 03 Feb 2014 14:51:01 +0100
-- Michael Prokop <mika@debian.org> Mon, 03 Feb 2014 14:51:01 +0100

jenkins-debian-glue (0.7.1) unstable; urgency=low

@@ -246,7 +252,7 @@ jenkins-debian-glue (0.7.1) unstable; urgency=low
"reprepro | freight" to jenkins-debian-glue instead
* [7588e3e] Add jenkins-debian-glue-buildenv-git to Recommends

-- Michael Prokop <prokop@grml-solutions.com> Thu, 24 Oct 2013 13:42:46 +0200
-- Michael Prokop <mika@debian.org> Thu, 24 Oct 2013 13:42:46 +0200

jenkins-debian-glue (0.7.0) unstable; urgency=low



+ 46
- 89
debian/control View File

@@ -11,13 +11,15 @@ Vcs-Browser: https://git.devuan.org/devuan-infrastructure/jenkins-devuan-glue/tr

Package: jenkins-debian-glue
Architecture: all
Depends: fakeroot,
reprepro | freight,
sudo | sudo-ldap,
Depends: sudo | sudo-ldap,
${misc:Depends}
Recommends: curl,
gnupg,
jenkins-debian-glue-buildenv-git
Recommends: jenkins-debian-glue-buildenv
Conflicts: jenkins-debian-glue-buildenv-git,
jenkins-debian-glue-buildenv-lintian,
jenkins-debian-glue-buildenv-piuparts,
jenkins-debian-glue-buildenv-slave,
jenkins-debian-glue-buildenv-svn,
jenkins-debian-glue-buildenv-taptools
Description: glue scripts for building Debian packages inside Jenkins
This package provides scripts which should make building Debian
package inside Jenkins (a Continuous Integration suite) easier.
@@ -25,98 +27,53 @@ Description: glue scripts for building Debian packages inside Jenkins
It's meant to make Q/A builds of Debian packages inside Jenkins
as manageable and homogeneous as possible.
.
Please note that you might want to also install the
jenkins-debian-glue-buildenv-git or the jenkins-debian-glue-buildenv-svn
package, depending on your needs.
You need to install the according packaging tools, depending on
your VCS type and the features you want to use. To get all tools
you can just install the jenkins-debian-glue-buildenv package.

Package: jenkins-debian-glue-buildenv
Architecture: all
Depends: cowbuilder,
Depends: build-essential,
cowbuilder,
devscripts,
dpkg-dev,
eatmydata,
fakeroot,
file,
git-buildpackages,
gnupg,
jenkins-debian-glue,
quilt, qemu-user-static,
ccache, eatmydata
Description: Virtual package to be used on build slaves
This virtual package depends on the software packages required
for using jenkins-debian-glue as build environment.

Package: jenkins-debian-glue-buildenv-git
Architecture: all
Depends: build-essential,
git-buildpackage,
jenkins-debian-glue-buildenv,
pristine-tar,
${misc:Depends}
Description: virtual package for Git build environment of jenkins-debian-glue
This virtual package depends on the software packages required
for using jenkins-debian-glue as standalone build environment
with Git.

Package: jenkins-debian-glue-buildenv-svn
Architecture: all
Depends: build-essential,
jenkins-debian-glue-buildenv,
subversion-tools (<= 1.6.18dfsg-1) | svn2cl,
xsltproc,
${misc:Depends}
Description: virtual package for Subversion build environment of jenkins-debian-glue
This virtual package depends on the software packages required
for using jenkins-debian-glue as standalone build environment
with Subversion.

Package: jenkins-debian-glue-buildenv-slave
Architecture: all
Depends: build-essential,
jenkins-debian-glue-buildenv,
openjdk-7-jre-headless | sun-java6-jre | java-runtime-headless,
lintian,
parallel | moreutils,
piuparts,
pristine-tar,
quilt,
reprepro| freight,
rsync,
${misc:Depends}
Description: virtual package for basic build environment of jenkins-debian-glue
This virtual package depends on the software packages required
for using jenkins-debian-glue on a slave system inside Jenkins.

Package: jenkins-debian-glue-buildenv-lintian
Architecture: all
Depends: lintian,
ruby,
${misc:Depends}
Description: lintian integration in Jenkins
This package provides the lintian-junit-report script. The
script can be used for generating JUnit reports of lintian
checking Debian packages inside Jenkins jobs.

Package: jenkins-debian-glue-buildenv-taptools
Architecture: all
Depends: devscripts,
file,
libperl-critic-perl,
parallel,
ruby,
${misc:Depends}
Recommends: pep8 (>= 1.4.6~)
Description: taptools integration in Jenkins
This package provides scripts to check Shell, Perl and Python
source code for common errors, using checkbashism and perlcritic
to generate according TAP reports for usage inside Jenkins.
.
The provided tap_tool_dispatcher script can be used to automatically
detect according Shell scripts, Perl and Python scripts to execute
checkbashism_tap, perlcritic_tap and pep8_tap accordingly.
.
Please notice that pep8 isn't a hard dependency because it's not yet
available in the required version in Debian/wheezy.

Package: jenkins-debian-glue-buildenv-piuparts
Architecture: all
Depends: piuparts,
ruby,
subversion-tools (<= 1.6.18dfsg-1) | svn2cl,
ccache,
${misc:Depends}
Description: piuparts integration in Jenkins
This package provides the piuparts integration for testing
package installation and upgrades of generated Debian packages.
The scripts can be used for generating TAP reports of piuparts
runs checking Debian packages inside Jenkins jobs.
Conflicts: jenkins-debian-glue-buildenv-git,
jenkins-debian-glue-buildenv-lintian,
jenkins-debian-glue-buildenv-piuparts,
jenkins-debian-glue-buildenv-slave,
jenkins-debian-glue-buildenv-svn,
jenkins-debian-glue-buildenv-taptools
Replaces: jenkins-debian-glue-buildenv-git,
jenkins-debian-glue-buildenv-lintian,
jenkins-debian-glue-buildenv-piuparts,
jenkins-debian-glue-buildenv-slave,
jenkins-debian-glue-buildenv-svn,
jenkins-debian-glue-buildenv-taptools
Recommends: curl,
libperl-critic-perl,
pep8 (>= 1.4.6~),
qemu-user-static,
shellcheck
Description: Virtual package to be used on build slaves
This virtual package depends on the software packages required
for using jenkins-debian-glue as build environment.

Package: jenkins-debian-glue-buildenv-devuan
Architecture: all


+ 8
- 3
debian/copyright View File

@@ -3,15 +3,20 @@ Upstream-Name: jenkins-debian-glue
Source: https://github.com/mika/jenkins-debian-glue

Files: *
Copyright: 2011,2012 Michael Prokop <mika@debian.org>
Copyright: 2011-2015 Michael Prokop <mika@debian.org>
License: MIT

Files: debian/*
Copyright: 2011,2012 Michael Prokop <mika@debian.org>
Copyright: 2011-2015 Michael Prokop <mika@debian.org>
License: MIT

Files: pbuilder-hookdir/B20autopkgtest
Copyright: 2012 Christoph Berg <myon@debian.org>
2013-2015 Michael Prokop <mika@debian.org>
License: MIT
Copyright (c) 2011,2102 by Michael Prokop <mika@debian.org>

License: MIT
Copyright (c) 2011-2105 by Michael Prokop <mika@debian.org>
.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal


+ 0
- 1
debian/jenkins-debian-glue-buildenv-lintian.install View File

@@ -1 +0,0 @@
debian/tmp/usr/bin/lintian-junit-report usr/bin/

+ 0
- 2
debian/jenkins-debian-glue-buildenv-piuparts.install View File

@@ -1,2 +0,0 @@
debian/tmp/usr/bin/piuparts_tap usr/bin/
debian/tmp/usr/bin/piuparts_wrapper usr/bin/

+ 0
- 5
debian/jenkins-debian-glue-buildenv-taptools.install View File

@@ -1,5 +0,0 @@
debian/tmp/usr/bin/adtsummary_tap usr/bin/
debian/tmp/usr/bin/checkbashism_tap usr/bin/
debian/tmp/usr/bin/pep8_tap usr/bin/
debian/tmp/usr/bin/perlcritic_tap usr/bin/
debian/tmp/usr/bin/tap_tool_dispatcher usr/bin/

+ 9
- 0
debian/jenkins-debian-glue.install View File

@@ -1,10 +1,19 @@
debian/tmp/usr/bin/adtsummary_tap usr/bin/
debian/tmp/usr/bin/build-and-provide-package usr/bin/
debian/tmp/usr/bin/checkbashism_tap usr/bin/
debian/tmp/usr/bin/generate-git-snapshot usr/bin/
debian/tmp/usr/bin/generate-reprepro-codename usr/bin/
debian/tmp/usr/bin/generate-svn-snapshot usr/bin/
debian/tmp/usr/bin/increase-version-number usr/bin/
debian/tmp/usr/bin/jdg-debc usr/bin/
debian/tmp/usr/bin/lintian-junit-report usr/bin/
debian/tmp/usr/bin/pep8_tap usr/bin/
debian/tmp/usr/bin/perlcritic_tap usr/bin/
debian/tmp/usr/bin/piuparts_tap usr/bin/
debian/tmp/usr/bin/piuparts_wrapper usr/bin/
debian/tmp/usr/bin/remove-reprepro-codename usr/bin/
debian/tmp/usr/bin/repository_checker usr/bin/
debian/tmp/usr/bin/shellcheck_tap usr/bin/
debian/tmp/usr/bin/tap_tool_dispatcher usr/bin/
debian/tmp/usr/share/jenkins-debian-glue/examples/debian_glue etc/jenkins/
debian/tmp/usr/share/jenkins-debian-glue/pbuilder-hookdir/ usr/share/jenkins-debian-glue/

+ 32
- 13
examples/debian_glue View File

@@ -30,9 +30,17 @@
# By default no user specific pbuilder configuration file is used.
# PBUILDER_CONFIG=/etc/jenkins/pbuilderrc

# Base directory for reprepro repositories
# Base directory for reprepro repositories, can be overwritten
# via REPOSITORY on demand
# Default:
# REPOSITORY='/srv/repository'
# DEFAULT_REPOSITORY='/srv/repository'

# In order to keep the build versions unique and to ensure that there
# are no duplicates, the version number from the changelog is padded
# with the date and the build number.
# To override this behaviour and preserve the version number from the
# changelog, set USE_ORIG_VERSION to true.
# USE_ORIG_VERSION=true

# By default reprepro repositories are not verified but assumed to be
# trustworthy.
@@ -88,14 +96,25 @@
# Default:
# ADT=


# Use eatmydata (https://launchpad.net/libeatmydata) to disable
# fsync(), O_SYNC and so on to speed up build.
# Default:
# USE_EATMYDATA=false

# Use ccache to speed up builds on packages that
# needs to rebuild the same source multiple times by
# caching build results
# Default:
# USE_CCACHE=false
# Control usage of eatmydata to speed up builds. If eatmydata is present on the
# host system and when building for a recent distribution (Debian/jessie +
# Ubuntu/vivid or newer) eatmydata is enabled automatically.
# To force its usage (skipping any host + distribution checks) set it to 'true'.
# To disable its usage set it to 'false'.
# USE_EATMYDATA=true

# Enable ccache (compiler cache for fast recompilation of C/C++ code)
# to speed up builds.
# USE_CCACHE=true

# Specify the format (as accepted by the 'date' command) for the generated
# build timestamp.
# Default: %Y%m%d%H%M%S
# TIMESTAMP_FORMAT="%Y%m%d%H%M%S"

# In case you are explicitly marking your new package version as "UNRELEASED"
# inside the debian/changelog file, you may not get a package build version
# number containing the SCM commit. Set this to 'true' in order to work around
# this.
# Default: false
# UNRELEASED_APPEND_COMMIT=false

+ 34
- 16
pbuilder-hookdir/B20autopkgtest View File

@@ -31,27 +31,45 @@ if [ -n "${ADT_OPTIONS:-}" ] ; then
echo "*** Using provided ADT_OPTIONS $ADT_OPTIONS ***"
fi

set -- /tmp/adt-*/
TMPADT="$1"
if [ ! -d "$TMPADT" ]; then
echo "Didn't find any /tmp/adt-*/ directory. It should have been created before invoking cowbuilder." >&2
ls -la /tmp
exit 1
# try to launch adt-run in a new PID namespace so several testsuites can run
# in parallel, newpid exists in jessie and newer only though
unset newpid_name
if ! apt-cache policy newpid | grep -q 'newpid:' ; then
echo "The newpid package doesn't seem to be available, not considering for installation"
else
echo "The newpid package seems to be available, considering for installation"
newpid_name='newpid'
fi

# runner/adt-run uses apt-utils's apt-ftparchive and
# pbuilder's pbuilder-satisfydepends-classic
apt-get install -y autopkgtest apt-utils pbuilder
apt-get install -y autopkgtest apt-utils pbuilder $newpid_name

# since autopkgtest 3.16 the --tmp-dir option is gone, make sure
# we've --output-dir available though before using it
if adt-run --help | grep -q -- --output-dir 2>/dev/null ; then
OUTPUT_OPTION='--output-dir'
else
OUTPUT_OPTION='--tmp-dir'
fi

mkdir -p "$TMPADT/out"
binaries=$(awk '/\.deb / { print "--binary ../" $1 }' debian/files)
mkdir -p /tmp/buildd/autopkgtest.out

adt-run \
--tmp-dir "${TMPADT}/out" \
--summary "${TMPADT}/summary" \
$newpid_name adt-run \
${OUTPUT_OPTION} /tmp/buildd/autopkgtest.out \
--summary /tmp/buildd/autopkgtest.summary \
/tmp/buildd/*.deb \
--built-tree "${PWD}" \
$binaries ${ADT_OPTIONS:-} --- adt-virt-null || EXIT=$?
${ADT_OPTIONS:-} --- adt-virt-null || EXIT=$?

# collect autopkgtest output in a single file so pbuilder automatically copies it
tar acf /tmp/buildd/autopkgtest.tar.gz /tmp/buildd/autopkgtest.out

# 4 means some test failed, exit 0 here and let adtsummary_tap report the failure
[ ${EXIT:-0} = 4 ] && exit 0
exit ${EXIT:-0}
case ${EXIT:-0} in
2|4|6|8) # let adtsummary_tap report the failure
exit 0
;;
*)
exit ${EXIT:-0}
;;
esac

+ 1
- 0
pbuilder-hookdir/C10shell View File

@@ -27,6 +27,7 @@ if ! [ -d /var/cache/pbuilder/build/ ] ; then
cat >&2 <<EOF
┌──────────────────────────────────────────────────────────────────────────────┐
│ Error: could not access /var/cache/pbuilder/build/ - missing in bindmount? │
│ Please make sure /var/cache/pbuilder/build is included in USER_BINDMOUNTS │
└──────────────────────────────────────────────────────────────────────────────┘
EOF
exit 1


+ 1
- 5
puppet/jenkins_debian_glue.pp View File

@@ -189,11 +189,7 @@ class jenkins::software {
}

package { [ 'jenkins-debian-glue',
'jenkins-debian-glue-buildenv-git',
'jenkins-debian-glue-buildenv-lintian',
'jenkins-debian-glue-buildenv-svn',
'jenkins-debian-glue-buildenv-taptools',
'jenkins-debian-glue-buildenv-piuparts' ]:
'jenkins-debian-glue-buildenv' ]:
ensure => present,
require => [
File['/etc/apt/sources.list.d/jenkins-debian-glue.list'],


+ 223
- 22
scripts/build-and-provide-package View File

@@ -1,8 +1,13 @@
#!/bin/bash

set -x
if [ -z ${JENKINS_DEBIAN_GLUE_QUIET:-} ]; then
set -x
fi
set -u

# Debian bug #531885: cowbuilder build fails with restrictive umask
umask 022

# make sure cowbuilder/pbuilder/... are available
PATH='/bin:/sbin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin'

@@ -54,10 +59,25 @@ set_debootstrap() {
}

checks_and_defaults() {
# backwards compatibility, see PR#94
if [ -z "${REPOSITORY:-}" ] ; then
local repository_is_missing_in_env=true
else
local repository_is_missing_in_env=false
fi

if [ -r /etc/jenkins/debian_glue ] ; then
. /etc/jenkins/debian_glue
fi

# backwards compatibility, see PR#94
if [ -n "${REPOSITORY:-}" ] && $repository_is_missing_in_env ; then
echo "*** WARNING: 'REPOSITORY' set in /etc/jenkins/debian_glue but should be DEFAULT_REPOSITORY ***"
echo "*** WARNING: Setting DEFAULT_REPOSITORY to $REPOSITORY for backwards compatibility ***"
echo "*** WARNING: Please replace REPOSITORY=... in /etc/jenkins/debian_glue with DEFAULT_REPOSITORY=... ***"
DEFAULT_REPOSITORY="${REPOSITORY}"
fi

# make sure cowbuilder/pbuilder has access to the variable
if [ -n "${DEB_KEEP_BUILD_ENV:-}" ] ; then
export DEB_KEEP_BUILD_ENV
@@ -89,8 +109,13 @@ checks_and_defaults() {
echo "*** REPREPRO_OPTS is unset, using default: $REPREPRO_OPTS ***"
fi

if [ -z "${DEFAULT_REPOSITORY:-}" ] ; then
DEFAULT_REPOSITORY='/srv/repository'
fi

# REPOSITORY can overwrite DEFAULT_REPOSITORY, so define only if unset
if [ -z "${REPOSITORY:-}" ] ; then
REPOSITORY='/srv/repository'
REPOSITORY="${DEFAULT_REPOSITORY}"
fi

if [ -z "${PBUILDER_HOOKDIR:-}" ] ; then
@@ -133,6 +158,14 @@ bailout() {
[ -n "${1:-}" ] && EXIT="${1}" || EXIT=0
[ -n "${2:-}" ] && echo "$2" >&2

rm -f "${build_lockfile}"
# if we have an aborted build we have to clean up left behinds,
# but we have to make sure we only clean up the files from our run
if [ -r "${update_lockfile_pid}" ] ; then
rm -f "${update_lockfile_pid}"
rm -f "${update_lockfile}"
fi

if [ "${SKIP_BINARY_REMOVAL:-}" = "true" ] ; then
echo "*** Skipping binary removal as requested via SKIP_BINARY_REMOVAL=true. ***"
else
@@ -140,7 +173,7 @@ bailout() {
rm -f "$WORKSPACE"/binaries/*
fi

${SUDO_CMD:-} rm -rf /tmp/adt-$$ /tmp/apt-$$
${SUDO_CMD:-} rm -rf /tmp/apt-$$

[ -n "$start_seconds" ] && SECONDS="$[$(cut -d . -f 1 /proc/uptime)-$start_seconds]" || SECONDS="unknown"
echo "*** Finished execution of $0 at $(date) [running ${SECONDS} seconds] ***"
@@ -224,12 +257,6 @@ dist_and_arch_settings() {
echo "*** architecture is set to ${architecture} ***"
fi

# only set $arch for other functions in this script if PROVIDE_ONLY is set
if [ -n "${PROVIDE_ONLY:-}" ] ; then
echo "*** Config variable 'PROVIDE_ONLY' is set, not setting COWBUILDER_BASE, COWBUILDER_DIST and DIST ***"
return 0
fi

if [ -n "${distribution:-}" ] ; then
local DIST="${distribution}"
else
@@ -254,6 +281,11 @@ dist_and_arch_settings() {
COWBUILDER_BASE="/var/cache/pbuilder/base-${COWBUILDER_DIST}-${arch}.cow"
echo "*** No COWBUILDER_BASE set, using $COWBUILDER_BASE as cowbuilder base.cow ***"
fi

local lockfiles="/var/run/lock/${COWBUILDER_DIST}-${arch}"
build_lockfile="${lockfiles}.building.$$"
update_lockfile="${lockfiles}.update"
update_lockfile_pid="${lockfiles}.update.$$"
}

cowbuilder_init() {
@@ -282,6 +314,9 @@ cowbuilder_init() {
echo 'export CCACHEDIR="/var/cache/pbuilder/ccache"' >> "$pbuilderrc"
fi

use_eatmydata
use_ccache

# allow setting main pbuilder configuration file from outside, then append data
# as needed without actually writing anything to user-provided $PBUILDER_CONFIG
if [ -n "${PBUILDER_CONFIG:-}" ] ; then
@@ -312,7 +347,7 @@ cowbuilder_init() {
echo "*** PBUILDER_CONFIG is not set, continuing with checks for Ubuntu workaround ***"

if ! [ -r "/usr/share/debootstrap/scripts/${COWBUILDER_DIST}" ] ; then
echo "*** Warning: distribution ${COWBUILDER_DIST} not supported by debootstrap, not considering Ubuntu workaround ***"
echo "*** WARNING: distribution ${COWBUILDER_DIST} not supported by debootstrap, not considering Ubuntu workaround ***"
else
# debootstrap scripts for recent versions of Ubuntu are all pointing to gutsy, use that
# to identify Ubuntu as the distribution we want to build instead of hardcoding all
@@ -328,9 +363,46 @@ cowbuilder_init() {
fi
fi

# ensure that we've access to Debian's archive keyring if we're
# building for recent Debian releases on Ubuntu, see issue#130
if lsb_release --id 2>/dev/null | grep -q Ubuntu ; then
echo "*** Looks like we're building on Ubuntu, checking for distribution target ***"
if [ "$(readlink -f /usr/share/debootstrap/scripts/${COWBUILDER_DIST})" != '/usr/share/debootstrap/scripts/sid' ] ; then
echo "*** Doesn't look like we're building for Debian, not considering Debian archive keyring workaround ***"
else
if ! [ -r /usr/share/keyrings/debian-archive-keyring.gpg ] ; then
echo "*** WARNING: /usr/share/keyrings/debian-archive-keyring.gpg does not exist. ***"
echo "*** If building fails with 'E: Release signed by unknown key ...' please ensure package debian-archive-keyring is installed ***"
else
echo "*** Package debian-archive-keyring is present, enabling its usage for keyring ***"
echo "DEBOOTSTRAPOPTS=(${DEBOOTSTRAPOPTS[@]:-} '--keyring' '/usr/share/keyrings/debian-archive-keyring.gpg' )" >> "$pbuilderrc"
fi
fi
fi

echo "*** Listing pbuilder configuration file as reference: ***"
cat "$pbuilderrc"

if ls "${COWBUILDER_BASE}.building."* >/dev/null 2>&1 ; then
echo "*** Skipping update run because a build is in progress ***"
return 0
fi

if ls "${update_lockfile}"* >/dev/null 2>&1 ; then
echo "*** Update run already taking place, skipping ***"
return 0
fi

(
# if we cannot get the lock then somebody else is already running
if ! flock --nonblock 9 ; then
exit 1
fi

# in order to be able to clean up aborted runs we need to
# mark that this file is ours
touch "${update_lockfile_pid}"

if [ ! -d "${COWBUILDER_BASE}" ]; then
echo "*** Creating cowbuilder base $COWBUILDER_BASE for arch $arch and distribution $COWBUILDER_DIST ***"
sudo DIST="${distribution:-}" ARCH="${architecture:-}" ${ADT:+ADT=$ADT} \
@@ -339,13 +411,37 @@ cowbuilder_init() {
--debootstrapopts --arch --debootstrapopts "$arch" \
--debootstrapopts --variant=buildd --configfile="${pbuilderrc}" \
--hookdir "${PBUILDER_HOOKDIR}"
[ $? -eq 0 ] || bailout 1 "Error: Failed to create cowbuilder base ${COWBUILDER_BASE}."
[ $? -eq 0 ] || exit 2
else
echo "*** Updating cowbuilder cow base ***"
sudo DIST="${distribution:-}" ARCH="${architecture:-}" ${ADT:+ADT=$ADT} \
cowbuilder --update --basepath "${COWBUILDER_BASE}" --configfile="${pbuilderrc}"
[ $? -eq 0 ] || bailout 1 "Error: Failed to update cowbuilder base ${COWBUILDER_BASE}."
if [ "${SKIP_COWBUILDER_UPDATE:-}" = "true" ] ; then
echo "*** Skipping cowbuilder update as requested via SKIP_COWBUILDER_UPDATE ***"
else
echo "*** Updating cowbuilder cow base ***"
sudo DIST="${distribution:-}" ARCH="${architecture:-}" ${ADT:+ADT=$ADT} \
cowbuilder --update --basepath "${COWBUILDER_BASE}" --configfile="${pbuilderrc}"
[ $? -eq 0 ] || exit 3
fi
fi
) 9>"${update_lockfile}" ||
{
# depending on the exit code we have to distinguish between different failures
case "$?" in
1)
echo "*** Someone else is holding the lock file ${update_lockfile}, skipping create/update ***"
return 0
;;
2)
echo "*** Something went wrong with the creation of the cowbuilder environment. Cleaning up. ***"
rm -rf "${COWBUILDER_BASE}"
bailout 1 "Error: Failed to create cowbuilder base ${COWBUILDER_BASE}."
;;
3)
bailout 1 "Error: Failed to update cowbuilder base ${COWBUILDER_BASE}."
;;
esac
}

rm -f "${update_lockfile_pid}" "${update_lockfile}"

[ -n "${pbuilderrc:-}" ] && rm -f "$pbuilderrc"
}
@@ -471,7 +567,15 @@ autopkg_run() {
if [ -n "${ADT_OPTIONS:-}" ] ; then
echo "*** Using provided ADT_OPTIONS $ADT_OPTIONS ***"
else
ADT_OPTIONS="--tmp-dir adt-external/out --summary adt-external/summary"
# since autopkgtest 3.16 the --tmp-dir option is gone, make sure
# we've --output-dir available though before using it
if adt-run --help | grep -q -- --output-dir 2>/dev/null ; then
local adt_output_option='--output-dir'
else
local adt_output_option='--tmp-dir'
fi

ADT_OPTIONS="$adt_output_option adt-external/out --summary adt-external/summary"
rm -rf adt-external
mkdir -p adt-external
echo "*** Using default ADT_OPTIONS $ADT_OPTIONS ***"
@@ -482,23 +586,71 @@ autopkg_run() {
adt-run --changes $changes_file ${ADT_OPTIONS:-} --- $ADT_RUNNER || bailout $?
}

use_ccache() {
if [ "${USE_CCACHE:-}" = 'true' ] ; then
echo "*** USE_CCACHE is set to true, enabling ccache support ***"
echo 'CCACHEDIR="/var/cache/pbuilder/ccache"' >> "$pbuilderrc"
fi
}

enable_eatmydata() {
echo 'EXTRAPACKAGES="$EXTRAPACKAGES eatmydata"' >> "$pbuilderrc"
echo 'export LD_PRELOAD="${LD_PRELOAD:+$LD_PRELOAD:}libeatmydata.so"' >> "$pbuilderrc"
}

use_eatmydata() {
if [ "${USE_EATMYDATA:-}" = 'false' ] ; then
echo "*** eatmydata is disabled via USE_EATMYDATA [$USE_EATMYDATA] ***"
return 0
fi

if [ "${USE_EATMYDATA:-}" = 'true' ] ; then
echo "*** eatmydata is enabled via USE_EATMYDATA [$USE_EATMYDATA], forcing usage (skipping host/distribution checks) ***"
enable_eatmydata
else
if ! dpkg-query --show --showformat='${Status}' eatmydata | grep -q '^install' ; then
echo "*** eatmydata missing on host system, not considering for usage (force via USE_EATMYDATA=true) ***"
else
if ! dpkg --compare-versions "$(dpkg-query --show --showformat='${Version}' eatmydata)" gt '26-2' ; then
echo "*** eatmydata version on host needs to be newer than v26-2 (>=82-6 is recommended), skipping eatmydata ***"
else
echo "*** eatmydata is present on host system, now checking distribution support ***"
case "${COWBUILDER_DIST}" in
etch|lenny|squeeze|wheezy)
echo "*** Debian release $COWBUILDER_DIST doesn't provide eatmydata >=82-6, skipping eatmydata ***"
;;
warty|hoary|breezy|dapper|edgy|feisty|gutsy|hardy|intrepid|jaunty|karmi|lucid|maverick|natty|oneiric|quantal|raring|saucy|utopic|precise|trusty)
echo "*** Ubuntu release $COWBUILDER_DIST doesn't provide eatmydata >=82-6, skipping eatmydata ***"
;;
*)
echo "*** Distribution ${COWBUILDER_DIST} should provide recent eatmydata support, enabling eatmydata ***"
enable_eatmydata
;;
esac
fi
fi
fi
}

autopkgtest_results() {
if [ -n "${SKIP_AUTOPKGTEST_RESULTS:-}" ] ; then
echo "** Skipping autopkgtest_results as requested via SKIP_AUTOPKGTEST_RESULTS ***"
return 0
fi

# copy autopkgtest results from /tmp
rm -rf adt
${SUDO_CMD:-} chmod -R go+rX /tmp/adt-$$
cp -a /tmp/adt-$$ adt
mkdir -p adt
if [ -r autopkgtest.summary ] ; then
mv autopkgtest.summary adt/summary
else # do not fail if no autopkgtest run took place
touch adt/summary
fi
}

cowbuilder_run() {
echo "*** cowbuilder build phase for arch $architecture ***"
mkdir -p "$WORKSPACE"/binaries/ /tmp/adt-$$ /tmp/apt-$$
mkdir -p "$WORKSPACE"/binaries/ /tmp/apt-$$

local BINDMOUNTS="/tmp/adt-$$ /tmp/apt-$$ /var/cache/pbuilder/build ${USER_BINDMOUNTS:-}"
local BINDMOUNTS="/tmp/apt-$$ ${USER_BINDMOUNTS:-}"

# make sure we build arch specific packages only when necessary
identify_build_type
@@ -581,6 +733,9 @@ EOF
fi


use_eatmydata
use_ccache

# allow setting main pbuilder configuration file from outside, then append data
# as needed without actually writing anything to user-provided $PBUILDER_CONFIG
if [ -n "${PBUILDER_CONFIG:-}" ] ; then
@@ -593,6 +748,19 @@ EOF
fi
fi

counter=3600
while ls "${update_lockfile}."* >/dev/null 2>&1 && [ $counter -gt 0 ] ; do
echo "*** Update (or creation) of ${COWBUILDER_BASE} is in progress, waiting up to $counter seconds ***"
sleep 1
counter=$(( counter - 1 ))
done

if [ $counter -eq 0 ] ; then
bailout 1 "Error: ran into timeout because parallel create/update operation for ${COWBUILDER_BASE} didn't finish in time."
fi

touch "${build_lockfile}"

case "$architecture" in
i386)
linux32 sudo DIST="${distribution:-}" ARCH="${architecture:-}" ${ADT:+ADT=$ADT} \
@@ -615,6 +783,7 @@ EOF
;;
esac

rm -f "${build_lockfile}"
[ -n "${pbuilderrc:-}" ] && rm -f "$pbuilderrc"
}

@@ -833,6 +1002,35 @@ trunk_release() {
fi
}

# Get rid of *unused* files to avoid uploading unreferenced files to incoming
# directory. When generating the source package on jessie this might result in
# a *.tar.xz file. But when building the binary package for an older release
# like wheezy this might generate a *.tar.gz file. When directly copying those
# files into reprepro's incoming directory (e.g. for release builds) the
# *unused* tarball will be kept around, so let's avoid that.
drop_unused_debfiles() {
if [ "${DROP_UNUSED_DEBFILES:-}" = "false" ] ; then
echo "*** DROP_UNUSED_DEBFILES is disabled, not removing unreferenced artifact files. ***"
return 0
fi

if ! [ -x "$(which dcmd)" ] ; then
echo "Error: dcmd executable not available, please install devscripts package." >&2
exit 1
fi

(
cd "${WORKSPACE}/binaries/"
for file in ./* ; do
filename="$(basename "${file}")" # get rid of trailing ./
if ! dcmd ./*.changes | grep -q -- "${filename}" ; then
echo "*** Removing file ${filename} ***"
rm -f "./${filename}"
fi
done
)
}

release_repos() {
echo "*** Environment variable 'release' is set, running through release steps. ***"

@@ -850,6 +1048,9 @@ release_repos() {
REPOS="${release}" remove_packages
fi

# get rid of files that aren't mentioned in the changes files before copying to incoming directory
drop_unused_debfiles

cp "${WORKSPACE}/binaries/"* "${REPOSITORY}/incoming/${release}/"
[ $? -eq 0 ] || bailout 1 "Error: Failed to copy binary packages to release directory."



+ 54
- 26
scripts/generate-git-snapshot View File

@@ -1,6 +1,8 @@
#!/bin/bash

set -x
if [ -z ${JENKINS_DEBIAN_GLUE_QUIET:-} ]; then
set -x
fi
set -e
set -u

@@ -15,7 +17,7 @@ export DEBEMAIL

if [ ! -d "${SOURCE_DIRECTORY}" ] ; then
echo "Please run the script in the jenkins workspace." >&2
echo 'NOTE: Make sure that Source Code Management -> Branches to build -> Advanced has "Local subdirectory for repo (optional)" configured to "source"' >&2
echo 'NOTE: Make sure that "Source Code Management -> Git -> Additional Behaviours -> Check out to a sub-directory" is set to "source"' >&2
exit 1
fi

@@ -30,6 +32,15 @@ if [ -z "${GIT_COMMIT:-}" ] ; then
exit 1
fi

# since git-buildpackage >=0.6.24 only the gbp command is available
if [ -x "$(which gbp)" ] ; then
GBP_DCH="gbp dch"
GBP_BUILDPACKAGE="gbp buildpackage"
else
GBP_DCH="git-dch"
GBP_BUILDPACKAGE="git-buildpackage"
fi

JENKINS_DEBIAN_GLUE_VERSION=$(dpkg --list jenkins-debian-glue 2>/dev/null | awk '/^ii/ {print $3}')
if [ -n "${JENKINS_DEBIAN_GLUE_VERSION:-}" ] ; then
echo "*** Running jenkins-debian-glue version $JENKINS_DEBIAN_GLUE_VERSION ***"
@@ -39,8 +50,8 @@ if [ -n "${DIST:-}" -a -z "${KEEP_SOURCE_CHANGES:-}" ]; then
echo "*** Warning: DIST is set but KEEP_SOURCE_CHANGES is not, this doesn't make sense. ***"
echo "*** Please consider setting KEEP_SOURCE_CHANGES=true when setting DIST. ***"

if ! git-dch --help 2>&1 | grep -q -- --distribution ; then
echo "Error: git-dch doesn't support the --distribution option. Upgrade git-buildpackage to version 0.6.7 or newer." >&2
if ! ${GBP_DCH} --help 2>&1 | grep -q -- --distribution ; then
echo "Error: ${GBP_DCH} doesn't support the --distribution option. Upgrade git-buildpackage to version 0.6.7 or newer." >&2
exit 1
fi
fi
@@ -64,26 +75,43 @@ version_information() {

# retrieve and adjust information
ORIG_VERSION=$(dpkg-parsechangelog --count 1 | awk '/^Version/ {print $2}')
INCREASED_VERSION=$(increase-version-number $ORIG_VERSION)
DISTRIBUTION=$(dpkg-parsechangelog --count 1 | awk '/^Distribution/ {print $2}')
APPLY_VERSION_WORKAROUND=false

# we want to get a version string like
# $ORIG_VERSION+0~$TIMESTAMP.$BUILD_NUMBER~1.$GIT_ID
# so the version is always increasing over time, no matter what
TIMESTAMP="$(date -u +%Y%m%d%H%M%S)"
BUILD_VERSION="${TIMESTAMP}.${BUILD_NUMBER}" # git-dch appends something like ~1.gbp5f433e then
if [ "${USE_ORIG_VERSION:-}" = "true" ] ; then
echo "*** USE_ORIG_VERSION is set to 'true', keeping original version [${ORIG_VERSION}] from changelog ***"
VERSION_STRING="${ORIG_VERSION}"
else
# we want to get a version string like
# $ORIG_VERSION+0~$TIMESTAMP.$BUILD_NUMBER~1.$GIT_ID
# so the version is always increasing over time, no matter what
INCREASED_VERSION=$(increase-version-number $ORIG_VERSION)
if [ -n "${TIMESTAMP_FORMAT:-}" ] ; then
echo "*** Using custom timestamp format as specified by TIMESTAMP_FORMAT ***"
TIMESTAMP="$(date -u +"$TIMESTAMP_FORMAT")"
else
TIMESTAMP="$(date -u +%Y%m%d%H%M%S)"
fi
BUILD_VERSION="${TIMESTAMP}.${BUILD_NUMBER}" # git-dch appends something like ~1.gbp5f433e then

# we do NOT raise the version number, if we detect an unreleased version,
# otherwise the released version would be older than our snapshot builds
if [ "$DISTRIBUTION" = "UNRELEASED" ] && dpkg --compare-versions "$ORIG_VERSION" lt "$INCREASED_VERSION" ; then
echo "*** Not increasing version number as distribution is set to UNRELEASED ***"
INCREASED_VERSION="$ORIG_VERSION"

if [ "${UNRELEASED_APPEND_COMMIT:-}" = "true" ]; then
# However, as we will use plain 'dch' for this, we still want to have
# the (short) GIT commit id inside the version...
echo "*** UNRELEASED_APPEND_COMMIT is set to 'true', manually appending GIT commit to version number ***"
BUILD_VERSION="${BUILD_VERSION}~${GIT_COMMIT:0:7}"
fi
APPLY_VERSION_WORKAROUND=true
fi

# we do NOT raise the version number, if we detect an unreleased version,
# otherwise the released version would be older than our snapshot builds
APPLY_VERSION_WORKAROUND=false
if [ "$DISTRIBUTION" = "UNRELEASED" ] && dpkg --compare-versions "$ORIG_VERSION" lt "$INCREASED_VERSION" ; then
echo "*** Not increasing version number as distribution is set to UNRELEASED ***"
INCREASED_VERSION="$ORIG_VERSION"
APPLY_VERSION_WORKAROUND=true
VERSION_STRING="${INCREASED_VERSION}~${BUILD_VERSION}"
fi

VERSION_STRING="${INCREASED_VERSION}~${BUILD_VERSION}"

if [ -n "${distribution:-}" ] ; then
echo "Distribution variable found. Adding distribution specific version."
VERSION_STRING="${VERSION_STRING}+${distribution//-/_}"
@@ -98,7 +126,7 @@ dch_opts() {
echo "Found environment variable DCH_OPTS, set to ${DCH_OPTS}"
else
DCH_OPTS="-S --multimaint-merge --ignore-branch"
echo "Using default git-dch options (${DCH_OPTS})"
echo "Using default ${GBP_DCH} options (${DCH_OPTS})"
fi
DCH_COMBINED_OPTS="${DCH_OPTS}"

@@ -110,7 +138,7 @@ dch_opts() {
if [ -n "${DIST:-}" ]; then
DCH_EXTRA_OPTS="${DCH_EXTRA_OPTS} --distribution ${DIST} --force-distribution"
fi
echo "Using extra git-dch options (${DCH_EXTRA_OPTS})"
echo "Using extra ${GBP_DCH} options (${DCH_EXTRA_OPTS})"
fi
DCH_COMBINED_OPTS="${DCH_COMBINED_OPTS} ${DCH_EXTRA_OPTS}"

@@ -182,7 +210,7 @@ git_dch_auto() {
dch -b --distribution=UNRELEASED --newversion=$VERSION_STRING -- \
"SNAPSHOT autobuild for unreleased $ORIG_VERSION via jenkins-debian-glue."
else
git-dch --auto $DCH_COMBINED_OPTS
${GBP_DCH} --auto $DCH_COMBINED_OPTS
fi
}

@@ -207,7 +235,7 @@ identify_latest_change() {
fi

echo "Latest tag [${last_tag:-}] / merge [${last_merge:-}] seems to be $since"
git-dch -s "${since}" $DCH_COMBINED_OPTS
${GBP_DCH} -s "${since}" $DCH_COMBINED_OPTS

local NEW_VERSION=$(dpkg-parsechangelog | awk '/^Version: / {print $2}')

@@ -302,10 +330,10 @@ changelog_generation() {
identify_latest_change
fi
elif [ -r debian/gbp.conf ] ; then
echo "Found debian/gbp.conf, using git-dch with auto mode."
echo "Found debian/gbp.conf, using ${GBP_DCH} with auto mode."
git_dch_auto
else
echo "Using git-dch with auto mode."
echo "Using ${GBP_DCH} with auto mode."
git_dch_auto
fi

@@ -460,7 +488,7 @@ if [ "${IGNORE_GIT_BUILDPACKAGE:-}" = "true" ] ; then
echo "Skipping git-buildpackage execution as requested via IGNORE_GIT_BUILDPACKAGE ***"
else
gbp_opts # set $GBP_OPTS
git-buildpackage ${GBP_OPTS:-}
${GBP_BUILDPACKAGE} ${GBP_OPTS:-}
fi

quilt_cleanup


+ 29
- 3
scripts/generate-reprepro-codename View File

@@ -1,19 +1,37 @@
#!/bin/bash

set -x
if [ -z ${JENKINS_DEBIAN_GLUE_QUIET:-} ]; then
set -x
fi
set -e

bailout() {
[ -n "${1:-}" ] && EXIT="${1}" || EXIT=0
[ -n "${2:-}" ] && echo "$2" >&2
rm -f /var/lock/jdg-generate-reprepro-codename."$(id -un)"
exit $EXIT
}

# backwards compatibility, see PR#94
if [ -z "${REPOSITORY:-}" ] ; then
repository_is_missing_in_env=true
else
repository_is_missing_in_env=false
fi

if [ -r /etc/jenkins/debian_glue ] ; then
echo "*** Sourcing /etc/jenkins/debian_glue ***"
. /etc/jenkins/debian_glue
fi

# backwards compatibility, see PR#94
if [ -n "${REPOSITORY:-}" ] && $repository_is_missing_in_env ; then
echo "*** WARNING: 'REPOSITORY' set in /etc/jenkins/debian_glue but should be DEFAULT_REPOSITORY ***"
echo "*** WARNING: Setting DEFAULT_REPOSITORY to $REPOSITORY for backwards compatibility ***"
echo "*** WARNING: Please replace REPOSITORY=... in /etc/jenkins/debian_glue with DEFAULT_REPOSITORY=... ***"
DEFAULT_REPOSITORY="${REPOSITORY}"
fi

if [ "${BUILD_ONLY:-}" = "true" ] ; then
echo "BUILD_ONLY is set to true, ignoring request to generate local repository."
exit 0
@@ -80,13 +98,19 @@ if [ -n "${JENKINS_DEBIAN_GLUE_VERSION:-}" ] ; then
echo "*** Running jenkins-debian-glue version $JENKINS_DEBIAN_GLUE_VERSION ***"
fi

if [ -z "${DEFAULT_REPOSITORY:-}" ] ; then
echo "*** Repository variable DEFAULT_REPOSITORY is unset, using default [$DEFAULT_REPOSITORY] ***"
DEFAULT_REPOSITORY='/srv/repository'
fi

# REPOSITORY can overwrite DEFAULT_REPOSITORY, so define only if unset
if [ -z "${REPOSITORY:-}" ] ; then
REPOSITORY='/srv/repository'
REPOSITORY="${DEFAULT_REPOSITORY}"
echo "*** Repository variable REPOSITORY is unset, using default [$REPOSITORY] ***"
fi

if ! ${SUDO_CMD:-} mkdir -p "${REPOSITORY}"/conf ; then
echo "Error creating ${REPOSITORY}/conf (forgot to create /srv/repository and chown jenkins?)" >&2
echo "Error creating ${REPOSITORY}/conf (forgot to create ${REPOSITORY} and chown jenkins?)" >&2
exit 1
fi

@@ -144,4 +168,6 @@ echo "Added $REPOS as new codename/repos to the reprepro configuration."

) 9>/var/lock/jdg-generate-reprepro-codename."$(id -un)" || bailout 1 "Error while generating reprepro codename/repos."

rm -f /var/lock/jdg-generate-reprepro-codename."$(id -un)"

# vim:foldmethod=marker ts=2 ft=sh ai expandtab sw=2

+ 9
- 2
scripts/generate-svn-snapshot View File

@@ -1,6 +1,8 @@
#!/bin/bash

set -x
if [ -z ${JENKINS_DEBIAN_GLUE_QUIET:-} ]; then
set -x
fi
set -e
set -u

@@ -93,7 +95,12 @@ build_snapshot() {

DISTRIBUTION=$(dpkg-parsechangelog --count 1 | awk '/^Distribution/ {print $2}')

TIMESTAMP=$(date -u +%Y%m%d%H%M%S)
if [ -n "${TIMESTAMP_FORMAT:-}" ] ; then
echo "*** Using custom timestamp format as specified by TIMESTAMP_FORMAT ***"
TIMESTAMP="$(date -u +"$TIMESTAMP_FORMAT")"
else
TIMESTAMP="$(date -u +%Y%m%d%H%M%S)"
fi

if [ "$DISTRIBUTION" = "UNRELEASED" ] ; then
# we do NOT raise the version number, if we detect an unreleased version


+ 26
- 2
scripts/remove-reprepro-codename View File

@@ -1,14 +1,38 @@
#!/bin/bash

set -x
if [ -z ${JENKINS_DEBIAN_GLUE_QUIET:-} ]; then
set -x
fi
set -e

# backwards compatibility, see PR#94
if [ -z "${REPOSITORY:-}" ] ; then
repository_is_missing_in_env=true
else
repository_is_missing_in_env=false
fi

if [ -r /etc/jenkins/debian_glue ] ; then
. /etc/jenkins/debian_glue
fi

# backwards compatibility, see PR#94
if [ -n "${REPOSITORY:-}" ] && $repository_is_missing_in_env ; then
echo "*** WARNING: 'REPOSITORY' set in /etc/jenkins/debian_glue but should be DEFAULT_REPOSITORY ***"
echo "*** WARNING: Setting DEFAULT_REPOSITORY to $REPOSITORY for backwards compatibility ***"
echo "*** WARNING: Please replace REPOSITORY=... in /etc/jenkins/debian_glue with DEFAULT_REPOSITORY=... ***"
DEFAULT_REPOSITORY="${REPOSITORY}"
fi

if [ -z "${DEFAULT_REPOSITORY:-}" ] ; then
echo "*** Repository variable DEFAULT_REPOSITORY is unset, using default [$DEFAULT_REPOSITORY] ***"
DEFAULT_REPOSITORY='/srv/repository'
fi

# REPOSITORY can overwrite DEFAULT_REPOSITORY, so define only if unset
if [ -z "${REPOSITORY:-}" ] ; then
REPOSITORY='/srv/repository'
REPOSITORY="${DEFAULT_REPOSITORY}"
echo "*** Repository variable REPOSITORY is unset, using default [$REPOSITORY] ***"
fi

if [ -z "${REPREPRO_OPTS:-}" ] ; then


+ 23
- 1
scripts/repository_checker View File

@@ -3,12 +3,34 @@
JENKINS_DEBIAN_GLUE_VERSION=$(dpkg --list jenkins-debian-glue 2>/dev/null | awk '/^ii/ {print $3}')
JENKINS_DEBIAN_GLUE_VERSION=${JENKINS_DEBIAN_GLUE_VERSION:-unidentified}

# backwards compatibility, see PR#94
if [ -z "${REPOSITORY:-}" ] ; then
repository_is_missing_in_env=true
else
repository_is_missing_in_env=false
fi

if [ -r /etc/jenkins/debian_glue ] ; then
. /etc/jenkins/debian_glue
fi

# backwards compatibility, see PR#94
if [ -n "${REPOSITORY:-}" ] && $repository_is_missing_in_env ; then
echo "*** WARNING: 'REPOSITORY' set in /etc/jenkins/debian_glue but should be DEFAULT_REPOSITORY ***"
echo "*** WARNING: Setting DEFAULT_REPOSITORY to $REPOSITORY for backwards compatibility ***"
echo "*** WARNING: Please replace REPOSITORY=... in /etc/jenkins/debian_glue with DEFAULT_REPOSITORY=... ***"
DEFAULT_REPOSITORY="${REPOSITORY}"
fi

if [ -z "${DEFAULT_REPOSITORY:-}" ] ; then
echo "*** Repository variable DEFAULT_REPOSITORY is unset, using default [$DEFAULT_REPOSITORY] ***"
DEFAULT_REPOSITORY='/srv/repository'
fi

# REPOSITORY can overwrite DEFAULT_REPOSITORY, so define only if unset
if [ -z "${REPOSITORY:-}" ] ; then
REPOSITORY='/srv/repository'
REPOSITORY="${DEFAULT_REPOSITORY}"
echo "*** Repository variable REPOSITORY is unset, using default [$REPOSITORY] ***"
fi

if [ -z "${REPREPRO_OPTS:-}" ] ; then


+ 1
- 1
tap/pep8_tap View File

@@ -67,7 +67,7 @@ class Issue
'code' => @code,
'source' => @source.join("\n"),
'file' => @file
}) + '---' # end of yaml
}) + '...' # end of yaml
return ("not ok #{counter} #{@firstline}\n " + body.gsub("\n","\n "))
end
end


+ 131
- 7
tap/tap_tool_dispatcher View File

@@ -2,6 +2,75 @@

set -u

BASE_BIN=${BASE_BIN:-/usr/bin}

# command line handling
CMDLINE_OPTS=disable-checkbashism,disable-pep8,disable-perlcritic,disable-shellcheck
CMDLINE_OPTS+=,file-list,help

_opt_temp=$(getopt --name tap_tool_dispatcher -o -f+h --long $CMDLINE_OPTS -- "$@")
if [ $? -ne 0 ]; then
echo "Try '$0 --help' for more information." >& 2
exit 1
fi
eval set -- "$_opt_temp"

# defaults
_opt_checkbashism=true
_opt_pep8=true
_opt_perlcritic=true
_opt_shellcheck=true

usage() {
echo "$0 - run code style quality tools in parallel

Usage: $0 [<options>]

Supported options:

--disable-checkbashism Disable checkbashism execution
--disable-pep8 Disable pep8 execution
--disable-perlcritic Disable perlcritic execution
--disable-shellcheck Disable shellcheck execution
-f|--file-list FILE File with the list of files to check
-h|--help Display this usage information and exit
"
}

while :; do
case "$1" in
--help|-h)
usage ; exit 0;
;;
--disable-checkbashism)
_opt_checkbashism=false
;;
--disable-pep8)
_opt_pep8=false
;;
--disable-perlcritic)
_opt_perlcritic=false
;;
--disable-shellcheck)
_opt_shellcheck=false
;;
--file-list|-f)
[ -f "$2" ] || { echo "No $2 file found" >&2; exit 1; }
FILE_LIST="$2"
shift;
;;
--)
shift; break
;;
*)
echo "Internal getopt error! $1" >&2
exit 1
;;
esac
shift
done

# sanity checks before executing anything
if [ -z "${REPORTS_DIRECTORY:-}" ] ; then
export REPORTS_DIRECTORY='reports'
fi
@@ -19,14 +88,69 @@ fi
# start with clean report dir
rm -rf "${REPORTS_DIRECTORY}"

# run all tap tools at once, in parallel
find source -type f ! -path '*.svn*' ! -path '*.git/*' ! -name '.gitignore' -prune | parallel -v -- \
mkdir -p ${REPORTS_DIRECTORY}'/'{//} ';' \
/usr/bin/checkbashism_tap '{}' '>' ${REPORTS_DIRECTORY}'/{}_checkbashism.tap' ';' \
/usr/bin/perlcritic_tap '{}' '>' ${REPORTS_DIRECTORY}'/{}_perlcritic.tap' ';' \
/usr/bin/pep8_tap '{}' '>' ${REPORTS_DIRECTORY}'/{}_pep8.tap' ';'
# set up a temporary file which executes the requested commands
TMPFILE="$(mktemp)" || exit 1

# GNU parallel supports the -v option, while parallel from moreutils doesn't
unset PARALLEL_OPTION
if parallel --version >/dev/null 2>&1 ; then
PARALLEL_OPTION='-v'
fi

if [ -n "${FILE_LIST:-}" ] ; then
LIST="sed 's#^#source/#g' < $FILE_LIST"
else
LIST="find source -type f ! -path '*.svn*' ! -path '*.git/*' ! -name '.gitignore' -prune"
fi

echo "$LIST | parallel ${PARALLEL_OPTION:-} -- \\
mkdir -p ${REPORTS_DIRECTORY}'/'{//} ';' \\" > "${TMPFILE}"

if ! [ -x "$(which checkbashisms)" ] ; then
echo "*** Skipping checkbashism checks as tool checkbashisms isn't present ***"
else
if $_opt_checkbashism ; then
echo "${BASE_BIN}/checkbashism_tap '{}' '>' ${REPORTS_DIRECTORY}'/{}_checkbashism.tap' ';' \\" >> "${TMPFILE}"
else
echo "*** Skipping checkbashism tests as requested via --disable-checkbashism ***"
fi
fi

if ! [ -x "$(which pep8)" ] ; then
echo "*** Skipping pep8 checks as tool pep8 isn't present ***"
else
if $_opt_pep8 ; then
echo "${BASE_BIN}/pep8_tap '{}' '>' ${REPORTS_DIRECTORY}'/{}_pep8.tap' ';' \\" >> "${TMPFILE}"
else
echo "*** Skipping pep8 tests as requested via --disable-pep8 ***"
fi
fi

if ! [ -x "$(which perlcritic)" ] ; then
echo "*** Skipping perlcritic checks as tool perlcritic (package libperl-critic-perl) isn't present ***"
else
if $_opt_perlcritic ; then
echo "${BASE_BIN}/perlcritic_tap '{}' '>' ${REPORTS_DIRECTORY}'/{}_perlcritic.tap' ';' \\" >> "${TMPFILE}"
else
echo "*** Skipping pep8 tests as requested via --disable-perlcritic ***"
fi
fi

if ! [ -x "$(which shellcheck)" ] ; then
echo "*** Skipping shellcheck checks as tool shellcheck isn't present ***"
else
if $_opt_shellcheck ; then
echo "${BASE_BIN}/shellcheck_tap '{}' '>' ${REPORTS_DIRECTORY}'/{}_shellcheck.tap' ';'" >> "${TMPFILE}"
else
echo "*** Skipping shellcheck tests as requested via --disable-shellcheck ***"
fi
fi

# generate the executed script
bash "${TMPFILE}"
rm -f "${TMPFILE}"

# get rid of empty files
echo "*** Getting rid of empty files ***"
find "${REPORTS_DIRECTORY}" -type f -empty -exec rm {} +

## END OF FILE #################################################################

Loading…
Cancel
Save