Browse Source

[REWRITE] This is a full rewrite.

This is done with the hope that auditing, tweaking, maintaining and improving
ReleaseBot is not a dreaded chore but is indeed something that can be
understood.

This is still Work In Progress but appears to work mostly fine.
tags/v0.2
Evilham 1 year ago
parent
commit
21e86ba8e0
15 changed files with 496 additions and 1616 deletions
  1. +2
    -0
      .gitignore
  2. +21
    -0
      Pipfile
  3. +8
    -0
      README.md
  4. +5
    -2
      config.example
  5. +0
    -0
      libs/__init__.py
  6. +0
    -106
      libs/pytea-master/.gitignore
  7. +0
    -7
      libs/pytea-master/CONTRIBUTORS.md
  8. +0
    -13
      libs/pytea-master/LICENSE
  9. +0
    -111
      libs/pytea-master/README.md
  10. +0
    -155
      libs/pytea-master/pytea/__init__.py
  11. +0
    -1076
      libs/pytea-master/pytea/resources.py
  12. +0
    -7
      libs/pytea-master/requirements.txt
  13. +0
    -15
      libs/pytea-master/setup.py
  14. +0
    -1
      pytea
  15. +460
    -123
      releasebot.py

+ 2
- 0
.gitignore View File

@@ -0,0 +1,2 @@
config
*.pyc

+ 21
- 0
Pipfile View File

@@ -0,0 +1,21 @@
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
python-language-server = {extras = ["mccabe", "pyflakes", "rope"],version = "*"}
pyls-mypy = "*"
pyls-isort = "*"
pyls-black = "*"

[packages]
python-jenkins = "*"
attrs = "*"
requests = "*"

[requires]
python_version = "3"

[pipenv]
allow_prereleases = true

+ 8
- 0
README.md View File

@@ -1,5 +1,13 @@
# Devuan Releasebot

## Dependencies

This depends exclusively on following python 3.5+ libraries:

apt install python3-requests python3-jenkins python3-attr

For development you may want to use `pipenv install --dev`.

## WARNING: this is alpha code written for internal use in Devuan,
do not use it if you don't know what you are doing.



+ 5
- 2
config.example View File

@@ -3,9 +3,12 @@
[git]

token: YOUR_PRIVATE_TOKEN
username: YOUR_USERNAME
username: YOUR_USERNAME (defaults to 'releasebot')
password: YOUR_PASSWORD
host: https://git.devuan.org
namespace: YOUR_NAMESPACE (defaults to 'devuan')
gitea_org: YOUR_GITEA_ORG (defaults to namespace)
gitea_team: YOUR_GITEA_TEAM (defaults to 'packages')

[jenkins]

@@ -18,4 +21,4 @@ build_job: gitea-test

suites: experimental,unstable
restricted_suites: jessie, ascii, beowulf
restricted_suffixes: proposed, proposed-updates
restricted_suffixes: proposed, proposed-updates, proposed-security

+ 0
- 0
libs/__init__.py View File


+ 0
- 106
libs/pytea-master/.gitignore View File

@@ -1,106 +0,0 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
.static_storage/
.media/
local_settings.py

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/


+ 0
- 7
libs/pytea-master/CONTRIBUTORS.md View File

@@ -1,7 +0,0 @@
Contributors
============

Pytea has been improved by:

+ [happydig](https://github.com/happydig): [issue: code formatted with YAML](https://github.com/arount/pytea/issues/1)


+ 0
- 13
libs/pytea-master/LICENSE View File

@@ -1,13 +0,0 @@
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
Version 2, December 2004

Copyright (C) 2004 Sam Hocevar <sam@hocevar.net>

Everyone is permitted to copy and distribute verbatim or modified
copies of this license document, and changing it is allowed as long
as the name is changed.

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

0. You just DO WHAT THE FUCK YOU WANT TO.

+ 0
- 111
libs/pytea-master/README.md View File

@@ -1,111 +0,0 @@
# Pytea

[Gitea API](https://try.gitea.io/api/swagger) wrapper for Python (3 only).


## Install

### From sources

```python
python setup.py install
```

### With Pip

```python
pip install git+https://github.com/arount/pytea
```


## Basic usage


```python
import pytea

api = pytea.API('http://192.168.100.10:3000')
api.get('/version')
api.get('/orgs/an-organisation/members')
```


### Authentification token


Setup authentification token:

```python
import pytea
api = pytea.API('http://192.168.100.10:3000', token="AUTH-TOKEN")
api.delete('/admin/users/arount')
```


### Main API methods


```python
api.get('route') # Send GET query to route
api.post('route') # Send POST query to route
api.patch('route') # Send PATCH query to route
api.put('route') # Send PUT query to route
```


### Alternative API method


```python
# Send GET query to route with parameters
api.call('route', method='get', params={"body": "Egg, bacon, sausages and SPAM")
```


## Exceptions


Exceptions are raised before sending query. If API respond error message no exception will be raised (for the moment, at least)


### Authentification token


If auth token is not set when you are trying to access to a protected resource:

```python
api = pytea.API('http://192.168.100.10:3000')
api.delete('/admin/users/arount')
```


```
pytea.PyteaRequestException: Resource '/admin/users/{username}' require an authentification token.
```


### Resource do not exists


```python
api.get('/fake/route')
```


```
pytea.PyteaRequestException: Path '/fake/route' did not match with any resource
```


### Method do not exists for resource


```python
api.delete('/markdown')
```


```
pytea.PyteaRequestException: Resource '/markdown' did not expect method DELETE
```


+ 0
- 155
libs/pytea-master/pytea/__init__.py View File

@@ -1,155 +0,0 @@
#!/usr/env/bin python

'''
Gitea API wrapper for Python.
API doc: https://try.gitea.io/api/swagger
'''

import parse
import requests

from pytea.resources import resources


class PyteaRequestException(Exception):
pass


class API(object):
'''
Gitea API wrapper.
'''

_api_baseroute = '/api/v1'

def __init__(self, baseuri, token=None):
if baseuri.endswith('/'):
baseuri = baseuri[0:-1]

self._baseuri = ''.join([baseuri, self._api_baseroute])
self._token = token

# Aliases to `call`, this should be prefered entry-points
def post(self, path, params=None):
return self.call(path, method='post', params=params)

def get(self, path, params=None):
return self.call(path, method='get', params=params)

def delete(self, path, params=None):
return self.call(path, method='delete', params=params)

def patch(self, path, params=None):
return self.call(path, method='patch', params=params)

def put(self, path, params=None):
return self.call(path, method='put', params=params)


def call(self, path, method, params=None):
'''
Compute, check and execute request to API.
Returns requests.Response object.
'''
# Handle parameters
if params is None:
params = {}
method = method.lower()
resource = self.get_resource(path).copy()

# Check if request needs auth token
if path.split('/')[1] == 'admin' and self._token is None:
raise PyteaRequestException(
'Resource \'{}\' require an authentification token.'.format(resource['path'])
)
else:
params['token'] = self._token

# Check if `resource` expect to be called with `method` HTTP method
if not self._resource_has_method(resource, method):
raise PyteaRequestException('Resource \'{}\' did not expect method {}'.format(
resource['path'],
method.upper()
))

# Check if all required parameters are given
required_params = self.clean_resource_params(resource['path'], resource[method]['parameters'])
for key in required_params:
if key not in params.keys():
raise PyteaRequestException('Resource \'{}\' with method {} expect parameter \'{}\''.format(
resource['path'],
method.upper(),
key
))

func = getattr(requests, method)
final_uri = ''.join([self._baseuri, path])
if 'body' in params.keys():
return func(final_uri, params=params, json=params['body'])
else:
return func(final_uri, params=params)


def clean_resource_params(self, resource_path, params):
'''
Remove params in resource hash already given in resource's uri.
Example:
/foo/{bar}/{baz} expect `bar` and `baz`, but since they are in uri we don't want
to check that again.

Returns cleaned params hash
'''
to_rm = []
for name in params:
if '{{{}}}'.format(name) in resource_path:
to_rm.append(name)

for key in to_rm:
params.pop(key, None)

return params


def get_resource(self, path):
'''
Returns resource's hash from URI (path)
Augment resource's hash with non-formated path for convenience.
Returns augmented resource hash.
'''
key = self._get_resource_path(path)
if key is None:
raise PyteaRequestException('Path \'{}\' did not match with any resource'.format(path))

resource = resources[key].copy()
resource['path'] = key
return resource


def _resource_has_method(self, resource, method):
'''
Check if `resource` (path to resource) accept `method` (HTTP method).
'''
if method not in resource.keys():
return False
return True

def _get_resource_path(self, path):
'''
Get raw resource's path (/foo/{bar}) from formatted one (/foo/bar).
'''
# Two or more routes can match with `path`
# The one with more characters is the good one.
# I guess?..
possibilities = []
for method in resources:
compiled = parse.compile(method)
reparsed = compiled.parse(path)
if reparsed is not None:
possibilities.append(method)

if len(possibilities) == 0:
return None

# Return only best possibility
return sorted(possibilities, key=len)[-1]


+ 0
- 1076
libs/pytea-master/pytea/resources.py
File diff suppressed because it is too large
View File


+ 0
- 7
libs/pytea-master/requirements.txt View File

@@ -1,7 +0,0 @@
certifi==2018.1.18
chardet==3.0.4
idna==2.6
parse==1.8.2
pkg-resources==0.0.0
requests==2.18.4
urllib3==1.24.2

+ 0
- 15
libs/pytea-master/setup.py View File

@@ -1,15 +0,0 @@
#!/usr/env/bin python

from setuptools import setup

setup(
name='pytea',
version='0.0',
description='Gitea API wrapper for python',
url='http://github.com/arount/pytea',
author='Arount',
author_email='arount@riseup.net',
license='WTFPL',
packages=['pytea'],
zip_safe=False
)

+ 0
- 1
pytea View File

@@ -1 +0,0 @@
libs/pytea-master/pytea

+ 460
- 123
releasebot.py View File

@@ -19,127 +19,464 @@
# along with Foobar. If not, see <http://www.gnu.org/licenses/>.
##########################################################################

import pytea
import jenkins
import abc
import configparser
import os, sys

# Wrapper to handle gitea API errors
def gitea_call(method, url, params=None):
global git
response = getattr(git, method)(url, params=params)
if response.status_code >= 400:
raise RuntimeError('Gitea API request {} failed with {}'.format(url, response.status_code))
return response

def issue_add_comment(issue, comment):
return gitea_call('post',
'/repos/{owner}/{repo}/issues/{index}/comments'.format(owner=issue['repository']['owner'],
repo=issue['repository']['name'],
index=['number']),
params={'id': 1, 'body': {'body': comment}})

def issue_close_with_comment(issue, comment):
issue_add_comment(issue, comment)
return gitea_call('patch',
'/repos/{owner}/{repo}/issues/{index}'.format(owner=issue['repository']['owner'],
repo=issue['repository']['name'],
index=['number']),
params={'body' : {'state': 'closed'}})

def build_issue(issue,testmode):
print('Checking for permissions')
teams=gitea_call('get', '/orgs/devuan/teams/search', params={'q' : 'packages', 'include_desc' : False }).json()['data']
if len(teams) != 1:
print('Failed to identify packages team id')
return

build_suites=[s.strip() for s in cfg.get('filters', 'suites').split(',')]
if issue['user']['is_admin']:
import itertools
restricted_suites=[s.strip() for s in cfg.get('filters', 'restricted_suites').split(',')]
build_suites.extend(restricted_suites)
build_suites.extend(['-'.join(s) for s in itertools.product(restricted_suites, [s.strip() for s in cfg.get('filters', 'restricted_suffixes').split(',')])])
elif not gitea_call('get', '/teams/{id}/members/{username}'.format(id=teams[0]['id'], username=issue['user']['username'])).ok:
print('User {username} is not a member of packages team.'.format(username=issue['user']['username']))
if not testmode:
issue_close_with_comment(issue, 'Unable to build this issue, {username} is not a member of the packagesteam'.format(username=issue['user']['username']))
return
print('Checking jenkins for job exixtence')
print('Connecting to jenkins...')
jk=jenkins.Jenkins(cfg.get('jenkins', 'host'),
cfg.get('jenkins', 'username'),
cfg.get('jenkins', 'password'))

jkjob_name=cfg.get('jenkins', 'build_job')
if not jk.job_exists(jkjob_name):
raise Error('Build job not found on jenkins')

buildfor=list(set(build_suites) & set([l['name'] for l in issue['labels']]))

if len(buildfor) < 1:
print('Error: no valid suite labels found')
# TODO: or should we default to experimental?
if not testmode:
issue_close_with_comment(issue, 'No valid suite label found.\n\n Available suites are: {suites}.\n\nClosing.'.format(suites=', '.join(str(i) for i in build_suites)))
else:
issue_add_comment(issue, 'Testmode: no valid suite label found.\n\n Available suites are: {suites}.'.format(suites=', '.join(str(i) for i in build_suites)))
return
for build in buildfor:
print('Building for '+str(build))
if not testmode:
jk.build_job(jkjob_name, {'codename': build, 'srcpackage': issue['repository']['name']})
issue_add_comment(issue, 'Triggered build for {build}.\n\n{url}{buildno}'.format(build=build,
url=jk.get_job_info(jkjob_name)['url'],
buildno=jk.get_job_info(jkjob_name)['nextBuildNumber']))
else:
issue_add_comment(issue, 'Testmode: build for {build}.'.format(build=build))

print('Processed all valid suite labels')
if not testmode:
issue_close_with_comment(issue, 'All builds queued')


def main(cfg, argv):
global git

testmode = False
if len(argv) > 1:
if argv[1] == 'test':
testmode = True

print('Connecting to git...')
git = pytea.API(cfg.get('git', 'host'), token=cfg.get('git', 'token'))

print('Getting list of issues...')
for issue in gitea_call('get', '/repos/issues/search', params={'state': 'open', 'q': 'build' }).json():
if issue['repository']['owner'] != 'devuan' or issue['assignee']['username'] != 'Releasebot':
if testmode:
print('Skipping '+str(issue['id']))
continue

# TODO: refuse reopened issues?

print('* Considering project {project} issue #{number}: {title}. *'.format(project=issue['repository']['full_name'],
number=issue['number'],
title=issue['title']))
if str(issue['title']).strip().lower() == 'build':
build_issue(issue, testmode)
# Add other commands here
else:
if not testmode:
issue_close_with_comment(issue, 'Issue title isn\'t a valid releasebot command.')
print('All issues processed')


if __name__ == '__main__':
rc_file='./config.test'
# Ensure the config file with secrets is mode 600
assert(os.stat(rc_file).st_mode & 0o177 == 0)
cfg = configparser.ConfigParser()
cfg.read(rc_file)
main(cfg, sys.argv)
import functools
import typing

import attr
import jenkins
import requests


def _list_helper(
config, section: str, item: str, separator: str = ","
) -> typing.Iterable[str]:
"""
Helper for comma-separated lists in .ini configurations
"""
return [s.strip() for s in config.get(section, item).split(separator)]


@attr.s
class ReleaseBotCommand(object):
"""
Data container for a ReleaseBot command.
Which commands are supported is determined by the cmd_COMMAND methods in the
ReleaseBot class.
"""

name = attr.ib(type=str)
package = attr.ib(type=str)
labels = attr.ib(type=typing.Iterable[str])


@attr.s
class ReleaseBotRequester(object):
"""
Data container for the requester and their permissions.
is_admin provides access to building anything.
permissions can be one of: 'member' or 'collaborator/PACKAGE'
'member' provides access to building anything but restricted suites and
prefixes
'collaborator/PACKAGE' provides access to building PACKAGE
This list of permissions is not guaranteed to be exhaustive, but should be
enough to make decisions about the build.
"""

name = attr.ib(type=str)
is_admin = attr.ib(type=bool)
permissions = attr.ib(factory=list, type=typing.Iterable[str])


@attr.s
class ReleaseBotTask(object):
"""
Data container for all necessary data to carry out a ReleaseBot task.
"""

id = attr.ib(type=str)
repo_store = attr.ib(repr=False, type= "GenericRepoStore")
command = attr.ib(type=ReleaseBotCommand)
requester = attr.ib(type=ReleaseBotRequester)
message = attr.ib(default="", type=str)

def notify(self, message: str, resolve: bool = False) -> type(None):
"""
Provide some progress notification regarding this task.
Tasks will only be resolved if this is not a dryrun.
"""
self.repo_store.notify_task(self, message, resolve)

def resolve_notify(self, message: str) -> type(None):
"""
Resolve a task with a notification.
Tasks will only be resolved if this is not a dryrun.
"""
self.repo_store.notify_task(self, message, resolve=True)


class GenericRepoStore(abc.ABC):
"""
Basic interface to abstract away the repository store (Gitea, GitLab, ...).
If we ever need a different subsystem, this would hopefully stay the same.
"""

def __init__(self, config, args):
self.config = config
self.args = args
self._host = self.config.get("git", "host")
self._token = self.config.get("git", "token")
self.namespace = self.config.get("git", "namespace", fallback="devuan").lower()
self.username = self.config.get(
"git", "username", fallback="releasebot"
).lower()

@abc.abstractmethod
def get_pending_tasks(self) -> typing.Iterable[ReleaseBotTask]:
"""
Get assigned issues/jobs/builds to the configured user in the namespace.

Underlying implementations should provide all information needed to make
a decision about whether or not and how such a request is to be processed.
"""
pass

def notify_task(self, task: ReleaseBotTask, message: str, resolve: bool):
"""
Provide some progress notification regarding a task and optionally
resolve it.
Tasks will only be resolved if this is not a dryrun.
"""
if self.args.debug:
print("[DEBUG] ", message, task)
if not self.args.dryrun:
self.do_notify_task(task, message, resolve)

@abc.abstractmethod
def do_notify_task(self, task: ReleaseBotTask, message: str, resolve: bool):
"""
Actually perform the notification. To be overridden in each implementation.
This is never called if this is a dryrun.
"""
pass


class GiteaRepoStore(GenericRepoStore):
"""
Gitea specific bits.
"""

def __init__(self, config, args):
super().__init__(config, args)
# These could be overridden in the config, but we have sane defaults
self.gitea_org = self.config.get("git", "gitea_org", fallback=self.namespace)
self.gitea_team = self.config.get("git", "gitea_team", fallback="packages")
try:
# python-requests 3
self._http = requests.HTTPSession()
except AttributeError:
# python-requests 2
self._http = requests.Session()
# This takes care of authentication against gitea
self._http.headers.update(
{
"Authorization": "token {}".format(self._token),
"Accept": "application/json",
"Content-Type": "application/json",
}
)

def get_pending_tasks(self) -> typing.Iterable[ReleaseBotTask]:
issues = self._get_build_issues()
for issue in issues:
# Basic data
command = issue["title"].strip().lower()
package = issue["repository"]["name"]
labels = set([l["name"] for l in issue["labels"]])
repo_owner = issue["repository"]["owner"].lower()
issue_assignee = issue["assignee"]["username"].lower()
task_id = "{}/{}/issues/{}".format(self.namespace, package, issue["id"])

if self.args.debug:
print("[DEBUG] Considering {}: {}.".format(task_id, command))

if repo_owner != self.namespace or issue_assignee != self.username:
# Silently skip if outside of the given namespace / user
if self.args.debug:
print(
"[DEBUG] Skipping {}/{}/{}".format(
repo_owner, package, issue["id"]
)
)
continue
# TODO: refuse reopened issues?
# does gitea provide this information in the API?

# Setup the ReleaseBotCommand
releasebot_command = ReleaseBotCommand(
name=command, package=package, labels=labels
)

# Setup the ReleaseBotRequester with their permissions
requester = ReleaseBotRequester(
name=issue["user"]["username"], is_admin=issue["user"]["is_admin"]
)
if self.is_packages_member(requester.name):
requester.permissions.append("member")
if self.is_package_collaborator(requester.name, package):
requester.permissions.append("collaborator/{}".format(package))

task = ReleaseBotTask(
id=task_id,
repo_store=self,
command=releasebot_command,
requester=requester,
)
yield task

def do_notify_task(self, task: ReleaseBotTask, message: str, resolve: bool):
url = "/repos/{}/comments".format(task.id)
self._request("post", url, params=dict(body=message))
if resolve:
url = "/repos/{}".format(task.id)
self._request("patch", url, params=dict(state="closed"))

def _get_build_issues(self):
"""
Overridable to facilitate testing.
"""
return self._request(
"get", "/repos/issues/search", params={"state": "open", "q": "build"}
).json()

@property
@functools.lru_cache()
def packages_team_id(self) -> int:
"""
We need this id internally for Gitea's API, it's cached per run.
"""
teams = self._request(
"get",
"/orgs/{}/teams/search".format(self.gitea_org),
params={"q": self.gitea_team, "include_desc": False},
).json()["data"]
if len(teams) != 1:
raise RuntimeError("Failed to identify packages team id")
return teams[0]["id"]

@functools.lru_cache()
def is_packages_member(self, username: str) -> bool:
"""
This is critical for permission management.
It is cached for each user for this run for consistency.
"""
response = self._request(
"get",
"/teams/{id}/members/{username}".format(
id=self.packages_team_id, username=username
),
)
return response.ok

@functools.lru_cache()
def is_package_collaborator(self, username, package) -> bool:
"""
This is critical for permission management.
It is cached for each user/package combination for this run.
"""
response = self._request(
"get",
"/repos/{owner}/{repo}/collaborators/{username}".format(
owner=self.gitea_org, repo=package, username=username
),
)
return response.ok

def _request(self, method: str, url: str, params: dict = dict()):
response = self._http.request(
method, "{}/api/v1{}".format(self._host, url), params=params
)
if response.status_code >= 400 and response.status_code != 404:
raise RuntimeError(
"Gitea API request {} failed with {}".format(url, response.status_code)
)
return response


class GenericJobStore(abc.ABC):
"""
Basic interface to abstract away the job store (Jenkins, buildbot, ...).
If we ever need a different subsystem, this would hopefully stay the same.
"""

def __init__(self, config, args):
self.config = config
self.args = args

def queue_build(self, suite: str, package: str) -> str:
"""
Queue a build for 'package' for 'suite' if not in dryrun mode.
Return the URL for the build or, if in dryrun mode, an empty string.
"""
url = ""
if self.args.debug:
print('[DEBUG] Queueing "{}" build for "{}"'.format(package, suite))
if not self.args.dryrun:
url = self.do_queue_build(suite, package)
return url

@abc.abstractmethod
def do_queue_build(self, suite: str, package: str) -> str:
"""
Actually queue the build. To be overridden in each implementation.
This is never called if this is a dryrun.
"""
pass


class JenkinsJobStore(GenericJobStore):
"""
Jenkins specific bits.
"""

def __init__(self, config, args):
super().__init__(config, args)
# Connect to Jenkins
self.jk = jenkins.Jenkins(
self.config.get("jenkins", "host"),
self.config.get("jenkins", "username"),
self.config.get("jenkins", "password"),
)

# Check job (sanity)
self.jkjob_name = self.config.get("jenkins", "build_job")
if not self.jk.job_exists(self.jkjob_name):
raise RuntimeError("Build job not found on jenkins")

if self.args.debug:
print(
"[DEBUG] Connected to Jenkins and checked build_job ", self.jkjob_name
)

def do_queue_build(self, suite: str, package: str) -> str:
"""
Queue build if not in dryrun mode.
Return the URL for the build or, if in dryrun mode, an empty string.
"""
self.jk.build_job(self.jkjob_name, {"codename": suite, "srcpackage": package})
buildno = self.jk.get_job_info(self.jkjob_name["nextBuildNumber"])
return "{}{}".format(self.jk.get_job_info(self.jkjob_name["url"]), buildno)


class ReleaseBot(object):
def __init__(
self, config, args, job_store: GenericJobStore, repo_store: GenericRepoStore
):
self.config = config
self.args = args
self.job_store = job_store
self.repo_store = repo_store

self.suites = _list_helper(config, "filters", "suites")
self.restricted_suites = _list_helper(config, "filters", "restricted_suites")
self.restricted_suffixes = _list_helper(
config, "filters", "restricted_suffixes"
)

def process(self):
for task in self.repo_store.get_pending_tasks():
yield self._process_task(task)

def cmd_build(self, task: ReleaseBotTask):
for suite in task.command.labels:
self.job_store.queue_build(suite, task.command.package)

def _process_task(self, task: ReleaseBotTask) -> ReleaseBotTask:
# 0. Check command is valid
cmd = getattr(self, "cmd_{}".format(task.command.name), None)
if cmd is not None:
# 1. check permissions
# TODO: check that suite exists and all that
if self.may_run_task(task):
# 2. Process command
cmd(task)
# 3. notify as necessary
task.resolve_notify("Finished queueing")
else:
task.resolve_notify("You don't have permissions to run this task")
else:
task.resolve_notify('Command not supported: "{}"'.format(task.command.name))
return task

def label_is_restricted(self, suite: str) -> bool:
"""
A label is considered restricted if on of following match:
- It starts with a restricted suite
- It ends with a restricted suffix
"""
# Restricted suites
for prefix in self.restricted_suites:
if suite.startswith(prefix):
return True
# Restricted suffixes
for suffix in self.restricted_suffixes:
if suite.endswith(suffix):
return True
return False

def may_run_task(self, task: ReleaseBotTask) -> bool:
"""
Decide whether or not a task may be run.
This is returns false if any suite does not match the requester's permissions.
See ReleaseBotRequester for the verbose description of the permissions.
"""
# Admins may run tasks
if task.requester.is_admin:
return True
# If there is at least one restricted suite, we refuse the whole task
restricted_build = any(
self.label_is_restricted(suite) for suite in task.commands.labels
)
# Restricted tasks are only for admins
if restricted_build:
return False
# Package members may run unrestricted tasks
if "member" in task.requester.permissions:
return True
# Collaborators may run unrestricted tasks on this package
if "collaborator/{}".format(task.command.package) in task.requester.permissions:
return True
# Default to deny
return False


def get_argparser():
"""
Get ReleaseBot's argument parser.

Any new parameters that are implemented should be added here.
"""
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("-c", "--config", type=argparse.FileType("r"), required=True)
parser.add_argument("--debug", action="store_true")
parser.add_argument("--dryrun", action="store_true")

return parser


def main(config, args):
"""
Perform a ReleaseBot run
"""
repo_store = GiteaRepoStore(config, args)
job_store = JenkinsJobStore(config, args)
releasebot = ReleaseBot(
config=config, args=args, repo_store=repo_store, job_store=job_store
)

print(list(releasebot.process()))


if __name__ == "__main__":
import sys

args = get_argparser().parse_args()

if args.config.name != "<stdin>" and not isinstance(args.config.name, int):
import os

# Eargsure the config file with secrets is mode 0600 or 0400
# This check is skipped if config is passed via stdin or file descriptor
if os.stat(args.config.name).st_mode & 0o177 != 0:
sys.stderr.write(
"Permissions are too broad ({})\n".format(args.config.name)
)
sys.exit(1)

config = configparser.ConfigParser()
config.read_file(args.config)

main(config, args)

Loading…
Cancel
Save