Browse Source

* Add support for reporting using http POST.

* new script /usr/share/popularity-contest/popcon-upload
   * new CGI /usr/share/doc/popularity-contest/popcon.cgi
   * Uploading using content-type text/plain, content-encoding x-gzip.
   * debian/cron.weekly:
     - do not run sendmail if it is not available.  This will make it
       possible to downgrade mail-transport-agent from Depend to
       Recommends if we move to HTTP.
     - try to run /usr/share/popularity-contest/popcon-upload if
       USEHTTP is enabled, disabled by default.
   * Code done by Bill Allombert.  Patch based on code from Ubuntu,
     though reimplemented in perl to avoid python dependency.  The
     ubuntu versions are included in the source package as
     popcon-submit-ubuntu.cgi and popcon-upload-ubuntu.
   * Added simple FAQ entry about the http POST feature.
   * Depend on ${misc:Depends} instead of debconf, to get the updated
     debconf dependency.
   * TODO:
       + Edit popcon-submit.cgi to report to the main server
       + Add timeout in the upload script.
       + Consider using existing perl module to submit file using http.
suites/ascii
Petter Reinholdtsen 17 years ago
parent
commit
f277c729e7
  1. 6
      FAQ
  2. 26
      debian/changelog
  3. 2
      debian/control
  4. 21
      debian/cron.weekly
  5. 3
      debian/rules
  6. 6
      default.conf
  7. 38
      popcon-submit-ubuntu.cgi
  8. 29
      popcon-submit.cgi
  9. 58
      popcon-upload
  10. 48
      popcon-upload-ubuntu

6
FAQ

@ -76,4 +76,10 @@ do_sendmail()
Make sure myuser account is properly configured to send email with a valid
return address.
Q) My system are unable to send email out to the Internet. How can I
participate.
An alternative is to use the new experimental http submission support.
This can be enabled using this command:
echo USEHTTP="yes" >> /etc/popularity-contest.conf

26
debian/changelog

@ -1,3 +1,29 @@
popularity-contest (1.30) UNRELEASED; urgency=low
* Add support for reporting using http POST.
* new script /usr/share/popularity-contest/popcon-upload
* new CGI /usr/share/doc/popularity-contest/popcon.cgi
* Uploading using content-type text/plain, content-encoding x-gzip.
* debian/cron.weekly:
- do not run sendmail if it is not available. This will make it
possible to downgrade mail-transport-agent from Depend to
Recommends if we move to HTTP.
- try to run /usr/share/popularity-contest/popcon-upload if
USEHTTP is enabled, disabled by default.
* Code done by Bill Allombert. Patch based on code from Ubuntu,
though reimplemented in perl to avoid python dependency. The
ubuntu versions are included in the source package as
popcon-submit-ubuntu.cgi and popcon-upload-ubuntu.
* Added simple FAQ entry about the http POST feature.
* Depend on ${misc:Depends} instead of debconf, to get the updated
debconf dependency.
* TODO:
+ Edit popcon-submit.cgi to report to the main server
+ Add timeout in the upload script.
+ Consider using existing perl module to submit file using http.
-- Petter Reinholdtsen <pere@debian.org> Sat, 2 Jul 2005 16:58:28 +0200
popularity-contest (1.29) unstable; urgency=low
* Please note when reading stats that sarge was released with 1.28.

2
debian/control

@ -8,7 +8,7 @@ Standards-Version: 3.6.1
Package: popularity-contest
Architecture: all
Depends: dpkg (>= 1.10), exim4 | mail-transport-agent, debconf, ${perl:Depends}
Depends: ${misc:Depends}, dpkg (>= 1.10), exim4 | mail-transport-agent, ${perl:Depends}
Recommends: cron
Suggests: anacron
Provides: popcon

21
debian/cron.weekly

@ -9,6 +9,8 @@ unset MAILFROM
unset MAILTO
unset MY_HOSTID
unset PARTICIPATE
unset SUBMITURL
unset USEHTTP
# get configuration information
. /usr/share/popularity-contest/default.conf
@ -24,6 +26,7 @@ if [ -z "$MAILTO" ]; then exit 0; fi
cd /var/log
umask 022
savelog -c 7 popularity-contest >/dev/null
POPCON=/var/log/popularity-contest
run_popcon()
{
@ -47,8 +50,18 @@ do_sendmail()
fi
}
# generate and transmit the popularity contest data
# generate the popularity contest data
run_popcon \
| tee /var/log/popularity-contest \
| do_sendmail
run_popcon > $POPCON
# try to email the popularity contest data
if [ -x "`which sendmail 2>/dev/null`" ]; then
do_sendmail < $POPCON
fi
# try to post the report through http POST
if [ "$SUBMITURL" ] && [ "yes" = "$USEHTTP" ]; then
setsid /usr/share/popularity-contest/popcon-upload \
-u $SUBMITURL -f $POPCON 2>/dev/null
fi

3
debian/rules

@ -26,6 +26,7 @@ install:
install -d debian/tmp/etc/
install -d debian/tmp/usr/share/popularity-contest/
install popularity-contest debian/tmp/usr/sbin/
install popcon-upload debian/tmp/usr/share/popularity-contest/
install popcon-largest-unused debian/tmp/usr/sbin/
install -m 644 default.conf debian/tmp/usr/share/popularity-contest/
@ -34,7 +35,7 @@ binary-indep: build install
dh_testdir
dh_testroot
dh_installdocs README FAQ
dh_installexamples prepop.pl popcon-process.sh popanal.py popcon.pl popcon-stat.pl README.examples
dh_installexamples prepop.pl popcon-process.sh popanal.py popcon.pl popcon-stat.pl popcon-submit.cgi README.examples
dh_installmenu
dh_installcron
dh_installman debian/popularity-contest.8 debian/popcon-largest-unused.8

6
default.conf

@ -28,6 +28,12 @@ MAILTO="survey@popcon.debian.org"
#
#MAILFROM="root@example.org"
# SUBMITURL is where to submit popularity-contest reports using http.
SUBMITURL=http://popcon.debian.org/cgi-bin/popcon.cgi
# USEHTTP enables http reporting. Set this to 'yes' to enable it.
USEHTTP="no"
# MY_HOSTID is a secret number that the popularity-contest receiver
# uses to keep track of your submissions. Whenever you send in a
# new entry, it overwrites the last one that had the same HOSTID.

38
popcon-submit-ubuntu.cgi

@ -0,0 +1,38 @@
#!/usr/bin/python
import os, sys, cgi, errno
import cgitb; cgitb.enable()
uploadDir = '/srv/popcon.ubuntu.com/popcon-data/'
logDir = '/srv/popcon.ubuntu.com/logs/'
def mkdirs(newdir,mode=0777):
try: os.makedirs(newdir,mode)
except OSError, err:
if err.errno != errno.EEXIST or not os.path.isdir(newdir):
raise
formStorage = cgi.FieldStorage()
fileitem = formStorage["popcondata"]
if fileitem.file:
header = fileitem.file.readline()
try:
id = header.split()[2].split(":")[1]
hash = id[:2]
hashDir = uploadDir + hash + '/'
filename = hashDir + id
mode = 'w'
except IndexError:
filename = logDir + "panic-popcon-submit-log"
mode = 'a'
mkdirs(hashDir,0755)
data = file(filename,mode)
data.writelines(header)
data.writelines(fileitem.file)
data.close()
print """Content-Type: text/plain
Thanks!
"""

29
popcon-submit.cgi

@ -0,0 +1,29 @@
#!/usr/bin/perl -wT
use strict;
my $email='test@popcon.debian.org';
$ENV{PATH}="";
print "Content-Type: text/plain\n\n";
if ($ENV{REQUEST_METHOD} ne "POST")
{
print "Debian Popularity-Contest HTTP-POST submission URL\n";
exit 0;
}
open POPCON, "|/usr/lib/sendmail -oi $email" or die "sendmail";
open GZIP, '/bin/gzip -dc|' or die "gzip";
close STDIN;
open STDIN, "<&GZIP";
print POPCON <<"EOF";
To: $email
Subject: popularity-contest submission
EOF
print POPCON while(<GZIP>);
close POPCON;
print "Thanks for your submission to Debian Popularity-Contest\n";
exit 0;

58
popcon-upload

@ -0,0 +1,58 @@
#!/usr/bin/perl -w
# Written by Bill Allombert for the Debian popularity-contest project.
# This file is placed in the public domain.
use strict;
use IO::Socket;
use Getopt::Std;
getopt("uf", \%opts);
my ($submiturl) = $opts{'u'} || "http://popcon.debian.org/cgi-bin/popcon.cgi";
my ($file) = $opts{'f'} || "-";
($host) = $submiturl =~ m%http://([^/]+)%;
# Configure the proxy:
my ($http_proxy,$proxy,$port,$remote);
$http_proxy=$ENV{'http_proxy'};
if (defined($http_proxy))
{
$http_proxy =~ m{http://([^:]*)(?::([0-9]+))?}
or die ("unrecognized http_proxy");
$proxy=$1; $port=$2;
}
$proxy=$host unless (defined($proxy));
$port=80 unless (defined($port));
# Compress the report:
my ($str,$len);
open GZIP, "gzip -c $file |" or die "gzip -c $file";
$str .= $_ while(<GZIP>);
close(GZIP);
$len = length($str);
# Connect to server
$remote = IO::Socket::INET->new(Proto => "tcp", PeerAddr => $proxy,
PeerPort => $port);
unless ($remote) { die "cannot connect to $proxy:$port" }
#Send data
print $remote <<"EOF";
POST $submiturl HTTP/1.1
Host: $host
Content-Type: text/plain; charset=utf-8
Content-Encoding: x-gzip
Content-Length: $len
EOF
print $remote $str;
#Get answer
my($answer)="";
$answer.=$_ while(<$remote>);
close ($remote);
#Check answer
exit (($answer =~ m/DEBIAN POPCON HTTP-POST OK/)?0:1);

48
popcon-upload-ubuntu

@ -0,0 +1,48 @@
#!/usr/bin/env python
import httplib, mimetypes, sys
def post_multipart(host, uri, fields, files):
content_type, body = encode_multipart_formdata(fields,files)
h = httplib.HTTPConnection(host)
h.putrequest('POST', uri)
h.putheader('content-type', content_type)
h.putheader('content-length', str(len(body)))
h.endheaders()
h.send(body)
response = h.getresponse()
return response.read()
def encode_multipart_formdata(fields, files):
BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
L = []
for (key, value) in fields:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s"' % key)
L.append('')
L.append(value)
for (key, filename, value) in files:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename))
L.append('Content-Type: %s' % get_content_type(filename))
L.append('')
L.append(value)
L.append('--' + BOUNDARY + '--')
L.append('')
body = CRLF.join(L)
content_type= 'multipart/form-data; boundary=%s' % BOUNDARY
return content_type, body
def get_content_type(filename):
return mimetypes.guess_type(filename)[0] or 'application/octet-stream'
upfile = 'popcon-data'
uploadname = 'popcondata'
val = sys.stdin.read()
data = ((uploadname,upfile,val),)
ret= post_multipart('popcon.ubuntulinux.org', '/popcon-submit.cgi', '' , data)
if not ret == 'Thanks!\n':
sys.stderr.write("%s\n" % ret)
sys.exit(1)
Loading…
Cancel
Save