Deploying Pelican Blog with Bitbucket Commit Hooks

Nginx

Nginx is used to serve our static files and also proxy to the uWSGI git service.

First, I disabled the default site:

rm /etc/nginx/sites-enabled/default

Next, I create a conf for my blog site:

vim /etc/nginx/conf.d/${fqdn}.conf

server {
     listen   80;
     root /var/www/${fqdn}/output;
     index index.html index.htm;
     server_name \*.${fqdn};
     location /api {
         include uwsgi_params;
         uwsgi_pass unix:///var/run/uwsgi/app/${fqdn}/socket;
         uwsgi_param UWSGI_SETENV PELICAN=/usr/local/bin/pelican;
     }

     location / {
             try_files $uri $uri/ /index.html;
     }
 }

This particular configuraion proxies to all requests made to the /api route to the backend server. However, you could modify the proxy to suit any number of configurations.

You may notice I pass a environment variable to uwsgi. The uWSGI process runs as the www-data user. In my testing, the user was unable to find the pelican binary when my server spawned a 'make regenerate'. To solve that problem, I set the PELICAN environment variable and changed the pelican Makefile to conditionally set its PELICAN var. You can do this by executing:

sed -i.bak 's|PELICAN=|PELICAN?=|g' Makefile

On the Makefile generated by pelican-quickstart.

virtualenv

I created a virtual environment with the necessary modules for my uwsgi server. It provides a nice segration between my system and application libraries. These config files assume the virtualenv is located at /var/www/$fqdn/env. Update them if you choose to drop it somewhere else.

virtualenv /var/www/$fqdn/env

uWSGI

Nginx needs something to proxy too and I used uwsgi. Gunicorn or any other wsgi compatible backend is perfectly acceptable. I created a file /etc/uwsgi/sites-available/$fqdn.ini with these contents

[uwsgi]
base = /var/www/$fqdn
gitpost = %(base)/utils/gitpost
module = gitpost
callable = app
app = app
# Generic Config
plugins = http,python
home = %(base)/env
pythonpath = %(base)/env
chdir = %(gitpost)

You must then link the file to the sites-enabled directory

ln -s /etc/uwsgi/sites-available/$fqdn.ini /etc/uwsgi/sites-enabled/

Bitbucket

Bitbucket is where the magic happens. Ok, it's not magic. It's just a POST hook on commits. But it's what sets everything in motion. To enable that feature you go to http://bitbucket.org/$username/$blog-reponame, then type 'r' and 'a' in quick succession (or click the little sprocket to the right). Next, in the left menu click "Services". You'll have a dropdown, select the "POST" service. Finally, enter your blog's fqdn and api url. Given a fqdn of blog.jameskyle.org, the examples above woudl produce an api endpoint of:

http://blog.jameskyle.org/api

My blog repo is public. However, if you wish to keep your raw source private you'll need to create a public/private keypair for checking out the blog. The private key should be stored in the www-data user's home directory. On Ubuntu systems, the key is stored in:

/var/www/.ssh/id_rsa

Or id_dsa, whichever.

gitpost

Of course, we need to have something listening on the other side of the POST hook or it's all for naught. I wrote a little server using flask to satisfy that function. Configurable variables are set at the top of the file. Specifically, you'll want to set the DOCROOT variable. In this case, the DOCROOT is the location of your blog repository.

My server also logs activity. By default, it does so to:

/var/log/gitpost/gitpost.log

This directory should be owned by the www-data user.

mkdir /var/log/gitpost
chown www-data:www-data -R /var/log/gitpost

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

import os
import json
import logging

from flask import Flask
from flask import request
from subprocess import Popen
from subprocess import PIPE
from werkzeug.contrib.fixers import ProxyFix

DOCROOT  = "/var/www/$fqdn"
LOGLEVEL = logging.INFO
LOG_FORMAT = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
DATE_FORMAT = "%Y-%m-%d,%H:%M:%S"
LOGFILE = "/var/log/gitpost/gitpost.log"

app = Flask(__name__)
app.wsgi_app = ProxyFix(app.wsgi_app)
app.logger.setLevel(LOGLEVEL)

formatter = logging.Formatter(fmt=LOG_FORMAT, datefmt=DATE_FORMAT)
fh = logging.FileHandler(LOGFILE)
fh.setFormatter(formatter)

app.logger.addHandler(fh)

app.logger.info("Starting gitpost api application")

def log_error(command, output, error, returncode):
    msg = """
    Command: {cmd} => {ret}
    --------------

    Output:
        {out}

    Error:
        {err}
    """.format(cmd=command, out=output, err=error, ret=returncode)
    app.logger.error(msg)

def pull():
    app.logger.info("Pulling changes from master")
    popen(["git", "pull", "origin", "master"])

def popen(cmd):
    app.logger.debug("Changing into {0}".format(DOCROOT))
    app.logger.debug("Executing: {0}".format(" ".join(cmd)))

    os.chdir(DOCROOT)
    p = Popen(cmd, stdout=PIPE, stderr=PIPE)
    p.wait()
    out, err = p.communicate()
    if p.returncode != 0:
        log_error(cmd, out, err, p.returncode)

def publish():
    app.logger.info("Publishing blog updates")
    popen(["make", "publish"])

def update():
    app.logger.info("Updating blog")
    pull()
    publish()

@app.route("/api", methods=["POST"])
def parse_request():
    app.logger.info("Parsing request")
    payload = json.loads(request.form["payload"])
    branch = payload["commits"][0]['branch']
    app.logger.debug("received commit from branch: {0!r}".format(branch))

    if branch == "master":
        app.logger.debug("Initiating master repo sync")
        pull()
        publish()

    app.logger.info("Finished publishing")
    return "Received:: {0}".format(request.form)

if __name__ == "__main__":
    app.run()

The uWSGI configuration files provided assume the gitpost script is located in:

/var/www/$fqdn/utils

And that the created virtualenv is located in:

/var/www/$fqdn/env