Keeping website sources safe

Last Friday I was thinking about making a backup of the nikola sources for my website. I have more than 12 years of blog posts in the archive now. While most of the contents are not worth much, there are some pieces I'd rather not lose.

Using normal backup procedures seemed boring, so I started thinking of other ways. Realising that most of the files were just basic text files some kind of version control system seemed appropriate. Since I had no intention of running my own VCS server Github seemed like a cool option, especially since it has private repositories.

So every time I update the site, the deploy step not only updates (using rsync) the webserver, it also calls on git to push the changes towards my private Github repository.

So for this post the procedure looked something like this (Edited for brevity and a little bit of obfuscation):

yyyyyyy@delphic:~/website/src/www.ramdyne.nl/src$ nikola new_post
Creating New Post
-----------------
Enter title: Keeping website sources safe
Scanning posts.....done!
Your post's text is at:  posts/keeping-website-sources-safe.rst
yyyyyyy@delphic:~/website/src/www.ramdyne.nl/src$ vi posts/keeping-website-sources-safe.rst
yyyyyyy@delphic:~/website/src/www.ramdyne.nl/src$ nikola build
Scanning posts.....done!
.  render_site:../output/categories/index.html
.  render_posts:cache/posts/keeping-website-sources-safe.html
.  render_indexes:../output/index.html
.  render_rss:../output/rss.xml
.  render_pages:../output/posts/keeping-website-sources-safe.html
.  render_tags:../output/categories/stack.xml
.  sitemap:../output/sitemap.xml
yyyyyyy@delphic:~/website/src/www.ramdyne.nl/src$ nikola deploy
Scanning posts.....done!
==> rsync -rav ../output/*
ramdyne@xxxxxx.org:/home/zzzzzz/www/ramdyne.nl/www
Password:
sending incremental file list
index-20.html
index.html
rss.xml
sitemap.xml
posts/
posts/keeping-website-sources-safe.html

sent 526893 bytes  received 85809 bytes  136156.00 bytes/sec
total size is 18639132  speedup is 30.42
==> /home/yyyyyyy/website/src/www.ramdyne.nl/git-push-website-to-master.sh
[master fbadf78] Another commit by nikola deploy
 448 files changed, 2205 insertions(+), 998 deletions(-)
 create mode 100644 output/.htaccess
 rewrite output/assets/js/tag_cloud_data.json (78%)
 create mode 100644 output/categories/git.html
 create mode 100644 output/categories/git.xml
 create mode 100644 output/categories/github.html
 create mode 100644 output/categories/github.xml
 rewrite output/categories/website.xml (80%)
 create mode 100644 output/posts/keeping-website-sources-safe.html
 create mode 100644
 src/cache/posts/keeping-website-sources-safe.html
 create mode 100644 src/posts/keeping-website-sources-safe.rst
Username for 'https://github.com': xxxxx
Password for 'https://xxxxx@github.com':
Counting objects: 890, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (459/459), done.
Writing objects: 100% (459/459), 129.89 KiB, done.
Total 459 (delta 444), reused 0 (delta 0)
To https://github.com/ramdyne/websites-ramdyne.nl.git
 3c519ad..fbadf85  master -> master
Successful deployment
yyyyyyyy@delphic:~/website/src/www.ramdyne.nl/src$

The eagle eyed viewers will have noticed a small script being used to do all the git steps:

yyyyyyyy@delphic:~/website/src/www.ramdyne.nl/cat git-push-website-to-master.sh
#!/bin/sh
cd /home/yyyyyyy/website/src/www.ramdyne.nl/
/usr/bin/git add '*'
/usr/bin/git commit -m "Another commit by nikola deploy"
/usr/bin/git push origin master

The script (and the cd step in the script) are needed because of the way the nikola deploy steps work. You can't do cool stuff like cd in the deploy steps themselves, if I understood this post correctly.

Now I only need to find a way to not have a boring fixed commit message for every change ("Another commit by nikola deploy"), preferably it would say what had been last updated or something like that. Or just ask me for a commit message.

The folder structure I am using now for my website sources is a little different from the default nikola setup, so can always stop not keeping all changes to the nikola output. Keeping sources and output in git is kind of redundant, isn't it.

Nikola default (with site in ~/Documents/site:

~/
|- Documents/
    |-site/
        |-cache/
        |-files/
        |-galleries/
        |-listings/
        |-output/
        |-posts/
        |-stories/
        |-themes/
        |-conf.py

My current directory structure

~/
|- Documents/
    |-site/
        |-output/
        |-src/
            |-cache/
            |-files/
            |-galleries/
            |-listings/
            |-posts/
            |-stories/
            |-themes/
            |-conf.py
        |-git-push-website-to-master.sh