Kris's blog

Posted Mo 09 März 2020

My publishing pipeline

Generally I wanted to have low resource consuming blog. The choice was to server static files. Serving static files is the king in serving any webpage. You have rendered final content in an HTML file. No code has to be run. No memcached or similar stuff. You enter URL address, server handles to you a file. Only bottle neck is reading from hard drive, but we today have SSD drives. Using Cloudflare such page will be fine.

I didn't even thought about a databases when I was building this project. Wordpress was no go. I wanted to go cheap as possible. As easy as possible. Static site generators like Pelican (used for this site) take content in markdown format, then render it into HTML files with styles and scripts attached. Product of a template, mardown-html converter and some config files. Rerendering has to be done each time you make a change (you need to change the modification date), but that's it. CMS using files, nano, Pelican, and git.

So the first workflow is like this: 1. write note in markdown with special header, inside folder content . 2. run make publish to generate HTML files inside output folder. 3. commit all changes 4. push stuff to Github 5. pull stuff from Github by cron script on server

I like this solution cause I don't have to do much. GNU make is great CI/CD local pipeline - all I need to do is run make publish . Steps 2, 3 and 4 are done by make command. Step 5 is automated using cron on the server.

Using Github like this is awesome automation. I publish online stuff that will be published online - so no security problem. I don't have to connect to the server - server downloads code from public space. Anonymously and automatically. No SSH keys or something, no users and passwords. Just a cron script that does git pull every hour.

What's more is that I can step 1 brake into smaller pieces - generating special header can be automated. I'm building something I call publishing_boy which is a tool that processes a folder full of mardown files by adding description headers for pelican . I even want to add some text processing tools into the chain like grammar checking or a style transfer. This will be cool :)

Category: blog coding projects organizing