I can has web site?
After thinking about doing it for a while, I finally went ahead and created a landing page for myself, along with a self-hosted copy of my DEV.to blog.
There are a number of options for doing this. Recently Stackbit has created a service that allows DEV.to users to automatically generate a copy of their blog:
I tried this service out, and it’s pretty cool! You can generate a blog hosted on Netlify with just a few clicks.
While I did model my list of articles on one of Stackbit’s themes, ultimately I decided to set up my own site. With generated sites, the CSS they use for layout can get a bit messy, which makes changing it more difficult. I ended up writing the CSS for my site from scratch. Also, the idea of creating something which didn’t have dependencies on another service appealed to me.
Python Scripts
The first step was to download the contents of my DEV.to blog. To this end, I wrote a few Python scripts: download_articles.py uses DEV.to’s REST api to download the markdown for my published articles; download_images.py then downloads all of the images used in these articles; copy_and_transform.py creates a copy of the original content, using regular expressions to apply some transformations to the markdown. There’s a master script, main.py which runs all of the above scripts. If you’re interested in taking a look, you can find a copy of this code on GitHub:
scripts to manage dev.to blog articles
Scripts that download and transform dev.to blog articles.
The main.py script runs three subordinate scripts, download_articles.py, download_images.py, and copy_and_transform.py. The following will run the main script, which downloads articles, images, and then makes copies which transform the original markdown content:
python main.py <username> --root <root dir> --download_dir <download dir> --transformed_dir <transform dir> --article <article name>
Options:
username refers to the dev.to user whose articles will be downloadedroot dir determines which base path the files are downloaded to - defaults to the current directory.download dir is the directory to which the markdown files and image files will be downloaded - defaults to downloaded_filestransform dir will contain a copy of the contents of download dir with changes applied to the markdown (localizes links and fixes some markdown to work with jekyll) - defaults to transformed_files- Once this script has been run for all articles, the
--article option can be used to download and transform additional articles one at a time. article name refers to the name of the article in the dev.to url - a wildcard match to this parameter is applied.
Example:
python main.py ben --root ~/downloaded_articles
When download_articles.py runs, it generates an articles_dict.json file. This file stores key-value pair mappings of article names to article titles. This is used by the copy_and_transform.py script to produce local links when an article links another article by the current author, as specified by username.
By default, these scripts download the markdown files to the _posts subdirectory, and the images to the assets/images subdirectory, to match the structure expected by the jekyll static site generator. This can be modified in common.py.
To delete an article, run:
python delete_matching.py <article name> --root <root dir>
This will delete matching markdown files and image directories, and will also remove the mapping for that article from articles_dict.json.
I wrote this code for my own purposes, so I can’t guarantee that it will work for everyone else. I did run the scripts against @ben’s posts and confirmed that they don’t crash.
Jekyll Static Site Generator
Next, I set up Jekyll to generate the HTML from these markdown files. I considered other generators like Hugo or Gatsby, but since DEV.to and Jekyll both use liquid tags and have similar formatting for front matter, Jekyll seemed like the most natural choice.
I set up the Python scripts to produce a Jekyll-compatible directory structure, with articles going into the _posts folder, and images going into assets/images. The output from the scripts is then copied to the corresponding folders in the Jekyll project:
Jekyll project for my professional landing page and blog.
To install (Ruby must already be installed):
To build:
The generated site will be in the _site folder.
To serve a local copy:
Jekyll has been quite helpful for several things: I’m using DEV.to’s support for a series of articles in a few places, and I found a bit of Jekyll template code to handle this.
I am also using the following plugins:
I’ve written a custom plugin as well, github_readme_tag.rb to support embedding a preview of GitHub projects.
Syntax highlighting has been a very useful aspect of using Jekyll. I simply downloaded the appropriate CSS theme for monokai, which tends to be my go-to theme, and voilà, I had a nice-looking display for code examples.
Design
One thing I found to be important was to design the appearance and layout separately from processing the content. I saved a couple of articles to a separate folder, and used that to create the CSS and HTML for the main components of my site: The landing page, the list of articles, and each individual article. This allowed me to focus on getting things to look the way I wanted them to. Once I felt this part was ready, I incorporated it into the Jekyll project.
Supporting comments seems to be a rather hairy area. I found a neat little project called utterances which I’ve incorporated into my article template. Reader comments are posted as issues to a dedicated GitHub project - blog_comments in my case. It does require commenters to have a GitHub account, but I like the simplicity and transparency of this solution.
Results
For the time being, I’ve deployed my small site to GitHub pages:
I used Google Domains for the custom domain registration. Even though I’m not a designer, I’m pretty happy with the results, and it feels good to have a self-hosted version of my blog, as well as a central spot for my online presence.
Writing some of these articles has been a lot of work, and it’s been gnawing at the back of my mind that if something catastrophic were to happen to DEV.to, I would lose a huge amount of work! I know, realistically, it’s not going to happen, but it still gives me some peace of mind knowing that I’ve got a back up.
As a matter of principle, it’s probably also worthwhile to host one’s own blog. If you decide to do this, don’t forget to update the canonical_url in the DEV.to front matter to point to your version of the same article. This applies for any other places where you may host copies of the same articles (see canonical link element for reference).
