The Sun and The Air

Code Crimes - A Bear to 11ty converter

I mentioned previously that I was considering the shift to 11ty, and I said I'd try it out a bit further before making a decision. Part of that would be porting over my Bear content to the new platform, and this post is about that process.

The Wrong Tool

One thing to know about me is that I don't like to learn new tools to solve problems that an existing tool could solve. Kind of like the inverse of how every tool's a hammer, I rarely encounter a programming problem that I can't solve with R.

R is a statistical programming language that, over its decades of use and adaptation by users, doesn't really have a single core design philosophy. Pretty much everything is vectorised, but other than that it's a bit of a free-for-all due to a truly extensive backlog of installable packages.

I found myself attempting to get into a Python or JS kind of mood, but I'd been using R a bunch at work this week so decided to go back to old faithful. This is definitely not the correct tool for the job (more on that later) but it took me about 20 minutes to get a working prototype together (ok, maybe an hour to hammer out some bugs), a timeframe that'd be impossible with more suitable tools that I don't know as well as R.

The Prototype

This is the whole thing:

library(tidyverse)

export <- read_csv('dat/post_export.csv') %>% 
  filter(publish) %>% 
  filter(!`is page`)

export_post <- function(post, out_dir="out"){
  
  clean_content <- post$content %>% 
    gsub("((\r\n)?___)?(\r\n)?### Comments.+$", "", .) %>% 
    gsub("\r\n", "\n", .)
  clean_tags <- post$`all tags` %>% 
    gsub('\\[|\\"|\\]', '', .) %>% 
    str_split(', ') %>% 
    el(1)
  clean_date <- as.Date(post$`published date`)
  clean_title <- post$title %>% 
    gsub(':', ' -', .)
  clean_meta_image <- coalesce(post$`meta image`, "")
  clean_meta_description <- coalesce(post$`meta description`) %>% 
    gsub(':', ' -', .)
  
  filename <- clean_title %>% 
    gsub("\\/", "-slash-", .)
  
  formatted_tags <- if(!all(clean_tags=="")){
    paste0('  - ', clean_tags, collapse='\n')
  } else {
    ""
  }
  
  md_blob <- paste0(
    "---\n",
    "title: ", clean_title, "\n",
    "date: ", clean_date, "\n",
    "tags:\n", formatted_tags, "\n",
    "meta_description: ", clean_meta_description, "\n",
    "meta_image: ", clean_meta_image, "\n",
    "---\n",
    clean_content
  )
  
  write(md_blob, file=glue::glue('{out_dir}/{filename}.md'))
}

map(1:nrow(export), ~{
  post = export[.x,]
  print(post$title)
  export_post(post, out="/home/crim/web/strawberry-starter-main/src/posts")
})

Yeah I used tidyverse just for basic filtering and pipes, that's kind of the vibe here.

Bear helpfully provides a csv export of all of my pages and posts, with the metadata in obviously labelled columns - I put mine in dat/post_export.csv.

I wanted to be able to automate adding the correct dates and tags, while maintaining compatibility with Obsidian - both Obsidian and 11ty are markdown-based, so this was very straightforward. I make the metadata blob between "---" lines and paste on the main content.

The only real problem due to using the Wrong Tool is that at several stages I need to escape or replace "functional" characters like brackets, slashes, and quotes. This is an extremely common problem in text processing, but rather than look up a solution I just subbed them and called it a day.

11ty demo

I also removed the comments section which was more of a pain than expected, but the result is a functioning version of my site, with a few issues.

A few issues

The biggest one is the URLs/internal links. If I've linked another of my posts (like in the first line of this post) I may have used the full URL (https://site.com/post-link-format) or the bear shorthand (/post-link-format I think? I never got fluent with this, drafting posts offline). In both cases though, the post title has been converted into a slug.

This means that to maintain compatibility with old posts, I can either -

I also need to sort images - I wouldn't want a new blog to reference the Bear image hosting. The Bear export doesn't include images, and the media interface doesn't include an export option. I can probably automate a scraper pretty easily to grab them all, the media page is very simply structured - it is a pain though.

However that is done, I could then just do the same mapping process as webpage links to get them pointing at my 11ty assets directory I assume. Simple enough once it's solved once.

The last major thing is just those special characters - fixing each one by hand is stupid, and there's probably a better way to export them to markdown.

A trickier aspect of this problem is that 11ty (or the Strawberry Starter pack that I used to get it off the ground - highly recommended) doesn't seem to support having those characters in the Titles - at least in the filenames which function as titles in Obsidian. That might be specific to the Strawberry Neocities uploader though - I patched in a workaround before tracking down exactly where it was breaking.

I don't have a clear solution for that last one - might just have to nut up and accept the limitations of the format.

Conclusion in progress

It was honestly pretty painless to get this going - I went from nothing to "some of my posts are now accessible and correctly formatted" in about half an hour. Strawberry Starter really takes all the hard stuff out, and gives a nice structure to work from to do more complicated stuff.

I don't know if it's how I'd implement my own site, but it let me test the important stuff incredibly quickly.

I'm still not totally sold on changing to 11ty - it's definitely less painful than I feared, but Bear honestly rules??? It's easy to work with, and while I think posting to an 11ty site would be easier for my workflow, I'd need it to offer a little more before switching. I'll have a look at some more of the demo sites to see if they inspire more enthusiasm.

The experience has been fun though, and it'll be good to have this in the back pocket if something goes horribly wrong with Bear - never bad to have an exit strategy.

✉️ Reply via email

Comments