The code can be found on GitHub.
At first glance this small CLI might not seem like it does much, and it really doesn’t, but while building it I ended learning quite a few things, three to be exact:
- How to use an external command line tool, such as
fzf, and feed it input, similar to using a unix pipe, like
echo "a\nb\nc" | fzf, as well as reading the output of the command, but programmatically, in Ruby.
- How to implement a flow similar to what happens when you type
git commitwithout the
-m/--messageoption and it prompts you with an editor,
- How to use the Gitub API to create a new commit, without using the
Before jumping in, here’s a summary of what
til actually does:
- It first loads the list of all existing categories in your TIL repo, and then uses
fzfto prompt you to pick the category for your new TIL. You can also choose to add a new category.
- Once you picked a category, it uses your default editor, as configured through the
$EDITORenvironment variables, or
viif none of those are defined. You can then type what you actually learned today.
- After saving and closing the text editor,
tilwill grab the content of the file, and commit it to the configured GitHub repo
- It also takes care of maintaining the
README.mdfile so that it contains a nicely organized index of all your TILs. It keeps a list of all the categories at the top, and includes a link to each TIL below, grouped by category
I don’t actually use it that much, but I love how improves CLI interactions. I thought it would be a great addition for the workflow I wanted with
til. Most of the time I will be reusing existing categories, and as someone who makes a lot of typos, I always look for ways to avoid having to type anything.
The main way that
fzf gets its input in from STDIN. You can test that from the command line:
echo "a\nb\nc" | fzf will load
fzf with three items
It’s worth mentioning that I could have used the backtick approach, but that was a tiny bit too much magic for me and also, there are few blog posts out there that recommend against using it, and favor the
system method. In this case we're not really dealing with user input, so it doesn't matter that much from a security standpoint.
Process.spawn method accepts a bunch of options, there are a lot, but these are the ones that are interesting to us here:
:in : the file descriptor 0 which is the standard input
:out : the file descriptor 1 which is the standard output
So, we can create a with and pass the reader as the
:in argument to
spawn, so that we can write with the writer from the main process and the process on the other end, the one created by
spawn will receive it.
Note: it is important to close the
writer in the initial process, if you don't,
fzf still thinks that there might be more to read, and it shows it by displaying a spinner in the bottom left corner of the terminal.
We can use the same approach to get the content that
fzf will output. Once a selection is made,
fzf writes it to STDOUT. So we create another pipe, and give the writer as the
:out option, the process started by
spawn will be able to write to it and we can then use the other end of the pipe to read from it and get the selection from the user in the main process.
This is what
til does, as you can see on GitHub.
A quick look at the C code defining the backtick function shows that it uses the
pipe syscall, so we're essentially reimplementing something fairly similar to what ruby does for us with
`echo 'a' | fzf`.
Note (another one): As I was writing this post, I realized that Ruby has another method related to spawning new processes and dealing with STDIN and STDOUT: . I haven’t looked too much into it yet, but it looks like it could simplify my code a little bit. That being said, the overall approach described above is still valid.
Using an external editor
For years I used
git from the cli and relied on its commit workflow without really wondering how it actually made that happen. In case you're not familiar with it, from the CLI, when you commit with
git commit you essentially have two options, you either provide the commit message inline, through the
-m/--message option, or you leave this option blank and
git opens an editor for you, by default
What is actually really cool with this is that you don’t have to use
vim, you could use pretty much any other editors, that being said, you probably want to pick one that is quick to start so you don't have to wait just to type a commit message. It's a little bit trickier with editors using a dedicated window such as vscode or macvim. GitHub's documentation has a page explaing how you can use the most common visual editors as git editors.
So, as a git cli enthusiast, I wanted to replicate the same worflow: you picked a category, now let’s write the content, in the editor you like using, so you can format things the way you want.
Reimplementing the git commit workflow turned out to be very little work, you first create a file, Ruby makes that easy with the class, we then use either
spawn with the value in
$VISUAL (we default to
vi, just in case, so we have something).
The main difference between
system here is how they return,
system does not return until the process is over, whereas
spawn returns a pid. Since we basically want to wait for the user to close the editor,
system is easier, with
spawn we would have had to use
waitpid to wait.
Once the child process is done, we can read the content of the file, and VOILA! We have the content, formatted by the user, in their favorite editor (unless they ended up in
vi and couldn't figure out how to exit).
You can see that
til does exactly what I just described
Creating a commit using the GitHub API
And now, the final piece of the puzzle, we have a category and a string representing a new TIL, it’s time to create a new commit on GitHub. The GitHub API must have an easy way to do this right?
You can do it! But is it easy? I’ll let you answer on your own.
The new commit needs to contain two changes, the new file we want to create, but also, and this is really one of the reasons why I wanted to create this tool in the first place, the updates to the README file, to keep the table of content and the links up to the date with the new TIL.
This blog post was really helpful but the fact that it didn’t include any code examples means that I spent a few hours (😭) trying to get things workings, here’s a summary of what I’m doing to create a new commit, I hope you’re ready:
- Get the
refof the latest commit on master
- Get the commit object, with the sha obtained in the previous step
- Get the tree object, based on the latest commit obtained in the previous step
- Get the readme content
- Create a blob for the new file — Interesting to note that a blob does not have a path, we only specify the path when adding blobs to a tree
- Get all blobs that are in the current tree, except the README — This is necessary because we don’t want the new tree to remove any files, so our new tree needs to contain all the old files, plus the new file, plus the updated readme, and we definitely don’t want the old README
- Figure out what the content of the README should be — We add a new category at the top if necessary and then we add a link to the new TIL at the bottom, keeping the categories sorted alphabetically, and the entries sorted, oldest at the top, newest at the bottom. The code isU (pause) GLY, but it works!
- Create a new blob, for the updated readme
- Add the new blob (at the correct path) and the blob for the updated readme to the blobs list
- Create a new tree with all blobs, all the old one, unchanged plus the new file and the updated readme
- CREATE A NEW COMMIT, FINALLY
- You thought we were done? Nope, we have to update master to point to the new commit, and now, we’re done
You can find documentation about the API endpoints I’m using on the following pages:
See it in action
The current version (0.0.4) is very very basic, but it gets the job done, and I’ve been using it for a few days already. I have a few thoughts about what I would like to do next:
- A “real” cli, probably written in go, so that it’s easier to distribute, with
brewfor instance. There doesn't seem to be any formulae/casks named
til, so I should hurry up!
- A chrome/firefox extension, so you can do the same without leaving your browser
- Improve the code (if you’ve looked at it, it’s … far from great, really far)
- Show a terminal spinner at the end when creating the commit, since it can take up to a few seconds. I recently learned how to use terminal escape sequences to do this! Read more on my TIL repo (see what I did there?!). But there are also at least two gems that do that for you,here and there.
Questions? Comments? Hit me up on Twitter!
There are a few similar gems, both in names and features available on ruby gems, I checked all of them before publishing mine:
- til_cli, there were two issues with this one. First, it does not seem to work with latest ruby 2.7.1 because of json 1.8.1 not compiling. And also, it runs git commands locally, it basically wraps
gitcalls, which is great, but means that the tool is not really a standalone tool, I wanted something I could run from anywhere, and that would work on its own.
- todayilearned: This is an interesting gem, but it works locally with sqlite, and while this is great, this is not what I wanted
- til: This seems to the same code as the one above, they both point to this GH repo: https://github.com/ip2k/todayilearned
- til_: This looks like an empty gem, there’s no “real” code in there as of today.
Originally published at https://blog.pjam.me.