Using the select syscall from Rust, with the libc crate

Disclaimer

  1. I am not a Rust expert, I am just getting started, so please take everything you read here with a grain of salt.
  2. You probably don’t want to use any of this in production code. There are libraries written by actual Rust developers providing similar features, in a way that will most certainly be more efficient and more reusable, such as Tokio and nix. Additionally, select is rarely use these days, kqueue and epoll tend to be preferred.

Good? Let’s get to it now.

The select syscall

The select syscall is really useful to write systems using an event loop. A good example is a database running as a TCP server. A database would want to maintain client connections open, and respond to them when then send queries to the server, and select can help with that. …


What does the “ampersand, symbol” thingy as an argument to a ruby method actually do? It creates a proc.

What’s with this weird looking syntax with an ampersand and a symbol

When &:object_id is used as an argument to a method call, it will convert the symbol :object_id into a Proc instance with the Symbol#to_proc method.

This syntax is really useful when mixed with Enumerable methods such as map and select:

irb(main):001:0> [ 1, 2, 3, 4, 5 ].select(&:even?)
=> [2, 4]
irb(main):002:0> [ 1, 2, 3, 4, 5 ].map(&:even?)
=> [false, true, false, true, false]

The result of :even?.to_proc is a Proc instance, very similar to proc { |x| x.even? } with one major difference. They handle arguments differently. …


In this chapter we will write our own hash table. This will allow to remove uses of the Ruby Hash class and use our Dict class, built from scratch.

This article is part of a series, Redis in Ruby, originally published at https://redis.pjam.me/. All chapters are available on Medium as well: https://medium.com/@pierre_jambet

What we’ll cover

So far we’ve been using the Ruby Hash class as the main storage mechanism for the key/value pairs received through the SET command. We also use it for the secondary dictionary necessary to implement the TTL related options of the SET command. We store the expiration timestamp of keys with TTLs, which allows us to know whether a key is expired or not.

Redis is written in C, which does not provide a collection similar to Ruby’s Hash. …


In this chapter we will focus on making RedisServer speak the Redis Protocol, RESP. Doing so will allow us to use Redis’ built-in client, redis-cli to communicate with our own server.

This article is part of a series, Redis in Ruby, originally published at https://redis.pjam.me/. All chapters are available on Medium as well: https://medium.com/@pierre_jambet

What we'll cover

By the end of this chapter RedisServer will speak the Redis Protocol, RESP v2. Doing this will allow any clients that was written to communicate with the real Redis to also communicate with our own server, granted that the commands it uses are within the small subset of the ones we implemented.

One such client is the redis-cli utility that ships with Redis, it'll look like this:

Image for post
Image for post
Using redis-cli against our Redis clone

RESP v2 has been the protocol used by Redis since version 2.0, …


Four runners
Four runners
Photo by RUN 4 FFWPU on Pexels

As I’m writing this, I’m finishing my fifth week of running after taking a five-week break, so things are starting to feel like they’re back to normal on that front. On the other hand, all the problems that were there before, racial injustice, and COVID being top of mind, are still here, and it’s really hard to gauge if progress is being made. But at least there is momentum to make things better, a lot of people are speaking up, on both ends, and that might be the silver lining.

On the protesting front, there are things we can do as runners. I’ve had the chance to participate in two runs for social change in the past few weeks. And let me be extremely clear here, I am in no way saying that by signing up for a run promoting civil rights you did your job, that you should pat yourself on the back and resume your daily life. There is a lot more that can, and probably should be done, such as donating, educating, engaging, debating, learning, and probably more. I know I’m working on it, and I don’t think I’m doing as much as I could. But I’m only hinting at the fact that there are ways, that we, as runners, can introduce ourselves to social change, if that’s not something we were used to before, through running. …


Image for post
Image for post
Photo by hannah joshua on Unsplash

This article is part of a series, Redis in Ruby, originally published at https://redis.pjam.me/. All chapters are available on Medium as well: https://medium.com/@pierre_jambet

What we’ll cover

We implemented a simplified version of the SET command in Chapter 2, in this chapter we will complete the command by implementing all its options. Note that we're still not following the Redis Protocol, we will address that in the next chapter.

Planning our changes

The commands accepts the following options:

  • EX seconds — Set the specified expire time, in seconds.
  • PX milliseconds — Set the specified expire time, in milliseconds.
  • NX — Only set the key if it does not already exist. …


A followup to the previous post about Railway Oriented Programming in Scala, adding support for parallel validations

Image for post
Image for post
Multiple parallel train tracks

Combining functions in parallel

In the previous post we attempted to translate the main F# constructs from the original Railway Oriented Programming (ROP) documents — the blog posts & the slide deck — to Scala. We also showed what an alternative approach looks like using Scala’s built-in class. The code is on GitHub. That was a long article, I want to keep this one short & sweet.

In this article, we will focus on the “Combining functions in parallel” piece.

The word “parallel” in this context can be confusing because it is conceptually different from “parallelism” used to describe tasks happening at the same time. …


Image for post
Image for post
Photo by Chander R on Unsplash

Good morning runners!

My last race was on March 1st, a little bit over four months ago now, and I’m starting to seriously miss it. There’s something special about the atmosphere in the corral, as you wait for the start of the race, shoulder to shoulder with a bunch of sweaty, smelly runners. The way NYRR staff greets you over the mic with their “Goooood morning runners!” — I haven’t really ventured much outside of NYRR races in the last few years — the national anthem, the whole thing! And then sure, the racing part is fun too, but the pre-race part, I don’t know, it’s a weird mix of excitement and anxiety that I really miss. …


Image for post
Image for post
Train tracks, including a switch

Intro

A few years ago, a coworker introduced me to Railway Oriented Programming (ROP). At the time we were using Ruby, and while the ideas in ROP made a ton of sense, I didn’t really find a way to apply them to what I was working on. The lack of types made it pretty hard to go beyond “I just read a blog post and I’m gonna pollute our codebase with it, because I can” and actually improve things. We all pretty much moved on.

Later on, I switched to a different project, using Scala, and ended up using the Either type a lot, to organize what we described at the time as a "pipeline of operations", or maybe, something that one could describe as a … "railway"?! …


This article is part of a series, Redis in Ruby, originally published at https://redis.pjam.me/. All chapters are available on Medium as well: https://medium.com/@pierre_jambet

What we’ll cover

In this chapter we will add support for efficient handling of multiple clients connected simultaneously. We will first isolate the problematic elements of the current implementation and explore different solutions before getting to the final one using the syscall.

First problem, accepting multiple clients

Let’s start with the new client problem. Our goal is the following:

Regardless of the state of the server, or what it might be doing, or whether other clients are already connected, new clients should be able to establish a new connection, and keep the connection open as long as they wish, until they either disconnect on purpose or a network issue occurs. …

About

pierre jambet

Software Engineer by day, Runner by day as well.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store