Off the Rails
I’ve undone it. Three years ago I wrote about the need to run a Ruby on Rails site on a larger server. It was a cheap and easy thing to do at the time, but it bugged me a bit because it meant I wasn’t running as lean-and-mean a setup as I could have. So what I set out to do, over time, was re-architect the system so that it was more frugal in using the limited resources that it actually needed.
The frontend for calendaRSS used to be an iOS app, which interacted with the Rails backend via a web API. It did all the standard stuff: manage user accounts and devices, track their subscriptions, and regularly update the published calendars from the source RSS feeds. But Rails has become pretty bloated when it comes to what resources it demands in order to do just that basic stuff and, in truth, the workflow actually isn’t a good fit for the underlying task that needed to be done.
For example, I didn’t really need user accounts on the server. In reality, all the server actually needed is a way to track an individual’s feeds, which can just as easily be accomplished by having an updated list sent in full from the iOS device whenever it changed locally. Even for a dozen URLs, it’s going to be less than 1k of data; a negligible amount in today’s world.
And that immediately leads to another thing that didn’t need to be stuck on the web stack: fetching the given RSS feeds and processing them into iCalendar files. It struck me as obvious that it’d be a good idea to move those methods out of the Rails app itself and into an “on demand” process that could work independently (and, in particular, be easily called from a cron job for regular updates without, again, all that unnecessary Rails overhead).
So as I started picking those pieces out of the Rails app, it quickly became apparent how the entire API was collapsing into something that didn’t need Rails. Fundamentally, everything I really needed on the web side of things could be accomplished by microservices. The needs of the web app break down into 3 distinct services: the HTML web site that people can view with a browser, the new “account-less” API that allows users to update feeds, and the legacy API that allows older software to access the account-based data (and ideally transition to the new API).
It was a very short search to find a good solution that allowed me to leverage my existing Ruby code: Sinatra. I’ve used Sinatra before for less complex web services, but this was going to be the first time I used it to reimplement a Rails app. And it only made sense for me in this case because I was explicitly going with an approach that eliminated the use of heavy-weight ORM libraries like ActiveRecord. This, of course, is not a path that is always going to make sense for every Rails-based app.
But something needs to function as a persistence/communication/coordination layer, because the system still has long-lived data (the feed URLs) that need to get from the web stack to the decoupled code that fetches and updates them. The answer I went with for that is lightweight middleware in the form of message queues. In particular, I used MQTT because it has demonstrated great utility in some the work I’ve done with Raspberry Pi devices. Retained messages act as a rudimentary datastore, and that’s all I really need for calendaRSS.
And that’s the system that’s been up and running since the beginning of August for calendaRSS. A very thin web layer does little more than wrap messages to the broker, which other very thin client code monitors and respond to in order to get things done. As an added bonus, the use of message queues means that I can employ just about any other computer to do that work. If I ever feel constrained by my tiny VPS again, I can simply set up just the decoupled code to run on another server, or even a desktop computer, and possibly even opening up the opportunity to bring in mobile devices to crowd source computation. For now, I’m really happy with the results, and I’m looking forward to applying this same design pattern on some other web projects I have in mind.