Singularity. It’s the oldest piece of software I’ve written that still gets active use today. For that same reason, it’s also the jankiest piece of software I still have to care about. I haven’t done much with it in years, and it shows. I was hoping to give it a dusting once I could upgrade to GTK4, but I’m still waiting on a certain library and I’m tired of relying on such a crusty program. It’s time for housekeeping.
Introduction to Singularity
With that said, some of you probably don’t know what Singularity is. I haven’t written about it since 2018, after all.
Singularity is an RSS reader that I built as a college freshman back in 2013, in response to Google Reader’s demise. Over the years it has gotten various features and the occasional rewrite, but the core concept remains the same: It’s an all-in-one app that runs on your PC, with no need to host any sort of server or rely on online services. You just pop it open and it handles everything, be it checking your subscriptions, tracking them in a database, or presenting them in the UI.
Over the years I’ve periodically looked at the competition, but I’ve found that most FOSS alternatives either rely heavily on online services, require a self-hosted server to function, or have difficulties handling some of the not-so-perfectly formed feeds lurking on the web. And so, the old beast lumbers on.
I haven’t fully decided what I want to do this time around, but having sworn off needless rewrites I think some refactoring and optimization would be a good start. I’ve learned a lot both from my professional work and my GTK4 testing from earlier in the year, and I think I can use that knowledge to make this software easier to maintain and improve. If you’d like to see it for yourself, check the project page for links to the code.
Cutting It Up
It has been a very long time since I sat down to work on this code. I vaguely recall highest-level structure though, I tried to organize the frontend and backend code into two separate projects as I usually do. The ‘app’ project has 15 code files and the ‘lib’ has… 43. There are no subfolders. Judging from the fact that our 58 code files are stored in only 2 folders, the first step is going to be working out what everything is for and sorting it out.
Thankfully, I don’t really care about what the ‘app’ is doing yet. It looks to mostly be UI code, so I’m much more interested in the next layer down. I’ve broken up that layer into a few categories:
Data Model (8 files)
For the first category, let’s start simple. We’ve got various classes representing objects like feeds, feed items, authors, attachments, etc. This is probably the most reasonable category.
Database Interaction (12 files)
Next, we have the code for interacting with the database. Singularity stores its data in a SQLite DB, and should probably hide that fact from the rest of the code but totally doesn’t.
Inside of this category we have the
DatabaseManager which processes requests and manages the schema. We also have the requests themselves, which uhhhh… build all of Singularity’s SQL queries with glued-together
printf statements and string concatenation. We should probably fix that.
Web Interaction (6 files)
On that note, we also have the other end of the pipeline: The code designed to actually get stuff off the internet. Honestly, this just calling into a library and it seems mostly OK. It could use better organization and smarter request handling to avoid hitting a server too frequently, but I’m not immediately concerned by this part.
Parser / Serializer Code (6 files)
Next, there’s the code for reading and writing feed data. These could use some refactoring, but like the web request code I’m not too worried about the actual functionality. I suspect this is one of the better-off parts of my codebase, since it’s the part that needs to work the most.
Things that really should be in the application layer (10 files)
And now, we arrive at the deep garbage. Remember how I mentioned that the ‘app’ layer is just UI code? That’s because the rest was tucked away in this layer. In here, we’ve got settings code, HTML/CSS generation for the web views, tree view code, and even web view interaction code… this definitely needs to be removed.
So there’s our 42 files organized. Now we just need to–wait there were 43 files. What’s the last one?
As a serious professional today, seeing a file in my codebase that’s simply named “IO” is really ominous. Inside is a collection of 9 similarly-ominous methods:
get_monthConverts a string containing either the name of a month or 3-letter abbreviation (but only in English) to an integer.
sql_str9 lines of code that allegedly escape a string for safe insertion into a SQL query.
strip_htm3 lines of code that allegedly escape special characters for display in HTML (It’s actually just < and > tho).
md5_guidGenerates the MD5 hash of a string and calls it a GUID.
clean_xmlAccording to the documentation comment, “Applies a few basic fixes to XML to increase the chances of successful parsing”. It does…something…to a string.
xml_to_plainAllegedly scrubs the markup tags out of XML, presumably just leaving behind the content?
resource_to_stringLoads a specified GLib resource file embedded in the binary as a string.
extract_imageScans an HTML string for an
<img>tag and returns the first image URL it finds.
html_to_genericScans an HTML string and replaces relative URLs with absolute ones.
Anyway, the punchline is only one of these methods does anything resembling I/O. I’m not really sure what to do with this menagerie yet, a lot of it seems like it should be closely scrutinized and unit-tested. That’s probably out of scope for this first stage, but I’ll figure something out later.
Speaking of Unit Tests
If you’ve looked at the code yourself, you might notice that I didn’t mention the unit tests. And if you actually looked at the unit test code, you will already know why! Almost all of my unit tests are unimplemented, and the few test cases I have don’t seem terribly useful. They’re tests like “does this constructor argument set this property”, which technically validate things but aren’t terribly useful. Hopefully we can get some of those in later, to validate our refactor. But first let’s make our codebase a little bit more sane.
Time to Work
Now that we’ve assessed the damage, it’s time to immediately yank out all of the code that should be in app and move it over there. Normally this would be a pretty scary task, but the end result was only 3 compiler errors. Those 3 errors stem from the application settings being directly referenced by some objects in our core. Funny enough, this is thanks to a refactor documented in a previous blog post:
“For a long time, I regarded globally-accessible data as a very bad thing. As a result, I avoided static functions and variables like the plague in most of my software. You don’t want to overuse these types of things, but it’s important to remember that there’s pretty much nothing in programming that should always be avoided.”
Old me has a point! There is nothing wrong with statics. But there’s a time and place for everything, and that place is definitely not here. I think we can all agree that settings and argument handling ought to live higher up the chain, regardless of your political leanings.
Looking at the code, we have 2 offenders:
should_update property is in our
Feed model, and I’m pretty sure it shouldn’t live there. Thankfully it’s checked once in the main application class and nowhere else, so we simply convert that into a one-liner and remove the property entirely. And as for #2…
Yes, it’s what you think it is. We’re calling a method to update the cookie db, and not passing the filepath in. So we do that, and as if by magic our compiler errors are gone! As it turns out, there really was no good reason for any of those 10 files to exist in our core at all.
And Now, Organization
We’re not quite done yet. I’d like to prevent this particular incident from happening again, so it’s time to introduce a little secret weapon: folders.
On the app side, I’ve put all of the actual UI widgets into a View folder with a second WebView folder for the web-specific view code. The rest sits at the base for now, because I expect it to change significantly as the refactoring progresses.
In lib, I’ve made DatabaseRequests, DataModel, Serializers, and WebRequests. This more or less follows the categories I created earlier, although some of the more generic code remains at the base (for now). Honestly, just having a clear separation between the *Request classes that operate on the network and the *Request classes that operate on the db is already huge.
And that’s where we’ll leave things for today. I’m going to avoid a complete play-by-play of this process (I expect that it’ll be pretty long), but try to use future posts to highlight the interesting parts instead. You might’ve noticed that in this first post, the amount of actual refactoring and code-touching was pretty minimal. Still, I think it helps a lot to start out like this. It gives me a much cleaner and easier to understand base to work on when I can see where where things are and what they do without having to check the code itself. Let’s call that Lesson 1:
No matter how well or poorly you write it, code takes time and attention to read. Through organization and documentation you can read less, and this will save you time in the long run.
Oh, and one more thing. If you’re wondering if I’ll address the rest of that old blog post, stay tuned. I have something planned for the blog archive.