Tech ramblings by Marcin

operating systems showdown

2020-12-05 00:00

Modern open source operating systems vs modern software development practices

The reason to write this blog post was a presentation I watched some time ago, and commented on it on Twitter:

So to be specific - the whole presentation was rather negative towards security model used along OpenBSD development. The presentation itself took a lot of time to go over recent security issues, false claims, etc.

What strikes me in it was how the development process for that particular operating system actually looks like. Working with software for quite some time now I have a pretty decent idea of how the development process of my half baked, security-assed code actually look like.

All that got me wondered - how OS developers actually deal with their daily load of issues? How do they fix bugs, monitor simple security issues, regressions, etc? And also what’s their tooling around that? It’s easy to say - just use git, or JIRA, or any other “normal” dev tool. But since you’re developing an actual OS things are’ perhaps’ not that obvious anymore. The code bases tend to be huge. With the multitude of contributions, mighty loads of bugreports.

The comment was about OpenBSD, but then I started wondering - how does this process actually look like for other major (or not necessarily) operating systems?

So I’ve decided to take some time and look into that space. I wanted to look into a few operating systems I tend to use, used to use, or just keep an eye for. Some of those are the key players in the OS space, some are just hobby projects. For all of them I just wanted to get a picture of how the development process looks.

Please keep in mind that this should be regarded mostly as a beginners guide, I have no significant experience in contributing to any of those code bases, so I was also looking into how approachable each of those communities is for newcomers. Also there may be a lot of obvious omissions for experienced contributions, but if it’s not mentioned in this post than I had a hard time finding relevant information - so anything that is a “secret lore” circulating among experienced contributors is not enough for me, a newbie.

With that disclaimer out of the way, I’d really welcome comments about my findings, so if you have any fell free to reach out to me and help with clarifying things here.

The process as I see it

So what actually constitutes the development process of a software project? Well for me this can be summed up into how the project handles the following key areas:

  • how should I approach the codebase when I want to start contributing to the code
  • what if I find a bug? how that looks like?
  • what happens when I submit the bugreport?
  • how the code is tested for regressions, or tested in general?

A good deal of this questions can actually be answered by discovering how each of those systems approach:

  • VCS
  • Testing
  • code-review process

Rationale for including each of the systems in this comparison

  • Linux - well, I just use it the most theses days. Which is fine, since the kernel is such a great code with a lot of innovative things happening. But since with just the kernel you can’t actually do that much, you always choose a specific distro that serves all your user needs. This comparison thou focuses solely on the kernel development process. I don’t want to delve into the fragmented space of Linux distros and they’re own ideas. I feel it would just dilute the overall impression of the efforts around Linux as a platform.
  • FreeBSD - a system I’ve been using for a really long time. It has all the great ideas of the BSD family, is extremely coherent in vision, as are all BSD systems since they’re not just a kernel but a whole package. You get kernel and userland with all the tools and basic integrations, which in my opinion is great. Choosing a specific BSD flavor actually means something, and not just a different UI experience as in the case of Linux.
  • OpenBSD - the reason this blogpost came to being, but I wouldn’t watch the presentation in the first place if not for me previous interest in this flavor of BSD. It was actually the first of the BSDs I’d started using. Liked it for it’s compactness, great documentation, ease of installation,
  • NetBSD - I’ve actually haven’t installed it, not even once. But this is such a great example of a system that tries to keep all those obsolete machines (which I adore) alive, I just couldn’t omit it from this comparison. Also what’s even more important, OpenBSD at some point forked from NetBSD due to difference in opinions between core developers. So both systems have common roots, let’s see how they actually compare,
  • FreeDOS - a free reimplementation of DOS - seems pretty complete and finished at this point, but since it’s still used in multiple places the community around it is quite vibrant,
  • ReactOS - another reimplementation - but this time of the Windows 95/2000 line of systems. It has this ambitious goal to be able to run Windows programs without problems. All this being accomplished with a lot of reverse engineering and of the original implementations. Last time I’ve check I still couldn’t run some programs I needed, but the effort is impressive enough for me to keep an eye on it from time to time.

Linux

Alt text

This is somewhat different, as due to its nature refers solely to the kernel. Other OSes in this comparison store kernel+base system in their repos.

FreeBSD

an image from splendid article -> splendid article

OpenBSD

Alt text

not underestimate the effort involved.

so this has come up before, and the answer remains the same. anyone can setup a bug tracker, and feed bugs into it. close the ones that get fixed, categorize the rest, etc.. do that for a few months and see how it goes.

i’m not really interested in looking at an empty bug database. nor one that’s filled with crap. so yeah, there’s a bootstrapping problem.

you don’t have to announce your bug database the first day you set it up. in fact, it’s better not to. but in a few months time, when somebody inevitably asks misc how do i contribute, where’s the todo list, you’ll have this handy list of unresolved bugs to point them at.

like a lot of projects that seem really easy, you’d think somebody would just do it if it were that simple. but the idea that nobody wants to chance investing time in a deadend project suggests they kind of know the time investment isn’t just a saturday afternoon.

If you look at a successful bug tracker, you will not be able to see most of this work without grovelling through dead tickets because the already-fixed issues and veiled support requests will no longer be visible.

NetBSD

Alt text

FreeDOS

Alt text

ReactOS

Alt text

Comparison table

OS name VCS Code review Bug reporting Contributors guide
Linux Git
Freebsd SVN / migrating to Git Phabricator Bugzilla commiters guide
Openbsd CVS / Github based Git readonly mirror CVS - core system, review via email, / Github pull-requests for GH-based projects No intention to have a bugtracker
Netbsd CVS commiters guide
FreeDOS Git Github-based sourceforge tracker
ReactOS Git reviews in git JIRA how to start contributing

Conclusion

The mentioned operating systems differ greatly in terms of modern tools adoption. The whole endeavor started as a way to compare how grounded in reality accusations towards OpenBSD really are. And from what I’ve found, also judging by the seemingly strong security concerns of Openbsd’s maintainers, and relative popularity of the OS itself, I must say that it now seems nothing more than a hobby OS, or a niche one, with usage in specific places like isolated network infrastructure. Not the one that should really be accessible from the outside world in any way. The effort around the OS has brought us a good deal of really useful and secure solutions, but as a whole its development process seems dubious.

As for the other Operating Systems - the clear contenders here are Linux and FreeBSD.The former as an unprecedented leader in terms of focus and the amount of contributions from various vendors and independent developers, and the latter because of the exquisite quality of the source code and value it brings to the table despite being foreshadowed by The Penguin.