In my previous post, Three Systemic Problems With Open-Source Hosting Sites I identified some missing features that create serious brittleness in or project-hosting infrastructure. The question naturally arises, why don’t existing hosting systems already have these facilities? I have looked into this question, actually examining the codebases of Savane and GForge/FusionForge, and the answer appears to go back to the original SourceForge. It offered such exciting, cutting-edge capabilities that nobody noticed its internal architecture was a tar-pit full of nasty kluges. The descendants — Savane, GForge, and FusionForge — inherited that bad architecture.
The central problem is implied by the implementation. It’s PHP pages doing SQL queries to a MySQL database. The query logic is inextricably tangled up with the UI. There is no separation of function! Now I understand why, back when I was a director at VA Linux, the original SourceForge team promised me a scriptable release process but never delivered. They couldn’t have done it without either (a) duplicating a significant number of SQL queries in some kind of ad-hoc tool (begging for maintenance problems as the SQL schema changed) or (b) prying the SQL queries loose from the GUI and isolating them in some kind of service broker, either an Apache plugin or a service daemon, that both the web interface and a scripting tool could call on.
Approach (b) would have been the right thing, but would have required re-architecting the entire system. It never happened. When, in my problem statement, I complained of forge systems being excessively tied to Web interfaces, I did not yet know how horribly true that was.
The rottenness of the architecture also accounts, indirectly, for some other features of these systems that have long puzzled me – like the reliance on cron jobs to do things like actually creating new project instances. The cron jobs are a half-assed substitute for a real service broker.
I conclude that the SourceForge/GForge/FusionForge architecture, as it is now, is an evolutionary dead end — overspecialized for webbiness. To tackle challenges like fixing the data-jail problem, scripting, and seamless project migration, one of these systems will need to be rebuilt from the inside out. The surface appearance of the GUI might survive at one end, and the SQL schema at the other, but everything — everything — between them needs to change.
As one of my regulars pointed out in commenting on the previous post, what we see here is a massive failure to apply the Unix philosophy. It needs to be fixed. I’m not sure what the best way to go about it is, though. I’d love to start with a clean sheet of paper; our tools are ten years better now, and with Python and Django I could probably have a proof-of-concept within weeks and deliver a forge system functionally superior to anything now deployed in a couple of months tops. The problem is, if I do that I have a serious political barrier to climb getting it deployed. The alternative would be for me to rebuild an existing forge system from the inside – much more difficult, because I’d be dragging around a lot of legacy, but I’d get to keep the userbase and the brand.
One of these things looks likely to happen. I’m not sure which, yet.