Sharing the load effectively

At the end of my last post I said I was wandering off to think about scalable, low-overhead recommendation systems.

It’s funny how preconceptions work. I know, I think better than most people, how often decentralizing systems to avoid single points of failure is good engineering. Yet I had to really struggle with myself to jettison the habits of thought that said “If you want to use money to help people, you’re going to have to build a centralized, heavyweight structure around the management of that task.”

But struggle I did. Because I’d already tried that, and failed.

I also had to get past the idea that identifying good funding targets can be crowdsourced. Nope. Identifying candidates and digging up information on them can be, but actually evaluating merit and centrality will take knowledge most contributors not only lack but have no strong reason to try to acquire.

Once I got my head clear, this is what came out:

http://www.catb.org/esr/loadsharers

The basic trick here is piggybacking not just on the payment transfer capacity of remittance systems but on their patron/client communications channels as well. That way Loadsharers doesn’t need to manage anything itself other a handful of adviser web pages and a bunch of trust relationships.

Also notice the implications of how I designed the Adviser role. By the time we have a half-dozen or so advisers I won’t be key man anymore. That’s intentional.

I also like the fact that there will, in effect, be a (mildly) competitive market in adviser skill, with loadsharer contributors tending to gravitate to advisers who exhibit activity and diligence. That’s intentional too.

37 thoughts on “Sharing the load effectively

  1. I think another role is needed: recruiter.

    Just funding the LBIPs doesn’t solve the problem: They don’t suddenly drop their projects because they can’t spare the time and expense of working on them, sure, but they will eventually die no matter how well you fund them.

    So we need a means of finding apprentices for LBIPs (at least two per LBIP), funding the apprentices, and continuing to fund both LBIP and apprentices after the apprentices are up to speed enough that the LBIP is no longer an LBIP. LBIPs are LBIPs because, out of all the people with the skillset to maintain a given project, there aren’t enough with the time, financial stability, and awareness of the project to give the project a sufficiently high bus factor.

    Even if the loss of a given person wouldn’t be a catastrophe, if that person is working on a project with a bus factor of two, the loss of that person would reduce the bus factor on that project to one. We don’t just need to fund LBIPs, we need to fund enough maintainers for critical projects that people don’t become LBIPs in the first place.

    Criticality should be split into two parts: how much we lose if each individual project a person works on collapses for any reason, and the bus factor for each project the person works on (the size of the smallest group such that, if the whole group were hit by a bus while crossing the street, the project would collapse).

    Someone who works on 10 projects, all with moderate criticality, but two of which have bus factors under three, is probably more critical than someone who works on 10 projects with civilization-ending consequences of failure, but none of which have bus factors under 50 (unless most of those 50 live in a war zone, or something).

    • >I think another role is needed: recruiter.

      All your points are valid. But I think you’re asking us to run while we’re learning to walk.

      In particular, I don’t think we’re ready to take on the need for apprenticeships yet. When there are ten thousand loadsharers and a couple of dozen advisers that will be a good thing to revisit.

      I think your idea of trying to evaluate bus factors is worthy. But have you thought about how to do that? Nobody is served if we set success criteria for advisers that no one can meet.

    • I agree that the apprenticeship concept isn’t appropriate to consider just now, but it’s interesting to me as I’ve thought on the bus problem for a long time.

      Some thoughts..

      Teaching, mentoring, learning and being apprenticed are a talent/skill most simply don’t have.

      Time and mind are taken away to mentor. This makes the project suffer in the short run with the assumption that a new person will make up for it in the long run. It would be interesting to reveal how many projects couldn’t even spare the mindshare to apprentice and so are really vulnerable to the bus problem.

      Some of those people are good because they are — charitably-speaking — atypically sane. They think and act in ways most humans don’t, why is why there’s only one of them. Coming from thought and experience, I don’t believe the mindset available to some of these people can be taught or learned. Projects and positions need to be simplified down to the level of “ordinary hackers” without it becoming yet another damned committee of developers. But how? This problem, and hopefully its solution, would be revealed when trying to apprentice.

  2. I am using libsodium, and value it enormously.

    If not for libsodium, we would be using crypto algorithms written by our evil enemies who hate us.

    So I would love to make a contribution to libsodium, and am working on tool to make it easier to use,but I would like to make a monetary contribution, but am not sure who or how.

  3. Just a thought. Infrastructure like roads is mostly maintained by governments all around the world. Volunteers maintaining the Internet infrastructure doesn’t simply mean you get it cheaper and better, what it really means is that if volunteers stop doing and the governments take over, you get spying on users or banning inconvenient ones built into the whole structure of the deep thing. Nightmare fuel.

    Of course there is a third option. If volunteers don’t maintain roads and I don’t want to take government roads, I can buy a very sturdy Jeep and risking driving through the prairie. So I would recommend putting some thought into how users could live without parts of the infrastructure in the worst-case scenario. What if my browser would store the IP of every website I have visited, my email client would store the IP of every domain I sent an e-mail to, so I could live with DNS being down a while? Maybe stupid idea, I am not really qualified to brainstorm about it, but just an example.

    • > I can buy a very sturdy Jeep and risking driving through the prairie.

      No, you can’t. It’s either owned by private parties, and hence fenced off or it’s owned by the government and you’re expected to keep to the existing trails.

    • ESR,

      Thinking further. From this angle, you could get donations not only from programmers but any rich dude, corporate manager who care about their privacy. Who want as little spying built into the infrastructure of the Internet as possible and therefore see it in their interest that it should be maintained by volunteers, not the government. One of your greatest achievements is selling Open Source to the suits who were originally skeptical about it, they mostly considered it hippie stuff. Maybe you can do that again.

  4. [Offtopic, shameless excuse for subscribing to future posts – your signature seems to still include a link to the now-defunct Google Plus.]

  5. The list of LBIP candidates at the Loadsharers page REALLY wants to be a table (one sure sign of this: repeating categories such as “criticality” and “need” for each individual is almost always a prescription for breaking out such information into columns, which are visually cleaner and easier to read).

    I have a mockup here that I’d be happy to send you. I’d offer to scratch out the HTML, but I doubt my own ability to do so in a sufficiently competent fashion.

      • Are you suggesting that I take a run through asciidoc to see how quickly I can learn to format tables?

        • >Are you suggesting that I take a run through asciidoc to see how quickly I can learn to format tables?

          No, I can do it. Not sure I will though. I did a half-conversion and it didn’t look so good. Running-text entries in tables tend not to.

    • >I’d like to donate via SubscribeStar rather than Patreon, Is there a way to do this?

      The LBIP you want to donate to would have to have an account there.

      By coincidence, I got my SubscribeStar ID approval earlier today. I’m esr there. Please forgive the somewhat stark appearance of my page, I have not had time to pretty it up yet.

  6. FYI your PayPal link to Simon Kelley is non-functional.

    Also, Stallman/FSF is noticeably absent in your list. They have had such a strong impact on the OSS system that I think they warrant a note, even if that is effectively “intentionally left blank” for whatever reason. Stallman has said, with some justice, that “it’s actually a GNU system with a Linux kernel”.

    (I suggest a vague mention of a reason is necessary for advisor reputational reasons).

    • >Also, Stallman/FSF is noticeably absent in your list.

      Because he’s safe. There’s a whole organization around RMS that exists to fund him and has been doing so for years.

      • Yes, I anticipated and agree with your reasoning. It’s one of his big accomplishments. I still suggest he warrants a mention.

        • >Yes, I anticipated and agree with your reasoning. It’s one of his big accomplishments. I still suggest he warrants a mention.

          If you look at the page now you’ll see he gets one.

    • >Also, how does one “claim a place” on the early adopters list?

      Take the pledge, give me a link to your Patreon or SubscribeStar contributor page so others can see you’re executing on it, and I’ll add it.

      • I wrestled with this because I much prefer to not be “out there.” But my near-future plans involve providing targeted technological funding and contributions for individuals and groups, so awareness matters I guess. If this list is currently running, I will submit for addition.

        https://www.patreon.com/user?u=21392675

  7. I used your LBIP page to give a $30 pledge to David Taht. I would have liked to give a pledge to Simon Kelley but the paypal link does not work. He also does not have a link to pledge money on his homepage.

    One thing to note is that Patreon filters the erotic artists and cosplayers I support from my “Whom I contribute to” list. This is a feature and not a bug.

    • > but the paypal link does not work

      Indeed it doesn’t. It did for me 2 days ago, I sent money and he said thank you. And my PayPal dashboard still shows the next payment due to go out in a month.

      I could post the email address he uses for his PayPal Account here, maybe you could find him that way, but I’m not sure if im supposed to.

      Anyway, ESR’s Loadsharers page says Simon will be on Patreon soon.

      • It’s good to know it worked at some point. The issue seems to be some token that expired. I can wait until another link is given or until Simon has a Patreon link. Dnsmasq came in handy at a former job. It would be nice to give back.

  8. Why are you still using centralised services such as Patreon instead of accepting donations directly to a bitcoin address?

    • >Why are you still using centralised services such as Patreon instead of accepting donations directly to a bitcoin address?

      Because Patreon and SuscribeStar have convenient setup for recurring donations.

      The goal of Loadsharers is to solve a funding problem, not be “Oooh, look! Crypto-geekery!” performance art.

      • “performance art”?

        Why so dismissive of technology that enables permissionless, decentralised and uncensorable electronic cross-border transactions?

        • >Why so dismissive of technology that enables permissionless, decentralised and uncensorable electronic cross-border transactions?

          I’m not dismissive of the technology at all. It has clear use cases.

          But if I were to use that as a sole payment channel, it would be a performance-art gesture – appealing to a comparative handful of people, not sufficient attack either my personal funding problem or the larger one I’m trying to solve.

          • > But if I were to use that as a sole payment channel, it would be a performance-art gesture

            You could, however, easily publish a donate-to bitcoin address for precisely zero cost as a parallel channel. Maybe the bitcoin channel would be infrequently used, but there seems little downside and at least theoretical upside in catching potential donors who won’t use paypal or similar services.

Leave a Reply

Your email address will not be published. Required fields are marked *