UPS I did it again

I had to buy a new UPS for my desktop machine yesterday after the old one succumbed to battery death, so Cathy and I made a run to the local MicroCenter.

UPS designers have been pissing me off since forever with designs that require you to throw away the entire device when the battery craps out, unless you’re willing to go to great length to avoid this – finding the exact right replacement battery from a specialty supplier, then taking the unit apart and reassembling it yourself.

This is never practical under time pressure, and I’ve never had the luxury of no time pressure when trying to cope with a dead UPS. Sure didn’t this time; my area was under a severe-thunderstorm watch.

Imagine my pleased surprise when I found a big stack of varied models branded APC that are not just significantly less expensive and with longer dwell times than when I was last UPS-shopping, but designed with removeable and replaceable batteries, too.

Progress does get made. Dunno whether this is a standard feature on all UPS brands yet, but doubtless it will be within a few years.

Some of you may find my UPS HOWTO of interest. I’ve shipped a 3.0 update with the glad news of replaceable batteries and a few other minor updates; it may be up by the time you read this.

58 thoughts on “UPS I did it again

  1. In my email, that paragraph reads:

    > Some of you may find my UPS HOWTO of interest. I’ve shipped a 3.0 update with the glad news of replaceable batters and a few other minor updates; it may be up by the time you read this.

  2. >It appears the post got cut off at “Some of you may find my”.

    Missing double-quote botch fixed.

  3. I’m going to have to go UPS shopping, I think. We had some heavy rains through here a couple of weeks ago and got water in the basement, just enough to puddle a bit on the floor and soak the carpet in the half that’s carpeted. Guess where the Linux server’s UPS was sitting?

    But yes, we’ve got a few APC UPSes with replaceable batteries. Dunno yet how easy they’re going to be to find replacements for.

  4. >But yes, we’ve got a few APC UPSes with replaceable batteries. Dunno yet how easy they’re going to be to find replacements for.

    You can buy ‘em off the APC website.

  5. I actually found replacements for my APC in a (er, THE) battery store in Central Australia. There were pretty crap batteries because they’re dead now (lasted about 2 years), and I’m unsure whether I want to replace them, or just get a new unit, it’s several years old.

  6. I have 3 classes of UPS that I deal with at home: (1) desktop UPS, (2) server UPS, (3) telecom integrated UPS.

    A desktop UPS is a standalone unit that exists primarily to give me time to gracefully save my work during a power outage. It needs to handle high current draw, since it’s powering my monitors in addition to my computer, but it really only needs to do that for a few minutes. Bonus points if it can signal my system to shutdown gracefully when power is low. I have these all over the house as needed (including one for my wife’s CPAP). I have many different brands, generally whatever was cheapest at MicroCenter the day I needed a replacement.

    The server UPS is a different beast entirely — it must have a long dwell time, it must have replaceable batteries, and it must communicate with VMware ESX to provide for graceful shutdown. This is currently an APC Back-UPS XS 1500 with an outboard battery pack, and is dedicated to the server. All of the switches share another UPS which will generally outlast everything else in the house. Bought it at MicroCenter (and bought the battery pack there as well).

    Finally, there’s the UPS that’s built into the FIOS ONT. It has a standard 12-Volt 7.2Ah SLA (Sealed Lead Acid) battery that I’ve had to replace twice since I got FIOS in 2008. I get these at Radio Shack.

  7. APC has been pretty good about replaceable batteries for a while. Most of their standalone and 2U+ units take standard SLAs. Virtually all of their RBCs are just standard cells with special cables and connectors, either stuck together with double sided tape or encased in a steel shell.

    Where they fall down is in communicating the battery number. When you buy one, immediately write down the model number and the RBC number. For some reason, their serial/USB protocol never contains the model number, or any other information useful to replacing the battery.

    Sadly, I see regress in the UPS world, not progress. The marketing numbers are all fiction, much like the “horsepower” ratings on tools, and the free features are vanishing before our very eyes. My most recent 2U model reports virtually nothing over USB, just the state-of-charge and estimated run time, while a similar model from a few years back would report utility voltage, load, temperature, date of manufacture, and date of last battery replacement.

    In case it helps anyone, the polarized plastic hermaphrodite connectors they use on many midsize units for internal and external battery hookups are Anderson Powerpole SB. You can find them on ebay and elsewhere, usually in sets that include the shell and a pair of crimpable pins. The shells are keyed, so you need to match colors, and there are different sizes (that you also have to match). They make different pins for crimping different wire sizes, so pay attention there too.

  8. Tired of cheap batteries which die necessitating replacement of the whole unit, I’ve been thinking about buying an AGM battery or two and an inverter and making my own UPS. It’s trivially easy and I may just extend this project by putting some 250 watt solar panels up on the roof and make my whole computer room solar powered. More expensive, sure, but a small project which should give me a better quality UPS and longer runtime for the money.

  9. Sucker. You did not need a new UPS. You just needed to go down to the local car part store and buy a set of jumper cables and tractor batteries. replace the silly 12V9A batteries your UPS comes with for some serious power at a fraction of the price.

    Turn you 20 min UPS into a 2 hour UPS for well under $100.

  10. >I’ve shipped a 3.0 update (…) it may be up by the time you read this.

    It still isn’t. This is the latest revision I see:
    Revision 2.2 2007-05-22

  11. The problem I’ve run into with the inexpensive desktop UPSs is, by the time I buy replacement batteries and pay shipping, it’s often cheaper just to buy another UPS locally.

  12. This is never practical under time pressure, and I’ve never had the luxury of no time pressure when trying to cope with a dead UPS. Sure didn’t this time; my area was under a severe-thunderstorm watch.

    I heard similar stories each time I worked with UPSs during my field circus career. My personal solution was to keep a spare unit with battery on hand, and the same at family member’s houses.

  13. >Sucker. You did not need a new UPS. You just needed to go down to the local car part store and buy a set of jumper cables and tractor batteries. replace the silly 12V9A batteries your UPS comes

    Gary, dude, what part of “under time pressure” did you fail to understand?

    I’m not going to try lashing together a custom rig from vague directions like that even when my desktop machine isn’t looking down the barrel of a severe thunderstorm watch. On the other hand, if you were to point me at or draft a HOWTO describing the build in detail…

    Now I’m wondering if this market might be a candidate to be disrupted by an open-source hardware design. Say, a minimalist UPS that ships without a battery but with terminals for an SLA, and upgradable open-source firmware. HackUPS, anyone?

  14. I am not sure of the open source firmware thing though but the UPS I used certainly used external batteries.

  15. I am not sure of the Open source part, but that UPS is specially designed for Indian power conditions, meaning managing heavy variations in voltage and power factor.

  16. …or you could, y’know, unplug your hardware and go read a book or two..

    ;)

    UPS costs do start getting a bit hairy when your hardware burns the thick end of a kilowatt….hence my more economical solution, above.

  17. Gary, there are problems with your approach. The charge circuits in the UPS are customized for the batteries that the UPS ships with. Putting different batteries in messes with the effective C for the system which will cook the batteries in fairly short order, if you are lucky. If you are unlucky, you’ll cook the charge circuit in the UPS.

    There are also problems with various suggestions to keep batteries on hand as spares. The most obvious problem is that batteries age even when not in use. A sufficiently smart charger can mitigate this to some extent, but not completely.

    If you’ve standardized on one UPS (or at least UPSs that all take the same battery), you can keep one spare on hand starting when your oldest battery hits 3 years (2 if you have frequent dropouts), or just replace them on schedule instead of on demand.

    You can merge the two ideas by rigging up a cable in advance to go from your UPSs battery connector to a car battery. In an emergency, borrow your car battery, but switch back to the proper size as soon as possible. If you the car battery gets deeply discharged, consider disconnecting it from the UPS and charging it with a car charger, particularly if it is much larger than the proper battery size for the UPS in question.

    This only works, of course, if your UPS uses 12v nominal for the battery pack, which a number of small to medium size units do. Larger units run at 24 or 48 volts, and you probably don’t have enough cars in your driveway to borrow 2 or 4 batteries.

    I think that most of the people that would really be interested in a hackable UPS eventually end up with a homebrew double conversion system, or, these days, with solar. A grid tie solar system with battery backup looks very much like a big UPS, particularly when you leave the solar panels off.

  18. I’m going to be a young whippersnapper and ask why the UPS HOWTO, while conceptually one document, is broken up into 8 separate HTML files. I’ve noticed that the format is common among older and well-respected hackers, but personally find it quite annoying. Am I correct in assuming that the practice originated because of bandwidth constraints in the early days of the Web, and continues due to the assumption that a significant portion of ones readership may still be on 56k or worse? Then again, the zsh man pages are similarly split up, and those are meant to be read locally.

  19. >I’m going to be a young whippersnapper and ask why the UPS HOWTO, while conceptually one document, is broken up into 8 separate HTML files.

    I don’t actually know. I’d make sure it rendered from DocBook as a single HTML page if it were up to me, but the way publishing through the LDP works is that I mail them a DocBook master and they render using their own stylesheet. Your conjecture about the historical origin of their policy may be correct.

  20. I guess the down-side to the following is that it is not “uninteruptable” , but it seems simple to have backup power – just buy 3 parts separately:

    - car batteries, or better, deep-cycle batteries (sometimes aka “marine batteries”)

    - a 12 volt to 120 volt inverter

    - a car battery charger (used when needed)

    Could this be turned into a UPS with a 120v-12v transformer, a relay to switch the 120v output from the mains to the inverter and a relay to switch the inverter on… the last part would require messing with the inverter or maybe leaving it on all the time but using a HEAVY-DUTY relay to switch the 12v input to it… Do inverters start producing 120v output fast enough?

  21. >>Dunno yet how easy they’re going to be to find replacements for.

    I just went to one of the local battery stores. It wasn’t cheap, but still was less than half of a new ups. If I had been willing to wait on shipping, I could have improved on that by a quarter or so,

    Jim

  22. ‘course relays use DC so throw in a diode bridge and an electrolytic capacitor, or otherwise provide a 120v AC to 12v DC to power relays if used.

  23. After working in IT support for a bit, my experience with UPS units was that all of them, at least up to a few years back, had replaceable batteries. Which is why this post was rather surprising. Granted, ours were probably somewhat more advanced models than those marketed to general public.

  24. > I’m going to be a young whippersnapper and ask why the UPS HOWTO, while conceptually one document, is broken up into 8 separate HTML files.

    There is also single-page version, but it is not linked from multiple-page version
    http://www.tldp.org/HOWTO/html_single/UPS-HOWTO/

    >> I’ve shipped a 3.0 update (…) it may be up by the time you read this.
    > It still isn’t. This is the latest revision I see:
    > Revision 2.2 2007-05-22

    I wonder how long it takes to publish an update on TLDP. Last HOWTO update is from Sep 2013, and there are some not updated since 1995.

    Eric, could you publish copy on http://www.catb.org/esr/writings/ ?

  25. @ESR: “Imagine my pleased surprise when I found a big stack of varied models branded APC that are not just significantly less expensive and with longer dwell times than when I was last UPS-shopping, but designed with removeable and replaceable batteries, too.

    “Progress does get made. Dunno whether this is a standard feature on all UPS brands yet, but doubtless it will be within a few years.”

    Horrible news, Eric. APC has been doing that for at least a decade, and from what I’ve seen, I must agree with kjj that all other vendors have been regressing, and APC isn’t in the twenty-first century with respect to communicating the necessary servicing information to customers and connected equipment (although I think they do at the high end where customers can afford lawyers.)

    Of course, the last time I saw an APC UPS in use, it was really solid, and really old. Its owner was so dumb (w.r.t. UPSes) that he did not know the battery had been dead for three years, and I think made nothing more than a shrug when I told him. Another owner of a similarly crippled APC UPS had kept it on the line for surge protection and accepted the risk of a potential sudden power-out. It’s pretty obvious to me that most of the Belkin UPSes and many of the newer-model APC UPSes for the bottom of the market (i.e. the ones that look like surge protection outlet bars with laptop batteries added …probably not an offensively inaccurate description, come to think of it) are designed for such users, and include much louder battery-out alarms.

  26. Okay… I got bored and wandered over to Slashdot and read a comment by “Gibgezr” that really had nothing to do with the post (Study: People Would Rather Be Shocked Than Be Alone With Their Thoughts) but really got me LOL…

    Erotic is using a feather. Perverted is using the whole chicken.

  27. That’s what I thought. But I have this fascinating story called “Featherwing Love” from which my “portrait” comes…

  28. Brian (and others) :

    60 hertz gives 8 1/3 milliseconds per half-cycle. Digikey has 120+ volt 15+ amp relays with 5 or 12 volt coils for under $5, and with ~8ms operation times.

    I’ve never seen a consumer inverter publish a startup time, but I’ve never looked either. You could just try it, or leave it on and idle.

    Your electronics will not be happy with the phase skew, but they’ll survive if you don’t do it too often (set your retransfer time high to prevent bouncing).

    Monitoring the utility power waveform should be trivial. You should be able to make the transition decision around the halfway point of any given semicycle, and reasonably fast relays should get you switched over during the next one.

    Designing a PCB with both 120v and sub-5v traces isn’t hard, but doing it safely is. Mark off one corner of your board for high voltage and surround it with a moat. Make sure no traces are routed across the moat, and the only components near it should be the relays and transformers that straddle it. Find someone to check your layout, or even the whole design.

    A double conversion unit is much easier to design, you basically just string the charger, the batteries, and the inverter together and call it a day. The downside, of course, is that each conversion has losses…

  29. A double conversion unit is much easier to design, you basically just string the charger, the batteries, and the inverter together and call it a day. The downside, of course, is that each conversion has losses…

    That is the best approach – it just plain works. It is sort of a CatB thing – a ups is sort of a cathedral whereas an inverter has very well defined input and output, and it plus the batteries and the charger can be bought at a Walmart or an auto-supply store. Of course, this would be a “dumb” ups, but sometimes that is a good thing.

    The relays to switch the computers’ source from the mains to the inverter would be trivial to put together – no pcb – use wires and perf board or breadboard. I don’t know if the charger should be connected/on all the time as it might not like it when the inverter starts drawing heavy current from the batteries.

  30. >That is the best approach – it just plain works. It is sort of a CatB thing – a ups is sort of a cathedral whereas an inverter has very well defined input and output, and it plus the batteries and the charger can be bought at a Walmart or an auto-supply store. Of course, this would be a “dumb” ups, but sometimes that is a good thing.

    I hear this, but I have a problem with it. I don’t grok analog electronics, I have no experience tinkering with custom or semi-custom hardware that handles mains current, and I’m nervous about trying.

    While I understand that in an objective sense a double conversion unit is simpler than a UPS with a bunch of semi-obscured firmware complexity, in practice simplicity is relative to the toolkit of the observer. Wrestling with software complexity is in my comfort zone. Wrestling with custom hardware that potentially delivers lethal shocks is not.

  31. I guess I’m a bit of a battery geek. Charger-converters are common in the RV and cottage worlds. Just add batteries. I don’t know what the changeover time is and I’m sure there’s no USB port for status but there’s an easy solution. If your existing UPS uses a 12V lead-acid battery then any 12V lead-acid will do. Just hook up an RV battery (with screw terminals) and you’re good to go. The main issue with changing the battery is overcharging but if you’re going bigger then you won’t be overcharging.

    Keeping a spare lead-acid battery on the shelf is a good way to end up with a dead battery. They need to be kept fully charged and will last 5 – 10 years like this. If your UPS battery is failing after a couple of years then there’s something wrong with the charging circuit. Or there’s something wrong with the 120V converter and the battery is fully charged and waiting for something to do. I have one of those.

  32. >then any 12V lead-acid will do

    I have to say that I’d be reluctant to use anything except a *sealed* battery indoors.

  33. Hi Eric, did you look at the UPS in my link earlier? Is that the kind of UPS you might be looking for?

    I think the issue for you is that it is designed for 220/240 Volts 50 Hz mains. Moreover the product is a local Indian product for Indian markets only. I have personally used the KRYKARD UPS and the battery is separate from the main unit and it works well in long power cuts (which is all too common here)

  34. OT:

    I’d like to ask for a bit of advice: A few months back, I purchased some hardware based on the list of hardware that a certain open-source project claimed to support. Upon actually trying to use the hardware, I found that the software technically does support it, but contains a bug that renders the hardware useless (a front-end program calls a library, which interacts with a driver. The front-end’s first attempt to access the hardware succeeds and returns useful data, but all subsequent attempts seem to hang until I exit and restart the front-end. For the hardware to be useful, the front-end must be able to access it several times without being restarted. This behavior is the same across three separate front ends).

    I filed a bug report on the project’s bug tracker, but that bug report has gone two months without any evidence that any of the maintainers have even looked at it. So the first bit of advice I’d like is this: Am I more likely to get results by playing the squeaky wheel and trying to rouse the maintainers, or by trying to debug the issue on my own? My programming skills are rather “spiky”: I’m fairly confident of myself in certain areas, but have rather significant gaps in my skill-set, such as not really knowing how to go about debugging code that I didn’t write myself, which leads to the second bit of advice I’d like to solicit: Can anyone provide any good pointers to resources on debugging other people’s code?

  35. >So the first bit of advice I’d like is this: Am I more likely to get results by playing the squeaky wheel and trying to rouse the maintainers, or by trying to debug the issue on my own?

    Normally I spend a couple hours on the second, then if that fails do the first.

    >Can anyone provide any good pointers to resources on debugging other people’s code?

    I don’t know any other way to learn that than hard experience. I would be as interested as you in any positive answers to this question.

  36. Re: debugging other people’s code:

    I’ve got to agree with ESR: hard experience is the main teacher. I can, however, hopefully at least point out the obvious: learn the tools. Knowing how to use tools like gdb, strace, objdump plus whatever language the code is written in, plus a good overview of general systems programming will give you a lot of the necessary context for debugging other people’s code – or software in general. You may need even more specialized contextual knowledge – I’m thinking here of ‘deep stack’ knowlege of things like django, tomcat, or gwt – but know the basics will get you asking at least the right questions of the docs for those specialized silos.

  37. >110-volt relays are trivially easy to find.

    Solid-state relays that will switch crazy amounts of current at fairly high voltages are not only easy to find, but cheap. The one I bought for my fermentation fridge when I switched its temperature controller over from an Apple II to a Raspberry Pi is rated for 25A @ 380V…probably shouldn’t trust that maximum, but 15A @ 120V should be well within its capabilities. With a heatsink, it was under $10 from some random eBay seller, shipped from China. For a UPS, I suspect you could use two of them (four, if you want to switch neutral as well as live) with a 74LS00 or equivalent to make sure either one or the other was on, but not both. It’d give you faster cutover between wall power and the battery, which would make a reset-triggering glitch less likely.

  38. Back when I was living in AU I had a small number of US (110/120v 60hz) electronic devices I wanted to run. There were some transformers in the house, but they only handed voltage upshifts, not phase conversion (meaning you put in 220v at 50hz, you get 120v out at 50hz). This was generally fine for pure electronic device (heck, most of them could handle straight 220), but was crap for motorized devices.

    I looked at building my own converter using a couple of 6 volt batteries in series and a “pure sine wave” converter at the other end. This would have had the additional benefit of conditioning the power, which is purportedly a lot better for the electronic devices.

    The one problem I had was the input side, I couldn’t find anything readily available that would reliably flow current to the batteries 24×7.

  39. I still can’t see any revision later than 2007. What’s going on?

  40. > Here’s a link to the updated version: http://www.catb.org/esr/ldp/UPS-HOWTO.html

    Thanks.

    BTW. I have noticed the following errors:

    In addition, some UPS or UPS/software combinations provide the following functions:
    [...]
    7. Provide alarms on certain error conditions.
    8. Provide alarms on certain error conditions.

    duplicated item

    it was not to be found on any consumer- or SOHO-grade GPS as recently as 2006.

    GPS -> UPS

    Below, you will find some suggestions for buying replacement batteries if your vebdor diesn’t supply them.

    vebdor -> vendor

    P.S. thanks for adding live preview… though it partially overlaps submit button in Chromium.

  41. > Revision 3.1 2014-07-09 esr Minor update. Note that SPSes may no longer exist.

    Emerson Network Power (Chloride Power Group) has Liebert PSP Off-Line UPS.
    CyberPower has Standby UPS Series.

  42. >Emerson Network Power (Chloride Power Group) has Liebert PSP Off-Line UPS.
    CyberPower has Standby UPS Series.

    So, not extinct but pretty rare. I googled for “standby UPS” to check.

    I’m actually a little puzzled why these have survived. AFAIK, the only advantage of SPSes was the lower cost of the control circuitry, and that was flattened by Moore’s Law years ago.

  43. “A surge is a substantial increase in voltage lasting a small fraction of a second, often caused when high powered appliances such as air conditioners are switched off.”

    Or such as someone plugging a floor buffer into the outlet on the other side of the wall.

    BT, DT, replaced the mainboard.

  44. For what it’s worth, I have adapted several APC UPSes in the 1-2kW range to use much larger SLAs than they came with. As an earlier commenter mentioned, the charging circuit in the units had no trouble with this, and slow charging doesn’t hurt most SLAs. I used this to run my machines during a three-day power outage in Hawaii after one of the earthquakes, in ’06 or ’07 if memory serves. Worked fine. This is a _very_ simple modification to make — if you can crimp or solder a terminal and perceive the difference between + and – or red and black, I think it can be done with little trouble.

  45. RE: The “Software Assistance” section of the HOWTO, some interesting tidbits on the software for a UPS I just bought:

    MATE’s power management components detected the UPS immediately without needing to install any extra software or do any special configuration, and appeared to offer sensible defaults for shutting down on low power, etc. However, in doing some reading around on the web, I saw some mention that GNOME’s power management suite will display notifications on UPS low battery states, but won’t actually take the configured action (such as shutdown) when the UPS battery drains, and that the underlying driver layer is being deprecated. I’m not sure how much of this predates the fork-off of MATE, so I’m not sure how functional MATE’s power management actually is.

    NUT seems to be completely dead in the water as far as functionality. The documentation is stale, the install scripts don’t create directories that the drivers need to run, and even once that’s resolved, NUT’s drivers don’t seem to think they can support the UPS.

    The box advertised that management software for the UPS was available for Linux, but contained no CD. Instead, the software is available in RPM, deb, and tarball formats on the manufacturer’s website. The software appears to be functional, but the manpages, while intelligible, were definitely not written by an English speaker.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>