How to Monitor Commercial Power & Battery Health in Your Remote Telecom Cabinets

I just got back from a trip last week. Among several other DPS clients, my fellow engineer Andy and I stopped by a telco (telephone & ISP) in a fairly rural part of Texas.

After a 2.5-hour drive, I sat down in a nice conference room with two network engineers responsible for many remote cabinets.

Monitoring Goal: Detect Commercial Power Failures at Remote Cabinets

This telecom network has about 75 cabinets to monitor. Some serve as few as 7 rural customers each.

They run on commercial power, and they have only UPS batteries (4 x 12VDC = -48 VDC string) as backup power. They don't have generators due to the small size of the cabinets. These enclosures have quite a limited power draw.

For this client, my proposed solution is to monitor power failures and battery health at their very remote telco cabinets - which can sometimes hit 170°F

The two engineers I met with described one particular incident that created a problem: A technician replaced the UPS batteries at a small cabinet site and forgot to flip the circuit breaker back on. This created an artificial power failure in the cabinet, so it was running on battery power only.

3 weeks later, that cabinet site went dead. Oddly enough, these clients were impressed by how long their batteries lasted. They're officially only rated for "24 hours".

Still, it really didn't matter whether the batteries would last for 3 weeks or 10. They needed to know that commercial AC was (due to the breaker being left open) effectively down.

This led to the first remote monitoring goal from this client:

"Is there a way to find out when power goes out, immediately?"

Other Equipment is Managed Separately by Another Department, Mostly via SNMP

In this network, there is quite a bit of existing SNMP equipment. Some rectifiers send SNMP, but there are also some older rectifiers that don't.

Solarwinds is a central manager already installed and running in their network. It collects SNMP from "blades" (the systems that provide data service to customers) only.

While this system works fine for what it does, the group I spoke with is looking for a dedicated central server. The existing system has a lot of data that, while it's critical for other departments at the company, isn't something they need to do their particular jobs.

Drive Times are Long, Making Remote Monitoring Very Important

The maximum drive time at this telco is 2 hours between territory edges, so truck rolls are problem. Even worse, it's possible to arrive at a site/cabinet without a spare device or part that is needed.

Currently, when generators run at the central offices, their contact closures trigger alarm dialers. These are working "OK", but could be better

Battery replacements are another recurring source of truck rolls (plus additional labor time) that could be eliminated.

For their battery monitoring, this client told me, "We replace batteries every 5 years just because we won't know when they eventually fail. They might actually last 10 years or more if we kept using them until they started to fail."

For that reason, we agreed on the concept of BVM sensors (Qty 4 of 12v, with perhaps few sites with). If that amount of monitoring wasn't feasible within their budget, a single RTU analog input for string voltage would offer at least half of the same value for a minimal expense.

Very Hot Cabinet Temperatures Present a Challenge

In this part of rural Texas, some telecom cabinets can reach temperatures of 170 F under the hot sun. Our "Industrial Temperature Rating" builds are clearly required for this project.

I Prepared a Proposal to Tackle All of this Client's Challenges

My proposal are intentionally detailed, because many people at your company will need to review it. You need a complete data packet that assumes no prior familiarity with either your needs or my proposed solution.

Here's some text from the introductory letter:

It was great meeting with you at your office last week. It's always great to hear the specific issues you're dealing with. The problems you described are certainly things that we've solved for clients before.

After speaking with you, I understand that you need:

1. Reliable power & battery monitoring for your 70 remote cabinets, which can sometimes reach internal temperatures of 170 degrees F.
2. 2. A dedicated central alarm collector for your team that will focus primarily on power, batteries, and other "facilities" concerns without involving clutter from other alarms.
3. 3. Detailed alarms and notifications that enable you to respond to power failures before batteries run out and ensure you arrive at the site with the correct parts.

The system I've quoted you here includes:

1. Our NetGuardian LT G2 (or, optionally, the newer G6 model). This RTU carries our highest "Industrial" temperature rating with a rated maximum of 158 F. Although you have seen hotter temperatures in your cabinets at times, I can confirm that NetGuardians with this rating have been used in Texas cabinets for many years. We try to rate our gear somewhat conservatively so that we will exceed the rating in the real world.
2. Our T/Mon MINI master station, which will be able to poll all 70 NetGuardians here (with a device limit of 128 total). T/Mon is also able to pull in alarm data from various third-party equipment (may require one-time purchase of a protocol module to enable SNMP, etc.), but it sounds like you're aiming for T/Mon to be quite focused on just the power data collected by the NetGuardians.
3. The ability to database detailed alarm labels and response instructions so that your team doesn't drive out to a site with partial information. You can also send alarms to emails & smartphones to allow for "stopping by" conveniently when a technician is already out in the field when an alarm occurs.

You have a choice between the NetGuardian LT G2 & the newer NetGuardian LT G6 with very detailed battery health sensing:

As you'll see from the Investment section, your decision to add the battery health sensing to our smallest NetGuardian has a large percentage impact on the overall price of the system. Still, batteries (and network uptime and customer satisfaction) are no less important at smaller sites - and the waste associated with replacing them earlier than necessary is no less costly.

The NetGuardian LT G2 is our existing hardware solution. It has built-in analog monitoring of the input power, which can be converted into a "battery life remaining" statistic to trigger alerts (and that you can view at any time).

The NetGuardian LT G6 is built on our new hardware platform released just this year (named "G6" to unify the all numbering with our flagship NetGuardian 832A G6). It will have a newer web interface and longer lifespan of ongoing firmware improvements (available free for the life of your NetGuardians).

The increased hardware resources on the G6 model will support our new battery triple-sensors (temperature, voltage, and internal resistance monitoring for each battery in your string).

As we discussed in our meeting, our roadmap for the battery sensor firmware also includes a new variable-timing system for using AC load to check internal resistance. When higher-voltage batteries are tested more frequently, they are automatically rebalanced to match the others. These slight "nudges" over time will tend to increase the lifespan of your batteries overall.

Also, with detailed battery monitoring, you won't need to proactively replace batteries after 5 years "just to make sure they're still good". That represents a potentially huge savings across the lifespan of NetGuardians, which historically has been well beyond 10 years for many DPS clients.

Soon After, I Modified My Proposal for an Initial Rollout

At your request, I've scaled down the project to just 10 of the NetGuardian LT G2 RTUs. This also allowed me to reduce the T/Mon to a "MINI" model that supports up to 16 devices (expandable via simple software upgrades later). You could choose to operate the LT G2 RTUs independently via web interface and email alerting initially. 10 units is right around the number where I recommend a T/Mon to aggregate all the RTU data onto one screen. You'll be able to grow your system later by adding RTUs and adding to your T/Mon device limit.

(For your reference, my original proposal letter remains below. Other than site count and the selection of the LT G2 and not the G6, the information remains valid.)

What Remote Monitoring Do YOU Need From DPS?

This is one (very specific) example of a system I proposed for purchase by one specific DPS client. Now, the question you must ask is:

What system do you need me to build for you?

To get started, just give me a call at 559-454-1600 or email me at sales@dpstele.com

Andrew Erickson

Andrew Erickson is an Application Engineer at DPS Telecom, a manufacturer of semi-custom remote alarm monitoring systems based in Fresno, California. Andrew brings more than 17 years of experience building site monitoring solutions, developing intuitive user interfaces and documentation, and opt...