Large water systems employ Supervisory Control And Data Acquisition (SCADA) to monitor and control the flow of water through the system. Because SCADA is so expensive and complex to install, program and maintain, SCADA is rarely used by small water operators. However, when SCADA is used for small water systems, it tends to be vulnerable to a number of failure scenarios. What happens when these events conspire to kill the PC at the water system location?
Recently, a small water district in Northern California was literally burned to the ground by a huge wild fire. Not only did they lose the outbuildings where the wells were, they lost the main district office where the treatment equipment was housed. And where the SCADA system was located.
Because SCADA systems are PC-based, the PC needs to be at the plant site. All operational data is stored on the PC. When a natural disaster strikes and the PC is damaged, it is often a fatal problem for the SCADA system. Backups did them no good, as we shall see later.
There are bad things far more common than natural disasters, however.
All it takes is one click on one link inside an e- mail on on a website for malware to be installed on the water system PC. The PC can then be commandeered to serve as a spam distribution node, upload any available data and e-mail lists. Or worse. The PC can be locked up and held for ransom. Or a hacker from any location in the world can get into the SCADA system and perform mischief on the water system itself.
The PC Treadmill
Because SCADA systems are PC-based, the SCADA software runs on a PC operating system such as Windows XP, Windows Vista, Windows 7, Windows 8 and so-on. PC’s are plagued by security problems, thus bi-weekly security patches are essential, especially for SCADA systems where they are often exposed to outside networks such as the Internet. Windows XP was dropped by MicroSoft support nearly two years ago, but a significant percentage of SCADA systems are still running under Windows XP!
Why wouldn’t people simply upgrade to a newer PC operating system? This is where the PC treadmill becomes apparent. Often, the new operating system will not run on the old hardware. And often, the hardware is five years old, which is a lifetime in PC terms. So, the hardware needs to be replaced.
Unfortunately, the SCADA software companies conveniently design their PC-SCADA software so that it will not run on a newer version of the PC operating system. This is not a total conspiracy since the SCADA software has to connect at a very low level in the PC operating system and those parts often change with each new PC operating system. So, you need to upgrade the SCADA software. This can costs many tens of thousands of dollars. And then you have to install, configure and then program the expensive new PC-SCADA software since the old SCADA program will often not run on the new SCADA software package. Guaranteed that you must purchase a lot of onsite expert consulting time. This is why we call it a treadmill!
What can a small water system operator do? Small water systems usually do not have an IT (Information Technology) department. The result is that this lone SCADA PC sits in some pumphouse or shed and runs with no effective backup or protection from failure. It’s not clear that backing up the PC would really help because when the PC fails, you really can’t fix it. You buy a new one. A new PC is purchased and the treadmill continues: the old backups may not not really be that useful.
When the PC fails, the small water system really has to pull out the checkbook and pay a huge sum. To add insult to injury, the historical information from the old SCADA system often cannot be imported into the new SCADA system because it’s new! And better! And different. And not compatible with the old system. Yes, consultants can be hired to do data migration to the newer platform, but the cost can be prohibitive if it’s even possible.
Natural Disasters just compound the SCADA PC problem
When our Northern California small water district was hit by the recent disastrous wildfires, they lost everything. They lost their SCADA PC. They lost all the data in the PC. They lost the SCADA programs. The install disks. And they lost a lot of plant manuals, plant operation information and years of hand-written data. They effectively lost their SCADA system along with their water plant.
The XiO Cloud SCADA® control system is entirely different
At XiO, our business is control systems for small water operators. We deliver better-than-old- SCADA performance for a tiny fraction of the price to mid-sized and very small water operations. As the name implies, the XiO Cloud SCADA system is Cloud Based meaning that data from the local site is immediately sent to the Cloud which consists of multiple redundant servers managed by some of the largest companies in the world. Therefore there is no local data to be lost in the event of a disaster.
At the same time, the XiO hardware onsite at the water company is simple and its program is held in the same cloud servers ready to be deployed to new hardware and sent to the field quickly. Therefore, if the local XiO hardware were burned up, new hardware could be sent to the field that would exactly replicate the burned up hardware. The new hardware would come complete with its program. When the new hardware is powered up, it sends its data to the XiO Cloud where this new data augments all the old data. Yes, there will be a gap in the data, but presumably, there was a corresponding gap in the water delivery as well!
As a result, the XiO customer has the peace of mind that their data is backed up all the time. Their data is held offsite. It can be downloaded and stored by the XiO customer as desired. In addition, the programs that run the water system are also archived offsite so that they can be replicated and shipped to site quickly.
Remarkably, the XiO Cloud SCADA system ongoing operational cost can be a few percent of old SCADA technology because there is no PC Treadmill, no constant upgrade cycle. Just a single, low monthly Cloud Service Fee.
XiO customers also have the ability to put important documents and instruction manuals on their secure XiO cloud website. How do you calibrate the pH meter? You can use your smartphone to download the correct manual with notes and instruction unique to the water company. Drawings, plans, procedures. They can all be stored securely where local disasters will not affect them.
And there are no PC’s in the XiO architecture! On site, there is only the elegant XiO Field Installable Unit (FIU). This small NEMA4X enclosure contains the controller and its software. This entire FIU can be quickly replicated and shipped to site after a natural disaster. And the cloud based data is not affected by the local disaster.
What do XiO customers do when their (old) water system PC fails? They haul it to the recycler!