Background
The complexity and sophistication of supervisory and control systems is growing at a steady rate to keep up with the need for production information and efficiency. At the same time, these systems need access to sensors and instrumentation through equally sophisticated PLCs so that they can log, measure and control events in real time. So we can say without exaggeration that industrial automation and industrial IT is not for the faint-hearted. In fact, a “data processing manager” of the latter half of last century would pale when considering what instrumentation engineers, production managers and others have to deal with today – because even he wouldn’t know what was going on.
That’s why it’s become vital to find people that could integrate production knowledge with system knowledge – such a person is Jacques Jansen, Solution Analyst, Automation, Exxaro Grootegeluk. He makes a point that, although they have a great knowledge of their industrial automation infrastructure (Exxaro implemented their systems with minimal assistance from system integrators), their expertise lies in coal beneficiation and not in the quirks of the latest operating system or monitoring scan overruns.
“One of the biggest problems we have in the automation environment has to do with changes,” says Jansen. “While everything seems to be operating well with a stable system, someone could have justifiably changed a system parameter whose effect isn’t immediately obvious but that eventually results in a problem over time. We don’t have the resources and skills on site to deal with such problems.”
Another change is the issue of system updates where IT personnel would update existing operating systems without regard as to what updates and patches were in place.
“All these changes would slow down the systems and even cause expensive downtime leaving the operators to ask for help,” says Jansen. “We’re looking to expand our operations in the Waterberg but with skills and resources becoming scarcer and technicians stretched to their limit looking after the plant, we decided to look for a proactive system monitoring tool that could detect issues before they cause downtime.”
Implementation
The implementation took place between January and February 2013 and included installing monitoring agents at each key node of the system. The first priority was to establish secure internet remote access through a dedicated port with only one-way traffic. Next on the agenda was a system assessment which included a system health audit followed by recommendations from IS³ and a stabilisation project.
“The system assessment revealed a number of issues such as excessive disc and CPU usage, scan overruns, exceeding best-practice thresholds and several other problems which first had to be addressed before routine monitoring could begin,” says Jansen. “Apart from the real-time alerts from the AVEVA software provided by IS³, we also meet once a month to go through an extensive report not only to discuss what happened but also to look at developments and recommendations as to where we should be heading. This is a very thorough report. We saw problems with the number of alarms operators were required to handle and we even detected an error in the configuration on a virtual machine that caused things to slow down unacceptably. These meetings help us to optimise the software life cycle, upgrade our roadmap for the future and review our alarming and security philosophies.”
