Tuesday, September 22, 2009

Aren't alarms alarming?

It always amazes me how poorly alarms are thought out and executed. It seems to me that well executed alarm code requires both a thorough understanding of the process under control and the relative importance of any deviation. Most alarm code that I have reviewed is like poorly choked birdshot. The philosophy seems to be grab everything that seems slightly out of design spec and bombard the operator with zillions of messages!

I am always amazed at the dexterity of the poor operator who can hit the acknowledge/clear button with a rapidity that is truly awesome! Then there is the alarm printer that jammed or off-line with reaps of useless garbage stacked on the floor. In either case it is quite likely that any real alarms that may identify the root cause of plant outage are never noticed or ignored. Even if the alarms are posted to log file and not to decimating forests, they still create a search nightmare. The operations/maintenance staff lose confidence in the SCADA or OI and attack the problem in some other empirical fashion.

The list goes on ...and on...
  • hardcoded limits and times that make it impossible to allow for valid process changes;
  • mixing up routine events (NOT actual alarms) with alarms;
  • 'clever' use of set-points that are meant to operate valves and pumps, but with the addition of a timer also produce an alarm;
  • seven or more levels of set-points so that you have to create tags like LA4H-725 to accommodate the rare situation when - well, you fill in the situation;
  • software suppliers who allow the creation of alarm log files, but use a binary format that is not searchable by any tool that a normal non-geek human can use;
  • use of a dozen or more colours and shades to distinguish different types of alarms (Let's use light magenta to identify problems with acid feed system) (More on this in another post);
  • meaningless or cryptic alarm messages, e.g. "PAH-123 High Pressure". This needs more description beyond the tag, like "PAH-123 High Feed Pressure at RO Inlet #1";
In reality you can cover nearly everything with two types of alarms. Let's call the first warning variety "Alerts". These should be colour-coded YELLOW and offer advice about situations that are somewhat outside the normal operating band. A nice is example is pump that is running at higher than normal rpms and producing normal flow. At some point this will mean the need for a rebuild, but not just yet. The warning gives the operator some time to schedule maintenance on the pump. Remember the flow is still okay, but the pump is working harder than it should. This type of alarm implies knowledge of the pump curve and how to code deviations, but it can prevent an unscheduled outage and will definitely pay for itself.

Aside from alerts, there are actual alarms. These are coded RED and indicate that some piece of equipment has failed and shut one or more parts of the process. For me the terms 'Alert' and 'Alarm' are very clear. They correspond to yellow caution and red stop lights on traffic signals and I believe are universally understood using the analogy. They are also nice short words and are alliterative to boot! I understand that some customers don't like these terms and find them confusing. I have never met any of these people. Whenever I have been training customers, I offer the traffic light analogy and they get it right away. Still there might be some valid reason to go in another direction. Let me know if you come across it.

(Aside - while the RED is dangerous and GREEN is safe standard is entirely consistent and logical, it is intuitively opposite to any one who has motor vehicle experience. Why confuse people unnecessarily?? Just asking. You might have a valid reason, but I have not been convinced so far. I have used both according to customer spec, but still.)

What is the bottom line? Make sure you communicate clearly what the problem is, when it happened and if possible some readily available advice on what to do to fix the problem. This brings up the subject of interlocks, which will be the subject another post.

If you find this post alarming, or stupid, start a discussion. It might be fun!

Friday, September 18, 2009

Discrete Controls and HMI Complexities

Since common HMI Status displays refer to integer variables, it is common to add extra PLC code and special registers to pass the correct integer value to the display status. A typical example would be:
  • Auto Off = 0
  • Auto On = 1
  • Manual Off = 2
  • Manual On = 3
  • Fault = 4
  • MCC Local Off = 5
  • MCC Local On = 6
  • etc.
This is all very nice in terms of easy to understand code, but it is wasteful of PLC resources and completely unnecessary.

Why not instead assign a byte or two to the control element and then assign individual bits to each discrete function, for example:

  • Bit 0 = Auto (0)/Manual (1)
  • Bit 1 = Contact Output
  • Bit 2 = Aux Contact Input
  • Bit 3 = Fault Contact
  • Bit 4 = Local (0)/Remote (1)
  • Bits 5 to 7 not used in this example
Then assign the HMI status to read the same PLC variable and interpret the byte with the following statuses:

  • Auto Off = 0
  • Auto Starting = 2
  • Auto Open/Running = 6
  • Manual Off = 1
  • Manual Starting = 3
  • Manual Open/Running = 7
  • To add Fault status to any of the above add 8
  • To add Remote Status to any of the above add 16
There is no rule that says HMI status has to be some simple sequence of numbers. The approach I have described is a little more complex to map out, but it is easily tested and once you have the first one working, it is a piece of cake to pull the standard status indicator from the library and assign the appropriate PLC variable to it. There is zero code in the PLC aside from the simple few discrete rungs that you need to drive the device.

You get smaller fast PLC code and less data to transfer to the HMI. What are you waiting for? Just do it!

Wednesday, September 16, 2009

Logic and Controls Design

It is interesting to me how many people involved in the design and coding of PLC and SCADA programs have no formal training (or any apparent understanding) of the simple rules of combinational and sequential logic. The basics of AND, OR, NOT and minimization techniques that are offered in any Logic 101 course are completely unknown to these people and yet they want to create complex applications to do real-functions like controlling water treatment plants or manufacturing lines. It 'bloggles' the mind!

If you are responsible for such teams. Make sure that each member of your design and programming teams has a working understanding of these basic elements of design and uses them consistently. It will pay big dividends for your margin, your customer's satisfaction and ultimately minimize life-cycle costs for the system. Oh and did I mention about the benefits when it comes to adding functionality or make minor changes?

Remember that the plant operator 'sees' the process primarily through the Operator Interface that you provide. You need to do everything humanly possible to provide a clear accurate window of what is going on in those tanks, pipes, reactor, mixers and whatever other unit processes are involved, but that is the subject for another posting.