Chemical Safety Board Report: Alarm Flooding

In 2022 the BP-Husky refinery in Toledo, Ohio experienced a large fire in which two men died. In this series of posts we discuss the valuable lessons that we can learn from the Chemical Safety Board (CSB) report to do with that event.

Previous Posts

Posts in this series to date are:

1. Chemical Safety Board Report: Naphtha Release and Fire. An introduction to the CSB’s report on this accident.

2. Chemical Safety Board Report: A Wicked Problem. We note that the refinery had implemented standard hazard analysis procedures and practices. These included Process Hazards Analyses (PHAs), Layers of Protection Analysis (LOPA), and Safety Instrumented Systems (SIS). Nevertheless, two men died. In that post we suggest that new ways of thinking are called for. The Wicked Problem approach is one option.

3. Chemical Safety Board Report: Stop Work Authority. We draw a distinction between ‘Stop Work’ and ‘Stop the Process’.

4. Chemical Safety Board Report: Training or Instrumentation ― Which Is It?

5. Chemical Safety Board Report: Abnormal Situation Management.

Alarm Flooding

The CSB report placed emphasis on the problem of alarm flooding. The preamble to the report states,

Board operators at the BP Toledo Refinery were receiving far more than 10 alarms in 10 minutes on average, a situation in which more alarms were annunciating than a human can effectively respond to, for nearly 12 hours preceding the incident . . .

Continued operation in an alarm flood state contributed to the incident by causing delays and errors in responding to critical alarms and shift-to-shift communications. Had the Tuesday, September 20, 2022, night shift board operators been less overloaded with alarms, they might have identified that the Coker Gas Plant Absorber Stripper Tower was overflowing naphtha . . .

The report has two recommendations on the topic.

2022-01-I-OH-R4
Revise the ‘Toledo Alarm Philosophy’ by incorporating the Engineering Equipment and Manufacturers Users Association (EEMUA) guidance for alarm rate following an upset and not limiting alarm performance to a single metric averaged over a month. In addition to including analyzing individual alarm flood events, the revised philosophy document should improve refinery alarm performance to reduce alarm flood duration and peak rate . . .

The second recommendation is directed to the International Society of Automation (ISA).

2022-01-I-OH-R7
Revise American National Standard ANSI/ISA 18.2-2016, Management of Alarm Systems for the Process Industries, to include performance targets for short-term alarm flood analysis so that users can evaluate alarm flood performance for a single alarm flood event. The performance targets should include:

a) number of alarm floods,
b) duration of each flood,
c) alarm count in each flood, and
d) peak alarm rate for each flood.

At a minimum, a target peak alarm flood rate should be defined, such as in the guidance provided by the ASM Consortium or Engineering Equipment and Materials Users Association (EEMUA), to establish trigger points that require alarm performance improvement actions.

Principles of Alarm Management

The principles of alarm management are described in detail in various engineering standards, including the EEMUA and ISA publications described below. Some general thoughts are provided here.

• Purpose
  The purpose of an alarm is to notify operators that the process is going or is outside its safe limits. Human intervention is called for. The alarm system lies between the basic control system and the safety instrumented system.

• Distinguish between Operating and Safety Alarms
  Some processes use alarms as part of their routine operations. For example, a batch process may require a vessel to be filled with a liquid every hour. An operating alarm will tell the operator that the vessel is full. A safety alarm on the same vessel will warn of potential overflow.

• Defined Response

  The alarm should call on an operator to take a defined action. He or she should then have sufficient time to carry out a defined response before the condition escalates.

• Ownership
  There should be no confusion as to who is meant to respond to an alarm, particularly with respect to unit interfaces and utilities such as steam, electrical power and cooling water.

• Frequency

  A general rule is that there should not be more than one alarm every ten minutes. The CSB report states that, ‘Between 6:50 a.m. and 6:49 p.m. September 20,

  2022, a total of 3,712 alarms were recorded’. This translates to about fifty alarms every ten minutes.

• Disabled Alarms

  Alarms should only be disabled as part of the Management of Change program. Alarms that are permanently disabled are a cause of (ahem) alarm.

• Human Factors

  The alarm system should be accommodate human capabilities and limitations.

• Alarm Sequence

  The sequence in which alarms annunciate should be recorded. This will help with any subsequent incident investigation. The sequence record can also be analyzed to determine ways of improving the operation.

• Process Hazards Analyses
  It is all too easy for a PHA/HAZOP team to recommend adding an alarm after a high risk hazard has been identified. In the days of panel board systems, alarms were regulated by both size and cost. There was limited board space and the cost of installing the alarm was significant. Sometimes, if a new alarm was needed, an old one had to be given up. With modern digital systems adding an alarm has become quick and easy. It is really just a matter of typing. This means, therefore, that the PHA team has be judicious when it comes to recommending more alarms.

Industrial Standards

Two important industrial standards to do with alarm management are provided by the EEMUA and the ISA.

EEMUA Publication 191, Chapter 6.5.1

The Engineering Equipment and Materials Users Association is a non-profit membership organisation for users of engineering equipment and materials. The title of EEMUA Publication 191 is Alarm systems: Guide to design, management and procurement. It has seven chapters and 20 appendices. The title of Chapter 6 is ‘Performance monitoring/improvement’.

ANSI/ISA 18.2-2016

The title of this document is ‘ANSI/ISA-18.2-2016, Management of Alarm Systems for the Process Industries’. As we have seen, the CSB identifies areas where they believe that this standard could be upgraded.