Standardized Assessment

Print Page
Risk assessment is not “rocket science.” Actually it is partly, but as for the expression, it does not have to be very complicated. It is a reasoned and disciplined approach to better understanding risk and more easily seeing how one proposed solution compares with another.

The process is often described as comparing, benefit to risk.

Three terms that get tossed around a lot are, subjective, objective, and arbitrary: “You’re not being objective!” “That’s just your subjective point of view, and it’s so arbitrary!”

Subjectivity, objectivity, and arbitrariness will all serve a purpose in making assessments. I will use objective mostly to refer to data, factual information that can often be expressed in numbers, like vehicles per hour, vehicle speed, or length of a walkway. “Subjective” is how that data can be interpreted: 40 MPH is too fast for this road; a 40 foot cross walk is excessively long. And being “arbitrary” in this case could be setting a speed of 35 MPH instead of 34 or 36 (ignoring for the moment state standards).

Subjective and arbitrary variables are one reason why charts and graphs are useful: they make it easier to get a picture of the issue, make comparisons, and grasp the relative meaning of what might be a complex situation. It makes comparisons easier to see, understand, and appreciate.

Assessment example

Traffic intersection risks can be affected by a number of things: the speed of vehicles, numbers of pedestrians, crosswalk length. Some risk factors can be represented factually, using measurements and numbers.

In this example, the chart represents all the possible combinations of risk factors for a T intersection. It is heavily trafficked, with a crosswalk.

The assessment should focus on those things that can produce the greatest harm or loss. Here, it is vehicle speed and length of the crosswalk.

The goal of performing risk assessments is to reduce the likelihood of causing harm, or if you prefer, to increase the benefit to be realized.

Four risk factors for pedestrians, plus a fifth

  1. Number of vehicles passing each hour.
  2. Number of pedestrians crossing during each hour (may be relevant as in the use of crossing guards).
  3. Speed of vehicles: faster = greater risk.
  4. Distance pedestrians travel to cross the street:
    longer = greater risk.

You might be thinking: “Waaaait just a minute. I didn’t sign up for some logic class in obsessive thinking!” And for that, I can sympathize. Shown above is a complete list of every possible combination of risk factors; it’s just to emphasize the function of risk factor variables.

In practice, a list will look much simpler – focusing on the critical factors, vehicle speed, and length of crosswalk. All the risk factors are still shown, but only those deemed most significant are rated.

In this case, shortening the length of the crosswalk – a major risk factor – was chosen to reduce the risk of pedestrian injury.

The method used was to square off the intersection (blackened triangle in illustration). Besides reducing pedestrian risk, making the intersection into a clear T formation also makes it clearer to drivers which roadway is the main thoroughfare.

The fifth risk factor is apparent – how apparent the risk is to those involved, drivers and pedestrians. If one can easily see the risk, that risk may be more easily avoided. If one cannot easily see the risk, that factor may be more significant. How apparent a risk is, is one way of assessing the likelihood of occurrence.

We have identified the risks to pedestrians, and have presented one way to lessen the risk. What about cost? Reducing risk will often be measured against cost. Does the benefit of reducing risk outweigh the downside of spending tax dollars?

In analyzing a situation, cost will be measured against the likelihood of something bad happening, as well as the severity of that bad thing: the greater the likelihood and severity of injury, the more willing we will be to consider spending money to lessen that risk.

The past does not predict the future

Past history can be tough to argue against: “Why are we looking at this issue, nothing bad has ever happened!”

The problem with relying on past experience is this: When an assessment is not based on risk factors, it’s an opinion, in this case, based on past performance. Facts of the past do not address the physical characteristics that make up the situation in the present.

It would be like saying the reason a bridge is not falling down today is because it never has. We know the reason it is still standing has to do with the current physical condition of the structure: level of corrosion, metal fatigue, road surface deterioration, and so on. The fact that it has not fallen down has no relation to those things.

However, past events can indicate that we should look at a situation’s risk factors: It’s easy to see that if accidents often occur at a particular intersection, maybe we should find out why by performing a risk assessment.

Before two Boeing 737 MAX planes crashed, everyone knew it was one of the safest commercial aircraft ever built: excellent safety record. After the crashes, well what do you know – when everyone took a careful look, many safety issues surfaced (aka, risk factors). It was no longer considered a very safe aircraft to fly or be on.

Boeing executives chose to believe that the plane’s safety record was more relevant than the current risk factors: of the need for air speed sensor redundancy and pilot retraining. Boeing’s beliefs were not a result of assessing risk, but were based on magical thinking: the aircraft would continue to be safe to fly because it always had been.

That is what they wanted to believe, or some of the executives wanted to. Others in the company were not so sure, but their voices were drowned out (something a risk management group can help avoid). One problem in bringing out the critical safety issues that downed the aircraft was the top administration’s deaf ear. The other was in not relying on a standardized assessment process and reporting.

There are two different kinds of past events and reactions.

  1. Nothing bad has ever happened – so why take any action?
  2. Something bad has happened – so I must consider corrective action.

Neither of these represent actual risk, or can be a risk factor. The fact that a particular situation has never harmed anyone is, by itself, no guarantee that things can not happen. And just because something bad has happened is, by itself, no guarantee it will ever happen again. The past is an event, not a condition. To use the past as a risk factor is a mistake because present circumstances can be missed, or misused.

It is critically important to distinguish between physical circumstances that put people at risk – risk factors – and how often those circumstances produce harmful effects. Like the stock market, past performance is no indicator of future results.

Management basics

Define the assessment process in a way that permits clear comparisons between different circumstances or events.

Ask questions: What is the situation that is of concern? What are the risk factors? What are the consequences or the degree of harm or loss? What is the likelihood of harm or loss occurring?

Explain the decision to take corrective action, or not to.

Document the process so decisions are made clearly and understood by everyone – how the level of risk relates to any proposed fix.

The alternative to a standardized approach is haphazard analysis that can be prone to biases and individual personalities. Risk assessment is fact-based, and management is a structured approach that includes subjective observation, but clarifies the reasoning.

Different name, same game

Dr. Atul Gawande, a Harvard-trained surgeon, wrote about the use of checklists in The New Yorker magazine. In “The Checklist Manifesto: How to Get Things Right” (2007), he wrote about the process of using them and whether or not – in hospitals or prior to aircraft takeoffs – the assurance that basic information is not ignored, reduces accidents.

How did it work? “We get better results,” he says. “Massively better results.” “We caught basic mistakes and some of that stupid stuff,” Gawande reports. But the study returned some surprising results: “We also found that good teamwork required certain things that we missed very frequently.” Like making sure everyone in the operating room knows each other by name. When introductions were made before a surgery, Gawande says, the average number of complications and deaths dipped by 35 percent.

Call it checklist, or call it risk management: following a formal procedure that defines what the issue is, and how aspects of it can influence outcomes, helps to assure better outcomes. And that last point about introductions is worth repeating: creating a formal risk management group can promote better relationships.

Standardized assessment means that no one in a position of responsibility will have to rely on excuses.

Next Section