Aircraft Accidents and Lessons Unlearned XL: American Airlines 587

American Airlines 587’s Vertical Stabilizer

On November 12, 2001, American Airlines flight 587 (AA587), an Airbus A300-605R, registration number N14053, crashed in Belle Harbor, New York shortly after taking off from John F. Kennedy airport. Recorded as National Transportation Safety Board (NTSB) accident number DCA02MA001, the A300 lost its rudder and vertical stabilizer during climb; the aerodynamic loads caused the loss of both engines before impact. Just two months after the September 11, 2001, terrorist attacks, the accident was thought to be terrorism, a case for the Federal Bureau of Investigation, but when it was determined to be non-terrorism in nature the NTSB shortly took over the investigation. The accident report, AAR-04/04, was adopted on October 26, 2004, almost three years later.

The NTSB determined that, “… the probable cause of this accident was the in-flight separation of the vertical stabilizer as a result of the loads beyond ultimate design that were created by the first officer’s unnecessary and excessive rudder pedal inputs. Contributing to these rudder pedal inputs were characteristics of the Airbus A300-600 rudder system design and elements of the American Airlines Advanced Aircraft Maneuvering Program (AAMP).”

This accident demonstrated the NTSB was ill-equipped for the technical and industry knowledge challenges of this investigation. This was my first major accident as an NTSB major accident investigator; I was also the only person in the NTSB who had, not only directly worked with composite materials (and I was no expert), but had worked the A300-600 series airliner. The NTSB’s Structures investigators’ inexperience with Composites was concerning; the Structures group should have pushed management for training on composites to better understand the important safety issues they eventually missed. Furthermore, analysis of Air Transat flight 961, A310-308 investigation, made it obvious how NTSB accident report AAR-04/04 did not capture the root failures that led to the AA587 accident.

Question: Could NTSB Board members, with no airline Operational or Technical experience, base a report’s conclusions on the insufficient skills of NTSB investigators long out of the industry? The Board Members faulted American Airlines’ AAMP, then placed full responsibility for AA587 on a qualified pilot with over a decade of Part 121 flight crew experience. According to AAR-04/04, 2.4.3 Summary, “First, the first officer had a predisposition to overreact [underline added] to wake turbulence encounters.” Who at the NTSB could speak to the FO’s ‘predisposition’; that the NTSB assumed the first officer (FO) was careless and unskilled? How did NTSB Board Members and investigators, lacking a career in piloting jet airliners, reach this embellished opinion of a professional they could not understand? Answer: Another captain, whose intentions were never questioned, said so. The FO had flown with this captain … four years earlier … on a B727 …  This captain was interviewed, stated his opinion, of the FO’s “… use of the [B727] rudder pedals was ‘quite aggressive’” (AAR-04/04 page 12).

What did this misdirection of a biased interview do? For one, this biased captain’s opinion placed blame on a professional – the FO – without proof. Second, the misplaced blame distracted attention away from the true cause, which in the case of AA587, was never determined.

What did happen? In 2003, the Allied Pilots Association, parties to the investigation, submitted their report to contradict the NTSB report. It stated, “And the Board found that the composite material used in constructing the vertical stabilizer was not a factor in the accident because the tail failed well beyond its certified design limits.” Did the NTSB physically test an A300 composite vertical stabilizer and rudder? During set-up for the structural tests, a lone A300 vertical stabilizer was found to be available in Asia. However, during shipping, the stabilizer came free of securing, left its pallet and fell overboard into the ocean. The test stabilizer became too structurally damaged and compromised with sea water. Instead, the NTSB was forced to resort to Finite Element Analysis (FEA), a computerized method for predicting how a product reacts to forces; and Progressive Failure Analysis (PFA), which uses C-one shell elements based on classical lamination theory to calculate the in-plane stress. In other words, the tests were accomplished without a static computer model to test on. But were these methods accurate?

No – for three reasons. First, when an aircraft is certified, structural stress exercises are accomplished, such as when the wingtips are bent upwards to see at what deflection the wings will snap, exceeding their physical limits. The manufacturer does not use a computer model for its products’ stress tests because the computer cannot duplicate the flaws unintentionally built into a product.

Second, the increased use of composite structures in aircraft components, such as all flight controls, was relatively new at the time of AA587. Industry mastered composites repairs – after – the composite aircraft were already in service … not before. Detecting composite material defects was an evolving science. Third, the NTSB did not have structural engineers with composite experience. Instead, the NTSB relied too heavily on Airbus to police their own product’s integrity.

What events occurred in the accident aircraft’s history? The accident aircraft, serial number (S/N) 420, was issued its airworthiness certificate on July 13, 1988 and it had flown for thirteen years before the accident. During the initial post-accident maintenance review, records showed that S/N 420 had ‘sat’ on its tail, overstressed the empennage.  This Maintenance Chairman’s Factual Report (MX Factual) should have recorded this. Aircraft ‘sit’ when a tail-heavy weight imbalance occurs, such as fuel in the stabilizer fuel tank. The tail could have struck the ground with great force, enough to damage the vertical stabilizer mounts or rudder hinges. Fact: an aircraft is not designed to ‘sit’; undetected structural flaws could form within the composite layers of components. Did such damage occur to S/N 420, the accident aircraft?

S/N 420 also had nine vertical stabilizer and/or rudder modifications (MX Factual page 12). Could any or could all of the nine modifications have introduced damage within the composite layers? Why did the NTSB fail to give structural integrity proper attention? Did the NTSB give full attention to the composite inspection programs, not only of American Airlines, but of all A300 operators? Where the AAR-04/04 investigation failed, the Transportation Safety Board of Canada (TSBC) accident report A05F0047, Air Transat flight 961 (AT961), found answers.

On March 6, 2005, AT961, an A310-308, was at Cruise approaching 35,000 feet of altitude (by contrast, AA587 was below 2000 feet and at Climb) when AT961’s rudder separated from the vertical stabilizer. The TSBC did not waste resources creating easy solutions, such as wake turbulence or pilot error. Instead the TSBC investigated structural integrity of the vertical stabilizer, the attach points for the rudder and maintenance programs that inspected the composite components’ integrity and the effects of various harmful agents, such as water/ice, deicing practices and chemicals.

The TSBC embarked on an in-depth series of structural examinations, in some cases using sister rudders designed and built to the same specifications as the separated rudder. The TSBC went beyond tests conducted by the manufacturer at rudder certification, such as static load, fatigue, damage tolerance and flutter tests. They thoroughly examined the rudder maintenance inspection program; subjected the test rudders and original vertical stabilizer to chemical and contamination tests, elasticity laminate tests, impact damage testing and bonding tests. The TSBC understood that composites, in relation to the industry, was a young science and that there was much to be learned, much to improve upon.

Where the TSBC exceeded the NTSB in report writing can still be seen in investigations today. For instance, the investigatory methods employed in the 1960s did nothing when investigating the B737-MAX accidents; the use of unqualified/inexperienced investigators and outdated investigatory practices brought the industry no closer to safety with the B737-MAX and only allowed the unsafe practices at Lion Air and Ethiopian Air to continue unimpeded.

Meanwhile, it was hoped that Airbus, knowing that AA587 received a less-than-adequate investigation, took it upon itself to build integrity into its composite products, increased the quality of maintenance structural inspections/testing and learned what was ignored during the AA587 investigation. Despite the errors of AAR-04/04, Industry learned; operators of Airbus products (and Boeing products) learned what was missed with AA587: how to design an aircraft to operate for its lifetime with some newfound safety built-in. Hopefully, someone took the initiative to get it right.

Aircraft Accidents and Going Virtual

Aircraft Maintenance Instruction in the Future

There is an old saying in the Education field, “There is no such thing as a stupid question.” For many years that line has held true. Despite this adage, it would be argued by me that, “The only stupid question is the one never asked …until it is too late.” And that is just not right.

In the (hopefully) final days of the COVID-19 scare, ideas have been presented in the aviation industry that aviation schools should move to decrease stand-alone (instructor classroom) instruction and replace/increase virtual instruction (VI), where students attend class through personal computers and the internet. It is floated as a ‘back-up’ plan, just in case, COVID continues to hang around, like a viral version of Jar-Jar Binks. COVID is the mystical magical virus that will suddenly disappear around Thanksgiving 2020, where someone at the table is bound to say, “Hey, do you remember a couple of weeks ago when we had to wear masks and maintain social distancing? Strange, huh?” Yeah, strange.

Back-up plans never seem to go away. Not emergency plans, like how to survive a Chicxulub asteroid strike, but back-up plans. Some back-up plans are good, e.g. buying a generator because of New Hampshire ice storms, while other back-up plans are not. They evolve into the norm, what we must do if we, as a society, are expected to survive the … (insert emotionally packed, crazy reason here). And that is the shame of it, that aviation, as one of the most noble industries the past century produced, might abandon common sense Training/Instructing just because we are afraid … no, conditioned, to fear being in the same room with more than three people.

My wife teaches 7th and 8th grade students. This past March the middle and high schools shut their brick-and-mortar doors; teachers taught virtually. There was a learning curve for high school age students as VI was introduced. The big negatives for VI were not the teachers’ instruction obstacles; they were student discipline. Parents, suddenly stuck in the same house with their little angels, surrendered to every whine and temper tantrum. The Department of Education, in its questionable wisdom, decided no student could fail. 7th through 12th grade students did not test because VI teaching upset their tender sensibilities. Video game proficiency replaced class work. And they all graduated to the next grade.

Colleges did not have better luck; thousands of students were restricted to the dorms for the closing months of the semester. Summer enrollment dropped. Classroom instruction adapted to the VI concept, which lacked the quality of classroom instruction. The problem was that high school and college students lost the structure that school schedules offered. Did some of these high school and college students engage in the civil unrest that has rocked our country since May? Could be. Would major cities have burned if students were in school? It is likely that students who studied … say, American History, would not have attacked Memorials like the 54th Regiment of Massachusetts for being … racist (???).

College age youth, fresh from high school, attend Trade Schools, e.g. aircraft maintenance (A&P) schools. Per Title 14 Code of Federal Regulations (CFR), Part 147.21 General Curriculum Requirements states that to acquire an Airframe (A) certificate requires 750 hours of Airframe-specific classroom instruction; a Powerplant (P) certificate requires 750 hours of Powerplant-specific classroom instruction. Both or either require 400 hours of General classroom instruction. Each A&P technician today had received 1900 hours of certification classroom instruction. Would A&P schools lobby the Federal Aviation Administration (FAA) to ease up on the required face-to-face classroom attendance hours in favor of VI, to prevent exposure to COVID-20, -21, -22 …?

For the time being, tech schools are not – cannot – eliminate the brick and mortar school. Why? Because there are still required hands-on labs. But, how far off are virtual labs? The technology exists to sit on a Florida beach while living in the Arctic, how long before we substitute real engine rebuilding with virtual versions in the lab. What would be the benefit of that?

In truth, brick and mortar buildings are disappearing. Companies like Circuit City have surrendered their classic building showroom for the cost-savings, e.g. rent, employee benefits, warehousing space, etc. for the dot-com. No doubt brick-and-mortar represents a heavy cost for the retail industry, but aviation is not a retail format. No links to click or search engines to use. One must turn a wrench or fly by instruments. Why? Because learning comes from INTERACTION; there is no substitute for it. One must get their hands dirty in order to learn. Virtual is too easy and it is not right.

Back in the early 1990s, my airline invested heavily in initial computer-based training (CBT) – an early version of VI – for its hundreds of pilots and mechanics. The airline converted a large room; one side of the room had dozens of computer monitors for pilots to train, the other side had dozens of monitors for mechanics and a small hall separated both kingdoms. If a monitor was inactive for fifteen minutes, a company logo screen saver bounced across the monitor. Many nights after work, I stood in the hallway, glancing left and right, seeing dozens of sleeping employees, as the company logo lazily bounced across the monitors. Despite happening decades ago, strict CBT was ineffective; it lacked INTERACTION.

The classroom instructions I took since the Eighties as an aircraft mechanic, as a National Transportation Safety Board investigator, as an FAA inspector, they were effective, not only because of the Instructors’ experiences, but because of the discussions that took place between students about the lesson material. Even as an A&P instructor, as an instructor at the FAA Academy or the NTSB Academy, learning was enhanced because students and instructors learned by speaking about their experiences, by interacting, by exchanging ideas and solutions.

Question: How would it be decided we should get back to pre-COVID stand-alone school instruction? Who decides? Would the trade schools push to go back? The problem is that once a regulation exemption is granted, how do we put the toothpaste back in the tube? Look what fun we are having with state Governors refusing to return to any semblance of pre-COVID-19 economies. Or maybe we would trust the hundreds of conflicting health experts who cannot remember what they said from Monday to Wednesday.

I have taught stand-alone classes; I have seen the conversion from stand-alone to VI. My opinion is it would not work, especially with high school graduates; they lack the discipline to learn in a VI trade school environment. Furthermore, they would not know how to transfer from a VI environment to the workforce; their work ethic would be irreparable. Students would be confined in four wall sterile environments, scattered across many miles, distracted by their cellphones and social media. Instructors would answer questions via texts with no controls, no discipline. Any Interaction would become non-existent.

Could regulation exemptions be abused? Sure, non-exempted regulations are abused all the time. The FAA exists to oversee certificate holders, e.g. technical schools, to assure regulations – the laws – are followed. Over a decade ago an A&P school was shut down, its owners imprisoned. Why? Because they were graduating unqualified technicians with A&P certificates. Aside from other issues, many of the new A&Ps could not speak, read or write English. Title 14 CFR 65.71(a)(2), Eligibility Requirements: General, has stated (since 1962), “To be eligible for a mechanic certificate [A and/or P] and associated ratings, a person must — be able to read, write, speak and understand the English language, or in the case of an applicant … certificate endorsed ‘Valid only outside the United States.’” This is the law. English is our primary language. By abusing the regulation, the guilty school made our industry unsafe.

Unfortunately, for a certificate holder to save money, the temptation to break the law can be a driving force. How much easier would it be to work around the system if the FAA approves VI on a permanent basis? Who would be flying our airliners? Who would be maintaining them? Would the flying public trust aviation again?

VI would be the easy thing to do. But would it be the right thing to do? We would end up with a generation of pilots and mechanics socially incapable of working with others, making undisciplined decisions without making the required effort. We would have a workforce with a high school mentality; we would have nothing. And that is just not right.

Nothing is easy. But who wants nothing?” President Donald Trump

Aircraft Accidents and Lessons Unlearned XXXIX: United Airlines Flight 266

B727 Engineer’s Panel

On January 18, 1969, United Airlines flight 266 (UA266), a Boeing B727-22C, registration number N734U, crashed into Santa Monica Bay. The airliner took off from Los Angeles International Airport (LAX) four minutes earlier; they were attempting to return, following a number one engine fire warning, when the accident occurred (accident report number AAR-70/6). After impact, the sections of UA266 sank to a depth of 950 feet in the Pacific Ocean. The cockpit gauges were destroyed. All three engines were recovered on February 11th, the flight data recorder (FDR) and cockpit voice recorder (CVR) between February 21st through March 4th.

The National Transportation Safety Board (NTSB) determined that the Probable Cause, “… of this accident was loss of attitude orientation during a night, instrument departure in which all attitude instruments were disabled by loss of electrical power.” The NTSB’s probable cause went on to say, “The Board has been unable to determine (a) why all generator power was lost or (b) why the standby electrical power system either was not activated or failed to function.” But is that what happened or was it a simple case of confusion brought about by technical inexperience?

In 1972, almost four years later, Eastern flight 401’s pilots fixated on an indication light bulb inflight. Unknowingly, with attention diverted, they casually flew the L1011 airliner into the Florida Everglades. What if UA266’s “loss of attitude orientation” was not the result of a power loss, but was because the crew simply lost focus on the job at hand: flying the aircraft – like Eastern 401?

The B727 aircraft entered service in 1963. There were three flight crew members: Captain (CP), first officer (FO) and second officer (SO). The CP and the FO flew the aircraft; the SO monitored the various systems’ panels. The B727 had three generators, one on each engine. After engine start, each generator was synchronized, then manually tied to their bus; the SO had to purposely engage the generator. One engine generator could power the airplane and would have to be manually tied.

AAR-70/06’s probable cause consisted of opinion, not fact; for this, it was confusing. “… all attitude instruments were disabled by loss of electrical power.” Little evidence suggested a loss of electrical power; that was pure theory. Instead, the report demonstrated a mismanagement of the aircraft electrical system and a group of investigators who were unfamiliar with the B727.

AAR-70/06 also showed why maintenance-experienced investigators would not only have understood what happened with the electrical power, they would have realized what the SO was doing … or more importantly, not doing. Why? Because United SOs were pilots but their function on the 727 was as an onboard technician. Pilot or engineer investigators would not understand the technical issues with aircraft electrical systems. The accident SO logged only 40 hours on the 727 (Appendix B); his job was to run the systems panel, troubleshoot systems’ problems in flight. He received basic technical training. At 40 hours (maybe 12 to 20 flights), it was unlikely he ever worked many deferred systems, calculated a fuel load, swapped generators or manually adjusted cabin pressure. Certainly, never in a high stress situation.

Consider the SO’s last words: “I don’t know what’s going on.” Classic straightforward confusion. The CP and FO were flying in low visibility, high-stress conditions, possibly disoriented. A #1 engine fire; sporadic contact with LAX; a SO with systems panel problems. Both pilots would have divided attention from flying to help the SO – seated behind them – figure out the panel. They could have missed any directions by the air traffic (ATC) Departure Controller (DC). Did the pilots even realize ATC was calling? If power was out, Standby could have been selected, the battery used to transmit on the #2 radio.

The report AAR-70/06, stated in Finding 14: “The No. 2 and No. 3 engines were developing power at impact.” The post-accident engine tear downs showed number (#) 2 and #3 engines were producing thrust at impact; therefore, the #2 generator was providing electric power the whole time. Did the SO mistakenly disconnect the bus tie? Did the SO not close the bus tie? Did he fail to select ‘GEN 2’ on the Essential Power selector switch? Did he accidentally disconnect busses that powered the CVR and FDR?

The CVR transcript showed unorganized engine shutdown procedures between the fire warning bell (1818:30) and CVR cut-out (1819:13.5). In that 43.5 seconds, did the FO return controls to the CP? Who was flying the aircraft at takeoff? Why did the FO have to ask the CP if he should retard the #1 throttle? Were the #1 engine extinguisher agents used? At 1818:45, an out-of-configuration or takeoff warning horn sounded when the #1 throttle was retarded. The horn was the only indication the crew attempted to shut down the #1 engine and there was no evidence that #2 Generator was selected.

More importantly, was the question of ‘indeterminate later time’, when the CVR and FDR were offline. Was it momentary? 30 seconds? One minute? At 1819:13.5, the CVR, the FDR and the transponder target cut out. The DC stated that UA266 did not respond to course directions. The ATC timer showed UA266 disappeared (impact) from the scope within two [radar] sweeps – four seconds each (Page 3, Note 4) at 1820:30. The CVR recorded nine seconds before impact, which was one second plus the two sweeps. The CVR had stopped for one minute and twenty-five seconds. The DC said he directed a right turn, but UA266 turned left and increased speed. Was UA266’s ‘increased speed’ the steep angled descent UA266 was found to hit the water at? Did UA266 even know they were descending?

At 0.5 seconds after CVR resumption, someone said, “… fields out.” Investigators believed the SO commented on the #2 generator’s electrical field, but if electrical power had been restored, why would the generator field be ‘out’? Besides, the “fields out,” speaker was unidentified. ‘Field’ could have been the airport or ‘field’. Investigators did not know. The “… field’s out” could have meant they had just discovered the communication problems with LAX. Was there stress in the crews’ speech? From the transcript, the SO never said that power had been restored. Did anyone notice the power returned or that power was even missed? The crew may not have known the radios, CVR, FDR or power were lost, because the FO or CP never commented about instrument recovery.

1.5 seconds after CVR returned, the SO stated, “We’re gonna get screwed up.” Two seconds later, the SO said, “I don’t know (what’s going on).” Question: If the SO failed to select Generator 2 on the Essential Power, would the instruments remain powered? Would the FDR and CVR have dropped offline? In his confusion with #2 generator, did he accidentally cut power to the busses powering the recorders?

In the last five seconds, the FO stated, “Keep it going up Arn [CP], you’re a thousand feet.” Two seconds later, the FO said, “Pull it up.” One second later: IMPACT. In those last five seconds, did the CP and FO return full attention to flying, like Eastern 401? The angle the aircraft hit the water suggested they were not aware of their attitude; the sudden call, “Pull it up,” suggested that neither pilot was focused on their rate of descent or pitch angle. The crew shut down the #1 engine without any procedures, no checklist. Could the crew have inadvertently put the aircraft out of configuration without realizing it? To answer the Standby system question, “(b) why the standby electrical power system either was not activated or failed to function”, if the crew did not know there was a power problem, they would not have select Essential Power to Standby. It was likely that the CP and FO focused attention on the young SO’s panel, then became disoriented when they looked back, just like Eastern 401.

Hindsight is 20/20; this is distinctly understood. However, accidents such as these should be reexamined and taught by/to investigatory agencies for the lessons unlearned, particularly mistakes made that could have prevented later accidents. UA266 represented lessons unlearned for investigatory agencies:

  1. Probable cause was useless in 1969 and is useless today. Root cause analysis should always have been pursued as the goal.
  2. Accident investigation reports have spiraled into opinions pieces, not factual analysis. Guessing may have saved time but what amateur opinions cost the aviation industry cannot be measured.
  3. The CVR and FDR data, analyzed while investigating accidents, must receive expert analysis by experienced aviation investigators.

UA266 represented lessons unlearned for the industry:

  1. An opportunity to improve cockpit resource management (CRM), a concept raised in the 1950s. The UA266 crew’s response to the #1 engine fire was disjointed, uncoordinated. CRM should have been a major focus.
  2. Better checklists and pilot-to-pilot challenges for flight crews to handle important events, such as terminating an engine fire or radio communication breakdowns.
  3. Enhanced ATC procedure reviews for communication losses with any aircraft in any stage of flight, whether takeoff, cruise and landing.
  4. Improved technical training for all pilots, specifically for the SO, whose real-life experience was as a pilot, not a technician.

Imagine what later accidents could have been avoided had some actual lessons been implemented in the UA266 accident report. As mentioned in Aircraft Accidents and Kobe Bryant, CVRs and FDRs are tools; if they are not used correctly, they are nothing more than paperweights. The post-tragedy of UA266 was that the data was not analyzed correctly by those who understood airline culture and training.