Learning from the evidence

For decades, the content of flight crew training programmes remained unchanged, according to regulation. Training curricula included repetitive exposures to a set of prescribed events, which, as technology evolved, became highly improbable with modern aeroplanes. In addition, these events were mainly examples of negative performance and we just expected crews to learn effectively from them. In some cases, we learned what NOT to do but rarely what could or should have been done to change the outcome of the event. This gives little help to the crew if faced with a similar or any other challenge.

Training programmes were consequently saturated with items that may not necessarily mitigate the real risks or enhance safety in modern air transport operations.

Furthermore, training approaches traditionally regarded non-technical skills (through Crew Resource Management courses) and technical flying skills (during simulator sessions) separately. Learning from past events, we have come to realize that air incidents and accidents rarely result from the improper use of a given skill alone, but rather involve a combination of both technical and non-technical aspects.

These observations clearly established the need for our industry to revisit some training principles.

In today’s rapidly changing flying environment, it is impossible to predict every single plausible situation that might arise in operations.

“When people and complex systems interact, there will always be an infinite number of possible outcomes”, explains Michael VARNEY, Senior Director Training Policy.

In other words, pilots need to be trained to mitigate the risks of the unpredictable. And means to reach this very objective do not seem obvious at first glance.

The ICAO Evidence-Based Training (EBT) concept is a safety-improvement initiative part of the ITQI project (IATA Training and Qualification Initiative) that precisely addresses this challenge by prioritizing the development and assessment of a finite number of key competencies. The EBT basically recommends to train competencies (not events) and choose training scenarios based on evidence collected from in service data to make sure pilots are able to demonstrate a good performance in front of realistic threats and errors.

Although this concept was primarily designed for recurrent training, the regulatory prescriptions of a Type-Rating skill test can also be achieved using this modern competency-based approach.

Competencies are the backbone of the new training concept; therefore the ones we wanted to use for our training courses needed to be defined with great care and accuracy. It was also important to base our approach on work previously and diligently undertaken during the development of the EBT concept.

In principle, such competencies needed to be defined against the real threats and risks airline pilots will face in everyday operations. This implied understanding those threats through a comprehensive review of safety-related evidence, i.e. data collected from both training results and operations, including the facts of pilot involvement in accident scenarios. Experts in the practice of flight crew training analysed the available wealth of data and agreed definitions of:

• core competencies and
• associated performance indicators aiming to enable training instructors to evaluate those competencies

Both were eventually introduced in ICAO Doc 9995.

Whilst keeping the current regulatory framework, Airbus adapted this system and introduced its own set of core competencies in the Autumn of 2013. These competencies are the key driver for all training and assessment in Airbus, including the A350 curriculum development.

The Airbus 9 core competencies are listed alphabetically as follows:

• Application of procedures
• Communication
• Flight path management – Automation
• Flight path management – Manual
• Knowledge
• Leadership and teamwork
• Problem solving and decision making
• Situation awareness
• Workload management

To create an effective training programme, we had to understand the relationship between this finite number of competencies, and the capability to manage an infinite number of operational situations. Doing so, our efforts were directed towards understanding the underlying conditions for effective performance, rather than on the outcome of events and maneuvers as was traditionally the case. Instructional technique needed to evolve to the person and not to the event.

Psychologists have developed a theory that our learning retention rate gets higher if we learn step-by-step by improving our performance, rather than just getting things right or wrong after a demonstration or lecture. Effective learning demands retention, which can be enhanced by enquiry, practice and feedback from performance.

Better than realizing a scenario went right or wrong, we want to understand why the outcome was negative or positive, and learn from this analysis. This includes viewing an incident not only in terms of its causes, but also with respect to what prevented a more serious outcome.

The role of the instructor becomes essential in this approach: (s)he no longer is just a speaker, but a facilitator and an advisor, continuously interacting with the trainee. He/she gives the trainee the necessary freedom to learn by discovering functionalities and guides him/her when necessary following his/her observations.

In this respect, our approach to training now includes “Facilitated de-brief” during which the instructor adopts a non-judgmental attitude and raises simple questions as a prelude to an open and unfiltered discussion with the trainee:

• “How did it feel?”
• “What did you do well? Why was it effective?”
• “What could have been done better?”

This new approach is meant to ease the discussion between the instructor and the trainee, and build a relationship of trust and honesty between them. With this mindset, it becomes possible for trainee pilots to develop from both their achievements as well as their mistakes.

“When an ILS approach is not completed successfully, rather than considering that the exercise is failed because it was “out of limits”, we want to ask ourselves why it was not completed satisfactorily.” mentions Michael VARNEY.

The upstream cause may be poor descent planning, FMS handling, communication or energy management leading to a rushed approach. A simple repeat of the ILS approach would probably ultimately result in a successful approach, but this method would not necessarily address any of the descent planning, FMS programming or energy management root causes.

To make learning effective, the relationship between the trainer and the trainee is crucial, both of them now being considered as partners in the process of developing confidence and capability in the operation of a modern jet transport aircraft.

Modernizing our training programmes involved reviewing our approach to trainees’ performance assessment. Traditionally, the successful completion of training courses was marked by the trainee’s ability to pass tests. However, those tests did not focus on a demonstration of performance to measure the trainee’s actual ability to fly an airplane and manage the threats a crew may face in today’s operational environment.

Since October 2013, Airbus Training instructors assess trainee pilots’ performance according to a new competency-based grading system. This new grading system requires a demonstration of each of the 9 core competencies, as opposed to the traditional assessment that was made up of right or wrong proposals.

“Training is more than ticking boxes”, says Jean-Michel BIGARRE, Director of Airbus Flight Training. “It is a matter of bringing the pilot and the aircraft work safely and efficiently together” he adds.

In line with this philosophy, the new grading concept uses Pass or Fail criteria, based on a list of observable performance indicators and a grading scale.

The performance indicators detail clearly observable criteria that were designed to enable the trainer to grade all skills in an objective manner – including the non-technical ones. At all times during the training or checking session, the instructor observes the trainee’s performance in relation to these pre-defined performance indicators. Then, by determining the consistency and frequency of these indicators, he/she is able to make a holistic appraisal of the trainee’s understanding of each competence, and his/her ability to exercise it using the grading scale. At the end of the training session, this grading scale offers a global picture of the trainee’s performance during the session by showing a mark between 1 and 5 against each of the 9 core competencies.

The new A350 Type-Rating course is fundamentally modelled on the EBT approach and it is built in accordance with the principles described earlier. But it also proposes beyond a new approach to delivering training to trainee pilots. Fundamentally, it focuses on engendering trainee pilots’ confidence in their own competencies as well as in the aircraft they interact with.

Among the 9 core competencies, the “Flight Path Management – Manual” appeared to us as one of the most challenging to maintain in line operations. Maintaining proficiency and adequate flying skills can be a challenge indeed, particularly for commercial aviation pilots who manual fly relatively infrequently, thus creating the potential for manual flying skills degradation from non-use. In effect, the loss of control in flight remains one of the main causes for accidents ; therefore efforts seeking to advocate for a return to the basics of manual flying clearly have a beneficial potential to invert this tendency.

For this reason, whilst developing the A350 Type-Rating curriculum, Airbus put a particular emphasis on this manual flying competence.

Traditionally, trainee pilots started practical training on a new aircraft with a maximum of flight guidance systems and automation engaged, making them reluctant to switch it off if needed operationally.

“The Airbus Golden Rules require to take over if the need arises, and our training regime needs to prepare them exactly to do this.”, says Dominique DESCHAMPS, Vice President Flight Operations and Training Support.

For its A350 Type-Rating course, Airbus has developed a hands-on “learning by discovery” process for trainee pilots to familiarize themselves with the aircraft and its manual handling characteristics through early and frequent practicing. This concept follows the principle described earlier of humans learning step-by-step with progressive skill improvement by cleverly sequencing theory in classroom, and practice in the Flight Training devices.

The goal of a Type-Rating program is to make a pilot proficient in interacting with a new complex technical system (the aircraft), using a new Human Machine Interface (the cockpit). Traditional Type-Rating programs introduce in priority the technical systems first (in the last decades by CBT style training). The cockpit is only introduced during the latter stages of the program using Flight Simulation Training Devices (FSTDs).

The Airbus A350 training concept has reversed this paradigm, introducing the cockpit interface and the Full Flight Simulator (FFS) as early as possible in the curriculum. According to this approach, practice on ACE is introduced from day 1 of the course.
Then, systems description modules are integrated in ACE as “quick user guides” to enable the pilot to start learning in the most realistic environment rapidly.

For FFS sessions, instructors are given guidance as to when they should let the trainee discover on his/her own, finding out how the aircraft works by experimenting, and when they should demonstrate. This principle echoes the way in which people frequently ignore the instructions guide when they try a new device, and simply turn it on and try to find out how it works by experimenting. They would consult the instructions guide after they had tried various actions, and eventually could not manage by themselves.
We have considered that the same happens with a new aircraft: we want to let trainee pilots get to grips with the aircraft by experimenting in the FFS early in the scheme, thus enabling them to develop their manual flying instinct.

“Our ambition is to re-ignite this very feeling they had when they first flew a real aircraft, by not overloading them with theory before they had a chance to try and experiment.”, says Franck VESSIOT, designer of the FFS training sessions.

Only after manual flying consolidation, automated systems are introduced gradually, one after the other, both in theory, and in practice in the FFS. The “Bird” (Flight Path Vector) comes first, then the Flight Director, Auto Thrust and eventually the Auto Pilot.

In the end, the approach to training is reversed: pilots are encouraged to appreciate flying the aircraft manually and consider automated systems a full benefit, rather than considering manual flying a degraded and potentially challenging configuration.

Incidents and accidents analyses show that abnormal situations rarely present themselves in a standardized or predictable sequence. Although the potential element of surprise is more suited in the recurrent training domain, we decided to include some elements of surprise while programming the A350 Type-Rating courseware.

Prior to the training session, trainees will be informed of the subjects that will be covered, but they will have no information about the exact nature or sequence of events the instructor will use. In the final sessions, the instructor will have a choice between several malfunctions and events intended to develop the same competencies, thus limiting, if not avoiding pre-planned or routinized actions by the crew.

This element of surprise brings training closer to real situations, and raises opportunities for the trainee and the instructor to discuss the available options and solutions on a case by case basis.
In addition, keeping it a surprise allows enough flexibility for the instructor to play scenarios the closest possible to the airline’s specific operations.

The training programme really is customized according to the airline’s operational constraints in order to ensure the pilots are confident in managing real threats and hazards as they commence supervised and ultimately unsupervised line flying.