The FDA’s 2011 Draft Guidance vs. 2000 Guidance for Human Factors

This summary discusses the following two FDA guidance documents:

• Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk Management (July 18, 2000)

• Draft Guidance for Industry and Food and Drug Administration Staff-- Applying Human Factors and Usability Engineering to Optimize Medical Device Design (June 22, 2011)

The FDA is making strides to increase awareness of the importance of the usability engineering process when developing a medical device.

The FDA formed a Human Factors group within the Office of Device Evaluation around 2006, but the FDA has reported an increasing trend of Human Factors adoption throughout industry.

In 2000, the FDA published its first Human Factors guidance, Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk Management, to inform manufacturers about the importance of usability engineering and to explain how to incorporate human factors in the design process. In 2011 the FDA released the Draft Guidance for Industry and Food and Drug Administration Staff - Applying Human Factors and Usability Engineering to Optimize Medical Device Design.

When the 2011 Draft Guidance for Human Factors was released, there was a lot of commotion in industry, and discussion of the “new requirements” for human factors. This blog post discusses the fact that many of the high-level recommendations for human factors were unchanged, such as:

The need to clearly state the device use, device user population, and device use environment

• The need for formative testing of medical devices

• The need to choose one of the four ways to improve the device when a use error is discovered

  • Modify the device design to remove a hazard or reduce its consequences

  • Make the user interface, including its operating logic, error-tolerant

  • Alert users to the hazard

  • Develop written procedures and training for safe operation

    
    

• The need for Verification and Validation testing

Though similar to its predecessor, the 2011 draft guidance is a significant improvement compared to the 2000 version because it offers specific guidance and has a clear, concise, process-driven outline. The flow of the guidance follows, step by step, the Usability Engineering process. The guidance also calls for specific deliverables such as a Human Factors Engineering Submission Report, which is now mandatory for all PMA applications. These changes help to minimize manufacturers’ confusion with regards to what exactly the FDA expects.

Particularly of note, the 2011 Draft Guidance sets clear expectations with regards to design of a manufacturer’s Usability Validation study. Here are a couple of new recommendations and information introduced by the 2011 Draft Guidance related to usability validation:

Identifies how many subjects are required during usability validation testing

• For example, since there were so many use-related hazards occurring with infusion pumps, the FDA added a suggestion that at least 25 people should be tested.

Source: fda.gov  

Introduces the idea of training decay.

• If a user receives training significantly prior to using the device in real life, it is inevitable that some information will be forgotten. It is essential to take into account the delay time after training and before testing otherwise risks that occur because of forgotten information will not be found.

• Clarifies expectations around Data Collection

  • Both subjective and performance data are essential to prove the device functions properly 

• Clarifies that the time it takes to complete a task with the device should only be taken into consideration if that matters for the real-life function of the device. 

• Defines the idea of “close calls,” where an error almost occurs.

• Discusses how to interpret the data from the study and introduces the concept of residual risk.

  • No matter how much testing is performed, a medical device may still have some risks associated with it. This idea is very important to get across because it is necessary to prove that the benefit of having a medical device on the market outweighs its “residual risks.”

  • Emphasizes that clinical validation should only happen after simulated validation so that as many errors as possible are found before actual clinical use

The following additions are also of note:

• The concept of performing a “cognitive walk-through” (where users can make comments as they are walked through how to use a device) was added as another Formative Study option for early stages of device development.

• The Draft Guidance additionally offers helpful information to determine sample sizes for formative (early-stage) usability testing. Appendix B includes data from Faulkner’s study in 2003 that relates the mean percent of errors captured based on the number of people tested. For example, a study of fifteen people will find 97.05% of all errors.

Interestingly, a couple of concepts from the 2000 guidance were omitted from the 2011 FDA Draft Guidance.

The 2011 FDA Draft Guidance fails to mention the effect stress and mental workload can have on a user’s ability to operate a medical device. This fact, which was described in the 2000 version, helps explain why alarm fatigue can occur---an individual may be overwhelmed in the face of too many alarms so may fail to distinguish them.

• Additionally, the analytic stage of the new draft states the importance of prioritizing risks, but does not give examples on how to do so. The old draft, on the other hand, provides information on how to organize data, such as through the use of an FMEA, which clearly lays out potential failures and their ensuing negative effects.

• Moreover, when searching for potential hazards, the idea of taking two different perspectives--“top-down” (identifying hazards and then finding uses that could cause those hazards) or “bottom-up” (identifying uses that could cause different hazards)--was removed in the new draft.

• The changes made to the Draft Guidance improved readability and offered clear expectations surrounding the human factors engineering process. The usability engineering process has been a requirement all along, and has not changed much over the years. Despite this fact, many manufacturers ignored the 2000 Draft Guidance. With increasing enforcement since 2006, and with the new-and-improved 2011 Draft Guidance, manufacturers have started to truly embrace human factors. Though the content is not significantly different, it has had a huge impact on the medical device industry.

References:

"Draft Guidance for Food and Drug Administration Staff: Applying Human Factors and Usability Engineering to Optimize Medical Device Design." U.S. Food and Drug Administration: Protecting and Promoting Your Health. U.S. Department of Health and Human Services, 22 June 2011. Web. 13 Aug. 2015. <http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM259760.pdf>.

"Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk Management." U.S. Food and Drug Administration: Protecting and Promoting Your Health. U.S. Department of Health and Human Services, 18 July 2000. Web. 13 Aug. 2015. <http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm094461.pdf>.