Statistical Techniques: Design of Experiments

Welcome back to MedTech Compliance Chronicles! This week, we’re shifting gears from our usual country-specific regulatory requirements to another discussion on a statistical technique. We will be discussing a powerful statistical tool used across regulated industries: Design of Experiments (DOE). DOE has many purposes depending on its application, for regulated industries it is invaluable in meeting the requirements of process validation, root cause analysis, corrective action verification, as well as understanding, optimizing, and controlling processes, product refinement, and process improvement.

While performing DOE is not a specific regulatory requirement, regulatory agencies expect manufacturers to use statistically sound methods when demonstrating process control, investigating failures, or optimizing product performance. This post will provide a brief introduction to DOE and break down how DOE can be applied in compliance-driven industries to enhance decision-making and improve quality outcomes.

The Process of DOE

The process of performing DOE is fairly straightforward, but as with all things, the devil is in the detail. Designing a proper experiment that will provide the maximum amount of information about what you are interested in studying requires a lot of up front planning. You must determine what it is you are interested in studying, a “response” as it is often termed (this would be the dependent variable from high school science class), then what factors/variables you believe influence it (this/these would be the independent variable(s)). One of the main benefits of DOE is the ability to study multiple factors in the same experiments, as opposed to changing only one factor at a time, many many times. This also brings the added benefit of being able to study the interaction effects of the factors, if any exist. After choosing the response and factors, you will determine the ranges (levels or settings, i.e. temperature from 190 C - 210 C) of the and set up the experimental runs in a particular order at various combinations of the settings. 

First, we will start with choosing your response and factors (dependent and independent variables). The response should be something you are interested in studying, usually the desired output of the process, a common example in medical device manufacturing might be the seal strength of a seal for a sterile barrier system. The next step is determining the factors that influence the response, in our seal strength example the most common factors would be sealing temperature, pressure applied by the sealing jaws and the time that the sealer exposes the packaging to the specified temperature and pressure. Factors must be chosen deliberately, based on the goal of the experiment. Depending on where you are at in your study of the response, the design of the experiment may be chosen to include many factors to see which have the greatest impact on the response (these are often called “screening experiments”). If the most relevant factors are known, then an experimental design with fewer factors can be chosen to hone in on the specific levels of those factors which give the most optimal value of the response. 

With the response and factors chosen, the next step is to determine the experimental design. This post will only briefly touch on two common experimental designs, as there are enough to write multiple books on if you wanted to comprehensively cover all types. For most problems in industry, the 2^k full factorial or fractional factorial designs will be adequate. The notation 2^k means that there are ‘k’ factors being studied, each with 2 levels. The mathematical result of 2^k equals the number of different combinations of factor levels that make up the required experimental runs of a full factorial design (i.e. 2^2 = 4 runs, 2^3 =8 runs and so on). As you can see, the number of runs required exponentially increases with each new factor in full factorial designs, which is why the fractional factorial design was invented. Next you will determine the levels of the factors you wish to study, for a 2^k design it will be 2 levels, one “high” and one “low.” It is important to choose a range for the levels that you expect a noticeable change in the response to occur. Too wide or too narrow of a range could result in missing valuable information. With the design and levels chosen, you will then code the runs in the format below:

A	B	C	AB	AC	BC	ABC
1	1	1	1	1	1	1
1	1	-1	1	-1	-1	-1
1	-1	1	-1	1	-1	-1
1	-1	-1	-1	-1	1	1
-1	1	1	-1	-1	1	-1
-1	1	-1	-1	1	-1	1
-1	-1	1	1	-1	-1	1
-1	-1	-1	1	1	1	-1

In the table above the single letters (A, B, C) correspond to the factors under study and the two and three letter combinations refer to the interactions of those factors. The -1’s in the table correspond to the low level of the factor for that column while the 1’s indicate the high level. The -1’s and 1’s for the interaction columns are found by multiplying the -1’s and 1 of the factors that are interacting. For example the interaction ABC designation in a row where A is 1, B is -1, and C is 1 will be 1*-1*1=-1, therefore -1 will be the value in the ABC column in the row where A, B and C have the stated settings. 

Fractional designs are found by replacing an interaction column with an additional factor. Say we wanted to use the above design to study 4 factors instead of 3, perhaps because we only have enough money or time for 8 runs of the experiment. We could replace the column ABC with factor D and set the high and low settings of factor D to the same that would have been the interaction ABC. The drawback to this is that you lose information on the interaction ABC. Any significance of the ABC interaction will be indistinguishable from the effects of factor D. In a 2^3 design, it is usually safe to replace the ABC interaction with a fourth factor. In higher order designs, you must analyze what is called the defining relationships to determine that you are not “aliasing” any important factors with each other. Aliasing and defining relationships are a bit beyond the scope of this post, but essentially aliasing occurs in fractional designs when two columns have their -1’s and 1’s at the same levels in the same runs, which makes their effects indistinguishable from each other, like the interaction ABC and factor D in the example above. 

With the design chosen, the next step is to run the experiments. An important consideration in running the experiments is randomization or blocking to reduce the unknown environmental effects or known effects that are not included in the study. In general, randomization is used to reduce unknown and environmental effects and blocking is used to reduce known effects that are not included in the study (usually because they either cannot be controlled or would be very expensive to control). Randomization involves running the experiments in random order. In the example above, instead of running the experiments sequentially 1-8, you would assign random numbers ranging 1-8 to each row and run them in that order, which might result in something like 7,3,2,5,8,4,1,6. An example of blocking would be if you have to run the experiment over multiple days or on two different machines, you would assign the experimental runs to days or machines. In general, you choose your blocks with either the highest order interaction for full factorials or the defining relationship for fractional and separate the blocks based on the high and low levels of these columns. For example, let's say the experiment above needs to be run on two machines, so you need to block the experiment by machine. You could take column ABC and assign all runs where this column is 1 to one machine and all runs where this column is -1 to the other. 

While the technical details of DOE can be deep and complex, even simple experiments can offer powerful insights. With a solid understanding of the fundamentals, manufacturers can apply DOE across a range of compliance-focused activities. Let’s explore how DOE can be applied within the medical device industry to support regulatory, quality, and business objectives.

DOE in Medical Devices

There are innumerable great opportunities to reap the benefits of a well designed experiment in the medical device industry. In fact, DOE can be a very economical way of complying with a lot of regulatory requirements. DOE can be invaluable in validation activities, root cause analysis and corrective action verifications. Additionally, it can be a great means of evidence based product and process refinement.

Regulatory bodies require manufacturers to demonstrate that their processes are capable, repeatable, and stable under normal operating conditions, through process validation. DOE can help define critical process parameters (CPPs) and their effect on critical to quality attributes, ensuring robust process performance. When using DOE in process validation, the CPPs are usually known and the goal is to establish the upper and lower limits of the settings, as well as the optimal settings on the specific machines that will be utilized in production. 

Continuing with our earlier example of validating a heat-sealing process for sterile packaging. The key input parameters we identified were seal temperature, dwell time and pressure. Using a 2³ full factorial DOE, you can model how these inputs affect seal strength and integrity. These can subsequently be used to determine which factors are most critical (the CPPs) and the acceptable ranges for each factor that ensure reliable seals. Once CPPs are identified, more advanced versions of DOE, such as response surface methods (RSM) can be used to map the safe operating window and define the optimum settings. For example, RSM might show that acceptable seal strength is maintained only when dwell time is above 0.8s, but not beyond 1.5s at certain temperatures. It could also show that the strongest values are at 1.3s. This allows you to establish a validated process window with strong statistical confidence, as well as what the ideal settings for the machines to be set to are.

When failures occur, DOE provides a systematic approach to identifying the most influential factors contributing to defects or deviations. Instead of trial-and-error problem-solving, DOE allows for an efficient, data-driven approach to investigating failures. Screening designs like the fractional factorial discussed in the previous section can efficiently identify key process variables affecting the problem quality characteristic. 

In product development, DOE is an essential tool for improving design robustness and performance and minimizing variability. Let’s say you’re developing a surgical catheter that must perform reliably across a range of fluid viscosities and temperatures. Rather than testing each factor in isolation, a factorial DOE allows you to simultaneously evaluate multiple inputs, such as material stiffness, wall thickness, operating temperature, fluid viscosity, etc. Using a DOE helps you identify not just the effects of these main factors but also their interactions (i.e., how viscosity and temperature affect performance together), allowing you to more precisely define the operating ranges of the device, as well as CPPs for the processes it will be made with. This makes your V&V activities more robust and defensible to regulators.

Conclusion

DOE is a powerful, structured approach to problem-solving and optimization, making it a critical tool for regulatory compliance in the medical device industry. By integrating DOE into process validation, root cause analysis, product refinement, and continuous improvement efforts, manufacturers can enhance quality, improve efficiency, and ensure compliance with regulatory expectations. Whether used to support regulatory submissions, optimize production, or troubleshoot failures, DOE provides data-driven insights that lead to more robust, reliable, and compliant products and processes.