Engineering Decisions: What Design of Experiments Reminds Us About Making Better Choices

Lionel Grealou Engineering Excellence Innovation 6 minutes

Product development is a discipline of trade-offs

Product development, as part of the broader product innovation process, is fundamentally a discipline of trade-offs. Every design choice involves balancing competing requirements, from performance, safety and reliability to cost, weight, manufacturability and sustainability. Engineering is not about maximising every attribute independently, but about understanding their interactions, managing uncertainty and making informed choices.

Consider the design of an automotive A-pillar. Engineers must reconcile structural crashworthiness with the driver’s field of vision — two competing dimensions of vehicle safety. Increasing the pillar’s cross-section may improve certain structural characteristics while obstructing visibility. Alternative geometries, materials and reinforcement strategies introduce further trade-offs involving weight, cost and manufacturing complexity.

Determining which compromises are acceptable requires more than experience or intuition. Engineers rely on analysis, simulation, experimentation and engineering judgement to evaluate alternatives and understand their consequences.

Design of experiments (DOE) provides a statistical framework for systematically investigating how influencing factors affect measurable responses, individually and through their interactions. An interaction occurs when the effect of one factor depends on the level of another. Established methods, including factorial designs, randomisation, replication and blocking, help engineers estimate these effects, account for experimental variability and evaluate alternatives systematically. These principles are documented in the NIST/SEMATECH Engineering Statistics Handbook and Douglas Montgomery’s Design and Analysis of Experiments.

Yet generating experimental evidence is only part of the challenge. Every experiment produces information that must be interpreted in context: what was tested, why, under which conditions, using which methods and with what results. Engineering decisions add more information: which alternatives were considered, which trade-offs were accepted, who made the decision, and on what basis.

This is where experimentation becomes a data and information management challenge.

Effective experimental information management connects evidence with product context, decision rationale and subsequent outcomes, ensuring that engineering knowledge remains traceable and reusable throughout the lifecycle.

Managing the information behind experimentation

Managing experimental information requires a structured relationship between six dimensions:

  1. Intent and inputs: The problem, hypotheses, requirements, product configuration, materials, experimental factors, constraints and expected responses.
  2. Experimental design and execution: Methods, procedures, equipment, people, operating conditions, factor levels, replication, and deviations.
  3. Outputs and evidence: Measurements, observations, statistical analyses, variability, uncertainty and unexpected results.
  4. Interpretation: What the evidence establishes, which interactions matter, what limitations remain and how competing alternatives compare.
  5. Decisions: Which options were selected or rejected, against which criteria, by whom and with what rationale.
  6. Outcomes: What was implemented, how it performed in practice and whether subsequent evidence supports the original conclusions.

These dimensions are related, but they are not interchangeable. A measurement is not automatically evidence supporting a hypothesis, and a statistically significant result does not necessarily establish the preferred engineering or business solution.

Returning to the A-pillar example, DOE can reveal how structural geometry, material properties, and reinforcement strategies interact to influence crash performance, weight, and other measurable responses. For example, the effect of changing pillar geometry on structural performance may depend on the material selected. These interactions matter because optimising individual parameters independently may overlook combinations that deliver better overall performance. Selecting the final design also requires considering driver visibility, tooling investment, manufacturing capability, supplier availability and platform commonality.

Experimental evidence informs engineering decisions, but selecting a solution also requires balancing technical findings against business constraints. Preserving the relationships between experimental conditions, results and decision rationale enables that evidence to be interpreted and reused beyond the original engineering team or laboratory.

From laboratory evidence to product and business context

Experimentation supports decisions throughout the product lifecycle: exploring hypotheses and technical feasibility during research, validating requirements and design alternatives during development, optimising manufacturing processes during industrialisation, and investigating performance or quality deviations in operations. Experimental evidence also supports intellectual property (IP) management, from invention disclosures and patent applications to protecting proprietary know-how.

The supporting information is often distributed across different systems and disciplines. LIMS manages laboratory samples, workflows, and test results; electronic laboratory notebooks (ELN) document experimental procedures and observations; PLM connects product definitions, requirements, configurations, and engineering changes; and MES records manufacturing execution and operational performance. Their capabilities overlap, but their information models and responsibilities differ.

Experimental evidence retains its value across the product lifecycle when its original context, assumptions and applicability remain traceable.

This requires maintaining relationships between experimental results, material formulations, product revisions, process conditions and the decisions they support. For IP management, provenance, contributor records and appropriate confidentiality controls are equally important.

As products, suppliers, manufacturing processes or regulatory requirements change, these relationships enable organisations to reuse existing evidence, assess its continued applicability and avoid unnecessary experimentation or decisions based on outdated assumptions.

Managing decisions, not just experimental records

Engineering organisations have long-established practices for documenting tests, reviewing designs, managing changes and approving technical decisions. However, the supporting context can become fragmented across systems, documents, disciplines and organisational boundaries.

Consider an alternative A-pillar design that met the technical requirements but was rejected because of manufacturing cost or tooling constraints. Years later, a new manufacturing technology or material may change those economics. The original alternative could become attractive again, provided the evidence and rationale behind the earlier decision can be reconstructed.

Managing decision context requires preserving the relationships between:

  • Alternatives and trade-offs: Options considered, selected or rejected, including the criteria behind those choices.
  • Evidence and assumptions: Experimental findings, simulations, uncertainties and conditions supporting the decision.
  • Product and business context: Applicable configurations, requirements, manufacturing capabilities and commercial constraints.
  • Accountability and validity: Decision ownership, rationale and the conditions that may warrant future reassessment.

An experiment may support several decisions, while a single decision may depend on evidence from multiple experiments, simulations and operational observations. Managing these many-to-many relationships is essential to reconstructing engineering rationale and assessing the consequences of change.

Not every laboratory experiment requires a formal decision process. Governance should reflect the significance, risk and intended application of the evidence. Decisions affecting product performance, safety, compliance, manufacturing capability or major investment warrant appropriate traceability.

An engineering decision is made under a particular set of assumptions and constraints. Its continued validity depends on whether those conditions still hold. A material substitution, new test result, unexpected field failure or change in operating conditions may justify reassessment. Effective change-impact analysis can identify affected decisions, determine which conclusions remain applicable and establish where further validation is required.

Designing the right loops around engineering decisions

Engineering experimentation illustrates how decision loops operate in practice. Laboratory evidence informs product and process choices, while manufacturing performance, quality deviations and field observations may reveal unexpected variability or challenge earlier assumptions, prompting further experimentation, process adjustments or reassessment of engineering decisions.

These feedback loops operate across different timescales. An experiment may inform an immediate design adjustment, while manufacturing trends or field performance may challenge decisions made years earlier. Their effectiveness depends on the ability to relate observed outcomes to the original expectations, distinguish meaningful changes from normal variability and determine where corrective action or further investigation is justified.

AI can support experimental planning, identify patterns, analyse historical results and help explore alternatives. It may also help identify relationships between emerging operational evidence and earlier engineering assumptions. However, retrieving laboratory data or generating recommendations does not establish the validity of experimental evidence or the rationale behind an engineering decision. Without reliable product context, traceability and governance, AI risks accelerating the interpretation of incomplete or inapplicable information.

Connecting experimental knowledge across the lifecycle makes evidence more accessible and reusable. A digital thread establishes information continuity, while effective decision loops connect that evidence to action, outcomes and learning.

From experimental learning to organisational learning

Experimental evidence has value beyond the immediate test or project. Its continued relevance depends on preserving the relationships between experimental conditions, product configurations, engineering assumptions and the decisions it supports.

Organisational learning also depends on making that knowledge available beyond the individuals and teams that generated it. This includes understanding which approaches succeeded, which alternatives were rejected and whether the original assumptions remain applicable as products, technologies and business conditions evolve.

The value of an experiment lies not only in what it establishes, but also in the choices it informs, the alternatives it eliminates and the assumptions it helps validate. Connecting experimental evidence to operational outcomes, and recognising when earlier conclusions warrant reassessment, turns individual experiments into reusable engineering knowledge and organisational learning.

What are your thoughts?

References:


Disclaimer: articles and thoughts published on v+d do not necessarily represent the views of the company, but solely the views or interpretations of the author(s); reviews, insights and mentions of publications, products, or services do neither constitute endorsement, nor recommendations for purchase or adoption. 

About the Author

Lionel Grealou

Lionel Grealou, a.k.a. Lio, helps original equipment manufacturers transform, develop, and implement their digital transformation strategies—driving organizational change, data continuity, operational efficiency and effectiveness, managing the lifecycle of things across enterprise platforms, from PDM to PLM, ERP, MES, PIM, CRM, or BIM. Beyond consulting roles, Lio held leadership positions across industries, with both established OEMs and start-ups, covering the extended innovation lifecycle scope, from research and development, to engineering, discrete and process manufacturing, procurement, finance, supply chain, operations, program management, quality, compliance, marketing, etc.

You may also like: