Are We Measuring Change at the Right Time?

Transformation readiness and adoption cannot simply be inferred from a green program dashboard. Readiness tells us whether the organization can operate; adoption tells us whether new ways of working are actually taking hold. Both need active management and measurement, but the required evidence changes as the transformation progresses.

This makes Organizational Change Management (OCM) a continuous discipline rather than a set of activities around go-live. What we measure, who needs to be involved, and who owns the outcomes must evolve from readiness through implementation, adoption, and ultimately value realization.

Another important dimension is stakeholder involvement. OCM cannot be delegated to a change team that periodically measures the organization from the outside. Sponsors, business leaders, process owners, managers, SMEs and impacted users need to remain involved throughout—not simply consulted at the beginning or trained at the end.

Prosci’s research consistently identifies active and visible sponsorship as a leading contributor to successful change and emphasizes the importance of building a coalition of support across the organization. The implication is straightforward: measurement, engagement and adjustment need to continue together throughout the transformation.

Before go-live: are we ready?

Before deployment, the emphasis should be on readiness, not adoption. Training completion, communications delivered and stakeholder engagement are useful, but they remain leading indicators.

Core measures should include:

Change-impact coverage: have impacted roles, processes and organizational units been identified?
Stakeholder engagement: are the right stakeholders actively participating in decisions and validation?
Training and competence: can people demonstrate the critical tasks required by their future roles?
Process readiness: can representative end-to-end scenarios be executed by the business?
Business readiness: are roles, decision rights, data, support arrangements and dependencies in place?
Readiness risks: what critical gaps remain, who owns them and what evidence supports closure?

The distinction between completion and competence is important. “95% trained” tells us that an activity occurred; it does not tell us whether 95% of users are ready.

Go-live and hypercare: can we operate?

Once the solution is deployed, the evidence changes. We no longer need to rely exclusively on plans, surveys and readiness assessments because we can observe real behavior.

The focus should shift toward:

Active usage: are the intended users actually using the new environment?
Critical process execution: are end-to-end processes operating successfully?
User proficiency: where do people still require exceptional support?
Support demand: what issues are recurring and why?
Process exceptions and workarounds: where is the intended process being bypassed?
Business continuity: can the organization operate without disproportionate program intervention?

Stakeholder involvement also needs to change. Users and managers now provide operational evidence rather than hypothetical feedback. Process owners need to determine whether exceptions are temporary stabilization issues or symptoms of deeper process, data or solution gaps.

A temporary spike in support after go-live may be perfectly normal. Continued dependency on the program team months later is more significant.

Post-implementation: has adoption really occurred?

This is where OCM measurement often becomes weaker just as it becomes more meaningful. The program may be closing, consultants may be leaving, and governance may be transitioning to business-as-usual, but adoption is not established simply because implementation has ended.

We should now measure sustained behavior:

Sustained usage: are people continuing to use the new capabilities?
Process adherence: are intended processes becoming normal practice?
Proficiency: can users operate effectively without exceptional support?
Shadow processes: have legacy systems and practices genuinely disappeared?
Workarounds: what is still happening in Excel, email, shared drives or local tools?
Local variation: where are teams deviating from the target operating model, and why?
Stakeholder sentiment: are managers and users identifying new barriers or improvement opportunities?

This last point matters. Stakeholder engagement should not end because the solution has gone live. Feedback now becomes more valuable because it is based on real experience.

Continued spreadsheet use, for example, is not automatically evidence of failed adoption. Excel may remain entirely appropriate for analysis and flexibility. But when people reconstruct processes, data or decisions outside the new platform, it provides evidence worth investigating.

The question is not simply “Are people still using Excel?” but “Why do they still need to?”

Beyond adoption: did anything actually improve?

Even sustained adoption does not prove that the transformation was worthwhile.

ERP, CRM, PDM/PLM, MES and other enterprise platforms are rarely implemented simply to generate system usage. They are intended to improve business capabilities and outcomes.

Measurement therefore needs eventually to move beyond OCM activity toward value realization:

Process performance: cycle time, throughput, quality or other relevant operational outcomes.
Data quality and availability: has information become more reliable and usable?
Productivity: has unnecessary manual effort and duplication decreased?
Business performance: are the outcomes used to justify the transformation improving?
Operating-model effectiveness: are functions collaborating and making decisions as intended?
Benefits realization: are the benefits in the business case actually materializing?

McKinsey’s transformation research illustrates why this longer-term perspective matters: while 56% of respondents reported initially achieving most or all transformation goals, only 12% said those gains were sustained for more than three years. Respondents estimated that an average 42% of potential financial benefits was lost during execution and sustaining phases.

The specific KPIs will obviously vary. PLM engineering-change performance differs from CRM sales effectiveness, ERP financial close, or MES manufacturing performance. OCM metrics eventually need to connect to the business outcomes the transformation was intended to change.

Avoid the single adoption score

There is an obvious temptation to aggregate all of this into an executive-friendly number: “Adoption = 87%.”

I would be cautious.

A high login rate can coexist with poor process adherence. High training completion can coexist with low proficiency. Strong transaction volumes can coexist with extensive Excel workarounds. Positive stakeholder sentiment can coexist with poor business outcomes.

Enterprise transformation is too interconnected to assume that one aggregated number adequately represents adoption.

The dashboard should help leaders ask better questions rather than manufacture certainty.

Track the transition, involve the stakeholders

Perhaps the simplest model is:

Before go-live: Readiness — Can we operate?
Go-live and hypercare: Stabilization — Are we operating?
Post-implementation: Adoption — Are the new ways of working becoming sustained behavior?
Beyond implementation: Outcomes — Is the transformation producing the intended value?

Throughout these phases, the measures, stakeholders and ownership should evolve. The program may initially coordinate much of the OCM activity, but business leaders, process owners, managers and users increasingly need to own the evidence, decisions and improvements.

This is consistent with the broader evidence on transformation sustainability: maintaining implementation rigor, active leadership and people-oriented practices into the later stages is associated with a much greater likelihood of sustaining performance improvements.

OCM therefore cannot be a workstream that finishes when the program finishes. It is a continuous process of engaging stakeholders, observing behavior, measuring outcomes and adjusting where the evidence tells us the transformation is not working as intended.

Perhaps the real test is simple:

If we stop engaging stakeholders and measuring adoption when the implementation team leaves, were we ever really measuring adoption—or just implementation?

The research basis is strong enough without overloading the article: Prosci says sponsors should remain “active and visible throughout the life of the project,” build coalitions and solicit management feedback; its broader research also identifies active and visible sponsorship as the leading contributor to change success. McKinsey provides the complementary post-implementation evidence: only 12% in its survey sustained transformation gains beyond three years, and organizations combining later-stage rigor, people goals and adequate resources were 3.4× more likely to sustain gains.

What are your thoughts?

References:

Disclaimer: articles and thoughts published on v+d do not necessarily represent the views of the company, but solely the views or interpretations of the author(s); reviews, insights and mentions of publications, products, or services do neither constitute endorsement, nor recommendations for purchase or adoption.

About the Author

Lionel Grealou

Lionel Grealou, a.k.a. Lio, helps original equipment manufacturers transform, develop, and implement their digital transformation strategies—driving organizational change, data continuity, operational efficiency and effectiveness, managing the lifecycle of things across enterprise platforms, from PDM to PLM, ERP, MES, PIM, CRM, or BIM. Beyond consulting roles, Lio held leadership positions across industries, with both established OEMs and start-ups, covering the extended innovation lifecycle scope, from research and development, to engineering, discrete and process manufacturing, procurement, finance, supply chain, operations, program management, quality, compliance, marketing, etc.