"How do you know when real change has occurred?" This is a question our Chief Strategy Officer, Colin Sloman, has asked many times. Colin shares his thoughts on using A/B Testing and data to deliver meaningful results that track change in large organizations. 

In my early years as a change management consultant, one daunting question from clients was, "How can we ensure the success of this change?" In simpler terms: would it work, and how would they know when real change occurred? During those years, I often relied on the 'hit-and-hope' method—delivering communications, training, and engagement activities without concrete assurance that these efforts would be effective. Effective measurement was also complicated by a frequent lack of investment in comprehensive data collection and analysis. 

What Has Changed? 

In 2016, I collaborated with the Cognician team, whose approach to behavioral science and nudges was gaining attention. This shift was inspired by the rise of behavioral economics and the influence of The Behavioural Insights Team in the UK government. The challenge was integrating neuroscience and behavioral science into existing change management methods—in essence, turning 'hit-and-hope' into a scientifically grounded process that ensures measurable change. 

Applying the Cognician Activation Framework 

My first experience applying the Cognician Activation Framework was with newly hired consultants at a global firm. While traditional face-to-face onboarding was adequate, we believed enhancements were possible. Partnering with Cognician, we developed a multi-day challenge (MDC) alongside the existing process. Using design thinking, we implemented an A/B test to measure time to proficiency, billable hours, and line manager effort, supported by behavioral measures and a financial business case. The impressive results—including a Brandon Hall Gold Award—highlighted the new approach’s benefits in comparison to traditional methods. Since then, the onboarding MDC has become a mainstay, benefiting thousands of new joiners by nurturing mindsets and behaviors aligned with organizational goals. 

Embracing A/B Testing 

At Cognician, A/B testing is a foundational practice. In a project for a banking client, we divided participants into three cohorts: Cohort A (standard MDC), Cohort B (enhanced behavioral MDC), and Cohort C (without Cognician support). Results showed consistently high contributions from Cohort B, demonstrating the impact of enhanced drivers. These insights inform our approaches and enable us to optimize change project outcomes through varied challenges and engagement strategies. 

Enhanced Measuring with Cognician 

We’ve refined our A/B testing methods to better assess change impact in multi-day challenges: 

  • Activation Index (AI): Measures engagement with change outcomes through intent (accepting and scheduling change challenges), action (fulfilling the challenge), and reflection (considering the challenge). 
  • Commitment Index (CI): Assesses participants' investment using natural language processing (NLP) across social, physical, intentional, cognitive, and emotional categories. CI helps gauge commitment, offering benchmarks via Standard Activation Reports for comparison with similar change projects. 

Rooted in over 30 years of behavioral science research, Cognician's programs deliver mindset and behavior change, driven by 8 Activation Catalysts and 48 Activation Tactics published here: Activation Tactics Cards. Continuously innovating through A/B testing, our new Activation Reports ensure confidence in ongoing change using the Activation Index and Commitment Index to measure success and buy-in. 

Let Cognician be your assurance of genuine transformation; trust us to prove real change has happened and that behavior and mindsets are evolving. For more information about our AI and CI measures, please contact us here.