Forty Years On

Michael Wagstaff • 2 September 2024

How my research skills have evolved

It's almost 40 years since I started working in research and over the course of my career, I’ve seen some pretty big changes in the skills and approaches needed to make a difference in this field. From my early days in academia to the fast-paced world of online surveys and now to text and data analytics for brands, the skills I’ve picked up along the way have definitely evolved.



The learning years

I started out in academic research, which is all about precision and rigour. Back then, my work was heavily focused on hypothesis testing, literature reviews and applying statistical techniques to ensure every finding was backed by solid evidence. This time was all about applying a methodical approach to problems, which really set the stage for everything that came after. It also introduced me to SPSS for analysing data which was an essential tool for uncovering insight.


Academia laid the foundations for a career in research and gave me a deep appreciation for data accuracy and the importance of evidence-based decision making.


The listening years

After academia, I joined a local government research team and that’s where things started to change. I had to learn to really listen to people to fully understand their needs. This time showed me the importance of empathy and communication—skills that have remained important throughout my career.


And as a commissioner of research, I also got to see what it's like being a client of a research agency. This showed me the importance of only promising what you can actually deliver!


The agency years

Next was a move into market research, mainly because it paid more! The pace was much faster and there was pressure on getting things done quickly and efficiently. And doing them online. For some of this time, I worked for the agency that pioneered online research and convincing clients (and indeed myself) of its merits was a challenge.  I still feel uncomfortable about applying probability based statistics to non-probability samples but acknowledge that that ship has well and truly sailed.


During the agency years, I had to be able to write questions that were both accurate and unbiased and surveys that were short and sharp. On top of that, I learnt how to blend qualitative insights with quantitative data to tell a more complete story. I also had to learn how to present insights in PowerPoint without sending the audience to sleep.


There was also a need to have a more commercial mindset. This meant making sure that the research had to directly help client businesses while at the same time making the agency money.


The analytics years

Today my work involves using text analytics, predictive modelling and techniques such as network analysis to extract customer insights. The tools I use have changed a lot, too—gone are the days of SPSS and PowerPoint; now it’s all about Python and advanced analytical techniques I didn’t even know existed when I started out.


It’s been hard work getting up to speed with all these new tools but it’s also been really exciting to see how much more we can do with the data we already have.


Looking back

Looking back, it’s clear that while the hard skills needed for research have evolved with the tools and technologies, the soft skills have remained pretty much constant. Being able to listen, communicate effectively and understand the needs of different people is still a huge part of the job. As an insight professional today, it’s not just about being technically skilled; it’s also about being able to translate complex data into insights that can really drive decision-making and make a difference.


The insight world is always changing and so are the skills we need to succeed in it. From the rigour of academic research to the fast-paced demands of online research and now to the data analytics focus of my current role, adaptability and a willingness to learn have been key.


It’s a constantly evolving field and I think that’s what makes it so exciting to still be be a part of 40 years on.

by Michael Wagstaff 8 April 2025
The huge volume of data available through consumer comments, reviews and surveys can make cutting through the noise difficult. In this article we discuss how text analytics combined with human expertise can uncover the insight.
by Michael Wagstaff 10 March 2025
Market research agencies are going all in on AI based models to generate next level consumer insight. But are these just more illusion than substance?
by Michael Wagstaff 3 March 2025
With the online survey already on the ropes due to poor quality, has data science finished it off?
by Michael Wagstaff 24 February 2025
Research agencies are pinning their futures on AI. Are they right to do so or are we missing trick by ditching the human?
by Michael Wagstaff 12 February 2025
Online surveys suffer from fake, ill considered and unrepresentative responses. What can be done to improve their reliability? Triangulation is the key.
by Michael Wagstaff 11 February 2025
With so many agency panels riddled with fake respondents resulting in poor quality data, are we witnessing the end of the online survey?
by Michael Wagstaff 6 February 2025
With the January transfer window closed, we run our predictive model to work out the probabilities of where each team will finish in the final Premier League table.
by Michael Wagstaff 5 February 2025
In this latest article in our series on the power of text analytics we look at how sentiment analysis can be used to really understand what customers think about product offerings.
by Michael Wagstaff 23 January 2025
The true value of review sites lies in going beyond the stars and analysing what reviewers are actually saying.
by Michael Wagstaff 17 January 2025
Start making sense of customer review and feedback data by using text analytics. In the first of a series of articles we discuss how it can help your business.
Show More