How two documents, and regular critiques lead to high quality research studies.
Leading a growing research team can be challenging. Being a good line manager is a full time job in itself. As well as being responsible for the people, research leaders also set the bar for the quality of the work being done by the team. Making sure research is high quality can suffer when it relies on a single research leader to enforce best practice.
When teams are small, a lot of the knowledge sharing which helps researchers excel can be managed through 1:1 interactions – talking amongst researchers and sharing best practice. However as teams scale this becomes more difficult, and it becomes necessary to document ‘how we do research here’.
Having a defined research process, combined with a culture of continual improvement, can help with this. In this article we’ll look at how to do this, and how defining and describing the research process can help research teams mature faster.
Why describe ‘how research works’ to researchers?
User research informs the decisions that product teams are making throughout development. These decisions are usually time sensitive, and slow studies will miss opportunities to influence those decisions, reducing the impact of research. Over time, user researchers identify efficiencies to shave time off the research process, speeding up study preparation, analysis or debriefing.
Research also needs to be high quality in order to earn trust and credibility from other disciplines. Studies go wrong in endlessly interesting ways. Participants go to the wrong building, the microphone is not turned on and audio is lost, or meeting invites are sent too late meaning that no-one comes to the debrief. Encountering these problems and deciding how to prevent them happening again means that the quality of studies is continually improving.
Being both ‘fast enough’ and ‘high quality’ is essential for a new research team to have an impact. These requirements for ‘speed’ and ‘quality’ means that each research team independently identifies some best practise, such as ‘what information do we give to participants before they attend’, ‘who do we inform when each study occurs’ or ‘where do we share our findings’. This guidance is usually bespoke to each organisation, determined by who the decision makers are, where the team is situated and who researchers work with.
Documenting the research process captures those best practices. This allows the knowledge to be shared, and prevent team members making the same mistakes – allowing a whole bunch of more interesting mistakes to be made instead. This is particularly valuable when training junior team members who are less experienced with running research studies.
It also reduces the ‘bus factor’ for the research team – how much information would be lost if one or two key researchers were hit by a bus and their knowledge was lost. Less tragically, if key team members did decide to leave the company, documenting what they know helps ensure that the team’s ability to deliver survives.
Two documents to document research practice
At the heart of describing how research works are two documents, the research checklist and the knowledge base.
The research checklist
Part of a research checklist
The research checklist describes every step that a researcher goes through for running a round of research. Starting from what to do before scheduling a research kick-off, it includes reminders for every step when preparing a study, and how to wrap up after a study is complete. An example checklist can found on this list of research templates.
Although the checklist works as a printed document, it could be made in trello or through other task management software. Template tickets can be made for each project, which makes tracking the progess of a new project simple. An example of a research trello board, with checklist tickets, is also available as a free template.
Many of the steps on the checklist will link to more information, for example “contact the participant recruiter”, could link to a page with the specific information on how to do that, and what information to include. That’s where the knowledge base comes in.
The knowledge base
Although the checklist can describe what the steps to run a study are, it doesn’t give any information on ‘how to do this’. A knowledge base fills this gap.
OneNote is one way of storing research process information in a structured way
A knowledge base is used to write down ‘here is some relevant information on how to do this’. Communal document writing tools, like OneNote or Google Docs, can allow the whole team to contribute templates, explanations and things to watch out for on each step of the research process.
Some tools also support hierarchy, allowing structure to be created – such as having subsections under participant recruitment for sub-topics, such as ‘what information do we need to give our recruiter’, ‘what are the contact details for our recruiter’, or ‘where do we keep the recruitment budget spreadsheet’.
By writing down this information, it will help disseminate it amongst the whole team, useful as team sizes grow and 1:1 knowledge sharing between all researchers is no longer possible.
Critiquing the research process
There’s a risk that describing how research works can freeze progress. Having ‘this is how we do research’ written down can discourage researchers from trying different approaches, and can become a crutch that reduces creativity. This is not what we want to encourage.
When introducing these documents to a team, be clear that the research process isn’t prescriptive, and a study doesn’t have to be run using the checklist or the way advised in the knowledge base. Researchers should be free to use their professional judgement to decide what the best approach is – the goal from this process is consistency, but not uniformity. However when there are deviations from the ‘normal’ process, this should be investigated to decide whether there’s anything new that the rest of the team could benefit from.
To achieve this, after every study, the research team should run a critique of how the study went – focused on the research process, not the findings. This meeting should look out for:
- How did the study differ from the process currently described?
- Did anything go wrong in the study that we should fix?
When the study deviates from the written process, the team should look at why, and decide whether there is anything new that can be incorporated into the knowledge base. This helps share any new best practice with the rest of the team, and creates the opportunity for the whole team to benefit from the new approach.
When things have gone wrong in a study, there is an opportunity to reflect on what could be done differently to avoid this happening again. This could lead to new guidance being written in the knowledge base, or additional items on the checklist.
Updates based on these meetings can feature in team meetings, to make sure that everyone is aware when the team’s documented best practise changes.
A continually improving research team
By using these documents to capture the team’s best practice, and encouraging researchers to continually review how they are approaching planning a study, the team will continually be improving how they run studies, and adapt to new challenges.
This gives research leaders time to focus on other aspects of the role, such as advocating within their company for user research or being an excellent line manager. It also creates opportunities for more junior researchers to take ownership over improving an aspect of how the team works, and develop their own experience. The book Building User Research Teams has more guidance on how to decide and track team tasks to help improve a user research team over time, and includes more helpful templates on it’s website.
I was introduced to this process by David Tisserand, who has written more about the steps involved in defining a research process in the book Games User Research. Thanks David!
Leave a Reply