How do we know our reviews ask the 'right' questions?

We asked Stephanie, one of our team of systematic reviewers, to write a series of blogs to help explain how we try ensure our reviews ask the 'right' questions. Stephanie's first blog introduces how we have worked with the James Lind Alliance to do this.  


Identifying priorities and asking the ‘right’ questions

When it comes to producing trusted evidence relating to mental health, the editorial base of the Cochrane Common Mental Disorders (CCMD) group works tirelessly. As part of Cochrane’s Mental Health and Neuroscience Network, not only are we working with systematic review authors across the globe, we also work with content and statistical specialists, peer reviewers, local NHS partners, charitable organisations, research satellites and Cochrane method groups, to help place relevant, timely evidence at the heart of healthcare decisions for people who experience common mental disorders. 

But, how do we know what the most pressing questions are? 

Since 2014, CCMD has worked with partners involved in four James Lind Alliance (JLA) priority-setting partnerships (PSP) covering the key topic areas of:

  1. mental health in children and young people;
  2. digital technology for mental health;
  3. bipolar; and
  4. depression.

The JLA, funded by the National Institute of Health Research (NIHR), has the scope of developing and using inclusive methods to identify priorities for medical research that are considered important by patients as well as their healthcare providers.

The James Lind Alliance, established in 2004 and endorsed by the Department of Health, was intended to serve as a communications and discussion forum on randomised controlled trials (RCT) between patients, practitioners and researchers. James Lind is often credited as the pioneer of the first clinical trial, when in 1747 the Scottish naval surgeon allocated sailors to one of six different daily treatments for a period of 14 days to investigate the best treatment for scurvy. Out of the six different treatments (including cider; dilute sulphuric acid; vinegar; sea water; a medicinal past of garlic, mustard seed, radish root and myrrh; or oranges and lemons), Lind found that ‘the most sudden and visible good effects were perceived from the use of oranges and lemons; one of those who had taken them being at the end of six days fit for duty.’ Though it took many more years after this experiment to truly understand the causes of scurvy (a deficiency in vitamin C), the foundations for the fair testing of treatments was laid, and it is under this philosophy that Lind was considered the perfect poster-boy to represent priority setting partnerships (PSPs) that cover various physical and mental health conditions.

Common mental health problems affect 1 in 6 people in any given week – meaning that most of us will ourselves have some experience of problems such as depression or anxiety. Or a family member, friend or colleague might be going through something similar, either with or without your knowledge.

As a result, there are likely to be questions that you want to ask about what you are experiencing, or about the treatments that may be offered. For example, will treatments work for me? What are the risks? Your doctor may be able to answer some of your questions using a combination of their clinical judgement, by using clinical guidance such as that provided by NICE clinical guidelines, and possibly by using evidence from systematic reviews such as those published in the Cochrane Library. However, there are still many important questions relating to mental health for which we do not have a clear answer. Researchers are constantly trying to find ways to identify these important areas of uncertainty so that they can address the gaps in our knowledge base. Finding these areas of treatment uncertainty is not always easy though, nor is it always inclusive of patients, service users or people with lived experience. 

Like other areas of research, the mental health research agenda has typically been shaped and led by groups other than people with lived experience, including the pharmaceutical industry, the research community of universities and clinical research academics, funding bodies, and healthcare policymakers. Until relatively recently, the value of involving patients and the public in research has been under-appreciated; however there is now undoubtedly a shift in culture, with many researchers engaging in greater levels of participant patient involvement (PPI), co-produced and user-led research. This partnering with patients in research design and conduct stage, as well as at the peer review and publication stages is intended to promote relevance, transparency, access to, and uptake of research. Cochrane is similarly calling for involvement from anyone who would like to join and contribute towards the work of the global organisations’ community (which currently includes 13,000 members and 50,000 supporters from over 130 countries – and growing!). 

Though there is still a long way to go to achieve optimal involvement of patients and the public in research, efforts are being made to support this. For example, funded and hosted by the NIHR, core support for identifying research priorities is offered by the James Lind Alliance, who abide by principles of inclusivity (equal involvement of patients, carers and clinicians working together as partners), transparency of methods, declaration of interests, and a commitment to using and contributing towards the evidence base. It’s these principles that help to ensure research questions are relevant to the people they concern. 

In the next blog, I’ll discuss in greater detail the Priority Setting Partnerships CCMD has been involved with, and give a personal reflection of my own hands-on involvement with the Digital Technology in Mental Health PSP.

Stephanie Sampson, March 2019


Stephanie is a Research Fellow in the Centre for Reviews and Dissemination at the University of York. Steph works closely with Cochrane Common Mental Disorders as part of an Economic and Social Research Council (ESRC) funded knowledge exchange collaborative with Tees, Esk & Wear Valley (TEWV) and Northumberland, Tyne and Wear (NTW) NHS Trusts. 

Previously, Stephanie has worked  with the Cochrane Schizophrenia Group at the Institute of Mental Health, University of Nottingham. Before working with CCMD she worked as a research consultant for MindTech’s Digital Technology in Mental Health James Lind Alliance Priority Setting Partnership (PSP), assisting health and social care workers, carers and people with lived experience develop their top ten questions for research.

Share