Time To Change stigma and discrimination: evaluation results

The April 2013 edition of the British Journal of Psychiatry included a special supplement of papers which focused on the Time To Change campaign.  This supplement included 7 evaluation studies of the first phase of the campaign, which ran between 2008 and 2011.

This Lifestyle Elf finds mental health promotion a fascinating area, and was excited to read these evaluation studies!  I’ve summarised the main findings for you below, and would love to know your thoughts about this campaign, and the direction of future mental health promotion.

What is ‘Time To Change’?

Time To Change  (TTC) is a large campaign aimed at reducing mental health stigma and discrimination in England.  The aims of the campaign were threefold;

  1. to increase public awareness of mental health,
  2. to empower people with mental illness to challenge discrimination, and
  3. to reduce discrimination and increase well-being through physical activity opportunities.

For further information, visit the TTC website here: http://www.time-to-change.org.uk/about

 Study 1: Public knowledge, attitudes and behaviour regarding mental illness

  • AIM: Assess whether public knowledge attitudes and behaviour towards people with mental illness has improved in England since the inception of TTC in 2009.
  • METHOD: Annual ‘Department of Health Attitudes to Mental Illness’ survey conducted with around 1700 quota-sampled respondents per year.
  • RESULT: Statistical analysis showed improvements in intended behaviour and a trend towards positive attitudes.  However no significant improvement in knowledge or reported behaviour, on a national level.

The authors concluded that the findings “provide support for the effectiveness of the national TTC programme against stigma and discrimination in improving attitudes and intended behaviour…” However the survey did not collect information about respondent awareness of TTC; thus it is difficult to know how much TTC contributed to these observed changes.  It is great that an annual national survey has been implemented, and this Lifestyle Elf hopes this will continue annually as the findings provide useful information regarding stigma-related outcomes by sociodemographic groups.

Study 2: Experiences of discrimination among mental health service users

  • Dogs were not included in the telephone survey

    Dogs were not included in the telephone survey

    AIM: Determine if TTC target to reduce discrimination by 5% had been achieved.

  • METHOD: Telephone questionnaire administered annually to mental health service users across different mental health trusts in England (2008 – 2011).
  • RESULTS: A small but significant overall increase in the proportion who had no experiences of discrimination was observed between 2011 and 2008. In all years, the majority of respondents reported at least one experience of discrimination. Discrimination in four of the 21 domains covered in the questionnaire increased slightly over the period (safety, benefits, marriage and transport).

The authors reported that there had been a reduction in reported discrimination amongst mental health service users, however they did not achieve the 5% target.  The key limitation of this study, acknowledged by the authors, is the low response rate (between 6 – 11%), making it difficult to feel confident that these results are representative.  This study highlights the difficulties in measuring discrimination on a population level, and as with study 1 – it is difficult to attribute the results directly the the TTC campaign.

Study 3: Newspaper coverage of mental illness in England (2008 – 2011)

  • paperAIM: Compare English newspaper coverage of mental health related topics each year.
  • METHOD:  Content analysis performed on articles found in 27 local and national newspapers, on two random days per month.
  • RESULTS: Significant increase in the proportion of anti-stigmatising articles between 2008 and 2011. No proportional decrease in stigmatising articles.  Decrease in mixed or neutral articles.

This research paper was reported well and the findings are encouraging.  However although this research was conducting during the same time frame as the TTC campaign – this is insufficient evidence to be confident that these changes were due to TTC.

Study 4: Mental health problems in the workplace (2006 – 2010)

  • Employees celebrating their new mental health workplace policy

    Employees celebrating their new mental health workplace policy

    AIM: to examine whether there had been improvements in mental health knowledge, attitudes and workplace practices among British senior employers

  • METHOD: Telephone survey of senior employers (sampled from all registered British businesses)
  • RESULTS: An increase in awareness of common mental health problems and workplace mental health policies was reported.  Employers reported being less likely to view colleagues attitudes to mental health as a problem in employing someone with a mental illness.

The authors reported that these findings were consistent with other surveys showing improves public attitudes to mental health and a reduction in experiences of workplace discrimination.  However there was a higher proportion of HR managers who participated in 2009 (49%) compared to 2006 (9%), and this is likely to bias the results as HR managers ought to be more knowledgeable about mental health and employment.  As well as the survey methodology changing over the years of observation, this Lifestyle Elf is a little skeptical of these results.

Study 5: Influence of TTC’s social marketing interventions on stigma

  • AIM: To assess the effectiveness of TTC interventions, specifically: a) mass media and b) mass social contact events
  • METHOD: a) on-line interviews before and after mass media social marketing, and b) questionnaires completed by adults who had attended a social contact event
  • RESULTS: No consistent improvement in attitude or knowledge at a population level, however small but significant improvement in relation in intended behaviour (especially following mass media intervention).  Awareness of the TTC campaign was positively associated with favourable knowledge, attitudes and behaviour.  Social contact, particularly good quality contact, was associated with positive attitude change and greater confidence to challenge stigma.

TTC has led the first national mental health anti-stigma social marketing campaign, which was evaluated in detail in this paper.  It was difficult to summarise the findings and lessons learnt from this complex intervention, so I would encourage you to read the full paper.  The short-term findings are promising, and this study shows that a social marketing approach has helped raise awareness of the TTC campaign.

Study 6: Anti-stigma training for medical students

  • medAIM: Assess the impact of Education Not Discrimination (anti-stigma training) 
    on medical students
  • METHOD: Both participants and controls completed questionnaire to assess stigma outcomes immediately and after 6 months with regard to knowledge, attitudes, behaviour and empathy.
  • RESULTS: Intervention group demonstrated significantly greater improvements in stigma-related knowledge and reductions in stigma-related attitudes and intended behaviour, relative to the control group. At 6 months’ follow-up, however, only one attitude item remained significantly better.

These results highlight that the anti-stigma benefits derived from training did not persist after 6 months, which the authors acknowledged to be disappointing.  It is also a limitation that only 2 out of the 4 medical schools involved were able to have a control group.  On a positive note, the study found that individuals with the most stigmatised attitudes towards mental health at baseline, benefited most from the intervention.

Study 7: Economic evaluation of the anti-stigma social marketing campaign

  • econAIM: To assess the economic impact of TTC’s social marketing campaign
  • METHOD: Campaign expenditure data was compared to outcome data. Return on investment was assessed using a decision model that estimated the impact on employment for people with depression.
  • RESULTS: Compared to other national public health social marketing campaigns, the economic benefits outweighed the financial costs.  Cost per person with improved intended behaviour was at most £4 (assuming TTC was responsible for 50% of the change).

These results are likely to be conservative estimates of the economic benefits, as the effects measured did not encompass all activities across the TTC campaign.  Assuming that TTC is an enabling factor supporting the population changes, these are very encouraging results.

 Final Thoughts

Initiatives to address mental health always tend to lag behind those for physical health, yet maybe, given time, the government’s mandate for mental health to be given equal priority to physical health will become a reality.  Numerous health campaigns have been implemented in the past, yet the TTC campaign was the first of it’s kind; addressing mental health stigma and discrimination on a national scale.

The benefits of working in partnership with academia (in this case the TTC evaluation was led by the Institute of Psychiatry (Kings College London)) to accompagny health improvement initiatives, are clear. It is fantastic to see such comprehensive evaluation – measuring attitudes, economics through to media coverage – which have provided a detailed picture of the range of activities and acheivments of the campaign.

There is however a recurring theme throughout these studies; the challenge of being able to evidence attribution.  The Bradford-Hill criteria (factors which help assess the evidence for causality) can be helpful to consider.  The TTC campaign is certainly a plausible explanation for the changes, and all the above evidence considers temporality (i.e. the intervention takes place before the outcomes).  It is difficult to determine that the changes are due specifically to TTC, and not to other factors (e.g. decline of anti-stigma attitudes in society generally, or result of other educational initiatives…etc).  Ideally, one would try to identify a ‘control’ for comparison – however this is often near-on impossible for population interventions, especially when your intervention covers a whole country.

The results of the TTC first wave evaluation are encouraging, and TTC report that they will build upon what they have acheived and learnt in moving into the second phase.

Overall, we aim to reach 29 million adults in England and increase the confidence of 100,000 people with mental health problems to challenge stigma and discrimination.

TTC is both ambitious and challenging; we at Lifestyle Elf wish them the best of luck and look forward to the second wave results!


Reducing Stigma and Discrimination: Evaluation of England’s Time To Change programme (2013) British Journal of Psychiatry, April 2013, Volume 202, Issue s55.  Table of Contents available here: http://bjp.rcpsych.org/content/202/s55.toc

The studies discussed above are available here:

  1. Sara Evans-Lacko,Claire Henderson,and Graham Thornicroft (2013) Public knowledge, attitudes and behaviour regarding people with mental illness in England 2009-2012, BJP April 2013 202:s51-s57 PDF
  2. E. Corker, S. Hamilton, C. Henderson, C. Weeks, V. Pinfold, D. Rose, P. Williams, C. Flach, V. Gill, E. Lewis-Holmes, and G. Thornicroft (2013) Experiences of discrimination among people using mental health services in England 2008-2011, BJP April 2013 202:s58-s63 PDF
  3. Amalia Thornicroft, Robert Goulden, Guy Shefer, Danielle Rhydderch, Diana Rose, Paul Williams, Graham Thornicroft, and Claire Henderson (2013) Newspaper coverage of mental illness in England 2008-2011, BJP April 2013 202:s64-s69, PDF
  4. Claire Henderson, Paul Williams, Kirsty Little, and Graham Thornicroft (2013) Mental health problems in the workplace: changes in employers’ knowledge, attitudes and practices in England 2006-2010, BJP April 2013 202:s70-s76; PDF
  5. Sara Evans-Lacko, Estelle Malcolm, Keon West, Diana Rose, Jillian London, Nicolas Rüsch, Kirsty Little, Claire Henderson, and Graham Thornicroft (2013) Influence of Time to Change’s social marketing interventions on stigma in England 2009-2011, BJP April 2013 202:s77-s88; PDF
  6. Bettina Friedrich, Sara Evans-Lacko, Jillian London, Danielle Rhydderch, Claire Henderson, and Graham Thornicroft (2013) Anti-stigma training for medical students: the Education Not Discrimination project, BJP April 2013 202:s89-s94; PDF
  7. Sara Evans-Lacko, Claire Henderson, Graham Thornicroft, and Paul McCrone (2013) Economic evaluation of the anti-stigma social marketing campaign in England 2009-2011, BJP April 2013 202:s95-s101; PDF; Data Supplement