Co-design and Evaluation of Help Functionality
The Synergy Online System, a platform that delivers mental health care to young people, like most suicide prevention apps and e-tools at the time, only offered a simple call button with a long static list of services. Our previous research revealed that young people want a tiered system of help based on their self-identified level of need (e.g. psycho-education, online chat or emergency services). To co-design the Help functionality to meet young people's needs, particularly people in crisis (suicide prevention needs), and to evaluate the functionality, we engaged young people, supportive others and health professionals working with young people in crisis.
Context
The Youth Mental Health and Technology Team at the University of Sydney, researches and develops innovative treatments for young people with emerging mental health disorders. The Australian Government funded Project Synergy, a $5M grant to co-design, build, and evaluate the Synergy Online System.
The Platform is a customisable digital tool kit designed to be integrated into in-person mental health services to facilitate shared decision making between clinicians and clients, helping clients meet their mental health goals using the most effective treatment.
​
Clients complete an initial online evidence-based assessment to populate a dashboard giving them comprehensive feedback about their needs (such as sleep, mood, physical health, suicidal thoughts, and social connectedness). Clinicians can then use the dashboard to help inform clinical care and monitor outcomes. Clients can track their progress in real-time, primarily using self-reported psychometric measures, which is then reflected on their dashboard and used to continuously inform clinical care, enabling a collaborative approach to care.
Team
The team included academic researchers, UX researchers, clinicians, and technologists combing expertise in mental healthcare, clinical research and technology.
My contribution included project management, participant recruitment, co-design workshop: ideation and co-facilitation, prototyping and user testing.
We Tackled The Project in Three Phases
1
Discover: To illuminate what we didn't know, better understand how best to support young people in crisis (with suicide prevention needs) and design help functionality to meet their needs, we conducted ten co-design workshops and created a wireframe prototype.
2
Explore: To better understand the problem space, design scope and ensure the prototype addressed the young people's needs appropriately, we completed 23 user testing sessions.
3
Test: To evaluate the demand and acceptability of the help functionality, 101 young people across Australia received access to a clickable prototype of the functionality and completed an online survey.
1 Discover
We used co-design workshops as a discovery tool to explore opportunities rather than produce one final solution. Based on the idea that everyone can be creative, participants took on the role of ‘user-as-a-partner’.
To illuminate what we didn't know, better understand how to support young people in crisis, and create help functionality to meet their needs, we held ten co-design workshops. In total 80 participants attended the workshops including young people (16 - 30 yrs), supportive others and health professionals living in four regions of Australia with high rates of suicide - Central and Eastern Sydney; Murrumbidgee; Central Queensland, Wide Bay, Sunshine Coast; and, Country WA.
​
We designed the workshops to discover how best to support active suicide prevention strategies and understand young people's pain points and frustrations when accessing immediate mental health support online. We also wanted to understand their needs and preferences for improving immediate mental health support online.
​
The co-design process was iterative, whereby the learnings from each workshop informed the agenda of the next one. Although each workshop was unique, they concentrated on three stages:
1
Discovery to illuminate what we didn't know and better understand how to support young people in crisis online. We aimed to validate and discard assumptions. An example of a discover activity would be a prompted discussion about accessing help and the kind of support they desired.
2
Evaluation to better understand the problem space and how to address young people's needs when accessing crisis support online. An activity might include evaluating current solutions - what participants liked, disliked and why, in functionality, content and design.
3
Prototyping to develop solutions to address needs. For example, paper prototyping solutions to better understand the different types of users, their mental models, preferred content, task flows, ideal functionality and general wish-fors.
After each workshop, the team debriefed reflecting on key takeaways and surprise discoveries to understand themes which served as a basis for further ideation and conversing in future workshops. After we completed all of the workshops, we reviewed the data from each workshop for themes. Similar findings were grouped, including personas, user journey maps, written comments made by co-designers related to real-life examples and transcribed detailed qualitative notes of the workshops.
Outcomes
Together with young people, supportive others and health professionals, we co-designed possible help functionality solutions to meet young people's needs when in crisis.
We discovered that young people needed:
-
Content to be personalised, appealing in terms of content and style, accessible, non-stigmatising, interactive and provide a greater breadth of options, catering for different individuals' needs, rather than traditional static information-based "need help now" functionality.
-
Functionality consolidated and links provided with other online services, face-to-face services, and options for support and information (covering the full spectrum of young people's needs, from prevention and early intervention stages to the crisis stage).
We refined and converged the prototypes that came out of the workshop to create one prototype recommendation.
The prototype includes three levels of help:​
1
Services: provides support services that are available online or nearby. Online services allow users to select from a range of apps, e-tools and other online resources that are quality assured using the Mobile App Rating Scale. Nearby services feature a mapping system that allows the user to select service attributes via tags to find the closest .mental health services
2
​Talk: provides young people with the ability to talk with others about their situation when not in an immediate emergency, including call, online chat and rant options. Call directs users to a telephone service. Online chat allows the user to converse with someone by text. Lastly, rant enables the user to type or voice their feelings and thoughts in a personal built-in mood journal then save or delete it in a visually appealing way. Human moderators would support rant to respond efficiently and appropriately.
3
Help: offers immediate emergency support. This feature provides an instantly clickable link to talk with emergency services (Triple Zero [000]) and other 24-hour services like Lifeline, Suicide Call Back Service and Kids Helpline.
Deliverables
Reports
To share our findings with the broader team and stakeholders, a report was created for each workshop, including the workshop's goals, participant demographics, methodology, a list of key insights contextualised with supporting details, user quotes, and recommendations.
Wireframes
Using the valuable insights and prototypes gathered from the workshops, the solutions were refined and converged into a set of wireframes used to have conversations with the team about the design direction of the help functionality.
2 Explore
We used user testing to better understand the problem space and design scope and ensure the prototype addressed the young people's needs appropriately.
Using a low-fidelity prototype, we completed 23 in-person user testing sessions with young people (age 16 to 30 years), supportive others, health professionals in four regions of Australia with high rates of suicide. Lived experience of suicidal ideation or behaviour was not an inclusion criterion, nor was it an exclusion criterion.
​
We designed the testing to discover problem areas, reveals ways to improve the functionality and understand how well the designs worked.
Outcomes
Identification of problems example
Participants frequently made a navigation error when asked to go to where they'd expect to find the number for emergency services for immediate support, navigating to the 'Talk' page instead of 'Help'. When asked 'why', many users said the pages' names were ambiguous, which led to confusion.
Opportunities to improve example
Many users suggested referrals to national and international health services should be added to resources offered.
Users behaviour and preferences example
The rant feature was the most liked.
Deliverables
Report
To share our findings with the broader team and stakeholders, we created a report for each workshop, including the workshop's goals, participant demographics, methodology, a list of key insights contextualised with supporting details, user quotes, and recommendations.
Clickable Prototype
The findings of the concept testing informed the design of a high-fidelity prototype.
3 Test
We used feasibility testing to evaluate the demand for and acceptability of the help functionality.
One hundred one participants (young people, supportive others, and health professionals) across Australia received access to a high-fidelity prototype and completed an online survey. Lived experience of suicidal ideation or behaviour was not an inclusion criterion, nor was it an exclusion criterion.
​
The online survey collected demographic information, device and internet use, usefulness, the likelihood of use, likability of components, rating on a 5 point Likert scale, an adapted version of the System Usability Scale, and open-ended qualitative feedback.
​The survey descriptive and frequency data were analysed using SPSS. "What were your first impressions of the functionality?" was coded using NVivo.
Outcomes
Some of the key findings:
-
Components that participants liked the most included: the mapping system (young people, 64%; supportive others and health professionals, 49%); recommended apps and e- tools (young people, 53%; supportive others and health professionals, 42%);
-
Rant component was one of the best-liked for many participants, (young people, 58%; particularly young people supportive others and health professionals, 36%). Though, it was also the most divisive component, with the most significant proportion of participants rating it as the least liked.
Constraints
Although recruitment of individuals was mainly through mental health service channels, the young people who participated would not necessarily have lived experience of suicide-related thoughts or behaviour, either personally or as a supportive other. Participants were not directly asked about these experiences due to ethical concerns, which could have provided another level of validity to the Help functionality.
​The online survey was designed only to assess acceptability and gather basic usability feedback. We know participants liked the functionality, but we did not test whether any measurable changes resulted from its use. Participants only had access to a prototype for viewing and feedback purposes, so they could not directly trial its use.
Impact
The tiered system of help (three levels) was added to the product roadmap, excluding the rant feature, which needed further safety evaluation.
What I Learnt
Always, but especially when researching sensitive topics, it's most important to create a "safe space" so the essential elements of co-design — building trust, finding a voice, sharing perspectives, and creating a shared vision — can occur.