V1 Clinical Evaluation Report

March 12, 2021

Uncategorized

By Matt2012

Published March 2020

Reviewed March 2021

Author: Dr Greg Burch: Clinical Director, Tiny Medical Apps

1.0 Executive Summary

2.0 Background

3.0 National assessment and local assurance

4.0 Evaluation

4.1 Content evaluation and evidence base

Evidence for asthma plans and self-management education reducing hospital attendances

NICE Asthma Quality Standard (QS25) Sept 2018

Validated Behaviour Change Techniques

4.2 Digital Health Passport assessment against NICE standards

The NICE Digital Health Technology tiers

Tier 2 Evidence for the Digital Health Passport

Partial Evidence for Tier 3a

4.3 NASSS Framework Evaluation

Assessment overview

4.4 Clinician Evaluation

4.5 Patient Evaluation

4.6 Technical Assessment

Evaluation Report Appendices

Appendix 1 – Clinician co-design and feedback

Appendix 2 – Patient feedback

1.0 Executive Summary

Thanks to the investment from Healthy London Partnership, the Digital Health Passport has overcome all of the barriers needed to be ready to scale. Projects are already underway in Manchester and Sheffield capitalising on the work done to date. This report gives some background to the work before the pilot and focuses on the initial evaluation work. Evaluation will be continuous as we gain more data and increase user numbers.

The Digital Health Passport

has been designed for teenagers to take control of their health and has an initial focus on asthma self-management with Asthma UK action plans.
enables remote tracking of symptoms, displaying emergency plans and accessing NHS support.
has been co-designed with young people and health care professionals including school nurses, asthma nurses, GPs and consultant asthma specialists from paediatrics and adult services.
addresses key recommendations in: NICE Asthma Quality Standard, BTS/SIGN clinical guideline 153, GINA 2019 and The National Review of Asthma Deaths (NRAD 2014).
meets all high level NHS assurances and has been accepted into the NHS Apps Library and is assured to use NHS login.
has been assessed to contain 14 Validated Behaviour Change Techniques.

From the NICE Digital Evidence Standards Framework the DHP demonstrates evidence of effectiveness at Tier 2 and partial evidence for Tier 3a.

TMA have evaluated the DHP Pilot using the NASSS Framework to inform both future adoption, use and commissioning.

Interviews were carried out with users of the DHP and gave overwhelmingly positive feedback. Valuable information has been obtained to influence the design and adoption of the next version.
Twelve site visits to observe and evaluate use of the portal and seven follow up interviews with clinicians were conducted to ensure their requirements were met.

2.0 Background

The Digital Health Passport has been designed for young people to take control of their health – creating asthma action plans, tracking symptoms and accessing NHS support.

The project has been led and commissioned by the NHS Healthy London Partnership, Children & Young People’s team as part of their work to improve asthma standards in London. In the past few years young people in the UK have had worse outcomes from asthma than in most other countries in Europe, and there have been a number of preventable asthma deaths.

Young people with a personalised asthma action plan are four times less likely to go to A&E – so a key feature of the app is the action plan from Asthma UK which gives instructions and advice of what to do if your asthma is getting worse.

The Digital Health Passport has been co-produced with young people, school nurses, GPs and asthma specialists in east London and is now available in the NHS Apps Library by invite only as it is further refined with the first users. It is being piloted and tested at the Royal London and Barts hospitals and at Chrisp Street GP practice in Tower Hamlets.

This innovative project has been recognised by NHS England and is one of only a handful of ‘Personal Health Record’ Apps to be evaluated around the country. It is now being rolled out in Greater Manchester and South Yorkshire with additional features planned to support young people with allergies, epilepsy and other long-term conditions.

The main features of the Digital Health Passport are

Asthma UK action plan
Emergency plan
Track symptoms on a visual timeline
NHS health advice and Asthma management educational information
Air quality levels (pollution, pollen and weather changes)

3.0 National assessment and local assurance

The Digital Health Passport requires a high level of regulatory assurance.

Significant effort has gone into delivering a safe and assured platform.

NHS Apps Library

Gaining acceptance to the NHS Apps Library has been time consuming, but is a huge assurance hurdle that has been overcome. The bedrock of the process is the Digital Assessment Questionnaire (DAQ) which requires passing assurance in seven domains from clinical need, evidence to information governance and security.

NHS Login

The developers Tiny Medical Apps were invited to be in the first wave of companies allowed to use NHS login. We have invested in attaining all of the additional assurance requirements to enable rapid integration with regional Local Health & Care Records such as One London.

Clinical Safety DCB 0129, DCB 0160:

These standards provide a set of requirements suitably structured to promote and ensure the effective application of clinical risk management by those health organisations that are responsible for the deployment, use, maintenance or decommissioning of Health IT Systems within the health and care environment.

ISO/IEC 27001

ISO/IEC 27001 Information Security Management system is designed to help organisations manage their information security processes in line with international best practice

Our certification is externally audited by BSI and our scope specifically covers the Digital Health Passport platform.

Cyber Essentials +

Cyber Essentials helps us to guard against the most common cyber threats and demonstrates our commitment to cyber security. We are Cyber Essentials + certified which means we are also externally audited.

SCAL

The Supplier Conformance Assessment List (SCAL) is a technical document which details the consumer supplier approach to information governance, clinical safety, functional testing and SMSP-PDS requirements. As part of our compliance and conformance assessment for NHS Login we successfully completed the requirements of the SCAL.

Data Security and Protection Toolkit

The Data Security and Protection Toolkit is an online self-assessment tool that allows organisations to measure their performance against the National Data Guardian’s 10 data security standards.

All organisations that have access to NHS patient data and systems must use this toolkit to provide assurance that they are practising good data security and that personal information is handled correctly.

4.0 Evaluation

4.1 Content evaluation and evidence base

The content of the Digital Health Passport is consistent with the best evidence for reducing asthma exacerbations and reducing unplanned hospital attendances as part of a supported case management approach. The asthma plan and the educational content is provided by Asthma UK and the NHS

Having a personalised asthma action plan is a NICE quality standard.

Improving outcomes will come from behaviour change and patient activation. From the first pilot we are introducing behaviour change techniques that will expand, test and refine in future iterations. A Queen Mary’s University study demonstrated 14 behaviour change techniques within the Digital Health Passport app.

Evidence for asthma plans and self-management education reducing hospital attendances

The evidence in favour of supported self-management for asthma is overwhelming. Self-management including provision of a written asthma action plan and supported by regular medical review, almost halves the risk of hospitalisation, significantly reduces emergency department attendances and unscheduled consultations, and improves markers of asthma control and quality of life (Pinnock, Breathe 2015).

The British Thoracic Society/Scottish Intercollegiate Guideline Network (BTS/SIGN) asthma guideline cites 261 randomised controlled trials reported in 22 systematic reviews in support of its grade A recommendation that “all people with asthma (and/or their parents or carers) should be offered self-management education which should include a written personalised asthma action plan and be supported by regular professional review”

NICE Asthma Quality Standard (QS25) Sept 2018

NICE Quality Statement 1

“People (5 years old and over) with asthma discuss and agree a written personalised action plan.”

https://www.nice.org.uk/guidance/qs25/chapter/Quality-statements

What the quality statement means for each audience:

Service providers ensure systems are in place for people with asthma to receive a written personalised action plan.
Healthcare professionals ensure they give people with asthma a written personalised action plan.
Commissioners ensure they commission services that give people with asthma a written personalised action plan.
People with asthma receive a written plan with details of how their asthma will be managed.

A written personalised action plan (such as Asthma UK’s asthma action plan) should be tailored to the person with asthma, enabling them to recognise when symptoms are worse. The plan should set out actions to be taken if asthma control deteriorates and who to contact.

Source guidance:

NICE guideline NG80, (2017) Asthma: diagnosis, monitoring and chronic asthma management, recommendations 1.10.1 and 1.10.2
BTS/SIGN clinical guideline 153(2016) British guideline on the management of asthma, recommendation 5.2.2
GINA 2019 Global Initiative for Asthma Ch 3, pg69
The National Review of Asthma Deaths (NRAD 2014) recommended the use of Personalised Asthma Action Plans as have multiple Coroners’ reports into avoidable deaths from asthma.

Validated Behaviour Change Techniques

The provision of an agreed self-management plan and educational materials are core features of the Digital Health Passport, however in order to maximise the potential of the tool we are including validated behaviour change techniques and plan to test their effectiveness in a large randomized controlled trial in east London in the coming years.

An analysis by Dr Samaresh Mazumdar and Dr Liz Edwards, under supervision of Prof Chris Griffiths and Dr Anna De Simoni of Queen Mary’s University London identified the 14 BCT’s in use in the app and recommended additional methods we could introduce based on analysis of 50 asthma apps used internationally. We jointly identified the Australian app ‘Kiss My Asthma’ as the leader in the field and worth emulating in many respects. Future versions will incorporate and evaluate further BCTs based upon the recommendations from QMUL researchers, particularly with greater ability to contribute to the care plan from patients with regard to goal setting, action planning and thus increased status within the team. Features such as medication reminders and ‘gamification’ are being introduced into the next version.

From the BCT taxonomy we can demonstrate the use of the following techniques:

1.2 – Problem Solving – with the use of information provided in videos about avoiding triggers

1.4 – Action Planning – in the action plan/emergency

2.3 – Self Monitoring of behaviour – logging of peak flows

2.4 – Self monitoring of outcomes of behaviour – symptom logging

3.1 – Social support unspecified – ‘my team’ section

4.1 – Instruction on how to perform behaviour – video instructions on PEFR/spacer use

5.1 – Information about health consequences – outlined in videos

5.4 – Monitoring of emotional consequences – mood log

6.1 – Demonstration of behaviour – video instructions on PEFR/spacer use

8.1 – Behavioural Practice – videos and encouraging daily peak flows/preventer use

8.3 – Habit Formation – encouraging daily use through the timeline/ calendar homepage

9.1 – Credible Source – Asthma UK/NHS branding

11.1 – Pharmacological Support – encouraging the use of inhalers

15.3 – Focus on past success – calendar homepage showing previous good days

Full taxonomy by Michie et al

It contains 93 techniques to change behavior that are hierarchically clustered into 16 groups.

4.2 Digital Health Passport assessment against NICE standards

The Digital Health Passport can now demonstrate evidence of effectiveness at Tier 2 and partial evidence for Tier 3a (Behaviour change techniques).

The NICE Digital Health Technology tiers

Digital Health Technology (DHTs) are classified by function and stratified into evidence tiers (based on level of risk)

Tier 1

DHTs with potential system benefits but no direct user benefits

Tier 2

DHTs which help users to understand healthy living and illnesses but are unlikely to have measurable user outcomes.

Inform
Simple monitoring
Communication

Tier 3a

DHTs for preventing and managing diseases. They may be used alongside treatment and will likely have measurable user benefits.

Preventative behaviour change
Self-manage

Tier 3b

DHTs with measurable user benefits, including tools used for treatment and diagnosis, as well as those influencing clinical management through active monitoring or calculation. It is possible DHTs in this tier will qualify as medical devices.

Treat
Active monitoring
Calculate
Diagnose

We aim to demonstrate complete evidence of effectiveness at Tier 3a in the next 12 months by evaluating with licensed Patient Activation Measure scores (skills, knowledge and confidence to self-manage).

Higher-risk Digital Health Tools require a higher level of evidence for the Tier. Children and vulnerable groups are at higher risk. This means a higher level of evidence is required for the Digital Health Passport than if it was only for adults.

Tier 2 Evidence for the Digital Health Passport

Credibility with UK health and social care professionals (tier 1)

“Has a plausible mode of action and reflects current standard/best practice in the UK health and social care system or provides an alternative to standard/best practice that is beneficial to users and the health and social care system”

A large number of asthma specialists, adult and paediatric have been involved in the co-design of this product from inception including Prof Chris Griffiths, (Deputy Director Asthma UK, Centre for Applied Research, Dr Chinedu Nwokoro (Children’s Asthma Lead, Royal London), Dr Paul Pfeffer (Severe Asthma Lead Adult, Bart’s Hospital), Tori Hadaway (Community Asthma Nurse), Dr Richard Iles (Paed Resp Cnslt Evelina), The Tower Hamlets School nurse team and multiple other stakeholders.

The content evaluation demonstrates use of asthma care plans that are the recommended best practice with a strong evidence base for the paper equivalent. The content and behaviour change techniques provide a plausible mode of action.

Relevance to current care pathways in the UK health and social care system (tier 1)

“For the best practice standard, evidence could include published or unpublished reports describing the successful implementation of the DHT showing benefits to users in the UK health and social care system.”

Having an asthma action plan, receiving advice on inhaler technique and completing symptom diaries are fundamental parts of asthma care pathways. We can demonstrate improvements in care pathways for clinical and patient users (eg, the ability to generate a completed pdf that can be uploaded without scanning, ability to send asthma action plan directly to a patient’s mobile phone, the ability for a patient to record a symptom diary on their phone and easily access educational resources).

Acceptability with users (tier 1)

“Some evidence to show that potential users of the DHT have tested it and found it to be usable and useful will help to show that implementing the DHT may be successful. Evidence could include reports from user or user group testing, or showing that users have been consulted in the design and development process.”

Patient and Clinician users have been interviewed and the feedback can be provided upon request. See Appendices 1,2,3.

Equalities considerations (tier 1)

“Consider whether the DHT helps to reduce any existing inequalities within the health and social care system. This could include factors such as digital exclusion, or use by hard-to-reach populations.”

“Indicate any equalities considerations needed when commissioning, adopting or implementing the DHT, particularly in reference to the Equality Act 2010.”

Digital Health Technology may have unforeseen consequences such as creating a two-tier system through digital exclusion – this may become more of a risk with the introduction of NHS login to access some services. The Digital Health Passport is an alternative to current paper based pathways which should remain in place.

Reliable information content (tier 2)

“Any information or advice to users concerning health, healthy living, lifestyle, diseases, illnesses or conditions must be correct and relevant.“

The content for the Digital Health Passport comes from trusted and reliable sources: Asthma UK and NHSgo.

Ongoing data collection to show usage of the DHT (tier 2)

“To ensure value for money to the health and social care system, the DHT owner must commit to providing data showing that the DHT is used as expected by the intended user group after adoption.”

Used ‘as expected’ defined as one of the following:

– view their care plan

– view their emergency plan

– complete a symptom tracker form (asthma review)

– watch a video, or link out to NHS go

– check an air quality or pollen level

For some people to do 2 or more of the following:

– view their care plan

– view their emergency plan

– complete a symptom tracker form (asthma review)

– watch a video, or link out to NHS go

– check an air quality or pollen level

For some people to do any of the following, on multiple occasions:

– view their care plan

– view their emergency plan

– complete a symptom tracker form (asthma review)

– watch a video, or link out to NHS go

– check an air quality or pollen level

Evidence of patient usage demonstrating that users have met these requirements is available upon request..

Ongoing data collection to show value of the DHT (tier 2)

“To ensure value for money to the health and social care system, the DHT owner must commit to providing data demonstrating that people using the DHT are showing the expected benefits from its use. This could include improvements in symptoms or general health measures.”

It is too early to show any improvements in symptoms or general health measures. This will require a much more robust evaluation. Over the next 12 months, whilst demonstrating Tier 3a evidence we will use the validated Patient Activation Measure score. This is a 13 question system to assess an improvement in skills, knowledge and confidence to self-manage.

Quality and safeguarding (tier 2)

“Some DHTs provide chat platforms or peer-to-peer communication, or link the user to support from third-party organisations. The DHT owner should be able to clearly identify who the user can interact with, describe why these interactions are appropriate, any risks in those interactions, and what safeguarding measures have been put in place.”

N/A – There is no 2 way communication from within the DHP

Partial Evidence for Tier 3a

Use of appropriate behaviour change techniques (tier 3a)

“DHTs that aim to change the behaviour of the users should be consistent with accepted and effective behaviour change techniques. The DHT owner should be able to describe which behaviour change techniques are used and provide references to these”

See content evaluation

From the BCT taxonomy we can demonstrate the use of 14 BCTs

Demonstrating effectiveness (tier 3a, best practice standard)

“A high quality intervention study using a quasi-experimental or experimental design would compare the effect of the DHT on a group of users with 1 or more groups having a different (or no) intervention. The study would report the difference between the groups. It would include statistical considerations such as sample size and statistical testing, report outcomes that are relevant to the condition, and be clear on reporting the outcomes of every person in the group testing the DHT. Ideally, the comparator group would be people having current standard care, but it could also be a before-and-after study (measuring people’s symptoms over a period of time before they use the DHT then comparing this with while they are using the DHT).“

Evidence plan – Use of PAMs with a larger number of users as a before and after study.

4.3 NASSS Framework Evaluation

Many promising technological innovations in health and social care are characterized by non adoption or abandonment by individuals or by failed attempts to scale up locally, spread distantly, or sustain the innovation long term at the organization or system level.

The NASSS framework has been developed by Trish Greenhalgh and others to be used at any time in a project lifestyle

Projects with too many domains in the complex/complicated zones will fail to achieve scale and sustainability. It can be seen that the Digital Health Passport is well positioned to scale.

https://www.ncbi.nlm.nih.gov/pubmed/29092808

Assessment overview

Current assessment position of DHP is highlighted in bold (updated March 2021)

Domain/question		Simple	Complicated	Complex
Domain 1: The condition or illness
	1A. What is the nature of the condition or illness?	Well-characterized, well-understood, predictable	Not fully characterized, understood, or predictable	Poorly characterized, poorly understood, unpredictable, or high risk
	1B. What are the relevant sociocultural factors and comorbidities?	Unlikely to affect care significantly	Must be factored into care plan and service model	Pose significant challenges to care planning and service provision
Domain 2: The technology
	2A. What are the key features of the technology?	Off-the-shelf or already installed, freestanding, dependable	Not yet developed or fully interoperable; not 100% dependable	Requires close embedding in complex technical systems; significant dependability issues
	2B. What kind of knowledge does the technology bring into play?	Directly and transparently measures [changes in] the condition	Partially and indirectly measures [changes in] the condition	Link between data generated and [changes in] the condition is currently unpredictable or contested
	2C. What knowledge and/or support is required to use the technology?	None or a simple set of instructions	Detailed instruction and training needed, perhaps with ongoing helpdesk support	Effective use of technology requires advanced training and/or support to adjust to new identity or organizational role
	2D. What is the technology supply model?	Generic, “plug and play,” or COTS solutions requiring minimal customization; easily substitutable if supplier withdraws COTS: customizable, off-the-shelf.	COTS solutions requiring significant customization or bespoke solutions; substitution difficult if supplier withdraws	Solutions requiring significant organizational reconfiguration or medium- to large scale-bespoke solutions; highly vulnerable to supplier withdrawal
Domain 3: The value proposition
	3A. What is the developer’s business case for the technology (supply-side value)?	Clear business case with strong chance of return on investment	Business case underdeveloped; potential risk to investors	Business case implausible; significant risk to investors
	3B. What is its desirability, efficacy, safety, and cost effectiveness (demand-side value)?	Technology is desirable for patients, effective, safe, and cost effective	Technology’s desirability, efficacy, safety, or cost effectiveness is unknown or contested	Significant possibility that technology is undesirable, unsafe, ineffective, or unaffordable
Domain 4: The adopter system
	4A. What changes in staff roles, practices, and identities are implied?	None	Existing staff must learn new skills and/or new staff be appointed	Threat to professional identity, values, or scope of practice; risk of job loss
	4B. What is expected of the patient (and/or immediate caregiver)—and is this achievable by, and acceptable to, them?	Nothing	Routine tasks, eg, log on, enter data, converse	Complex tasks, eg, initiate changes in therapy, make judgments, organize
	4C. What is assumed about the extended network of lay caregivers?	None	Assumes a caregiver will be available when needed	Assumes a network of caregivers with ability to coordinate their input
Domain 5: The organization
It is assumed organisations will have limited slack resources	5A. What is the organization’s capacity to innovate?	Well-led organization with slack resources and good managerial relations; risk taking encouraged	Limited slack resources; suboptimal leadership and managerial relations; risk taking not encouraged	Severe resource pressures (eg, frozen posts); weak leadership and managerial relations; risk taking may be punished
There is currently strong appetite for Digital tools across the NHS	5B. How ready is the organization for this technology-supported change?	High tension for change, good innovation-system fit, widespread support	Little tension for change; moderate innovation-system fit; some powerful opponents	No tension for change; poor innovation-system fit; many opponents, some with wrecking power
The formation of ICSs will likely lead to improved regional decision making and faster adoption	5C. How easy will the adoption and funding decision be?	Single organization with sufficient resources; anticipated cost savings; no new infrastructure or recurrent costs required	Multiple organizations with partnership relationship; cost-benefit balance favorable or neutral; new infrastructure (eg, staff roles, training, kit) can mostly be found from repurposing	Multiple organizations with no formal links and/or conflicting agendas; funding depends on cost savings across system; costs and benefits unclear; new infrastructure conflicts with existing; significant budget implications
	5D. What changes will be needed in team interactions and routines?	No new team routines or care pathways needed	New team routines or care pathways that align readily with established ones	New team routines or care pathways that conflict with established ones
	5E. What work is involved in implementation and who will do it?	Established shared vision; few simple tasks, uncontested and easily monitored	Some work needed to build shared vision, engage staff, enact new practices, and monitor impact	Significant work needed to build shared vision, engage staff, enact new practices, and monitor impact
Domain 6: The wider context
	6A. What is the political, economic, regulatory, professional (eg, medicolegal), and sociocultural context for program rollout?	Financial and regulatory requirements already in place nationally; professional bodies and civil society supportive	Financial and regulatory requirements being negotiated nationally; professional and lay stakeholders not yet committed	Financial and regulatory requirements raise tricky legal or other challenges; professional bodies and lay stakeholders unsupportive or opposed
Domain 7: Embedding and adaptation over time
	7A. How much scope is there for adapting and coevolving the technology and the service over time?	Strong scope for adapting and embedding the technology as local need changes	Potential for adapting and coevolving the technology and service is limited or uncertain	Significant barriers to further adaptation and/or coevolution of the technology or service
Dependent on organisation	7B. How resilient is the organization to handling critical events and adapting to unforeseen eventualities?	Sense making, collective reflection, and adaptive action are ongoing and encouraged	Sense making, collective reflection, and adaptive action are difficult and viewed as low priority	Sense making, collective reflection, and adaptive action are discouraged in a rigid, inflexible model

4.4 Clinician Evaluation

The following clinicians have given feedback into the design, functionality and validation of the Digital Health Passport during the pilot phase:

Dr Chin Nwokoro (Paediatric Consultant and Respiratory Lead, Royal London Hospital)

Dr Paul Pfeffer (Respiratory Consultant RLH and Difficult Asthma Lead for East London)

Dr Julia Moody (GP, Chrisp Street and CCG Lead for Children and Maternity)

Dr Jim Cole (GP, Chrisp Street, and QMUL Clinical Effectiveness Group0

Charlotte Carrick (Children’s Respiratory Nurse, RLH)

Tori Hadaway (Community Asthma Nurse, Tower Hamlets)

Rachel McCready (Practice Nurse, Chrisp St)

Jane Simpson (Respiratory Nurse, Bart’s)

Anne-Marie Casey (Respiratory Nurse, Bart’s)

Dr Jonathan Grigg (Paediatric Consultant, RLH)

Summary of key points

Twelve site visits to observe and evaluate use of the portal
Seven follow up interviews were conducted
High ‘Did not attend’ rate and overall low number of teenage patients attending asthma reviews despite proactive measures
Nurses are the main users, then GPs, not used by consultants
Feedback about the patient app was very positive
Feedback about the clinical portal was mixed
- It is functional and little training is required to use
- Issues around remembering another login
- Not interoperable with EMIS (or any system)
- Time constraints
- Staff awareness (forget to use or to book longer appointments)
Change requests to portal were prioritised and implemented if considered high enough priority during the pilot (eg re-engineering for old versions of MS Explorer)
Changes need to made to work flow processes as well as technology
Targeting CYP in schools may be better approach

See Appendix 1 (Clinician feedback)

4.5 Patient Evaluation

“Easy to log onto and good to know what level my asthma is at – knowing if I need to wear a coat or hat is really good.” Alice, DHP User

“We are always trying to encourage Alice to look after her asthma herself and it helps her be independent…..really great as she has been discharged from hospital and the app helps her monitor her asthma and she lets us know how she is getting on” Susan, Alice’s Mum

“I really liked it because it was simple to use” Jared, DHP User

“The look and feel is really good – not just a boring NHS App and the interactive background is especially good for younger users” Lucy, DHP User

“As an asthmatic in the last two years I’ve been to A&E twice and if I’m having trouble breathing I can take my phone out and say look at this plan. That would be really handy. The name as a Digital Health Passport is exactly it – you can travel around with it and use it as and when you enter into a service.” Saira, DHP User

“It’s really cool” Robert, DHP User

Summary

Lower initial signup numbers than anticipated
Useful and usable insights gained
Easy to use and no negative comments about design or functionality
Most people use some features
Small number of users use intensively
Many additional feature requests and ideas to further improve the design

See Appendix 2 (User feedback)

4.6 Technical Assessment

In order to guide the next stages of the development of the Digital Health Passport Tiny Medical Apps have been commissioned by Healthy London Partnership to appraise the technical requirements of scaling the platform in London. This has involved a number of technical meetings with stakeholders in London and outside which are listed in the attached separate report. The key ambition is to support a generic approach to Personal Health Records across the One London Local Health Care Record region to reduce the costs and risks for this project and other similar digital approaches in the future.

Whilst we have highlighted the fact that requirements for scalability rely on working with novel solutions at a regional and national level these risks are somewhat mitigated by a clear urgency to support these across the NHS at local, national and regional levels. This has been demonstrated by a willingness to provide in kind support.

Key requirements for scalability are:

Patient authentication (via NHS Login).
Two-way interoperability of Care Records (via Local Health Care Records using FHIR Care Connect Profiles) – still innovative in London at least for PHRs
Data Persistence approach for Patient entered data by NHS regions – this is still at an early stage of development in most regions.
A longer term requirement for standards around Digital Care Plans is still an unmet need. This should not be a barrier initially but our approaches are discussed in more detail within this report.

The full technical evaluation is available upon request.

Evaluation Report Appendices

Appendix 1 – Clinician co-design and feedback

Appendix 2 – Patient feedback

Appendix 1 – Clinician co-design and feedback

“This app is just brilliant and life saving”

Anne Marie, Respiratory Nurse, St Bartholomew Hospital, London

“Really useful when patients come back into clinic recording peak flow rather than paper is a

real positive. The links are really good – one patient this morning had poor inhaler technique and

having additional information that they access in one place is really useful”

Jane, Respiratory Nurse, St Bartholomew Hospital, London

The following clinicians have given feedback into the design and functionality of the Digital Health Passport Clinical Portal during the pilot phase:

Dr Chin Nwokoro (Paediatric Consultant and Respiratory Lead, Royal London Hospital)

Dr Paul Pfeffer (Respiratory Consultant RLH and Difficult Asthma Lead for East London)

Dr Julia Moody (GP, Chrisp Street and CCG Lead for Children and Maternity)

Dr Jim Cole (GP, Chrisp Street, and QMUL Clinical Effectiveness Group0

Charlotte Carrick (Children’s Respiratory Nurse, RLH)

Tori Hadaway (Community Asthma Nurse, Tower Hamlets)

Rachel McCready (Practice Nurse, Chrisp St)

Jane Simpson (Respiratory Nurse, Bart’s)

Anne-Marie Casey (Respiratory Nurse, Bart’s)

Dr Jonathan Grigg (Paediatric Consultant, RLH)

Summary of key points

Twelve site visits to observe and evaluate use of the portal
Seven follow up interviews were conducted
High ‘Did not attend’ rate and overall low number of teenage patients attending asthma reviews despite proactive measures
Nurses are the main users, then GPs, not used by consultants
Feedback about the patient app was very positive
Feedback about the clinical portal was mixed
- It is functional and little training is required to use
- Issues around remembering another login
- Not interoperable with EMIS (or any system)
- Time constraints
- Staff awareness (forget to use or to book longer appointments)

Change requests to portal were prioritised and implemented if considered high enough priority during the pilot (eg re-engineering for old versions of MS Explorer)
Changes need to made to work flow processes as well as technology
Targeting CYP in schools may be better approach

Dr Julia Moody (GP)

Adoption requires system change within the practice
Appointment type needed to be identified ahead of time – training of assistant team required as well as clinician.
Chrisp Street sent out an invitation to targeted cohort of 110 patients via text message to book an asthma review using app and received 1 response
Personal invitation to come to asthma review worked better
Barriers included 2 not having smart phones, 1 no data plan
Language barrier also highlighted as barrier
Younger people seemed more enthusiastic – maybe start earlier
Consider how can be installed on parents phone / home tablet due to parents reluctance for children to have their own
Difficulty remembering portal URL
Login – so many logins how to integrate with standard NHS
Instructions on correct way to upload to EMIS (and other systems)
Doing the review and entry in 10mins was a challenge

Create peer groups in schools and secondary school – group consultations and care plan development would be effective route to adoption
Contact of 16 year olds is a challenge as telephone numbers are often still parents . Chrisp Street is reregistering 16 year olds contact details
Workforce issues – need to train staff to book the correct length of appointments and inform clinicians and patients about use of the app.
Ability to train to practice nurses being able to do reviews as well – they need to gain confidence and training.
Look at Ready, Steady, Go transition programme as transition model in development
Agreed that patient entered data in general in PHR was a positive move
Need to access portal in different settings and various clinicians – eg school and surgery

Charlotte Carrick (RLH Childrens’ Respiratory Nurse)

Sign up somewhat slow as working in general respiratory clinic
Best results from screening appointments and giving advance notice of use of app
Six sign ups since the start
One really enthusiastic response and no negative feedback
Feeling that 12/13 year olds may be an engaged age range going forward
Comments back from patients about concerns around storage space on their phone and data plans
Noted that 4 patients did not have phones
Some have tables with no mobile number
Felt that after a few uses patients can be on board and care plan produced in 10 min appointment
Too much free text input time consuming
Add drop down menus for standard medication, colours of inhalers
Peak flows as number and then calculate 80/50 reduction amounts in plan
Charlotte has done one review on care plan and made changes on the portal – all worked well
Looks and feels better and is more simple to enter the required information than old self populating printed pdf plans
Product development: provide alerts for parents/carers on peak flow reduction and when a care plan needs to be reviewed.
The app not for everyone as not suitable for people with very poor condition control / management

Tori Hadaway (Community Children’s Asthma Nurse)

Difficulty finding suitable patients as most are U12
Number of Do Not Attend high for reviews (60%)
Need 30mins to undertake review
GP’s and School Nurses should be doing majority of CYP plans in school on a yearly basis
Challenges may be not allowing phones in schools, lack of wifi, data and having a smart phone at that age
Secondary schools may be better – lower sixth may be a engaged age group
Encourage adoption needs proactive approach and associated resource and workflow (eg leaflets, patients contact, follow up etc)
Keen to explore links within adoption is schools in Tower Hamlets

Anne-Marie Casey (Bart’s Adult Asthma Nurse)

“This app is just brilliant and could save a life”
Great example of joined up care
Potential to extended to Royal London Asthma wards
Can add additional features to health tracker
Additional leaflets / instructions would be good

Jane Simpson (Bart’s Adult Asthma Nurse)

“Really useful when patients come back into the clinic, recording peak flow rather than paper is a real positive. The links are really good – one patient this morning had poor inhaler technique and having additional information that they access in one place is really useful”
Allergies – there is a double entry between emergency and care plans and medical record – can this be integrated to reduce error possibility
Graph of peak flows on app or portal for clinician and patient to review together
Time to use the system a concern especially when not used to it
Inhaler Techniques videos really useful – reduce time in clinic as can refer to them
Roll back does portal save old plans as useful to refer back in same format
Possibility to use pre-discharge as inpatient to bridge transition and record peak flows prior to first asthma clinic appointment
Issues around Do Not Attend patients reducing number using the app
Can app prompt to bring inhalers to asthma review and give a reminder to book a review
Medication tracking – based on care plan can work out when medication may be running low and alert

Dr Jim Cole (GP)

“The app is brilliant, really impressive”
Lack of interoperability with EMIS
Difficulty remembering portal URL
JC cut and pasted from EMIS to emergency plan (medications, allergies, contact details)
NOTE: Patient was moved manually in database from Barts to Chrisp Street highlighting interoperability issues

Rachel McCready (Practice Nurse, Chrisp Street)

Currently not used as lack of suitable patients
Difficulty remembering portal URL
Tech issues are embarrassing if you can’t remember so don’t try
Video and instruction sheet would be useful (note – instructions previously provided, but not all staff aware – need better system)
Accurix – text messaging sits on EMIS as widget – similar would be a reminder and easy link
Switching between patients of differing conditions – easy to forget to use a new system
Appointment system – Add a comment to remind clinician to use DHP
Text message patient to download app before appointment
Need upload to EMIS (and other systems)

Appendix 2 – Patient feedback

“Easy to log onto and good to know what level my asthma is at – knowing if I need to wear a coat or hat is really good.” Alice, DHP User

“I really liked it because it was simple to use” Jared, DHP User

“The look and feel is really good – not just a boring NHS App and the interactive background is especially good for younger users” Lucy, DHP User

“As an asthmatic in the last two years I’ve been to A&E twice and if I’m having trouble breathing I can take my phone out and say look at this plan. That would be really handy. Saira, DHP User

“It’s really cool” Robert, DHP User

Summary

Useful and usable insights gained
Easy to use and no negative comments about design or functionality
Most people use some features
Small proportion of users use intensively
Many additional feature requests and ideas to further improve the design

Survey comments

Works perfectly, health plan really beneficial
Loved the colour scheme and bright and bold interactive features
Cool to have sharing emergency plan and health tracker with doctor
Useful for the record and to share the record, ambulance to look at plan etc
Forget to use it, but feels good, looks good
Push notifications (would be good) to take inhaler and fill out diary
Some more features and dropdown boxes
Not complicated

Workshop feedback

A co-design workshop was held with 2 CYP and one parent

On-boarding easy via text message – no problems
Like the ‘magic links’ – no password is good
Like the bright colours and the timeline – it could be customised
Easy to navigate and use the app
Could have a character that ‘gets weaker’ if you don’t take medications
Need to make sure it doesn’t use too much memory
Hadn’t found the air quality alerts – notifications would be good
Different graph views with effects would be good
Medication reminders would be good
More condition care plans – eg for eczema
A tutorial on the app about it’s features would be good
Should have information about how to contact the team
Could have peer groups to share information with
Could get ‘badges’ for completing tasks or watching videos
It would be good to be able to book/cancel appointments and get reminders
Could have themes from video games that get unlocked with points
Health tracker has about the right number of questions (not everyone thinks this) but not clear how often you should use it
Health tracker could record healthy activities
Could see more of the week on the timeline or better way to display older information (graphs and charts)

“It’s really cool” – Robert, age 12