Skip to main content

RiskRadar: development and pilot results of a technical intervention targeting combination prevention regarding HIV, viral hepatitis, sexually transmitted infections and tuberculosis

Abstract

Background

The HIV pandemic impacts the lives of millions and despite the global coordinated response, innovative actions are still needed to end it. A major challenge is the added burden of coinfections such as viral hepatitis, tuberculosis and various sexually transmitted infections in terms of prevention, treatment and increased morbidity in individuals with HIV infection. A need for combination prevention strategies, tailored to high-risk key populations arises and technology-based interventions can be a valuable asset. The COVID-19 pandemic challenged the delivery of existing services and added stress to existing public health and clinical structures but also highlighted the potential of exploiting technical solutions for interventions regarding infectious diseases. In this paper we report the design process, results and evaluation findings from the pilots of ‘RiskRadar’—a web and mobile application aiming to support combination prevention, testing and linkage to care for HIV, viral hepatitis, various sexually transmitted infections and tuberculosis.

Methods

RiskRadar was developed for the INTEGRATE Joint Action’s aim to improve, adapt and pilot innovative digital tools for combination prevention. RiskRadar was designed iteratively using informed end-user-oriented approaches. Emphasis was placed on the Risk Calculator that enables users to assess their risk of exposure to one or more of the four disease areas, make informed decisions to seek testing or care and adjust their behaviours ultimately aiming to harm/risk reduction. RiskRadar has been piloted in three countries, namely Croatia, Italy and Lithuania.

Results

RiskRadar has been used 1347 times across all platforms so far. More than 90% of users have found RiskRadar useful and would use it again, especially the Risk Calculator component. Almost 49.25% are men and 29.85% are in the age group of 25–34. The application has scored 5.2/7 in the User Experience Questionnaire, where it is mainly described as “supportive” and “easy-to-use”. The qualitative evaluation of RiskRadar also yielded positive feedback.

Conclusions

Pilot results demonstrate above average satisfaction with RiskRadar and high user-reported usability scores, supporting the idea that technical interventions could significantly support combination prevention actions on Sexually Transmitted Infections.

Background

HIV, viral hepatitis, Tuberculosis (TB) and other Sexually Transmitted Infections (STIs) continue to pose major public health challenges globally and gaps in the coverage of the continuum of care services are widely acknowledged [1, 2]. The term “combination prevention” refers to approaches that promote a combination of biomedical, behavioral, and structural interventions adapted to the needs of specific communities [3].

However, one-size-fits-all approaches have failed to support the broad and complex spectrum of HIV, viral hepatitis and STIs prevention needs [4]; Effective prevention strategies for the different key populations most at risk [5] (i.e. Men who have sex with Men (MSM), People Who Inject Drugs (PWID), transgender people, sex workers, people living in prisons or other closed settings and migrants) indicate that interventions designed for specific populations and social contexts have enhanced the adoption of prevention behaviours and the uptake of care services [69]. To this end, a number of studies exploring the effectiveness and acceptability of such efforts show encouraging results regarding the uptake and actionable takeaways from Information and Communication Technologies (ICT)-assisted interventions [1014]. However, it should be emphasized that, regardless of how innovative a technical intervention might seem, it must also take social issues into account [15].

The INTEGRATE Joint Action (JA) [16] has implemented integrated activities to improve awareness, prevention, early diagnosis and linkage to care for HIV, viral hepatitis, STIs and TB in Europe. It aimed to bring together ICT experts with infectious disease scientists, clinicians, civil society and members of academia to design, pilot and assess the effectiveness of a new ‘INTEGRATE ICT Tool’, i.e. the RiskRadar (RR). This web and mobile application offers basic information on the four disease areas and tools for finding testing sites across Europe, notifying sexual or needle sharing partners as well as for self-assessing protective behaviours to prevent HIV infection, hepatitis, other STIs and TB. In this paper we describe the design process and report on the results of the RiskRadar pilot test conducted in Croatia, Italy and Lithuania during the second half of 2020.

Methods

The RiskRadar application design process (see Fig. 1) began with a mapping exercise of existing ICT tools for prevention, testing and linkage to care for four disease areas, i.e. HIV, viral hepatitis, TB and STIs. 115 ICT tools were initially proposed for evaluation; the list was then reduced to 53 tools and ordered by topic (Prevention, Harm Reduction, PrEP, Testing, Partner Notification (PN), Linkage to Care). INTEGRATE’s Steering Committee (SC) and Advisory Board (AB) members rated the mapping matrix while a technical overview was also conducted to assess adaptability, reproducibility, data relevance of the selection. Since effectiveness studies on the tools under review were sparse [17], the evaluation was based on the expertise of the various stakeholders and their extensive experience utilising such tools in their workflow. Several tools were considered for adaptation (see Table 1) due to features providing added value to the toolkit under development. “YOUR ENDING HIV TOOLKIT” offered a comprehensive suite of tools, “Chemsex Care Plan” contained insights on harm reduction and non-judgmental language, “PrEP in Europe” provided the most up-to-date information about PrEP while “PrEP Locator” pointed to PrEP providers (albeit in the USA). “What’s your number” was proposed as a base for the Risk Calculator and “Let Them Know” and “Don’t Spread It” introduced anonymous Partner Notification via SMS and/or email.

Fig. 1
figure1

RiskRadar design and development methodology

Table 1 Shortlisted tools for HIV, viral hepatitis, STIs or TB as rated by the mapping exercise

These shortlisted tools drove the initial proposal of the RiskRadar structure; the adaptation and integration process aimed to accommodate all four disease areas and the pilot-specific target audiences, while considering the pre-existing tools’ strengths, weaknesses and perceived effectiveness to address the different information and prevention aspects. The overall technical design of RiskRadar was based on informed User eXperience (UX), end-user oriented design and communication approaches [1822]. The following priorities were defined: (a) understandable and adaptable content to fit diverse target groups; (b) acceptable, non-judgemental, non-threatening and non-fear-inducing language and imagery; (c) privacy and confidentiality along GDPR, and (d) emphasis on user empowerment by exploiting insights of user perspectives as contributed by the pilot partners involved, in an iterative design and development process; All involved stakeholders (mainly NGOs and patient organisations) examined RiskRadar’s design and offered valuable points towards the inclusion of specific group needs, such as promoting PrEP and Hepatitis B vaccination to MSM, providing Hepatitis A&C vaccination advice to migrants, including a lightweight web application option to be accessed at various low-threshold points for PWID without smartphones etc. This “feedback loop” was repeated for each version of RiskRadar until a consensus was reached in a final stakeholders meeting.

The RiskRadar components can be summarized as following: (a) Factsheets (FS) offering basic information about the four disease areas (a TB infographic is presented individually to highlight the differences to the other three), (b) Risk Calculator (RC) that helps users assess their risk of exposure to the different diseases according to their behaviour and to make informed decisions in order to adjust their protective behaviours, (c) PrEP and Undetectable = Untransmittable (U=U) information, (d) Test Finder (TF) with filtering capabilities per disease and country, (e) Partner Notification (PN), an anonymous, non-traceable SMS service to notify one’s partners that can be accessed via a unique random code (in QR form for smartphones and in serial form for web users) that only diagnosed patients receive from an authorized healthcare professional, (f) Reminders (only featured in the mobile app) for the users with an estimated risky behaviour to get tested every 6 months that can be disabled at any time, and (g) Evaluation Questionnaire (EQ) [23] to collect information regarding end-user satisfaction and perceived impact. For the most challenging components, the RC and PN, a Data Protection Impact Assessment was conducted to identify and mitigate any risk in terms of information security.

The functionality of the Risk Calculator component aimed to facilitate the individual risk assessment based on the decision trees (a) described in WHO and ECDC guidelines [2426] and (b) derived from the analysis of existing single-disease risk assessment tools for HIV, hepatitis or STIs considered during the mapping exercise. To this end, various risk factors were identified through 16 questions that follow conditional logic (see Table 2) while upon the Risk Calculator’s completion, a comprehensive answer is presented, depicting the estimated risk overall and per risk factor. Additionally, if the user reports condom use, they can opt to answer 3 further questions to get advice on how to select the correct condom type. The final decision tree (see Fig. 2) is based on input from a wide range of infectious disease scientists, clinicians and civil society members participating in the project, through an iterative design process aiming to balance the sufficient granularity of answers while ensuring user retention. Vital risk factors resulting from the users’ answers were selected to be stored anonymously in RiskRadar’s secure database for further analysis. Other factors (e.g. PrEP use) drove the conditional logic of the Risk Calculator component as well, but were not stored.

Table 2 Risk assessment factors for the Risk Calculator Module
Fig. 2
figure2

Decision tree for the Risk Calculator

RiskRadar is available via a web application [27], an Android mobile app [28] and an iOS based app [29] for Apple devices ecosystem (iPhones, iPads etc.). It was translated in the native languages and piloted during the second half of 2020 in Croatia, Italy and Lithuania, targeting different key populations to evaluate the application’s impact in diverse settings. RiskRadar’s various components (see Fig. 3 part a) were evaluated concerning their overall usability and acceptability per country; The respective metrics include usage statistics from Google Analytics, Google Play and the Apple Store for the web, Android and iOS apps respectively, risk factor metrics from the Risk Calculator component and the answers to the evaluation questionnaires submitted via the RiskRadar toolkit. Finally, a qualitative evaluation was conducted in Italy and Lithuania, in the form of semi-structured online interviews and face-to-face focus groups. The participants were introduced to a validation scenario describing the basic actions to be performed while using the RiskRadar, aiming to solicit reactions and thoughts during the hands-on inspection of the toolkit. The full User Experience Questionnaire [23] was completed at the end of those sessions to measure the users’ satisfaction.

Fig. 3
figure3

a RiskRadar's components (web application view) and b Risk Calculator answer (mobile app view)

Regarding promotional activities, RiskRadar banners and articles have been featured on pilot and affiliated partner’s websites and social media accounts (including targeted social media paid advertisements) and on Italy’s and Lithuania’s Ministry of Health websites. RiskRadar has been presented in multiple occasions at national and regional meetings to public health specialists and other involved stakeholders. Targeted promotion was handled by each organisation as needed, using available communication channels and through their day-to-day interactions with potential end-users. To this end, paid campaigns were launched on Grindr, a prominent social networking app for gay, bi, trans and queer people in Italy, and on the gay.it website. Moreover, the toolkit was presented to people accessing low-threshold service locations where it could gain visibility among vulnerable risk groups.

Results

RiskRadar was officially launched on July 6th, 2020 -after some months’ delay due to the COVID19 pandemic; the piloting phase lasted until December 31st, 2020. In total, it was accessed 817 times via web, while 270 and 260 users have downloaded the app on Android and iOS respectively. The Risk Calculator component (see Fig. 3 part b), one of the main attractions of RiskRadar as a whole, has been used 1106 times.

In terms of adoption during the pilot period, most users were attracted in Italy (Table 3). The web application shows an average session duration of 14 min and 51 s while only 9.70% of users navigate away from the landing page without exploring the application further; both indicators denote a high retention and interest of visitors in RiskRadar. Regarding localisation in general, it should be noted that country information for each web application visit or mobile download is deducted by the user-selected language. In terms of privacy, no location data are retrieved by the user’s browser or mobile device, in accordance to GDPR. Consequently, the English version metrics contain data that are impossible to attribute correctly to any pilot country. Furthermore, the Lithuanian language isn’t readily available for the native iOS app -only as an option after the initial English installation- so all Lithuanian Apple ecosystem devices are included in the metrics for the English version.

Table 3 RiskRadar usage per pilot country

Tables 4 and 5 present the aggregated results (overall and per language) collected through the Risk Calculator component to gain insights on population groups using this particular module and by extension, the target audiences of the RiskRadar pilots. As expected, MSM constitute a sizable portion of the user base, reaching up to 43.50% in Italy where the pilot focused on MSM and migrants. 6.33% reported being a migrant and a joint percentage of 3.53% answered that they were using injectable drugs. The condom test was only taken 47 times, with 38.29% reporting that condoms fit too tightly, 21.28% noticing irritation due to latex and 19.15% complaining about diminished/inadequate feeling with condom use. It should be noted that users may choose to repeat the Risk Calculator test to explore risky behaviours, however due to privacy issues, the application did not keep information to identify unique users. Although this process obviously lacks accuracy, there is nevertheless an opportunity to extract some comparative insights per country. Also, although users may change their answers on risky behaviours, some basic user categorization is expected to be consistent such as sexual preference, immigration status etc. More accurate information regarding the targeted populations and intended actions would be gathered by the piloting organisations who provide linkage to testing and care but they were cancelled due to the pandemic.

Table 4 Risk factors recorded by users' answers in the RC (HIV status, sex-related risks)
Table 5 Risk factors recorded by users' answers in the RC (Drug injection risk, vaccination and immigrant status)

To evaluate RiskRadar in a systematic manner, a special component was developed containing (a) questions about its perceived usefulness and (b) a short user experience evaluation questionnaire (UEQ) [23]. This component, designed to be as unobtrusive as possible, was completed 67 times which corresponds to 4.97% of total visits, an acceptable response for this kind of in-app surveys [30]. Overall, 92.53% of responses found RiskRadar useful; 85.07% were positive to using RiskRadar again in the future. The various STIs Factsheets were ranked first regarding end-users’ interest (67.16%), followed by the Risk Calculator (64.18%), the Test Finder (38.80%) and U=U and PrEP information (41.79% and 40.29% respectively). Interestingly, 76.11% of responses indicated further action upon consulting the toolkit, mainly by seeking further advice (35.83%) and getting tested (32.83%) as seen in Fig. 4. Concerning the demographics, 49.25%, 20.89% and 2.99% of end-users identified as men, women and trans women respectively. 29.85% were in the 25–34 age group followed by 16.41% of users aged 35–44. RiskRadar achieved an average user experience score of 5.2 (with 7 being the highest possible score) and the highest ranked characteristic of the RiskRadar experience was support and ease-of-use (5.5/7) while the lowest was the excitement and inventiveness aspect (see Figs. 5 and  6).

Fig. 4
figure4

Action taken after RiskRadar use

Fig. 5
figure5

Average scores for the main 8 aspects of User Experience Questionnaire reported in-app

Fig. 6
figure6

Likert rating scale results regarding the RiskRadar User Experience reported in-app

Regarding the qualitative evaluation, in Italy 10 participants were interviewed online by Italian pilot partners; 80.00% were men (Italian partners address mainly MSM) and 40.00% of them belonged in the 45–54 age range followed by the 35–44 and over 75 age range (20.00% each). Self-reported computer skills were average and above. Figure 7 presents the user experience scores where RiskRadar is rated as understandable, enjoyable, good, easy, efficient, clear, organized and friendly by all respondents, while it scores lower regarding creativity and inventiveness. All respondents appreciated the integration aspect of the tool.

Fig. 7
figure7

Likert rating scale results from the User Experience reported in the Italian evaluation

In Lithuania, face to face focus groups were organised instead, involving 12 patients with diagnosed addictive disorders, however only 7 of them completed the UEQ after the meetings. All respondents were males; 71.40% belonged in the 35–44 age range and 28.60% in the 45–54 range. 6 participants self-assessed as having “bad” or “moderate” computer skills. Overall, the tool received positive feedback (see Fig. 8) however uptake of the tool was challenging in participants with insufficient computer literacy, in which case assistance and support was deemed necessary.

Fig. 8
figure8

Likert rating scale results from the User Experience reported in the Lithuanian evaluation

Finally, it should be noted that majority of the feedback received in the form of free-text comments and interviews focused on expanding the toolkit towards harm reduction advice, self-testing and PrEP services that are becoming more popular across the EU.

Discussion

The development of RiskRadar exploited input from experts and stakeholders across the INTEGRATE JA to achieve the additive and synergistic effects of various combination prevention approaches, targeting certain vulnerable population groups. RiskRadar aims to accommodate various target audiences, countries and regions, disease areas accompanied by a broad scope of prevention, testing, empowerment, treatment and capacity building strategies. The developed application offers significantly increased levels of security, privacy and confidentiality that enhances its acceptability and impact on a wide audience in the EU.

RiskRadar’s development encountered numerous challenges, especially concerning the integration of TB-oriented artefacts due to the differences in the routes of transmission, prevention and care compared with the other diseases. Moreover, the coordination of information and translations across the 3 pilot countries, bearing in mind the particularities of each country’s settings, capacity and target populations, was far from trivial, especially during the COVID-19 pandemic. The Risk Calculator component proved to be the most challenging, since the integrated risk assessment algorithm for all 4 disease areas had to be created by prioritising risk based on each specific target group’s characteristics and the messages communicated by the RC answers required broad stakeholder input.

Regarding the limitations of this study, an inherent user selection bias should be noted. The INTEGRATE consortium designed the pilots aspiring to reach as wide an audience as possible and utilise any channels at their disposal, starting with already established networks (patients/clients already visiting NGO and low threshold sites, actively involved members of the civil society etc.). In the future, a controlled study engaging a representative sample of the targeted populations could provide more robust results. Furthermore, low computer literacy, prevalent in some vulnerable groups, has been identified as key factor which might hinder the uptake of similar interventions. Moreover, it should be highlighted that as COVID-19 pandemic affects most key populations disproportionately by hindering or at least delaying access to testing and treatment, especially for vulnerable populations [31, 32] and adding strain to the healthcare systems of low and middle income countries, where TB is endemic and patients may also be co-infected with HIV and other STIs [33]. To this end, COVID-19 significantly hindered the pilot roll out of RiskRadar, limiting the pilot application range and the potential impact of the intervention as a whole, as for example the ability to direct users to local organizations for continuity of care was restricted to traditional phone/email communications showcased in RiskRadar’s dedicated “Contact” section. On the other hand, it should be noted that digital tools have the potential to address some of the imposed challenges and to mitigate some of the added burden [31, 3436];

As a whole, we consider RiskRadar’s uptake to be positive, taking into account the difficulties introduced by the COVID-19 pandemic, especially regarding the dissemination of the tool among the target groups. The majority of users in the pilots originate from Italy, partly due to the timely and coordinated efforts of the partners and the experience and traction gained from previous similar campaigns, but also because of the disparities in population size. All pilot countries managed to intensify their efforts during the last months while targeting key population groups and supporting their use of RiskRadar, which was clearly shown in the daily breakdown of usage metrics.

The overall evaluation of the tool was satisfactory with RiskRadar’s understandability and user friendliness scores being rated as highly positive. Furthermore, INTEGRATE’s Scientific Committee and Advisory Board members have suggested the additional expansion of the RiskRadar components to other combination prevention issues such as HIV and STIs self-tests, chemsex, harm reduction advice and services etc.

Outlining future work paths, RiskRadar’s further development should focus on (a) following an adaptive model to address the variety of needs per population group, (b) systematic evaluation and (c) long-term sustainability plans to maximise its impact.

Conclusion

RiskRadar targets population groups at risk for HIV, hepatitis, STIs and TB with clear and simple combination prevention messages through several modular core components. Based on the overall experience of the RiskRadar development, we argue that RiskRadar and similar interventions have the potential to provide support towards combination prevention efforts, provided that considerable resources to ensure its accuracy, reliability, relevance and usefulness are invested in the future. As the COVID-19 pandemic has also clearly highlighted, the need for timely monitoring, prevention, testing and risk assessment constitute a necessity that ICT technologies can significantly address.

Availability of data and materials

Not applicable.

Abbreviations

AB:

Advisory Board

COVID-19:

Coronavirus disease

ECDC:

European Centre for Disease Prevention and Control

EU:

European Union

GDPR:

General Data Protection Regulation

HIV:

Human Immunodeficiency Viruses

ICT:

Information and Communications Technology

JA:

Joint Action

MSM:

Men who have Sex with Men

NGO:

Non-Governmental Organisation

OS:

Operating System

PN:

Partner Notification

PrEP:

Pre-Exposure Prophylaxis

PWID:

Person Who Injects Drugs

RC:

Risk Calculator

RR:

Risk Radar

SC:

Steering Committee

STIs:

Sexually Transmitted Infections

TB:

Tuberculosis

U=U:

Undetectable = Untransmittable

UEQ:

User Experience Questionnaire

UI:

User Interface

UX:

User Experience

WHO:

World Health Organization

References

  1. 1.

    Unaids. 90–90–90: An ambitious treatment target to help end the AIDS epidemic. 2020.

  2. 2.

    Fast-Track strategy to end the AIDS epidemic by 2030|UNAIDS. 2018 (cited 2018 Feb 27). http://www.unaids.org/en/resources/campaigns/World-AIDS-Day-Report-2014.

  3. 3.

    Hayes R, Brown A, Azad Y, Delpech V. HIV Combination prevention Monitoring implementation of the Dublin Declaration on partnership to fight HIV/AIDS in Europe and Central Asia: 2018 progress report HIV: Combination prevention (cited 2020 Sep 17); www.ecdc.europa.eu.

  4. 4.

    Singh A, Gibbs J, Estcourt C, Sonnenberg P, Blandford A. Are HIV smartphone apps and online interventions fit for purpose? ACM Int Conf Proceeding Ser. New York, New York, USA: Association for Computing Machinery; 2017 (cited 2020 Aug 31). p. 6–15. http://dl.acm.org/citation.cfm?doid=3079452.3079469.

  5. 5.

    Macdonald V, Verster A, Baggaley R. A call for differentiated approaches to delivering HIV services to key populations. J Int AIDS Soc. 2017 (cited 2020 Sep 17). p. 28–31.

  6. 6.

    Kurth AE, Celum C, Baeten JM, Vermund SH, Wasserheit JN. Combination HIV prevention: significance, challenges, and opportunities. Curr HIV/AIDS Rep. 2011;8:62–72.

    Article  Google Scholar 

  7. 7.

    Bekker L-G, Beyrer C, Quinn TC. Behavioral and biomedical combination strategies for HIV prevention. Cold Spring Harb Perspect Med. 2012;2:8.

    Article  Google Scholar 

  8. 8.

    Young I, Flowers P, McDaid LM. Barriers to uptake and use of pre-exposure prophylaxis (PrEP) among communities most affected by HIV in the UK: findings from a qualitative study in Scotland. BMJ Open. 2014;4:e005717.

    Article  Google Scholar 

  9. 9.

    avac.org. Multipurpose prevention technologies—an introductory factsheet. https://www.avac.org/sites/default/files/resource-files/mpt_product_dev.pdf

  10. 10.

    Gannon B, Davis R, Kuhns LM, Rodriguez RG, Garofalo R, Schnall R. A mobile sexual health app on empowerment, education, and prevention for young adult men (MyPEEPS Mobile): Acceptability and usability evaluation. JMIR Form Res. 2020;4.

  11. 11.

    Sullivan PS, Driggers R, Stekler JD, Siegler A, Goldenberg T, McDougal SJ, et al. Usability and acceptability of a mobile comprehensive HIV prevention app for men who have sex with men: a pilot study. JMIR mHealth uHealth. 2017;5:e26.

    Article  Google Scholar 

  12. 12.

    van Rooijen MS, Gotz H, Vriens P, Heijman T, Koekenbier R, van Veen M, et al. Sender and receiver acceptability and usability of an online partner notification tool for sexually transmitted infection in the Netherlands. Sex Transm Dis. 2018;45:354–7.

    Article  Google Scholar 

  13. 13.

    Lessard D, Aslan A, Zeggagh J, Morel S, Michels D, Lebouché B. Acceptability of a digital patient notification and linkage-to-care tool for French PrEPers (WeFLASH©): key stakeholders’ perspectives. Int J STD AIDS. 2019;30:1397–407. https://doi.org/10.1177/0956462419878043.

    Article  PubMed  Google Scholar 

  14. 14.

    Bilardi JE, Fairley CK, Hopkins CA, Hocking JS, Kit Sze J, Chen MY. Let them know: evaluation of an online partner notification service for chlamydia that offers E-mail and SMS messaging. Sex Transm Dis. 2010;37:563–5.

    Article  Google Scholar 

  15. 15.

    Gibbs J, Gkatzidou V, Tickle L, Manning SR, Tilakkumar T, Hone K, et al. “Can you recommend any good STI apps?” A review of content, accuracy and comprehensiveness of current mobile medical applications for STIs and related genital infections. Sex Transm Infect. 2017;93:234–5.

    PubMed  Google Scholar 

  16. 16.

    INTEGRATE|Joint Action on integrating prevention, testing and linkage to care strategies across HIV, viral hepatitis, TB and STIs in Europ (cited 2020 Sep 17). https://integrateja.eu/.

  17. 17.

    Guy RJ, Micallef JM, Mooney-Somers J, Jamil MS, Harvey C, Bateson D, et al. Evaluation of chlamydia partner notification practices and use of the “let them know” website by family planning clinicians in Australia: cross-sectional study. J Med Internet Res. 2016;18:173.

    Article  Google Scholar 

  18. 18.

    Pilemalm S, Timpka T. Third generation participatory design in health informatics-Making user participation applicable to large-scale information system projects. J Biomed Inform. 2008;41:327–39.

    Article  Google Scholar 

  19. 19.

    Gkatzidou V, Hone K, Sutcliffe L, Gibbs J, Sadiq ST, Szczepura A, et al. User interface design for mobile-based sexual health interventions for young people: design recommendations from a qualitative study on an online Chlamydia clinical care pathway. BMC Med Inform Decis Mak. 2015;15:72. https://doi.org/10.1186/s12911-015-0197-8.

    Article  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Aicken CRH, Fuller SS, Sutcliffe LJ, Estcourt CS, Gkatzidou V, Oakeshott P, et al. Young people’s perceptions of smartphone-enabled self-testing and online care for sexually transmitted infections: qualitative interview study. BMC Public Health. 2016;16:974. https://doi.org/10.1186/s12889-016-3648-y.

    Article  PubMed  PubMed Central  Google Scholar 

  21. 21.

    World Health Organization. Global health sector strategies on HIV, viral hepatitis and sexually transmitted infections, for the period 2016–2021. 2016. https://apps.who.int/iris/bitstream/handle/10665/246178/WHO-HIV-2016.05-eng.pdf;jsessionid=46F4C7556AA64E61C596C597E58E8F2F?sequence=1.

  22. 22.

    European Centre for Disease Prevention and Control. Communication strategies for the prevention of HIV, STI and hepatitis among MSM in Europe. Stockholm; 2016. Available from: https://www.ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/Communication-strategy-prevention-HIV-STI-hepatitis-MSM.pdf.

  23. 23.

    User Experience Questionnaire (UEQ) (cited 2020 Sep 10). https://www.ueq-online.org/.

  24. 24.

    World Health Organization. Consolidated Guidelines on Hiv Testing Services. 2019; https://apps.who.int/iris/bitstream/handle/10665/179870/9789241508926_eng.pdf?sequence=1LB-0ojl.

  25. 25.

    European Centre for Disease Prevention and Control. Public health guidance on HIV, hepatitis B and C testing in the EU/EEA—an integrated approach. Stockholm; 2018 (cited 2019 Oct 10); https://www.ecdc.europa.eu/sites/default/files/documents/hiv-hep-testing-guidance_0.pdf.

  26. 26.

    European Centre for Disease Prevention and Control. HIV and STI prevention among men who have sex with men. (cited 2020 Sep 14); www.ecdc.europa.eu.

  27. 27.

    Risk Radar (cited 2020 Sep 10). https://integrateja.eu/riskradar/.

  28. 28.

    RiskRadar—Apps on Google Play (cited 2020 Sep 10). https://play.google.com/store/apps/details?id=com.tdermaris.integratetoolkit.

  29. 29.

    RiskRadar on the App Store [Internet]. [cited 2020 Sep 10]. https://apps.apple.com/app/id1469764662

  30. 30.

    Mobile Survey Response Rates: Averages by App Category—apptentive (cited 2021 Apr 7). https://www.apptentive.com/blog/2016/10/04/mobile-survey-response-rates/.

  31. 31.

    Ridgway JP, Schmitt J, Friedman E, Taylor M, Devlin S, McNulty M, et al. HIV Care Continuum and COVID-19 Outcomes Among People Living with HIV During the COVID-19 Pandemic, Chicago, IL. AIDS Behav. Springer; 2020 (cited 2020 Aug 31). p. 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7203502/.

  32. 32.

    Survey shows that the COVID-19 pandemic increases vulnerability of LGBTI people|UNAIDS (cited 2020 Sep 14). https://www.unaids.org/en/resources/presscentre/featurestories/2020/july/20200710_survey-covid19-lgbti-people.

  33. 33.

    Togun T, Kampmann B, Stoker NG, Lipman M. Anticipating the impact of the COVID-19 pandemic on TB patients and TB control programmes. Ann Clin Microbiol Antimicrob. 2020; 21.

  34. 34.

    Hightow-Weidman L, Muessig K, Claude K, Roberts J, Zlotorzynska M, Travis Sanchez. Maximizing digital interventions for youth in the midst of Covid-19: lessons from the adolescent trials network for HIV interventions.

  35. 35.

    Sittig DF, Singh H. COVID-19 and the Need for a National Health Information Technology Infrastructure. JAMA. 2020 (cited 2020 Sep 7). p. 2373.

  36. 36.

    Hoagland B, Torres TS, Bezerra DRB, Geraldo K, Pimenta C, Veloso VG, et al. Telemedicine as a tool for PrEP delivery during the COVID-19 pandemic in a large HIV prevention service in Rio de Janeiro-Brazil. Brazilian J Infect Dis. 2020;24:360.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the contribution of the Integrate Joint Action Steering Committee for the implementation support, strategic guidance and quality assurance throughout the process and the Advisory Board members for reviewing and rating the selected ICT tools. The authors would also like to thank Cary James for his input to the decision tree development. Similarly, ECDC is acknowledged for providing the initial information used for the creation of the Factsheets and for the TB infographic, NAM for providing the data for the Test Finder and Teymur Noori from ECDC for facilitating communication with ECDC and NAM. Finally, the authors would also like to highlight the major contribution of the recently deceased Dr Vassilis Koutkias, who led this work on behalf of the INAB/CERTH team.

About this supplement

This article has been published as part of BMC Infectious Diseases Volume 21, Supplement 2 2021: Results from INTEGRATE - the EU Joint Action on integrating prevention, testing and linkage to care strategies across HIV, viral hepatitis, TB and STIs in Europe. The full contents of the supplement are available at https://bmcinfectdis.biomedcentral.com/articles/supplements/volume-21-supplement-2.

Funding

The INTEGRATE Joint Action was co-funded by the 3rd Health Programme of the European Union under Grant Agreement No. 761319.

Author information

Affiliations

Authors

Contributions

CK and LC performed the mapping exercise and designed the RiskRadar application, assisted by all pilot partners (DS, MM, NJ, TNB, ZD, DP, IJ, ASZ, LS and AS). CK, VKD and TD performed the technical development of the backend, web application and mobile apps for the RiskRadar toolkit. ITI/CERTH (EP, VD, RK, AP, TT, KV and DT) contributed to the deployment of the RiskRadar web application focusing on the implementation of the authorisation module of the Partner Notification component along with the SMS sending service. CM coordinated the initial rollout of the RiskRadar mobile apps for the pilot launch. CK analysed and interpreted user data. CK drafted the manuscript with the help of PN. The work was supervised by INAB/CERTH’s eHealthLab (PN). All authors read and approved the final manuscript.

Corresponding author

Correspondence to Christine Kakalou.

Ethics declarations

Ethics approval and consent to participate

Not Applicable. We have conducted a Data Protection Impact Assessment for RiskRadar along the GDPR guidelines and thoroughly evaluated and mitigated potential risks, emphasizing on data anonymization and encryption. Detailed cookies and privacy policies are featured prominently on all RiskRadar applications and users are prompted to read them carefully and consent to continue using the RiskRadar application. Only aggregated data are reported in this manuscript.

Consent for publication

Not applicable.

Competing interests

All authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kakalou, C., Polychronidou, E., Drosou, V. et al. RiskRadar: development and pilot results of a technical intervention targeting combination prevention regarding HIV, viral hepatitis, sexually transmitted infections and tuberculosis. BMC Infect Dis 21, 866 (2021). https://doi.org/10.1186/s12879-021-06501-0

Download citation

Keywords

  • Human Immunodeficiency Viruses (HIV)
  • Integrated approach
  • Combination prevention
  • eHealth
  • Risk assessment