A randomized trial to assess retention rates using mobile phone reminders versus physical contact tracing in a potential HIV vaccine efficacy population of fishing communities around Lake Victoria, Uganda

Abstract

Background

High retention (follow-up) rates improve the validity and statistical power of outcomes in longitudinal studies and the effectiveness of programs with prolonged administration of interventions. We assessed participant retention in a potential HIV vaccine trials population of fishing communities along Lake Victoria, Uganda.

Methods

In a community-based individual randomized trial, 662 participants aged 15–49 years were randomized to either mobile phone or physical contact tracing reminders and followed up at months 1, 2, 3, 6, 12 and 18 post-enrolment. The visit schedules aimed at mimicking a vaccine efficacy trial representing an early interval (months 1–6) where most vaccinations would be administered and a later period of post-vaccination follow-up. The primary outcome was retention measured as the proportion of post-baseline follow up visits completed by a participant. Retention was estimated in early and later follow-up intervals, and overall for all the six follow-up visits. Adjusted differences in retention between the study arms were determined by multivariable logistic regression using Stata® 14. One participant was later dropped from the analysis because of age ineligibility discovered after enrolment.

Results

Of the expected total follow up visits of 3966 among 661 participants, 84.1% (3334) were attained; 82.1% (1626/1980) in the phone arm and 86% (1708/1986) in the physical tracing arm (p = 0.001). No statistically significant differences in retention were observed between the study arms in the first 6 months but thereafter, retention was significantly higher for physical contact reminders than mobile phones; 91.5% versus 82.1% (p < 0.0001) at month 12 and 82.8% versus 75.4%, (p = 0.021) at month 18. Controlling for sex, age, education, occupation, community location, length of stay and marital status, the odds of good retention (completing 5 out of 6 follow-up visits) were 1.56 (95% CI;1.08–2.26, p = 0.018) for physical contact tracing compared to mobile phone tracing. Other statistically significant predictors of good retention were residing on islands and having stayed in the fishing communities for 5 or more years.

Conclusions

Among fishing communities of Lake Victoria, Uganda, 84% of follow-up visits can be attained and participant retention is higher using physical contact reminders than mobile phones.

Trial registration number

PACTR201311000696101 (http://www.pactr.org/). retrospectively registered on 05 November, 2013.

Background

HIV still remains a significant problem especially in sub-Saharan Africa where the rate of new HIV infections is still unacceptably high despite the increased availability of proven interventions and attainment a tipping point (a condition where the number of persons initiating anti-retroviral therapy is greater than the number of new infections). An effective HIV preventive vaccine is urgently needed to complement the current HIV prevention and treatment strategies. A mathematical modeling study has showed that the UNAIDS 95–95–95 target alone could avert about 25.3 million new HIV infections and the addition of a vaccine with 50% efficacy and 70% coverage to the 95–95–95 target could prevent 31 million infections by 2035. Moreover, an HIV vaccine alone was predicted to prevent 17 million incident HIV infections beyond status quo interventions [1]. A number of vaccine candidates are currently under development and have either reached human trials stage or will do so soon [2, 3].

Trials evaluating candidate vaccines require high levels of adherence to scheduled study visits and investigational products. Levels of participant retention (follow-up rates) affect the validity of study outcomes, the statistical power to detect meaningful differences, and the efficacy and effectiveness of intervention [4, 5]. Attainment of high retention rates requires minimizing loss to follow-up (LTFU) rates and this is a challenge to research studies and implementation programs. Barriers to retention include transport, inflexible work schedules and study visits, forgetting research appointments, participant/patient fatigue, demanding domestic responsibilities, mistrust of researchers and uncertainty of study outcomes [6,7,8]. Observational studies among potential populations for HIV vaccine efficacy trials have reported retention rates of 69–85.2% at 12 months and 76% at 18 months of follow-up [9,10,11,12,13,14]. These rates are below the desired minimum of 90%. A randomized trial in Kenya that assessed the effect of mobile phone use (text messages/audio calls) and in-person (physical contact) reminders on visit attendance did not have a separate physical contact arm making it impossible to determine the effect of the latter [15]. Other limitations of the aforementioned studies include the absence of follow-up schedules that mimic vaccine efficacy trials and the hypothetical assessment in the absence of an actual HIV vaccine efficacy trial. To address some of the above limitations, we conducted a randomized trial of mobile phone reminders versus physical contact reminders, to assess retention rates using a follow-up schedule that mimicked an HIV vaccine efficacy trial.

Methods

Study design and population

Between April 2012 and April 2014, we conducted a community-based individual randomized unblinded two arm trial in four fishing communities (2 islands and 2 mainland lakeshore villages) along Lake Victoria, Uganda, and assessed retention rates using mobile phone (short-text message and or audio) reminders versus physical contact tracing. The objective of the trial was to determine whether retention rates differed by method of reminding - mobile telephones (calls and text messages) versus physical contact tracing. The primary outcome was the proportion of post-baseline follow up visits completed by each participant. A follow up visit was counted as completed if a participant seen and interviewed by the study team on a pre-scheduled date or within the visit window period. Six hundred and sixty two participants aged 15–49 years were enrolled and randomized to either mobile phone or physical tracing. Participants in both arms had a total of six post-enrolment follow up visits at months 1, 2, 3, 6, 12 and 18. Each scheduled visit had a window of ± one week from the scheduled date. The visit schedule was designed to mimic an actual HIV vaccine efficacy trial follow up schedule, representing two periods typically encountered in actual HIV vaccine trials; the first six months where vaccinations are administered (earlier interval) and the post 6 months period for assessment of endpoints/outcomes (later interval). Participants in both arms were reminded twice about an upcoming visit; 7 and 3 days to the scheduled date. In the physical tracing arm reminders were done community health mobilizers (CHMs) who were members of local village health teams via face-to-face discussions. Approximately two weeks prior to each visit, CHMs were given lists of participants in their villages with dates of scheduled visits and their associated window intervals. CHMs were instructed to thank participants for their previous visits participation and to remind them of the upcoming next visit including the day and date. Phone call reminders were done by a member of the research team who was stationed at the central coordination office and had no direct physical contact with participants. Each participant in the phone arm was called and informed about an impending follow up visit. If a participant’s audio call did not go through, further attempts were made twice a day (one in the morning and the other in the afternoon) till the expiration of the visit window interval. Short text messages were sent to all phone arm participants at the lower end of the window interval and one day prior to the scheduled date. The text message was “Hello, thank you for participating in the STAR research study. We are reminding you of your next study visit which is on (day and date). STAR study team.” All messages were in Luganda, the most commonly spoken local language in the study communities.

Sample size determination

Our sample size estimation was based on a previous study in the similar communities that found a cumulative retention of 80% at 18 months of follow up. We hypothesized a retention of 80% (or p = 0.8) among the physical contact arm and an increase of 8.75% mobile phone arm 18 months post-baseline. Using the sample size formula for binary outcomes, a two-sided alpha of 0.05, power of 80%, and adjusting for a non-response proportion of 15% and cross-over – a drop in rate (R_I) of 5% and dropout rate (R_O) of 3%, we estimated a total sample size of 662 participants (331 per arm).

Screening and eligibility criteria

Inclusion criteria: Eligible participants had to be: 1) aged 15–49 years; 2) capable of providing written informed consent/assent; 3) HIV sero-negative; 4) at risk of HIV infection defined as having had any of the following in the past 3 months: a symptom of a sexually transmitted infection (STI), more than one sexual partner, a new sexual partner, or sex with a worker; 5) willing to undergo HIV testing and receive HIV results; 6) owning a cell phone; and 7) willing to provide locator information. Exclusion criteria: Participation in another interventional or behavioral study at the time of enrolment and presence of any condition which, in the opinion of the investigator or designee, would interfere with achieving the objectives of the study.

Randomization

Participants were randomized to either phone or physical contact reminder using permuted block randomization with blocks of size of 6 and a 1:1 allocation ratio. Randomization numbers were generated using Stata™ software version 11 and individual random assignment sheets were placed and sealed in opaque envelopes in batches of multiple blocks to minimize predictability that would occur if single blocks were used. Generation of randomization numbers and their concealment was done by a Biostatistician based at the Makerere University Clinical Trials Unit who never had contact with participants. She provided concealed envelopes to the Study Coordinator in numbers corresponding to the expected enrolments for given time periods. Accountability for earlier randomization envelopes was given prior to release of subsequent ones. Each consenting participant was asked by a field data collector to select an envelope which s/he opened to see the randomization number and study arm. All randomization numbers were monitored and logged daily in the field, and electronically via a data base at a data management center.

Laboratory testing

HIV-1 serostatus was determined by rapid HIV tests performed in the community by certified laboratory technologists and EIA confirmation in the laboratory at Uganda Virus Research Institute. Blood samples were first tested with Determine® HIV assay (Alere Medical Co., Ltd., Chiba, Japan), and if negative, results were reported as negative. Determine® positive samples were then tested with HIV 1/2 Stat-Pak® assay (Chembio Diagnostic Systems, Inc. Medford, NY, USA), and if positive too, results were reported as positive. But if negative on Stat-Pak®, Uni-Gold™ HIV test (Trinity Biotech plc, Bray, Ireland) was used as a tie-breaker.

Statistical analysis

Participants socio-demographic characteristics were summarized in frequencies and compared between those screened and enrolled, and across study arms using chi-square and Fisher Exact tests for categorical variables. Retention was estimated as the proportion of visits that a participant completed, divided by the total number of expected follow up visits. The follow-up period was categorized into the early interval (comprising the first 4 post-baseline visits of month 1, 2, 3 and 6) corresponding to the intensive phase of a vaccine trial, and a later period (the last 2 visits of month 12 and 18) which represented the post-vaccination follow-up phase. Retention was also estimated at each month of follow-up, by period interval (earlier and later), and overall for all the six follow-up visits. We defined good retention as completion of at least five of the six scheduled follow-up visits. We assessed if retention differed between study arms at each follow up visit, across socio-demographic and behavioral characteristics in early and later follow-up intervals, using intention-to-treat analysis. Crude and adjusted odds ratios of retention were estimated using multivariable logistic regression modeling, testing for differences between the study arms and associations with various socio-demographic and behavioural characteristics. Covariates were included in the multivariable models based on biological plausibility and a bivariate statistical significance at an alpha (α) of < 0.15. Models were built using the stepwise logical model building approach and the final model adjusted for age, gender, education, marital status, occupation, location of community and duration of stay in the communitiy. All statistical analyses were performed with Stata® 14 (StataCorp, College Station, TX) software. This article adheres to CONSORT guidelines.

Ethics and consent

The protocol was reviewed and approved by the Uganda Virus Research Institute Research and Ethics Committee, and the Uganda National Council for Science and Technology. Written informed consent was obtained from all study participants aged at least 18 years. Participants aged less than 18 years (minors) gave assent in addition to parent or guardian consent. Persons who were found HIV sero-positive at screening or during the follow up visits were referred to ART clinics for further management. HIV treatment and care was provided by ART clinics in the study communities that were operated by the UVRI-IAVI HIV Vaccine Program and MildMay Uganda. Participants who opted for non-Program clinical services were referred to Entebbe General Hospital, a public health facility that serves the region where the study was done.

Results

Baseline characteristics of the study population

A total of 879 potential participants were screened of whom 83.6% (735) were found eligible and 662 were enrolled and randomized (Fig. 1). Among those excluded, 144 did not meet the inclusion criteria while 73 were eligible but the sample size had already been attained; only one screened participant did not have a mobile phone. During the trial, we discovered that one participant in the phone arm was wrongly enrolled by attaining his 15th birthday; he was dropped from the final analysis leaving a total of 661 participants. The major reason for study ineligibility was HIV seropositive at screening, 14.6% (128/879). There were no statistically significant differences between the enrolled and screened participants with regard to age, education, tribe, occupation, community of stay (island or mainland), and length of stay in the community (Table 1). However, statistically significant differences were observed with regard to gender, religion and marital status; more males, (p = 0.004), Muslims (p = 0.035) and marrieds (p = 0.024). Participants randomized in the two study arms did not differ in baseline socio-demographic characteristics except occupation; a higher proportion of housewives was enrolled in phone arm while more bar/lodge/restaurant workers were recruited in the physical contact tracing arm. Of the approximately 1324 audio calls made to the phone arm participants during the study, 12% (159) did not go through and reported phone loss at the end of the study was about 8.2%. This is likely an underestimate given that we captured this info from only those who called in to the study office using other numbers and reported phone loss.

Fig. 1

Trial profile

Table 1 Baseline socio-demographic and behavioural characteristics of screened and enrolled participants

Retention

For each enrolled participant there were six scheduled post-baseline follow up visits, making a total of 3966 follow-up visits among the 661 enrolled participants. Overall, 84.1% (3334) follow-up visits were attained; 82.1% (1626/1980) in the phone arm and 86% (1708/1986) in the physical tracing arm (p = 0.001). Retention was 79% at month one, 87.6% at month two, 85.2% at month three, 86.4% at month six, 86.8% at month 12, and 79.1% at month 18.

Retention did not differ between study arms during the first 6 months of follow-up (Fig. 2). Thereafter, physical contact reminding had significantly higher retention than mobile phones at month 12 (91.5% versus 82.1%, p < 0.0001) and at month 18 (82.8% versus 75.4%, p = 0.021). In early follow-up phase, 64.3% (425) of participants completed all the four scheduled visits, and retention was significantly higher among participants from island communities compared to their counterparts on the mainland (72.4% versus 57.2%, p < 0.0001) and in participants who had stayed in fishing communities for more than 10 years (72.8%, p = 0.024) relative to those with less than 10 years of residence. During this phase of follow-up, no differences in retention rates were observed between the two study arms nor by other socio-demographic characteristics.

Fig. 2

Proportion of follow-up at scheduled visits by study arm

During the later follow-up period, 75.2% (497) completed the two scheduled visits. In that interval, retention was higher in the physical contact arm than in the mobile phone arm (80.1% versus 70.3% respectively, p = 0.004) and those in polygamous marriages (81.1%) relative to monogamous marriages (76.9%) and the unmarried (68.9), p = 0.026. Retention was 84.8% among participants who had stayed in fishing communities more than 10 or more years, 83.3% in those with 5–10 years, 70.3% in those with 2–4 years, and 57.7% among ones with less than two years of stay in the communities, p < 0.0001 (Table 2).

Table 2 Retention in the early and later phases by socio-demographics characteristics

Overall, 74.9% (495/661) participants attained good retention (Table 3), and it was significantly higher in the physical tracing arm compared to the phone arm (78.5% versus 71.2%, p = 0.03) and among island based participants relative to mainland dwellers (79.2% versus 71.1%, p = 0.016). After adjusting for sex, age, education, occupation, community location, length of stay in the community, and marital status in a logistic regression analysis, the odds of good retention were 1.56 (95% CI;1.08–2.26, p = 0.018) for physical contact tracing compared to mobile phone tracing (Table 3). The adjusted odds of good retention were statistically higher among island dwellers than mainland participants, AOR = 1.62 (95% CI; 1.08–2.43, p = 0.019), and in participants who had stayed in the fishing communities for 5–10 years, AOR = 2.88 (95% CI; 1.58–5.28, p = 0.001) and for more than 10 years, AOR = 3.46 (95% CI; 1.78–6.73, p < 0.0001), relative to those with less than one year of stay. Statistically borderline higher adjusted differences in good retention were observed for females compared to males, AOR = 1.70 (95% CI; 0.99–2.91, p = 0.052) and for participants aged 20–24 years relative to those aged 15–19 years, AOR = 1.99 (95% CI; 0.98–4.04, p = 0.057).

Table 3 Proportions and odds ratios of good retention by study arm and socio-demographic characteristics

Discussion

In this individual randomized trial among residents of fishing communities on Lake Victoria, Uganda, we found that participants’ reminders via physical contact tracing yielded statistically significant higher retention rates than mobile phones. This observation occurred with regard to the proportion of scheduled visits completed overall, and after the first six months of follow-up when retention was assessed by month of study visit, adjusting for adjusting for sex, age, education, occupation, community location, length of stay in the community and marital status. Good retention (defined as completion of 5 out of 6 scheduled follow-up visits) was significant higher in the physical contact reminder arm, among island dwellers and participants who had stayed in the fishing communities for 5 or more years.

We think that physical contact tracing performed better than mobile phone because the following reasons. While physical contact reminders are time and energy consuming and require effort on the part of the tracer, at individual level this approach provides direct contact with participants which in turn allows for dialogue on visit attendance and any other study related issues. Those advantages seem to work well particularly in the latter part of follow-up when excitement about continued participation in the study may have waned due to fatigue and or study-related unwanted effects. At interpersonal level, physical tracing provides room for other persons (family, workmates, community members) to inform the participant that a mobilizer was looking for him/her in case the former was absent at the time of tracing. This is may inadvertently remind a participant about the visit since he/she could guess why the mobilizer was looking for him/her. In addition, in the small and closely knit communities in Africa, it is likely that a community member may alert a mobilizer about the presence of a person he/she was looking for earlier on and this would enable the mobilizer to make another attempt to remind a participant. These additional efforts are unlikely to occur in the phone arm given the privacy associated with phones. Furthermore, phone loss and unavailability due to lack of power at the time of reminders may have led to fewer reminders in the phone arm. Mobile phone technology (text messages or audio call) removes the need for physical contact, allows getting in touch with participants at a wide range of time (day or night) regardless of their physical location. Nonetheless, it has several disadvantages especially in developing countries. These include but are not limited to unreliable connectivity, lack of constant power to charge phones, temporary and permanent change of phone ownership, loss of phones, and inability to comprehend a message especially text messages [16, 17]. Other barriers to retention like lack of transport, inflexible work schedules and forgetting research appointments are addressed by both physical contact tracing and mobile phone reminders, so they may not explain the differences retention levels between the two approaches.

The overall scheduled visit completion rate in this study was similar to that observed in Thai RV 144 trial but higher than that observed among a female sex workers (FSWs) study in Barcelona that assessed retention in the absence of an actual vaccine trial [18]. The retention observed in the mobile phone arm of this study at 6 and 12 months in this study was higher than that reported in the phone arm among FSWs in Barcelona (85% versus 76% at month 6 and 82% versus 69% at month 12, respectively) and that found in the enhanced arm (SMS and phone call reminders or in-person reminders without a phone, 59%) in a randomized trial of repeat HIV testing an persons with acute HIV infection in Kenya [13, 15]. The differences could be partly explained by dissimilarities in study populations between this study and the one in Spain, and age group with the Kenyan trial which enrolled participants aged 18–29 years.

In this study we also found that only 64% of participants completed all the four follow-up visits in the early phase that corresponds with the period where vaccinations are administered. This was much lower than the 85% observed among women in the Thai RV 144 trial [19].

The strength of this study was the ability to test two modes enhancing participant retention through a randomized trial design, using a visit schedule that mimicked an HIV vaccine trial, in a HIV high-risk population that it potentially eligible for HIV vaccine efficacy trials. However, there were some limitations. First, there was blinding of the intervention and we did not track cross-over of intervention. Although physical contact tracers were strictly warned and repeatedly reminded against using phones to track participants, there was a possibility of that happening and if it occurred it could have diluted the difference between the study arms. Second, there were no actual investigational products administered so the assessment of retention, even though it followed a visit schedule typically seen in actual HIV vaccine trials, was hypothetical. Third, we did not conduct formal qualitative assessments on participants’ perspectives on being contacted using either of the two different methods or both in a hybrid process where those not traceable via mobile phones would have physical contact tracing and vice versa. We also did not formally assess the community health mobilizers’ experiences on physical contact tracing. However, informally many mobilizers told us that participants always thanked them about the reminders and indicated they had forgotten about the visits. Such qualitative data would have enhanced the interpretation of our findings. Lastly, retention was assessed in absence of actual HIV vaccine trial and the hypothetical findings may not necessarily translate into actual retention in real HIV vaccine trial willingness observed in real vaccine trials [20, 21].

Conclusion

Among fishing community populations of Lake Victoria, Uganda, 84% of follow-up visits can be attained and participant retention is higher using physical contact reminders than mobile phones. Regardless of method of reminding, persons residing on islands and those whose duration of stay in the fishing communities was five years or more have significantly higher follow-up rates.