A pilot study of the acceptability, feasibility, and preliminary impact of SITA (SMS as an Incentive To Adhere): A mobile technology-based intervention informed by behavioral economics to improve ART adherence among youth in Uganda

Studies report serious adherence problems among youth (individuals age 15 to 24 years of age) in Uganda. Recent growth in mobile phone ownership among Ugandan youth has highlighted the potential of using text-based interventions to improve ART adherence. We therefore developed and conducted a randomized controlled trial and examined the acceptability, feasibility, and preliminary impact of a pilot intervention providing weekly real-time antiretroviral adherence feedback, based on information from a smart pill box, to HIV-positive Ugandan youth. We randomized participants to a control group, or to receive messages with information on either their own adherence levels (Treatment 1 - T1), or their own adherence and peer adherence levels (Treatment 2 – T2). We conducted six focus groups with providers and youth ages 15-24, double coded interviews, and achieved a Cohen’s Kappa of 0.79. The quantitative and data show that the was deemed acceptable and The


Abstract
Background Studies report serious adherence problems among youth (individuals age 15 to 24 years of age) in Uganda. Recent growth in mobile phone ownership among Ugandan youth has highlighted the potential of using text-based interventions to improve ART adherence. We therefore developed and conducted a randomized controlled trial and examined the acceptability, feasibility, and preliminary impact of a pilot intervention providing weekly real-time antiretroviral adherence feedback, based on information from a smart pill box, to HIV-positive Ugandan youth.

Methods
We randomized participants to a control group, or to receive messages with information on either their own adherence levels (Treatment 1 -T1), or their own adherence and peer adherence levels (Treatment 2 -T2). We conducted six focus groups with providers and youth ages 15-24, double coded interviews, and achieved a Cohen's Kappa of 0.79.

Result
The quantitative and qualitative data show that the intervention was deemed acceptable and feasible. The direction and magnitude of the treatment effects for T2 are promising.
Adherence in the control group decreased from 80% to 70% by the end of the study. In the T2 group, adherence increased initially and remained between 80% and 85% for the duration of the study. Regression models estimate a 3 percentage point increase in adherence in the initial 9 weeks of the study in the T2 group relative to the control, which grows to a 9 percentage point increase by the last 9 weeks of the study. The T1 intervention shows no signs of impact; adherence in the T1 group decreased from 84% to 74%. This pilot study was not powered to detect statistical differences.

Conclusions
Improving youth's adherence by supplementing information about their adherence with information about adherence of peers is a promising new strategy that should be further evaluated in a fully-powered study. Providing one's own adherence information alone appears to have less potential.

Background
In Uganda, studies report serious adherence problems among youth 15-24 years of age [1]. Recent growth in mobile phone ownership among youth in resource-poor settings [2] has highlighted the potential of using text-based interventions to improve ART adherence [3,4]; however, evidence of their effectiveness is mixed [5,6] and novel strategies to use phone-based messages are needed. Behavioral economics (BE) offers novel insights into systematic decision making errors ('biases') that might contribute to suboptimal adherence, potentially offering a way to enhance the effectiveness of text-based interventions.
This pilot study tested approaches rooted in BE that use text messages to improve ART adherence among youth (ages 15-24). We focused on two well-documented BE biases that may be particularly relevant for youth. We designed one intervention component to address 'optimism bias'-the tendency of individuals to overestimate their capabilities [7].
People tend to overestimate the likelihood of positive experiences and underestimate the likelihood of negative ones and youth in particular tend to be particularly overoptimistic [8]. Preliminary analyses of data from our previous studies found that adults overestimate their own capability to adhere to their medication; self-reported adherence was 91% on average, while electronically measured adherence was only 80%. Yet despite this poor performance, the large majority of patients (81%) believed that they would show 100% adherence in the subsequent month. This finding underscores the importance of feedback to make respondents aware of their true adherence level. Therefore, in our intervention we used text messages to give participants weekly feedback about their recent ART adherence to counter optimism bias.
We designed a second intervention component to leverage 'reference dependence bias'the tendency of individuals to want to equal or surpass the performance of their peers [9].
Peer comparison has been effective at improving health worker performance [10], voting behavior [11], and energy efficiency [12]. Youth are particularly attuned to the behavior of their peers, so leveraging this bias may be particularly effective with this age group.
We used text messages to give individuals information about the adherence of a reference group of their peers and to show how their own adherence compared to the group's adherence, with the hypothesis that this would lead the participants to try to equal or even surpass the adherence level of their peer group.
We based the text messages on data collected by Wisepill, a device that electronically records when pills are removed from the container and sends these data to a study computer. One treatment intervention arm of the study was given only their own individual ART adherence levels (T1), and the second intervention arm was given both individual and group ART adherence levels (T2). The control arm received the usual standard of care as provided by the clinic, including any adherence support mechanisms.
In this paper, we describe the results from our pilot study regarding the acceptability, feasibility, and preliminary impact of the intervention.

Methods
Our quantitative [13] and qualitative methods [14] are described in detail elsewhere; here we provide a brief summary of the data collected and methods used.
Study population and recruitment: Study participants age 15-24 were recruited from The AIDS Support Organization (TASO) at Mulago clinic in Kampala, the capital of Uganda, and in the suburb of Entebbe during scheduled clinic visits. Eligibility criteria included: 1) knows own HIV status, has disclosed to caretaker (if minor); 2) in care at TASO for at least three months, intends to seek care from this facility for the next year, and not currently participating in another health-related study; 3) taking ART or co-trimoxazole; 4) has regular access to a cell-phone (at least one hour per day, five days a week); 5) are not in boarding school (as mobile phones are often forbidden).
Once recruited, participants were given a Wisepill device and told to begin using it immediately. We monitored each participant's adherence via Wisepill for two months before announcing their assignment to the intervention or control arms of the study. We excluded recruited clients who demonstrated less than 20% adherence during this twomonth period because it suggested they were not using the Wisepill device, and consistent use of the device was necessary to accurately measure adherence. Twenty-four of 179 initially recruited clients (13%) were excluded for falling below this 20% threshold. The study team then randomly assigned the remaining 155 participants to one of the three study arms using a random number generator in Stata. Random assignments were determined in the pre-baseline period between recruitement and the start of the intervention.
Interventions: After this two-month monitoring period, we informed the remaining 155 participants of their randomly assigned group. In the control group (n=59), participants received care as usual, including any adherence support offered in the clinic. In the first treatment intervention arm (T1; n=40), clients received a weekly text informing them of their adherence level in the previous week as measured by the Wisepill device. This intervention was designed to provide feedback to counter the observed bias of overestimating one's own adherence. In the second treatment intervention arm (T2; n=56), clients received information about their own adherence as well as information about the adherence level of their peers in the intervention (see Appendix Table A1 for exact wording of the weekly messages). To avoid sending low group adherence statistics, we used either the adherence of the person in the 65 th percentile (if equal to or higher than 80%) or a randomly picked adherence level between 80% and 93%.
Quantitative data: We use two sources of quantitative data: 1) two waves of participant surveys (baseline and 9-month follow-up) to collect demographics and beliefs/behaviors related to HIV treatment; 2) data recorded by the Wisepill device, which recorded the number of doses taken by each participant during the study.
Qualitative data: We conducted six exit focus groups (FG) -1 with providers and 5 with youth between December 2016 and March 2017 to evaluate satisfaction with the intervention arms and to identify areas for improvement (Table 1).
To recruit participants for the provider FG, we invited all providers to attend the focus group; each participating provider was given approximately $16 USD as is customary in this setting. We conducted separate FGs for youth aged 18 and older and for those younger than 18 (i.e. minors). To recruit youth for the FGs, we generated a list of studyeligible youth attending the clinic on the day of the FGs; we asked 6-8 of them to attend each focus group and gave each approximately $8 USD. The youth (as requested by the study IRBs) provided verbal consent. All youth were reminded that participation was completely voluntary and that their responses would in no way impact provison of services going forward.
All FGs were transcribed, translated from Luganda into English, and uploaded into the qualitative analysis software Dedoose. We used a directed content analysis approach: our relevant research provided guidance for identifying the intial themes (e.g., acceptability of receiving individual versus group adherence information) while also providing flexibility for additional themes to emerge (e.g., implementation challenges) [15]. Therefore we developed our preliminary codebook based on initial themes that we had anticipated, and revised it as two researchers (SM and AMG) jointly reviewed a subset of transcripts.
Revisions to the codebook were made; the final version included definitions for code with example text when helpful. The two researchers established inter-rater reliability based on a pooled Cohen's Kappa of 0.79 and 0.80. The remaining interviews were single-coded and issues raised were discussed weekly. We complemented qualitative findings with summary notes from study staff.

Assessment of acceptability and feasibility:
We drew on existing frameworks in the peerreviewed literature to discuss core components of acceptability [16] and feasibility [17].
We describe acceptability based on the framework provided by Sekhon and colleagues [16] that assesses acceptability of an intervention based on cognitive and emotional responses to it. Tickle-Degnen [17] suggests determining feasibility based on four types of assessments: management, resource, scientific, and process. Adequate management of the study and adequate resources to conduct it are requirements for NIH funding. Here we focus on the scientific and process assessments (e.g., reliability of our measurement tools, adherence to study procedures) that determine the feasibility of large-scale implementation.
Analysis of preliminary impact: To assess the preliminary impact of the two modes of textbased interventions, we used an intention to treat framework. We applied linear regression to compare adherence in the intervention and control groups. We estimated average treatment effects with all post-intervention periods pooled and assessed how treatment effects evolved over time, splitting the 36 -week study into four 9-week intervals. We clustered standard errors by individual to account for potential autocorrelation in the error term. Although we estimate standard errors and confidence intervals, this is a pilot study; it is not powered to detect statistically significant treatment effects.

Results
Demographic characteristics of participants are shown in Table 2. All participants had a similar baseline adherence level. Compared with the control group, participants in the two treatment arms were more likely to be male and have English as their first language. They also had a higher monthly income.

Acceptability
Intervention coherence -Do participants understand SITA? Qualitative data showed that participants understood the intervention protocols. They viewed SITA as an intervention to improve their adherence and emphasized the helpful role of Wisepill and receipt of adherence information.
Affective attitude -How do participants feel about SITA? Participants had positive attitudes about SITA. In the survey data, 98% of participants reported that they would remain in the intervention if they had the choice, and 85% said there was nothing about SITA that they did not like. In the FGs, participants reflected on enjoying the competition with their peers, noting that SITA boosted their morale and prompted them to take their ART medication.
Self-efficacy -Are participants able to perform the SITA activities? The intervention has two key components: use of the Wisepill device and use of the mobile phone to which SMS were delivered. Wisepill recorded an 88% median level of adherence for participants, suggesting that most people used the Wisepill device to store their medication. Moreover, participants were generally very fond of the device: more than half of participants reported that Wisepill was the part of the study they liked the most, and many said their least favorite part of the study was having to give the device back. Participants experienced some challenges with the device --e.g., difficultly charging it; however, overall they appreciated its benefits. Anecdotal evidence from study staff noted some challenges in receiving Wisepill devices (e.g., fees required upon receipt at the airport) and preparing the devices for distribution (e.g., packaging the device along with the required cables, batteries, and plugs).
The second key protocol component was use of the mobile phone to which text messages were delivered. Exposure to the text messages was high, suggesting strong usage.
Seventy-five percent of participants reported reading the messages every week, and 85% said they read them most weeks.
However, the wrong individual adherence information was sent on 82 occasions. Staff notes suggest that the errors were due to technological difficulties with SIM cards when trying to register them with the Ugandan phone service provider. In the interviews, participants reported that receiving inaccurate information undercut their desire to further improve their adherence. The study team subsequently resolved this problem.
Perceived effectiveness -Did participants think SITA was effective? Participants also thought SITA was effective. In the survey, 98% reported benefiting from being part of the SITA program, and all participants reported that other youth at the clinic would benefit from being part of SITA. The FGs revealed that SITA consistently helped participants to take their medications on time. Further, participants noted that SITA's impact extended beyond the specific act of taking their medication: in many ways, the program improved their overall outlook on life and generated renewed focus on their health.
Representative qualitative quotes that support the key findings on acceptability are shown in Table 3.

Feasibility
Scientific assessments -Is SITA safe and standardized, and does it use valid measures?
Study safety was established through the ethics approval processes at the RAND Corporation, TASO, and the Uganda National Council for Science and Technology (UNCST).
In addition, the study protocol was published on the clinical trials registry (ClinicalTrials.gov Identifier: NCT02514356 [13]). All measures of adherence are established using a medication monitoring system through the Wisepill device -the gold standard for measuring medication adherence.
Process assessment -What is the recruitment process and are study procedures followed? Of the 229 approached, 32 individuals were ineligible, most often because they did not have a working phone or did not meet the age requirement; 18 declined to participate in the study because they were not interested or they provided no explanation. Staff notes also highlight that participants were concerned about potential disclosure of their HIV status (e.g., as a result of receiving calls from study staff associated with TASO, a known HIV service provider in the area). Despite these issues, we easily achieved our target numbers. Of those recruited for the study, 24 failed to achieve sufficient adherence in the pre-baseline period to warrant further study participation. Attrition was low, with only 8 of 155 participants lost to follow-up (5.16%). shows preliminary impact over time. It is common in studies that electronically measure adherence to see a decrease in adherence over time. Electronic measurement devices are a novelty that increases adherence initially but they become less salient over time. Over the course of the study, adherence in the control group and in the T1 (own adherence information) group steadily dropped off. The control group began at over 80% adherence but fell to about 70% by the end of the study. The T1 group decreased from around 84% to 74%. In the T2 group, adherence increased initially and the subsequent drop off was much less stark than in the other two groups. Adherence in the T2 group remained between 80% and 85% for the duration of the study. Our linear regression models show a positive treatment effect in the initial 9 weeks (3 percentage point increase in adherence) that grows to 9 percentage points by the last 9 weeks of the study (see Table 4).

Preliminary Impact
Because this is a pilot study, our sample size is not large enough to give us sufficient power to detect statistically significant effects. However, the direction and magnitude of the treatment effects for T2 are promising; the T1 intervention shows no signs of impact.
In other words, giving adolescents information about their own adherence (T1) does not appear to improve adherence, but giving them information about their own adherence relative to their peers (T2) shows promise for increasing adherence.

Discussion
In this paper, we use established criteria to determine the acceptability, feasibility, and preliminary efficacy of a pilot intervention testing a novel approach to improve ART adherence based on behavioral economics and mobile health technologies. Our analysis of acceptability focused on determining whether providers and youth considered SITA to be appropriate, beneficial, and not prohibitively burdensome. Both our quantitative and qualitative data show that youth understood SITA and felt positive about it. Further, consistent with other studies using Wisepill in low-resource settings [18], the participants felt strongly that Wisepill helped improve their adherence. The sample was selected based on their use of Wisepill in the pre-baseline period (13% were excluded because they did not use the device consistantly within the first two months of receiving the device) and some people declined to participate (9% of those eligible). Other strategies for increasing adherence may be needed for individuals who are not comfortable using technology in this context.
The pilot established the feasibility of sending text messages with information on a participant's own adherence and adherence performance relative to peers; initial problems were subsequently resolved. While use of Wisepill has been shown to be effective in other resource-poor settings [18], our study highlights the need to maintain clear lines of communication with Wisepill distributors and local phone companies, and to conduct routine data checks with participants to ensure accurate reporting of adherence.
We also examined SITA's safety, reliability of our measurement tools (e.g., the Wisepill device), feasibility of our recruitment process (size of eligible population, refusal and recruitment rates, and attrition), and adherence to study procedures (e.g., consistent use of the Wisepill device and retention of mobile phones).
Feedback from study staff highlighted several changes that could improve study implementation going forward. For example, they recommended identifying one individual to manage logistics associated with Wisepill, potentially helping to overcome some of the challenges in successfully receiving the devices in-country. Further, study staff noted that creating stronger contractual agreements with the phone company could facilitate communications when challenges arose (e.g., ensuring the phone company is willing to provide regular reports about the number of text messages sent, the number of messages bouncing back due to disconnected lines). Finally, study staff provided suggestions about increasing use of Wisepill among those participants who are hesitant to engage with technology. For example, participants who were concerned that the Wisepill device would signal their HIV status could be given potential responses when asked what the device was for (e.g., storage of vitamins). Alternatively, if participants, especially those in more rural areas, referenced concerns about their ability to consistently charge their cell phones, it could be helpful to make additional batteries available, or send texts reminding them to charge their phone.
Findings from this pilot study suggest that giving individuals information on their own adherence does not improve adherence, but giving them information on their adherence relative to their peers improves their adherence. A subsequent study at scale should be implemented to confirm these results and investigate whether the intervention works through the conceptual pathways hypothesized--countering optimism bias by providing own adherence information, and activating the power of social norms by providing information on the performance of the peer group.

Limitations
The study has both limitations and strengths. We have limited data on participants who were excluded or dropped out during the intervention, minimizing our understanding of how such factors might affect future scale up of SITA. Additionally, only the quantitative data include demographic information; thus despite our structured sampling frame, we cannot adequately compare how similar or different clients were who participated in the focus groups to the intervention as a whole.
These limitations are balanced with significant strengths. Using both quantitative and qualitative data, our study assesses the feasibility and acceptability of a novel approach to using SMS messages to promote adherence. We used existing frameworks for our analysis of 'acceptability' and 'feasibility,' terms that are commonly used but rarely defined further.

Conclusion
The intervention tested in this pilot study was found to be acceptable and feasible. The study provided preliminary evidence that giving youth information on their adherence relative to their peers can improve youths' adherence. Availability of data and material: The datasets generated and/or analysed during the current study are not publicly available due confidentiality concerns, but are available from the corresponding author on reasonable request.    Increased Morale: "When you have the wise pill device, it comes to your mind that someone will know whether you took drugs or not so it was always boosting morale." (Treatment Group 2)** Helpful Reminders: "[SITA] was so good generally especially to us who used not to take our drugs well so it reminded us so much (Treatment Group 1)

Self-Efficacy
Use of Wisepill Device: "You can easily move with it without anyone knowing what you are carrying and its only you that knows. One can easily mistake it for a power bank and not mind about it. It was very good especially making it easy to move with it rather than moving with medication containers while making noise." (Treatment Group 2) Use of SMS Messages: "I was eager to see the message because I very much wanted to see my percentage. Whenever, I saw my percentage for example like 30% or 50%, I would ask myself why it is 50%. I would ask myself, [and] it seemed here I skipped some minutes, so I am going to start to have to be punctual because poor adherence denied me a chance of the right percentage." (Treatment Group 1)

Perceived Effectiveness
Taking medication on time: "According to me, [Wisepill] is good because I used to take my drugs on a daily basis but not on time. It used to motivate me and I would say, let me take drugs on time so that I score good marks because someone is monitoring me, so it always reminded me to take drugs on time. That was its merit." (Treatment Group 1) Improved their overall quality of life: "It made me happy because before they gave it to me, I   Intervention effects over time