Defining cognitive impairment in people-living-with-HIV: the POPPY study

Background The reported prevalence of cognitive impairment (CI) varies widely in cohorts of people living with HIV (PLWH); this may partly be due to the use of different diagnostic criteria. Agreement between diagnostic criteria of CI, the optimal definition to use, and associations with patient-reported cognitive symptoms have not been fully investigated. Methods Two hundred ninety PLWH aged >50 years and 97 matched negative controls completed a detailed assessment of cognitive function and three questions regarding cognitive symptoms. Age- and education-adjusted test scores (T-scores) determined if subjects met the following definitions of CI: Frascati, global deficit score (GDS) and the multivariate normative comparison (MNC) method. Results PLWH were more likely than controls to meet each definition of CI (ORs were 2.17, 3.12 and 3.64 for Frascati, GDS and MNC, respectively). Agreement of MNC with Frascati and GDS was moderate (Cohen’s k = 0.42 and 0.48, respectively), whereas that between Frascati and GDS was good (k = 0.74). A significant association was found between all the three criteria and reporting of memory loss but not with attention and reasoning problems. The 41 (14 %) PLWH meeting all the three criteria had the lowest median global T-score (36.9) and highest rate of symptom reporting (42 %). Conclusions Different CI criteria show fair diagnostic agreement, likely reflecting their ability to exclude CI in the same group of individuals. Given the lower overall cognitive performance and higher rates of symptom reporting in those meeting all three criteria of CI, further work assessing this as a definition of CI in PLWH is justified. Electronic supplementary material The online version of this article (doi:10.1186/s12879-016-1970-8) contains supplementary material, which is available to authorized users.


Background
In recent years, despite the introduction of combination antiretroviral therapy (ART), a high but varying prevalence of cognitive impairment (CI) has been reported among HIV-positive individuals. Robertson et al. [1] described a 26 % prevalence of CI among HIV-positive subjects who had received ART for at least 20 weeks, while Heaton et al. [2] reported a prevalence of 36 % among ART-treated asymptomatic individuals. Among those with long-standing viral suppression, Winston et al. [3] and Simioni et al. [4] reported prevalences of 51 and 74 %, respectively.
The optimal screening tools to identify CI are unknown. The current European AIDS Clinical Society (EACS) guidelines recommend three questions as one form of assessment to guide the initial evaluation of HIV-positive individuals with suspected cognitive problems [5]. These patient-reported measures may be a quick and practical method of screening for cognitive impairment in clinical practice, however, doubts about their utility remain [6]. When cognitive function is assessed through objective neuropsychological tests, differences in the reported prevalence of CI may be due to the use of different diagnostic criteria. Three criteria in current use include the 'Frascati' criteria, proposed by Antinori et al. [7] and also known as HIV-associated neurocognitive disorder (HAND) criteria, the global deficit score (GDS) [8] and the multivariate normative comparison (MNC) [9]. These criteria differ in the way they combine scores from a battery of neuropsychological tests to classify subjects as either cognitively impaired or normally functioning. Several studies have reported contrasting results when using different criteria of CI on the same set of patients [10,11]. However, they only assessed prevalence rates by criteria and did not specifically investigate whether they identified the same people as impaired/not impaired.
The associations between CI and patient-reported outcomes of cognitive function have not been fully investigated and remain unclear. Some studies have found a relationship between subjective cognitive complaints and actual impairment in neuropsychological tests [12,13], while other studies have not [14,15]. While these studies tended to include high rates of untreated or unsuppressed subjects, there is currently a lack of data relating cognitive function with self-reported cognitive complaints in populations of well-treated HIV-positive individuals.
The aims of this study are threefold. Firstly, to compare the prevalence of CI in HIV-positive individuals over 50 years of age and demographically matched HIV-negative controls according to the Frascati criteria, GDS and MNC and combinations of these definitions. Secondly, to assess the level of agreement between these three criteria when identifying HIV-positive people with CI. Finally, to investigate the association between different definitions of CI and their combination with patient-reported symptoms of cognitive dysfunction.

Study design and participants
The Pharmacokinetic and Clinical Observations in People Over Fifty (POPPY) study is a prospective, multicentre, observational study that aims to examine the effects of ageing on the clinical outcomes of people living with HIV in UK and Ireland. To address its aims the study has established cohorts of HIV-positive people aged over 50, younger HIV-positive controls less than 50 years old and demographically matched HIV-negative controls aged over 50 years. For the present analysis only the two older cohorts were considered as the purpose of the younger HIV-positive cohort is to provide a younger control group which is not directly relevant to this analysis. HIV-positive participants were recruited at HIV outpatient clinics around UK and Ireland. Inclusion criteria were: documented presence of HIV infection, self-defined white or black-African ethnicity, likely route of HIV acquisition via sexual exposure (either by male to male exposure if white or by heterosexual exposure if white or black-African) and ability to comprehend the study patient information leaflet. HIV-negative controls were frequency matched to the HIV-positive group on gender, ethnicity, sexual orientation and location (in or out of London) and were recruited from sexual health clinics affiliated with the HIV clinics, as well as from community events, churches, adverts in targeted publications and community groups. Recruitment was from January 2013 to September 2014. The study was approved by the UK National Research Ethics Service (NRES; Fulham London, UK number 12/LO/1409). Written informed consent was obtained from all participants prior to undertaking any study specific procedures.

Cognitive symptoms
All enrolled participants completed questionnaires detailing physical and mental health status. In particular, participants answered the three questions on cognitive symptoms described by the EACS guidelines [5] regarding memory loss (do you experience frequent memory loss [e.g., do you forget the occurrence of special events even the more recent ones, appointments, etc.]?), reasoning (do you feel that you are slower when reasoning, planning activities, or solving problems?) and attention (do you have difficulties paying attention [e.g., to a conversation, book or movie]?). Individuals answering 'Yes, definitely' , as opposed to 'Never' and 'Hardly ever' were classified as experiencing the related cognitive symptom. In addition a positive answer to at least two of the three questions was considered indicative of self-reported cognitive problems.

Assessment of cognitive function
Assessment of cognitive function was performed using the CogState battery [16], a computerized battery of neuropsychological tests that has been used in different clinical settings [17][18][19][20], including HIV-positive cohorts [21][22][23][24]. The battery covered six cognitive domains commonly affected by HIV-associated CI, including visual learning, psychomotor function, visual attention, executive function, verbal learning and working memory (see Additional file 1: Table S1 for details of individual tests and how they map onto cognitive domains). Raw test scores were logtransformed or arcsine root-transformed where necessary (as recommended by the CogState guidelines for analysis) and converted into demographically-adjusted T scores (with a mean of 50 and a standard deviation of 10) using the scores of the HIV-negative group as normative scores. Briefly, a linear regression was fit for each test in order to estimate regression coefficients for age, gender, ethnicity and education using scores from the HIV-negative group. These regression coefficients were then used to determine the normative scores depending on subjects' age, gender, ethnicity and education. The difference between the normative score and the actual score for each subject was then standardized into T-scores. A single T-score was calculated for each of the 6 cognitive domains by averaging individual test T-scores within each domain. A global Tscore was also obtained by averaging T-scores across the six domains. For all T-scores a higher value indicates better cognitive function.

Classification of CI
For each subject, the T-scores were then used to determine if the individuals met three definitions of CI, Frascati, GDS and MNC, using published methods. Frascati and GDS are the most extensively adopted definitions in previous studies of CI in HIV, while the MNC is a relatively newer approach showing promising results in reducing the false positive rate [10]. First, according to the Frascati criteria [7], CI was defined as at least two cognitive domain T-scores below 40 (i.e., one or more standard deviations below the average normative score). Second, a GDS [8] was computed for each subject by converting domain T-scores into deficit scores (0: T-score ≥ 40, 1: 34 < T-score < 40, 2: 29 < Tscore ≤ 34, 3: 24 < T-score ≤ 29, 4: 19 < T-score ≤ 24, 5: Tscore ≤ 19). An overall GDS was obtained by averaging domain deficit scores and CI was defined as a mean score equal or greater than 0.5. Finally, the MNC method [9] was applied. The MNC is a statistical method that simultaneously compares multiple cognitive scores of each study participant to the average scores of the same tests in the control group (in our case the HIV-negative group), taking into account the variances and covariance between all scores. For each participant, a continuous measure of the deviation of the participant's cognitive profile from the average cognitive profile in the control group is then obtained. If this deviation (also called Hotelling's T 2 ) exceeds a critical value associated with a 5 % significance the individual is classified as cognitively impaired (so that the chance of erroneously concluding that an individual has CI while this is not the case, i.e., the false positive rate, is approximately 5 %). Definitions of CI included the three criteria listed above plus all combinations of patients meeting individual criteria and two or all three of these criteria.

Statistical analysis
Group comparisons of baseline characteristics were assessed using Chi-square, Wilcoxon rank-sum and t-tests (two-tailed) as appropriate. Comparisons of the prevalence of CI in HIV-positive and HIV-negative persons were performed using the Chi-square test with odds ratios used to provide a comparative estimate of the prevalence in the two groups. The agreement between criteria of CI was assessed using Cohen's κ statistics [25] and interpreted following Landis and Koch [26] guidelines. The null hypothesis that Cohen's κ equals zero (i.e., no agreement between criteria other than what would be expected by chance) was tested using the asymptotic test [27]. Internal consistency between the three patient-reported cognitive problems was assessed using Cronbach's α. Associations between each definition of CI and self-reported cognitive problems were assessed using the Chi-square test. All analyses were performed using SAS v9.4 with p-values <0.05 considered as statistically significant.

Participant characteristics
A total of 290 HIV-positive and 97 HIV-negative participants were enrolled into the study between January 2013 and September 2014 and completed the CogState battery. Demographic, lifestyle and HIV-related characteristics are summarised in Table 1

Agreement between definitions of CI
Overlap in the classification of CI between the three criteria and Cohen's κ statistics are reported in Fig. 1.  (Fig. 2).

Association between definitions of CI and self-reported cognitive problems
Overall, 14, 15 and 17 HIV-positive individuals did not answer or had missing information on memory loss, reasoning and attention problems, respectively. Among those with complete information, 79 (28.6 %) reported frequent memory loss, 105 (38.2 %) reported reasoning problems and 79 (28.9 %) attention problems; moreover 90 (32.5 %) reported at least two of the three problems. Internal consistency of the three patient-reported measures was excellent with a Cronbach's α (95 % CI) of 0.9 (0.82, 0.98). A significant association was found between memory loss and all the three definitions of CI (Table 2): using the Frascati criteria, 38 % of subjects with CI reported frequent memory loss, while only 25 % did so among those without CI (p = 0.02), with

Discussion
HIV-positive individuals exhibit poorer cognitive function when compared to an appropriate HIV-negative control group. Although the difference in the overall cognitive score is statistically significant, this would not be considered clinically meaningful (for T-scores, a 5point difference is usually considered relevant from a clinical point of view [28]). The prevalence of CI in older  HIV-positive individuals may vary from 35 to 22 % depending on the criteria used.
Commonly-used criteria of CI show fair agreement, especially Frascati and GDS. However, this agreement is mainly driven by the ability of criteria to exclude CI in the same set of subjects rather than their ability to identify CI. As expected, subjects meeting all the criteria have generally poorer cognitive function (with particularly poor performances in the psychomotor and visual attention domains) and are more likely to experience cognitive problems. Similar cognitive scores were also observed in subjects classified as impaired by Frascati and GDS (but not MNC).
Consistent with several published studies [14,15], but contrary to others [12,13], the associations with selfreported cognitive symptoms are generally poor for all the three definitions of CI considered. Whilst CI, defined with all three criteria, correlate with memory loss, this is not the case for either attention or reasoning problems. These results, based on a cohort of mainly treated and virallysuppressed subjects, shed further light on the association between patient-reported and objective cognitive impairment in the post-ART era. The lack of association found may reflect the pattern of cognitive changes we observed, namely, poorer verbal learning (which may relate to patient reported memory problems) but no significant differences in executive function (reasoning) or working memory (attention). These results suggest a potential lack of a clear relationship between subjective measures of cognitive function and more objective measures based on neuropsychological tests, particularly in those with mild impairment. Several reasons may account for this lack of association, such as the over-reporting of cognitive symptoms and the subjectivity of the EACS questions. Moreover, depressive disorders have been previously reported to affect both subjective and objective cognitive function [29], and can therefore confound the association between the two. We did not co-vary depression in our analyses, but from preliminary analysis, depression did not seem to change the associations between definitions of CI and patient-reported symptoms (data not shown).
A missing answer to questions about cognitive complaints may, in theory, be informative as it may indicate poor understanding or attention. However, almost 95 % of data was complete so it's unlikely that the exclusion of this small group of individuals from analyses has introduced substantial bias. Although ART prescribing was in line with national guidelines, 3.1 % of enrolled HIV-positive individuals were not receiving suppressive antiretroviral therapy at study entry. This is justified by the aim of the study of recruiting a 'real-world' sample of PLWH in UK and Ireland. Nonetheless those not receiving ART had a median CD4+ cell count of 664 cells/μL, which makes it unlikely that lack of suppression of HIV replication in a minority of the sample has led to substantial bias to our findings.
Given the lack of a gold standard in defining CI, it is difficult to ascertain the validity of different definitions. An optimal definition of CI would capture subjects with the lowest cognitive performance scores and the greatest number of cognitive symptom complaints. In our study we observed the lowest overall cognitive score in subjects meeting all three definitions (median global Tscore of 36.9 for those meeting Frascati, MNC and GDS). Moreover, in this group of subject, the number reporting cognitive complaints was highest (41.5 %). Given these findings, we consider further work to assess longitudinal outcomes in HIV-positive individuals meeting this definition of CI within the POPPY study, compared to other definitions of cognitive impairment, is justified. Other definitions of cognitive impairment in PLWH, such as the Frascati criteria, have been criticised for overcalling the number of PLWH with cognitive deficits [11]. On the converse argument, the Frascati criteria has attempted to define the presence of cognitive deficits prior to the onset of clinical symptomatology and classifies large numbers of subjects with CI. The rationale behind such criteria being interventions at an early stage of disease, if effective, may prevent the onset of clinically apparent conditions. However, to date, longitudinal data both in the HIV-field and in other neurodegenerative diseases have not provided convincing evidence for the diagnosis of a pre-morbid cognitive state [30,31].
There are several other problems when utilising nonstringent definitions of CI. Firstly, as large numbers of patients will meet a diagnostic criteria, unnecessary anxiety for patients could be created. Secondly, non-stringent definitions will include subjects with cognitive impairment but will also include subjects who do not have cognitive impairment. Within interventional trials, this may lead to the null hypothesis being proven when in fact an intervention does work. By including patients without a disease state in an interventional study, the results of the study may suggest an effective interventions has no effect whereas, if the effective intervention was trialled in the diseased population, an effect may be observed.

Conclusions
Commonly-used criteria of CI show fair agreement, especially in identifying subjects without CI. However their association with patient-reported symptoms is generally weak and we hypothesise that our POPPY Study definition of cognitive impairment may be a pragmatic approach to adopt at the current time. Our definition, by defining subjects with the highest rates of symptomatology and the lowest global cognitive score is likely to capture patients with true pathological cognitive impairment and requires validation within other HIV-cohorts and in longitudinal settings.

Additional file
Additional file 1: Table S1.