Skip to main content

Table 3 Inter–rater reliability

From: Chest radiograph reading and recording system: evaluation in frontline clinicians in Zambia

Agreement Index Parenchymal abnormalities Large opacifications Small opacifications Cavitation Pleural abnormalities Central abnormalities Any abnormality Abnormalities consistent with TB
CRRS–certified medical officers
 HIV Positive patients (N = 231)a
  Both readers agree abnormality present 38 8 24 2 32 17 78 69
  Only Reader 1 says abnormality present 56 38 36 30 13 21 41 28
  Only Reader 2 says abnormality present 7 5 10 0 16 17 22 29
  Both readers agree abnormality not present 130 180 161 199 170 176 90 105
  Percent Agreement (95 % CI) 0.73 0.81 0.80 0.87 0.87 0.84 0.73 0.75
(0.67–0.78) (0.76–0.86) (0.75–0.85) (0.83–0.91) (0.83–0.92) (0.79–0.88) (0.67–0.78) (0.70–0.81)
  Kappa (95 % CI) 0.38 0.20 0.40 0.10 0.61 0.38 0.46 0.49
(0.27–0.50) (0.06–0.35) (0.26–0.53) (–0.03–0.24) (0.48–0.74) (0.22–0.54) (0.34–0.57) (0.38–0.61)
  Strength of agreement (based on kappa) Fair Poor Fair Poor Good Fair Moderate Moderate
 HIV Negative patients (N = 335)a
  Both readers agree abnormality present 58 10 41 6 60 18 136 122
  Only Reader 1 says abnormality present 100 74 64 39 24 32 55 31
  Only Reader 2 says abnormality present 5 1 12 0 17 15 9 19
  Both readers agree abnormality not present 172 250 218 290 234 270 135 163
  Percent Agreement (95 % CI) 0.96 0.78 0.77 0.88 0.88 0.86 0.76 0.85
(0.64–0.74) (0.73–0.82) (0.73–0.82) (0.85–0.92) (0.84–0.91) (0.82–0.90) (0.72–0.81) (0.81–0.89)
  Kappa (95 % CI) 0.35 0.16 0.39 0.21 0.66 0.36 0.62 0.70
(0.27–0.43) (0.07–0.26) (0.29–0.50) (0.07–0.35) (0.57–0.76) (0.21–0.50) (0.54–0.70) (0.62–0.77)
  Strength of agreement (based on kappa) Fair Poor Fair Fair Good Fair Good Good
CRRS-oriented clinical officers
 HIV Positive patients (N = 231)a
  Both readers agree abnormality present 46 33 5 1 55 14 107 102
  Only Reader 3 says abnormality present 33 28 22 5 16 41 40 35
  Only Reader 4 says abnormality present 44 37 11 11 63 21 41 43
  Both readers agree abnormality not present 108 133 193 214 97 155 43 51
  Percent Agreement 0.67 0.72 0.86 0.93 0.66 0.73 0.65 0.66
(0.61–0.73) (0.66–0.78) (0.81–0.90) (0.90–0.96) (0.60–0.72) (0.67–0.79) (0.59–0.71) (0.60–0.72)
  Kappa (95 % CI) 0.28 0.31 0.16 0.08 0.32 0.15 0.24 0.29
(0.16–0.41) (0.18–0.44) (–0.02–0.34) (–0.12–0.28) (0.21–0.43) (0.01–0.29) (0.11–0.37) (0.17–0.42)
  Strength of agreement (based on kappa) Fair Fair Poor Poor Fair Poor Fair Fair
 HIV Negative patients (N = 335)a
  Both readers agree abnormality present 88 58 9 1 107 36 177 165
  Only Reader 3 says abnormality present 42 38 43 10 18 70 52 51
  Only Reader 4 says abnormality present 36 48 11 15 86 22 48 48
  Both readers agree abnormality not present 169 191 272 309 124 207 58 71
  Percent Agreement (95 % CI) 0.77 0.74 0.84 0.92 0.69 0.73 0.70 0.70
(0.72–0.81) (0.70–0.79) (0.80–0.88) (0.90–0.95) (0.64–0.74) (0.68–0.77) (0.65–0.75) (0.66–0.75)
  Kappa (95 % CI) 0.51 0.39 0.18 0.04 0.40 0.28 0.32 0.36
(0.41–0.60) (0.28–0.50) (0.05–0.31) (–0.10–0.18) (0.31–0.49) (0.17–0.38) (.21–0.42) (0.25–0.46)
  Strength of agreement (based on kappa) Moderate Fair Poor Poor Fair Fair Fair Fair
  1. a2 CXRs among HIV-positive and 3 among HIV-negative patients were deemed ‘Unreadable’ by one or more reader and excluded from analyses