- Open Access
Simple-to-use nomogram for predicting the risk of syphilis among MSM in Guangdong Province: results from a serial cross-sectional study
BMC Infectious Diseases volume 21, Article number: 1199 (2021)
The purpose of this study was to develop and validate a simple-to-use nomogram for the prediction of syphilis infection among men who have sex with men (MSM) in Guangdong Province.
A serial cross-sectional data of 2184 MSM from 2017 to 2019 was used to develop and validate the nomogram risk assessment model. The eligible MSM were randomly assigned to the training and validation dataset. Factors included in the nomogram were determined by multivariate logistic regression analysis based on the training dataset. The receiver operating characteristic (ROC) curves was used to assess its predictive accuracy and discriminative ability.
A total of 2184 MSM were recruited in this study. The prevalence of syphilis was 18.1% (396/2184). Multivariate logistic analysis found that age, the main venue used to find sexual partners, condom use in the past 6 months, commercial sex in the past 6 months, infection with sexually transmitted diseases (STD) in the past year were associated with syphilis infection using the training dataset. All these factors were included in the nomogram model that was well calibrated. The C-index was 0.80 (95% CI 0.76–0.84) in the training dataset, and 0.79 (95% CI 0.75–0.84) in the validation dataset.
A simple-to-use nomogram for predicting the risk of syphilis has been developed and validated among MSM in Guangdong Province. The proposed nomogram shows good assessment performance.
Syphilis remains an urgent public health priority worldwide . There are an estimated 6 million new cases of syphilis globally among people aged 15 to 49 years each year. MSM are disproportionately burdened with syphilis infection globally , especially in low- and middle-income countries (LMICs), including China . The overall prevalence of syphilis was 7.1%  and the incidence was 9.6 cases per 100 persons in 2015 among Chinese MSM .
Given the high risk of syphilis transmission among MSM, identifying MSM who are at high risk of syphilis infection can help design tailored behavior interventions to prevent the spread of syphilis . Many studies have investigated the risk factors of syphilis infection, and it was found that MSM who had unprotected anal sex, commercial sex, multiple sexual partners or used drugs were more likely to be infected with syphilis in China [7,8,9,10]. However, previous studies only focused on identifying the associated factors without quantification and validation for the risk factors and failed to predict the probability of syphilis infection. Prediction of the probability of syphilis can help identify MSM who are at high risk, thus undergoing further intervention, testing and treatment. Therefore, the aims of this study were to investigate factors predictive of syphilis infection, and to develop and validate a simple-to-use nomogram for the prediction of syphilis infection among MSM in Guangdong Province in China.
Study design and participants
A governmental sentinel surveillance network has been set up for regular syphilis surveillance among MSM in four cities (Foshan, Jiangmen, Shenzhen, and Yunfu) in Guangdong Province in Southern China from April to June annually since 2017. (Additional file 1). Guangdong Province has a high burden of syphilis infection and has been consistently ranked first on syphilis incidence for the last decade among general population in China .
A standardized sentinel survey protocol designed by the Guangdong Provincial Centre for Sexually Transmitted Diseases Control was sent to sentinel surveillance network sites for data and sample collection each year. The sentinel surveillance was conducted by local public STD hospitals in each city. Each sentinel surveillance hospital has set up the MSM outreach team consisting of nurses, clinical physicians and public health physician. These outreach teams have extensive experiences of MSM outreach service, which can provide syphilis testing, condom promotion, comprehensive sexual health education, and behavioral intervention to reduce risks. The participants were recruited through offline outreach services, including pub, disco, tearoom and club. As a rule of thumb and based on the recommendation of World Health Organization, a minimum sample size of 200 participants was required for each sentinel surveillance hospital [12, 13]. Eligibility criteria for sentinel surveillance included: born biologically as a male, more than 18 years old, and had anal sex with a man at least once in the past 6 months. Among the eligible subjects, those who have already participated in this survey in other cities during the same year were excluded through face-face interview. Since we collected mobile phone numbers to inform the participants of the results of the syphilis test, we also excluded participants with the same mobile phone number.
The data was collected using paper questionnaires in this research. The questionnaire items were determined through discussions with STD experts, MSM and local sentinel surveillance hospital staff. We also piloted the survey with 20 volunteer MSM to test questionnaire items. The pilot data were not included in the final analysis. After providing written informed consent, MSM who agreed to participate in the study were referred to a separate, quiet room to complete the questionnaire with the help of a trained investigative assistant. We collected the phone number of each participant for notification of syphilis testing result. Each participant was given a disturbance allowance of 30 Yuan (about 4.3 USD) as an incentive for participation.
Social-demographic and behavioral variables
Socio-demographic characteristic variables included age, marital status, residency, ethnicity and educational level. Sexual behavioral variables included commercial anal sex with men as well as vaginal sex with women, condom use consistently with men or women in the last 6 months, and condom use in last commercial sex. Drug use in lifetime and STD infection history in the past year were also collected. Drug use was defined as the using of any of the following drugs in the past year, cannabis, heroin, cocaine, crack, ecstasy, amphetamines, poppers, ketamine and methamphetamine. STD infection history was defined as the diagnosis of any of the following diseases in the past year: syphilis, chlamydia, gonorrhea, herpes and HPV infection. Condom use consistently over the last 6 months was defined as consistently using condoms in this study.
Syphilis laboratory testing
Blood samples were collected from all eligible participants for syphilis testing by nucleic acid amplification tests. The rapid plasma regains (RPR) test (Lizhu Biotech Inc, Zhuhai, China) was used for syphilis screening and Treponema pallidum particle agglutination (TPPA, Rongsheng Biotech Inc, Shanghai, China) was used for syphilis test confirmation. The syphilis test kits were approved by China’s State Food and Drug Administration. All participants would receive the syphilis test results through mobile text message within one week.
All the data in this study were double-entered using the Epidata 3.0 software (EpiData Association from Denmark). All the dataset was randomly assigned into the training and validation datasets at the ratio of 2:1 (seed = 163,407) . Categorical data (socio-demographic characteristics, sexual behaviors and syphilis infection characteristics) were presented as the number and percentage of MSM. The Chi-square test was used to compare qualitative variables between syphilis-infected and uninfected groups.
Nomogram is a graphical calculating device, which can reduce statistical risk factors to a single numerical estimate tailored to the individual patient’s profile and can generate an individual probability of an event [14, 15]. Analysis steps were as follows: First, a multivariate binary logistic regression model with stepwise variable selection was used to fit the prediction model based on the variables with a P-value less than 0.1 in univariate analyses using the training dataset [16, 17]. Second, a nomogram was developed depending on the independent significant syphilis infection risk factors, which were found in multivariate binary logistic regression using the training dataset [16, 17]. Each independent risk factor was assigned a corresponding score according to its location on the nomogram. The final risk score was calculated by adding up the score of each risk factor. A risk probability curve also demonstrated the association between total risk scores and syphilis infection. Third, to evaluate the performance and discrimination of the nomogram, we created a receiver operating characteristic (ROC) curve based on the total risk scores that were calculated by the nomogram in training and validation dataset. The optimal cut-off point for the predictive probability was determined by the maximization of Youden's index. In addition, the calibration was performed to assess performance of the nomogram by plotting the calibration curve to compare the observed prevalence and the predicted probability . We used 5000 bootstrap resamples for calibration curves. Furthermore, the sensitivity, specificity, the positive predictive value (PPV), and the negative predictive value (NPV) were also calculated based on the optimal cut-off point. All analyses were conducted on R version 3.6.3 software (The R Project for Statistical Computing).
In total, 2184 participants were recruited in this survey. Of those participants, the majority were between 18 and 35 years old (64.4%, 1406/2184), were never married (63.7%, 1392/2184), were of Han ethnicity (98.5%, 2152/2184), and were local residents (65.3%, 1427/2184). About one-third had a college degree or higher (37.5%, 819/2184). (Table 1).
Sexual behaviors and syphilis prevalence
The majority used internet to seek male sexual partners (60.6%, 1324/2184), and consistently used condom in the past 6 months (68.0%, 1485/2184). About one-tenth had commercial sex with men (10.0%, 218/2184) and had sex with women in the last 6 months (15.5%, 339/2184). The overall prevalence of syphilis was 18.1% (396/2184). The prevalence of syphilis in 2017, 2018 and 2019 was 12.4% (77/620), 18.9% (127/613) and 21.5% (192/891), respectively.
Characteristics of the training and validation datasets
Of all the MSM participants, 1456 MSM were randomly selected as the training dataset, and the remaining 728 were regarded as the validation dataset. The differences between the characteristics of the training dataset and the validation dataset were not statistically significant, suggesting that the randomization into training and validation subset worked very well (Table 2).
Independent influencing factors
Table 3 lists the independent risk factors for syphilis infection among Chinese MSM by the multivariate logistic regression model. Stepwise logistic regression was used to determine independent risk factors of syphilis prevalence based on the variables with a P-value less than 0.1 in univariate analyses. The multivariate analysis indicated that MSM who were older (over 45 years old vs. 18–25 years old) (odds ratio (OR) = 1.90, 95%CI:1.15–3.14), sought sexual partners through non-internet route (OR = 10.49, 95%CI:7.38–14.91), had not consistently used condom in the last 6 months (OR = 1.41, 95%CI:1.03–1.96), had commercial sex with men in the last 6 months (OR = 2.50, 95%CI:1.59–3.94) and had been infected with STD (OR = 3.70, 95%CI:3.18–8.19) in the past year were more likely to have syphilis infection. (Table 3).
Development of a nomogram
On the basis of the multivariate logistic regression model, a predictive nomogram was constructed to evaluate the risk of syphilis. The scores for each factor of over 45 years old, seek sexual partners through non-internet route, inconsistent condom use with men in past 6 months, had commercial sex with men in past 6 months and had been infected with STD in past year were 27,100,15,39 and 69, respectively. (Fig. 1 and Additional file 1). A risk probability curve demonstrating the association between total risk scores and syphilis infection is shown in Additional file 1.
Validation of the assessment accuracy of nomogram
The C-index value was 0.80 (95% CI: 0.76–0.84) in the training dataset, indicating a fair power to identify syphilis infection (Fig. 2). As a cut-off point of 61.50 was determined by the maximization of Youden's index up to 0.509, it was shown that predictive accuracy of a nomogram comprised a sensitivity of 86.9%, a specificity of 64.0%, PPV of 35.2% and NPV of 95.6%. The C-index value was 0.79 (95% CI: 0.75–0.84) in the validation dataset (Fig. 2). The calibration curves of syphilis rates showed good agreement in training set and validation set, respectively (Fig. 3). In the validation set, we used the same cut-off value (61.5), the sensitivity, and the specificity, the PPV and the NPV were 81.4%, 68.3%, 94.5% and 35.6%, respectively.
In this study, a syphilis risk-prediction nomogram model was developed and validated using a relatively large MSM dataset in Guangdong Province. To the best of our knowledge, this is the first report of a nomogram incorporating significant risk factors for syphilis prediction among MSM globally. ROC indicated that this nomogram had significantly high sensitivity and specificity to distinguish individuals at a high risk of syphilis. This constructed nomogram model provides a simple-to-use and individualized tool for syphilis risk assessment, which can help target higher risk populations for timely syphilis testing and risk-reduction intervention among MSM.
In this study, we found that the overall prevalence of syphilis was 18.1%, and this prevalence among MSM was higher than the overall rate across China , and similar with that in developed cities among MSM in China . The prevalence was higher than those in many developed countries, such as Argentina , Netherlands , Germany , and United Kingdom . Our result also showed that the prevalence of syphilis was increasing year by year. The high prevalence may be due to the indulgence of unprotected anal or commercial sex intercourse in a large proportion of MSM in China . The high prevalence of syphilis along with high-risk sexual behaviors may further accelerate the spread of the HIV and other STDs among MSM. Individual syphilis risk assessment can help identify populations at high risk for timely syphilis testing, treatment and risk-reducing behavior intervention.
In this study, five predictors were incorporated into the nomogram for the prediction of syphilis infection. We found that MSM who used condom consistently or had commercial sex in the past 6 months were more likely to be infected with syphilis [8,9,10, 19], which was consistent with other studies among Chinese MSM. Consistent and correct use of condoms has been proven to effectively reduce the risk of syphilis . MSM who mainly seek male sexual partners through non-internet (traditional meeting places) were more likely to be infected with syphilis, this was consistent with other study . This may be due to the fact that participants who seek sexual partners through non-internet route have more sexual partners and unprotected sexual behaviors . Meanwhile, we also found that MSM who had STD history in the past year were more likely to be infected with syphilis, which was consistent with other study . Unlike previous studies on syphilis infection prediction models, which only identified the associated risk factors, our study quantified and validated those risk factors, which can accurately predict the risk of syphilis. The discriminatory accuracy of our model appears to be good, which can be used for public health intervention, syphilis screening and personal syphilis risk assessment . This nomogram risk evaluation model provides MSM with an accurate reflection of their vulnerability to syphilis infection due to their highly risky sexual behaviors. This tool can also be applied in other regions and countries with our method, while using their own MSM serial cross-sectional data to simulate their local parameters and develop a specialized syphilis risk assessment model.
The most important aspect of this syphilis nomogram model is its public health applicability and ease of use in a wide variety of health systems. As an example, a male aged 20 years who mainly used internet to seek sexual partners, consistently used condom with men in the last 6 months, had no commercial sex with men and ever had STD infection last year, would have a total risk score of 69 points, which corresponds to a probability of syphilis infection of 13%. In contrast, a male aged 40 years who mainly used non-internet methods to seek sexual partners, used condom inconsistently with men over the past 6 months, had commercial sex with men and ever had STD infection in the past year, would have a total risk score of 244 points, which corresponds to a probability of syphilis infection of 90%. The current findings support the risk assessment potential of the developed and validated nomogram, which is relatively straightforward to understand and can be obtained using a simple intake form in a time-saving way.
There were several limitations in the development of this syphilis infection nomogram model. First, there is a lack of several routinely available data in this study, such as the average number of sexual partners with men and women and the role of sexual behavior. Constructing a risk assessment model using both the factors identified in our model and other variables would thus be beneficial in creating a much more accurate syphilis risk prediction model. Second, the susceptibility of the self-reported behavioral data to social desirability bias might lead to misclassification in our study , especially those related to sexually risky behaviors. Third, the study was of cross-sectional design, thus there was generally no evidence of a temporal relationship between syphilis infection and factors. Last but not the least important, we only excluded the eligible subjects who have already participated in this survey in other cities through face-to-face interview and the same mobile phone number, there may still be repeated participants in the study.
In conclusion, we developed and validated a simple-to-use nomogram for the prediction of syphilis among MSM. The nomogram model showed good assessment performance. The introduction of a syphilis risk nomogram model into syphilis control practice can help reduce the sexually risky behaviors and the prevalence of syphilis. Further studies to explore effective methods of promoting individualized syphilis risk assessments among MSM are warranted.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Men who have sex with men
Receiver operating characteristic
Sexually transmitted diseases
Low- and middle-income countries
Rapid plasma regains
Treponema pallidum particle agglutination
Positive predictive value
Negative predictive value
Wang C, Cheng W, Li C, Tang W, Ong JJ, Smith MK, Fu H, Marks M, Nie J, Zheng H, et al. Syphilis self-testing: a nationwide pragmatic study among men who have sex with men in China. Clin Infect Dis. 2019. https://doi.org/10.1093/cid/ciz603.
Abara WE, Hess KL, Neblett Fanfair R, Bernstein KT, Paz-Bailey G. Syphilis trends among men who have sex with men in the United States and Western Europe: a systematic review of trend studies published between 2004 and 2015. PLoS ONE. 2016;11(7): e159309.
Kojima N, Klausner JD. An update on the global epidemiology of syphilis. Curr Epidemiol Rep. 2018;5(1):24–38.
Fu R, Zhao J, Wu D, Zhang X, Tucker JD, Zhang M, Tang W. A spatiotemporal meta-analysis of HIV/syphilis epidemic among men who have sex with men living in mainland China. BMC Infect Dis. 2018. https://doi.org/10.1186/s12879-018-3532-8.
Chen G, Cao Y, Yao Y, Li M, Tang W, Li J, Babu GR, Jia Y, Huan X, Xu G, et al. Syphilis incidence among men who have sex with men in China: results from a meta-analysis. Int J STD AIDS. 2016;28(2):170–8.
Stahlman S, Plant A, Javanbakht M, Cross J, Montoya JA, Bolan R, Kerndt PR. Acceptable interventions to reduce syphilis transmission among high-risk men who have sex with men in Los Angeles. Am J Public Health. 2015;105(3):e88–94.
Zhang Y, Wu G, Lu R, Xia W, Hu L, Xiong Y, Xie J, Yu Q, Ye M. What has changed HIV and syphilis infection among men who have sex with men (MSM) in Southwest China: a comparison of prevalence and behavioural characteristics (2013–2017). BMC Public Health. 2019. https://doi.org/10.1186/s12889-019-7730-0.
Hu Q, Xu J, Zou H, Liu J, Zhang J, Ding H, Qian H, Li S, Liu Y, Jiang Y, et al. Risk factors associated with prevalent and incident syphilis among an HIV-infected cohort in Northeast China. BMC Infect Dis. 2014;14(1):658.
Wu Z, Xu J, Liu E, Mao Y, Xiao Y, Sun X, Liu Y, Jiang Y, McGoogan JM, Dou Z, et al. HIV and syphilis prevalence among men who have sex with men: a cross-sectional survey of 61 cities in China. Clin Infect Dis. 2013;57(2):298–309.
Ruan Y, Luo F, Jia Y, Li X, Li Q, Liang H, Zhang X, Li D, Shi W, Freeman JM, et al. Risk factors for syphilis and prevalence of HIV, Hepatitis B and C among men who have sex with men in Beijing, China: implications for HIV prevention. AIDS Behav. 2009;13(4):663–70.
Wong NS, Chen L, Tucker JD, Zhao P, Goh BT, Poon CM, Yang L, Yang B, Zheng H, Huang S. Distribution of reported syphilis cases in South China: spatiotemporal analysis. Sci Rep-UK. 2018. https://doi.org/10.1038/s41598-018-27173-y.
Wang C, Tang W, Zhao P, Tucker J, Chen L, Smith MK, Wong NS, Dong W, Yang B, Zheng H. Rapid increase of gonorrhoea cases in Guangdong Province, China, 2014–2017: a review of surveillance data. BMJ Open. 2019;9(11): e31578.
Qin Q, Tang W, Ge L, Li D, Mahapatra T, Wang L, Guo W, Cui Y, Sun J. Changing trend of HIV, syphilis and hepatitis C among men who have sex with men in China. Sci Rep-UK. 2016. https://doi.org/10.1038/srep31081.
Choi NY, Kim EH, Baek CH, Sohn I, Yeon S, Chung MK. Development of a nomogram for predicting the probability of postoperative delirium in patients undergoing free flap reconstruction for head and neck cancer. Eur J Surg Oncol (EJSO). 2017;43(4):683–8.
Lionte C, Sorodoc V, Jaba E, Botezat A. Development and validation of a risk-prediction nomogram for in-hospital mortality in adults poisoned with drugs and nonpharmaceutical agents. Medicine. 2017;96(12): e6404.
Cowley LE, Farewell DM, Maguire S, Kemp AM. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn Progn Res. 2019;3:16.
Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605.
Harrell FE, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Stat Med. 1984;3(2):143–52.
Zhou Y, Li D, Lu D, Ruan Y, Qi X, Gao G. Prevalence of HIV and syphilis infection among men who have sex with men in China: a meta-analysis. Biomed Res Int. 2014;2014:1–12.
Bissio E, Cisneros V, Lopardo GD, Cassetti LI. Very high incidence of syphilis in HIV-infected men who have sex with men in Buenos Aires city: a retrospective cohort study. Sex Transm Infect. 2017;93(5):323–6.
van Aar F, den Daas C, van der Sande MAB, Soetens LC, de Vries HJC, van Benthem BHB. Outbreaks of syphilis among men who have sex with men attending STI clinics between 2007 and 2015 in the Netherlands: a space–time clustering study. Sex Transm Infect. 2017;93(6):390–5.
Fuchs W, Kreuter A, Hellmich M, Potthoff A, Swoboda J, Brockmeyer NH, Wieland U. Asymptomatic anal sexually transmitted infections in HIV-positive men attending anal cancer screening. Br J Dermatol. 2016;174(4):831–8.
Mohammed H, Mitchell H, Sile B, Duffell S, Nardone A, Hughes G. Increase in sexually transmitted infections among men who have sex with men, England, 2014. Emerg Infect Dis. 2016;22(1):88–91.
Holmes KK, Levine R, Weaver M. Effectiveness of condoms in preventing sexually transmitted infections. Bull World Health Organ. 2004;82(6):454–61.
Das A, Li J, Zhong F, Ouyang L, Mahapatra T, Tang W, Fu G, Zhao J, Detels R. Factors associated with HIV and syphilis co-infection among men who have sex with men in seven Chinese cities. Int J STD AIDS. 2015;26(3):145–55.
Yun K, Xu J, Leuba S, Zhu Y, Zhang J, Chu Z, Geng W, Jiang Y, Shang H. Development and validation of a personalized social media platform-based HIV incidence risk assessment tool for men who have sex with men in China. J Med Internet Res. 2019;21(6): e13475.
Solomon MM, Mayer KH. Evolution of the syphilis epidemic among men who have sex with men. Sex Health. 2015;12(2):96.
Zhao P, Tang W, Cheng H, Huang S, Zheng H, Yang B, Wang C. Uptake of provider-initiated HIV and syphilis testing among heterosexual STD clinic patients in Guangdong, China: results from a cross-sectional study. BMJ Open. 2020;10(12): e41503.
We thank all the study participants and staff members at Jiangmen, Yunfu, Foshan and Yangjiang city who contributed.
This work was supported by the National Natural Science Foundation of China (Grant Numbers 82173619 and 81773544), the Science and Technology Planning Project of Guangdong Province (Grant Number 2020B1212030008) and Medical Scientific Research Foundation of Guangdong Province (Grant Number B2021297).
Ethics approval and consent to participate
This study was approved by the Ethical Committee of Dermatology Hospital of Southern Medical University (GDDHLS-20181207). All participants provided written informed consent. The study was conducted in accordance with the Declaration of Helsinki, and also the local and international ethical guidelines.
Consent for publication
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Map of Guangdong Province, China, highlighting the sentinel surveillance cities. Figure S2. Predicted probability curve. Table S1. Quantification of syphilis infection influencing factors.
About this article
Cite this article
Zhao, P., Yang, Z., Li, B. et al. Simple-to-use nomogram for predicting the risk of syphilis among MSM in Guangdong Province: results from a serial cross-sectional study. BMC Infect Dis 21, 1199 (2021). https://doi.org/10.1186/s12879-021-06912-z
- Men who have sex with men