Clinical Practice & Epidemiology in Mental Health




ISSN: 1745-0179 ― Volume 14, 2018
RESEARCH ARTICLE

Quick Identification of the Risk of Psychosis: The Italian Version of the Prodromal Questionnaire-Brief



Antonio Preti1, 2, 3, *, Andrea Raballo4, Georgios D. Kotzalidis5, Rosanna Scanu2, Tamara Muratore2, Mersia Gabbrielli2, Debora Tronci2, Carmelo Masala2, Donatella Rita Petretto2, Mauro G. Carta1
1 Center of Liaison Psychiatry and Psychosomatics, University Hospital, University of Cagliari, Cagliari, Italy
2 Section on Clinical Psychology, Department of Education, Psychology, Philosophy, University of Cagliari, Cagliari, Italy
3 Genneruxi Medical Center, Cagliari, Italy
4 Department of Psychology, Psychopathology and Development Research, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
5 NESMOS Department, Sapienza University, School of Medicine and Psychology, Sant'Andrea Hospital, Rome, Italy

Abstract

Background:

Diagnosing people during the prodromal phase of an incipient psychosis can improve the chance of better outcome. In busy clinical settings, the ideal tool is a brief, easy-to-complete self-report questionnaire.

Objective:

To test the psychometric properties of the Italian version of one of the most used screening tools for the identification of the risk of psychosis, the Prodromal Questionnaire-Brief (PQ-B).

Methods:

Cross-sectional design. A convenience sample of college students was enrolled via snowball procedure (n=243; men: 45%). After understanding and signing the consent form, the participants received a booklet containing the following questionnaires: the 21-item Prodromal Questionnaire-Brief (PQ-B); the 12-item General Health Questionnaire (GHQ-12), and the 74-item Schizotypal Personality Questionnaire (SPQ). Receiver operating characteristic (ROC) analysis was used to assess the capacity of the PQ-B to identify individuals at risk of psychosis as independently defined based on the combination of GHQ-12 and SPQ thresholds.

Results:

The Italian version of the PQ-B revealed good internal consistency, test-retest reliability, and adequate convergent and divergent validity. The Youden method retrieved a cut-off = 7 for the PQ-B frequency score and a cut-off = 22 for the PQ-B distress score. Both PQ-B scores had a perfect (99%) negative predictive value.

Conclusion:

The PQ-B is a promising screening tool in two-stage protocols. The major advantage of the PQ-B is to exclude cases that are unlikely to be at risk of psychosis.

Keywords: PQ, Screening, Schizophrenia, Psychosis, Ultra high risk, Questionnaire, Distress score.


Article Information


Identifiers and Pagination:

Year: 2018
Volume: 14
First Page: 120
Last Page: 131
Publisher Id: CPEMH-14-120
DOI: 10.2174/1745017901814010154

Article History:

Received Date: 16/02/2018
Revision Received Date: 18/04/2018
Acceptance Date: 23/04/2018
Electronic publication date: 23/05/2018
Collection year: 2018

Article Metrics:

CrossRef Citations:
0

Total Statistics:

Full-Text HTML Views: 755
Abstract HTML Views: 627
PDF Downloads: 327
ePub Downloads: 143
Total Views/Downloads: 1852

Unique Statistics:

Full-Text HTML Views: 509
Abstract HTML Views: 225
PDF Downloads: 204
ePub Downloads: 98
Total Views/Downloads: 1036
Geographical View

© 2018 Preti et al.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


* Address correspondence to this author at the Center of Liaison Psychiatry and Psychosomatics, University Hospital, University of Cagliari, via Costantinopoli 54, 09123 Cagliari, Italy; Tel: 0039 070 6093498; E-mail: antolink@yahoo.it




1. INTRODUCTION

The early intervention model in psychiatry posits that detection and treatment of people in their early stage of psychosis can greatly improve the course of the condition [1McGorry PD. Early intervention in psychosis: Obvious, effective, overdue. J Nerv Ment Dis 2015; 203(5): 310-8.[http://dx.doi.org/10.1097/NMD.0000000000000284] [PMID: 25919380] , 2Malla A, Iyer S, McGorry P, et al. From early intervention in psychosis to youth mental health reform: A review of the evolution and transformation of mental health services for young people. Soc Psychiatry Psychiatr Epidemiol 2016; 51(3): 319-26.[http://dx.doi.org/10.1007/s00127-015-1165-4] [PMID: 26687237] ]. There is some evidence that the time spanning from the initial manifestations of symptoms of psychosis, such as hallucination and delusions, and the start of an appropriate treatment with drugs and psychosocial interventions is related to the medium and long-term outcome of a psychosis in the spectrum of schizophrenia [3Penttilä M, Jääskeläinen E, Hirvonen N, Isohanni M, Miettunen J. Duration of untreated psychosis as predictor of long-term outcome in schizophrenia: Systematic review and meta-analysis. Br J Psychiatry 2014; 205(2): 88-94.[http://dx.doi.org/10.1192/bjp.bp.113.127753] [PMID: 25252316] , 4Santesteban-Echarri O, Paino M, Rice S, et al. Predictors of functional recovery in first-episode psychosis: A systematic review and meta-analysis of longitudinal studies. Clin Psychol Rev 2017; 58: 59-75.[http://dx.doi.org/10.1016/j.cpr.2017.09.007] [PMID: 29042139] ]. The longer the duration of untreated psychosis, the poorer the outcome of the psychosis [3Penttilä M, Jääskeläinen E, Hirvonen N, Isohanni M, Miettunen J. Duration of untreated psychosis as predictor of long-term outcome in schizophrenia: Systematic review and meta-analysis. Br J Psychiatry 2014; 205(2): 88-94.[http://dx.doi.org/10.1192/bjp.bp.113.127753] [PMID: 25252316] , 4Santesteban-Echarri O, Paino M, Rice S, et al. Predictors of functional recovery in first-episode psychosis: A systematic review and meta-analysis of longitudinal studies. Clin Psychol Rev 2017; 58: 59-75.[http://dx.doi.org/10.1016/j.cpr.2017.09.007] [PMID: 29042139] ].

Full-blown psychotic episodes are often preceded by a period of progressive impairment in social functioning, associated to nonspecific affective symptoms and attenuated manifestations of the symptoms that characterize the positive (hallucination- and delusion-like experiences) and negative (blunting anhedonia and apathy, autism-like social withdrawal) dimensions of the schizophrenia-spectrum psychoses [5Sullivan HS. The onset of schizophrenia. 1927. Am J Psychiatry 1994; 151(6)(Suppl.): 134-9.[http://dx.doi.org/10.1176/ajp.151.6.134] [PMID: 8192188] , 6Modinos G, McGuire P. The prodromal phase of psychosis. Curr Opin Neurobiol 2015; 30: 100-5.[http://dx.doi.org/10.1016/j.conb.2014.11.003] [PMID: 25464375] ].

Diagnosing people during the prodromal phase of an incipient psychosis can improve its outcome. For this reason, several tools have been developed to allow the early detection of people with high at-risk mental states (HARMS) for psychosis, in order to increase their early access to treatment [7Kline E, Schiffman J. Psychosis risk screening: A systematic review. Schizophr Res 2014; 158(1-3): 11-8.[http://dx.doi.org/10.1016/j.schres.2014.06.036] [PMID: 25034762] , 8Addington J, Stowkowy J, Weiser M. Screening tools for clinical high risk for psychosis. Early Interv Psychiatry 2015; 9(5): 345-56.[http://dx.doi.org/10.1111/eip.12193] [PMID: 25345316] ]. A two-stage model is usually adopted with self-report screening followed by a clinical interview, this procedure has been proven accurate and efficient in other public health-care settings [9Pickles A, Dunn G, Vázquez-Barquero JL. Screening for stratification in two-phase (‘two-stage’) epidemiological surveys. Stat Methods Med Res 1995; 4(1): 73-89.[http://dx.doi.org/10.1177/096228029500400106] [PMID: 7613639] ].

The Prodromal Questionnaire (PQ) is the most used tool for the initial screening of potential HARMS cases [10Savill M, D’Ambrosio J, Cannon TD, Loewy RL. Psychosis risk screening in different populations using the Prodromal Questionnaire: A systematic review. Early Interv Psychiatry 2018; 12(1): 3-14.[http://dx.doi.org/10.1111/eip.12446] [PMID: 28782283] ]. The initial version of the PQ included 92 items [11Loewy RL, Bearden CE, Johnson JK, Raine A, Cannon TD. The prodromal questionnaire (PQ): Preliminary validation of a self-report screening measure for prodromal and psychotic syndromes. Schizophr Res 2005; 79(1): 117-25.[http://dx.doi.org/10.1016/j.schres.2005.03.007] [PMID: 16276559] , 12Kotzalidis GD, Solfanelli A, Piacentino D, et al. The Italian version of the 92-item Prodromal Questionnaire: Concurrent validity with the SIPS and factor analysis in a sample of 258 outpatients aged 11-36years. Schizophr Res 2017; 189: 50-6.[http://dx.doi.org/10.1016/j.schres.2017.02.008] [PMID: 28254200] ]. Although, helpful for epidemiological purposes [13Loewy RL, Johnson JK, Cannon TD. Self-report of attenuated psychotic experiences in a college population. Schizophr Res 2007; 93(1-3): 144-51.[http://dx.doi.org/10.1016/j.schres.2007.02.010] [PMID: 17459662] , 14Loewy RL, Therman S, Manninen M, Huttunen MO, Cannon TD. Prodromal psychosis screening in adolescent psychiatry clinics. Early Interv Psychiatry 2012; 6(1): 69-75.[http://dx.doi.org/10.1111/j.1751-7893.2011.00286.x] [PMID: 21883972] ], such a long version of the PQ may result cumbersome in the busy clinical setting. Therefore, two shorter versions have been developed: the Prodromal Questionnaire – brief version (PQ-B) [15Loewy RL, Pearson R, Vinogradov S, Bearden CE, Cannon TD. Psychosis risk screening with the Prodromal Questionnaire-brief version (PQ-B). Schizophr Res 2011; 129(1): 42-6.[http://dx.doi.org/10.1016/j.schres.2011.03.029] [PMID: 21511440] ], including 21 items, and a 16-item version of the Prodromal Questionnaire, the PQ-16 [16Ising HK, Veling W, Loewy RL, et al. The validity of the 16-item version of the Prodromal Questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophr Bull 2012; 38(6): 1288-96.[http://dx.doi.org/10.1093/schbul/sbs068] [PMID: 22516147] ]. There is evidence that all versions of the PQ can reliably identify people at high risk of psychosis [10Savill M, D’Ambrosio J, Cannon TD, Loewy RL. Psychosis risk screening in different populations using the Prodromal Questionnaire: A systematic review. Early Interv Psychiatry 2018; 12(1): 3-14.[http://dx.doi.org/10.1111/eip.12446] [PMID: 28782283] ]. The PQ-B, in particular, showed good convergent and discriminant validity and test-retest reliability in US samples [17Kline E, Wilson C, Ereshefsky S, et al. Convergent and discriminant validity of attenuated psychosis screening tools. Schizophr Res 2012; 134(1): 49-53.[http://dx.doi.org/10.1016/j.schres.2011.10.001] [PMID: 22036199] ], excellent sensitivity to emerging psychosis, and strong agreement with clinician evaluations of attenuated psychosis symptoms in longitudinal studies [18Kline E, Thompson E, Demro C, Bussell K, Reeves G, Schiffman J. Longitudinal validation of psychosis risk screening tools. Schizophr Res 2015; 165(2-3): 116-22.[http://dx.doi.org/10.1016/j.schres.2015.04.026] [PMID: 25975827] , 19Kline E, Thompson E, Demro C, Bussell K, Reeves G, Schiffman J. Self-report instruments for clinical monitoring of psychosis risk states. Psychiatr Serv 2016; 67(4): 456-9.[http://dx.doi.org/10.1176/appi.ps.201500063] [PMID: 26567937] ]. The PQ-B also provides measurement invariance across different ethnic groups, as demonstrated in multiethnic samples in the US [20Cicero DC, Krieg A, Martin EA. Measurement invariance of the prodromal questionnaire-brief among white, asian, hispanic, and multiracial populations. Assessment 2017; 1073191116687391: 1073191116687391. Epub ahead of print[http://dx.doi.org/10.1177/1073191116687391] [PMID: 28092988] ]. Generally, higher cut-offs were required to identify cases at risk of psychosis in non-help-seeking samples than in general help-seeking populations, or in samples highly enriched with ultra high-risk participants [10Savill M, D’Ambrosio J, Cannon TD, Loewy RL. Psychosis risk screening in different populations using the Prodromal Questionnaire: A systematic review. Early Interv Psychiatry 2018; 12(1): 3-14.[http://dx.doi.org/10.1111/eip.12446] [PMID: 28782283] ].

The PQ and its shorter versions have been translated in various countries, including China [21Zhang T, Li H, Woodberry KA, et al. Prodromal psychosis detection in a counseling center population in China: An epidemiological and clinical study. Schizophr Res 2014; 152(2-3): 391-9.[http://dx.doi.org/10.1016/j.schres.2013.11.039] [PMID: 24387999] ], Spain [22Fonseca-Pedrero E, Gooding DC, Ortuño-Sierra J, Paino M. Assessing self-reported clinical high risk symptoms in community-derived adolescents: A psychometric evaluation of the Prodromal Questionnaire-Brief. Compr Psychiatry 2016; 66: 201-8.[http://dx.doi.org/10.1016/j.comppsych.2016.01.013] [PMID: 26995254] ], Nigeria [23Okewole AO, Ajogbon D, Adeniji AA, et al. Psychosis risk screening among secondary school students in Abeokuta, Nigeria: Validity of the Prodromal Questionnaire - Brief Version (PQ-B). Schizophr Res 2015; 164(1-3): 281-2.[http://dx.doi.org/10.1016/j.schres.2015.01.006] [PMID: 25640525] ], and Brazil [24Gonçalves PD, Martins PA, Gordon P, et al. Prodromal Questionnaire: Translation, adaptation to Portuguese and preliminary results in ultra-high risk individuals and first episode psychosis. J Bras Psiquiatr 2012; 61(2): 96-101.[http://dx.doi.org/10.1590/S0047-20852012000200007] ]. The Italian version of the PQ-92 has been validated in a sample of 258 outpatients aged 11-36 years, who applied to the psychiatric mental health services in a large semi-rural area outside Rome [12Kotzalidis GD, Solfanelli A, Piacentino D, et al. The Italian version of the 92-item Prodromal Questionnaire: Concurrent validity with the SIPS and factor analysis in a sample of 258 outpatients aged 11-36years. Schizophr Res 2017; 189: 50-6.[http://dx.doi.org/10.1016/j.schres.2017.02.008] [PMID: 28254200] ]. No Italian version of the PQ has been evaluated in non help-seeking populations, which are the primary target of epidemiological surveys on the prevalence of psychosis-related symptoms and conditions.

This study illustrates the psychometric properties of the PQ-B in a sample of college students, a population at a higher risk of mental distress particularly during the first years due to the new pressures in the academic setting [25Hope V, Henderson M. Medical student depression, anxiety and distress outside North America: A systematic review. Med Educ 2014; 48(10): 963-79.[http://dx.doi.org/10.1111/medu.12512] [PMID: 25200017] , 26Griggs S. Hope and Mental Health in Young Adult College Students: An Integrative Review. J Psychosoc Nurs Ment Health Serv 2017; 55(2): 28-35.[http://dx.doi.org/10.3928/02793695-20170210-04] [PMID: 28218927] ].

2. METHODS

The institutional review board approved the study protocol in accordance with the guidelines of the 1995 Declaration of Helsinki, as revised in Tokyo in 2004, and further revised in Fortaleza, Brazil, in 2013 [27World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA 2013; 310(20): 2191-4.[http://dx.doi.org/10.1001/jama.2013.281053] [PMID: 24141714] ].

2.1. Participants

Young adults attending a university college in Center-South Italy were invited to take part in the study. The undergraduate sample was enrolled via a snowball procedure. Recruiters asked initial participants to take part in a study by completing a booklet and recruiting further participants among their colleagues, who were requested, in turn, to enroll other subjects, and so on. This method is designed to recruit a variegated array of individuals and avoid the bias of self-selection that occurs when recruiters choose from their personal social network only [28Snijders T. Estimation on the basis of snowball samples: How to weight. Bull Methodol Sociol 1992; 36: 59-70.[http://dx.doi.org/10.1177/075910639203600104] ]. Anonymity was ensured.

We targeted a minimum sample size of 210 participants, 10 per each PQ-B item to assure adequate variance across the items. We increased the sample size to 300 to account for potential refusal. Out of 31,729 potential candidates among those attending university during the period of the study, 327 people were contacted: 26 declined after having a look at the booklet; 301 people accepted to fill in the questionnaire; 256 participants actually returned the booklet; 13 cases were rejected because their questionnaires were left blank in some essential parts (data on age or gender, or some items in two or more questionnaires); 243 participants were included in the study out of the 301 people who had accepted to participate (81%), and out of the 327 people who had been asked to take part in the study (74% overall participation rate). Because of the enrollment method (snowball procedure), we were unable to control from which faculty the university students involved in the study come from.

Participation was voluntary and no fee or other compensation was given for taking part in the study. All participants provided informed consent.

2.2. Measures

After having understood and signed the consent form, participants received a booklet containing the following questionnaires: the 21-item Prodromal Questionnaire-Brief (PQ-B); The 12-item General Health Questionnaire (GHQ-12), and the 74-item Schizotypal Personality Questionnaire (SPQ).

The PQ-B is a yes/no 21-item self-report questionnaire recording the positive symptoms experienced over the past month. For each endorsed symptom, responders rate whether they found it distressing or impairing, ranging from 1 (strongly disagree) to 5 (strongly agree), with 4 or 5 indicating distress. As a screening tool, the PQ-B is rated by using the total number of endorsed items (range: 0–21), the number of items that are identified as distressing (range: 0–21), and the total distress score (range: 0–105), which is the method recommended by Loewy and colleagues [15Loewy RL, Pearson R, Vinogradov S, Bearden CE, Cannon TD. Psychosis risk screening with the Prodromal Questionnaire-brief version (PQ-B). Schizophr Res 2011; 129(1): 42-6.[http://dx.doi.org/10.1016/j.schres.2011.03.029] [PMID: 21511440] ].

Standard procedures of translation and back-translation were used to translate the Italian version of the PQ-B from the original English version [29Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000; 25(24): 3186-91.[http://dx.doi.org/10.1097/00007632-200012150-00014] [PMID: 11124735] ]. Translation accuracy was confirmed by an English-speaking translator and optimized with the help of the authors of the PQ and of the Italian translator of the PQ-92 (see Appendix for the Italian PQ-B).

The GHQ-12 is a screening tool aimed at identifying people in need of clinical attention [30Goldberg DP. The detection of psychiatric illness by questionnaire. London: Oxford University Press 1992.]. The validated Italian version of the GHQ-12 was used in the study [31Politi PL, Piccinelli M, Wilkinson G. Reliability, validity and factor structure of the 12-item General Health Questionnaire among young males in Italy. Acta Psychiatr Scand 1994; 90(6): 432-7.[http://dx.doi.org/10.1111/j.1600-0447.1994.tb01620.x] [PMID: 7892776] ]. According to past studies, scores equal or above 4 on the GHQ-12 were considered indicative of clinically relevant psychological distress (i.e., needing clinical attention [31Politi PL, Piccinelli M, Wilkinson G. Reliability, validity and factor structure of the 12-item General Health Questionnaire among young males in Italy. Acta Psychiatr Scand 1994; 90(6): 432-7.[http://dx.doi.org/10.1111/j.1600-0447.1994.tb01620.x] [PMID: 7892776] ]) However, patients with psychosis tend to score higher on the GHQ-12 than patients with anxiety or depressive disorders, and scores equal or above 6 better differentiate people diagnosed with psychosis from healthy people than the usual threshold of 4 [32Preti A, Rocchi MB, Sisti D, et al. The psychometric discriminative properties of the Peters et al Delusions Inventory: A receiver operating characteristic curve analysis. Compr Psychiatry 2007; 48(1): 62-9.[http://dx.doi.org/10.1016/j.comppsych.2006.05.003] [PMID: 17145284] ]. Cronbach's alpha of 0.81 was found for the Italian validation study of the GHQ-12 [31Politi PL, Piccinelli M, Wilkinson G. Reliability, validity and factor structure of the 12-item General Health Questionnaire among young males in Italy. Acta Psychiatr Scand 1994; 90(6): 432-7.[http://dx.doi.org/10.1111/j.1600-0447.1994.tb01620.x] [PMID: 7892776] ]. The GHQ-12 has been translated and tested in a wide range of cultures and has proved itself a valid screening tool for common mental disorders in both Western and non-Western countries [33Ali GC, Ryan G, De Silva MJ. Validated screening tools for common mental disorders in low and middle income countries: A systematic review. PLoS One 2016; 11(6): e0156939.[http://dx.doi.org/10.1371/journal.pone.0156939] [PMID: 27310297] ].

The SPQ is a 74-item self-report with a true/false format [34Raine A. The SPQ: A scale for the assessment of schizotypal personality based on DSM-III-R criteria. Schizophr Bull 1991; 17(4): 555-64.[http://dx.doi.org/10.1093/schbul/17.4.555] [PMID: 1805349] ], which was developed to assess schizotypal personality disorder according to the Diagnostic and Statistical Manual of Mental Disorders-Revised, Third Edition (DSM-III-R) [35American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 3rd Edition-Revised (DSM-III-R). Washington, DC: American Psychiatric Association; 1987.]. The Italian version of the SPQ was used in the study [36Fossati A, Raine A, Carretta I, et al. The three-factor model of schizotypal personality: Invariance across age and gender. Pers Individ Dif 2003; 35: 1007-19.[http://dx.doi.org/10.1016/S0191-8869(02)00314-8] ]. There is general agreement that the SPQ measures a multimensional construct, including a cognitive-perceptual deficits domain (ideas of reference, odd beliefs or magical thinking, unusual perceptual experiences, and suspiciousness subscales); an interpersonal deficits domain (excessive social anxiety, no close friends, constricted affect, and suspiciousness subscales); and a disorganized domain (odd or eccentric behavior and odd speech subscales) [37Raine A, Reynolds C, Lencz T, Scerbo A, Triphon N, Kim D. Cognitive-perceptual, interpersonal, and disorganized features of schizotypal personality. Schizophr Bull 1994; 20(1): 191-201.[http://dx.doi.org/10.1093/schbul/20.1.191] [PMID: 8197415] , 38Reynolds CA, Raine A, Mellingen K, Venables PH, Mednick SA. Three-factor model of schizotypal personality: Invariance across culture, gender, religious affiliation, family adversity, and psychopathology. Schizophr Bull 2000; 26(3): 603-18.[http://dx.doi.org/10.1093/oxfordjournals.schbul.a033481] [PMID: 10993401] ]. The reproducibility of the first-order, nine-subscale structure of the SPQ and of its second-order domains has been demonstrated [39Preti A, Siddi S, Vellante M, et al. Bifactor structure of the schizotypal personality questionnaire (SPQ). Psychiatry Res 2015; 230(3): 940-50.[http://dx.doi.org/10.1016/j.psychres.2015.11.010] [PMID: 26607431] ]. The SPQ has been translated into many languages, and there is wide evidence of its cross-national reliability and structural validity [40Fonseca-Pedrero E, Debbané M, Ortuño-Sierra J, et al. The structure of schizotypal personality traits: A cross-national study. Psychol Med 2018; 48(3): 451-62.[http://dx.doi.org/10.1017/S0033291717001829] [PMID: 28712364] ].

General socio-demographic information from self-report data was collected for the following variables: age, gender and socioeconomic status. To measure socioeconomic status we used the highest level of parental education [41Galobardes B, Shaw M, Lawlor DA, Lynch JW, Davey Smith G. Indicators of socioeconomic position (part 1). J Epidemiol Community Health 2006; 60(1): 7-12.[http://dx.doi.org/10.1136/jech.2004.023531] [PMID: 16361448] ], further subdivided into three categories, i.e., lower than high school, high school diploma, college graduate or higher.

2.3. Statistics

In the database, there were no missing data, since any questionnaire lacking data in the essential part of the booklet was excluded (n=13). An independent research assistant rechecked the data after they were entered: Error rates were less than 1% and all were corrected following the questionnaires.

All data were coded and analyzed using the Statistical Package for Social Sciences (SPSS) version 20. Additional analyses were carried out in R [42R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.Rproject.org/ 2017].

All tests were two-tailed, with alpha set at p < 0.05.

Means with standard deviations were reported for continuous variables. Counts and percentages were reported for categorical variables. Parametric or non-parametric tests, as appropriate, were used to compare continuous variables between groups. Chi-square tests or Fisher's exact tests were used to analyze categorical data. Correlation coefficients were compared according to Steiger’s Z-test [43Steiger JH. Tests for comparing elements of a correlation matrix. Psychol Bull 1980; 87: 245-51.[http://dx.doi.org/10.1037/0033-2909.87.2.245] ].

Scales reliability was measured by Cronbach’s alpha. For group comparisons, reliability values of 0.70 are considered quite satisfactory, and when dealing with subscales derived from a single questionnaire, values around 0.60 are considered acceptable [44Nunnally JC. Psychometric Theory, 2nd ed. New York, NY: McGraw-Hill 1978.].

Test-retest reliability of the PQ-B was evaluated in a subgroup of 120 participants, who were invited to complete the PB-Q again after 30 days. Participants included in the test-retest assessment were randomly recruited among those students whose university registration number was an odd number until reaching the quota of n=120.

Follow-up completion rate was 95% for the test-retest reliability sample (6 participants only did not return the booklet). Test-retest stability was assessed with the intraclass correlation coefficient (ICC), with 95% Confidence Interval (CI). The ICC is dimensionless statistics describing the reproducibility of repeated measurements in the same population: ICC values ≥ 0.60 are considered as acceptable for clinical use [45Brennan P, Silman A. Statistical methods for assessing observer variability in clinical measures. BMJ 1992; 304(6840): 1491-4.[http://dx.doi.org/10.1136/bmj.304.6840.1491] [PMID: 1611375] ].

To assess agreement at retest for the PQ-B frequency score, we used the Bland and Altman [46Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1(8476): 307-10.[http://dx.doi.org/10.1016/S0140-6736(86)90837-8] [PMID: 2868172] ] method (the PQ-B distress score depends on frequency scores, hence reproducing its Bland-Altman plot is unnecessary). The Bland-Altman plot visualizes the agreement between the scores of a test measured at two different assessment points by plotting the difference between test- and retest-scores against the mean of test- and retest-scores for each participant. Confidence intervals for the mean difference are calculated to determine if the latter deviates significantly from zero, which should not be. The plot draws the upper and lower limits of agreement, indicating the range within which 95% of the test scores in the two assessments can be expected to vary.

According to Raine [34Raine A. The SPQ: A scale for the assessment of schizotypal personality based on DSM-III-R criteria. Schizophr Bull 1991; 17(4): 555-64.[http://dx.doi.org/10.1093/schbul/17.4.555] [PMID: 1805349] ], no more than a half of those scoring in the top 10 percent of SPQ would receive a diagnosis of schizotypal personality disorder. We assumed that those scoring in the top 10 percent of SPQ were more likely to have a schizotypal personality disorder and to be at risk of psychosis as well when they also manifested intense psychological distress. Subjects were identified as being High at-Risk Mental States (HARMS) when they scored above the cut-off ≥ 6 on the GHQ-12 [32Preti A, Rocchi MB, Sisti D, et al. The psychometric discriminative properties of the Peters et al Delusions Inventory: A receiver operating characteristic curve analysis. Compr Psychiatry 2007; 48(1): 62-9.[http://dx.doi.org/10.1016/j.comppsych.2006.05.003] [PMID: 17145284] ] and scored in the top tenth percentile on the SPQ [34Raine A. The SPQ: A scale for the assessment of schizotypal personality based on DSM-III-R criteria. Schizophr Bull 1991; 17(4): 555-64.[http://dx.doi.org/10.1093/schbul/17.4.555] [PMID: 1805349] ].

Receiver Operating Characteristic (ROC) analysis was used to assess the capacity of the PQ-B in identifying HARMS individuals as independently defined according to the combination of GHQ-12 and SPQ thresholds. Optimal cut-off points for frequency and distress scores were established according to the Youden method, using the Optimal Cutpoints package running in R [47Lopez-Raton M, Rodriguez-Alvarez MX, Cadarso-Suarez C, et al. Optimal cutpoints: An R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw 2014; 61(8): 1-36.http://www.jstatsoft.org/v61/i08/[http://dx.doi.org/10.18637/jss.v061.i08] ]. ROC analysis was based on a logistic regression. The fit of the models was assessed with le Cessie - van Houwelingen - Copas - Hosmer unweighted sum of squares test [48le Cessie S, van Houwelingen JC. A goodness-of-fit test for binary regression models, based on smoothing methods. Biometrics 1991; 47: 1267.[http://dx.doi.org/10.2307/2532385] ], and Tukey-Pregibon test [49Pregibon D. Goodness of link tests for generalized linear models. J R Stat Soc Ser C Appl Stat 1980; 29: 14-5.]. In these tests, the null hypothesis assumes that the model has a good fit, thus p < 0.05 (rejection of the null hypothesis) indicates misspecification of the model. The McFadden [pseudo]R2 and the adjusted McFadden [pseudo]R2 were used as a measure of explained variance, with values from 0.20 to 0.40 indicating good model fit [50McFadden D. Conditional logit analysis of qualitative choice behavior. In: Zarembka P, Ed. Frontiers in econometrics. New York, NY: Academic Press 1994: pp. 104-142.]. Adjusted Odds Ratio (OR), with 95% confidence interval (95%CI), and estimated Wald test’s p were reported for each predictor in addition to the Area under the Curve (AUC). AUC threshold are: 0.80 to 0.90, good; 0.70 to 0.80, fair; <0.70, poor. ROC analysis was conducted with the pROC package running in R [51Robin X, Turck N, Hainard A, et al. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 2011; 12: 77.[http://dx.doi.org/10.1186/1471-2105-12-77] [PMID: 21414208] ].

3. RESULTS

The sample included 109 participants who identified themselves as men and 134 participants who identified themselves as women (Table 1). The age range was 19 to 34 years old, with mean age of 24.3 years (SD, 3.5; median, 24), with no difference by gender. No differences were observed in the distribution of PQ-B scores by age or socio-economic status, and a barely significant difference was found for the PQ-B distress score by gender, negligible in terms of effect size (Hedges’ g: -0.17; 95%CI: -0.42 to 0.09).

Table 1
General characteristics of the sample (n = 243).


3.1. Internal Consistency and Test-Retest Reliability

Internal coherence, as measured by Cronbach’s alpha, was optimal for both the frequency and the distress scores of the PQ-B. Internal coherence was good to acceptable for the other scales and subscales that were used in the study (Table 2).

Table 2
Mean scores on the measures of psychopathology used in the study, and inter-correlation among them and the PQ-B in the sample (n = 243).


Test-retest reliability for the PQ-B frequency scores, as measured by ICC, was 0.89 (95%CI = 0.86 to 0.92), and was 0.89 (0.87 to 0.91) for the PQ-B distress score.

By plotting the differences and the means of the two assessments in the Bland-Altman plot, 7 cases only out of 114 were outside the upper and lower limits of agreement (Fig. 1).

3.2. Distribution of Scores on the PQ-B

Endorsement of items varied depending on the experience. Items concerning mistrustfulness and suspiciousness or unusual beliefs were endorsed by a large majority of participants, with only a minority endorsing items pertaining to visual hallucinations (Fig. 2).

Participants endorsed an average of 5 positive psychotic experiences. The mean for the PQ-B distress score in the sample was 14. Only a minority of participants agreed that the experience was distressing (item rated 4 or 5). 42 participants (17.3%) rated just one experience as distressing; 33 participants (13.6%) rated two experiences as distressing, the remaining participants (n=61, 25.1%) rated three or more experiences as distressing. The experiences that were more often rated as distressing were those described in items 18 (mistrustfulness or suspiciousness of other people, 31%), 12 (worry that something is wrong with one’s own mind, 16%), and 21 (people sometimes finding it hard to understand what the subject is saying, 16%). The probability of rating a psychotic experience as distressing was related to the frequency of endorsement, but did not coincide with it; Spearman’s rho was 0.699, p<0.0001 (Fig. 3).

Fig. (1)
Bland-Altman plot of the PQ-B frequency score at test-retest assessment (n=114). The horizontal axis shows the mean of the two measurements ([test + retest]/2), while the vertical axis reports the absolute difference between the test and the retest (retest – test). Most values are expected to be within the 95% limit of agreement (average difference ± 1.96 standard deviation of the difference).


Fig. (2)
Distribution of scores by item for the Italian version of the Prodromal Questionnaire-Brief (PQ-B) in the sample (n=243). The items are numbered as they appear in the questionnaire, and are ranked according to the frequency of positive endorsement (response: yes).


Fig. (3)
PQ-B distress score = 4 / 5 (positive psychotic experiences rated as distressful) and correlation with the PQ-B frequency score.


3.3. Convergent and Divergent Validity of the PQ-B

Both the frequency and the distress scores of the PQ-B were related to psychological distress as measured by the GHQ-12. The PQ-B was positively correlated to SPQ subscales, but showed stronger links with the cognitive-perceptual deficits and disorganization domains than with the interpersonal deficits domains, or the measure of general psychological distress (Table 2).

3.4. Predictive Ability of the PQ-B

In the sample, 59 participants (24.3%) scored ≥ 6 on the GHQ-12; 25 (10.3%) scored in the top tenth percentile on the SPQ, and 10 (4.1%) were identified as HARMS.

Both scores of the PQ-B were able to detect HARMS cases, with high accuracy (Table 3).

Table 3
Receiver operating characteristic (ROC) analysis of the links between PQ-B and being a carrier of high at-risk mental states (HARMS) according to GHQ-12 and SPQ thresholds.


The fit of the model in both analyses was optimal, with McFadden [pseudo]R2 > 0.20.

The PQ-B distress score had a better AUC than the PQ-B frequency score, with a small but statistically significant advantage (Fig. 4).

Fig. (4)
Comparison of the ROC curves between the PQ-B frequency score and the PQ-B distress score against the target, i.e., high at-risk mental states (HARMS) cases as defined on the basis of the validated GHQ-12 and SPQ thresholds. The statistical significance of the difference between the areas under the two dependent ROC curves (derived from the same cases) with the method of DeLong et al. [58DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 1988; 44(3): 837-45.[http://dx.doi.org/10.2307/2531595] [PMID: 3203132] ] is reported.


The Youden method retrieved a cut-off = 7 for the PQ-B frequency score and a cut-off = 22 for the PQ-B distress score. Both the PQ-B frequency and distress scores had a very high negative predictive value (99%).

4. DISCUSSION

The Italian version of the PQ-B revealed good internal consistency, test-retest reliability, and adequate convergent and divergent validity. The predictive capacity of the tool is promising, and as in past studies, the PQ-B was able to detect people with HARMS with high accuracy, precision and performance (AUC close to 0.90).

It should be noted that the validity criterion that was used in this study is very conservative. The findings of this study should be intended preliminary as far as the predictive capacity of the tool is considered, since our criterion for HARMS case was entirely based on self-report tools. Findings of the present investigation need to be corroborated by further studies using a standardized interview as the gold standard. Nevertheless, the optimal threshold scores for the Italian PQ-B in this study were very close to the threshold suggested by Savill et al, [10Savill M, D’Ambrosio J, Cannon TD, Loewy RL. Psychosis risk screening in different populations using the Prodromal Questionnaire: A systematic review. Early Interv Psychiatry 2018; 12(1): 3-14.[http://dx.doi.org/10.1111/eip.12446] [PMID: 28782283] ] for general or mental health services in their comprehensive review of the studies where PQ was used as a screening instrument.

Noteworthy in this study is that positive psychotic experiences that were rated as distressful did correlate with their frequency of occurrence. Essentially, the more positive psychotic experiences the candidate reported, the more likely s/he had been distressed by these experiences. Thus, the frequency score of the PQ-B can be a reasonable summary score of the tool. Nevertheless, the distress score, as hypothesized by the authors of the instrument, is more accurate in detecting HARMS cases than the mere sum of the occurrence of the experiences (frequency score).

In this study, as in past studies, the major advantage of the PQ-B is to exclude cases that are unlikely to be at risk of psychosis rather than detecting potential cases at risk for further evaluation with a standardized interview. In a longitudinal study in the US, Kline et al. [18Kline E, Thompson E, Demro C, Bussell K, Reeves G, Schiffman J. Longitudinal validation of psychosis risk screening tools. Schizophr Res 2015; 165(2-3): 116-22.[http://dx.doi.org/10.1016/j.schres.2015.04.026] [PMID: 25975827] ] concluded that an individual scoring below the recommended threshold score would be extremely unlikely to develop psychosis in the short or medium term. Albeit helpful, a “negative” screening tool requires a large-scale usage to produce a public health impact. To date, the PQ, in its various versions, has been used systematically to screen young adults seeking help from mental health services only in the Netherlands [52Rietdijk J, Klaassen R, Ising H, et al. Detection of people at risk of developing a first psychosis: Comparison of two recruitment strategies. Acta Psychiatr Scand 2012; 126(1): 21-30.[http://dx.doi.org/10.1111/j.1600-0447.2012.01839.x] [PMID: 22335365] ].

Despite these limitations, at the moment the PQ-B is the best screening tool for investigating the risk of psychosis in a two-stage scenario, for both epidemiological and clinical studies, and monitoring people with HARMS who are already in treatment [10Savill M, D’Ambrosio J, Cannon TD, Loewy RL. Psychosis risk screening in different populations using the Prodromal Questionnaire: A systematic review. Early Interv Psychiatry 2018; 12(1): 3-14.[http://dx.doi.org/10.1111/eip.12446] [PMID: 28782283] , 53Kline E, Wilson C, Ereshefsky S, et al. Psychosis risk screening in youth: A validation study of three self-report measures of attenuated psychosis symptoms. Schizophr Res 2012; 141(1): 72-7.[http://dx.doi.org/10.1016/j.schres.2012.07.022] [PMID: 22921375] ].

Due to its low positive predictive value, the better use of the PQ-B is probably in samples with a higher proportion of at-risk people (e.g., help-seeking people, patients’ relative).

4.1. Strengths and Limitations of the Study

The assessment was conducted through self-report tools, which might have introduced some bias in responding, including the one related to social desirability. On the other hand, self-report measures allow the enrollment of large samples, and the guarantee of anonymity might have made participants more forthcoming when filling in the questionnaires. Unfortunately, we had not the opportunity to conduct a follow-up in order to further evaluate the people identified as being at potential (psychometric) risk with a dedicated interview, such as the Comprehensive Assessment for At Risk Mental States [CAARMS] [54Yung AR, Yuen HP, McGorry PD, et al. Mapping the onset of psychosis: The comprehensive assessment of at-risk mental states. Aust N Z J Psychiatry 2005; 39(11-12): 964-71.[http://dx.doi.org/10.1080/j.1440-1614.2005.01714.x] [PMID: 16343296] ], the Structured Interview for Prodromal Syndromes/Scale of Prodromal Symptoms [SIPS/SOPS] [55McGlashan TH, Walsh BC, Woods SW. The psychosis-risk syndrome, handbook for diagnosis and follow-up. New York: Oxford University Press; 2010.], or the Structured Interview for Prodromal Schizophrenia Proneness Instrument - Adult [SPI-A] [56Schultze-Lutter F, Addington J, Ruhrmann S, Klosterkötter J. Schizophrenia Proneness Instrument, Adult Version (SPI-A). Italian edition edited by FM Ferro and G Aiello. Rome: Giovanni Fioriti Editore; 2007.]. The latter has shown better long-run ability to predict conversion to psychosis than the other structured interviews [57Schultze-Lutter F, Michel C, Schmidt SJ, et al. EPA guidance on the early detection of clinical high risk states of psychoses. Eur Psychiatry 2015; 30(3): 405-16.[http://dx.doi.org/10.1016/j.eurpsy.2015.01.010] [PMID: 25735810] ]. It should be noted that participants were undergraduates still attending university courses, and were therefore unlikely to have a full-blown episode of psychosis at the time of the study. However, since the participants were university/college students, the results cannot be immediately generalizable to the 19-34 year-old general population. We were unable to control which faculty/school did the university students involved in the study come from; thus, we cannot exclude that there has been some sort of sampling bias, for example, more students deriving from humanistic courses than from scientific ones. However, recruiters were specifically instructed to avoid sampling from psychology courses, to make sure that the knowledge of the topic did not introduce a bias in responding. Finally, we cannot exclude some sort of self-selection bias, inasmuch as those students who were more interested in the topic may have been also more prone to agree to participate in the study.

CONCLUSION

This study confirms the promising psychometric properties of the PQ-B as a screening tool in two-stage protocols. The authors believe that its use could be helpful for both clinical and epidemiological purposes. For this reason, we have made available to the Italian clinicians and researchers the Italian version of the PQ-B and of the PQ-16 (Appendices A and B). Future studies could hopefully focus on comparing the two instruments to identify which is the most sensitive in excluding noncases.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

The Institutional Review Board approved the study protocol.

HUMAN AND ANIMAL RIGHTS

No Animals were used. All the reported experiments involving humans were in accordance with the ethical standards of the committee responsible for human experimentation (institutional and national), and with the Helsinki Declaration of 1995, as revised in Tokyo in 2004, and further revised in Fortaleza, Brazil, in 2013.

CONSENT FOR PUBLICATION

All participants provided informed consent.

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

ACKNOWLEDGEMENTS

Decleared none.

REFERENCES

[1] McGorry PD. Early intervention in psychosis: Obvious, effective, overdue. J Nerv Ment Dis 2015; 203(5): 310-8.[http://dx.doi.org/10.1097/NMD.0000000000000284] [PMID: 25919380]
[2] Malla A, Iyer S, McGorry P, et al. From early intervention in psychosis to youth mental health reform: A review of the evolution and transformation of mental health services for young people. Soc Psychiatry Psychiatr Epidemiol 2016; 51(3): 319-26.[http://dx.doi.org/10.1007/s00127-015-1165-4] [PMID: 26687237]
[3] Penttilä M, Jääskeläinen E, Hirvonen N, Isohanni M, Miettunen J. Duration of untreated psychosis as predictor of long-term outcome in schizophrenia: Systematic review and meta-analysis. Br J Psychiatry 2014; 205(2): 88-94.[http://dx.doi.org/10.1192/bjp.bp.113.127753] [PMID: 25252316]
[4] Santesteban-Echarri O, Paino M, Rice S, et al. Predictors of functional recovery in first-episode psychosis: A systematic review and meta-analysis of longitudinal studies. Clin Psychol Rev 2017; 58: 59-75.[http://dx.doi.org/10.1016/j.cpr.2017.09.007] [PMID: 29042139]
[5] Sullivan HS. The onset of schizophrenia. 1927. Am J Psychiatry 1994; 151(6)(Suppl.): 134-9.[http://dx.doi.org/10.1176/ajp.151.6.134] [PMID: 8192188]
[6] Modinos G, McGuire P. The prodromal phase of psychosis. Curr Opin Neurobiol 2015; 30: 100-5.[http://dx.doi.org/10.1016/j.conb.2014.11.003] [PMID: 25464375]
[7] Kline E, Schiffman J. Psychosis risk screening: A systematic review. Schizophr Res 2014; 158(1-3): 11-8.[http://dx.doi.org/10.1016/j.schres.2014.06.036] [PMID: 25034762]
[8] Addington J, Stowkowy J, Weiser M. Screening tools for clinical high risk for psychosis. Early Interv Psychiatry 2015; 9(5): 345-56.[http://dx.doi.org/10.1111/eip.12193] [PMID: 25345316]
[9] Pickles A, Dunn G, Vázquez-Barquero JL. Screening for stratification in two-phase (‘two-stage’) epidemiological surveys. Stat Methods Med Res 1995; 4(1): 73-89.[http://dx.doi.org/10.1177/096228029500400106] [PMID: 7613639]
[10] Savill M, D’Ambrosio J, Cannon TD, Loewy RL. Psychosis risk screening in different populations using the Prodromal Questionnaire: A systematic review. Early Interv Psychiatry 2018; 12(1): 3-14.[http://dx.doi.org/10.1111/eip.12446] [PMID: 28782283]
[11] Loewy RL, Bearden CE, Johnson JK, Raine A, Cannon TD. The prodromal questionnaire (PQ): Preliminary validation of a self-report screening measure for prodromal and psychotic syndromes. Schizophr Res 2005; 79(1): 117-25.[http://dx.doi.org/10.1016/j.schres.2005.03.007] [PMID: 16276559]
[12] Kotzalidis GD, Solfanelli A, Piacentino D, et al. The Italian version of the 92-item Prodromal Questionnaire: Concurrent validity with the SIPS and factor analysis in a sample of 258 outpatients aged 11-36years. Schizophr Res 2017; 189: 50-6.[http://dx.doi.org/10.1016/j.schres.2017.02.008] [PMID: 28254200]
[13] Loewy RL, Johnson JK, Cannon TD. Self-report of attenuated psychotic experiences in a college population. Schizophr Res 2007; 93(1-3): 144-51.[http://dx.doi.org/10.1016/j.schres.2007.02.010] [PMID: 17459662]
[14] Loewy RL, Therman S, Manninen M, Huttunen MO, Cannon TD. Prodromal psychosis screening in adolescent psychiatry clinics. Early Interv Psychiatry 2012; 6(1): 69-75.[http://dx.doi.org/10.1111/j.1751-7893.2011.00286.x] [PMID: 21883972]
[15] Loewy RL, Pearson R, Vinogradov S, Bearden CE, Cannon TD. Psychosis risk screening with the Prodromal Questionnaire-brief version (PQ-B). Schizophr Res 2011; 129(1): 42-6.[http://dx.doi.org/10.1016/j.schres.2011.03.029] [PMID: 21511440]
[16] Ising HK, Veling W, Loewy RL, et al. The validity of the 16-item version of the Prodromal Questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophr Bull 2012; 38(6): 1288-96.[http://dx.doi.org/10.1093/schbul/sbs068] [PMID: 22516147]
[17] Kline E, Wilson C, Ereshefsky S, et al. Convergent and discriminant validity of attenuated psychosis screening tools. Schizophr Res 2012; 134(1): 49-53.[http://dx.doi.org/10.1016/j.schres.2011.10.001] [PMID: 22036199]
[18] Kline E, Thompson E, Demro C, Bussell K, Reeves G, Schiffman J. Longitudinal validation of psychosis risk screening tools. Schizophr Res 2015; 165(2-3): 116-22.[http://dx.doi.org/10.1016/j.schres.2015.04.026] [PMID: 25975827]
[19] Kline E, Thompson E, Demro C, Bussell K, Reeves G, Schiffman J. Self-report instruments for clinical monitoring of psychosis risk states. Psychiatr Serv 2016; 67(4): 456-9.[http://dx.doi.org/10.1176/appi.ps.201500063] [PMID: 26567937]
[20] Cicero DC, Krieg A, Martin EA. Measurement invariance of the prodromal questionnaire-brief among white, asian, hispanic, and multiracial populations. Assessment 2017; 1073191116687391: 1073191116687391. Epub ahead of print[http://dx.doi.org/10.1177/1073191116687391] [PMID: 28092988]
[21] Zhang T, Li H, Woodberry KA, et al. Prodromal psychosis detection in a counseling center population in China: An epidemiological and clinical study. Schizophr Res 2014; 152(2-3): 391-9.[http://dx.doi.org/10.1016/j.schres.2013.11.039] [PMID: 24387999]
[22] Fonseca-Pedrero E, Gooding DC, Ortuño-Sierra J, Paino M. Assessing self-reported clinical high risk symptoms in community-derived adolescents: A psychometric evaluation of the Prodromal Questionnaire-Brief. Compr Psychiatry 2016; 66: 201-8.[http://dx.doi.org/10.1016/j.comppsych.2016.01.013] [PMID: 26995254]
[23] Okewole AO, Ajogbon D, Adeniji AA, et al. Psychosis risk screening among secondary school students in Abeokuta, Nigeria: Validity of the Prodromal Questionnaire - Brief Version (PQ-B). Schizophr Res 2015; 164(1-3): 281-2.[http://dx.doi.org/10.1016/j.schres.2015.01.006] [PMID: 25640525]
[24] Gonçalves PD, Martins PA, Gordon P, et al. Prodromal Questionnaire: Translation, adaptation to Portuguese and preliminary results in ultra-high risk individuals and first episode psychosis. J Bras Psiquiatr 2012; 61(2): 96-101.[http://dx.doi.org/10.1590/S0047-20852012000200007]
[25] Hope V, Henderson M. Medical student depression, anxiety and distress outside North America: A systematic review. Med Educ 2014; 48(10): 963-79.[http://dx.doi.org/10.1111/medu.12512] [PMID: 25200017]
[26] Griggs S. Hope and Mental Health in Young Adult College Students: An Integrative Review. J Psychosoc Nurs Ment Health Serv 2017; 55(2): 28-35.[http://dx.doi.org/10.3928/02793695-20170210-04] [PMID: 28218927]
[27] World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA 2013; 310(20): 2191-4.[http://dx.doi.org/10.1001/jama.2013.281053] [PMID: 24141714]
[28] Snijders T. Estimation on the basis of snowball samples: How to weight. Bull Methodol Sociol 1992; 36: 59-70.[http://dx.doi.org/10.1177/075910639203600104]
[29] Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000; 25(24): 3186-91.[http://dx.doi.org/10.1097/00007632-200012150-00014] [PMID: 11124735]
[30] Goldberg DP. The detection of psychiatric illness by questionnaire. London: Oxford University Press 1992.
[31] Politi PL, Piccinelli M, Wilkinson G. Reliability, validity and factor structure of the 12-item General Health Questionnaire among young males in Italy. Acta Psychiatr Scand 1994; 90(6): 432-7.[http://dx.doi.org/10.1111/j.1600-0447.1994.tb01620.x] [PMID: 7892776]
[32] Preti A, Rocchi MB, Sisti D, et al. The psychometric discriminative properties of the Peters et al Delusions Inventory: A receiver operating characteristic curve analysis. Compr Psychiatry 2007; 48(1): 62-9.[http://dx.doi.org/10.1016/j.comppsych.2006.05.003] [PMID: 17145284]
[33] Ali GC, Ryan G, De Silva MJ. Validated screening tools for common mental disorders in low and middle income countries: A systematic review. PLoS One 2016; 11(6): e0156939.[http://dx.doi.org/10.1371/journal.pone.0156939] [PMID: 27310297]
[34] Raine A. The SPQ: A scale for the assessment of schizotypal personality based on DSM-III-R criteria. Schizophr Bull 1991; 17(4): 555-64.[http://dx.doi.org/10.1093/schbul/17.4.555] [PMID: 1805349]
[35] American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 3rd Edition-Revised (DSM-III-R). Washington, DC: American Psychiatric Association; 1987.
[36] Fossati A, Raine A, Carretta I, et al. The three-factor model of schizotypal personality: Invariance across age and gender. Pers Individ Dif 2003; 35: 1007-19.[http://dx.doi.org/10.1016/S0191-8869(02)00314-8]
[37] Raine A, Reynolds C, Lencz T, Scerbo A, Triphon N, Kim D. Cognitive-perceptual, interpersonal, and disorganized features of schizotypal personality. Schizophr Bull 1994; 20(1): 191-201.[http://dx.doi.org/10.1093/schbul/20.1.191] [PMID: 8197415]
[38] Reynolds CA, Raine A, Mellingen K, Venables PH, Mednick SA. Three-factor model of schizotypal personality: Invariance across culture, gender, religious affiliation, family adversity, and psychopathology. Schizophr Bull 2000; 26(3): 603-18.[http://dx.doi.org/10.1093/oxfordjournals.schbul.a033481] [PMID: 10993401]
[39] Preti A, Siddi S, Vellante M, et al. Bifactor structure of the schizotypal personality questionnaire (SPQ). Psychiatry Res 2015; 230(3): 940-50.[http://dx.doi.org/10.1016/j.psychres.2015.11.010] [PMID: 26607431]
[40] Fonseca-Pedrero E, Debbané M, Ortuño-Sierra J, et al. The structure of schizotypal personality traits: A cross-national study. Psychol Med 2018; 48(3): 451-62.[http://dx.doi.org/10.1017/S0033291717001829] [PMID: 28712364]
[41] Galobardes B, Shaw M, Lawlor DA, Lynch JW, Davey Smith G. Indicators of socioeconomic position (part 1). J Epidemiol Community Health 2006; 60(1): 7-12.[http://dx.doi.org/10.1136/jech.2004.023531] [PMID: 16361448]
[42] R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.Rproject.org/ 2017
[43] Steiger JH. Tests for comparing elements of a correlation matrix. Psychol Bull 1980; 87: 245-51.[http://dx.doi.org/10.1037/0033-2909.87.2.245]
[44] Nunnally JC. Psychometric Theory, 2nd ed. New York, NY: McGraw-Hill 1978.
[45] Brennan P, Silman A. Statistical methods for assessing observer variability in clinical measures. BMJ 1992; 304(6840): 1491-4.[http://dx.doi.org/10.1136/bmj.304.6840.1491] [PMID: 1611375]
[46] Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1(8476): 307-10.[http://dx.doi.org/10.1016/S0140-6736(86)90837-8] [PMID: 2868172]
[47] Lopez-Raton M, Rodriguez-Alvarez MX, Cadarso-Suarez C, et al. Optimal cutpoints: An R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw 2014; 61(8): 1-36.http://www.jstatsoft.org/v61/i08/[http://dx.doi.org/10.18637/jss.v061.i08]
[48] le Cessie S, van Houwelingen JC. A goodness-of-fit test for binary regression models, based on smoothing methods. Biometrics 1991; 47: 1267.[http://dx.doi.org/10.2307/2532385]
[49] Pregibon D. Goodness of link tests for generalized linear models. J R Stat Soc Ser C Appl Stat 1980; 29: 14-5.
[50] McFadden D. Conditional logit analysis of qualitative choice behavior. In: Zarembka P, Ed. Frontiers in econometrics. New York, NY: Academic Press 1994: pp. 104-142.
[51] Robin X, Turck N, Hainard A, et al. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 2011; 12: 77.[http://dx.doi.org/10.1186/1471-2105-12-77] [PMID: 21414208]
[52] Rietdijk J, Klaassen R, Ising H, et al. Detection of people at risk of developing a first psychosis: Comparison of two recruitment strategies. Acta Psychiatr Scand 2012; 126(1): 21-30.[http://dx.doi.org/10.1111/j.1600-0447.2012.01839.x] [PMID: 22335365]
[53] Kline E, Wilson C, Ereshefsky S, et al. Psychosis risk screening in youth: A validation study of three self-report measures of attenuated psychosis symptoms. Schizophr Res 2012; 141(1): 72-7.[http://dx.doi.org/10.1016/j.schres.2012.07.022] [PMID: 22921375]
[54] Yung AR, Yuen HP, McGorry PD, et al. Mapping the onset of psychosis: The comprehensive assessment of at-risk mental states. Aust N Z J Psychiatry 2005; 39(11-12): 964-71.[http://dx.doi.org/10.1080/j.1440-1614.2005.01714.x] [PMID: 16343296]
[55] McGlashan TH, Walsh BC, Woods SW. The psychosis-risk syndrome, handbook for diagnosis and follow-up. New York: Oxford University Press; 2010.
[56] Schultze-Lutter F, Addington J, Ruhrmann S, Klosterkötter J. Schizophrenia Proneness Instrument, Adult Version (SPI-A). Italian edition edited by FM Ferro and G Aiello. Rome: Giovanni Fioriti Editore; 2007.
[57] Schultze-Lutter F, Michel C, Schmidt SJ, et al. EPA guidance on the early detection of clinical high risk states of psychoses. Eur Psychiatry 2015; 30(3): 405-16.[http://dx.doi.org/10.1016/j.eurpsy.2015.01.010] [PMID: 25735810]
[58] DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 1988; 44(3): 837-45.[http://dx.doi.org/10.2307/2531595] [PMID: 3203132]

Endorsements



"Open access will revolutionize 21st century knowledge work and accelerate the diffusion of ideas and evidence that support just in time learning and the evolution of thinking in a number of disciplines."


Daniel Pesut
(Indiana University School of Nursing, USA)

"It is important that students and researchers from all over the world can have easy access to relevant, high-standard and timely scientific information. This is exactly what Open Access Journals provide and this is the reason why I support this endeavor."


Jacques Descotes
(Centre Antipoison-Centre de Pharmacovigilance, France)

"Publishing research articles is the key for future scientific progress. Open Access publishing is therefore of utmost importance for wider dissemination of information, and will help serving the best interest of the scientific community."


Patrice Talaga
(UCB S.A., Belgium)

"Open access journals are a novel concept in the medical literature. They offer accessible information to a wide variety of individuals, including physicians, medical students, clinical investigators, and the general public. They are an outstanding source of medical and scientific information."


Jeffrey M. Weinberg
(St. Luke's-Roosevelt Hospital Center, USA)

"Open access journals are extremely useful for graduate students, investigators and all other interested persons to read important scientific articles and subscribe scientific journals. Indeed, the research articles span a wide range of area and of high quality. This is specially a must for researchers belonging to institutions with limited library facility and funding to subscribe scientific journals."


Debomoy K. Lahiri
(Indiana University School of Medicine, USA)

"Open access journals represent a major break-through in publishing. They provide easy access to the latest research on a wide variety of issues. Relevant and timely articles are made available in a fraction of the time taken by more conventional publishers. Articles are of uniformly high quality and written by the world's leading authorities."


Robert Looney
(Naval Postgraduate School, USA)

"Open access journals have transformed the way scientific data is published and disseminated: particularly, whilst ensuring a high quality standard and transparency in the editorial process, they have increased the access to the scientific literature by those researchers that have limited library support or that are working on small budgets."


Richard Reithinger
(Westat, USA)

"Not only do open access journals greatly improve the access to high quality information for scientists in the developing world, it also provides extra exposure for our papers."


J. Ferwerda
(University of Oxford, UK)

"Open Access 'Chemistry' Journals allow the dissemination of knowledge at your finger tips without paying for the scientific content."


Sean L. Kitson
(Almac Sciences, Northern Ireland)

"In principle, all scientific journals should have open access, as should be science itself. Open access journals are very helpful for students, researchers and the general public including people from institutions which do not have library or cannot afford to subscribe scientific journals. The articles are high standard and cover a wide area."


Hubert Wolterbeek
(Delft University of Technology, The Netherlands)

"The widest possible diffusion of information is critical for the advancement of science. In this perspective, open access journals are instrumental in fostering researches and achievements."


Alessandro Laviano
(Sapienza - University of Rome, Italy)

"Open access journals are very useful for all scientists as they can have quick information in the different fields of science."


Philippe Hernigou
(Paris University, France)

"There are many scientists who can not afford the rather expensive subscriptions to scientific journals. Open access journals offer a good alternative for free access to good quality scientific information."


Fidel Toldrá
(Instituto de Agroquimica y Tecnologia de Alimentos, Spain)

"Open access journals have become a fundamental tool for students, researchers, patients and the general public. Many people from institutions which do not have library or cannot afford to subscribe scientific journals benefit of them on a daily basis. The articles are among the best and cover most scientific areas."


M. Bendandi
(University Clinic of Navarre, Spain)

"These journals provide researchers with a platform for rapid, open access scientific communication. The articles are of high quality and broad scope."


Peter Chiba
(University of Vienna, Austria)

"Open access journals are probably one of the most important contributions to promote and diffuse science worldwide."


Jaime Sampaio
(University of Trás-os-Montes e Alto Douro, Portugal)

"Open access journals make up a new and rather revolutionary way to scientific publication. This option opens several quite interesting possibilities to disseminate openly and freely new knowledge and even to facilitate interpersonal communication among scientists."


Eduardo A. Castro
(INIFTA, Argentina)

"Open access journals are freely available online throughout the world, for you to read, download, copy, distribute, and use. The articles published in the open access journals are high quality and cover a wide range of fields."


Kenji Hashimoto
(Chiba University, Japan)

"Open Access journals offer an innovative and efficient way of publication for academics and professionals in a wide range of disciplines. The papers published are of high quality after rigorous peer review and they are Indexed in: major international databases. I read Open Access journals to keep abreast of the recent development in my field of study."


Daniel Shek
(Chinese University of Hong Kong, Hong Kong)

"It is a modern trend for publishers to establish open access journals. Researchers, faculty members, and students will be greatly benefited by the new journals of Bentham Science Publishers Ltd. in this category."


Jih Ru Hwu
(National Central University, Taiwan)


Browse Contents



Advertisements


Webmaster Contact: info@benthamopen.com
Copyright © 2018 Bentham Open