A concurrent nested mixed method research design was used (Fig. (1)). We assessed attendees at 4 primary health care centers run by the government of the republic of Zambia between December 2008 and May 2009. These clinics were purposely selected within the city of Lusaka, two of which were clinics in very high density areas (Kalingalinga and Mtendere) while the others were clinics in a medium density area (Chilenje and Chelston). The residents of these areas speak a number of languages but mainly English and Nyanja.

Fig. (1). Study design.

Fig. (2). Item by Item analysis of responses to SRQ-20.

Fig. (3). Sensitivity, Specificity AUC for SRQ-20, SRQ-10 and GHQ-12 (combined).

Fig. (4). Sensitivity, Specificity AUC for SRQ-20, SRQ-10 and GHQ-12 (Males).

Fig. (5). Sensitivity, Specificity AUC for SRQ-20, SRQ-10 and GHQ-12 (Females).

A pilot study was first conducted at Kabwata clinic (outside the study sites) (Fig. (1)). Forty-five outpatients were interviewed and based on the results it was decided that the questionnaire would be read to all the participants irrespective of their education level. A time sample of 400 clinic attendees aged 16 years and over was asked to participate in the study between January and March 2009. The purpose of the study was explained to each participant by the research assistants and consent was asked for. Each clinic was sampled randomly on selected days, 3 times each week. On the selected day, interviews were conducted with consecutive clinic attendees at the clinic outpatients department.

A brief social and demographic questionnaire was administered to all the participants by research assistants who had received training in carrying out interviews. The interviews lasted approximately 10 minutes. Information on participant’s demographic characteristics, including age, gender, educational attainment, residence and marital status, was collected using standard questionnaire items. The participants were also asked in what language they wanted the interview to be carried out. Socioeconomic position was assessed using the participant’s educational attainment, employment status and an asset index based on items intended to reflect household wealth. These included household ownership of appliances (TV, radio, refrigerator, electricity, bicycle, plough, cattle and donkey) and other household resources (running water in the home, type of toilet, type of floor, and type of roofing material). A summative wealth index was then constructed which was categorized into low, medium and high wealth index. The participants were also asked to rate their own health status by answering the question: How would you say your health is at the moment? Is it, (1) Very poor, (2) Poor, (3) Fair, (4) Good, or (5) Excellent? The recent life events were evaluated by events occurring in the previous 12 months based on whether the participant had experienced (1) Break-up of a marriage (2)Break-up of a sexual relationship, (3) Physical abuse, (5) Neglected or disowned by family or (6) loss of a loved one.

The SRQ-20 and the GHQ-12 were used to measure global mental distress. These interviews were conducted by interviewers of the same sex as the participant. The participants were then classified into two groups according to their scores on the SRQ-20 (low, 0-7; high 8+) and GHQ-12 (low, 0-3; high, 4+). Subsequently these participants were directed to a medical officer who held a clinical interview with them for the ailments that brought them to the clinic as well as conducting a psychiatric inquiry where the DSM-IV schedules for common mental disorders was used to determine the presence and diagnosis of a psychiatric disorder. The general health assessment and the mental distress assessment were done at the same time so that the patients were not delayed due to the study. The clinical interview was conducted blind, without the knowledge of the questionnaire results.

In the second part of the study, in-depth interviews were conducted in a subsample of 28 participants nested within the quantitative sample. The sample consisted of participants who were classified as being high scorers (14 participants) and low scorers (14 participants), on the basis of the SRQ-20 score >7 and GHQ-12 score >3. These interviews were used to assess face and content validity.

All the instruments were translated into Nyanja and Bemba as these are the most predominantly spoken languages in Lusaka. The results from the pilot study also confirmed that participants who did not speak English opted to be interviewed in Nyanja or Bemba. These instruments were then back translated to English by bilingual translators from the linguistics department of the University of Zambia. Discrepancies that were found were discussed further by a group that included the principle investigator, translators and a medical doctor from the psychiatric hospital. This was to ensure face validity as well as conceptual meaning. Few final changes were made after the pilot study.

A team of three male and three female interviewers who had no experience in mental health care administered the SRQ-20 and the GHQ-12. They, however, all had previous experience administering questionnaires in other epidemiological studies. A three day training session was conducted in administering the instruments. This involved explanation and discussion of conceptual definitions of each item in the instruments and role playing. This was followed by a 1 day field test.

The Research and Bioethics Committee of the University of Zambia and the Ministry of Health, through the Lusaka District Health Office approved this study. Permission to conduct the study was also further sought from the authorities in charge of the Primary Health Centres. The study was conducted in accordance with the guidelines of Good Clinical Practices in biomedical research.

The data was analysed using SPSS version 15. In this study, receiver operating characteristic (ROC) analysis was used to identify a cut-off point for the SRQ-10, SRQ-20 and GHQ-12 as defined with the DSM-IV as the gold standard. This plots sensitivity against 1-specificity for each possible cut-off point. The sensitivity and specificity here being the fraction of true positive cases and true negative cases correctly identified by the screening tools respectively. Each ROC is characterised by an area under the curve (AUC) which generally indicates the overall accuracy of the questionnaire over a range of cut-off points to distinguish between cases and non-cases. AUC ranges between 0.0 to 1.0 with 1.0 indicating perfect prediction and 0.5 indicating a prediction equal to chance. Hence we used the AUC to compare the screening tools over the total range of scores. We performed a factor analysis with varimax rotation to check for measurement equivalence. This refers to the equivalence of construct or theoretical validities across populations, which is a prerequisite for the comparison of prevalence rates or mean scores of the scales [25]. Independent t-tests were performed to compare the scales between sexes while the Pearson Chi-square was used to compare the psychiatric diagnosis in the same groups. We also calculated Pearson correlation coefficients to examine the relationship between the scales.

The sample was composed of 400 respondents who completed the SRQ-20 and the GHQ-12 and were subsequently referred to the Medical Doctor for clinical interview using the DSM-IV. These respondents were visiting the four Primary Health Care (PHC) centres for various medical reasons. Ten patients were not included because they refused the clinical interview. There were, however, no significant differences between the total sample and the participants that refused the clinical interview in sex ratio, wealth status, marital status and educational attainment. The respondents who were ethnically from the Bemba speaking tribes accounted for 26% of the total study population, while 16% were Nyanja and only 12% were Tonga. However almost half of the respondents preferred English as the language for the interview, while the others preferred Nyanja and Bemba (38.8% and 8.5% respectively). The sample had 167 (41.8%) men and 233 (58.3%) women (Table 1). The male patients ranged in age between 16 and 67 years with a mean of 32 years (SD=11.1). Female patients ranged between 16 and 65 years with a mean of 29 years (SD=9.4). The majority of participants were married (64%). Most of the patients had more than 8 years of education (secondary 56% vs. tertiary 19.5%) while 3.8% were illiterate. There were no statistical differences between the clinics serving the medium and high density catchment areas in terms of marital status (t= 1.139, p=0.06, eta² =0.00), wealth index (t=0.198, p=0.418, eta² = 0.00) and educational level (t=0.284, p=0.777, eta ²= 0.00).

Principal component analysis with varimax rotation of the SRQ-20 items revealed a two factor model (common disorders and social disability) that explained 50.1% of the variance.

A similar model was extracted from the SRQ-10 and explained 50.2% of the variance, while three factors (Common disorders, social dysfunction and loss of confidence) were extracted from GHQ-12 items by the same procedure explaining 49.9% of the variance. The factor structure of these instruments was similar to that reported in other studies [2, 26, 27]. We as a result found support for the measurement equivalence between the SRQ and GHQ-12 instruments. The correlation between the SRQ-20 and SRQ-10 was 0.85 while the correlation between these instruments and GHQ-12 scales was found to 0.60 and 0.52 respectively. Independent t-tests were used to compare differences in the continuous instrument scores between men and women and no significant differences were found. For comparison of definitive psychiatric diagnosis between males and females chi-square test was used and found to be insignificant (p=0.370). Overall the prevalence of common mental disorder as diagnosed by the DSM-IV classification was 13.6%, and was found to be mainly depression (10.8%) anxiety disorders (1.8%). The prevalence tended to be higher in females than males (women 14% vs. men 12.9%, p=0.743). An item-by-item analysis of the SRQ also revealed that females on average reported more symptoms of mental distress than the males (Fig. 2).

This part of the analysis focuses on the ability of the SRQ-20, GHQ-12 and SRQ-10 to screen for psychopathology (mental distress). Fig. (3) shows that SRQ-20 and SRQ-10 performed well with the area under the curve (AUC) being 0.96 and 0.95 respectively while the GHQ-12 had a modest AUC of 0.81. When analyzed separately for men and women no clear tendencies to perform better by sex were noted (Figs. 4, 5). Table 2 shows the sensitivity, specificity, positive predictive values, and negative predictive and kappa’s values of the scales with different cut-off points. The most appropriate cut-off point was a trade off between sensitivity and specificity. Since these instruments are meant to be used as screening instruments, the optimal cut-off point is one with high sensitivity and an acceptable specificity. The optimal cut-off for both SRQ-20 (sensitivity 0.85, specificity 0.94) and SRQ-10 (sensitivity 0.81, specificity 0.96) was 7, while that for GHQ-12 was 2 (sensitivity 0.66, specificity 0.86). Further analysis by sex did not reveal any significant differences in cut-off points.

The study to assess the content validity was conducted in a subsample of the quantitative study. It included 28 respondents, 15 (53.6%) of whom were male while 13 (42.9%) were female. The respondents had an average of 9 school years being slightly higher in males than in females (10 years vs. 8 years respectively). Over half (53.6%) reported that they were married, 39.3% were single while less that 1% were either, divorced, separated or widowed. Half of the respondents preferred to have the interview conducted in English while 23% preferred Bemba while 28.6% preferred Nyanja. The 28 respondents gave the yes-answer a total of 205 times on the SRQ. Invalidity of these answers was considered on two main stages listed below. The results are presented in Table 3.

Differences in conceptualization of the question by the respondent were recorded in 25% of the yes-answers given. “Do you have headaches often?” All the invalid answers given to this question were attributed to the presence of other intercurrent illness namely hypertension, malaria and toothaches. However the question largely managed to uncover information indicating the headache as a symptom of depression and/or anxiety.

“Do you have uncomfortable feelings in your stomach?” - Among those giving invalid answers, this question was understood as an inquiry into presence of gastrointestinal ailment. The reasons most frequently given were: “Yes because I suffer from “gas” in my stomach” and “Yes I get uncomfortable feeling when I eat beans”. Contrary the questions;” Is your digestion poor? and Is your appetite poor?” performed very well with the most frequent answer among the valid answers being: “Yes, I don’t feel like eating because I have many thoughts and even when I feel like eating I have problems swallowing or I get full easily”.

Anxiety Items: “Are you easily frightened? Do your hands shake? Do you feel tense or worried?” These items seemed to have a narrow meaning in the context of our study, and were interpreted as being an enquiry into literal feeling or state of being afraid. The most frequent answer was: “Sometimes, especially if I am threatened or if I am in trouble with spouse”. We also probed the no-answers to these items and we found the same responses suggestive of the fact that being frightened, hands shaking or feeling tense or worried is associated with literal fear. This concept does not seem to exist in our sample unless there is a clear reason for it and so the items failed to uncover the information suggestive of anxiety.

“Do you feel tired all the time?” Was interpreted by the respondents as asking about whether they get tired easily as regards work rather than an enquiry pertaining to depression. The most frequent answer was “Yes I get tired because of work since I work very long hours”.

We assigned a yes-answer to this invalidity category if the question had to be repeated one or more times or if it needed further explanation before an answer was obtained. We also assigned, to this category, respondents who said they didn’t understand or who answered “I do not know” to the questions posed. We also included in this category respondents who insisted on the yes-answer but were unable or unwilling to give further details or examples of experiences that would help us to clearly define the underlying psychopathology. Respondents who also directly indicated that they thought by participating in the interview they would be “fast-tracked” to see the doctor were also assigned to this category, although these accounted for less than 1%. This kind of invalid answers were observed in 15.6% of the yes-answers and was attributed to not understanding the content of the question and complexity of the words used.

Within the subsample we also assessed the face validity of the SRQ by asking the respondents what they thought the instrument was supposed to measure and we also probed further by asking the respondents what they thought the aim of these questions were. The SRQ was found to have good face validity with 71.4% of the respondents saying that we were assessing mental health. The most common response was that we were measuring “problems of the mind and soul” (53.6%) while 17.9% said we were assessing stress and depression. The proportion who said they did not know the aim of the questions was 28.6%.