PATIENTS AND METHODS

Sampling

Three groups of subjects were enrolled consecutively from the rheumatology outpatient clinics of 10 French university hospitals: 235 patients with RA, 175 with SpA, and 195 controls with other rheumatological disorders. All diagnoses were confirmed by a senior rheumatologist. Recruitment was restricted to rheumatology outpatients to increase the likelihood of false positive errors (that is, subjects being identified as having RA or SpA when in fact they had another inflammatory rheumatic condition) and minimise failure to detect genuine cases.

At least 150 subjects in each group were required to obtain sensitivity and specificity estimates from 80 to 95% with a precision of 3% at a type I error threshold of 5%.

Diagnosis

Subjects were assigned to one of the three groups on the basis of a clinical diagnosis confirmed using biological data, radiological findings, medical records, and prescription information gathered during the recruitment visit. Patients with RA and SpA were also required to fulfil American College of Rheumatology (ACR) 1987⁹ and European Spondylarthropathy Study Group (ESSG) 1991¹⁰ criteria, respectively. These classification criteria were also used as an alternative to clinical diagnosis in the analysis. Rheumatologists fully documented their findings on standard forms.

Questionnaire

A questionnaire suitable for use by patient interviewers was designed by a group of rheumatologists and epidemiologists from the epidemiology section of the French Society of Rheumatology. Some questions covered the relevant signs and symptoms and asked for a self reported diagnosis, and others were based on the RA (ACR 1987) and SpA (ESSG 1991) criteria. Pretesting for comprehensibility by a doctor and research nurse in a group of 95 patients improved the presentation of some of the 33 items.

Design

Three to 10 volunteers in each setting, themselves patients with RA or SpA, were recruited from each of several French self help groups and social organisations (see “Acknowledgements”). They were then trained by a member of staff from a polling company in how to conduct telephone interviews, including how to deliver questions in a standardised manner. Training also involved telephone role playing exercises, and preliminary test interviews followed by debriefing.

At the start of the survey proper, the newly trained interviewers were given lists of telephone numbers but remained unaware of the identities of the respondents, whether they were patients or controls, and of the results of medical evaluation and diagnosis. The first session of interviews in each setting was supervised by the trainer. The protocol had ethics committee approval.

Statistical analysis

The sensitivity and specificity, with 95% confidence intervals, of information gained using each item of the questionnaire was compared with corresponding data from the medical examination, and with the clinical diagnosis and classification. Two separate RA-control, and SpA-control comparisons used logistic regression modelling to identify the set of items best able to discriminate between cases and controls under the most difficult circumstances. The stability of each model was checked by running stepwise forward and backward regressions. The overall classification in each group was used, and the Hosmer-Lemeshow goodness of fit test of the model was used for the data. To provide an estimate of the performance of selected sets of items, Harrel’s c value (an approximation of the area under the curve for item performance) was used, which varies between 0 and 1; the closer to 1, the lower the performance error.¹¹ The performance of each item was assessed both separately and in combination (or/and). The percentage concordance of a set of items indicates overall concordance—that is, the total proportion of well classified subjects in both control and RA or SpA groups.

To record the practicality of these items for use in epidemiological population prevalence surveys, positive and negative predictive values were computed on the basis of an expected prevalence of 0.5%, 1%, and 2%.

All analyses were carried out using SAS version 8.0 statistical software.

RESULTS

Of 605 subjects recruited consecutively from 10 rheumatology outpatient clinics, 235 had RA, 175 had SpA, and 195 were controls. Their mean ages were 55.6, 47.1, and 55.0 years, respectively, and the female/male ratios were 2.6, 0.7, and 2.2, respectively (table 1). Almost all patients with RA (230) satisfied the ACR 1987 classification criteria. Individual criteria were positive in a range from 49.5% for the presence of nodules to 97.1% for radiographic changes. Patients with SpA satisfied preliminary criteria for classification in 122/175 cases. Criteria were positive in 39.7% for synovitis and 66.4% for inflammatory spinal pain, and in a range from 6.9% for inflammatory bowel disease to 57.8% for alternating buttock pain.

Control patients had osteoarthritis (n=67), low back pain (n=41), other systemic diseases (n=28), soft tissue rheumatism (n=27), osteoporosis (n=11), or another, miscellaneous, condition (n=31). The total is greater than 195 because some subjects had more than one disorder.

None of the selected patients and controls refused to participate. Interview administration lasted less than five minutes, on average, and no particular difficulty was noted during the telephone interviews. When the clinical diagnosis was taken as the standard, sensitivities of items on the questionnaire ranged from 89% to 0% in RA, and from 83% to 7% in SpA. Restricting the analysis to patients with RA satisfying ACR criteria (89%–0%) and to patients with SpA satisfying ESSG criteria (84%–6%) had no significant effect. Among controls, the specificities of items ranged from 99% to 22% (table 2).

A diagnosis was self reported by 203/235 (86%) patients with RA, 135/175 (77%) with SpA, and 154/195 (79%) controls. In RA, these reports reached 99% sensitivity and 87% specificity compared with the clinical diagnosis; in SpA, the figures were 85% and 96% respectively. Patients who did not report a diagnosis were older, more likely to be female in the RA group, and male in the SpA group, and had more severe disease (as indicated by the number of positive answers to interview questions).

Items that contributed significantly to discrimination between patients with RA and controls in logistic regression modelling included self reported diagnosis and five items providing 97.7% agreement (Harrel’s goodness of fit test c=0.984, p=0.953) when using clinical diagnosis as a standard, and self reported diagnosis and six items providing 98.5% agreement (c=0.988, p=0.946) when using ACR classification criteria. The overall contribution of items to discrimination between SpA and control included self report of diagnosis and three items accounting for 94.4% agreement (c=0.95, p=0.02) with clinical standard, and self report of diagnosis plus five items providing 93.8% agreement (c=0.92, p=0.11) with ESSG criteria (table 3).

When self reported diagnosis was excluded, a combination of 10 items including two general queries about peripheral (Q1) and spinal (Q2) pain provided 90.4% agreement (c=0.911, p=0.44) with a clinical diagnosis of RA; and a combination of nine items (of which eight were shared with above-mentioned 10 items) provided 89.7% agreement (c=0.90, p=0.005) with ACR classification criteria. With regard to SpA, excluding self reported diagnosis reduced the performance to a combination of nine items, including pain in peripheral joints, that provided 79.1% agreement (c=0.6, p=0.32) with the clinical standard, and eight items (of which seven were shared with above-mentioned nine items) providing 80.6% agreement (c=0.65, p=0.17) with ESSG criteria (table 4).

Some combinations of items, whether exclusive or inclusive, achieved high specificities and sensitivities close to 100% (table 5). Combinations of self reported diagnosis plus hand and wrist x ray data, morning stiffness, or rheumatoid factor testing, were very close to self reported diagnosis alone in RA. Self reported diagnosis plus a family history of spondylitis or reactive arthritis, rheumatoid factor testing, or spinal pain was close to a self reported diagnosis alone in SpA.

To further record the practicality of these items for use in epidemiological population prevalence surveys, positive and negative predictive values were computed on the basis of an expected prevalence of 0.5–2%. An excellent negative predictive value (99%) was seen in all projections, and positive predictive values were correct to low (table 6).

DISCUSSION

These results show that cases of RA and SpA can be detected using a limited set of items including self reported diagnosis and information about spinal and peripheral joint involvement, supplemented with classification criteria for RA and SpA as adapted for use in a questionnaire delivered by lay patient interviewers to lay respondents (see “Appendix”). The value of a self reported diagnosis was similar to that found elsewhere.¹² Other reports of the development of questionnaires to detect rheumatic diseases have been published, some of which have been successfully applied in various situations and countries.⁵ An attempt to develop instruments specific for knee osteoarthritis has been hampered by limitations such as the effect of age, and the low performance of a single stage procedure.⁶

The questionnaire described here is suitable for use in telephone surveys of the general population aimed at detecting cases in order to calculate reasonable estimates of prevalence. However, detection of cases by lay patient interviewers must be followed by a confirmation of these cases by doctors. The advantage of such a two step approach to case ascertainment is that it reduces the number of people to be examined by doctors without increasing the proportion of cases missed.

A preliminary estimate of the effect of specificity error on the burden on doctors showed that if the questionnaire achieved 70, 80, 90, or 95% specificity, the numbers of false positive cases in a population of 4000 with a 1% prevalence would be 1500, 1000, 600, and 400, respectively. Present findings of 98% specificity in RA and 80% in SpA may therefore be considered acceptable. The figures would be even higher in a general population rather than among outpatient controls deliberately selected to maximise false positive (specificity) errors. Enrolled patients were outpatients at clinics of public rheumatology departments, and thus cases and controls would be closer each other than in a community population making the distinction more difficult, but were not inpatients to avoid bias towards severe disease. In such public departments, outpatients are probably less well educated about their disease than those in the private sector, the latter being generally attended by a more highly educated population in France.

Although there is a possibility that milder diseases may remain undetected in part, the goal was not to uncover undiagnosed cases but to detect patients with diagnosis. The potential of this questionnaire to detect undiagnosed cases remains unknown, but sensitivity estimates derived from pre-identified cases are within an acceptable range. Theoretically, perfect sensitivity would require full knowledge of all cases. Although there is no standard approach to detecting undiagnosed cases in the field, preliminary use of the present strategy in a current field survey has already identified some of these cases.¹³

Nevertheless, integration of the questionnaire into a case detection strategy has its limitations. Firstly, answers to some items are likely to be missing despite careful formulation and testing to ensure comprehensibility. To minimise such problems, interviewers should be trained to ask questions in a standardised form, and to avoid rephrasing them.

Secondly, the use of telephone interviews is only appropriate if large proportions of the population have personal telephones, and if operators allow access to their numbers. When the present survey was conducted in 2001, the availability of mobile telephone numbers was slightly restricted, but more than 95% of French households had a fixed telephone and appeared on directory listing. When there is insufficient coverage by phone, a face to face interview is recommended. A questionnaire which is already satisfactory on the phone will be valid in a face to face interview. This way of using the questionnaire is currently being tested.

Thirdly, the design of the study may limit external validity. Because of the low prevalence¹⁴ of RA and SpA, a case-control design was used to assess the properties of each item. Consequently, patients knew some weeks in advance that they would be given a telephone interview about their health. In the forthcoming field prevalence survey, interviews to detect cases will take place without notice and potential interviewees may refuse to participate.

Fourthly, specificity in the present study is likely to be less than in the field survey, as a high proportion of the population in the latter survey will be free of any rheumatic condition. When the performance of self reported diagnosis was calculated using data from a previous prevalence survey,¹⁵ its sensitivity was as high as 96.6%. High sensitivity (nearly 100%) will be mandatory in the field survey to minimise false negative reports (failure to detect RA or SpA). This has already been considered in the pretest phase.

The fact that interviewers had either RA or SpA themselves was not intrinsically important. The value of this to the investigation was that these interviewers were motivated to help without being paid. Active participation in research by patients from self help groups is one way of advancing awareness of their condition. In this particular collaboration, interviewers were not only voluntary research subjects but also active partners in the investigative process. Such partnership could be extended to the field prevalence survey. Other validation studies of this questionnaire are currently continuing in several European countries as part of an international project.

In conclusion, strategies for the case ascertainment of rare diseases like RA are costly, and require high levels of validity. However, if detection of cases and case confirmation are carried out separately, patient interviewers can be involved in the former, thereby saving money. Confirmation of cases by a rheumatologist remains necessary. An accurate and reliable questionnaire that is suitable for use in telephone surveys of a general population and can be administered by trained patient interviewers, such as that described here, is clearly a useful tool.

APPENDIX: QUESTIONNAIRE FOR THE DETECTION OF CASES OF RA AND SpA IN A COMMUNITY SURVEY

Subjects are asked to answer “yes” or “no” to the following questions:

• – Q1 Are you at present experiencing, or have you in the past experienced, pains in your joints?

• – Q2 Have you, or have you had, pain in your neck, your back, or your buttocks?

If yes to Q1 or Q2:

• – Q3 What was the diagnosis?

Ask Q4 to Q16 if “yes” to Q1:

• – Q4 Are your joints swollen or have they been in the past?

Ask Q5 to Q9 if “yes” to Q4:

• – Q5 Are or were your joints symmetrically affected, that is to say about the same on each side?

• – Q6 Are or were your hands affected?

• – Q7 Are or were your lower limbs affected (that is to say, your groin, your hip joint, your knees, your ankles, or your feet)?

• – Q8 Are or were more than three joints affected?

• – Q9 Has the pain lasted or did it last more than six weeks?

• – Q10 Does or did the pain wake you?

• – Q11 Are or were your joints stiff in the morning?

Ask Q12 if yes to Q11:

• – Q12 For about how many minutes?

• – Q13 Have you or have you had nodules under the skin on your elbows or hands?

• – Q14 Have you had the rheumatoid factor test, sometimes called the latex test or the Waaler-Rose test?

Ask Q15 if yes to Q14:

• – Q15 Do you know if it was positive?

• – Q16 Have you had x ray examinations of your hands and wrists?

Ask Q17 to Q22 if yes to Q2:

Now we are going to talk about your lumbar, back, or neck pain.

• – Q17 Did your pains start before you were 45 (if the person was born before 19__)?

• – Q18 Did they start gradually?

• – Q19 Do you have more pain when you exert yourself?

• – Q20 Do you have or have you had a feeling of stiffness in the morning?

Ask Q21 if yes to Q20:

• – Q21 On average, how long does or did this last in the morning?

• – Q22 Has your pain lasted or did your pain last more than three weeks?

For all subjects:

Do you know if in your family, apart from yourself, there have been cases of:

• – Q23 Spondylitis?

• – Q24 Psoriasis?

• – Q25 Uveitis?

• – Q26 Reactive arthritis?

• – Q27 Crohn’s disease or ulcerative colitis?

Have you yourself had:

• – Q28 Psoriasis?

• – Q29 Diarrhoea that lasted a long time?

• – Q30 Pain in your heel?

• – Q31 Pain in your buttocks?

• – Q32 Have you had x ray examinations of your back or pelvis?

• – Q33 What treatment are you receiving for your problem?