Methodology

Creating the Surveys

We first create a skeleton algorithm based on the existing research literature and published guidelines to identify key decision points in the everyday treatment of patients with the disorder in question. We then highlight important clinical questions that had not yet been adequately addressed or definitely answered. Then we develop written questionnaires concerning medication and psychosocial treatments.

The Rating Scale

The survey questionnaires use a 9-point scale slightly modified from a format developed by the RAND Corporation for ascertaining expert consensus. We present the rating scale to the experts with the following instructions:

Extremely Inappropriate 1 2 3 4 5 6 7 8 9 Extremely Appropriate

9 = extremely appropriate: this is your treatment of choice
7-8 = usually appropriate: a 1st line treatment you would often use
4-6 = equivocal: a 2nd line you would sometimes use (e.g., patient/family preference or is 1st line treatment is ineffective, unavailable, or unsuitable)
2-3 = usually inappropriate: a treatment you would rarely use
1 = extremely inappropriate: a treatment you would never use

Analyzing and Presenting the Results

In analyzing the results of the survey questions, we first calculate the mean (Avg), standard deviation (SD), and confidence interval (CI) for each item. The CI is a statistically calculated range which tells you that there is a 95% chance that the mean score would fall within that range if the survey were repeated with a similar group of experts. We designate a rating of first, second, or third line for each treatment option, determined by the category into which the 95% CI of its mean score falls.

The Ratings

First line treatments are those strategies that came out on top when the experts responses to the survey were statistically aggregated. These are options that the panel feels are usually appropriate as initial treatment for a given situation. Treatment of choice, when it appears, is an especially strong first line recommendation (having been rated as 9 by at least half the experts). In choosing between several first line recommendations, or deciding whether to use a first line treatment at all, clinicians should consider the overall clinical situation, including the patient s prior response to treatment, side effects, general medical problems, and patient preferences.

Second line treatments are reasonable choices for patients who cannot tolerate or do not respond to the first line choices. Alternatively, you might select a second line choice as your initial treatment if the first line options are deemed unsuitable for a particular patient (e.g., because of poor previous response, inconvenient dosing regimen, particularly annoying side effects, a general medical contraindication, a potential drug-drug interaction, or if the experts don t agree on a first line treatment).

For some questions, second line ratings dominate, especially when the experts did not reach any consensus on first line options. In such cases, to differentiate within the pack, we label those items whose confidence intervals overlap with the first line category as high second line.

Third line treatments are usually inappropriate or used only when preferred alternatives have not been effective.

From Survey Results to Guidelines

After analyzing the survey results and assigning ratings, we then use the experts recommendations to create user-friendly guidelines. The guidelines are organized so that clinicians can quickly locate the experts treatment recommendations. We usually begin with choice of initial treatments and then go on to describe options for patients who do not respond to the initial treatments. Note that, whenever the guideline gives more than one treatment in a rating category, we list them in the order of their mean scores.

Limitations and Advantages of the Guidelines

These guidelines can be viewed as an expert consultation, to be weighed in conjunction with other information and in the context of each individual patient-physician relationship. The recommendations do not replace clinical judgment, which must be tailored to the particular needs of each clinical situation. We describe groups of patients and make suggestions intended to apply to the average patient in each group. However, individual patients will differ greatly in their treatment preferences and capacities, their history of response to previous treatments, their family history of treatment response, and their tolerance for different side effects. Therefore, the experts first line recommendations will certainly not be appropriate in all circumstances.

We remind the readers of several other limitations of these guidelines:

  1. The guidelines are based on a synthesis of the opinions of a large group of experts. From question to question, some of the individual experts would differ with the consensus view.
  2. We have relied on expert opinion precisely because we are asking crucial questions that are not yet well answered by the literature. One thing that the history of medicine teaches us is that expert opinion at any given time can be very wrong. Accumulating research will ultimately reveal better and clearer answers. Clinicians should therefore stay abreast of the literature for developments that would make at least some of our recommendations obsolete. We will continue to revise the guidelines periodically based on new research information and on reassessment of expert opinion to keep them up-to-date.
  3. The guidelines are financially sponsored by the pharmaceutical industry, which could possibly introduce biases. Because of this, we have made every step in guideline development transparent, report all results, and take little or no editorial liberty.
  4. These guidelines are comprehensive but not exhaustive; because of the nature of our method, we omit some interesting topics on which we did not query the expert panel.

Despite these limitations, these guidelines represent a significant advance because of their specificity, ease of use, and the credibility that comes from achieving a very high response rate from a large sample of the leading experts in the field.

No set of guidelines can ever improve practice if read just once. These guidelines are meant to be used in an ongoing way, since each patient s status and phase of illness will require different interventions at different times. Locate your patient s problem or your question about treatment in the Table of Contents and compare your plan with the guideline recommendations. We believe the guideline recommendations will reinforce your best judgment when you are in familiar territory and help you with new suggestions when you are in a quandary.

References

  1. McEvoy JP, Weiden PJ, Smith TE, Carpenter D, Kahn DA, Frances A. The expert consensus guideline series: treatment of schizophrenia. J Clin Psychiatry 1996;57(Suppl 12B):1-58
  2. Frances A, Kahn D, Carpenter D, Frances C, Docherty J. A new method of developing expert consensus practice guidelines. Am J Man Care 1998;4:1023-1029
  3. Kahn DA, Docherty JP, Carpenter D, Frances A. Consensus methods in practice guideline development: a review and description of a new method. Psychopharmacol Bull 1997;33:631-639
  4. Brook RH, Chassin MR, Fink A, et al. A method for the detailed assessment of the appropriateness of medical technologies. International Journal of Technology Assessment in Health Care 1986;2:53-63
  5. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC:American Psychiatric Association; 1994