construct.
Inter-rater reliability—Inter-rater reliability asks how similar two or more individuals are when they observe and rate specific behaviors. Psychopathology researchers often rate the emotional responses of children as they engage in various activities. An index of inter-rater reliability would measure how consistent different observers would be in rating the same situation. Historically, one of the motivating factors for developing the DSM classification system was the discovery that different clinicians in different locations watched a film of a person with a mental health disorder and diagnosed it in different ways.
We expect our bathroom scale to show the same reading if our weight has not changed. Likewise, researchers are concerned with the reliability, or consistency of measurement, by assessment instruments.
© iStockphoto.com/tetmc
Assessment Validity
Validity asks whether the instrument we are using is accurate. A clock, for example, could be reliable if it was always 5 minutes fast, but it would not be accurate. Unlike time, for which there is a definition in terms of atomic clocks, psychopathological disorders lack exact unchanging definitions. Although measures such as neuropsychological tests, brain images, and molecular and genetic changes suggest possible variables to be considered, there is currently no exact measure by which to diagnose psychopathology. This makes validity an important but complex concept. Partly for this reason, we consider a number of types of validity.
Content validity—the degree to which an instrument measures all aspects of the phenomenon. If a final exam only had questions from 1 week of the course, it would not be representative of what the students had learned. A variety of psychopathological disorders, such as depression, for example, have cognitive, emotional, and motor components. A measure that just asks if a person felt negative about the future would be seen as a less useful measure of depression than one that also asks about feeling sad and thoughts about suicide and self-worth.
Predictive validity—the degree to which an instrument can predict cognitions, emotions, or actions that a person will experience in the future. If an IQ test in high school predicted college performance, then it would be seen to have predictive validity. Many medical tests such as cholesterol measurements are designed to predict who is at risk for later medical conditions such as cardiovascular problems.
Concurrent validity—the ability of an instrument to show similar results as other established measures of the construct.
Construct validity—the extent that an instrument measures what it was designed to measure (Cronbach & Meehl, 1955). If a test was designed to measure what students learned in a course, then it would be a problem if the test was also sensitive to other factors such as intelligence or the ability to understand test questions asked in terms of double negatives.
Ecological validity—the manner in which data collected has been considered beyond the local context. For example, considering which cultural factors could be influencing the information obtained would improve the ecological validity of the data. This would also hold true in research studies involving mental illness in different cultures. That is, the meaning of a concept in one culture may be different from that in another.
Concept Check
What are five critical areas mental health professionals cover in an initial clinical interview?
Why is it important for mental health professionals to understand the cultural context of an individual’s mental disorder? What kinds of information does the CFI help mental health professionals obtain?
In terms of assessment, what are four types of reliability you should be concerned with, and why?
In terms of assessment, what are five types of validity you should be concerned with, and why?
Models of Assessment
In this section, I will consider different ways of assessing signs and symbols. These range from simply asking a person about his or her symptoms to comparing the person to others who have a similar disorder. Specifically, I will discuss symptom subtests such as the Beck Depression Inventory, personality inventories that are based on psychometric formulations such as the Minnesota Multiphasic Personality Inventory, projective techniques such as the Rorschach inkblot test and the Thematic Apperception Test, neuropsychological approaches such as intelligence tests, and neuroscience approaches.
Symptom Questionnaires
At times, it is important to know what a person’s symptoms are and how that person may compare with others in terms of reporting these symptoms. A variety of questionnaires have been developed that focus on particular sets of symptoms such as those associated with pain, sleep disorders, anxiety, and depression.
Beck Depression Inventory (BDI): a questionnaire useful for determining the level of depressive symptoms that a person is reporting
The Beck Depression Inventory (BDI) has been used in both clinical and research settings to assess symptoms associated with the experience of depression (A. T. Beck & Beck, 1972). The BDI has 21 items, each of which is presented in a four-choice format where the individual is asked to indicate which choice best fits his or her current experience. Here is an example:
I am not particularly discouraged about the future.
I feel discouraged about the future.
I feel I have nothing to look forward to.
I feel the future is hopeless and that things cannot improve.
A questionnaire such as the BDI is useful for determining the level of depressive symptoms that a person is reporting. Given that the measure has been in use for more than 40 years, there is considerable clinical and research data available in terms of level of depressive severity. The measure is also useful for noting changes in depression level during various types of treatment. During psychotherapy, for example, the measure could be given weekly to document changes in depressive experiences. A newer version of the scale (BDI-II) was developed in 1996 in response to the publication of DSM–IV, which changed a number of the diagnostic criteria for depression.
Personality Tests
For at least the past 2,000 years, there has been an understanding that individuals have a particular style for relating to the world and others. At the beginning of the last century, the personality styles of introversion and extraversion were studied. There was also an effort to examine the relationship between personality styles and psychopathology. A number of questionnaires have been developed to this end. One of the best known of these is the Minnesota Multiphasic Personality Inventory (MMPI).
Minnesota Multiphasic Personality Inventory (MMPI)
The Minnesota Multiphasic Personality Inventory (MMPI) is an assessment measurement of personality traits that is composed of more than 500 items of a true–false nature. The person taking the test simply indicates yes or no to statements such as, “I have trouble falling asleep.” The test was developed in an interesting manner.
