plan.
If you are conducting an evaluative study, you will determine the research design. This design instructs you on the procedures for the collection of your data on client outcome. One design, for example, calls on you to measure a group of clients once before treatment begins and once at the end of treatment. This design measures client progress.
Step 3: Collecting and Analyzing Data
You will collect data according to the relevant protocol. The term data refers to the discrete information that you have, such as the age of each client, the depression score for each client, and so forth. You must have data for each variable in your study. You cannot have a research question that contains age as a variable unless you collect data on age for each study subject.
Your data collection procedure may mean giving a questionnaire to a group of people at one point in time. It might entail the administration of a tool more than once to the same group of people. In this book, you will have exercises where you will collect data from a group of people (perhaps the members of your class) at one point in time. The instrument used will have items designed to measure each of the variables in your study. One of the variables may designate the group the respondent is in, so that two groups can be compared.
You will select a statistic for each of your research questions. Some of these statistics will be descriptive in nature, such as a frequency or a mean. Some will be explanatory in nature because the statistic helps you determine if there is a relationship between two variables that cannot be explained by chance (i.e., the data are statistically significant). Later in this book, guidance is provided on how to find the appropriate statistic and how to employ it in the examination of your data.
One of the issues you will encounter in both explanatory and evaluative research is whether your data can be explained by chance. Statistical significance refers to the likelihood that a given set of data would occur by chance. You will see much on this theme later. If your data can be explained by chance, you cannot logically conclude that the data are meaningful. For example, if you find that your clients’ posttest scores for anxiety are better than their pretest scores but you find that the data failed to be statistically significant, you cannot logically declare that you found that the clients gained with regard to reduced anxiety. You cannot do this because you found data that can be explained by chance, so the next time you do the same study, you are just as likely to find that they did not gain as to find that they did. Your data must do better than being the result of chance to be taken seriously.
Step 4: Drawing Conclusions
Based on your data findings, you draw conclusions about your research question. Is this the place for you to offer your opinions about the research question? No! Your conclusions should stick to the data you analyzed. It does not matter how strong your opinion is or how much experience you have had with it. The research study is about the collection and analysis of data in a certain manner using certain measures of certain variables. Your conclusions must adhere to the data analyzed. However, you may offer suggestions regarding future research or your opinion about the strengths and limitations of your research methods. But your opinions on the research question of your study should be kept apart from the study conclusions.
To summarize this presentation on the process of social work research, let’s take an example from evaluative research. The target behavior is depression because you have a group of 11 clients who have entered your special program for the treatment of depression. The research question is whether these 11 clients will have lower depression at the end of the treatment than before. Your method of measurement is the Beck Depression Inventory, a highly tested tool for measuring depression. Your intervention is cognitive–behavioral therapy, which has more positive evidence for the treatment of depression than any other treatment method. The outcome is that the mean score for these 11 clients at the end of the treatment is 40% higher than their scores before the treatment began. These differences between the scores before and after treatment were found to be statistically significant. This means that the gain cannot be easily explained by chance and, therefore, can be taken seriously. You draw the conclusion that cognitive–behavioral therapy was effective in the reduction of depression for this group of 11 clients. However, you are not in a position to generalize these findings to depressed people who were not in your study, because you did not employ a random sample. Instead, you used a convenience sample. You used this type of sample because your primary concern was to find out whether your treatment was effective with this group of clients.
The research process is illustrated in Figure 2.1. In this presentation, a group of at-risk middle school students were randomly divided into two subgroups: one subgroup would get tutoring during the first grading period and the other subgroup would get tutoring in the second grading period. To test the effectiveness of this tutoring intervention, the grades of these two groups would be compared for the first grading period. It would be expected, of course, that the first subgroup (who had the tutoring) would have higher grades during this first grading period than the second subgroup (who did not have tutoring in this grading period). In this example, the tutored group had grades that were 34% higher than the nontutored group.
Figure 2.1 ■ The Research Process
Chapter Practice Exercises
For this chapter, there are two practice exercises. In the first exercise, you will examine a selected aspect of program evaluation for a familiar human service agency. The second exercise calls on you to provide a brief report on certain aspects of an evaluative study you might undertake in a familiar agency.
Practice Exercise 1: Evaluating a Human Service Program
Here are a few parts of the service system that could serve as the focus of one part of a program evaluation. You should select one of these questions and see what you can find in the information available to you at your agency for this program.
1 This program should be serving people in the target population. Important characteristics of this population might include poverty, having young children, being pregnant, being at risk for delinquency, facing death or dying, having marital problems, and so forth.How would you characterize the target population?Does this agency seem to be serving this population? Do you have any data that confirm it? Do you have informed opinions of staff that would support this conclusion?
2 Services should be accessible to those in need. In other words, prospective clients should be able to take advantage of what your agency offers. For example, employed persons may have a difficult time going to your agency before 5:00 p.m. If so, does your agency have service hours after 5:00 p.m.? Certain persons may have transportation problems. Does your agency facilitate transportation in any way?
3 Clients should be appropriately screened at intake to ensure that services are being used by those most in need. What does your agency do to ensure this? Is there any evidence available to you that would suggest that this is true?
4 The program should be documenting outcomes for the program (e.g., higher grades in school, lower depression, higher employment, fewer acts of delinquency, improved marital relations, etc.). Does the agency collect data to evaluate client outcomes?
5 Clients should be appropriately terminated from a service when the need has been met. What does the agency do to ensure that this happens?
6 Services should have an impact on indicators of need, such as rate of child abuse,
