able to remind the youths of the reasons for the study and tell them ourselves about the voluntary nature of their participation (i.e., give them our own informed consent). In this particular situation, we asked the supervisor to ask the officer to approach it differently, but we were unable to change the situation. Consequently, we were able to complete very few interviews with youths on his particular caseload, which meant we had a lot of missing interview data from this group. The bottom line, though, is that this probation officer saw our study as a hassle (because his clients were in the control group and not receiving services) and did not buy-in to the value of the research. He remained hard to convince, although we did work to gain buy-in from the other probation officers in charge of the control group in general.
One way to increase staff buy-in is to listen to and address their concerns. This can be challenging because researchers often do the negotiating with higher-level administrators or heads of agencies, but must work directly with the line staff. Avoiding power struggles even within agencies can be delicate. As one example of how we worked with line staff in the SOCP study, we listened to staff express concern that they would have to complete additional forms that we created for data collection (e.g., contact forms indicating how often they contacted the youths as well as why and where) when they were already overwhelmed with work. While we continued to have the staff for the SOCP complete these forms to document staff contact with youth (see appendices B and C), we created an actual stamp with wooden handle and stamp pad for the control probation officers that allowed them to enter the data in their chronological notes in the youths’ files, and the research team then coded the information into contact sheets for data entry (see figure 2). The managers of their units also agreed to allow this information to serve as their records for the typical files, unless other relevant information needed to be added to it. That is, we made it as easy as we could to get the data from staff without imposing too much on their time or daily activities.
Figure 2. Contact stamp for staff with client caseloads in the control (routine probation) group in SOCP. Source: Authors
Gaining buy-in from peripheral agencies or staff (e.g., people who manage clients in the control group or who provide only a few services, such as contracted educational or drug services) may be tough in many projects. We recommend having initial meetings with the group to describe the importance of the study in detail and answer questions, but also maintaining regular, ongoing conversations with relevant staff to ensure all is still going well, monitor their “compliance,” quell any concerns, and so on. If there are not researchers on site, we recommend scheduling regular site visits to ensure research staff can monitor daily activities. It also helps at the outset to (1) agree to measure information that would help them or their own agency and (2) to give them regular feedback on the results. One group that often can understand the research value are agency leaders, because they can be convinced that data results can be useful for selling the program or garnering more resources. Sometimes, especially in paramilitary organizations like many corrections agencies, it is helpful to get buy-in from leaders so they can encourage compliance with the research among their staff. In more than one instance, we had to talk to managers to request they “help” get their staff to collect agreed-upon data, complete forms, follow the study protocol, and so forth.
Researchers conducting program evaluations may also need to convince correctional leaders and staff to use random assignment, which means to randomly assign offenders to experimental or control groups. Using random assignment means that subjects have an equal chance of being assigned to either group. Random assignment can occur by doing something simple, such as flipping a coin, or by using random number generators or a computer program with this capability.
Random assignment is considered the “gold standard” for evaluation research (Boruch 1997; Farrington 2003), but it can be difficult to get agencies or their staff to understand the need or to understand that random assignment is legal and ethical. That is, staff may want to use assessments or staff knowledge of client needs to determine who gets access to special programming. In our experience, for example, SOCP staff often wanted to use their personal knowledge about a youth or experience over time in the field to help determine which youths were assigned to which programming rather than rely on luck of the draw. That is, staff may be personally convinced that particular youths need a new or special program and want to bypass any procedure that would leave that assignment to chance. We have found that two points can be convincing to staff. First, random assignment is the fairest way to put youths into groups, if the program cannot serve all people who need the help it provides. It prevents favoritism and ensures all have an equal chance of being selected. Second, random assignment allows for the strongest results, meaning that agency leaders can put more faith in findings than if it were not in place. They can better sell the results, whatever they are, when asking for more funding, for example. Additionally, in some cases, if we had enough degrees of freedom, we allowed a small number of youths to be sent to the project each year (e.g., fewer than 10 percent) while bypassing the random assignment process. These youths, then, are just not to be analyzed for the study but are served by the program.
In our experiences collaborating with staff, we found that some were easier to work with than others, which we expect is typical of most correctional studies. It was important to get to know the personalities and preferences of both the facility and agency administrators and their staffs, because it helped ease the working relationship as we periodically arrived to conduct research. The key, though, was working within the constraints of the facility or agency while trying to maximize our ability to conduct rigorous research. In other words, we were very willing to compromise on details (e.g., times, dates, places to collect data, and managing other idiosyncrasies) of each facility and its staff as long as it did not jeopardize methodological rigor.
With some staff members and in some situations we could be more open about ourselves (e.g., our religious preferences or personal lives) and with others we felt we needed to be more guarded to encourage a smoother working relationship. In our Florida Faith and Community-Based Delinquency Treatment Initiative project, for example, religious beliefs were part of regular conversation among staff, because faith was such an integral part of the program. This meant we had to be especially attuned to our attitudes and how we expressed them, and we could not show judgment when they expressed religious or even political attitudes with which we disagreed.
In this faith project, we also had to be careful about how we responded to behaviors we personally thought were not appropriate. For example, in one facility, we regularly saw areas in physical disarray, such as unmade beds and clothes strewn around, youth jumping across furniture, and expired food (which together signaled to us a lack of good management), while in others important paperwork was not appropriately stored. In one facility, home addresses were listed on youths’ bedroom doors, which we thought might be a safety issue for the residents. This facility was later shut down for many problems. While we jotted issues such as these in our research notes, we did not express judgment or report these sorts of problems to their agency managers because we wanted to maintain the working relationships and understand the facilities as they were, not just as they were supposed to be. We allowed the quality assurance process within the agency more broadly to manage the agency’s rule enforcement for these issues. Of course, had we seen something abusive or illegal, we would have reported it to the appropriate authorities. If the research team sees something concerning, they should be encouraged to report it to the lead researcher or principal investigator, who may wish to call the university attorneys to inquire about the appropriate course of action (if any) for the researcher.
CONVINCING THE TARGET POPULATION TO PARTICIPATE
Once researchers have obtained agency permission to collect data and secured buy-in from the staff, the next challenge researchers face is often how to convince people to participate. The following is a list of specific tips that have worked for us to build and maintain rapport with participants:
•Make eye contact, be friendly, and
