first half contained features associated with Debbie’s identified style and the second half contained the closing features associated with Jamie’s previously identified style.
Further to this, scattered along the time series of disputed emails, a substantial number of texts could not be attributed on the basis of the feature set. This is primarily because they were too short and therefore did not contain any of the features identified. However, aside from the first two texts in the series, all of the texts were either clearly attributable to Jamie Starbuck or not attributable to either author due to lack evidence or mixed evidence. After May 3, 2010, we found no texts with features that were inconsistent with Jamie’s style.
Overall, we therefore found strong evidence that the style of the disputed emails from this date was substantially more similar to the style of Jamie’s known emails than to that of Debbie’s known emails, therefore clearly signaling that Jamie had taken over the account.
EVALUATION OF METHOD
At this stage of the analysis, three aspects of the procedure come to the fore.
First, many of the features derived in the initial analysis of known texts simply did not appear in the disputed material. This is a cost of structuring the analysis as we did and might represent considerable wasted effort in identifying and counting features that turned out to be irrelevant to the analysis. We would argue that although this is wasted effort, it is a worthwhile price to pay to mitigate potential bias. Furthermore, some of the features extracted through this process were crucial to our analysis and may not have been identified otherwise. Although this did not figure in this case, this approach also allows for active consideration of features’ non-occurrence in disputed material and whether conclusions against an attribution might be drawn on this basis.
Second, as noted, many texts were not attributable. Features did not occur, or the picture was too mixed, to make an attribution. In the Starbuck case, this did not significantly affect the overall conclusions as there were sufficient disputed emails we could attribute to support writing an evidential report. Also, the date-stamping of the texts allowed us to place the attributable texts in the time series and from this support reasonable (nonlinguistic) conclusions to be drawn about other emails in the timeline.
Third, the combination of stylistic and stylometric methods meant that we minimized the risk of finding new features in the disputed material, which retrospectively might have been detectable in the analysis of known emails. There is a risk in deriving features only from the texts of known authorship that important features might be missed. The whole effort to avoid bias is set against retroactively considering new features not identified at first pass. This should not be done as such post hoc fishing for features to further bolster an analysis once a conclusion is partially drawn is, by definition, biased. Once one has made a firm attributive decision, by definition looking for further features to support that decision counts as looking for confirmation.
A case might be made to return to the analysis to look for features to contradict the conclusion, but this would require careful planning and discussion in advance as to how any results might be treated. It is thus important to employ a strategy that minimizes missing features in the analysis of known materials. One consideration for future analyses might be that it would be worthwhile to post hoc search the disputed material for evidence contrary to the conclusion drawn as a check on the attribution, but this would require careful planning and discussion in advance as to how any results might be treated. That was not done in this case.
OUTCOME TO THE STARBUCK CASE
TG wrote a formal expert witness report, explaining the method and crediting JG with his role in the analysis. The opinion was expressed in terms that the later disputed material was consistent with the previous emails of Jamie Starbuck and inconsistent in style with the previous emails sent by Debbie Starbuck. In the first instance, this report was used as part of an application for an international arrest warrant. However, in response to increasingly forceful inquiries from the Nottinghamshire Police, Jamie Starbuck returned home to England voluntarily. He was arrested at Customs and later confessed to murdering Debbie before leaving the country. The timeline he indicated matched with the conclusion of our authorship analysis.
Our evidence was never tested in court, illustrating how investigative forensic analysis does not always, or even often, result in evidence (see Grieve & Woodfield, 2020). Jamie was convicted of murder on the basis of his confession and received a life sentence with a minimum term of 30 years in prison. Debbie’s body has never been found. The court heard evidence that Jamie dismembered and burned her body to hide the evidence of murder. He has not cooperated with police in disclosing the location of any remains.
REFLECTION ON THE ATTEMPT TO MITIGATE BIAS
The design of our analysis in this case was a first for both of us. There are clear advantages in working as a pair with one forensic linguist adopting the role of “case manager” and the other as “analyst.” Both these roles require expertise and understanding of issues in authorship analysis work and, by separating out these functions, there is indeed an opportunity to mitigate some of the potential for unconscious bias in the analysis.
The first mitigating aspect of our procedure was the shielding of the analyst from the police briefing in the case. It would be foolish, however, to suggest that this created or could ever create a complete firewall in a case of disputed authorship. JG was fully aware that this was a serious missing person investigation and the emails in the known set, in and of themselves, told some of the story, as is always the case with textual data as opposed to other forms of forensic data. In addition, not only was JG exposed to some of the media reports about the case at the time, but the very fact of the police request for an authorship analysis was also a strong communication that the police believed that Jamie might be writing the disputed emails and possibly that harm had come to Debbie.
A more effective part of the strategy to mitigate bias was the restriction in the flow of text to JG as the analyst. Providing a descriptive, comparative analysis without sight of the disputed material, and locking that analysis before moving to examine the disputed material, strongly reduces the possibility of bias in feature selection. As discussed already, it does come at the cost of time and energy devoted to defining and investigating features that were simply not present in the disputed texts. It is our view that this is a cost worth bearing as the result is not only a more rigorous and certain analysis, but as such also an analysis that is harder to attack as a subjective view in an adversarial court system.
The issue of unconscious confirmation bias in all forms of forensic analysis is serious and present and entirely separate from that of an unscrupulous analyst who, regardless of which approach they take, can easily manipulate the results in the direction they choose. It is incumbent on all authorship analysts to consider how bias will affect their judgments and where and whether it can be designed out. A failure to recognize the risks here and a failure to take action to mitigate these risks is a failure of professionalism in our practice.
What we have reported here are just some possible ways to mitigate the risk of bias, and this separation between analyst and case manager along with a careful consideration of the order of analysis, has now been effectively applied to other cases. It is not always easy to do and there is no one correct method. Nonetheless, the approach in the Starbuck case shows how confirmation bias can be mitigated in authorship analysis and how the results can contribute positively to a case outcome. Risks of unconscious confirmation bias can be mitigated but perhaps never avoided altogether.
NOTES
1 1 This case is wholly complete in the courts
