For the Love of Evidence

Over Xmas I listened to Serial, and had a few thoughts about it. Post Xmas I have been catching up on Undisclosed, the pseudo? Spiritual? Something “sequel” to Serial, which follows the exploits of Rabia Chaudry and her friends, as they get to gripes with the case of the murder of Hae Min Lee by Adnan Syed. 1

Both Serial and Undisclosed are interested in their use of evidence. Serial takes the journalistic avenue of searching for the “smoking gun”, which will either show Adnan to be clearly innocent or guilty. Undisclosed takes the more lawyerly approach of trying to show that the inferences the prosecution made in the case are clearly unsound. Both podcasts are about the evidence, but they privilege different kinds of evidence.

Evidence is awkward. “Smoking guns” – pieces of evidence which uncontroversially support one and only one explanatory hypothesis – are rare. Most “smoking guns” end up being examples of evidence which at best strongly supports one hypothesis and at worst supports one hypothesis only if a certain number of assumptions are taken to be true. Many things which are called “smoking guns” aren’t; the evidence everyone says showed Richard Nixon was guilty with regards to Watergate – the missing Oval Office recordings – only look conclusive if you assume the lapse in the recording is relevant to the Watergate Affair (and not, say, a discussion about Nixon covering up the existence of a space navy).

Serial is very much a journalist’s very long form take on Adnan’s presumed innocence. Sarah Koenig ums-and-ahs over whether she thinks Adnan can really be guilty (relying heavily, I should add, on just how nice and normal he seems to her on the telephone), eventually claiming that she couldn’t have convicted him should she have been on the jury. Her reason: the State’s case against Adnan doesn’t survive scrutiny because there is no decisive which supports the claims the Prosecution made.

The argument presented by Koenig in Serial for Adnan’s likely innocence is interesting precisely because the evidence used by the Prosecution did, in fact, lead to a verdict of guilty; Koenig and her producers end up claiming that it’s not so much the evidence which decided Adnan’s guilt but, rather, the way in which it was selectively cited. Koenig and co. do not go so far as to allege misconduct from the police and the courts in this matter; it’s entirely plausible for a listener of the podcast to think “That’s just how things were done in Baltimore back then.” Which is to say that the things which allowed certain parts of the evidential record to be unquestioningly presented by the Prosecution was just a feature of the system at the time.2

The argument in Serial, then, is that the evidence was the evidence, but it did not support the prosecution’s take. The fact it was not challenged by the defence was a fault of the Defence. That’s a different take from that in Undisclosed. Undisclosed asserts some kind of conspiracy by the police. Or, at least, some of the hosts toy with the idea without necessarily ever stating it outright.3

Undisclosed takes Serial as a starting point for the analysis of the inadequacies of the prosecution of Adnan Syed, but rather than looking for the “smoking gun”, Chaudry and co. look at the inferences the police made in the case. They argue not only are these inferences wrong (such as how they rely on cellphone pings to track Adnan on the day they suspect the murder took place), but that the police knew of these issues; the investigating officers were not blithely ignorant of the inconsistencies but, rather, knowingly working around them in order to secure a conviction.

Undisclosed – if you will excuse a cliche – goes for the jugular: the police not only decided early on that Adnan was the killer, but they quite deliberately coached witnesses to ensure that he was convicted, even in the face of evidence they knew indicated he probably could not have committed the crime the way they claimed it played out.4

The idea that the police secure convictions unethically is uncontroversial.5 What tends to be said these days is that it doesn’t happen as often as it did. In that respect Chaudry and co. thesis is not remarkable, especially since as lawyers they are probably much more sensitive to, and aware of, cases where the police do not play fair. However what they claim is, in the end, a conspiracy theory: the police and the prosecution knowingly kept from both the defence and the jurors evidence that would have cast doubt on Adnan’s innocence, and they have continued to maintain that the conviction is secure to this day.

Chaudry and co. come to this conclusion of conspiracy via an analysis of the way the prosecution sorted and presented the evidence. The hosts of Undisclosed are interested in how inferences were drawn from the available evidence, and how said inferences were either followed up, ignored or presented. Unlike Koenig, Chaudry and co. are willing to entertain the notion that the authorities had an agenda.6

So, why is this interesting, and why is this of interest to me? Well, because evidence is something which is poorly understood by most people. People fetishise evidence, and some professionals fetishise particular kinds of evidence.

Take Serial. Koenig really wants a “smoking gun” which shows either Adnan is guilty or innocent. She does not find one, and so ends up saying that if she had been on the jury, then she could not have convicted him. That is to say she finds in favour of reasonable doubt (an understandable position), but can’t shake the idea that maybe he was guilty after all. The thing which would sway her belief – the “smoking gun” evidence – is just not available.

The Undisclosed discussion of evidence is all about poking holes in the official theory of the murder of Hae Min Lee. What the hosts are interested in is the hypotheses you can reasonably infer from other evidence, which is not just a kind of evidence, but the kind of evidence we typically rely on a day-by-day basis. For example, you think your child is eating all the biscuits, and your only evidence is that whenever you want a ginger nut there aren’t any. You know you bought some earlier, but now the kids are back from school, and there are none left. From that you infer “The kids ate the biscuits” and you charge them with this most heinous of crimes.

For the hosts of Undisclosed the issue is not that they have a smoking gun, but rather that the inferences drawn from the available evidence in no way support the idea Adnan could have committed the crime the way the State claimed he did.7 For the people behind Undisclosed, the case falls over because the details the investigators inferred from the morass of evidence fail to survive scrutiny.

On one level this is good, because it shows up issues in the investigation, and casts doubt on the security of Adnan’s conviction. Yet it also doesn’t provide evidence for an alternative explanatory hypothesis of who committed the crime.8 As such, the evidence which undergirds the analysis in Undisclosed can lead to a “So?” kind of response. “Sure”, people might say, “the case against Adnan looks shaky, but who else could have committed the crime?”9 That is to say, if you understand any explanation is going to rely on a certain amount of selective evidence use, and that all explanations of social phenomena (like murder) are going to have loose ends, then apparent inconsistencies can be waved away as being part of the difficulty of ever getting the full story of what really happened.

Now, this is to be expected; the hosts of Undisclosed are lawyers and all they need show is that there is reasonable doubt about the theory Adnan murdered Hae. However, an awful lot of the theory which drives their approach to instilling a feeling of reasonable doubt is about bolstering the theory the police and prosecution conspired to get a guilty verdict. Admittedly, they present a lot of evidence to support this theory, from weird plea deals with a key witness, previous cases in which the investigating officers obtained insecure convictions, the way in which evidence in favour of the defence was never given to Adnan’s legal team… It certainly looks suspicious. However, some of that suspicion comes simply from believing in the innocence of Adnan and seeking to explain away the guilty verdict.

Now, assumptions are crucial when making inferences; evidence is only evidence with respect to surrounding theories. The hosts of Undisclosed are practicing lawyers and they know how to show up the problems with rival theories. However, their strategy with dealing with the official theory is – to my mind – as frustrating as Sarah Koenig’s eventual conclusion. They present grounds for reasonable doubt, based upon good evidence of a conspiracy. Yet they downplay response to their conspiratorial claims either by simply not recognising that there could be non-conspiratorial explanations for some features of the case, or claiming such non-conspiratorial explanations are prima facie unlikely. It seems the rhetoric of defending Adnan is more important than a careful analysis.

And there’s the rub; maybe it should be. If you were really convinced that an injustice had been done, would you pepper your argument with “But, of course…” or would you do your best to persuade others of your righteousness? I wager it’s the latter, and that, just maybe, I’m judging the epistemic merits of podcasts when, really, I should be much more interested in their rhetoric.

Notes

  1. Adnan was accused, and eventually convicted, of the murder of Hae Min Lee back in February, 2000. The evidence used to convict him mostly consisted of a congruence between the sole key witness in the case, Jay Wilds, and a series of cellphone pings which placed Adnan in the vicinity of where Hae Min Lee’s body was buried on the day it was alleged she was murdered.
  2. Both Serial and Undisclosed emphasise that things have changed in the intervening years.
  3. I should point out that I’m only halfway through the current crop of Undisclosed episodes, so this issue of conspiracy might get addressed more fully in a later episode.
  4. Not only that, but due to the success of Serial, the key witness and prosecutors have slightly changed their stories; see this interview in The Intercept
  5. Undisclosed spends an entire episode – almost an hour and a half – detailing previous cases of dubious and eventually overturned convictions which were investigated by the detectives who charged Adnan.
  6. One of the interesting aspects of Serial was Koenig’s claim that she thought it was unlikely the pursuit of Adnan as prime suspect could have been motivated by racism. Chaudry and co. seem to think it’s quite possible Adnan’s arrest and then denial of bail was – given what was said in court – very racist indeed.
  7. A constant issue I have with Undisclosed is how they move from “The State’s case is implausible” to the attendant claim “Adnan did not murder Hae Min Lee”. It’s not at all obvious that the details of the prosecution case being at fault means the general thesis is false (although it does show it’s hard to believe). It’s quite possible Adnan killed Hae Min Lee, the police somehow know he killed Hae Min Lee, but the actual details are so fuzzy that no clear story has emerged. But that’s by-the-by, really.
  8. Whilst the hosts of Undisclosed do think that Jay Wilds is a suspicious character, as of yet they are not claiming he was the real murderer.
  9. I do have an issue with the strict literalism the makers of Undisclosed engage in; if someone says in their testimony “it happened a year ago”, then they take it that it pretty much happened 12 months ago. This kind of “If someone says x, then they only mean x” is a common trope in reasoning, which doesn’t really reflect the fact most of us are not all that precise when we speak. Most people gesture towards detail, as opposed to give a frank and full accounting, so the idea we should take all testimony literally does, I think, in many cases suggest radical inconsistencies where only minor details are at stake.

Another update on North Head: Am I in danger of changing my mind?

A few years ago, Martin Butler provided me with a copy of his book, “Tunnel Vision”, which I reviewed here. Last year Martin updated his book (The front cover calls it “An Explosive Update”) which I’ve now read and am in the process of reviewing. I think it’s a better book now than it was a few years ago, although I’m not entirely convinced by all of Butler’s claims. That is by-the-by, however, because earlier this week I met Martin at the Torpedo Yard cafe, at the base of North Head, and I came away from that meeting a little swayed in my thinking. I’m not saying I’m now a firm believer in the existence of a cover-up to hide decaying ammunition in one of the country’s most expensive suburbs. I am, however, willing to go so far as to say there are some anomalies in the public record which suggest there is more to the North Head story that certain authorities would have us believe.

I’ve been mulling this over the last few days. My good friend and colleague, Lee Basham (of South Texas College) has long argued that I should not have closed my book with a declaration that the best conspiracy theory about the events of 9/11 is the “Al-Qaeda was responsible” theory. Rather, Lee thinks I should have just provided the methodology for the analysis of conspiracy theories and left the generation of conclusions to those who would employ my analysis. His argument was that my analysis does not need to be hitched to any particular claim to be useful. The North Head issue is a good example: when Martin and I met in person for the first time one of the first things he said to me was “So, you’ve been a skeptic about all of this for a very long time, haven’t you?”

Being known as a skeptic of something has, in the past, been something I’ve celebrated and shouted to the rooftops. However, now I think that it can be a bit of a millstone. I have no issue in changing my mind; I went from being a very devout theist to an atheist (of the “There’s no good proof for the existence of the gods, so I’m not going to believe in them until there is” variety), and I went from being a racist to a non-racist. I even started out writing a PhD on the warrant of conspiracy theories believing that we had grounds to claim said theories were prima facie unwarranted, and we’ve all seen where that got me.

So, being known as a skeptic of the view there might be something more to the North Head story can be a bit of problem. This is because sometimes people take skepticism to mean “Here is my view on x, and you are stupid to believe otherwise.” However, my skepticism of the Hidden Tunnels conspiracy theory has always been about a lack of good evidence (and there’s a phrase which needs careful unpacking). Meeting with Martin and seeing and hearing about some of the new evidence he has brought to light, has shifted my thinking.1

Here’s a quote from the end of my book.

When inferring to an explanation, ordinary reasoners might fail to consider:

1. The extent to which the available evidence that the phenomenon being explained renders the hypothesis probable (the posterior probability),
2. The degree to which the hypothesis is independently likely (the prior probability),
3. The likelihood of the hypothesis, relative to the other hypotheses being considered (the relative probability) or
4. The possibility that there are some worthwhile hypotheses which have not been considered.

Three of these issues are to do with how we consider the probability of a given hypothesis. The fourth is about the failure of ordinary reasoners to consider other worthwhile hypotheses.
(Dentith, M. R. X. ‘The Philosophy of Conspiracy Theories’, Palgrave Macmillan, 2014, p. 147)

Martin’s research, at least for me, means there is new evidence to consider. His new evidence consists of additional information about the military use of North Head and how North Head fitted into the military command structure across Aotearoa. Not just that, but he also has some interesting examples of inconsistencies in official correspondence. Some of this evidence changes the posterior probability of some version of the non-official, conspiracy theory because it not just opens up holes in official reports and statements from Ministers and senior personnel, but it also shows that people where either very lackadaisical with the truth or that they lied to either the public or members of the Government.

Now, I say “some version of the non-official, conspiracy theory” quite deliberately; if I can going to concede that it seems there is more evidence for a cover-up than I initially thought, that doesn’t require me to believe a specific conspiracy theory that says, for example, that there is decaying ammunition deep within North Head. I can believe there is evidence for a cover-up about something without having to believe something about what is being covered up. But, and this is important, I think Martin’s research increases the likelihood that some version of a conspiracy theory about North Head is true. The question is, does it change it such that it is the most probable explanatory hypothesis?

Obviously there is a tension between the posterior and relative probability of a set of hypotheses; as the posterior probability of some version of, in this case, the conspiracy theory goes up you should expect it to become relatively more probable than some other hypotheses for the same event. This is where I am at right now: the new evidence certainly increases the posterior probability of some conspiracy theory about North Head, but has the relative probability of the rival, official and non-conspiracy theory been lowered, such that some version of the conspiracy theory is now the most likely explanation? For the moment, I have no concrete answer. My gut tells me that the official theory is still the most likely explanation, but it’s not as likely (to my mind) as it was a week ago. But why trust my gut on this, rather than go back and re-examine the evidence?

Which is what I am going to do. More on this soon.

Notes

  1. I keep wanting to say things like “a little bit” and I’m honestly not sure whether that’s because I’m simply resistant to changing my mind on some of the issues.

Chapter 5 (or 6 or 7) – The Inference to the Existence of a Conspiracy

The final chapter of my thesis, which is number 7 (in the file directory), number 6 (according to the PDF) or number 5 (as my supervisors would have it) is my third peg, so to speak, in why I think we have a prima facie case for our suspicion of conspiracy theories; they are typically examples of Inferences to Any Old Explanation (which you might know better as “Just So” stories, ala Mr. Kipling1 ).

Originally I was going to base this chapter on large chunks of Peter Lipton’s book on the Inference to the Best Explanation (which, funnily enough, is called “Inference to the Best Explanation” and was published in 2004 (the second edition) by Routledge), and not just because he specifically mentions (and then glosses over) conspiracy theories on page 60.

“Perhaps some conspiracy theories provide examples of this. By showing that many apparently unrelated events flow from a single source and many apparent coincidences are really related, such a theory may have considerable explanatory power. If only it were true, it would provide a very good explanation. That is, it is lovely. At the same time, such an explanation may be very unlikely, accepted only by those whose ability to weigh evidence has been compromised by paranoia.”

Lipton runs a contrast between the loveliness of explanations (just how powerful they are as explanatory hypotheses, essentially) and the likeliness of such explanations (i.e. just how probable is the explanatory hypothesis); he thinks2 that conspiracy theories were good explanations only in the lovely sense as they were unlikely.

Lipton doesn’t come back to conspiracy theories, which is useful for me, because, in some important respects, it rather gives away the kind of analysis I have in kind for chapter 5/6/7.

As I wrote, my original intention was to develop Lipton’s view with specific respect to conspiracy theories, but that is no longer the case. Instead, I am developing the Inference to Any Old Explanation analysis that Dr. Jonathan McKeown-Green and I worked up for our Critical Thinking course (PHIL105 to the fans) at Auckland. The Lipton material is useful in talking about when the inferential practices of epistemic agents can be said to `go right’ but my analysis is really about when such practices `go wrong.’ It is much easier to work up a bespoke philosophy/epistemology than it is to try and make the work of someone else fit your particular analysis (ask me about my MA thesis for detailed reasons as to how that doesn’t necessarily work out for the best).

Still, I should point out that whilst I was writing this post I realised that there was a particular part of Lipton’s analysis of IBE (as the cool kids these days call “Inference to the Best Explanation”) which I could use to fill this particular hashed out section of the chapter (the “%” marks are playing the role of hashes in my LaTeX documents):

%However, we should also be aware that there is a kind of tradeoff between the probability of an hypothesis and the extent to which said hypothesis suggests the explanans.

%[This is the stuff that motivates Bayesianism. We might need to say a bit about it somewhere to help set up this discussion of the Inference to Any Old Explanation.] – Will need my Goldman…

I’m no Bayes scholar; I know how the theory works and the difference between prior and posterior probabilities (which it is important never to confuse), but the specific details… Well, I’d need to spend quite some time with a primer and a notepad to get myself sufficiently up to speed on Brother Bayes and his mathematical theorems. However, Lipton has a gloss on Bayes, since Bayesianism is often trotted forth as a contender for a theory of the Inference to the Best Explanation, and so I might use the Lipton gloss (which I’ve partially written up) after all.

Which goes to show that this new “thesis-centric” blog ethos is already delivering. Huzzah.

Next time: What it is I am actually trying to say in chapter 5/6/7.

Notes

  1. And his marvellous cakes.
  2. Well, thought; Peter Lipton is dead.