What is evidence?

We talk about evidence here all the time but what do we really mean? What things do or do not qualify? In this post I hope to discuss this idea and see if we can’t clarify some of those concepts. Like anyone looking for a definition our first stop is to grab a dictionary. Here’s what Mirriam-Webster says:

The thing about dictionaries is that, by design they are descriptive. Not prescriptive. Definitions are built by examining the context in which words are use. What gets included in a dictionary is based on several factors — primarily age and frequency. This means that when using a dictionary definition as part of an argument, the only thing you can really do is justify your use of a term. You can’t use it to exclude a definition. At worst you could say that a definition which differs from the dictionary is idiosyncratic. Even so, it’s worth pointing out that the standard dictionaries use is pretty conservative.

Consider the partial phrase “doesn’t jive with me” and a similar phrase “doesn’t jibe with me” both could be used by an English speaker to communicate that something presented to them doesn’t quite fit with their understanding, or tastes. The former use is arguably not only well understood but far more frequently used (Google phrase search shows 72,000 uses of “jive” vs 17,000 uses of “jibe”) but dictionaries still only mention the “jibe” as posessing this meaning. Merriam-Webster actually wrote an article talking about the differences between these terms (as well as “gibe”) the author clearly recognizes the modern use of the term “jive” — it’s just that sense has only been in use for about 80 years.

So, to get the nuance of any modern word we need to look at the term in a modern context. Something like the following:

Bob: I think I'll get vaccinated tomorrow.
Dob: I wouldn't do that. There's a lot of evidence saying that vaccines will harm you.

Without concerning ourselves with the truth of either statement, lets look at whether either sense 1a or 1b fully captures what is being said. I'd argue that 1a isn't quite strong enough. It is reasonable that a person who has an a discolouration on their arm at the same time that they are infected with a microorganism to suspect that the discolouration could be evidence of the infection. Most would also agree that if we examined many other individuals who had this disease and were to carefully control for other variables that the discovery of a high degree of correlation between the discoloration and the infection would suggest that the discolouration is evidence of this kind of infection.

However, if instead of examining several other people. We simply re-examined the same person repeatedly. Few would conclude — even with similar degrees of control and similar numbers of examinations that the discolouration is necessarily evidence of the infection.

Why not? This may seem obvious but if the discolouration and infection are unrelated events. We would expect that both would persist to some degree over multiple examinations. Especially if the examinations are done in rapid succession. Which means that checking the same person repeatedly doesn’t add much (if anything) to the likelihood that these two events are related. Experimental science has a name for this: pseudoreplication. When the same event occurs more than once in the same sample.

So 1a isn’t quite strong enough because while an outward sign doesn’t necessarily imply evidence.

Looking now at sense 1b, it almost seems a little too strong. “Proof” has a host of definitions but most of them indicate something of considerable strength (Somewhat ironically the dictionary also gives “evidence” as a definition of “proof”). Dob’s statement above however implies that evidence can be greater or lesser in magnitude. We can have a lot or a little. Assuming this distinction is meaningful it’s reasonable to believe that evidence can be both strong and weak. If so, then 1b doesn’t quite allow for that.

So, given the context that we use the term “evidence”. I think there is a better definition:

Evidence is anything that shifts the likelihood of some given claim or hypothesis.

This definition fits well with the conversation between A and B. Since it both requires a shift in direction but no specific level of strength.

Hey! That’s not evidence.

Consider the following exchange:

Bob: My friend Ted said that aliens landed in his backyard.
Dob: That's not evidence. Ted could say anything.
Bob: It is evidence and you are in denial.

I see conversations like this all the time and while it appears that Bob and Dob are arguing about what is or is not evidence. However, often what is actually happening is they are disagreeing as to what is sufficient evidence to accept a premise — in this case Bob’s claim that aliens landed in Ted’s backyard. Bob starts by claiming that something is true by virtue of some evidence. — this implicitly carries the claim that the evidence is sufficient to accept the assertion as reasonable if not true. Dob counters by saying that “That’s not evidence.” while it sounds like Dob is denying that Ted’s word is evidence. The next sentence makes it clear that what they are stating is that Ted’s word does not represent sufficiently strong evidence.

Bob’s response to Dob then is interesting. Instead of focusing on the strength of Ted’s word they shift the discussion entirely from what is sufficient evidence to whether Ted’s claims fall under the definition of evidence. People might blame Dob for this shift in subject, but Dob’s response utilizes a common English shortcut where very low values are expressed as equivalent to zero. Bob’s quibble with this makes no more sense than quibbling with someone who says “I’m not doing anything today.” — when clearly they are breathing, perspiring, etc…making Bob’s claim that Dob is “in denial” a strawman (and Bob’s argument is equivocation) as Dob is not actually stating that Ted’s statements do not fall under the definition of evidence but rather that they are insufficient.

That said: Dob could have stopped Bob cold simply by forcing Bob to qualify or quantify their evidence.

Dob: Can you demonstrate that Ted’s testimony is more likely correct than incorrect?

When we are mindful that evidence requires more than just a direction but also a change in likelihood helps us focus the conversation on how we determine the size of that change. Which is far more important than just pointing out that a change might occur. Since in the final analysis what matters is the totality of evidence. Which can only be understood if we know both the direction and size of all the evidence presented.

Then is anything evidence?

No, while it may not always be clear how much something changes the likelihood of our hypothesis. We should be able to identify or argue that something does. There are two notable exceptions:

Pseudoreplicaiton: We’ve already discussed this. Evidence can not be double counted toward establishing the same hypothesis. This incidentally is one (of the many problems) with collecting anecdotal evidence. Many books about dubious health cures will publish or aggregate stories about people who have had positive experiences with health cure X. Several websites may also collect these stories too. However, even if these are all true. Anecdotes are not tracked in a way that guarantees they represent specific individuals. What seems like hundreds of experiences could be only dozen. Medical experiments, in contrast track and identify people. Most labs take steps to ensure that the people involved did not participate in other studies.

Ambiguous data: If it is just as likely for event X to support hypothesis A or hypothesis B. Then it does not represent a shift in likelihood between the two. A symptom that happens just as frequently in two different disease pathologies may be an indication that a patient has some illness (quite possibly A or B) but it can’t shift the likelihood toward a diagnosis of A or B.

So, what’s all this mean?

By framing our discussions in a way that recognizes evidence as something that has not just a direction but a value. Is a technique which helps us focus on the important parts of a discussion: What ideas are the most likely and are therefore worth our consideration.

BoredNow

Article URL : https://en.wikipedia.org/wiki/Bayesian_inference