A Brief Introduction to Bayesian Reasoning, with a Calculator for Day-to-Day Use

Bayes’ Theorem is a short equation for calculating the conditional probability of a hypothesis being true, given some data. Put differently, Bayes’ Theorem allows the question, “I’ve seen these data; having seen them, how much should I believe in my hypothesis?” Bayesian thinking is especially noteworthy for its ability to incorporate background knowledge on a topic. In this way, Bayes accurately reflects the world: if you’ve ever entered into a discussion of politcs with someone who comes into the discussion, e.g., 99% certain that $x$ political party is good/bad/etc., even the most moving of evidence to the contrary will likely move that individual’s certainty only a small amount. On the contrary, for a person who enters into a discussion with less certainty (e.g., 50%), strong evidence, either in support or to the contrary of his or her hypothesis, will likely be more moving to him or her. ClearerThinking.org recently published a 35-minute interactive introduction to the topic.

Bayesian reasoning is power-giving stuff. And it can be used in everyday situations: if one knows or can guess a few background statistics, Bayes’ Theorem allows for objective calculations on how much one should change one’s mind in the face of new evidence, taking into account how certain one was before seeing that evidence, and how likely it would be to see the evidence if the hypothesis were true, and if it were not true.

Bayes’ Theorem is typically written like this:

$$P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$$

where $P(A|B)$ is said, “The probability of A, given B.” This can be re-written with $A$ and $B$ filled in:

$$P(Your~Hypothesis~being~true~|~The~data~you~just~saw) = \frac{P(The~data~you~just~saw~|~Your~Hypothesis~being~true) \times P(The~data~you~just~saw)}{P(Your~hypothesis~being~true,~pretending~that~you~didn’t~just~see~some~new~data)}$$

Below is a quick script that can be used (e.g., with a bash “alias” file, which sets shortcuts for the Unix-style command line) for making decisions in a Bayesian way. It’s based off of Nate Silver’s version of Bayes’ Theorem (printed on line 18 below — it’s the same equation as above, just rearranged). It can be run with python /path/to/this/script.py. I wrote it when I first learned about Bayesian reasoning, and use it occasionally in my daily life when I recognize myself getting anxious about this or that but recognize that I might be unconsciously overusing the availability heuristic.

The availability heuristic is a cognitive bias, following which examples that are easy to bring to mind (e.g., because they are easy to visualize or are emotionally-charged) are misunderstood to be more likely to occur, simply because they are so easy to bring to mind.

#!/bin/python
# The above line just tells the interpreter that this is a python script.

# Jacob Levernier, 2013
# This code is released CC0 (https://creativecommons.org/publicdomain/zero/1.0/) (Public Domain)

# THIS APPLIES BAYES' THEOREM (AS PRINTED ON P. 247 FF. OF NATE SILVER'S BOOK /THE SIGNAL AND THE NOISE/):

print "BAYES THEOREM:"

# (See https://en.wikibooks.org/wiki/Python_Programming/Input_and_output and http://docs.python.org/2/library/functions.html#input)
x=float(input('Make an estimation: How likely (out of 1.00) would you have estimated it would be for your hypothesis to be correct, before seeing this new information (The "prior")? '))

y=float(input('How likely (out of 1.00) do you think it is to see these results if your hypothesized explanation is correct? '))

z=float(input('How likely (out of 1.00) do you think it is to see these results if your hypothesized explanation is NOT correct? '))

posteriorProbability=(x*y)/( (x*y)+(z*(1-x)) )

print "The revised estimate of how likely (out of 1.00) it is to see these results if your hypothesis is correct, given current circumstances, is",posteriorProbability

Say, for example, that we have received a test for a medical disease. Let’s say that we know from the test’s creator that the test has an 80% true-positive rate (i.e., if a person does have the disease, the test will correctly say so 80% of the time, or in 80 out of 100 cases). Let’s say that we also know that the test has a 20% false positive rate (i.e., if a person does not have the disease, the test will incorrectly say that the person has the disease 20% of the time, or in 20 out of every 100 cases). Let’s also say that 5% of the population has the disease, and you’re worried that you may be part of that 5%. If we ran the script above, and entered the above example statistics, we would see the following output:

BAYES THEOREM:
Make an estimation: How likely (out of 1.00) would you have estimated it would be for your hypothesis to be correct, before seeing this new information (The "prior")? 
	.05

How likely (out of 1.00) do you think it is to see these results if your hypothesized explanation is correct? 
	.8

How likely (out of 1.00) do you think it is to see these results if your hypothesized explanation is NOT correct? 
	.2

The revised estimate of how likely (out of 1.00) it is to see these results if your hypothesis is correct, given current circumstances, is 0.173913043478

Whereas a person uninitiated

Notably, medical professionals are often not taught about Bayesian reasoning, and so many times misunderstand statistics like these.
into Bayesian reasoning might see the 80% true positive rate of the test and think, “I have an 80% probability of being afflicted with this disease!” we can see that there’s actually a much lower 17% probability that we have the disease, taking into account that very few people (5 out of every 100 people in the population) have the disease to begin with, and that some people who don’t have the disease will still get a (false) positive result. Bayes’ Theorem is powerful in its ability to incorporate this “prior” knowledge.
In Bayesian terminology, $P(A)$ is often called the “Prior probability,” while $P(B|A)$ is called the “Likelihood.” $P(B)$ is called the “Evidence,” and $P(A|B)$ (which is what we’re trying to figure out) is called the “Posterior probability.”

For anyone trying to figure out how Silver’s version of Bayes’ Theorem matches up with the standard version printed above, the answer is that they’re the same, except that the Evidence term is written out more fully. Using the standard Bayes equation, our math for the example above would be as follows: $\frac{.8 \times .05}{(.05 \times .8)+(.95 \times .2)} = 0.173913$. The Evidence term here is still the probability of getting a positive result, just written out:

$$(Probability~of~having~the~disease \times Probability~of~Positive~Test~Result~Given~Having~the~Disease) \
+ \
(Probability~of~NOT~having~the~disease \times Probability~of~Positive~Test~Result~Given~NOT~Having~the~Disease)$$

Note: The code in this post was also recently posted at the University of Oregon’s student-run Bayes’ Club blog.

Related

Next
Previous