Understanding Bayes’ Theorem: Updating Beliefs with Evidence

📚 Topic: Introduction to Philosophy

Bayes’ Theorem is a standard result from probability theory that formalizes how rational belief should change when new information becomes available.

It is not just a mathematical formula.
It captures a general principle of reasoning:

beliefs should be updated in light of evidence, but in proportion to how strong and how informative that evidence actually is.

This post works through that idea carefully, using concrete examples.

---

🧠 What Is Bayes’ Theorem?

Bayes’ Theorem gives a rule for calculating the probability of a hypothesis after observing some evidence.

It connects three things:

• what you believed before seeing the evidence
• how expected the evidence would be if the hypothesis were true
• how common that evidence is overall

Formally:$$P(H \mid E)=\frac{P(E \mid H)\times P(H)}{P(E)}$$P(H∣E)=P(E)P(E∣H)×P(H)​

Where:

• P(H | E) — the probability of the hypothesis given the evidence (the updated belief)
• P(H) — the prior probability of the hypothesis (what you believed before)
• P(E | H) — the probability of observing the evidence if the hypothesis were true
• P(E) — the overall probability of observing the evidence at all

A useful way to read the formula is:

start with how plausible the hypothesis already was,
strengthen or weaken it depending on how well it predicts the evidence,
and scale that by how common the evidence is in general.

---

🧪 Example 1: Symptoms and a Rare Disease

Suppose you wake up feeling sick and notice symptoms that seem to match a rare disease called Hypothetica.

You naturally ask:

“Given these symptoms, how likely is it that I have the disease?”

Assume the following:

• P(H) = 0.0001
  (1 person in 10,000 has the disease)
• P(E | H) = 0.95
  (if someone has the disease, they almost always show these symptoms)
• P(E) = 0.01
  (1% of people show these symptoms for various reasons)

Using Bayes’ Theorem:$$P(H \mid E)=\frac{0.95\times 0.0001}{0.01}=0.0095$$P(H∣E)=0.010.95×0.0001​=0.0095

So the updated probability is:

• 0.0095, or 0.95%

What matters here

Even though the symptoms are very likely given the disease, the disease itself is extremely rare, and the symptoms occur for many other reasons.

Bayes’ Theorem forces both facts to be taken into account at the same time.

---

🎯 The Base Rate Insight

A common mistake is to confuse these two probabilities:

• P(E | H) — how likely the symptoms are if you have the disease
• P(H | E) — how likely you have the disease given the symptoms

They are not the same.

Ignoring how rare something is (its base rate) leads to overconfidence.
Bayes’ Theorem makes that mistake explicit and correctable.

---

📦 Example 2: Hoodie and Package Theft

Now consider a different case.

You see someone wearing a hoodie near your porch shortly after a neighbor warned about package theft.

Let:

• H = “This person is a package thief”
• E = “This person is wearing a hoodie”

Assume:

• P(H) = 0.001
  (1 in 1,000 people are package thieves)
• P(E | H) = 0.95
  (most thieves wear hoodies)
• P(E) = 0.3
  (30% of people wear hoodies)

Bayes’ Theorem gives:$$P(H \mid E)=\frac{0.95\times 0.001}{0.3}=0.00317$$P(H∣E)=0.30.95×0.001​=0.00317

That is about 0.3%.

What this shows

Wearing a hoodie slightly increases suspicion, but not nearly as much as intuition might suggest.

The evidence is weak because hoodies are common.
Bayes’ Theorem prevents overreaction to familiar-looking “signals.”

---

🧠 What Bayes’ Theorem Is Really Doing

At a conceptual level, Bayes’ Theorem is about updating, not jumping.

• Evidence should change your belief
• but the size of that change must be constrained by:
  • how rare the hypothesis is
  • how common the evidence is

Strong-looking evidence can still lead to small probability shifts when base rates are low.

---

🍪 Example 3: The Missing Cookie

There are 20 coworkers. One cookie is gone.

Before any clues:

• P(Sarah did it) = 1/20 = 5%

This is your prior.

Now you learn:

“The person who took the cookie is left-handed.”

Suppose there are 4 left-handed coworkers, including Sarah.

What changes?

• Sarah was 1 of 20 possible suspects
• she is now 1 of 4 plausible suspects given the new evidence

So her probability increases from 5% to 25%.

This is Bayes’ reasoning in informal form:

• start with a baseline
• incorporate new information
• restrict attention to the group consistent with that information

The belief changes, but it does not become certain.

---

📌 What to Remember

• Bayes’ Theorem formalizes rational belief updating
• It combines prior belief with new evidence
• Rare hypotheses remain unlikely unless evidence is unusually specific
• The goal is not certainty, but better calibrated confidence

Even without the formula, the core idea is this:

new evidence should shift what you believe
but only as much as the evidence actually justifies.

🎶 Use This Song to Memorize It

🎧 While studying this, the core definitions were turned into a short song as a memory aid.
The song doesn’t add content, it simply repeats the same ideas in another form.

Lyrics are included below so you can read, sing, or listen along if repetition helps.

🎤 Song Lyrics:
(Sing, read, or hum along, repetition helps!)

Update The Odds

Bayes’ theorem shows me how
To change beliefs, here and now
When I learn a brand new clue
My old belief updates too

Not yes or no, not black or white
Just raise or lower odds at night
I keep the base, adjust the guess
Bayes gives structure to the mess

🧠  Prior belief, then evidence
How likely is that clue if the case makes sense?
Multiply both, then don’t forget:
Divide by how likely the clue is... you’re set

Say a cookie has gone missing
Lots of kids, but only one did it
You hear the thief wore red shoes
Sarah’s one, but so are a few

If red shoes are common gear
The clue’s weak, the odds stay near
But if red shoes are super rare
Then Sarah’s chance climbs in the air

🧠  Prior belief, then evidence
How likely is that clue if the case makes sense?
Multiply both, then don’t forget:
Divide by how likely the clue is... you’re set

Don’t confuse these two, they’re not the same:
P of E given H - not P of H given E’s name
If you skip the base rate, you might fall
That’s the base rate fallacy, it fools us all

Prior belief, then evidence
Bayes helps you think with intelligence
Odds go up, or maybe down
One clue alone won’t wear the crown