Quartz 4

Information Theory — Why Is Information Physical?

7 min read

Information Theory — Why Is Information Physical?

In 1948, Claude Shannon published “A Mathematical Theory of Communication” and gave information a precise mathematical form. In 1961, Rolf Landauer showed that erasing a single bit of information must release a minimum amount of heat into the environment. In the 1990s, John Archibald Wheeler proposed that physical reality itself may be made of information — his “it from bit” conjecture. In 2025, physicists experimentally verified Landauer’s principle at the quantum many-body level.

The question “why is information physical?” has graduated from philosophy to experimental physics.

Confidence level: established (Shannon entropy, Landauer principle) / emerging (it from bit, infodynamics)

Shannon Entropy — Information as Surprise

Shannon’s entropy is deceptively elegant:

H = −Σ pᵢ log₂ pᵢ

H measures the average surprise in a probability distribution. If an event is certain (p = 1), it carries no information (log₂(1) = 0). If an event is maximally uncertain (coin flip), it carries 1 bit. The formula looks abstract — but it turns out to be the same formula as thermodynamic entropy (up to a constant: k_B · ln2 instead of log₂).

This is not a coincidence. Boltzmann’s statistical mechanics defines entropy as a count of microstates. Shannon’s information entropy is a count of distinguishable messages. They are the same thing measured in different units.

Key properties of Shannon entropy:

  • Basis-independent: doesn’t depend on what the symbols mean, only their statistics
  • Additive: independent sources add their entropies
  • Maximum for uniform distributions: all symbols equally likely = maximum ignorance
  • Connection to data compression: H gives the theoretical minimum bits needed to encode a message (Shannon’s source coding theorem)

Landauer’s Principle — The Cost of Forgetting

In 1961, IBM physicist Rolf Landauer made a startling claim:

Any logically irreversible operation — particularly the erasure of a bit — must dissipate at least k_B T ln(2) of heat into the environment.

Where k_B is Boltzmann’s constant and T is temperature. At room temperature (300K), this is ~2.85 × 10⁻²¹ joules per bit — impossibly tiny by 1961 standards to measure directly.

This principle has profound implications:

  • Information has physical weight. Erasing a file is a physical process with a thermodynamic cost. Your laptop runs hot not just because of resistive heating but because logical operations (including memory writes and cache clears) are irreversible.
  • Maxwell’s Demon is exorcised. The famous thought experiment (a demon sorting fast and slow molecules to reduce entropy “for free”) is impossible: the demon must erase its memory of past measurements, and that erasure costs exactly the entropy it seemed to gain.
  • The physical universe cannot process information for free. Any computation, any neural firing, any quantum measurement that collapses state — all generate heat proportional to information destroyed.

2012: First Experimental Verification

Researchers at ENS Lyon used a colloidal particle in a double-well optical trap to perform a controlled single-bit erasure and measured the heat dissipated. The result matched k_B T ln(2) to within experimental error — Landauer’s principle confirmed after 51 years.

2025: Quantum Many-Body Extension

A 2025 Nature Physics paper (TU Vienna, FU Berlin, University of British Columbia) extended Landauer’s principle to quantum many-body systems — the regime where quantum entanglement between system and environment cannot be ignored.

Using a quantum field simulator of ultracold Bose gases, they performed a global mass quench (suddenly changing the system’s effective mass) and tracked the entropy production using dynamical tomographic reconstruction. The quantum field theoretical calculations matched the experimental results, confirming that:

  • Landauer’s principle holds even when the “bit” being erased is entangled with the bath
  • The free energy cost depends sensitively on system-bath entanglement and on the number of quantum bath states
  • Classical Landauer is a limiting case of a deeper quantum thermodynamic framework

Separately, a 2024 paper in J. Physical Chemistry Letters derived Landauer’s principle directly from the quantum Shannon entropy, providing a first-principles quantum derivation that had been missing for 60 years.

It From Bit — Wheeler’s Radical Conjecture

John Archibald Wheeler, co-author of Gravitation and coiner of “black hole” and “wormhole,” spent his final decades developing a single idea:

“It from bit.” Every particle, every field, every spacetime event — every “it” in the physical world — derives its existence from answers to yes/no questions (bits). The physical universe is built from information.

Wheeler’s conjecture is non-trivial because it goes further than saying physics can be encoded in information. It says information is the substance, not the encoding. Matter is information in disguise.

Evidence pointing in this direction:

  1. Black hole thermodynamics — Bekenstein-Hawking entropy (S = A/4 in Planck units) says a black hole’s entropy is proportional to its surface area, not volume. This means the information content of a region of space is bounded by its boundary area — a 2D surface encodes a 3D volume. The universe may be fundamentally 2D (concept-holographic-principle).

  2. Quantum mechanics and binary outcomes — Every quantum measurement is a binary question (spin up or down?). The wavefunction is a probability distribution over possible answers. Quantum mechanics is a theory of information.

  3. Digital physics / cellular automata — If spacetime has a Planck-scale discrete structure, the universe literally runs on bits. Konrad Zuse proposed this in 1969; Stephen Wolfram’s computational irreducibility framework (2002) developed it.

  4. Quantum error correction = holography — The AdS/CFT correspondence, which says gravity in a volume is equivalent to a field theory on its boundary, turns out to be mathematically identical to quantum error correction (concept-holographic-error-correction). Spacetime geometry is encoded in quantum correlations — entanglement IS geometry (concept-spacetime-from-entanglement).

Vopson’s Infodynamics

Melvin Vopson (University of Portsmouth) proposed a Second Law of Infodynamics (2022): information entropy remains constant or decreases over time — the opposite of thermodynamic entropy. He finds empirical support in genomic entropy reduction in SARS-CoV-2 mutations, where Shannon information entropy decreases linearly over time despite increasing genetic mutations.

Vopson argues this law, combined with Landauer’s principle, implies that information has mass (~10⁻³⁵ kg/bit at room temperature) and that a “fifth state of matter” — information — permeates reality. This remains speculative and contested, but it generates experimental predictions: electron-positron annihilation should produce a slight asymmetry in photon energies due to information destruction. No confirmed experimental replication as of 2026.

The Arrow of Time Connection

Landauer’s principle beautifully resolves one aspect of the arrow-of-time puzzle (concept-arrow-of-time). Thermodynamic entropy increases because information is constantly being erased — memories of past states are overwritten by new interactions. The flow of time is literally the flow of information loss. The past is more knowable than the future not because of some metaphysical asymmetry in time, but because the future erases records of the past.

Key Facts

  • Shannon entropy H = −Σ pᵢ log₂ pᵢ is identical in form to Boltzmann’s thermodynamic entropy (up to k_B · ln2 scaling)
  • Landauer limit at room temperature: k_B T ln(2) ≈ 2.85 × 10⁻²¹ J per erased bit
  • Modern computer chips erase bits at ~1,000× the Landauer minimum — enormous room for efficiency improvement
  • The human brain erases ~10^16 bits per second (rough estimate from metabolic data and Landauer); this accounts for a non-trivial fraction of brain heat
  • Black hole evaporation (Hawking radiation) may be the universe’s thermodynamically mandated process for releasing information about matter that fell in (concept-black-hole-information-paradox)
  • 2024 paper: Landauer applies not just to erasure but to creation of information — any irreversible information-creating process also generates heat

Cross-Realm Connections

See Also

Graph View

Backlinks