The demons haunting thermodynamics

In a recent Physics Today article [1], Katie Robertson wonders if the philosophical demons that haunt thermodynamics have been exorcised. She begins by stating that thermodynamics is a strange theory because although it is fundamental to our understanding of the world, it differs dramatically from other physical theories. I do not agree. First, thermodynamics is not a theory. Thermodynamics is a scientific discipline and, as in other sciences, we have theoretical, computational, and experimental flavors. Second, I do not know what she mean when she claims that thermodynamic theory differs dramatically from other physical theories. I do not find any fundamental difference with electrodynamics, with mechanics or with general relativity. In this newsletter, I will avoid "the bizarre philosophical implications" and focus on the scientific aspects.

The "many oddities" that Robertson assigns to thermodynamics are actually oddities of statistical mechanics and, more specifically, of the Gibbs-Boltzmann approach. Thermodynamic theory is rigorous and self-consistent. Problems have always been on the side of the statistical mechanics attempts to ‘derive' or ‘explain' thermodynamic laws. Or put in a more poetic wording, no demon stalks the rich and fruitful village of thermodynamics.

The difficulties of statistical mechanics have not been solved by quantum mechanics, just as the problems of classical electrodynamics have not been solved by quantum mechanics. This is admitted in The Feynman Lectures on Physics [2]: "There are difficulties associated with the ideas of Maxwell's theory which are not solved by and not directly associated with quantum mechanics [...] However, when electromagnetism is joined to quantum mechanics, the difficulties remain".

The main difficulties of classical statistical mechanics regarding its foundations, the extension of Gibbs ensemble theory to nonequilibrium regimes or the problem of the arrow of time remain in quantum statistical mechanics. It is not true that a "quantum exorcism" banishes the demons once and for all.

LOSCHMIDT'S DEMON

We are all familiar with irreversible processes (that is, processes that only go in one direction, but not in the reverse), but this observed irreversibility contrasts with the usual laws of mechanics, with are time-reversible and cannot distinguish the process \( A \to B \) from \( B \to A \).

Boltzmann is one of the founders of statistical mechanics, but he did not found thermodynamics. This merit goes to Carnot, Clausius, Kelvin, Caratheodory, and others. Carnot is often called the father of thermodynamics. Boltzmann was concerned that the irreversibility of thermodynamics, as encoded in its second law was, conflicted with the reversibility of classical mechanics. Boltzmann tried to derive the second law from mechanics and failed.

Robertson states that the "underlying classical or quantum mechanics are time reversible". This is not true for quantum theory, which introduces a dual evolution for quantum systems: (i) a time-reversible evolution given by the Schrödinger equation, and (ii) a time-irreversible evolution given by the collapse postulate.

To claim that, throughout his career, Boltzmann pursued a range of strategies to explain irreversible equilibrating behavior from the underlying reversible dynamics, strikes me as a surprisingly compassionate writing of what actually happened. Over the years, many objections were raised to Boltzmann's attempt to derive irreversibility from reversibility, in particular to his assertion that probability did not enter into the derivation. Boltmann's response to the objections was inconsistent, sometimes invoking probabilistic argument to justify his thinking and sometimes saying that it was not necessary to invoke probability theory.

The essence of Josef Loschmidt's objection to Boltzmann is not that classical mechanics allows for the possibility of the moments being reversed, which would lead to antithermodynamic behavior incompatible with experience, but that classical mechanics cannot discern between thermodynamic and antithermodynamic behavior.

Normal and time-reversed motion of simple particles in a box.

We do not need to manually reverse the momentum of all the particles in a system to see this, so Boltzmann's challenge of trying to reverse the moments is irrelevant, as this does not explain why such initial conditions are not observed in nature. This is not just a "matter of practical impossibility" as Robertson asserts, since we do not prepare thermodynamic systems by fixing the momentum of each particle. Let us consider the case of an expanding gas, we start from the gas in the bottom box and we always observe the evolution drawn on the left and never the one drawn on the right. The laws of mechanics state that both evolutions are possible. The laws of thermodynamics state that only the evolution on the left is possible. The evolution on the right has never been observed.

Spontaneous expansion (left) and compression (right) of a gas

Recent technological developments have allowed us to experimentally run a version of the idea of reverse all the moments. In the spin-echo experiment, it is possible to manipulate, by means of a RF pulse, atomic spins that have dephased and become disordered and return they to their earlier state. While it is still technologically impossible to reverse the moments of all the particles in a gas, it is now possible to do so on a computer. In a molecular dynamics simulation, we can stop the simulation, reverse all the moments, restart the simulation, and see on the screen how the system returns to an earlier state. What Robertson does not tell us is that after a short period in both the spin-echo experiment and the molecular dynamics simulation the system returns to an equilibrium state again. One only can ignore dissipation for short periods of time.

Why do not we see antithermodynamic behavior in nature? Why does not a gas compress back into a smaller volume? Why does not eggs unsmash or cups of coffee spontaneously warm up? Robertson's answer is based on the old myth of initial conditions, but as we saw in the figure above, the gas starts from the same initial condition in both cases. If the initial condition was all we need to explain irreversible behavior, the second law had never been invented. The first law had been enough. Why do we observe \( A \to B \) but not \( B \to A \)? Is it because the system started from the initial condition \( A \)? It might seem so at first glance, but if we work out the details we find that \( B \to A \) is equivalent to \( A \to Z \) and the question now is the following: if the system starts from the initial condition \( A \), what will we observe \( A \to B \) or \( A \to Z \)? Will we observe the spontaneous expansion of a gas or the spontaneous compression? The first law of thermodynamics cannot answer this and that is the reason why the second law was invented.

The above discussion is exclusively from a macroscopic point of view. It is often claimed that if we study the microscopic details we find that antithermodynamic evolutions require atypical microscopic initial conditions. The usual counterargument is that, by the same rule, thermodynamic evolutions require atypical microscopic final conditions. Thus, it is very difficult to pretend to reject atypical conditions in one case, but not the other.

However, there is another deeper counterargument to rejecting the argument about atypical microscopic initial conditions, and it is that this distinction between "typical" and "atypical" uses a non-dynamical and anthropic concept of probability, often called the thermodynamic probability. The name is a misnomer because it has nothing to do with thermodynamics. This thermodynamic probability is not only based on an arbitrary distinction between microscopic and macroscopic states, which also ignores any intermediary state (mesoscopic, nanoscopic…), but it can be shown from pure dynamics that the "atypical" initial conditions are as typical as the typical conditions. Look again at the figure of a gas-filled box with a partition in the middle. For every microscopic trajectory that leads to the evolution on the left, there is a microscopic trajectory that leads to the evolution on the right. As Ilya Prigogine once wrote, Boltzmann opened the door to the uncritical mix of probabilistic and dynamical arguments in physics. Physicists and philosophers continue to repeat Boltzmann' mistakes some hundred and fifty years later.

MAXWELL'S NIMBLE-FINGERED DEMON

Maxwell imagined a being that observes individual molecules in a gas-filled box with a partition in the middle. If the demon sees a fast-moving gas molecule, it opens a trapdoor in the partition that allows fast-moving molecules through while leaving slow-moving ones behind. Repeatedly doing that would allow the buildup of a temperature difference between the two sides of the partition. A heat engine could use that temperature difference to perform work, which would contradict the second law of thermodynamics.

The problem here is that Maxwell forgot to consider the being from a thermodynamic point of view. We could also imagine a ‘demon' that takes molecules at rest and gives them momentum. If we do not include this ‘demon' in our mechanical calculations, we might com to the (wrong) conclusion that it is possible to build an engine that contradicts the laws of mechanics.

Have been able to realize Maxwell's demon in modern experiments? Robertson argues not. The second law has been not violated in those experiments, She is right that we have to include the demon in our calculations, but some of her other arguments deserve further comment. For example, it is not true that a process violates the second law only if the entropy of the total system decreases. I assume that by "total system" she really means an isolated system. The more rigorous form of the second law states that the Markovian evolution of the average entropy \( \langle S \rangle \) cannot decrease its value between two equilibrium states. The law says nothing about non-Markovian evolutions or fluctuations. A spontaneous fluctuation \( \delta S \) can temporally decrease the value of the entropy in a system at equilibrium, but this is not a violation of the second law, since the law only describes the behavior of \( \langle S \rangle \) and not of the \( S = \langle S \rangle + \delta S \). This is not an observation exclusive to thermodynamics. All the laws of physics refer to averages. Newton laws or Maxwell equations are only about averages not about fluctuations. Even the Boltzmann kinetic equation is only valid for averages in the number density \( \langle f \rangle \) and has to be amended and extended if we want study fluctuations \( \delta f \).

\[ \frac{\partial \langle f_1 \rangle}{\partial t} = - \frac{\boldsymbol{p}_1}{m} \cdot \boldsymbol{\nabla}_1 \langle f_1 \rangle + \!\!\iint_0^{2\pi}\!\!\!\int_0^\infty\!\! b g \;[ \langle f_1' \rangle \langle f_2' \rangle - \langle f_1 \rangle \langle f_2 \rangle ]\; \mathrm{d}b \mathrm{d}\phi \mathrm{d}\boldsymbol{p}_2 \] \[ \frac{\partial \delta f_1}{\partial t} = - \frac{\boldsymbol{p}_1}{m} \cdot \boldsymbol{\nabla}_1 \delta f_1 + \!\!\iint_0^{2\pi}\!\!\!\int_0^\infty\!\! b g \;[ \delta f_1'f_2' + f_1' \delta f_2' - \delta f_1f_2 - f_1 \delta f_2 ]\; \mathrm{d}b \mathrm{d}\phi \mathrm{d}\boldsymbol{p}_2 + \zeta \]

Boltzmann kinetic equation (top) and extension to fluctuations (bottom).

It is a bit ironic that the same physicists who claim that a fluctuation in entropy violates the second law and that the second law is only a statistical law, do not make the same claims about Newton laws, Maxwell equations, or the Boltzmann equation. Tell me, how many headlines on science new sites do you know that have claimed that the Maxwell equations of electrodynamics are just statistical laws continuously violated by fluctuations?

I agree with Robertson that today's alleged Maxwellian demons are skilled illusionists rather than true magicians.

QUANTUM STEAMPUNK

We saw above that Katie Robertson cannot distinguish between scientific discipline and theory. Now we will know that she misunderstands the concepts of classical and quantum. She uses "quantum" to mean microscopic, and "classical" to mean macroscopic. This distinction is not valid because we can build classical models of small systems (think for example of the Lorentz equation, which applies to a single electron) and we can build quantum models of very large systems (systems with \( N \to \infty \) and \( N \to \infty \) but \( N/V \) finite).

Robertson is correct that thermodynamics is not limited to macroscopic systems, but research in thermodynamics did not progress from the steam engines that powered the Industrial Revolution to the atomic-scale "quantum steampunk" revolution. Research in macroscopic thermodynamics was followed by thermodynamic research at the mesoscopic scale, this by nanothermodynamics (also called thermodynamics of small systems), and finally by quantum thermodynamics. Robertson mentions quantum information theory, but this discipline has nothing to do with quantum thermodynamics, just as classical information theory has nothing to do with classical thermodynamics.

Note: There are many myths regarding information theory and thermodynamics, from misconceptions that thermodynamic entropy is a measure of ignorance, to the claim that information theory provides a foundation for thermodynamics.

I do not know what Robertson means when she writes that entanglement and coherence can be used as "fuel". If thermodynamics is not limited to macroscopically large systems, is it universal? Robertson mentions that many physicists believe that it is, she also quotes Albert Einstein. Thermodynamics is a branch of physics and chemistry that deals with concepts such as entropy, temperature, and heat. Those concepts are universal, therefore thermodynamics is. She adds that today thermodynamics is used to understand topics as varied as quantum thermal engines, globular clusters of stars, black holes, bacterial colonies, and –more controversially– the brain.

We would differentiate physics from fiction. The so-called thermodynamics of black holes is not a true thermodynamics. It is an inconsistent and useless discipline where certain concepts are given the same name of thermodynamics concepts, but that is all. We have a black hole internal energy that is not a thermodynamic internal energy, a black hole entropy is not a thermodynamic entropy, a black hole temperature is not a thermodynamic temperature, you get the idea. A part of my series of books on Common misconceptions in Physics is devoted to "black hole thermodynamics" and I explain all in detail therein, including how the proposed generalized second law is nonsense. I will simply mention here that the concept of black hole temperature fails from both theoretical and empirical perspectives; it fails because it is not an intensive quantity that characterizes thermal equilibrium and it fails because it cannot be measured by an thermometer and therefore has no physical validity.

Brains are just physical systems and they have to verify the laws of physics, chemistry, biology,… this includes the laws of thermodynamics. No, brains cannot violate the laws of thermodynamics. There is nothing controversial here.

IS THERMODYNAMICS ANTHROPOCENTRIC?

The short answer is "no". Thermodynamics is a branch of natural science that deals with physical systems and their properties. Entropy, temperature, pressure, chemical potential, heat, work,… are physical properties.

Maxwell's philosophical position on the nature of thermodynamics is untenable. The distinction between ordered (work) and disordered (heat) motion is fundamental to thermodynamics, but this distinction is entirely physical. If a given energy exchange mechanism causes the volume of the system under study to vary, we know this mechanism is work and not heat. The distinction between ordered and disordered motion is physical and that is why all observers agree when something is work and when it is heat. If the distinction was "only in relation to the mind which perceives them", then some people would observe heat when others observe work and vice versa.

I have always found it interesting how certain physicists and philosophers try to dismiss the laws of thermodynamics simply because those laws do not sustain their geometrical vision, their idealized view, of Nature. The distinction between ordered and disordered motion in thermodynamics is similar to the distinction between external and internal motion in mechanics, but no one has claimed that mechanics is anthropocentric.

Robertson ignores the physical nature of thermodynamic systems and writes that thermodynamics is anthropocentric, or observer dependent, because thermodynamic features such as entropy might look different –or not exist at all– if we were a different type of creature, such as a honeybee. This is absurd. I will use her own example. A honeybee sees a garden very differently than we do because its eyes are sensitive to a different part of the electromagnetic spectrum than ours are. Does this mean that one of the main branches of physics, electromagnetism is anthropocentric? Obviously no. This example highlights what I wrote above about how certain physicists and philosophers try to dismiss thermodynamics and its laws. If a honeybee detects a different electromagnetic spectrum this does not mean anything for electromagnetism, but if the insect detects a different thermodynamic feature such as entropy, then it is used as ‘proof' that thermodynamics is anthropocentric. Double standard.

I want to end this section by remarking that quantum mechanics has not made many people comfortable with the observer being seemingly inescapable from physics. Only some interpretations of quantum mechanics do. It is perfectly possible to build a quantum formalism without even mentioning observers.

LEAVING IGNORANCE OUT OF IT

Robertson writes that "in classical statistical mechanics, the key postulate –often called the fundamental assumption– is that each accessible microstate of a system must be equally likely". This is a serious misunderstanding. The postulate of equal a priori probabilities is only valid for an isolated system in equilibrium. If the system is out of equilibrium, then the probabilities of the accessible microstates are not identical. The probabilities are also different for a nonisolated system at equilibrium. Thus, the probabilities pᵢ for the canonical ensemble are given by the well-known expression \( p_i = (1/Z) \exp(-E_i/kT) \). The higher the energy of the microstate, the lower its probability.

We saw above that Boltzmann opened the door to the uncritical mix of probabilistic and dynamical arguments in physics, and that physicists and philosophers continue to repeat Boltzmann' mistakes hundred and fifty years later. Paul and Tatyana Ehrenfest continued Boltzmann's tradition of leaving rigor out of statistical physics, while Edwin Jaynes and Claude Shannon continued Boltzmann's tradition of bringing nonphysical concepts into physics.

Robertson, quoting Jaynes, states that thermal physics is anthropocentric. This is not true. Thermal physics deals with thermal properties, and those are entirely physical. For instance, the temperature of a physical system is the same when we know only a pair of macroscopic parameters of the system as it is when we know the detailed position and velocity of each one of its atoms. The same happens with entropy. Some physicists and philosophers confuse thermodynamic properties with their information theoretic cousins. Shannon's entropy is zero when we know all the microscopic details about a system, but its Clausius entropy is not zero. Of course, only the Clausius entropy is physical and a thermodynamic property of the system.

The lack of rigor and the confusions introduced by Boltzmann and later continued by Ehrenfest, Jaynes, Shannon, and others have literally ruined the field of statistical physics and the reason why this discipline is still in the air, without a solid foundation and with not accepted theory valid for arbitrary nonequilibrium regimes. The Physics Today article we are reviewing is another nail in the coffin, repeating old mistakes and myths.

It is not true that we "must assume each state is equally likely because we do not know which exact microstate the system is in". If we do a molecular dynamics simulation, we know exactly the microstate, but we need to average over different microstates, because the quantities of classical thermodynamics correspond to a macroscale level of description, since they are measured with apparatus whose relaxation time is macroscopic. That is the reason why we have to average over microscopic states in order to calculate the classical temperature of a thermodynamic system. We have to average over all the microstates visited by the system during the duration of the macroscopic measurement associated to a thermometer. This is not unique to thermodynamics. If we want to compute the pressure of classical mechanics (for example, the pressure used in hydrodynamics) by means of a molecular dynamics simulation, we have to do an average as well, but nobody will claim that classical mechanics is anthropocentric.

The Gibbs entropy \( S = \int \rho \ln \rho \mathrm{d}^Nq \mathrm{d}^Np \)is not the entropy of a thermodynamic system, but the entropy of an ensemble. The best way to see this is by setting \( \rho = 1 \), which corresponds to the case when we know the system's exact microstate with certainty in the microcanonical ensemble. In that case, the Gibbs entropy reduces to zero, but the thermodynamic entropy is not zero. In fact, the Clausius entropy can be calculated from molecular dynamics methods. We know exactly the microscopic state and using thermodynamic integration we can obtain the value of the entropy of the system, or more exactly the difference between two states, just as in the experimental situation.

Robertson claims that the Laplace's demon can be exorcised by shifting to a quantum perspective on statistical mechanics. Her argument is that probabilities are an additional ingredient added to the microdynamics of systems in classical statistical mechanics, but in the quantum case, probabilities are already an inherent part of the theory, so there is no need to add ignorance to the picture. "In other words, the probabilities from statistical mechanics and quantum mechanics turn out to be one and the same". This is not entirely correct.

Probabilities enter in two different ways in quantum theory. The probabilities associated with the wavefunction do not represent ignorance, but indeterminacy. But the probabilities associated with von Neumann approach to quantum statistical mechanics represent ignorance. Von Neumann introduced a statistical operator and quantum ensembles in parallel with the Gibbs' approach in classical statistical mechanics. For von Neumann, the probabilities associated with his operator describe our ignorance about the quantum system, when we do not know its wavefunction.

Robertson claims that quantum mechanics and quantum statistical mechanics appear to clash, because the former only can assign "a definite state known as a pure state" to an isolated system, whereas quantum statistical mechanics can assign to the system "an inherently uncertain state known as a maximally mixed state". There is no clash here, simply quantum statistical mechanics deals with a more general type of quantum state. Using the purity \( \gamma \) of a quantum state, quantum mechanics is restricted to systems whose state \( \gamma = 1 \), but quantum statistical mechanics (and quantum thermodynamics) do not, because they can describe quantum states in the range \( 0 \leq \gamma \leq 1 \). As Beretta claims [3]: "Quantum thermodynamics is a nonstatistical generalization of quantum theory".

Scope of quantum mechanics (circumference) and quantum thermodynamics (circle)

The problem with Robertson seems to be that she thinks that quantum states only can be given by wavefunctions, but already Dirac showed that a statistical operator is the more general description of a quantum state. There other alternative ways to describe quantum states in its more general way. In Nine formulations of quantum mechanics [4], readers can find alternatives along with the limitations of the wavefunction approach.

It is not true that the supersystem composed of a qubit that is entangled with a surrounding heat bath has a quantum state described by a wavefunction. If this was true, we could obtain the heat bath state from the wavefunction and we cannot. Quantum statistical mechanics operators must be postulated in the same way that classical ensembles must be postulated in classical statistical mechanics.

I hope that with this newsletter I have ‘exorcised' thermodynamics and statistical mechanics, removing the common misconceptions that have plagued both disciplines since pioneering work of Boltzmann and Gibbs in statistical mechanics and gas kinetics. To know more, you can refer to the mentioned references for some technical details and to my forthcoming book series Common misconceptions in Physics for a detailed discussion and further references.

NOTES

1. https://physicstoday.scitation.org/doi/10.1063/PT.3.4881
2. The Feynman Lectures On Physics, Vol 2; Mainly Electromagnetism And Matter; Second Printing 1964: Addison-Wesley Publishing Company, Inc.; Reading, Massachusetts. Feynman, Richard P.; Leighton, Robert B.; Sands, Matthew.
3. Quantum thermodynamics of nonequilibrium. Onsager reciprocity and dispersion–dissipation relations 1987: Found. Phys. Vol. 17(4) 365–381. Beretta, G. P.
4. Nine formulations of quantum mechanics 2002: Am. J. Phys. 70(3), 288–297. Styer, Daniel F.; Balkin, Miranda S.; Becker, Kathryn M.; Burns, Matthew R.; Dudley, Christopher E.; Forth, Scott T.; Gaumer, Jeremy S.; Kramer, Mark A.; Oertel, David C.; Park, Leonard H.; Rinkoski, Marie T.; Smith, Clait T.; Wotherspoon, Timothy D.