David M. Friscia’s Homepage

17 Equations That Changed The World

entropy

Rudolf Clausius (1822 – 1888) is the originator of the concept of entropy. Clausius described entropy as the  transformation-content. This was in contrast to earlier views, based on the theories of Isaac Newton, that heat was an indestructible particle that had mass.

Entropy is a measure of the number of specific ways in which a system (Cosmos) may be arranged, often taken to be a measure of disorder.

The entropy of an isolated system never decreases, because isolated systems spontaneously evolve towards thermodynamic equilibrium, which is the state of maximum entropy. Entropy was originally defined for a thermodynamically reversible process as [ Delta S = int frac {dQ_{rev}}T ] where the entropy (S) is found from the uniform thermodynamic temperature (T) of a closed system divided into an incremental reversible transfer of heat into that system (dQ).

The above definition is sometimes called the macroscopic definition of entropy because it can be used without regard to any microscopic picture of the contents of a system. In thermodynamics, entropy has been found to be more generally useful and it has several other formulations. Entropy was discovered when it was noticed to be a quantity that behaves as a function of state. Entropy is an extensive property, but it is often given as an intensive property of specific entropy as entropy per unit mass or entropy per mole.

In the modern microscopic interpretation of entropy in statistical mechanics, entropy is the amount of additional information needed to specify the exact physical state of a system, given its thermodynamic specification. The role of thermodynamic entropy in various thermodynamic processes can thus be understood by understanding how and why that information changes as the system evolves from its initial condition. It is often said that entropy is an expression of the disorder, or randomness of a system, or of our lack of information about it (which on some views of probability, amounts to the same thing as randomness). The second law is now often seen as an expression of the fundamental postulate of statistical mechanics via the modern definition of entropy.

Interdisciplinary Applications Of Entropy

Although the concept of entropy was originally a thermodynamic construct, it has been adapted in other fields of study, including information theory, psychodynamics, thermoeconomics/ecological economics, and evolution.

The three nested systems of sustainability - the economy wholly contained by society, wholly contained by the biophysical environment.

Information Theory

One of the first major applications of Boolean algebra came from the 1937 master's thesis of Claude Shannon, one of the most important mathematicians and engineers of the 20th century. Shannon realized that switches in relay networks, like in a telephone network, or an Shannon's innovation made the design of switch networks vastly easier: rather than needing to actually play around with network connections themselves, the techniques developed by Boole and his successors provided a mathematical framework allowing for more efficient network layouts.

I thought of calling it 'information', but the word was overly used, so I decided to call it 'uncertainty'. [...] Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage.'
Conversation between Claude Shannon and John von Neumann regarding what name to give to the attenuation in phone-line signals.

When viewed in terms of information theory, the entropy state function is simply the amount of information (in the Shannon sense) that would be needed to specify the full microstate of the system. This is left unspecified by the macroscopic description.

In information theory, entropy is the measure of the amount of information that is missing before reception and is sometimes referred to as Shannon entropy. Shannon entropy is a broad and general concept which finds applications in information theory as well as thermodynamics. It was originally devised by Claude Shannon in 1948 to study the amount of information in a transmitted message. The definition of the information entropy is, however, quite general, and is expressed in terms of a discrete set of probabilities :

.

In the case of transmitted messages, these probabilities were the probabilities that a particular message was actually transmitted, and the entropy of the message system was a measure of the average amount of information in a message. For the case of equal probabilities (i.e. each message is equally probable), the Shannon entropy (in bits) is just the number of yes/no questions needed to determine the content of the message.

The question of the link between information entropy and thermodynamic entropy is a debated topic. While most authors argue that there is a link between the two, a few argue that they have nothing to do with each other.

The expressions for the two entropies are similar. The information entropy H for equal probabilities is where k is a constant which determines the units of entropy. There are many ways of demonstrating the equivalence of "information entropy" and "physics entropy", that is, the equivalence of "Shannon entropy" and "Boltzmann entropy". Nevertheless, some authors argue for dropping the word entropy for the H function of information theory and using Shannon's other term "uncertainty" instead.

Information Paradox

The entropy of a black hole is given by the equation:
where S is the entropy,
c is the speed of light,
k is Boltzmann's constant,
A is the surface area of the event horizon,
h (“h-bar”) is the reduced Planck's Constant (or Dirac's Constant) and
G is the gravitational constant. (Hawking's Paradox)

Boolean Algebra

Sometimes, developments in pure math can, decades or centuries later, turn out to have amazing applications.

Remarkably, about a century after Boole's initial investigations, mathematicians and scientists discovered an extremely powerful set of applications for formal logic, and now this apparently abstract mathematical and logical tool is at the heart of the global economy.

Boolean algebra, and other forms of abstract propositional logic, are based on dealing with compound propositions made up of simple propositions joined by logical connectors like "and", "or", and "not". Boolean algebra — taking true and false values, manipulating them according to logical rules, and coming up with appropriate true and false results — is the fundamental basis of the modern digital computer.