A small note on Entropy in Information Theory and Entropy in Physics

It is rather an amazing fact that if one asks for a function that will capture how much "choice" there is in selection of an event (to use Claude Shannon's term) we arrive at an equation that is exactly the one called entropy in statistical mechanics i.e \( S= - k \sum_i p_i \ln p_i\) where k is some constant. So the obvious question then is that, is this a coincidence or something deep? We have spent a bit of time carefully go through the thermal physics and ultimately arriving at entropy and drawing conclusions from the principle of increasing entropy and all throughout was the ever present notion of a quasi-static process because we always wanted to stay in equilibrium in order that we may use state functions.  This clearly has no direct relation with the entropy in thermal physics but perhaps thermal physics is the wrong place to look for the relation.  Obviously, we are led to invoke some notions from statistical mechanics.

Suppose we have a system A with entropy \( S_1\) and add then we have another system B with entropy \( S_2\). If we consider the composite system, we have that the combined entropy will be  \( S_1 + S_2\). Now concurrently, we can consider the number of states (however we define that)available to each system. Then we will have \( \Omega_1 \) states for system A and \( \Omega_2 \) states for system B. How then do we get the total number of states for the combined system? The answer is \( \Omega_1 \Omega_2 \). If we are to look for a relation between the two concepts it will have to obey

\begin{equation}
 S(A) + S(B)=  f( \Omega_A \Omega_B ) \hspace{10mm} \text{eq.1}
\end{equation}
where f is some undetermined function.

Let us for the moment switch gears to information theory. Say I have some random variables X and Y which can take on values \( x_i\) and \(y_i \) respectively. We now want some function ,I, that characterizes the amount of "choice" I have from a specific probability event. Clearly this function must satisfy some intuitive properties. I only state the property that is relevant for this discussion. If I have two different probabilistic events \( x_i\) and \(y_i \), then surely we want the property that the amount of choice of both these two events will be  \( I(x_i) + I(y_i)\) and this will be the amount of information \( I(x_iy_i) \) for the observed outcome \( x_i y_i\). So we have that

\begin{equation}
 I(x_i y_i)=   I(x_i) + I(y_i) \hspace{10mm} \text{eq.2}
\end{equation}
 .
It should be clear now why in both cases the desired function turns out to be natural log.
In both situations we are solving Cauchy's functional equation but can we go deeper? As far as I can tell the answer is no(so far). No one as far as I can tell has come up with a conceptual understanding that ties the two areas together. People use the word entropy in both situations, it is the same equation but this does not imply that there is the same concept at work. Notice that the formula for entropy looks like an average where the function being averaged over is the natural log. So we are calculating the average information over some probabilistic distribution but here comes the crucial point; who said it had to be the Boltzmann distribution or any distribution relevant for physics? The differential equation for the simple harmonic oscillator appears in very disparate situations but then no one argues for a connection between the situations.

There is a paper by E.T Jaynes called \( \textit{Information Theory and Statistical Mechanics} \) claiming to make the connection. I am not sure I buy it since it merely takes advantage of the fact that the equation is the same in both cases and goes on from there. In fact he goes on the derive the Boltzmann distribution doing exactly the same kind of calculation done in Statistical mechanics the difference is how he interprets the calculation. He interprets it in such a way that one never needs to consider any physical assumption! To quote  part of the abstract:

" It is concluded that statistical mechanics need not be considered as a  physical theory  dependent for its validity on the truth of additional  assumptions not contained in the laws of mechanics (such as ergodicity, metric transitivity, equal a priori probabilities etc)''

In other words if experiments falsified his predictions(which they don't), his interpretation would still stay where it is namely, this is provably what you get when you consider statistical inference. It just so happens that this statistical inference (containing no physical assumptions or necessities) corresponds to reality. So if he takes out the physics from statistical mechanics why should we conclude that he has found a connection between information theory and physics?

Clearly there is something deep happening but again no one has gone further than commenting on the fact that the same equation appears in both fields. As a result we can draw no conclusion.

Principle of Increasing Entropy, consequences and Maxwell Relations

Note that I am not calling the principle of increasing entropy the second law of thermodynamics because it was a consequence two previous states that I called the second law of thermodynamics. This can all be thrown aside as semantics that does not contain any invariant information. Setting aside this debate, we can move on to derive the consequences.
Let us first note, that the first law of thermodynamics however powerful it is, is not directly relate-able to experiments, after all it says that \(U = TdS - PdV,\) but how are we going to construct a knob that we can use to control entropy. We therefore need other kinds of state functions that also have units of energy and can easily be transferred to experiments. We shall introduce these functions by taking legendre transforms of U.

First let us introduce one that is important to chemists but not for physicist.
Consider the function \(H= U +PV\). We now take the differential of both sides to get \(dH = dU + VdP + PdV\) but we know what dU is from the first law, so we that
\(dH = TdS -PdV + VdP +PdV = TdS + VdP \) so H(S,P).  Now, we recall that the function we are dealing with is a state function and therefore does not depend on the path in otherwords it is an exact differential.
\(F = AdX + BdY \) is exact when \( \frac{\partial A}{\partial Y})_X= \frac{\partial B}{\partial X})_Y\). We apply this to H to get \( \frac{\partial T}{\partial P})_S =  \frac{\partial V}{\partial S})_P \). This is our first example of Maxwell Relation, derived painlessly without jumping into a morass of partials. Notice the natural variables of the state function are those that are held constant on each side the relation. This is how one can tell which state function to use in order to derive the relation.

Our second state function is important to physicists and that is got by taking the following legendre transform \(F= U - TS\). We play the same game as before and take the differential of both sidesto get that  \(dF = -SdT - PdV\). This is an exact differential so \( \frac{\partial S}{\partial V})_V=  \frac{\partial P}{\partial T})_T \)

The third example is got by considering \(G = F+PV\). This gives a differential of
\(dG = -SdT + VdP\). Applying the condition for exact differential gives  \( \frac{\partial S}{\partial P})_T = - \frac{\partial V}{\partial T})_P \). The reader can do the same exercise for the internal energy U.

The purpose of these other state functions is to ease analysis of equilibrium states. If one has control over the temperature and volume of a system and considers those as the important variables then one should not insist with working with the internal energy but instead work with F or the Helmholtz free energy or if the thermodynamic coordinates are T and P the natural state function to consider is the the Gibbs free energy, G.

Let us now consider two situations which one is likely to come upon in thermodynamics and we shall use the principle of increasing entropy and the first law of thermodynamics to draw important conclusions.
Let there be a system coupled with a thermal reservoir at  temperature \( T_0\). We shall assume that the system can't expand in volume, but there might be some internal degrees of freedom of the system that are not in equilibrium that allow for heat, Q, to be transferred to the reservoir. These situation might represent a chemical reaction or may a cup of ice and water sitting on table. The cup is the system and surrounding is the reservoir at temperature \( T_0\). Let us further assume that the system and the reservoir make up our universe i.e they both are thermally isolated. We know from the principle of increasing entropy that
 \begin{equation}
    \Delta S + \Delta S_0 \geq 0  \hspace{10mm} \text{eq.1}
\end{equation}.
S, here is the entropy of the system and \( S_0\) is the entropy of the reservoir. We know that \( S_0 = - \frac{Q}{T_0}\) and plugging this into eq.1 we get
\begin{equation}
Q - T_0  \Delta S \leq 0  \hspace{10mm} \text{eq.2}
\end{equation}
We know from the first law that \( \Delta U = Q \), since there is no change work being done by the system and plugging this into eq.2 we get
\begin{equation}
\Delta(U - T_0 S ) = \Delta F \leq 0  \hspace{10mm} \text{eq.3}
\end{equation}
Therefore for a system coupled to a thermal reservoir at fixed volume minimizes the free helmholtz energy if we obey the principle of increasing entropy. So as the system achieves thermodynamic equilibrium F goes to a  minimum.

Let us consider another situation where we have a system that can do work that is calculated by \(P_0 \Delta V\) (this is the only work we will consider). This time the reservoir is a thermal reservoir at temperature  \( T_0 \) and a pressure reservoir with pressure at \( P_0 \). Again we will there will be some heat exchange and we apply the principle of increasing entropy i.e eq.1 and follow by the same arguments as before to eq.2. This time we know from the first law that \( \Delta U = Q - P_0 \Delta V\). Solving for Q and plugging into eq.2 we arrive at

\begin{equation}
\Delta (U + P_0V - T_0 S) = \Delta G \leq 0 \hspace{10mm} \text{eq.4}

\end{equation}

But this is precisely the Gibbs free energy. So the principle of increasing entropy implies for a system coupled to a thermal and pressure reservoir that Gibbs Free energy is minimized.

Eq.3 and Eq.4 apply for both reversible and irreversible processes since state functions do not care about history or path but simply the endpoints which are equilibrium states.
 

What is to come?

What is usually described as the second law of thermodynamics, namely that entropy of an isolated system stays constant or increases as has been demonstrated is a result of more fundamental statements that have been discussed. But now that we know entropy always increases, do we stop here? Surely, this has some consequences. In the next few discussions we shall investigate what results as a consequence of the never decreasing entropy.
After which we shall leave the world of equilibrium processes and investigate the world of non-equilibrium processes. We shall not go over the necessary basic statistical mechanics as excellent resources (in my opinion) are readily available.

Clausius Inequality and Entropy

The  Clausius Inequality arises when we consider cyclic processes. This leads to the concept of entropy. For this discussion recall that
\( \frac{Q_2}{Q_1}= \frac{T_2}{T_1} \). In particular \( \delta Q_2 = \frac{T_2}{T_1} \delta Q_1 \).  Now imagine there is an engine doing work in a cyclic process, this will be done with help of carnot engines. There ill be one ultimate reservoir from where we will get our supply of hear. The engine will be taken from one temperature \( T_i\) to \( T_{i+1} \) by an auxillary reservoir at temperature \( T_i \) to keep the reservoir repleished it gets a supply of heat \( \delta Q_i \) for a carnot engine \( C_i\) but this engine gets it heat from another auxillary reservoir in the amount \( \delta Q_i \frac{\tilde{T}}{T_i} \) and this auxillary reservoir got its heat \( \delta Q_i \frac{\tilde{T}}{T_i} \) from the big reservoir at temperature \( \tilde{T} \). We use the index because movement from one temperature \( T_i \) to \( T_{i+1} \) requires a carnot engine \(C_i \). Everything is done in a cycle so that ultimately \( \Delta U = 0\). The total heat transferred to the engine in the cycle in \( \sum \frac{\delta Q_i \tilde{T}}{T_i} = Q \).

From first law we have that \( Q=W \). But this is a violation of second law because we have converted all the incoming heat into work. This can be avoided if Q and W are negative i.e all the work we put in is turned into heat. Therefore we have that \( Q=W \leq 0 \) so \( \tilde{T} \sum_{i} \frac{\delta Q_i}{T_i} \leq 0 \).

For infinitesimal \( \delta Q_i \) we have that
\begin{equation}
 \oint \frac{dQ}{T} \leq 0 \hspace{10mm} \text{This is the Clausius Inequality.}
 \end{equation}
 But we can reverse the direction of the cycle to get
\( \int \frac{dQ}{T} \geq 0 \). Therefore for a reversible process \begin{equation}
\oint \frac{dQ}{T}= 0  \hspace{10mm} \text{Reversible process}
\end{equation}

Now for a reversible process consider going in a cycle from initial state to final state and back to the initial state. We this we have
\( \int_i^f \frac{dQ}{T} + \int_f^i \frac{dQ}{T} = 0 \). So we have that
\( \int_i^f \frac{dQ}{T} = \int_i^f \frac{dQ}{T} \)
This implies the existence of a state function where \( S_f - S_i\).

We have finally arrived at the concept of entropy. Now we can show that the change in this state function always increasing or stays the same. So consider a process we go from some initial state to final state by some irreversible process and go back to the initial state by a reversible process. We therefore have
\begin{equation}
 \int_{i(irreversible)}^f \frac{dQ}{T} + \int_{f(reversible)}^i \frac{dQ}{T} \leq 0
\end{equation}
so
\begin{equation}
 \int_{i(irreversible)}^f \frac{dQ}{T} \leq \int_{i(reversible)}^f \frac{dQ}{T} = \Delta S
\end{equation}
 The first integral was over an irreversible process while the second was over a reversible process. So we have that the first integral over the irreversible process is
\begin{equation}
\int _{i(irreversible)}^f \frac{dQ}{T} \leq \Delta S \hspace{10mm} \text{Integral over an irreversible process}
\end{equation} .

Thermally isolate system we have dQ=0 so \( \Delta S \geq 0 \). This is what people call the second law of thermodynamics.

Equivalence of the two statements of the second law.

We know prove the equivalence of the two statements of the second law of thermodynamics. Recall that they are the following:

1. No engine or process exists such that its sole purpose is to turn all the heat it extracts from a reservoir into work. It must release some energy into a colder reservoir.

2. No engine or process exists whole sole purpose is to transfer heat from a cold reservoir to a hot reservoir with no other effects.

We shall prove their equivalence but assuming the violation of one of them and showing that implies the violation of the second.

Proof
Let us assume the violation of the first statement. This means that the engine E get Q from a reservoir and produces work, W, such that Q=W. Now imagine a composite system with this engine E and a second engine C. Let the work done by engine E be put in the engine C so that C can get heat \( Q_2\) from a cold reservoir and supply  \(Q_2 +W \) to the hot reservoir. So in terms of the composite system (E and C) we got Q from the hot reservoir and used engine C to put back \( Q_2 +W = Q_2 +Q_1\). Therefore we transferred \( Q_2\) from cold reservoir to the hot reservoir with no work supplied i.e from the view of the composite system, the process consisting of the engine E and engine C, the second statement of the second law was violated.

For the second part of the proof, we assume that the second statement was violated and the goal will be to show that this implies the violation of the first statement. This time assume that engine E takes  \( Q_2\) from a cold reservoir and places it all in the hot reservoir with any work supplied and with no other effects. Now imagine another engine C getting \( Q_1\) from the hot reservoir doing work, W ,and leaking   \( Q_2\) into the cold reservoir. Thus the amount of work done by engine C is \( W = Q_1 -Q_2 \). Again let us look at these two engines as one system. From this point of view . Thus the total amount of heat got from the hot reservoir was  \( Q_1 -Q_2\) and we used all of it as work. This violates the first statement.

Hence the two statements are equivalent.

Note: The first statement does not forbid us from putting work, W, into a system and the engine turning all of this into heat, Q, which is then dumped into a reservoir. This process is what will be used to arrive at the concept of entropy.

Carnot Engines and the Second Law of Thermodynamics

We now come to discuss two statements of the second law of thermodynamics which we shall see gives us a upper bound to the efficiency of any imaginable engine used to do work.
We start with a few experimental facts that were known by Carnot who did the original work on this topic.

Facts
1. We can get work from an engine if it is working between two heat sources of different temperatures. Clearly, we would like our engine to leave our heat sources unchanged and we would like the engine to return to its original state after a while in order to begin another cycle. These two requirements can be satisfied if our engine uses processes that are reversible.
2.It is possible for no work to be done when heat moves from a hot body to a cold body while the system is returning to equilibrium. Therefore any return to thermal equilibrium where no work has be done must counted as loss. We therefore want the engine to work between heat sources that are close in temperature as possible. This way we can reduce any in-efficiency as much as possible because, remember that inefficiency is defined as \( \nu = \frac{W}{Q_1}\)  where W is the work done and \(  Q_1 \) is the heat got from the heat source as opposed to the sink(reservoir at the colder temperature). Now one obvious way of getting the best efficiency namely, 1 is to make \( W= Q_1 \) but is this possible? The answer is a resounding no. This leads us to two statements that are the second law of thermodynamics which  are actually equivalent. The second law of thermodynamics says that the efficiency of an engine can never be one and that in fact there is an upper bound that is less than 1.

Second Law of Thermodynamics

1. It is impossible to construct an engine whose sole purpose is to extract heat from a source and convert all of it into work.

2. It is impossible to construct a device that operating in a cycle has the sole purpose of transferring  heat from a cold reservoir to a hot reservoir with no other effect.

As will later be proved these statements are indeed equivalent but they both infer the existence of an amount of heat, \( Q_2 \) which leaks from the engine to the  cold reservoir  so that not all the original heat  \( Q_1\) is converted into work. Therefore the amount of work done can never equal to \(Q_1 \) but must be equal to \( Q_1 -Q_2 \). Plugging this into the equation for efficiency we get that 
\( \nu = 1 - \frac{Q_2}{Q_1}\).

Let us now prove that there exists an upper-bound  \( \nu_c \). This will be proved by contradiction. We shall assume that the upper-bound can be violated and show that this would involve a violation of the second law.

Theorem: There exists an efficient engine,C, with efficiency,\( \nu_c \). This will extract heat ,\( Q_{c_1} \) from the heat source and leak heat \( Q_{c_2} \)to the cold reservoir. This engine C has the property that  \( \nu_c \) is the upper-bound for the efficiency of any system.

Proof
Let us assume there is a more efficient engine,E, than our brilliant engine and let us further assume that they both do the same work difference.  So we have that
\(\frac{W}{Q_1}=\nu > \nu_c \) therefore it must be that \( Q_{c_1}> Q_1\) since we are assuming that \( W=W_c \). Now comes the key idea or trick.

Let us imagine a composite system composite system composed of these two engines, E and C except that the engine that is working at efficiency \( \nu_c\), C, is working backwards (it is a refrigerator) and the hypothetical engine , E,that has more  efficiency  than our upper-bound is doing work that is being put as an input to run the refrigerator.
Recall, that an engine working backwards requires us to put in work so that energy from a cold reservoir can be put into a hotter reservoir. In other words,the engine that is violating our bound on efficiency is running the refrigerator.
To make everything very explicit, E is extracting \( Q_1 \) from a hot reservoir and doing work, W, and leaking \( Q_2\) into the cold reservoir. Engine C is using this same work W, to extract \( Q_{c_2}\) from the cold reservoir and placing into into \( Q_{c_1}\) into the hot reservoir.
This means that the composite system is extracting from the cold reservoir positive  \(Q_{c_1} -W - (Q_1- W)= Q_{c_1} -Q_1  \) and placing the same amount of work into the hot reservoir and the reservoirs are unchanged by the amount of heat added or extracted and stay at the same temperature. But this entails a violation of the second formulation of the second law of thermodynamics. This device  extracts energy from the cold reservoir and put all of it into the hot reservoir with no work done.
      So what has gone wrong? Well, we got that \( Q_{c_1}> Q_1\) which followed from the fact that E was more efficient than C. So this assumption has to be wrong.What we have at this stage is that there is an upper bound to the efficiency that must be less than one. The engine that produces this upper-bound famously goes by the name Carnot's engine and the cycle that the engine goes through in-order to produce this efficiency is called a Carnot cycle.
     Note that we have not mentioned anything about entropy or disorder. We shall see that as a consequence of the formulation of the second law we shall prove the existence of a state function whose change in values from one equilibrium state to another is never negative. This we shall call entropy.
 The second statement of the second law is what Clausius had in mind when he discovered or defined the notion of entropy. It must be emphasized that entropy will then be given the notion of disorder but we can not do that now as we do not have yet the concept of micro-states or atoms as was true in Clausius' time and also by the fact that thermal physics only cares about properties of macro-systems in equilibrium and says nothing about micro-states.

First Law of Thermodynamics

It is interesting to note just how much has been already covered without explicitly mentioning the famous three laws of thermodynamics. It is very tempting to begin with mentioning the three laws and then studying thermal physics, after all supposedly everything banks on them. It is my opinion that this mode of proceeding while it works well for classical mechanics or quantum mechanics does not work well (pedagogically) for thermal physics.  Rather than stating the laws and seeing the consequences as is done in classical mechanics with Newton's laws , it is far better in thermal physics to see how the laws emerge.  Remember our goal is to find a more natural way of formulating the second law that fits well in Thermal physics and does not involve bringing in concepts that are better left in statistical mechanics.

We begin our discussion with an experimental fact that was discovered by Joule.

 If a thermally isolated system is brought from one equilibrium state to another the work necessary to achieve this change is independent of the process used.

Note: Before we argued correctly that the amount of work done was path dependent. But for the special case of adiabatic work it is path independent.

So we are going from one equilibrium state to another and calculating some quantity \( W_{adiabatic}\) that is path independent (does not depend on its history). Recall, this was the special and lovable feature of state functions. Therefore this existence of this experimental fact implies the existence of a state function so that  \( W_{adiabatic} = U_2 - U_1 = \Delta U \). The choice of using the letter U is supposed to be suggestive. Remember the system is thermally isolated and yet we are able to change it from one equilibrium state to another. This must mean that we are changing its internal energy.

But let us suppose that the system is not thermally isolated. Then as pointed out before, the work done on it depends on the path taken. Therefore we can imagine to sorts of scenarios: We have a system A and we done work on it to bring it from one equilibrium state ,a, to another,b, while it is thermally isolated to get  \( \Delta U \) but we can also put it into thermal contact with its surrounding and perform enough work to change it from equilibrium state a, to equilibrium state b. We will not do the same amount of work for both these processes. The difference between these two is heat, Q i,e Q = \( \Delta U - W\). Rearranging, the variables we arrive at what is called the first law of thermodynamics

\begin{equation}
 \Delta U = Q + W  \hspace{10mm} \text{eq.1}
\end{equation}

Note that in contrast to other discussions we have not talked about whether the process is reversible or not ,whether it is quasi-static or not. This is what gives it its generality. Heat therefore is the non-mechanical exchange of energy between the system and the surroundings because of their temperature difference.

There is a subtle difference between what we mean by heat and what we mean by work, or rather what physically heat means as opposed to work. When we have heat put into a system we increase he random motion of the constituent molecules, but when we change the energy of a system by doing work we displace the molecules in an ordered way. From a quantum mechanical perspective, n particles can exist in discrete energy levels \( \epsilon_i \). Let us say that there are \( n_i \) particles on level  \( \epsilon_i \). The total internal energy U = \(  \sum n_i \epsilon_i \). When we do work we change the levels \( \epsilon_i \) with the populations remaining the same, but the populations \( n_i \) can be changed and this is heat.

We can now use the first law of thermodynamics to justify the statement that heat lost is heat gained. This is a crucial assumption or may a obvious assumption used in carnot cycles and engines.
Imagine we a system that we have an adiabatic system so that no heat comes in or goes out and this system and has two subsystems A and B. We can apply the first law to each subsystem separately as follows:
\begin{eqnarray}
 U_f^A - U_i^A &=& Q^A + W^A \\
 U_f^B - U_i^B &=& Q^B + W^B
\end{eqnarray}

We add the two equations to get

\begin{equation}
U_f^A + U_f^B - (U_i^A + U_i^B) = Q^A +Q^B + W^A + W^B
\end{equation}

The term of the left hand side of the above equation is the change in internal energy of the composite system. The first term on the right hand side is the net flow of heat in the composite system and the second term is the work done on the composite system. But remember our assumption was that the system was adiabatic so it must be that  \(Q^A + Q^B = 0\) and therefore \( Q^A = -Q^B\).

Reversible Processes and Work

Reversible Process 

We are interested in processes that take us from in equilibrium state to another. I  general this will be done irreversibly. But the wonder of state functions is that we can describe or imagine an ideal process that does the job. A simple definition of reversible is that it should be able to return to its original state but this is not all. This process must leave its surroundings unchanged.
Consider the following system: A pendulum and gravitational field. We can apply a force to swing it so we are the environment or the surrounding acting on the system. Imagine pushing the pendulum with an infinitesimal amount of force to move it from one angle to another. We do this to ensure that the pendulum is stays in a state of equilibrium along its path. Ensuring this is what is called a quasi-static process. Now imagine reducing this force so that the pendulum swings back and does work on us and return to its original state. The amount of work it does on us(environment) will be equal to the amount we applied as long as there are no dissipative forces otherwise some work will have to be done to fight against the dissipative forces. Therefore the pendulum has left its surrounding unchanged.
Therefore moving through equilibrium state is a quasi-static process and therefore a reversible process is a quasi-static process where no dissipative forces are present.

Work 

Let us consider, the all to common system involving a gas in cylinder covered by a piston one side. The gas is kept in by a balancing force from the frictionless piston. If the piston is let go slowly enough so that the expansion is quasi-static, the gas will start to expand. All the work done by the gas goes to the environment since there are no dissipative forces. In this simplified and ideal case we can easily calculate the work done by the gas.
\begin{eqnarray}
  dW &=& F dx \\
        &=& P A dx\\
        &=& P dV  \hspace{10mm} \text{reversible processes}
\end{eqnarray}

Therefore we have that:
\begin{eqnarray}
 W = \int_{v_1}^{v_2} P\, dV
\end{eqnarray}

This equation has been derived for reversible process but there are special irreversible processes where it can still be applied because there are no finite pressure drops. So, imagine a cylinder and piston system but this time put small reacting solids that slowly produce gas. This gas does work on the piston which is frictionless. Pressure in cylinder is only infinitesimally smaller than the surrounding and piston is always in mechanical equilibrium and surrounding has pressure  \( P_o\)  so

\begin{equation}
W = \int_{v_1}^{v_2} P\, dV = \int_{v_1}^{v_2} P_o \,dV
\end{equation}

Notice the process is not reversible since the chemical reaction that produces gas is not reversible.
Looking at a P-V diagram we clearly see that the total work done is path dependent as it is the integral under the curve. Therefore dW is and in-exact differential .
Note: The sign 

Example Calculations

1. Suppose we have an ideal gas at T= 300K and isothermally compressed from 1 atm to 10 atm. What is the work done on the gas.

We have that \( \int_{v_1}^{v_2} P \,dv \) but we are not told the volumes but instead told the pressure so obviously we need to change our integration variable. We the use the ideal gas law (PV = nRT) to get the relation between dV and dP . So we have \( dP = - \frac{nRT}{V^2} dV\). So we have that  \(  \int _{P_1}^{P_2} \frac{PV^2}{nRT} \,dP\) we can replace V using the ideal gas law to finally have the integral \(  \int_{P_1}^{P_2} \frac{nRT}{P} dP \) and therefore the formula for work done is \( nRT \ln(\frac{P_2}{P_1}) \). This gives the answer of 57431 J.

2. During and adiabatic irreversible process \( PV^ {\gamma} =c \) all through out the process (not just in this question but in general). Calculate the work done by the gas. \( \gamma \) and c are constants.

The process is reversible so we can use \( \int_{v_1}^{v_2} P \,dV \). We can plug in what P is in terms of and do the integral to get c\( (V_2^{-\gamma+1}-V_1^{-\gamma +1}) \frac{1}{-\gamma +1} \). We have that  \( P_1 V_1 ^{\gamma}=c\) and \( P_2 V_2 ^{\gamma}=c \). We pull back c into the two terms to get  \( W= \frac{P_1V_1 - P_2V_2}{\gamma-1} \)

3. Imagine a gas of n moles at high pressure in a container with volume V and with diathermal walls. The gas is let to leak out slowly through a valve to the surrounding at atmospheric pressure \( P_o\).

This is an irreversible process but it is one of those irreversible processes that is special. The point is that the gas is leaking out very slowly so that there is no finite pressure drop. So we can still use our beloved formula to get \( W= P_o(nv_o -V) \). Here \( v_o \) is the molar volume of the gas outside the container.

The Notion of Equilibrium

Thermodynamics is a perfect science any small inconsistency and the whole structure is destroyed. It therefore pays to be very clear and explicit about each little detail that is to be brought forth to understand anything. For now, we concentrate on what we mean by equilibrium and slightly formalize what we mean by it.
First we curve out a universe for ourselves which consists of everything we care to know about, we call this the system.  Anything that is not contained in it is called the environment or the surrounding. The two spheres of course do not live un-related by are separated by a boundary or a wall.
For a closed system the boundary does not allow for any interaction(no matter exchange) while an open system does allow for matter exchange.

ASIDE: The same kind of taxonomy appears also in quantum mechanics. There for a closed system we have nice unitary evolution wonderfully described by the Schrödinger’s equation and for open systems we have non-unitary evolutions in which classical behaviour emerges. It is often said sloppily that quantum is for the small and classical is for the big, it is more accurate to say quantum is for closed systems and classical is for open systems.

Let us consider a closed system. From experience we know that after a while system reaches a state when no changes occur. In particular, pressure becomes uniform. This state can be labelled by two independent variables (P,V). The amazing thing is that knowing P,V and the masses in the system fixes all other bulk properties that one might what to know about.
We therefore arrive at what we mean by and equilibrium state: this is one in which all bulk physical properties of the system are uniform throughout the system and do not change with time.

I have chosen the pair of variables P,V  but in fact we can choose two any two independent variables e.g for a wire you might choose the tension and the length as your variables. These pairs are called thermodynamic variables or co-ordinates.
It  turns out there are functions which take on unique values as a function of these thermodynamic variables at different equilibrium states. These are called state functions. The concept of a state function is very important because of a very important property namely, the values it takes at different equilibrium states do not depend on history of the system. First of all this is how we shall arrive at state functions but this property is important because in the thermal dynamics we often run into irreversible processes that connect two equilibrium states and we rarely ever have nice neat formulae for these processes but that does not matter because of state functions. All we need to do is describe reversible processes connecting these two equilibrium states (which is nice because reversible processes are easy to describe) and we can understand the physics. This is how we shall arrive at the concept of entropy.

We know we can bring two systems together so that they interact thermally and after a while there will be no further changes to take place in pressure and volume. When no further changes take place we say these two systems are in thermal equilibrium. A transitive property applies to states in thermal equilibrium i.e If A is thermal equilibrium with B and B is in thermal equilibrium with C then A is in thermal equilibrium with C. We can of course apply this argument ad infinitum to describe a whole series of systems that are in thermal equilibrium. This is in essence the zeroth law of thermodynamics.

The amazing thing is that as a consequence of the zeroth law we can prove that all systems in thermal equilibrium share one property, we call this the temperature. The proof which is borrowed from Zemansky's book "Heat and Thermodynamics" will follow momentarily.

We first note a subtle distinction often skirted over namely the difference between thermal equilibrium and thermodynamic equilibrium. Thermal equilibrium does not guarantee thermodynamic equilibrium. In order to have thermodynamic equilibrium we must have mechanical equilibrium i.e all forces are balanced and we must have chemical equilibrium i.e no chemical reactions should be taking place between the two systems.

Now at thermal equilibrium it must be that P,V and T are not independent but are related by some functional equation f(P,V,T)=0. This is called the equation of state. For an ideal gas we have the famous PV -NKT=0.

We now present a proof that given the zeroth law there exits one property that all systems in thermal equilibrium with one another share. This of course we know a head of time is temperature.

Suppose we have to systems A and C with thermal dynamic variables being X and Y for A and X'' and Y'' for C. We know there is and equation of state \begin{equation}
f_{AC} (X,Y,X'',Y'') =0  \hspace{10mm} \text{eq.1}
\end{equation}.
Also let us say there is another system B with variables X' and Y' which is in equilibrium with C so that there is another equation of state
\begin{equation}
f_{BC} (X',Y',X'',Y'') =0 \hspace{10mm} \text{eq.2}
\end{equation}

Now we can solve for Y'' in both equations and set the resulting equations equal to each other
\begin{equation}
g_{AC}(X,Y,X'') = g_{BC}(X',Y',X'') \hspace{10mm} \text{eq.3}
\end{equation}

The zeroth law guarantees that A is in equilibrium with B so that
\begin{equation}
f_{AB} (X,Y,X',Y') =0 \hspace{10mm} \text{eq.4}
\end{equation}
but in particular eq.4 and eq.3 must be in fact equal (remember these are state functions and take on unique values at equilibrium states). This means that X'' is an extraneous variable and can be removed so that at thermal equilibrium we have
\begin{equation}
g_{A}(X,Y) = g_{B}(X',Y') = g_{C}(X'',Y'') = t \hspace{10mm} \text{eq.3}
\end{equation}.
So we have that there exists a function which  parametrizes these sets of co-ordinates and these functions are all equal at thermal equilibrium. This common value is empirically known as the temperature.
 We now see the importance of the zeroth law it guarantees the existence of temperature. I often wondered about the importance of the zeroth law. It is mentioned in the first few pages of thermodynamic textbooks and never makes any appearance as one moves forward. Well, now we know it allows us to talk about thermal equilibrium and once that has happened it is forgotten.