Thermodynamics of Gambling Demons

Gonzalo Manzano,1, 2 Diego Subero,3 Olivier Maillet,3 Rosario Fazio,1, 4 Jukka P. Pekola,3 and Edgar Roldan1

1International Centre for Theoretical Physics ICTP, Strada Costiera 11, I-34151, Trieste, Italy2Scuola Normale Superiore, Piazza dei Cavalieri 7, I-56126, Pisa, Italy

3PICO group, QTF Centre of Excellence, Department of Applied Physics, Aalto University, 00076 Aalto, Finland4Dipartimento di Fisica, Universita di Napoli “Federico II”, Monte S. Angelo, I-80126 Napoli, Italy

The stochastic nature of games at the casinoallows lucky players to make profit by means ofgambling. Like games of chance and stocks, smallphysical systems are subject to fluctuations, thustheir energy and entropy become stochastic, fol-lowing an unpredictable evolution. In this con-text, information about the evolution of a ther-modynamic system can be used by Maxwell’sdemons to extract work using feedback control.This is not always the case, a challenging task isthen to develop efficient thermodynamic proto-cols achieving work extraction in situations wherefeedback control cannot be realized, in the samespirit as it is done on a daily basis in casinosand financial markets. Here we study fluctua-tions of the work done on small thermodynamicsystems during a nonequilibrium process that canbe stopped at a random time. To this aim weintroduce a gambling demon. We show that bystopping the process following a customary gam-bling strategy it is possible to defy the standardsecond law of thermodynamics in such a way thatthe average work done on the system can be belowthe corresponding free energy change. We derivethis result and fluctuation relations for the workdone in stochastic classical and quantum non-stationary Markovian processes at stopping timesdriven by deterministic nonequilibrium protocols,and experimentally test our results in a single-electron box. Our work paves the way towardsthe design of efficient energy extraction protocolsat the nanoscale inspired by investment and gam-ing strategies.

Maxwell’s demon, as introduced in 1867 [11], is a lit-tle intelligent being who acquires information about themicroscopic degrees of freedom of two gases held in twocontainers at different temperatures, and separated bya rigid wall. In this way the demon is able to controla tiny door, allowing fast particles from the cold con-tainer pass to the hotter one, hence generating a persis-tent heat current against a temperature gradient. Thisparadoxical behavior challenging the second law of ther-modynamics, has its roots in the link between informa-tion and thermodynamics, which has fascinated scientistsfrom more than a century [22]. Nowadays, it is well under-stood that Maxwell’s demon is a paradigmatic exampleof feedback control, for which modified thermodynamiclaws apply [33–66] which have been tested experimentallyfor both classical [77–99] and quantum systems [1010, 1111].

F F

WW

Time

W

WWW

WW W

WW

F F

F

W

WWW

FIG. 1. Illustration of a gambling demon. The demonspends work (W , silver coins) on a physical system (slot ma-chine) hoping to collect free energy (F , gold coins) by execut-ing a gambling strategy. In each time step, the demon doeswork on the system (introduces a coin in the machine) anddecides whether to continue (”play” sign) or to quit gamblingand collect the prize (”stop” sign) at a stochastic time T fol-lowing a prescribed strategy. In the illustration, the demonplays the slot machine until a fixed time T = 3 (top row) un-less the outcome of the game is beneficial at a previous time,e.g. T = 2 (bottom row). Under specific gambling schemes,the demon can extract on average more free energy than thework spent over many iterations, a scenario that is forbiddenby the standard second law.

Here we propose and realize a gambling demon whichcan be seen as a variant of the original Maxwell’s thoughtexperiment (Fig. 11). This demon invests work by per-forming a nonequilibrium thermodynamic process andacquires information about the response of the systemduring its evolution. Based on that information, the de-mon decides whether to stop the process or not followinga given set of stopping rules and, as a result, may re-cover more work from the system than what was invested.However, differently to Maxwell’s demon, a gambling de-mon does not control the system’s dynamics, hence ex-cluding the possibility of proper feedback control. Thisis analogous to a gambler who invests coins in a slot ma-chine hoping to obtain a positive payoff. Depending onthe sequence of outputs from the slot machine, the gam-bler may decide to either continue playing or stop thegame (e.g. to avoid major losses), according to someprescribed strategy. How much work may the gamblingdemon save/extract on average in a given transformationby implementing any possible strategy?

arX

iv:2

008.

0163

0v2

[co

nd-m

at.s

tat-

mec

h] 8

Aug

202

0

2

In this article, we derive and test experimentally uni-versal relations for the work and entropy production fluc-tuations in Markovian nonequilibrium processes subjectto gambling strategies that stop the process at a finitetime during an arbitrary deterministic driving protocol.Our theoretical results apply to both classical and quan-tum stochastic dynamics, and are derived applying thetheory of martingales, which are a paradigmatic class ofstochastic processes fruitfully applied in probability the-ory [1212] and quantitative finance [1313]. More recently mar-tingale theory has been successfully applied in nonequi-librium thermodynamics [1414–1818], providing further in-sights beyond standard fluctuation theorems, e.g. uni-versal bounds for the extreme-value and stopping-timestatistics of thermodynamic quantities [1414, 1515, 1919–2222].

Work fluctuation theorems at stopping times.We consider thermodynamic systems in contact with athermal bath with inverse temperature β = 1/kBT . TheHamiltonian H of the system depends on time throughan external control parameter λ(t) following a prescribeddeterministic protocol Λ = {λ(t); 0 ≤ t ≤ τ} of fixedduration τ . The evolution of the system is subject tothermal fluctuations and thus we will describe its ener-getics using the framework of stochastic thermodynam-ics [2323–2525]. We denote the state (continuous or discrete)of the system at time 0 ≤ t ≤ τ by x(t), and the prob-ability of observing a given trajectory x[0,t] ≡ {x(s)}ts=0

associated with the driving protocol Λ by P (x[0,t]). Weassume its dynamics is stochastic and Markovian withprobability density %(x, t). Thermodynamic variablessuch as system’s energy E(t) = H(x(t), λ) and entropyS(t) ≡ −kB ln %(x(t), t) are then stochastic processesgiven by functionals of the stochastic trajectories x[0,t].A key result from stochastic thermodynamics is the workfluctuation theorem 〈e−β(W−∆F )〉 = 1 [2626, 2727], which im-plies the second-law inequality 〈W 〉 − 〈∆F 〉 ≥ 0, wherethe averages 〈 · 〉 are done over all possible trajectories ofduration τ produced in the nonequilibrium protocol Λ.

We now ask ourselves whether the work fluctuationtheorem and the second law still hold when averagingover trajectories stopped at random times, following acustom “gambling” strategy. In particular we considergambling strategies defined through a generic stoppingcondition that can be checked at any instant of time tbased only on the information collected about the sys-tem up to that time. In each run, the demon gamblesapplying the prescribed stopping condition, and decideswhether to stop gambling or not depending on the sys-tem’s evolution. In this work, we consider stopping timesobeying T (x[0,t]) ≤ τ for any trajectory x[0,t], i.e. demonswhich are enforced to gamble before or at the end of thenonequilibrium driving. For this class of systems we de-rive the inequality

〈W 〉T − 〈∆F 〉T ≥ −kBT 〈δ〉T , (1)

which involves averages of functionals of trajectories eval-uated at stopping times 〈O〉T =

∑x[0,T ]

P (x[0,T ])O(T ),

i.e. taken over many trajectories x[0,T ], each stopped at a

stochastic time T . In Eq. (11), W (T ) ≡∫ T

0dt ∂tH(x(t), t)

is the work exerted on the system up to time T , and∆F (T ) ≡ F (T ) − F (0) the nonequilibrium free energychange, with F (t) ≡ E(t) − TS(t). Importantly, thequantity δ, denoted here as stochastic distinguishability

δ(t) ≡ ln

[%(x(t), t)

%(x(t), τ − t)

], (2)

is a trajectory-dependent measure of how distinguish-able is %(x, t) with respect to the probability distribution%(x, τ−t) at the same instant of time in a reference time-reversed process which is defined as follows. Its drivingprotocol Λ = {λ(τ − t); 0 ≤ t ≤ τ} is the time-reversedpicture of the forward protocol and its initial distribu-tion is the distribution obtained at the end of the for-ward protocol, i.e. %(x, 0) ≡ %(x, τ). We derive Eq. (11)by extending the martingale theory of stochastic thermo-dynamics to generic driven Markovian processes startingin arbitrary nonequilibrium conditions. This leads us tothe following fluctuation relation at stopping times

〈e−β(W−∆F )−δ〉T = 1, (3)

which implies Eq. (11) by Jensen’s inequality. For theparticular case of deterministic stopping at the end ofthe protocol T → τ , we get δ(T ) → 0 and thus Eqs. (11)and (33) recover respectively the standard second law andthe work fluctuation theorem, as expected.

Equation (11) reveals that the time-asymmetry intro-duced by the driving protocol, 〈δ〉T ≥ 0, enables fora ”second-law violation” at stopping times as the av-erage work cost can be below the free energy change〈W 〉T < 〈∆F 〉T when 〈δ〉T is sufficiently large. We re-mark that 〈∆F 〉T is the free energy change between thefinal state (a mixture of trajectories stopped at differenttimes T ) and the initial state. Thus, upon stopping tra-jectories at stochastic times T requires a work payoff thatcan be below the free energy change of a conventionalthermodynamic process transforming the initial distri-bution to the distribution at stopping times, thereby cir-cumventing the standard second law. Strikingly Eqs. (11)and (33) are valid for any stopping strategy thereby in-troducing a new level of universality. We next put tothe test our results applying one specific set of stoppingtimes to experimental data.

Experimental verification. The experimental setupthat we used to test the aforementioned predictions(shown in Fig. 22a) consists of two capacitively-coupledmetallic islands with small capacitance forming a single-electron transistor (SET) as a detector, and a single-electron box (SEB) as the system [2828, 2929]. The SEB,with capacitance C, is left unbiased: the offset chargeng of the SEB can be externally tuned with a gate volt-age Vg,sys = eng/Cg. At low temperature kBT < e2/2Cthe box can be approximated as a two-state system withcharge number states n = 0 and n = 1, and the off-set charge tuning enables the control of individual elec-trons on the island through the change in its electro-static energy Ec(n− ng)2. The other SET is used as an

3

A

Vg,sys

Cg

Vg,elElectrometer

Vbias

SystemSET

Iel

0.0

0.5

1.0

0.00 0.01 0.02 0.03 0.04 0.05

0.00 0.01 0.02 0.03 0.04 0.05

0.00

0.02

0.04

0.06

0.08

Det

ecte

d st

ates

0.50

0.55

0.60a b

c

FIG. 2. a. Scanning electron micrograph of the single-electronbox (SEB) with false-color highlight on the Cu island (red)and the Al superconducting lead (turquoise). The supercon-ducting leads are tunnel-coupled through thin oxide barriers(yellow) to the island. The DC SET electrometer is coupledcapacitively to the box through a bottom electrode (blue) de-tects the excess charge of the box n(t). b. Representativetime traces of the current measured through the electrometer(red solid line) and its digitized version (black solid line). Theblue dashed line correspond to the driving protocol ng(t) ofduration τ = 0.05s. c. Example traces of the stochastic workdone on the box as a function of time. We execute the follow-ing gambling strategy: the process is stopped at T < τ (blackline) only when the work reaches a threshold value Wth (reddashed line) before τ . On the contrary, the process is stoppedat final protocol time T = τ if the work threshold is neverreached during the driving protocol (blue line).

electrometer biased with a low voltage: through capac-itive coupling to the box, its output current is sensitiveto the box charge state, taking two values correspond-ing to the system states. The tunnelling of an electroninto the island corresponds to a jump between the statesn = 0 and n = 1 and is associated with an energy costε(ng) = Ec(1− 2ng). Through continuous monitoring ofthe box state n(t) (see Fig. 22b), we experimentally eval-uate at real time the heat exchange between the systemand the bath during a driving protocol of the gate voltageng(t). The tunnelling (i.e. heat exchange) events occurat rates of order Γd ∼ 230 Hz. If a jump occurs at time twithin a sampling time δt = 20 µs� Γ−1

d at gate voltageng ≡ ng(t), the work increment is δW = 0 and the heatincrement is δQ = ε(ng) (δQ = −ε(ng)) for an electrontunneling into (out) of the island. Conversely, if no jumpoccurs, δQ = 0 and δW = 2Ec(ng − n)ngδt.

The experimental driving protocol Λ of duration τ isdepicted in Fig. 22b. The system is initially preparedat charge degeneracy, i.e., ng(0) = ng(1) = 1/2 at ther-mal equilibrium where the initial energies of states areequal, following a uniform distribution. Then the en-ergy splitting ε[ng(t)] is tuned according to a linear ramp,λ(t) = 1/2 + ∆ngt/τ , with ∆ng = 0.1 fixed throughout

the experiment. The protocol is repeated several times(∼ 500− 1000) to acquire sufficient statistics. The gam-bling strategy that we chose consists on stopping the dy-namics at stochastic times T when the work exceeds athreshold value Wth (red dashed line) or at τ otherwise.In Fig. 22c we present two examples of stopped work tra-jectories where one reaches the threshold value at a timeT < τ (black line), while the other remains below thethreshold until the final time τ (blue line).

Experimental values of 〈W 〉T −〈∆F 〉T and −kBT 〈δ〉Tare shown in Figure 33a and 33d for two different rampsof durations τ = 0.05s (a) and τ = 0.2s (d) as a func-tion of the work threshold Wth. These results are val-idated and are in good agreement with numerical sim-ulations over the entire threshold range when includingthe experimental uncertainty. For both ramp durations〈W 〉T − 〈∆F 〉T is negative at small Wth, defying theconventional second law but is yet in agreement withEq. (11) within experimental errors. We find that thefaster is the protocol, the more negative 〈W 〉T − 〈∆F 〉Tbecomes, which can be understood as a consequence ofthe irreversibility (and hence 〈δ〉T ) associated with theramp driving speed. For large values of Wth, almostall trajectories are stopped at τ and the conventionalsecond law is recovered, as 〈δ〉T becomes small. Fur-thermore, Figs. 33b and e report the exponential aver-ages 〈e−β(W−∆F )〉T and 〈e−β(W−∆F )−δ〉T evaluated atthe stopping times. Notably, the conventional work fluc-tuation theorem 〈e−β(W−∆F )〉T = 1 only holds for largeWth, while for small Wth, 〈e−β(W−∆F )〉T is significantlygreater than one within experimental errors. On theother hand, we obtain an excellent agreement (with accu-racy ∼ 99.5%) of our fluctuation relation (33) for all valuesof Wth and both ramp speeds. To gain further insights, inFigs. 33c and 33f we show histograms of the stopping timesT and the value of the work at the stopping time W (T ).For small thresholds we observe that the distribution ofT is broad and includes stopping events that take placeat short times T . Γ−1

d (Fig. 33c, top panel). Its corre-sponding distribution of W (T ) (Fig. 33f, top panel) hasa peak at Wth arising from trajectories stopped beforeτ and a tail W (T ) < 〈∆F 〉T from trajectories ending atthe end of the protocol. By increasing the threshold value(Fig. 33c and 33f, middle panels) we reduce the number oftrajectories that stop before τ hence the distribution ofT becomes narrower (Fig. 33c, bottom panel). This effectis accompanied by a broadening of the W (T ) distribu-tion recovering a Gaussian-like shape with mean abovethe free energy change for large enough Wth (i.e. typi-cally far outside the standard fluctuation interval of W ),see Fig. 33f bottom panel.

Quantum gambling. The gambling demon can alsobe extended to the quantum realm by considering theframework of quantum jump trajectories [3030]. This isnot a mere transposition of the classical result in Eq. (33),since new features appear. Here the pure state of thesystem |ψ(t)〉 follows stochastic evolution conditioned onthe measurement outcomes generated by the continuous

4

a b c

d e f

FIG. 3. Dissipated work 〈W 〉T − 〈∆F 〉T (blue) and stochastic indistinguishability at stopping times (red) −kBT 〈δ(T )〉 (dots:experimental data, solid lines: simulation) in charging energy Ec units averaged over many realizations for protocol durationsτ= 0.05 s (a) and τ = 0.2 s (d) as a function of work threshold values. b,e. test of the generalized work fluctuation relation andof Eq. (33) (dots: experimental data, solid lines: simulation) for τ = 0.05 s (b) and τ = 0.2 s (e). c,f. Distributions of stoppingtimes T (c) and corresponding work values W (T ) (f) for a ramp time τ = 0.05 s for work thresholds Wth = 5× 10−4, 3× 10−2

and 10−1Ec. The total uncertainty is shown by shadowed areas; it is the combination of the statistical uncertainty and erroron temperature (about 10%).

monitoring of the environment [3131–3333]. However, the in-trinsic invasiveness of quantum measurements has severenon-trivial consequences for the thermodynamic behav-ior of the system when gambling strategies are to be em-ployed to stop the process.

In this case, we derive the following quantum stopping-time work fluctuation relation (see Methods)

〈e−β[W−∆F ]−δq+∆Sunc〉T = 1, (4)

where again W and ∆F are respectively the workperformed and free energy change during trajectoriesstopped at T . The term δq(t) ≡ ln〈ψ(t)|ρ(t)|ψ(t)〉 −ln〈ψ(t)|Θ†ρ(τ − t)Θ|ψ(t)〉 is the quantum analogue ofEq. (22), ρ and ρ being the density operators in the for-ward and backward process respectively, and Θ the time-reversal (anti-unitary) operator in quantum mechanics.As before, time-inversion at the final instant of time τimplies δq(τ) = 0. The key difference of the quantumfluctuation relation (44) with respect to its classical coun-terpart in Eq. (33) is the appearance of a genuine entropicterm associated to quantum measurements, namely the

“uncertainty” entropy production

∆Sunc(T ) = − ln

(〈n(T )|ρ(T )|n(T 〉)〈ψ(T )|ρ(T )|ψ(T )〉

). (5)

This quantity measures how much more surprisingis a particular eigenstate |n(t)〉 of ρ(t) with respectto the stochastic wave function |ψ(t)〉, as character-ized by the logarithm of the squared Ulhman fidelity,〈ψ(t)|ρ(t)|ψ(t)〉 [2020]. In general, |ψ(t)〉 can be an ar-bitrary superposition of the instantaneous eigenstates|n(t)〉. In the classical limit the stochastic evolutionof |ψ(t)〉 is given by jumps between energy levels andthus |ψ(T )〉 = |n(T )〉. Consequently ∆Sunc(T ) = 0in Eq. (55) and δq(T ) = δ(T ) for any T , thus recov-ering Eq. (33) in the classical limit. The correspond-ing stopping-time second-law inequality for quantum sys-tems reads 〈W 〉T − 〈∆F 〉T ≥ −kBT (〈δq〉T − 〈∆Sunc〉T ),where 〈∆Sunc〉T modifies the entropic balance. Even if〈∆Sunc〉 ≥ 0 for any fixed time t ≤ τ , the average overstopped trajectories 〈∆Sunc〉T may be either positive ornegative depending on the selected gambling strategy.Therefore, the quantum fluctuations induced by mea-

5

surements may act either as an entropy source or as anentropy sink, respectively limiting or enhancing the de-moniac effects reported above.

Discussion. We have introduced and illustrated thestochastic thermodynamics of gambling demons, a lit-tle intelligent being that, by observing the evolution ofa thermodynamic process, applies gambling strategies tostop it at a convenient time. As a consequence the stan-dard formulation of the second law of thermodynamicscan be bypassed. We have exploited this result experi-mentally by applying finite-time horizon gambling strate-gies to a single-electron box driven away from equilib-rium. For this example, we have shown how a drivingprotocol performing work on average can be turned ontoa work extracting protocol by stopping the dynamics us-ing work thresholds.

Our results generalize the second law to arbitrary stop-ping (“gambling”) strategies for classical and quantumsystems driven out of equilibrium. Even though all finite-time horizon gambling strategies fulfil the fluctuation re-lation at stopping times (33) and the second-law-like in-equality (11), not all guarantee average work extractionabove the limits set by the conventional second law. Neg-ative dissipation at stopping times requires an adequategambling strategy and a time asymmetry in the drivingprotocol, hence representing a genuine non-equilibriumeffect. This contrasts with heat and information engineswhich achieve maximal work extraction in the slow qua-sistatic limit.

Our relations are fundamentally different to the en-ergetics of Maxwell’s demons, where the information ac-quired by a feedback controller from the system I leads tothe inequality 〈W 〉− 〈∆F 〉 ≥ −kBT I [33, 44]. In contrast,for gambling demons the stochastic distinguishability setsthe maximum deviations from the standard second law.Thus, our bounds only depend on statistics of the systemand not on the measurement device. Applications to ex-perimental quantum devices [3434, 3535] may allow to exploitquantum superpositions to enhance work extraction be-yond the classical limits, as follows from our quantumextension. Finally, it will be interesting to explore in thefuture optimization of stopping strategies using knowl-edge in quantitative finance (e.g. option pricing, arbi-trage, etc.) and gambling such as Parrondo games [3636].

Acknowledgments. We acknowledge fruitful discus-sions with Christopher Jarzynski. R.F. research has beenconducted within the framework of the Trieste Insti-tute for Theoretical Quantum Technologies (TQT). Thiswork was funded through Academy of Finland Grant No.312057 and from the European Unions Horizon 2020 re-search and innovation programme under the EuropeanResearch Council (ERC) programme.

METHODS

Experimental details. The non-equilibrium occupa-

tion probabilities for forward and backward trajectories,necessary to compute the nonequilibrium free energy, areobtained by numerically solving the master equation overthe full protocol duration. The numerical ”forward” so-lution is in fair agreement with the experimentally recon-structed probability.

The gambling analysis was performed on two sets oftrajectories corresponding to protocol times τ = 0.05 sand 0.2 s. Each point in Fig 33 a,b,d,e corresponds to thesame dataset analyzed with a different threshold condi-tion.

Quantum jump trajectories. In order to describeMarkovian stochastic quantum dynamics, we use the for-malism of quantum jump trajectories [3030]. This frame-work allows to describe the evolution of a pure state ofthe system, |ψ(t)〉, conditioned on a set of outcomes re-trieved from continuous monitoring of the environment.The evolution consist in periods of smooth dynamics in-tersected by quantum jumps occurring at random times,which produce abrupt changes in the state of the system.The occurrence of such jumps is linked to the exchangeof excitations between system and reservoir (e.g. emis-sion and absorption of photons) captured by the detector.Such dynamics is described by the Stochastic Schrodingerequation:

d |ψ(t)〉 = dt

(− i~H(λ) +

∑k

〈L†kLk〉ψ(t) − L†kLk2

)|ψ(t)〉

+∑k

dNk(t)

Lk√〈L†kLk〉ψ(t)

− 1

|ψ(t)〉 . (6)

Here H(λ) is a Hermitian operator (usually the systemHamiltonian), and the operators Lk(λ) for k = 1...K arethe Lindblad (or jump) operators, both of which maydepend on the control parameter λ(t) following the driv-ing protocol Λ = {λ(t); 0 ≤ λ ≤ τ} up to time τ . Therandom variables dNk(t) are Poisson increments associ-ated to the number of jumps Nk(t) of type k detectedup to time t in the process. This variables take mostof the time the value 0, and they become 1 only at spe-cific times tj when a jump of type kj is detected in theenvironment. Here we denoted 〈A〉ψ(t) ≡ 〈ψ(t)|A |ψ(t)〉the quantum-mechanical expectation values, and 1 theidentity matrix.

Recording the different type of jumps occurring dur-ing the stochastic dynamics and the times at which theywere detected, one may construct a measurement recordRτ0 = {(k1, t1), ..., (kJ , tJ)}, where (kj , tj) denotes a jumpof type kj observed at time tj , where j = 1, ..., J for atotal number of jumps J , and 0 ≤ t1 ≤ t2 ≤ ... ≤ tJ ≤ τ .If the average over many processes is taken, the evolutionreduces to a Markovian process for the density operatorof the system ρ(t), ruled by a Lindblad master equa-tion [3737]

ρ(t) = −i~[H, ρ(t)]+∑k

Lkρ(t)L†k−1

2{L†kLk, ρ(t)}. (7)

6

In the case of a thermal environment all jumps occurin the energy basis, leading to the exchange of discreteenergy packets Ek(λ) with the environment that can beinterpreted as heat [3131, 3232]. When the Hamiltonian hasa fixed basis during all the control protocol Λ, a classicalMarkovian process is recovered. In this case, taking onlythe diagonal elements of ρ in the energy basis, we recoverfrom Eq. (77) a classical master equation.

Quantum stochastic thermodynamics. The frame-work of quantum jump trajectories is particularly wellsuited for extending stochastic thermodynamics to thequantum realm [3131–3333, 3838–4545]. An important feature ofquantum setups is the need to place the driven processeswithin a two-measurements scheme. Here the system issubjected to projective measurements in the density op-erator eigenbasis both at the beginning [ρ(0)] and at theend [ρ(τ)] of the protocol Λ. Therefore, in a trajectoryγ{0,τ} ≡ {(n(0), 0);Rτ0 ; (n(τ), τ)} the system is preparedin an eigenstate |n(0)〉 with probability pn(0)(0) in thefirst measurement. Then the state |ψ(t)〉 evolves fromt = 0 up to time t = τ according to a given environmentalmeasurement record Rτ0 = {(k1, t1), ..., (kJ , tJ)}, wherejump processes kj were detected at stochastic times tj .Finally, the system is projected in |n(τ)〉 in the sec-ond measurement. The changes in observables of thesystem such as energy and stochastic entropy are givenby ∆E(τ) = 〈n(τ)|H(λ(τ))|n(τ)〉 − 〈n(0)|H(λ(0))|n(0)〉,and ∆S(τ) = − ln pn(τ)(τ) + ln pn(0)(0), with pn(τ)(τ) =〈n(τ)| ρ(τ) |n(τ)〉 and pn(0)(0) = 〈n(0)| ρ(0) |n(0)〉 theeigenvalues of ρ(τ) and ρ(0), respectively. Averagingthese quantities over many trajectories we recover thestandard expressions for the energy change 〈∆E(τ)〉 =Tr[H(λ(τ)ρ(τ)] − Tr[H(0)ρ(0)] and von Neumann en-tropy change of the system 〈∆S(t)〉 = −Tr[ρ(τ) ln ρ(τ)]+Tr[ρ(0) ln ρ(0)].

A key quantity measuring the irreversibility of thephysical process along single trajectories is the stochasticentropy production

∆Stot(τ) = lnP (γ{0,τ})

P (γ{0,τ})= ∆S(τ) +

J∑j=1

∆Skjenv, (8)

where P (γ{0,τ}) is the probability that trajectory γ{0,τ}is generated, and P (γ{0,τ}) is the probability to obtain

the time-reversed trajectory γ{0,τ} = {n(τ); R0τ ;n(0)} in

the time-reverse or backward process. In the backwardprocess, the time-reversed protocol Λ is implemented overthe (inverted) final state of the system in the forward pro-

cess, ρ ≡ Θρ(τ)Θ†. The term ∆Skjenv in Eq. (88) is the en-

vironmental entropy change due to the jump kj [3333]. Thestochastic entropy production obeys the integral fluctu-ation theorem 〈e−∆Stot(τ)〉 = 1, leading to the secondlaw inequality 〈∆Stot(τ)〉 ≥ 0, where here the average istaken over complete trajectories γ{0,τ}.

In the case of a driven system in contact with a single

thermal reservoir at temperature T , we have∑j ∆S

kjenv =

−Q(τ)/T , where Q(τ) is the heat realeased by the reser-voir during the trajectory. In such case the entropy pro-duction reads:

∆Stot(τ) = β [W (τ)−∆F (τ)] , (9)

where W (τ) = ∆E(τ)−Q(τ) is the stochastic work per-formed during the trajectory, and ∆F (τ) = ∆E(τ) −kBT∆S(τ) the non-equilibrium free energy change.

Stopping quantum trajectories. The introductionof the two-measurements scheme has non-trivial conse-quences for the thermodynamic behavior of the systemwhen gambling strategies are to be employed to stop theprocess. The reason is that thermodynamic quantitieslike work or free energy are only well defined once thesecond measurement in the scheme has been performed,which requires performing the second measurement atthe time at which the trajectory is stopped. However,if the trajectory is stopped before the end of the pro-tocol, the introduction of a projective measurement atany time t ≤ τ may disturb the trajectory. A quantumgambling demon willing to decide to stop or not the pro-cess at T must take the decision before the second mea-surement is performed, since otherwise quantum Zenoeffect will trivialize the whole evolution. Therefore, thegambling demon decides to stop or not at T accordingto a selected stopping condition based on the informa-tion {(n(0), 0);RT0 }. If he stops, then the final measure-ment is performed in the ρ(T ) eigenbasis, completing thestopped trajectory γ{0,T } = {(n(0), 0);RT0 , (n(T ), T )},otherwise the measurement is not performed and the evo-lution continues. This process introduces a final unavoid-able disturbance of quantum nature in the stopped tra-jectories, that the gambling demon is not able to predictand/or control, with thermodynamic consequences.

In order to handle the thermodynamics of the measure-ment disturbance, we use the following decomposition ofthe stochastic entropy production in Eq. (88):

∆Stot(t) = ∆Sunc(t) + ∆Smar(t). (10)

Here the first term is the “uncertainty” entropy produc-tion already introduced in Eq. (55), and we denote thesecond term in Eq. (1010) as the “martingale” entropy pro-duction:

∆Smar(t) = − ln

(〈ψ(t)|ρ(t)|ψ(t)〉

pn(0)(0)

)+

J∑j=1

∆Skjenv. (11)

This quantity represents a “classicalization” of thestochastic entropy production (88), containing a slightlymodified boundary term which gets ride of the final pro-jective measurement impact (first term), and the full ex-tensive part due to the environmental entropy fluxes (sec-ond term).

Quantum Martingale theory. Our resutls for clas-sical work fluctuation relations at stopping times derive

7

from a more general martingale theory for entropy pro-duction that applies to both quantum and classical ther-modynamic systems. This theory relates irreversibility,as measured by entropy production, in generic nonequi-librium processes with the remarkable properties of mar-tingales processes.

A martingale process is a stochastic process defined ona probability space whose expected value at any time tequals its value at some previous time s < t when condi-tioned on observations up to that time s. More formally,M(t) is a martingale if it is bounded 〈M(t)〉 < ∞ forall t, and verifies 〈M(t)|M{0,s}〉 = M(s), where the lateraverage is conditioned on all the previous values M{0,s}of the process up to time s [1212].

We consider conditional averages of entropy produc-tion over trajectories with common history up to a cer-tain time t ≤ τ before the end of the protocol Λ, whichconstitutes the key ingredient for developing a martingaletheory [1414, 1515]. We introduce the conditional average of ageneric stochastic process O(t) defined along a trajectoryγ{0,t} as 〈O(τ)|γ[0,t]〉 =

∑n(τ),Rτt

O(τ)P (γ{0,τ}|γ[0,t]),

where the condition is made with respect to the ensem-ble of trajectories γ[0,t] ≡

⋃ts=0 γ{0,s} including all out-

comes of trajectories eventually stopped at all intermedi-ate times in the interval [0, t]. However, as shown in theSI, we have P (γ{0,τ}|γ[0,t]) = P (γ{0,τ}|γ{0,t}), and then〈O(t)|γ[0,t]〉 = 〈O(t)|γ{0,t}〉.

We identify the following martingale process (see SI)

〈e−∆Smar(τ)−δq(τ)|γ[0,t]〉 = e−∆Smar(t)−δq(t), (12)

where we recall the definition of the quantum version ofthe uncertainty distinguishability

δq(t) = ln

(〈ψ(t)|ρ(t)|ψ(t)〉

〈ψ(t)|Θ†ρ(τ − t)Θ|ψ(t)〉

). (13)

Notably, the average of δq(t) at fixed times t ≤ τequals the relative entropy (Kullback-Leibler divergence)between the forward and backward density operators〈δq(t)〉 =

∑γ P (γ[0,t])δq(t) = D[ρ(t)||Θ†ρ(t)Θ] ≡

Tr[ρ(t)(ln ρ(t) − ln Θ†ρ(t)Θ)], which provides aninformation-theoretical measure of the irreversibility inthe process [4646, 4747]. Moreover, we proof in the SI thatthe uncertainty entropy production in Eq. (55) fulfills thegeneralized fluctuation relation 〈e−∆Sunc(τ)|γ[0,t]〉 = 1.

Applying Doob’s optional sampling theorem [4848, 4949] tothe martingale process in Eq. (1212), and using the expres-sion of the split (1010) of entropy production, we obtain

〈e−∆Stot−δq+∆Sunc〉T = 1, (14)

with ∆Stot given in Eq. (88) and the average 〈O〉T =∑γ P (γ{0,T })O(T ) is taken over stopped trajectories.

Here T is a bounded stopping time, meaning that T < cfor some arbitrary constant c. A proof of Eq. (1414) is givenin SI. If we assume a single thermal reservoir, hence weget Eq. (44) in terms of the work by means of Eq. (99).Classical Martingale theory. The classical limit ofour results is obtained when the whole evolution occursin the Hamiltonian eigenbasis, [ρ(t), H(t)] = 0 at alltimes. Then the stochastic wavefunction |ψ(t)〉 is alwaysan eigenstate of ρ(t), that is 〈ψ(t)| ρ(t) |ψ(t)〉 = pn(t)(t) ≡%(n(t), t). This leads to classical trajectories where everyjump corresponds to a change in the system micro-stateand therefore we get γ{0,τ} = {n(t)}τt=0, while the initialand final measurements of the two-measurements schemebecome superflous.

Therefore we recover from Eq. (1111) the classical ex-pression of the stochatic entropy production [2424], namely

∆Smar(t) = ∆Stot(t) = ∆S(t)− βQ(t). (15)

Analogously, from Eq. (55) we obtain ∆Sunc(t) = 0 forall t and Eq. (1313) reduces to its classical counterpart inEq. (22). Substituting into Eq. (1212) we obtain the Mar-tingale:

〈e−∆Stot(τ)−δ(τ)|γ[0,t]〉 = e−∆Stot(t)−δ(t), (16)

leading to the following stopping-times fluctuation theo-rem for the entropy production, and second law at stop-ping times:

〈e−∆Stot−δ〉T = 1 ; 〈∆Stot〉T ≥ −〈δ〉T . (17)

Finally, using the expression for the entropy productionin Eq. (99) in terms of work and free energy, we obtain thesecond law inequality in Eq. (11) and the work fluctuationrelation in Eq. (33). If the system remains in a (time-symmetric) steady state during the evolution, that is,%(n, τ − t) = %(n, t) ≡ %st(n), then δ(t) = 0 for all t,and our results reduce to the steady-state second law atstopping times, 〈∆Stot〉T ≥ 0, or equivalently 〈W 〉T −〈∆F 〉T ≥ 0 [2121].

[1] A. Rex and H. S. Leff, Maxwell’s demon 2: entropy, clas-sical and quantum information, computing (Taylor andFrancis, 2002).

[2] K. Maruyama, F. Nori, and V. Vedral, Rev. Mod. Phys81, 1 (2009).

[3] T. Sagawa and M. Ueda, Phys. Rev. Lett. 100, 080403(2008).

[4] T. Sagawa and M. Ueda, Phys. Rev. Lett. 104, 090602(2010).

[5] L. Del Rio, J. Aberg, R. Renner, O. Dahlsten, and V. Ve-dral, Nature 474, 61 (2011).

[6] J. M. Parrondo, J. M. Horowitz, and T. Sagawa, NaturePhys. 11, 131 (2015).

[7] E. Lutz and S. Ciliberto, Phys. Today 68, 30 (2015).

8

[8] M. Gavrilov and J. Bechhoefer, Phys. Rev. Lett. 117,200601 (2016).

[9] M. Ribezzi-Crivellari and F. Ritort, Nature Phys. 15, 660(2019).

[10] P. A. Camati, J. P. Peterson, T. B. Batalhao, K. Micadei,A. M. Souza, R. S. Sarthour, I. S. Oliveira, and R. M.Serra, Phys. Rev. Lett. 117, 240502 (2016).

[11] N. Cottet, S. Jezouin, L. Bretheau, P. Campagne-Ibarcq,Q. Ficheux, J. Anders, A. Auffeves, R. Azouit, P. Rou-chon, and B. Huard, PNAS 114, 7561 (2017).

[12] D. Williams, Probability with martingales (Cambridgeuniversity press, 1991).

[13] S. Pliska, Introduction to mathematical finance (Black-well publishers Oxford, 1997).

[14] R. Chetrite and S. Gupta, J. Stat. Phys. 143, 543 (2011).

[15] I. Neri, E. Roldan, and F. Julicher, Phys. Rev. X 7,011019 (2017).

[16] C. Moslonka and K. Sekimoto, arXiv preprintarXiv:2001.04842 (2020).

[17] H. Ge, C. Jia, and X. Jin, arXiv preprintarXiv:1811.04529 (2018).

[18] Y.-J. Yang and H. Qian, Phys. Rev. E 101, 022129(2020).

[19] R. Chetrite, S. Gupta, I. Neri, and E. Roldan, EPL 124,60006 (2019).

[20] G. Manzano, R. Fazio, and E. Roldan, Phys. Rev. Lett.122, 220602 (2019).

[21] I. Neri, E. Roldan, S. Pigolotti, and F. Julicher, J. Stat.Mech. 2019, 104006 (2019).

[22] I. Neri, Phys. Rev. Lett. 124, 040601 (2020).[23] K. Sekimoto, Stochastic energetics, Vol. 799 (Springer,

2010).[24] U. Seifert, Rep. Prog. Phys. 75, 126001 (2012).[25] C. Jarzynski, Annu. Rev. Condens. Matter Phys. 2, 329

(2011).[26] C. Jarzynski, Phys. Rev. Lett. 78, 2690 (1997).[27] U. Seifert, Phys. Rev. Lett. 95, 040602 (2005).[28] O. Maillet, P. A. Erdman, V. Cavina, B. Bhandari, E. T.

Mannila, J. T. Peltonen, A. Mari, F. Taddei, C. Jarzyn-ski, V. Giovannetti, and J. Pekola, Phys. Rev. Lett. 122,150604 (2019).

[29] J. P. Pekola and I. M. Khaymovich, Annu. Rev. Condens.

Mat. Phys. 10, 193 (2019).[30] H. M. Wiseman and G. J. Milburn, Quantum measure-

ment and control (Cambridge university press, 2009).[31] F. W. J. Hekking and J. P. Pekola, Phys. Rev. Lett. 111,

093602 (2013).[32] J. M. Horowitz, Phys. Rev. E 85, 031110 (2012).[33] G. Manzano, J. M. Horowitz, and J. M. Parrondo, Phys.

Rev. X 8, 031037 (2018).[34] Z. K. Minev, S. O. Mundhada, S. Shankar, P. Reinhold,

R. Gutierrez-Jauregui, R. J. Schoelkopf, M. Mirrahimi,H. J. Carmichael, and M. H. Devoret, Nature 570, 200(2019).

[35] K. W. Murch, S. Weber, C. Macklin, and I. Siddiqi,Nature 502, 211 (2013).

[36] G. P. Harmer and D. Abbott, Nature 402, 864 (1999).[37] G. Lindblad, Comms. Math. Phys. 48, 119 (1976).[38] J. M. Horowitz and J. M. Parrondo, New J. Phys. 15,

085028 (2013).[39] B. Leggio, A. Napoli, A. Messina, and H.-P. Breuer,

Phys. Rev. A 88, 042111 (2013).[40] M. Campisi, J. P. Pekola, and R. Fazio, New J. Phys.

17, 035012 (2015).[41] G. Manzano, J. M. Horowitz, and J. M. Parrondo, Phys.

Rev. E 92, 032129 (2015).[42] Z. Gong, Y. Ashida, and M. Ueda, Phys. Rev. A 94,

012107 (2016).[43] F. Liu and J. Xi, Phys. Rev. E 94, 062133 (2016).[44] C. Elouard, D. A. Herrera-Martı, M. Clusel, and

A. Auffeves, npj Quant. Info. 3, 9 (2017).[45] B. Karimi and J. P. Pekola, Phys. Rev. Lett. 124, 170601

(2020).[46] R. Kawai, J. M. Parrondo, and C. Van den Broeck, Phys.

Rev. Lett. 98, 080602 (2007).[47] T. Sagawa, Second Law-Like Inequalities with Quan-

tum Relative Entropy: An Introduction in Lectures onQuantum Computing, Thermodynamics and StatisticalPhysics, edited by M. Nakahara and S. Tanaka, KinkiUniversity Series on Quantum Computing (World Scien-tific, 2012).

[48] J. Doob, Stochastic Processes (John Wiley and Sons,1953).

[49] J. Doob, Measure Theory, Vol. 143 (Springer Science andBusiness Media, 1994).