Recently my intellectual curiosity has driven me towards the topics of stochastic thermodynamics, analog computing, and probabilistic ML all connected to the the recent pop in thermodynamic computing by start-ups Extropic and Normal Computing. I can't say I know enough to really explain what the value is, other than knowing that these types of computing paradigms are extremely energy efficient and are one of the best prospects for scalable AI. Therefore in light of this, I'm going to pull a little back from my self-learning activities on graph NN (😢) and focus on these topics; I have no idea for how long or how much steam I'll maintain. My future post will probably be on these topics, or I might just keep with posting on new "tools" I'm exploring.
How am I going to achieve some baseline knowledge? Well I already know thermodynamics pretty well, but it's all from an equilibrium perspective with thermodynamic potentials and state variables. What I need is to study a bit more non-equilibrium thermodynamics, which is the focus on the time evolution of the state variables. But in particular to thermodynamic computing is stochastic1 thermodynamics, which is the non-equilibrium dynamics due to random fluctuations that occur on small collection/subset. In other words, the study of random dynamics in microscopic/mesoscopic collections of particles. The reason this is different that just non-equilibrium thermo, at least I think, is that when you deal with small collections of micro/meso-scopic systems, the statistical law of large numbers does not guarantee the same convergence of ensemble averages to thermodynamic values as in macroscopic systems. Meaning that expectation values remain meaningful but must be interpreted with consideration of significant fluctuations around these values. For instance, temperature can still be considered as an average kinetic energy of the ensemble, but with an increased importance of understanding the distribution and fluctuations of these energies.
Futhermore, stochastic thermodynamics distinguishes itself by explicitly accounting for the random, fluctuating dynamics of individual particles or small assemblies of particles. One of the key concepts in stochastic thermodynamics is that the fluctuation theorem, which quantifies the probability of observing fluctuations away from the average behavior in non-equilibrium systems, is dominant. These theorems provide a deep understanding of the irreversibility and the second law of thermodynamics in a statistical sense, showing that while the second law holds on average, there are observable, albeit rare, violations on the microscopic scale due to thermal fluctuations.
In essence, stochastic thermodynamics provides a comprehensive framework to study the energetics and dynamics of systems where fluctuations cannot be ignored, making it crucial for understanding processes at the molecular and nanoscale, especially in biological systems and nanotechnology.
Warning
This is what I think it all means! I'll find out if I am wrong or right as I learn more.
So what do I need to do? Well I have the following set of books that I'm going to read through:
- L. Peliti, S. Pigolotti, Stochastic Thermodynamics: An Introduction, Princeton University Press, 2021. URL.
- Ulmann, Bernd. Analog Computing, Berlin, Boston: De Gruyter Oldenbourg, 2022. DOI
- K.P. Murphy, Probabilistic Machine Learning: An introduction, MIT Press, 2022. URL.
I'm going to read 1 and 2 in tandem, and will read 3 over the next 6 months. The goal is to understand stochastic thermodynamics and analog computing thoroughly. Then I'll read through the papers [1-3] on thermodynamic computing. Finally I'll try to connect 3 to 1 and 2.
In the process of this learning, I'm going to write my notes up in a small handbook using the latex format graciously provide by Francois Fleuret[4]. No promises on the quality of the book, only that it will contain my notes on 1 and 2. I'll also write blog post, but these will most likely be renditions of my notes in to book. I'll probably make my notes available on my website if I'm happy with the final result.
Footnotes
-
The use of the term "stochastic" refers to random fluctuations in non-equilibrium thermodynamic processes at microscale. ↩
References
[1] T. Conte, et al., Thermodynamic Computing, (2019). https://doi.org/10.48550/arXiv.1911.01968.
[2] P.J. Coles, et al. Thermodynamic AI and Thermodynamic Linear Algebra, in: Machine Learning with New Compute Paradigms, 2023. https://openreview.net/forum?id=6flkWTzK2H.
[3] A.B. Boyd, J.P. Crutchfield, M. Gu, F.C. Binder, Thermodynamic Overfitting and Generalization: Energetic Limits on Predictive Complexity, (2024). https://doi.org/10.48550/arXiv.2402.16995.
[4] François Fleuret’s git - littlebook.git/tree. https://fleuret.org/git/littlebook (accessed March 13, 2024).
No comments:
Post a Comment
Please refrain from using ad hominem attacks, profanity, slander, or any similar sentiment in your comments. Let's keep the discussion respectful and constructive.