Stochastic Bits

We are all familiar with a bit, a 0 or 1, a false or true, that describes a discrete binary outcome. We can also think of a probabilistic bit, representing a continuous probability between 0 and 1. We then have quantum bits, which represents the superposition state $\vert\Psi\rangle = a\vert 0 \rangle + b\vert1\rangle$, with the probability of measuring a $\vert 0 \rangle$ or $\vert 1 \rangle$ given by the probability amplitude squared. But what if we have the scenario where randomness is introduced to the information state?

s-bit

A stochastic bit, s-bit, which is a bit which experiences random dynamics (i.e., stochastic) between two states:

$$ \begin{equation} s(t) = \begin{cases} 0 & p(t) \\ 1 & 1-p(t) \end{cases} \label{eq:sbit_discrete} \end{equation} $$

This says that the state at any given time of an s-bit, $s(t)$, is 0 or 1 given the probability $p(t)$. The signal, $s(t)$, could look something like:

Stochastic bit.

If we had a continuous scenario where we deal with nats¹, we can write this as:

$$ \begin{equation} s(t) = \int_{-\infty}^{\infty} x \cdot p(x,t) \, dx \label{eq:sbit_cont} \end{equation} $$

with $x$ being a random variable with probability density $p(x,t)$ corresponding to the signal state and $t$ the stochastic dynamic time variable. We can get something like:

Stochastic unit.

This is emblematic of random-walk or Brownian motion, that is, a drift in the signal due to random motion (i.e. noise). Typically when it includes time domain processes, it is usually called the continuous-time stochastic process or Wiener process.

Why use s-bit

As Patrick J. Coles puts it in his good overview video [1], some ideas are to think about how stochasticity can be used as a resource in ML and AI. He gives some examples on how random neuron dropout is used to prevent overfitting and sampling for generative AI uses random noise. Moreover, the interesting thing is the shift from discrete to continuous variables, since in ML/AI the weights, features, and probabilities are over continuous values. The idea is to use the stochastic behavior of physical systems to process information. In the video Patrick calls these s-units which seems fitting. All this obviously leads to thermodynamic computing which I'm diving into².

Footnotes

---

1. When dealing with continuous variables we use nats, a natural unit of information, which can be calculated from the Shannon entropy as $H(s) = -\int_{-\infty}^{\infty} s(t) \ln(s(t)) \, dt$ ↩

2. See my post on my recent focus on thermodynamic computing. ↩

References

[1] Thermodynamic AI and the Fluctuation Frontier | Qiskit Seminar Series with Patrick Coles, Qiskit Quantum Seminar, 2023. URL (accessed March 19, 2024).

[2] P.J. Coles, C. Szczepanski, D. Melanson, K. Donatella, A.J. Martinez, F. Sbahi, Thermodynamic AI and the fluctuation frontier, (2023). arXiv.

Reuse and Attribution