Probability is the language of measurable spaces and measurable functions.
There are two flavors of probability.
In the classical world, the state space X and the value set V of a given observable are both measure spaces.
The observable is a measurable function X → V.
To measure it is to compute the pullback of measurable sets along this function.
(Or rather, that is the act of inferring something about the state from the measurement.)
Nowhere in this setup is it necessary to reify individual points.
We only care about measurable functions up to measure-zero equivalence:
Only the σ-algebra really matters.
(And one never has the absolute precision of individual points in real life.)
This omission has a remarkable analogue in the setting where measure spaces are replaced by topological spaces.
I mean how the notion of a locale (i.e., infinitely distributive lattice) generalizes Hausdorff’s point-set topology.
The open sets of a topological space form a locale, but not every locale is built out of points.
There are “pointless” locales,
but it would be more helpful to discourage thinking of locales as structured sets altogether.
Otherwise said: When we do topology, we may enlarge our conception of what a point has to be.
Topology, unlike geometry, need not start from Euclid’s primitives
(“A point is breadthless width…”).
Locales would be banal if they did not say something new about functions.
Sheaves on locales are the language of functions without points.
It is the infinitary distributive law that unifies our intuitions about
(1) locality,
(2) the commutativity of functions, and
(3) classical logic.
Topoi are essentially the categories formed by sheaves on locales.
Grothendieck was led to them when Hausdorff’s axioms proved inadequate to transplant the analytic topology of the complex numbers to other fields.
Merely to treat the Zariski spectrum of an algebraic closure as both a point and a covering space simultaneously, we must overhaul the classical formalism.
The proof of the Weil Conjectures is part of this heritage of ideas.
In the quantum world, it is no longer relevant that X be a measure space.
Instead, X is the projective space of lines through the origin of a fixed Hilbert space H.
The observable is no longer a function, but merely the structure of the pullback:
That is, a map from measurable subsets of V to self-adjoint projections of H,
subject to axioms of countable additivity, normalization, and so forth.
We may regard this pullback as a homomorphism of rings,
from the commutative ring of bounded measurable functions on V
into the noncommutative ring of bounded linear operators on H
(in practice, distributions on V, not functions, are needed).
To measure the observable is no longer to pull back measurable sets to measurable sets.
Rather,
(1) A range (more generally, a distribution) of observed values defines a projection (more generally, a bounded operator) T on H.
(2) The operator produces a function on X by way of the Hilbert inner product:
x ↦ ⟨x|T|x⟩.
This inner product is the probability that in state x, we would obtain the range of values that we have observed.
In the quantum world, what replaces the distributive lattice of measurable subsets of X is the “orthomodular” lattice of closed subspaces of H.
The failure of distributivity in the latter captures the failure of our intuitions about locality, commuting functions, etc. in quantum mechanics.
(Exercise: Phrase the double-slit experiment as a failure of the distributive law for propositions about the world.)
One might ask, naively:
If we were subatomic creatures, taking quantum notions rather than classical notions as primitive intuitions,
what sort of mathematics might we invent in place of the macroscopic mathematics of topology and geometry?
(A mathematics in which locality is not a self-evident axiom.)
What would be the quantum replacements of open sets, topologies, locales, and so on?
How would a subatomic creature study Diophantine equations?
Before making any attempt to draw crude analogies, we observe two ways in which the quantum formalism is clumsier than the classical one.
(1) The state space and the space of observable values are not on the same footing:
They are not objects of the same type.
To fix this, one might view the algebra of bounded functions L^∞(V) as an algebra of bounded operators on L^2(V).
Then an observable would at least become a morphism from one ring of bounded operators to another.
This is how one arrives at the notion of a von Neumann algebra
(of which L^∞(V) and B(H) are both examples).
But an asymmetry between L^∞(V) (commutative) and B(H) (noncommutative) remains.
Otherwise said: If L^2(V) is regarded as an abstract Hilbert space L,
then picking out L^∞(V) inside B(L) is almost as much structure as picking out V such that L^2(V) ≃ L.
We haven’t really gotten rid of V.
So the morphism-structure of quantum mechanics is not quite the structure of the category of von Neumann algebras.
(2) The Hilbert space is essential to the measurement formalism.
I mean that the construction T ↦ (x ↦ ⟨x|T|x⟩) is crucial,
being part of what replaces the whole operation of pullback of measurable functions.
Is it a measure-theoretic structure that admits no purely topological analogue?
It feels banal or wrong to replace the Hilbert inner product with an arbitrary inner product.
Experience teaches us that positivity usually means something important.
We could ask what aspects of ⟨ | ⟩ may be expressed in terms of the lattice itself,
if we believe that the salient property of the points x in X is that they form maximal elements of the orthomodular lattice.
Regardless, the thrust of this observation is the essential role of the field of complex numbers.
Experience shows that the analogues of L^2 structures over arithmetic (e.g., finite) fields are not obvious, and usually, impossible to guess.

Have you ever looked at Manin's works on quantum aspects of arithmetic? He talks about a few of these things you mention above. I'm also curious by what you might mean by analogues of L^2 structures over arithmetic fields, if you're willing to elaborate or at least point to an example of what you're referring to.
I have seen, but not read, the papers of Manin to which you refer! This piece was more of a personal reaction to learning some basic principles of quantum logic. (I was also vaguely inspired by parts of Manin’s essay about notions of dimension, and by Cartier’s essay, “A Mad Day’s Work…”)
As for the L^2 stuff: I was suggesting that the Hilbert spaces in QM — even those whose elements are pictured as functions on measure spaces — should not be analogized to the rings that appear in algebraic geometry, where we picture of the elements of a ring R as functions on Spec R. In particular, inner products on vectors over finite or nonarchimedean local fields are not genuine analogues of the inner product on a Hilbert space. People do study L^2 functions on measure spaces of arithmetic origin, such as p-adic groups endowed with their Haar measure, but this is yet another direction unrelated to my point in the essay/outline.
My main point, phrased anew: Weil’s Rosetta Stone draws a surprising analogy between how we manipulate the arithmetic of integers and how we perceive the geometry of polynomials. But the latter emerges from a classical, not quantum, experience of the physical world. It depends on our being macroscopic creatures for whom “local” and “global” have a specific meaning. In QM, locality is an emergent phenomenon, created by the special situation of observables that commute. So one is led to ask whether there’s some deeper Rosetta Stone, still delivering insight into the integers from physical intuitions, of which Weil’s Rosetta Stone would be the macroscopic limit.
Ah, I see what you're getting at. Thanks for the background and additional detail. I think even seasoned hard analysts would agree that L^2(R) is way more complicated than R[x]. Embarrassingly, I still don't have a good grasp on why the standard functional analytic machinery doesn't work so well over p-adics, aside from the facile observation that the naive topology is almost useless for most of analysis, the (triumphant!) harmonic analytic works you alluded to in your comment aside.
If you like thinking about these kinds of things, Manin is really fun place to jump off from. He's not as technical as his (many) students, thinks very philosophically, and his love for the subfields he's trying to combine really shines. Here's an oft-quoted line from the introduction of Frobenius Manifolds, Quantum Cohomology, and Moduli Spaces: "The mathematical language of classical physics is based upon real numbers...The mathematical language of quantum physics is based upon complex numbers, and it would be natural to expect that the complex analytic and the algebraic geometry should replace the differential geometry of the classical period." He then casts a number of milestones in late 20th century physics in that framework, despite the fact almost all of them are written in the language of the latter. A program? Not even a sketch of one, alas, but a great read nonetheless... and maybe that thing I've been putting off learning would be just perfect for getting over that pesky difficulty in that example or cleaning up that annoying argument...
A quantum version of the Langlands program recovering the classical in the limit would indeed be something truly awesome, in the big as well as the small.