Operator-qualified sensing and physical inference

Scope

This chapter defines what an adaptive system is allowed to claim from a physical measurement. A sensor does not deliver a scene, object, or fact. It delivers a finite observation produced by an aperture, illumination pattern, medium, detector, clock, calibration state, acquisition policy, and noise process. Inference adds assumptions and prior information.

The architectural consequence is precise: decoded tensors may be convenient runtime inputs, but they are not self-describing evidence. The system must keep enough of the measurement operator, uncertainty, validity envelope, and lineage to know what the observation could resolve, what it could not resolve, and when another measurement is worth buying.

This chapter is operationalized by:

1. the optics, photonics, and inverse-sensing audit;
2. the operator-qualified optical-inference mathematics; and
3. Fixture F-007, which tests the full contract against inverse-method, active-sensing, calibration, control, digital-accelerator, and passive-optics baselines.

Biological observation

Biological sensing is already physical inference. An eye has finite aperture, spectral sensitivity, sampling density, integration time, dynamic range, blind regions, motion, adaptation, and a body that can change viewpoint. A useful percept can therefore depend on both received evidence and prior structure. Movement can reveal a surface that one view leaves ambiguous; longer exposure can buy photons while losing temporal resolution; adaptation can extend useful operation while changing the response function.

The transferable observation is not a particular visual anatomy. It is the closed coupling among:

• a bounded physical measurement channel;
• an internal estimate that remains conditional on that channel;
• actions that change future observability;
• calibration and adaptation over multiple timescales; and
• task-specific decisions made before every latent detail is known.

Optics makes those constraints measurable. Finite apertures, null spaces, photon statistics, ambiguity classes, saturation, and drift are established properties of physical sensing (C-970–C-988). They prevent a fluent reconstruction from silently becoming stronger evidence than the acquisition supplied.

Proposed AI translation

Preserve the evidence-producing operator

For latent physical state $x_e$ in episode $e$, let acquisition $t$ produce

where:

• $y_{e,t}$ is the raw observation in detector counts [count] or another declared sensor unit;
• $a_{e,t}$ is the acquisition action, such as viewpoint, exposure, wavelength, or illumination pattern;
• $c_{e,t}$ is the calibrated parameter vector in declared native units;
• $\nu_{e,t}$ is the immutable operator-version identifier;
• $\mathcal H_{\nu_{e,t}}$ is the physical forward operator;
• $g_{e,t}$ is detector conversion, clipping, and readout response; and
• $n_{e,t}$ is only the residual noise represented by an explicit likelihood.

The observation record travels with operator version, calibration covariance, saturation/dead-time mask, capture and receipt times, preprocessing lineage, and validity envelope. A compact representation can replace the raw record only for a registered family of future queries and only while reconstruction, uncertainty, and provenance obligations remain satisfied. This connects the versioned observation contract, semantic compaction, and value-aware retention. A friction coefficient is one interface-specific instance: its parties, motion, load, environment, history, operator, support, and evidence age travel with the estimate (C-1497).

Separate measured information from prior-supported reconstruction

For a linearized operator $H=U\Sigma V^*$, a component along $v_j$ with singular value $\sigma_j=0$ lies in the measurement null space. A decoder may still propose a plausible value for that component, but the value comes from a prior, another measurement, or a convention—not from this observation.

Every output therefore carries three distinguishable uncertainty sources:

The separation matters under compressed sensing, phase retrieval, computational super-resolution, blind calibration, and learned reconstruction (C-972, C-976–C-981, C-985). Pixel count, sharpness, or confidence cannot substitute for newly identified physical information.

The existing Fixture F-007 likelihood plot makes that distinction explicit. It is an analytical identifiability example, not an empirical superiority result.

Buy another measurement only when it changes the decision frontier

Let $b_t$ be the current belief, $a$ a safe acquisition action, $d$ a downstream decision, and $U(d,\theta)$ task utility for uncertain state $\theta$. The expected value of information is

No scalar acquisition price is assumed. The controller compares EVI against a cost vector containing at least photons [count], energy [J], latency [s], dose or disturbance in its task-specific unit, actuator wear [cycles], and risk on a declared scale. An action is admissible only inside its safety and authority envelope.

This converts active perception from “collect more data” into a resource allocation problem. The active path must beat fixed acquisition, greedy value of information, Bayesian experiment design, POMDP planning, and model-predictive control at equal opportunity and cost (C-975).

Monitor validity instead of trusting calibration indefinitely

Calibration is versioned state, not a one-time property of a device. Reference channels and task-independent residuals monitor alignment, gain, timing, temperature, background, saturation, and component aging. A threshold crossing does not identify the cause; it changes what action is permitted.

A validity transition can trigger, in order:

1. a qualified reduction in confidence or supported query set;
2. a new reference or calibration acquisition;
3. rerouting to a different sensor or computational path;
4. a digital or conservative fallback;
5. reset, repair, or replacement; and
6. abstention when none of those paths restores the evidence contract.

Blind self-calibration is tested for identifiability, and task residuals are not allowed to conflate scene shift with device drift (C-984–C-989).

Route a transform to the substrate that actually makes it cheap

Passive and active optical hardware can execute physically matched linear transforms, sometimes before an observation becomes a large digital tensor. That is valuable when the input is already optical, the transform is reusable, conversion can be avoided, and required precision fits the device envelope.

It is not a general preference for an optical path. The route record declares:

• input locality and format;
• transform identity, reuse count, sparsity, and required precision;
• source, modulator, detector, ADC/DAC, control, thermal, and host work;
• device-specific calibration, mismatch, yield, drift, and age;
• accepted-output latency and quality; and
• fallback and migration cost.

Routing then compares passive optics, a photonic core, a digital accelerator, and hybrid compositions on the same workload and service boundary. Optical propagation is credited only for work it actually displaces (C-989–C-999).

One closed contract

flowchart LR
    world["Hidden physical state<br/>scene · phase · spectrum · dynamics"] --> operator["Versioned measurement operator<br/>aperture · illumination · optics · detector · calibration"]
    action["Costed sensing action<br/>pattern · angle · wavelength · exposure"] --> operator
    regime["Hidden regime<br/>photon flux · background · drift · temperature · saturation"] --> operator
    operator --> raw["Raw observation + metadata<br/>counts · timestamps · masks · covariance · operator version"]
    raw --> ambiguity["Physical limits<br/>diffraction · shot noise · null space · phase ambiguity"]
    ambiguity --> reconstruct["Qualified inference<br/>inverse method · prior · uncertainty · abstention"]
    reconstruct --> query{"More information worth its cost?"}
    query -->|"yes, safe"| action
    query -->|"no"| decide["Decision or retained artifact"]
    monitor["Reference channel + residual monitor<br/>drift · alignment · gain · thermal state"] --> operator
    monitor --> gate{"Inside validity envelope?"}
    gate -->|"no"| recover["Recalibrate · reroute · digital fallback · reset"]
    recover --> operator
    route["Operator-qualified route<br/>passive optics · photonic core · digital accelerator"] --> transform["Physical or digital transform"]
    raw --> route
    transform --> reconstruct
    compact["Query-registered compaction<br/>raw · calibrated sufficient state · task output"] --> decide
    raw --> compact
    nulls["Mature null stack<br/>inverse methods · uncertainty · phase retrieval · compressed sensing<br/>computational imaging · AO/control · digital accelerators · passive optics"] --> compare{"Matched information + hardware + lifecycle budget"}
    decide --> firewall["Separate outcome firewall<br/>aperture · photons · phase · priors · drift · saturation · fusion<br/>transform · conversion · analog error · fabrication · safety · lifecycle"]
    ledger["Complete ledger<br/>samples · photons · dose · bytes · seconds · person-hours · joules"] --> compare
    fabrication["Device population<br/>yield · mismatch · trimming · thermal control · aging"] --> route
    firewall --> compare
    compare --> verdict["Retain measured residual<br/>or retire the composition"]

Editable source: operator-qualified-physical-inference.mmd.

Efficiency mechanism

The contract permits four efficiency gains, each with a matching way to fail:

1. Acquire selectively. Spend photons, time, and motion only where another observation changes an accepted decision. It fails when the acquisition controller costs more than fixed sensing or shifts risk outside the ledger.
2. Transform before expansion. Use a physical operator to filter, aggregate, or project local optical information before high-volume digital movement. It fails when conversion, source, control, or recalibration erases the saving.
3. Retain the sufficient level. Store raw evidence, calibrated sufficient state, or task output according to registered future queries and recovery obligations. It fails when later queries expose discarded information.
4. Route by validity and reuse. Amortize a stable transform on hardware that suits its precision and repetition. It fails under workload shift, fabrication spread, thermal control, low utilization, or short lifetime.

For each accepted service unit,

where every $E$ term is energy [J] measured over the same accepted-output boundary. Embodied energy includes fabrication, packaging, yield loss, replacement, and end-of-life treatment amortized over accepted lifetime service. The comparison also reports quality, calibration, latency, risk, photons, bytes moved, and human maintenance effort; joules alone cannot hide a worse sensing contract.

Evidence status

The sources support the constraints and component mechanisms. They do not yet show that their full composition improves this project's quality–risk–latency– energy frontier. F-007 is therefore a hostile fixture, not an architecture promotion.

Speculative extensions

Learned operator compaction

Learn the smallest operator state that preserves a registered family of likelihoods, counterfactual acquisitions, and calibration decisions. Compare it with explicit metadata, sufficient-statistic storage, low-rank calibration, and recomputation from raw evidence. A compact state that cannot answer a new registered query is rejected.

Joint query, sensor, and substrate routing

Let one controller decide whether to answer from retained state, acquire a new physical observation, or move the transform to another substrate. The claim is interesting only if joint control beats three separately optimized controllers after coordination and monitoring cost.

Population-calibrated physical modules

Treat fabrication variation as measured device identity rather than nominal noise. Assign workloads by the calibrated envelope of each device, then test whether characterization, placement, spares, and migration work less than trimming every device to one specification.

Future-query-aware sensing

Choose acquisitions that serve both the immediate decision and declared future queries. This could favor a slightly more expensive measurement now if it prevents reacquisition or unsafe inference later. The future-query distribution must be registered before results are inspected.

Failure modes

Measurable predictions

1. Null-space honesty. On paired physical states that share an observation under the tested operator, an operator-aware system will retain ambiguity or abstain more accurately than a tensor-only decoder without reducing identifiable-task performance.
2. Prior-shift qualification. Under held-out scene structure, operator-aware uncertainty will predict super-resolution and compressed-recovery failure better than confidence from the reconstruction model alone.
3. Costed active acquisition. At equal photons, dose, latency, action count, risk, and joules, active selection will improve accepted task utility beyond fixed acquisition and one-step EVI—or the learned acquisition mechanism is retired.
4. Noise-regime crossover. Multiplexed and focused acquisition will exchange rank at a reproducible noise boundary predicted before the confirmatory run.
5. Drift-aware recovery. Versioned monitoring will reduce invalid confident outputs and recovery time under hidden alignment, gain, timing, and thermal changes beyond periodic calibration at equal reference and maintenance cost.
6. Heterogeneous crossover. A physical path will improve end-to-end accepted outputs per joule only in preregistered regions of transform reuse, precision, input locality, utilization, and device validity; digital routing will win outside them.
7. Device-population validity. Routing by measured device envelope will improve yield-adjusted lifetime service beyond nominal routing and uniform trimming after characterization, migration, spare, and control costs.
8. Query-preserving compaction. A query-registered retained state will use fewer stored bytes and lifecycle joules than raw retention while meeting every registered reconstruction, uncertainty, provenance, and recalibration tolerance.

All predictions are evaluated through F-007. A positive result remains bounded to its measurement operator, physical regime, query set, hardware population, workload, and lifecycle boundary.