Low confidence can be useful without being visible

An uncertain signal is not automatically useless. It may be too weak to show today, yet valuable tomorrow when a repeated observation turns a possible pattern into a supported one.

Systems lose that continuity when they treat confidence as a single keep-or-delete switch. A better design separates observation from presentation: retain bounded evidence for later comparison, then apply a stricter rule before anything becomes user-facing.

Store observations independently of visibility

This section gives uncertain evidence a durable representation without implying that it has been approved for display. The record captures what was inferred, why, and under which version of the inference logic.

A compact observation contract makes the distinction visible:

export type SignalObservation = {
  id: string;
  subjectId: string;
  kind: string;
  value: string;
  confidence: number;
  evidenceKeys: string[];
  observedAt: string;
  inferenceVersion: string;
};

export type VisibleSignal = {
  observationId: string;
  label: string;
  explanation: string;
};

There is no visible boolean on the observation. Visibility depends on current policy, history, and request context, so storing it as if it were an intrinsic property creates stale decisions. The same observation may remain hidden in one policy version and qualify under another.

The record also avoids storing raw prompts or unrestricted model prose. Long-term continuity usually needs a normalized kind, a bounded value, evidence references, and version information rather than the entire inference transcript.

Evaluate continuity before display

This section turns multiple weak observations into a deterministic presentation decision. Recency, repetition, evidence support, and policy version should all be explicit inputs to that decision.

The policy first removes expired or unsupported observations, then groups comparable signals within a meaningful time window. Only distinct observations count as repetition. The result records whether the signal is visible and why, rather than mutating the underlying history.

This policy should be intentionally plain. Product teams need to explain why something appeared, and an evaluator needs to replay the exact decision. A model may produce each observation, but it does not decide that repetition equals sufficient evidence.

Deduplicate meaning before counting repetition

This section prevents retries and equivalent observations from masquerading as corroboration. Continuity is useful only when the stored history represents distinct evidence.

Build identity from the subject, normalized signal meaning, supporting evidence, and a time bucket appropriate to the domain. A retry with the same inputs should resolve to the same identity, while genuinely new evidence should create a new observation.

A uniqueness constraint or idempotent upsert can enforce that identity. The time bucket should match the signal’s meaning: a daily observation may use a date, while an event-level anomaly may need a finer interval. Choosing that granularity is a semantic decision, not merely a storage optimization.

Make the hidden path observable

This section records why observations did not surface without exposing their contents in logs. Hidden data still affects later decisions, so its lifecycle needs operational visibility.

Count decisions by signal kind, visible or hidden outcome, bounded reason, policy version, and inference version. Useful hidden reasons include insufficient support, insufficient history, expiry, and incompatibility with the current policy.

Avoid subject identifiers, generated summaries, confidence values with high cardinality, or raw evidence in metric dimensions. Aggregated counts are enough to detect a policy that suddenly hides everything or a model version that produces unusually weak observations. Sampled, access-controlled traces can carry deeper diagnostics when an aggregate moves unexpectedly.

Uncertain evidence needs two gates, not one verdict. Preserve a compact, auditable observation for continuity, and make user visibility a separate deterministic decision that can change without rewriting history.