Tail Messages

Summary

This document explains how the platform uses a “tail message” to indicate that a patient is done being queried for a given phase, and how that signal flows across services:

  • Data Acquisition → Normalization → Measure Evaluation
  • For both Initial and Supplemental query/normalization phases

The tail message allows Measure Evaluation to know precisely when to proceed with evaluation for the patient in each phase, avoiding premature or duplicate evaluations.

What is a tail message?

A tail message is an event that marks the end of a patient’s resource stream for a specific reporting context (facility/reportTrackingId) and phase (Initial or Supplemental). It is modeled via the acquisitionComplete boolean carried on acquisition/normalization events:

  • ResourceAcquired.acquisitionComplete: boolean
  • ResourceNormalized.acquisitionComplete: boolean

When acquisitionComplete is true, it signals that, for the identified patient, no more resources for the current phase will arrive for the given report context, and downstream consumers may proceed.

Relevant schemas in this catalog:

  • Events → Resource Acquired → contains acquisitionComplete, patientId, queryType, scheduledReports, reportableEvent
  • Events → Resource Normalized → contains acquisitionComplete, patientId, queryType, scheduledReports, reportableEvent

Note: On ResourceNormalized, acquisitionComplete exists but is not a required property; when present and true, it functions as the tail signal.

Correlation and scope of a tail

A tail applies to the intersection of the following:

  • Patient: patientId
  • Report context: scheduledReports[].reportTrackingId (and associated time window startDate/endDate and frequency)
  • Phase: queryType = Initial or Supplemental

Measure Evaluation correlates resources using this tuple so that multiple patients and/or reports can be processed concurrently without interference.

End-to-end flow (Initial phase)

  1. Data Acquisition produces one ResourceAcquired per resource as it queries the EHR for the initial phase.
  2. After the last initial-phase resource for the patient/report is produced, Data Acquisition emits a tail ResourceAcquired with acquisitionComplete = true (resource payload may be absent).
  3. Normalization consumes ResourceAcquired, transforms resources, and forwards them as ResourceNormalized events. For the tail, Normalization propagates the end-of-stream by publishing a ResourceNormalized with acquisitionComplete = true.
  4. Measure Evaluation consumes ResourceNormalized and buffers per patient/report/phase until it receives the tail (acquisitionComplete = true). It then performs the Initial evaluation for that patient/report window. If reportable, it may emit Measure Report Generated (for informational/tracking purposes) or request supplemental data.

End-to-end flow (Supplemental phase)

  1. Data Acquisition executes the Supplemental plan only for patients deemed relevant (e.g., reportable) by Initial evaluation and produces ResourceAcquired events per resource.
  2. After the last supplemental resource is produced, Data Acquisition emits a tail ResourceAcquired with acquisitionComplete = true.
  3. Normalization normalizes the supplemental resources and emits ResourceNormalized events; for the tail, it publishes a ResourceNormalized with acquisitionComplete = true.
  4. Measure Evaluation buffers supplemental resources until it receives the supplemental tail, then performs the final evaluation for the patient/report. Upon completion it emits Measure Report Generated and per-resource ResourceEvaluated as applicable.

Why a tail message?

  • Ordering and completeness: Resource arrivals can be out-of-order and across partitions. The tail provides a definitive completion signal for the current phase.
  • Determinism: Evaluations only run once the phase’s resource stream is known to be complete for the patient/report tuple.
  • Efficiency: Prevents unnecessary re-evaluations while resources are still in-flight.

Message and field references

Tail message structure examples

Initial tail (acquisition complete) carried on Resource Acquired:

{
  "acquisitionComplete": true,
  "patientId": "12345",
  "queryType": "Initial",
  "scheduledReports": [
    {
      "reportTypes": ["NHSNdQMAcuteCareHospital"],
      "frequency": "Monthly",
      "startDate": "2026-01-01T00:00:00Z",
      "endDate": "2026-01-31T23:59:59Z",
      "reportTrackingId": "7d1f8f0e-0b1a-4db2-8e3b-7a63c4a9e4c1"
    }
  ],
  "reportableEvent": "EOM"
}

Propagated tail on ResourceNormalized:

{
  "acquisitionComplete": true,
  "patientId": "12345",
  "queryType": "Initial",
  "scheduledReports": [
    {
      "reportTypes": ["NHSNdQMAcuteCareHospital"],
      "frequency": "Monthly",
      "startDate": "2026-01-01T00:00:00Z",
      "endDate": "2026-01-31T23:59:59Z",
      "reportTrackingId": "7d1f8f0e-0b1a-4db2-8e3b-7a63c4a9e4c1"
    }
  ],
  "reportableEvent": "EOM"
}

Supplemental tails use queryType: "Supplemental" with the same correlation semantics.

Aggregation logic in Measure Evaluation (conceptual)

  • Group incoming Resource Normalized by (patientId, reportTrackingId, queryType)
  • Buffer resources until a message with acquisitionComplete = true is observed
  • On tail:

Operational considerations and edge cases

  • Idempotency: Tails may be retried; consumers should treat (patientId, reportTrackingId, queryType, acquisitionComplete=true) as idempotent.
  • Ordering: While per-partition ordering is preserved, resources and the tail can be on different partitions. Consumers must not rely on strict ordering across the entire topic—buffer/correlate instead.
  • Missing tails: Monitor for stragglers/timeouts. If a tail is missing beyond an SLA window, alert or re-request acquisition.
  • Duplicates: Deduplicate on a stable key (e.g., resourceId + patientId + reportTrackingId + queryType) before evaluation.
  • Replays: On replay or reprocessing, tails should deterministically retrigger evaluation only if prior outputs are absent or invalidated.

Where this is implemented

These behaviors are derived from Link Cloud’s real-world implementation and the event schemas in this catalog:

  • Resource Acquired and Resource Normalized include acquisitionComplete for tail signaling
  • Measure Evaluation waits for the tail per patient/report/phase, then proceeds with evaluation
  • Initial evaluation may request supplemental acquisition; supplemental evaluation produces final Measure Report Generated

See also:

Relationships

flowchart LR
ntail_messages_4E1D6482["Design: Tail Messages"]