Signal Quality: What Bad Telemetry Data Looks Like and How to Handle It

Raw telemetry from a commercial vehicle OBD device is not a clean data stream. Before a single byte reaches the Octane processing pipeline, it has traveled through the vehicle's CAN bus, been read by a hardware device that may have intermittent power delivery, transmitted over a cellular network that may have variable signal, and arrived via an API layer that may deliver frames out of order. Each of those stages introduces a class of data quality problem. Getting the detection layer right requires addressing all of them before the anomaly model ever sees a reading.

This is the part of the engineering work that is not visible in a product demo but determines whether the system is usable in production. A detection model that runs on raw, unvalidated telemetry will generate false positives on almost every fleet that has older vehicles, routes through low-connectivity areas, or hardware installed by different technicians with different configurations. Those false positives are not minor inconveniences. Each one is an investigation that costs a fleet manager time and erodes their trust in the alerting system.

Category one: corrupted frames

The most common data quality problem in fuel sensor telemetry is a corrupted frame: a reading where the reported value is physically impossible given the surrounding data. A fuel level reading that jumps from 42 percent to 187 percent between two consecutive frames is clearly corrupted. So is a reading that shows a 60-liter drop in 30 seconds on a stationary vehicle with no pump access nearby.

Less obvious corruptions are more problematic. A sensor with age-related drift may report values that are consistently 8 to 12 percent lower than reality across a vehicle's entire operating day. This is not a single corrupted reading; it is a calibration error that will cause the baseline model to underestimate the vehicle's actual fuel level. The model will then interpret legitimate consumption as anomalous consumption because its reference point is wrong.

Octane handles these in two separate ways. Frame-level corruption (the single bad reading) gets caught by a sliding-window plausibility check that compares each value against what is physically possible given the adjacent readings and the vehicle's consumption characteristics. Values outside the plausible range are marked as corrupted, excluded from the fuel curve, and replaced with an interpolated estimate. The interpolated value is flagged in the data so it is visible if someone later reviews the raw record.

Calibration-level drift is harder to catch automatically because it does not produce a single obviously wrong reading. We handle it through a comparison with engine load and fuel consumption indicators from other OBD PIDs. If the fuel level sensor and the fuel flow rate sensor disagree systematically over a week of operation, that points to a calibration issue on the level sensor. We surface this as a sensor health flag rather than a theft alert, because the distinction matters for what the fleet manager does next.

Category two: coverage gaps

Egypt's road network includes long corridors where cellular coverage is sparse or absent. The Cairo to Luxor highway, sections of the Sinai coast road, and parts of the Western Desert route all have dead zones ranging from a few minutes to several hours. A telemetry system that requires continuous connectivity simply loses data on these routes.

The hardware Octane supports includes local buffering capability. When connectivity drops, the device stores frames locally with accurate timestamps. When connectivity resumes, it uploads the buffered batch. The Octane ingestion layer detects these buffered uploads, identifies the offline period from the timestamp gap, and reconstructs the fuel curve for the missing window using the buffered frames in correct chronological order.

The challenging case is when the fuel level changes substantially during a dead zone. If a vehicle enters a dead zone with 70 liters and emerges with 40 liters, the buffered frames should explain that transition through a series of consumption readings. If they do not because the device was powered off during the gap rather than just out of coverage, the system flags the gap as an unresolvable period and produces an alert with a lower confidence score. The alert format distinguishes between "this happened during continuous monitoring" and "this was inferred from a coverage gap reconstruction." That distinction is important for how the fleet manager responds to the alert.

Category three: delayed and out-of-order frames

Even with continuous connectivity, frames do not always arrive in the order they were transmitted. Network retransmissions, server queue depth variations, and the retry logic on some OBD device firmware can cause a frame timestamped ten minutes ago to arrive after a frame timestamped two minutes ago.

Inserting a late frame at its arrival position rather than its timestamp position creates a false anomaly in the fuel curve. If a frame showing fuel level at 65 percent arrives after a frame showing 55 percent, and you insert it in arrival order, the curve shows an impossible level increase followed by a drop. That is exactly the signature of a rapid undocumented refuel followed by consumption, which is a category the detection model is trained to notice.

The Octane ingestion pipeline maintains a short reorder buffer. Incoming frames are held briefly and inserted at their correct timestamp position rather than arrival position. The buffer window is calibrated to cover typical late-arrival variance without introducing meaningful latency into the real-time alerting path. The tradeoff is that very late frames (arriving more than a few minutes after their timestamp) may arrive after the buffer has closed its window for that time range. These are logged as late arrivals and reconciled in the historical record without affecting real-time alert timing.

Why this layer comes before everything else

The reason we invested heavily in data quality infrastructure before building out the detection model is that the detection model's accuracy is bounded by the input quality. A model trained on clean data and then fed corrupted inputs will generate confident-looking but wrong outputs. Confidence scores do not help if the underlying signal is noise.

When we first tested the detection model against raw, unfiltered telemetry from a mixed fleet in late 2025, the false positive rate was high enough to make the alerting system operationally useless. Most of the false positives traced back to one of the three categories above: corrupted frames driving anomalous readings, coverage gap transitions creating apparent rapid changes, or late frame insertions creating impossible level reversals. Fixing the data quality layer dropped the false positive rate to a level where the alerts had enough signal-to-noise ratio to be worth investigating.

The fleet operators we work with do not interact with this layer directly. They see the alerts, the confidence scores, and the fuel curves. But the quality of what they see depends entirely on the processing that happens before their data reaches the dashboard. Getting the data plumbing right is the prerequisite for everything else.