DATA ENGINEERING
Three SQL Keywords for Finding Missing Data
Using QuestDB to find missing data
Introduction
Whether you are just starting to work with a specific data set or monitoring activities and reports based on existing data sets, one of the first things you need to consider is the quality of the data you’re dealing with. Continuity is one of the most critical factors in gauging the quality of time-series data. Time-series systems usually serve use cases where data needs to be consumed, processed, and acted upon with urgency.
Take the example of a public transport vehicle. For the safety of passengers and the timeliness of the service, vehicles need their various sensors — GPS, proximity sensors, pressure sensors, engine diagnostics sensors, and so on. Continuously using the data from these sensors helps the public transport service guarantee timeliness, safety, and reliability. However, a break in the data coming from these sensors would mean that there’s a problem.
Most data access frameworks, including query languages and importable libraries, allow you to filter and see columns or rows where data is missing. The concept of data continuity and completeness isn’t more relevant anywhere than when you’re talking about time-series data. By definition, time-series…
