Data Logging and Storage
The data logging subsystem in Vayu provides persistent storage of system data for debugging, analysis, and future extensions such as flight data recording. It is built on top of the virtual file system (VFS) layer and is designed to ensure reliable operation without impacting real-time performance.
File-Based Logging Architecture
Logging is organized into multiple categories, each mapped to a dedicated file on storage. The current implementation supports separate logging streams for:
System-level events
Communication (Navlink) data
General-purpose logging
Each logging stream operates independently, allowing selective recording of different types of data without interference.
Preallocation and Deterministic Writes
To ensure predictable performance, log files are preallocated to fixed sizes during system initialization. This avoids dynamic file growth and reduces fragmentation, which is critical for maintaining consistent write latency in embedded storage systems.
Before logging begins, each file is expanded to its configured size and initialized, ensuring that all subsequent writes occur within a known memory region.
Circular Logging Mechanism
Logging is implemented using a circular buffer approach at the file level. Each log file maintains a write pointer that advances with each write operation. When the end of the file is reached, the write pointer wraps around to the beginning, overwriting older data.
This approach ensures continuous logging without requiring explicit file management or risking storage exhaustion.
Thread-Safe Operation
To support concurrent access from multiple tasks, each logging stream is protected using mutexes. This guarantees safe and consistent writes even when multiple subsystems attempt to log data simultaneously.
Write operations follow a structured sequence:
Acquire mutex
Seek to current write position
Write data to storage
Synchronize file to ensure persistence
Update write pointer
Release mutex
This design ensures data integrity while maintaining simplicity.
Integration with System Components
The logging subsystem is integrated with other parts of Vayu, including communication and telemetry. Log data can be generated by various modules and stored persistently, while selected logs may also be transmitted in real time through the telemetry system.
Currently, the logging system is used to store calibration data and system-level information. These stored values can be retrieved across system restarts, enabling persistent configuration.
Future Extensions
The logging infrastructure is designed to support future expansion. Planned extensions include:
Full flight data recording (sensor, control, and state data)
Blackbox-style logging for post-flight analysis
Configurable logging levels and formats
Efficient binary encoding for high-frequency data
With the underlying VFS and SD card support already in place, these features can be integrated without significant changes to the system architecture.
System Behavior
The logging subsystem operates independently of time-critical control tasks. By using preallocation, circular buffering, and mutex protection, it ensures that storage operations remain predictable and do not introduce blocking behavior in high-priority tasks.