Kellect: a Kernel-Based Efficient and Lossless Event Log Collector

As an essential element for log analysis, the system kernel-based event can be effectively employed in the hybrid computing environment integrated with cloud, edge, and endpoint for intelligent threat detection. However, the issues of massiveness, heterogeneity, and semantic redundancy have become the biggest challenges in event-based security analysis. Unfortunately, there is no comprehensive tool to collect and analyze its kernel logs for the widely used OS Windows. This paper proposes a kernel-based event log collector named Kellect, a multi-thread tool built on ETW(events tracing for Windwos). Kellect can provide very compressed but most valuable kernel event data for general-purpose analysis on software anomaly detection. Experimental results in real-world show that Kellect can collect kernel event logs generated from FileIO, Process, Thread, Images, Register, and Network, with efficient and lossless. The total performance is three times higher than that of existing tools. The CPU cost stays only at around 1%, while the memory consumption is less than 50MB. As an important application case, the data collected by Kellect is proved to be utilized to build proper model to detect APT after transformed into provenance graphs with complete semantics. At last, a large experiments for the full techniques from ATT&CK are conducted, and the full relevant log dataset is collected using Kellect. To our best knowledge, it is the first precise and public benchmark sample dataset for kernel event-based APT detection.