Log Management as an Enabler for Data Protection and Automated Threat Detection

Log management is an important and foundational aspect of security; however, it is often overlooked. It is common to view security logging and monitoring as one function—that is, logging-and-monitoring. It is easy to lose sight of the fact that security logging is distinctly different from security monitoring, although they are interdependent. Security event logging is a critical component that underpins many capabilities, including security monitoring. Similarly, automation, artificial intelligence (AI)-enabled tools, analytics, incident response, forensics and eDiscovery are all dependent on proper event logging.

The threat landscape is constantly evolving, and as a result, organisations need to be more vigilant. Without a proper log management capability, organisations may not be able to detect and respond to potential threats in a timely and effective manner. Effective data protection and threat detection requires robust log management and monitoring capabilities; however, implementing and maturing these capabilities can be complex and challenging.

The Threat Landscape Is a Big Data Problem

As digital disruption and expansion continue, there are more attacks, and as a result, more signals, logs, and telemetry than ever before, resulting in cybersecurity becoming a big data problem. This is supported by an experiment T-Mobile performed in 2022; T-Mobile placed a honeypot on the Internet to see how often adversaries would target it. It was targeted 65 million times a day.6

Threats have become so advanced that it is increasingly difficult to differentiate between authorized and unauthorized users. People can work from any location at any hour of the day. Traditional baselining is simply not enough. Security teams must correlate, normalize, and sift through big data sets to find anomalies.

The reality is that only a limited volume of data can be handled efficiently using traditional manual approaches, and this is where advanced analytics and AI-enabled solutions can add value. According to the IDC, there will be 41.6 billion Internet of Things (IoT) devices by 2025,7 all generating logs and telemetry. The challenge is no longer looking for a needle in a haystack—it is looking for a needle in a pile of needles.

Conclusion

In 2015, CrowdStrike published an article titled, “The Importance of Logs.” The opening line encapsulates the entire article: “Across all of the nation-state targeted attacks, insider thefts, and criminal enterprises that CrowdStrike has investigated, one thing is clear: logs are extremely important.”17

In any investigation, the first thing incident response or forensic practitioners do is review the logs. In recent years, the importance of event logging has increased significantly due to the sophistication and volume of attacks. In addition, compliance regulations mandate logging and monitoring to support forensic investigations and to identify and respond to threats in real time.

Note: This is an extract from a an ISACA journal. You can read the full article here: Log Management as an Enabler for Data Protection and Automated Threat Detection (isaca.org)