Security Log Management and It's Importance

A log is a record of the events generated by various network and security devices. Apart from information related to a specific event it also contains records related to computer security that has occurred within a network. These computer security logs are generated by many security sources such as operating system on servers, workstation, and networking equipment; firewalls, antivirus software, and IDS/IPS.

In recent years, the number, volume, and variety of security logs have seen significant increase. This has created the need to correctly generate, transmit, store, analyze, and dispose security log data. In other words there is a need of security log management.  Log management ensures that security records are stored for an appropriate period of time in sufficient detail as these logs are useful when performing auditing and forensic analysis. Routine analysis of logs are useful for following purposes

• to identify security incidents,
• to identify policy violations,
• to identify fraudulent activity,
• to identify operational trends and problems,
• to support internal investigations, and
• to establish baselines

Moreover, it is essential to store and analyze certain logs for compliance purpose. Various regulatory compliance requires organizations to store logs for certain period of time. Some of such compliance with this requirements are as follows

• Federal Information Security Management Act of 2002 (FISMA), 
• Health Insurance Portability and Accountability Act of 1996 (HIPAA), 
• Sarbanes-Oxley Act of 2002 (SOX), 
• Global Data Protection Regulation (GDPR)
• Gramm-Leach-Bliley Act (GLBA), and
• Payment Card Industry Data Security Standard (PCI DSS)

A fundamental problem that occurs with security log management is effectively balancing log management resources with a huge supply of log data. Log generation, collection and storage can be complicated by several factors that includes, 

• a high number of log sources; 
• inconsistent log content, formats, and timestamps among sources;
• increasingly large volumes of log data;
• necessity for detection;
• necessity for alert triage and incident response;
• necessity as context data for another log source;
• compliance requirements to collect and retain this log type;
• compliance requirements to monitor this data source and/or system; and
• ease of integration of the log source.

Lastly, log management also involves protecting the confidentiality, integrity, and availability of logs.