Legacy SIEMs often fall short in meeting the demands of modern security landscapes, with cost and complexity being particularly painful. It's time to explore the capabilities that the next generation of SIEM solutions must have to address the limitations of legacy systems and empower security teams to stay ahead of evolving threats.
They Take Modern Log Volumes Seriously
One of the most significant challenges faced by legacy SIEMs is the exponential growth of log volumes. With organizations generating terabytes of data per day, the cost of storing and analyzing this data using legacy SIEMs can easily reach millions of dollars annually. This is simply unsustainable.
The pattern we see too often with legacy SIEMs is that teams are forced to drop most of their logs or retain logs for only a few days, which are not serious solutions to the problems of modern log scale.
Next-gen SIEMs must prioritize cost reduction and search speed optimization. The goal should be to bring the cost down to tens of cents per gigabyte or lower, making it feasible to handle log volumes without breaking the budget. There are two approaches that next-gen SIEMs take to tackle this:
- SQL-based data lake for querying highly-structured data.
- Purpose-built indexing for flexible search on semi-structured data.
There are tradeoffs to each approach, but they both rely on the same fundamental idea: cloud storage is the medium where log data should be stored because it is easy to scale and incurs lower costs than operating expensive, brittle clusters.
They Respect Data Sovereignty with Bring-Your-Own-Data
Data sovereignty is a critical concern for organizations, especially those operating in regulated industries or across multiple jurisdictions. Next-gen SIEMs must adopt a bring-your-own-data approach, allowing teams to keep their data within their own cloud storage rather than requiring it to be shipped to a vendor's infrastructure.
By leveraging the data lake pattern, next-gen SIEMs can perform analysis on security data without compromising its sovereignty. This approach not only ensures compliance with data regulations but also gives users greater control over their security data and avoids vendor lock-in.
They Simplify SIEM Operations
Managing legacy SIEM infrastructure can be a complex and time-consuming task, often requiring dedicated teams to maintain clusters and perform ongoing maintenance. Next-gen SIEMs must simplify operations by eliminating the need for infrastructure maintenance altogether.
Moreover, next-gen SIEMs should make it easy to analyze raw data from various schemas and file types. Teams should not need to undertake an extensive data engineering effort to add a new log source.
By providing intuitive integration interfaces and intelligent data normalization capabilities, next-gen SIEMs can empower security teams to focus on threat detection and response rather than data wrangling.
They are Programmable: X-as-Code and APIs
To keep pace with the ever-evolving threat landscape, next-gen SIEMs must be adaptable. Security teams need the ability to use code to adapt the core components of the SIEM, which makes the platform more flexible, and gives teams the ability to collaborate better through source control. For example, all of these should be programmable and definable as code:
- Detections: To automate common investigations.
- Dashboards: To summarize data and highlight what to investigate next.
- Response Workflows: To automate common responses and reports.
Next-gen SIEMs should also provide robust APIs that facilitate running log searches from internal tools, integrating with SOAR platforms, and enabling the use of data science tools like Jupyter notebooks for advanced analysis.
Programmability also enables organizations to share best practices with the broader community by opening up public Github repositories. By leveraging a code-driven approach, next-gen SIEMs foster better adaptability and collaboration within teams and between organizations.
They Use AI the Right Way: To Summarize and Brainstorm
AI has the potential to revolutionize the way security teams operate, but the current generation of LLMs (large language models) have pitfalls, like hallucinations and inaccurate insights. As a result, next-gen SIEMs should use AI as a co-pilot to enhance productivity rather than relying on it for autonomous decision-making.
Thankfully, LLMs are particularly good at two things that are highly useful to security analysts:
- Summarizing complex data quickly.
- Brainstorming ideas to solve a problem.
AI-based features can do many powerful things, like produce a short single paragraph summary of thousands of alerts, suggest high-priority follow-ups, and brainstorm ideas about what to investigate next to speed up mean-time-to-resolution.
By providing intelligent summaries and idea brainstorming, next-gen SIEMs can empower security analysts to make informed decisions while still relying on human interpretation to mitigate the risk of AI hallucinations.
Embracing the Next-Gen SIEM
For CISOs, it's useful to start recognizing the limitations of legacy SIEMs and explore the possibilities offered by the next generation. The next wave of SIEMs will prioritize cost-efficiency, data sovereignty, operational simplicity, programmability, and AI-powered insights. They will make it much easier for security teams to manage ever-increasing log volumes and stay on top of threats with less work.
