Security Analytics Use Case

Sqrrl Enterprise enables Big Data Security Analytics.

Sqrrl Enterprise is the world’s most secure, scalable, and flexible NoSQL database for real-time Big Data applications and is powered by Apache Accumulo and Hadoop. One type of application that customers are building or integrating with Sqrrl Enterprise is Big Data Security Analytics applications. Big Data Security Analytics have the potential to greatly improve an organization’s ability to detect anomalous activity within their networks.


Organizations are utilizing Sqrrl Enterprise to securely integrate vast amounts of multi-structured data (e.g., tens of petabytes) onto a single Big Data platform and then are building real-time applications using this data and Sqrrl Enterprise’s analytical interfaces. The secure integration is enabled by Accumulo’s innovative cell-level security capabilities and Sqrrl Enterprise’s security extensions, such as encryption. The real-time analytical applications take advantage of Sqrrl Enterprise’s JSON, full-text search, SQL, statistics, and graph search capabilities. In this general sense, customers are utilizing Sqrrl Enterprise as a massively scalable, secure, and flexible NoSQL database for Big Data.

Big Data Security Analytics Overview

Some of Sqrrl’s customers in sectors such as financial services, telecommunications, and government are applying this general “secure data lake” use case to cybersecurity. In response to the continued rise of advanced threats and the need for greater visibility across networks, a new concept is starting to take hold in the cybersecurity industry around Big Data Security Analytics. Sqrrl Enterprise is uniquely positioned to help organizations build a Big Data Security Analytics capability.

Figure 1. Big Data Security Analytics Architecture Figure 1. Big Data Security Analytics ArchitectureBig Data Security Analytics refers to collecting and analyzing massive cybersecurity-related datasets in real-time to uncover hidden malicious patterns in the data. Big Data Security Analytics differ from traditional Security Information and Event Management (SIEM) tools in several different ways:

  • Volume: Most SIEM tools struggle to scale past tens of terabytes; Sqrrl Enterprise easily scales to tens of petabytes;
  • Variety: Most SIEM tools are limited to log and event data; Sqrrl Enterprise can easily ingest any data source, including emails, web data, host data, IDS/IDP/firewall information, identity context data, social activity, external threat intelligence, etc.;
  • Value: Most SIEM tools are expensive in both software and hardware costs; Sqrrl Enterprise has a significantly lower price point, because it is largely based on free, open source software and runs on low-cost commodity hardware.
  • Velocity: Most SIEM tools require time-consuming data modeling before the data is ready for analysis; Sqrrl Enterprise’s flexible schemas allow users to perform analytics on a variety of sources with minimal up-front modeling.
    Given these difference, the key benefits of using Sqrrl Enterprise for building a Big Data Security Analytics capability are:
  • Complete Visibility: Analyze across security and operational data of varying types
  • Massive Scalability: 10s of petabytes; access to both active and historical content
  • High Performance: Analyze large datasets in seconds, not hours
  • Standardized Interfaces: JSON, full-text search, SQL, statistics, and graph search
  • Data Security and Privacy: Cell-level security and encryption; access to only authorized & needed data

Sqrrl Enterprise is designed to complement instead of replace existing SIEM tools. Users will typically ingest a variety of datasets (e.g., log files, event files, Netflow, identity context information, vulnerability information, configuration management, external threat intelligence, etc.) into Sqrrl Enterprise and interrogate the data in a variety of ways to discover new suspicious patterns of behavior. Organizations may build new lightweight analytical applications to search for these patterns or integrate existing apps with Sqrrl Enterprise. Once these patterns are discovered, security analysts will train their SIEM tools to look for these patterns in real-time.

Big Data Security Analytics Examples

A Big Data Security Analytics capability enabled by Sqrrl Enterprise can help security organizations perform deeper and more thorough analysis across a variety of cybersecurity scenarios. Some of these scenarios could include the following:

  • A network Intrusion Detection System fires on malware Command and Control traffic; research the root cause on a specific system
  • A spear phishing attack is detected on a system; find other targeted systems
  • A correlation rule fires in a SIEM; full contextual awareness is required
  • The cybersecurity industry releases a new indicator of compromise; assess impacted systems
  • An employee is expected of an insider attack; track all activity of that employee

The Need For Big Data Security Analytics Is Rapidly Growing

“According to ESG Research, 44% of enterprises say that security data collection and analysis would be considered big data within their organizations today, while another 44% believe that they will likely consider security data collection and analysis big data within the next 24 months.”

Source: Jon Olstik, Enterprise Strategy Group, 3/13 Market Landscape Report: Evolution of Big Data Security Analytics

Better Security Models are Needed for Big Data in Healthcare and Life Sciences

“Traditional Security Information and Event Management (SIEM) systems suffer from several limitations. Security analytics were supposed to be anchored by SIEM) systems, a staple technology at most large enterprises. Unfortunately, many SIEM platforms can no longer keep up with mushrooming requirements due to technology, scalability, or usability flaws.”

Source: Jon Olstik, Enterprise Strategy Group, March 2013 Market Landscape Report: The Evolution of Big Data Security Analytics Technology