Security Analytics Part 1: The Game is On

NOTE: This is the first part of a three part series exploring the Security Analytics market. 

“The world is full of obvious things which nobody by any chance ever observes.”  – Sherlock Holmes

Deep inside the systems, networks, data and applications is the threat.  It is everywhere and nowhere. Hidden inside the gigabytes of data, traffic, and system calls, lurks all manner of malicious code, just waiting to commit a crime and slip away undetected.  The IT security industry is losing this game. The hackers and criminals are more devious, daring, and destructive than ever. We need a new kind of detective who can find this malicious code. We need a virtual Sherlock Holmes.

If Sherlock Holmes was looking for a career in the modern era it is likely, he would be drawn into the world of hackers, malware, cyberwar and hacktivism as an information security analyst.  Information security probably pays better than Lestrade, and the complexities of detecting and tracking modern attackers is something that would appeal to the intensely analytical mind of the world’s greatest “consulting detective.”

However, most of us lack the deductive powers of Sherlock. We need technology to pick through the mountains of data and packets to figure out where the threats are.  While many security technologies have matured to the point commoditization, security analysis remains a clumsy affair that requires many different tools, most of which do not interoperate.

Security technologies are therefore at a cross roads.  After decades of pouring resources into defensive technologies such as firewalls, intrusion prevention systems (IPS), data loss prevention (DLP), anti-virus (AV), application white listing, mobile security, and security information and event management (SIEM) there is scant evidence that attacks are abating.  In fact, some recent high-profile attacks have used special malware that was designed to avoid detection technologies.  Apparently, Moriarty has a new profession as well.

Hackers and their tools are getting smarter and more dynamic.  New advanced persistent threats (APT) can automatically detect and identify security controls and alter their behavior to avoid detection.  If our security technologies are failing to protect us, then what or who will?

As Holmes might say, “the game is on.”

The security game has a new player aimed at putting all that Sherlock style analysis into a box: Security Analytics (SA).  This is an emerging class of technologies that seeks to bridge the gap between what current security controls do and what people need to know.  Security Analytics aims to sift through all that big data and find the smoking gun of an attack.

In this paper, we will explore this emerging market.  We will define the technology components, players, and how they work together. We will begin this case at the scene of the crime, where Security Analytics was born.

The Failure of SIEM and Anti-Virus

The origins of the Security Analytics market centers around the failure of two widely deployed technologies: Security Information and Event Management (SIEM) and anti-virus (AV).  Both of these technologies are dwindling in value, in part because they are not capable of responding to new threats.

If there is one technology that embodies the difficulty in protecting modern systems, it is AV, or more specifically, the failure of AV.  AV is so entrenched in enterprise IT thinking, that nobody really questions its use.  This has made AV vendors extremely lazy.  McAfee, Trend Micro and Symantec, who dominate the AV market, just coast along on their renewals and infuriating management platforms.  Challengers like Sophos, ESET, and Bitdefender have attempted to enter this ultra-commodity market with moderate success.

Modern malware is slipping past AV with alarming ease.  Moving these engines into unified threat management/next-generation firewall (UTM/NGFW) platforms has helped.  It added diversity to the scanning, but these products are still dependent upon old AV technology.  IT leaders are clamoring for AVs replacement.  The situation took an interesting turn just recently, when a vice-president for Symantec announced “AV is dead” which has been said before, but is gaining momentum.

If AV is dead, SIEM is looking pretty sick.  Originally, SIEM products were supposed to gobble up tons of data and spit out useful analysis.  However, anybody who has spent more than a few hours with a SIEM product, knows that is rarely the case.  SIEM technologies have, for the most part, been an abject failure as a technology.  They are universally loathed and consistently fail to deliver anything close to what their vendors promise.

The problem with SIEM is the immense amount of effort necessary to make it useful.  SIEM technologies provide a framework for analyzing data, but they do not provide the intelligence to analyze that data sensibly.  At the heart of SIEM products is a collection of correlation rules.  These rules look for a sequence of events based upon static rule definitions.  These rules must be constantly maintained and updated to make them useful.  There is no dynamic rule generation based upon current conditions. Therefore, the management of SIEMs rapidly devolves into busy work, keeping all the rules relevant.

The evidence of SIEMs failure is everywhere.  In 2009 there were dozens of upstart SIEM products fighting to be the next big thing.  Today, there are only a few viable players and they see limited development or evolution.  Moreover, the promise of a product that is more than merely log storage was never quite realized.

It would be easy to call Security Analytics the “next-generation SIEM” as some have.  However, that is only partially correct.  While many SA platforms can integrate with SIEM products, they do not share a feature set.

The evidence of this is that most of the existing SIEM vendors are struggling to transition into SA products.  IBM, McAfee, HP and Splunk all field decent SIEM products, none of which provide any competitive SA capabilities.  Only RSA has sucessfully transitioned their SIEM product to SA through merging their widely unloved enVision product with their Netwitness acqusition.

The failure of these two technologies, in tandem with the sucess of UTM/NGFW products, primed the IT security market for a new kind of security technology that would compensate for the deficies of AV and SIEM and close the gap on emerging threats.

Enter Security Analytics

Typically, the discussion of SA begins with the phrase “big data.”  The meaning of this abused expression has become diluted.  What was once meant to convey an actual large quantity of data has become a buzzword for sales people in cheap suits trying to say that “our product is impressive and important.” However, in fairness, SA really does need a lot of data to be useful.  The whole point of SA is to render massive, unwieldy quantities of data into something useful and actionable.

All SA products consume and manage data in some manner.  This can be log data, file data, full network packet captures, and/or behavioral data from systems or networks.  However, merely consuming this data is only half the story.  There must be intelligence in the system that helps render that data into something useful.

As such, Security Analytics is the fusion of big data with threat intelligence.

Elementary Security Analytics

The SA market is comprised to two classes of technologies: data analyzers and behavior analyzers.  To be effective, these technologies leverage threat intelligence.  Therefore, to understand the SA market, we need explore what these three elementary components are and how the fit together.

Threat Intelligence

Threat Intelligence (TI) is the foundation of all SA technologies.  That is to say, all SA products must use some form of threat intelligence to do what they do.  This TI can either be data that feeds analytical engines to make them more effective or a specialized methodology an analytical engine would use to dynamically detect threats.  The ultimate purpose of TI is to quickly and accurately identify threats.

The most common TI is an intelligence data feed.  This feed is comprised of criteria that catalog known bad actors.  The criteria can include many different things, such as known bad IP addresses, file combinations, system behaviors, or any collection of defining characteristics.  The criteria are often used to augment the correlation capabilities of a SIEM. However, TI data could also be used to enhance the abilities of IDS/IPS, anti-virus, UTM/NGFW, or DLP solutions.  The source of their intelligence is either from an install-base of technologies, honeypot data gathering, and/or some kind of crowd-sourced information.

TI can also be packaged and coded directly into a SA product.  For example, Cylance has a unique mathematical process for analyzing system behavior and detecting threats.  In this case, Cylance has put their TI directly into the analysis engine.

There are some stand-alone TI products from companies like ThreatConnect and iSIGHT. However, most TI is integrated directly into SA products.

TI in and of itself is not SA.  Rather, SA products depend on TI to make them useful.  Therefore, TI is a defining characteristic of any SA product.  Moreover, the quality of the TI has a profound impact on how effective the SA product is as a whole.

Data Analyzers

Data Analyzers (DA) are the primary class of SA products.  These technologies sift through data to identify evidence of an attack.  DA products often consume either log data or raw network packet data.  RSA Security Analytics, BlueCoat Solara, and RazorThreat fall into this category of products.

These products analyze big data to find the “needle in the haystack” evidence of an attack.  These technologies are particularly beneficial to incident responders when investigating a potential attack.  They enable incident responders to more quickly and thoroughly review evidence and pinpoint attack details.

DA products are dependent upon threat intelligence to work.  They use the TI to analyze data and pick out events or sequences of events that conform to know attack patterns.  In some ways, DA products are really just a form of automation to the management of a SIEM product.

Behavior Analyzers

If DA products look at data to find attacks, Behavior Analyzers (BA) analyze the behavior of an environment to find attacks.  Behavior-centric products are a diverse and rapidly growing segment of the SA market.  This market includes some well-known anti-APT products, like FireEye; some UTM/NGFW bolt-on components like Palo Alto’s WildFire; and some innovative endpoint products such as Cylance, CounterTack, and CrowdStrike.

Behavior analyzers often use some combination of sandboxing or sophisticated heuristic analysis to identify malware-like activity in an environment.

Stay Tuned for Part 2

Join us tomorrow for next part of this analysis report: Security Analytics 2: An Innovation Explosion. In this report, we will dig deeper into threat intelligence, behavior analyzers and data analyzers and how they all fit together to define the security analytics market.