Threat Intelligence and Security

Does your business want to set up or improve its threat intelligence security operations? If you find it more complicated than you thought, you have plenty of company. Many companies wanting to benefit from Threat Intelligence are learning a hard truth: setting up successful TI operations takes a lot more than subscribing to a TI service and running data feeds through a TI platform. So, what’s in the “fine print” of setting up successful TI operations.

In a previous post, many must-have Threat Intelligence resources involve hardware, software and (for some businesses) cloud-based services. But, there are many good-to-have resources, too. Clearly defined business goals and people who can transform analytics results into security methods are just two examples. We made a list of suggested business and IT tasks and resources that go beyond the obvious hardware and software Threat Intelligence requirements. When the list passed 10 items, we knew we should share the information with you.

What is Threat Intelligence?

Threat intelligence is not just any platform output. It is output that relates to your security ops (relevant). Put into the context of your security operation (contextualized) It can be used in specific security functions such as processes, best practices, etc. (operationalized). Is easily understood and used.

Here’s a list of basic Threat Intelligence capabilities:

• Collects data from many of feeds and brings it to a single location.
• Receives alerts in real time.
• Normalizes feed data (remove duplicates, enables user-set rules, etc.)
• Integrates with SIEM, firewall logs, etc.
• Creates reports
• Provide actionable indicators used to identify potential threats
• Identify and contain new attacks automatically
• Analyzes security data automatically
• Integrates TI capabilities with other security tools.

Threat Intelligence Business-Side Planning

The most surprising thing about this assignment was how important business-related thinking and planning was. The idea was to suggest thinking ahead, setting goals and connecting successful Threat Intelligence system functions with the business results your company wants.

Here are planning-related topics. Consider them when it’s time to set up or upgrade your company’s threat intelligence resources and methods.

Value Review

Before you dive into the details of how you are going to use threat intelligence, it pays to have goals and capabilities needed to achieve those goals. High-speed, high-volume data handling performance might please the IT folks, but the value is the key to the hearts of the folks paying the bills. So, start with business value.

What do your CIO and CISO want to achieve in the long term? What do they want to avoid? For example, do they want:

• More efficient operations? If making TI more useful and manageable is the goal, look for Threat Intelligence platforms and services that reduce the time and effort needed to find, gather and analyze routine threat data.
• Faster, more accurate response to attacks? If reducing post-attack downtime and avoiding damage to your company brand is key, consider platforms or services that focus on rapid-response capabilities.

These are just examples. There are bound to be additional, value-related scenarios that are relevant to your security operations. But, answers to these questions provide general directions for your TI efforts. Setting goals to TI capabilities answer the question, “How well are we doing?”

Goal Setting

You’ve identified the security tasks that are likely to deliver maximum value to your Threat Intelligence operations. Next step: setting clearly defined business goals for those operations.

Approaching Threat Intelligence with clear operational goals influences your understanding of how and where intelligence can provide value to TI operations. Use these goals to customize your TI program and solve specific security problems.

Critical success factors (CSFs) and key performance indicators (KPIs) are the only way to identify what and how you want to measure TI ops behavior. Use these goals, (which should use bona fide units of measurement), when you:

• Define high-value security business goals.
• Identify TI operations process or task for each high-value business goal.
• Identify potential security operations problems for each process or task,

IT-Side Resource Planning

Identifying the human and IT resources you’ll need for successful Threat Intelligence operations is key. Here are some IT-related considerations for Threat Intelligence program designers.

People & Threat Intelligence

Successful TI is not a machine-only proposition. You’ll need a human as well as IT assets to solve your security ops problems and meet your business goals. Consider the need for these members of your TI team:

• Security ops specialist – Translates results run through data analytics (and possibly a SIEM system) into relevant security processes and tasks.
• Security data analytics specialist – This individual knows how to set up data queries in high-volume data analytics programs and interpret pattern matching and machine learning results.

And next, some cures for some expensive data issues.

Getting to Know Your Data Sources

Organizations that engage in the plug-and-play school of Threat Intelligence quickly become overwhelmed by the sheer quantity of low-yield alerts these sources provide. That’s because the threat “intelligence” tends to be raw data, not information, let alone intelligence.

Relying purely on open-source data feeds sets your TI program up for “alert fatigue” and possibly for a lost opportunity. There are many different sources of threat data, each with its own advantages and drawbacks. Best results (most relevant data hits and fewest false positives) occur when you automatically combine multiple data sources to confirm and customize threat information before you hand it off to a human analyst.

A Tale of Two Data Streams

Transform Threat Intelligence Data to Intelligence – This is a comprehensive list of steps that convert raw threat data to intelligence. Not all TI solutions have all these capabilities, but more and more solutions use them.

• Combine many formats of data coming from many sources.
• Send threat data to a single, centralized location (portal).
• Use natural language processing to capture unusual speech patterns in data.
• Use advanced data analytics to automatically perform pattern recognition tasks and convert raw data into an easily understood format.

Integrate Threat Intelligence and SIEM solutions – Analytics processes eliminate many false positives. SIEM provides the context necessary for a human analyst to triage security events up to 10 times more quickly than with manual methods.

• Scenario 1: your TI platform gathers huge amounts of raw TI data from many sources without your customizing the data feeds. Result: the data flow overwhelms human analysts with false positives.
• Scenario 2: Data feeds are customized to reflect the unique security operations of your business. Result: The value of your threat information soars, and the number of false positives plummets.

The remarkable difference in these scenarios relies on data processing software and methods and on your choice of data sources.

Don’t Analyze Irrelevant Data

Answering these questions will help you avoid wasting time and resources analyzing irrelevant data:

• Think back to your business goals and what you want to occur or avoid.
  What mix of open-source and premium data feeds do you need to analyze to meet your goals?
• How many and which types of data feeds does your system use?
• Where do your data feed come from?
• How much of the data they provide applies to a company in your industry or sector?
• How will your TI provider deliver data?
  If you rely on hosted TI, your provider will offer it as a premium threat feed, a pre-packaged software product, or as a single-customer report.

Check to see whether your provider uses a mix of human and automated security operations, and harvests intelligence from open and closed data sources. Is this the mix that will help you achieve your goals?

Organizing Threat Intelligence Data and Security Ops

Before you can correlate, enrich and customize Threat Intelligence data, you must organize it. Here are a few ways to organize the data and its users.

• Threat Intelligence Data Hub: Centralizing TI data removes barriers between silos of internal and external data sources. And, it makes it easier to find, clean up the data and transforming everything into a single data model.
• Common workspace: Having a centralized place for executives, TI, risk and security ops specialists to create and share their work is a proven way to operate more efficiently.
• Threat Intelligence playbooks: Guide your intel processes by creating or assessing intelligence methods based on internal and external sources.

Reviewing Threat Intelligence Platforms and Services

When you assess TI platform capabilities, consider this:

The goal of Threat Intelligence platforms is a balance. Receive and analyze enough data to improve the likelihood of receiving alerts your organization needs to remain secure without being overwhelmed by irrelevant data and false positives. When you sign up for a TI platform or service, what do you get? Capabilities. That’s the nitty-gritty. You pay for the ability to do specific security-related tasks with specific types of data and tools.

Kicking TI Capabilities UP a Notch

These more advanced capabilities are also included in TI platforms and services:

Big data analytics – This enables organizations to collect and customize a very large quantity of potentially valuable data and separate useful and irrelevant data automatically.

Data Integration & SIEM solutions -The analytics results are handed off to the SIEM, which now processes far fewer false positives. This enables human analysts to look at and respond to far fewer potential threats more quickly.

Machine Learning & Threat Patterns – TI solutions produce genuine threat intelligence by using machine learning capabilities to discover, combine and contrast threat data and information from a broad range of external and internal sources. The key thoughts here are “faster” and “more relevant.” The very best threat intelligence solutions can compare your TI alerts with other data sources, internal telemetry, and a detailed understanding of your organization’s IT infrastructure.

Machine Learning Capabilities

Although most of the AI-related claims that TI vendors make is futuristic hoo-hah; machine learning has a modest place in security operations. We mention machine learning here because security threat response time is a big concern among CSOs and security professionals. The longer something goes undetected, the greater the risk of lost revenue, employee productivity and damage to a company’s brand.

Advanced TI systems include data analytics in machine learning scenarios. Analytics solutions scan a wide range of external intelligence sources to identify security system baselines, outliers, and anomalies.  Analysts who use analytics in TI systems process data (firewall logs, for example) 10 times faster than the manual inspection of alerts.

Drowning in a Tidal Wave of Alerts 73% of enterprises surveyed say they ignore security events because they’re overcome by a deluge of alerts. Enterprise Strategy Group report (201x)

Big Data Analytics & Threat Intel

A deluge of data is drowning security teams, who must sift, separate, and correlate the real threats from the false positives and irrelevant information. Around 30% of survey respondents say they analyze about 11 different threat intelligence feeds. This overwhelming surge of information isn’t much help if security pros can’t prioritize and use information intelligently.

Algorithms used in large-scale business intel data processing are familiar tools. All analysts have to do is use frameworks such as Apache Hadoop and inexpensive, industry-standard servers and storage hardware. A high-speed, high-volume analytics solution is born. TI vendors now add big data analytics capabilities to TI systems to filter indicators of compromise (IOCs) and other threat information for security event and information management (SIEM) systems, which weren’t built to process millions of IOCs.