8 Ways to Avoid Threat Intelligence Disappointment
More than 35 years ago, when biotech was young, the trade press bombarded readers with a litany of praise about the newest biological drugs, processes and medical treatments that biotech would bring to pass.
As usual with high-tech product announcements, some of this verbiage was based on hope, not evidence. Long before Gartner established its IT HypeCycle, the biotech hype machine was working overtime. Reporting what different biological technologies could and could not deliver was a big part of an analyst’s job.
VoIP Product Maturity and Performance
Communications and security technologies are going through the same transformation, from gee-whiz, cool technology concepts to mainstream enablers that businesses of all sizes can use.
But business VoIP involves more than a single technology or process. It includes many interdependent enablers that deliver the Internet-based communications we’re learning to depend on. Each has its own development maturity, capabilities and product track record. And, each has a current level of dependability, which can be reviewed and measured.
So, we’ve made a list of currently available communications products, services and processes to see how they perform in enterprise and mid-size business environments. We’ll note where performance is solid, in which environments they work best and where improvement is needed.
Threat Intelligence Platforms and Services
When we get under the hood of threat intelligence products and services, we focus on TI technology maturity, as measured by the variety of TI capabilities, the variety of platform designs and approaches, and track record—performance—of those offerings.
This post and the posts that follow are written with a single idea in mind. What you do or don’t know might save a lot of pain, disappointment, and grief in the corporate bank account. So, sit back and consider these ideas…
1. Not everything called “threat intelligence” really is.
Not everything that comes out of internal systems logs or external security feeds is TI. No, it’s probably raw data.
If a system analyzes and prioritizes the raw data, the product is information, but, it’s still not intelligence. To qualify as intelligence, TI information must be:
Relevant to your unique security situation.
Put into the context of your specific business.
Transformable into security processes.
Easy to understand and put into action.
This raw data-to-intelligence transformation requires a lot of high-volume, high-speed data discovery and handling and a lot of time, effort and other resources. Your goal: finding a high-value TI platform or service that will help you avoid or reduce these resources.
2. Define business goals early on to maximize chance for TI success.
Identifying potential attacks and risks is an important part of security operations. But there will always be more risks than you can find and neutralize. The only way to avoid being overwhelmed is to identify and prioritize your most important security operations and business goals.
For example, are you most concerned about losing revenue and customer trust? You might want to focus on incident response and the tools and methods that optimize it. Your strategy: reduce lost revenue by identifying and prioritizing the TI platform or service capabilities that you want TI to deliver.
Then, back into the product or service approach that’s most likely to deliver it in your business environment.
3. Match TI platform capabilities to your business goals.
Gartner, the arbiter of IT development and markets, describes TI technology in its Adolescent stage. That means (among other things) that there are different ways to design and use TI platforms that are on the market.
Here is a list of those approaches:
Centralized data management. Offer a single portal to analyze data received from internal logs, commercial feed providers and open-source threat data providers such as US-CERT.
TI data and process integration. Use the tools and methods that make threat data and information part of your business security processes. (You might have come across this idea, expressed with the awful term “operationalizing” threat information).
Best-of-breed tools and analytics. Provide the best analytics tools and capabilities for data and risk analysts.
TI software integration. Integrate information analyzed from external data feeds with internal SIEM systems. The up-front analysis, often provided by machine learning solutions, drastically reduces false positives and irrelevant feed information.
TI collaboration and sharing. Advance the security information sharing process within and outside your organization.
Your operations might require more than one approach on this list. Fine, just remember: have your security ops team or specialist identify capabilities that solve high-priority business problems. Then, it’s time to match security tools and approaches to the goals.
4. When marketers make AI or machine learning claims, slow down!
As you might expect from a hot-ticket, high-tech item, TI platforms and services are soaked in marketing claims. And why not, AI is super-sexy, right? Maybe, but that doesn’t mean that every TI offering has real AI capabilities. To avoid a wasted investment in TI you must understand how AI and machine learning work in security tools.
AI Questions for Your Vendor
§ When you say, “AI” what do you mean?
Do you mean (something like) “a solution that monitors threat information behavior, learns, adapts to current conditions and solves security problems?”
Or do you mean pattern matching?
§ What types of machine learning does your system or service use?
§ Decision tree
§ Bayesian analysis
§ Clustering
§ All the above (best option)
So, when in doubt, ask. We have a list of questions at right. What some vendors claim to be AI might be pattern matching.
When you ask your questions, keep these in mind:
No customization, no AI. If the tool or service requires no customization to your SO environment, it’s pattern matching, not AI.
Pattern matching alone doesn’t provide solid value. Your vendor must make constant updates to your tool or service. Of course, this goes on your bill.
Decision tree learning is common in pattern-matching systems. This approach provides value by reducing the time and effort of repetitive, formerly manual tasks your security team performs.
Bayesian analysis observes or monitors status of many variables, based on a set of probabilities of malicious activity.
Faster response, less damage. Bayesian analysis can drastically reduce time to identify and respond to a threat.
More data, better results. The Bayesian model requires a lot of data to work well. Not all businesses can meet this requirement.
Looking for outliers. Clustering (or k-means clustering) plots a graph of expected behavior in a clustered model. It maps what normal behavior looks like and identifies outliers.
The complete lineup. A full-bodied machine learning profile should include:
A decision tree to look for known patterns.
Bayesian analysis to detect problems.
Clustering to describe and monitor security baselines.
5. Know the hurdles to get the most out of TI technology.
The relative immaturity of TI technology puts up serious hurdles to getting good performance and maximum value from TI platforms and services. TI technology developers are working non-stop to jump the barriers. But, some of those barriers involve humans, not technology.
Here is a list of what we know now.
Technology
There are many sources and data formats of threat information. Several formats are used most often, but there are no current standards. This can make it difficult to integrate TI tools into existing IT infrastructures.
Many threat data streams contain information that isn’t relevant to the security operations of specific organizations.
SIEM data requires a lot of filtering. Simply pouring threat data into a SIEM system can create an overabundance of false positives. Best practice now involves using big data analytics for first-pass data discovery and context. Using BDA as a filter enables SIEM tools to work on a drastically reduced number of alerts.
SIEM systems might not support the tools that security data analysts use to evaluate threat data.
Human Users
Matching specific business-centric actions with specific kinds of threat data and information still requires human attention.
There’s so much threat information that it’s difficult to organize it and develop a response quickly enough to have value
It’s difficult and time-consuming for security specialists to choose the tools and practices suited for specific organizations.
6. Make sure that your organization is a good candidate for TI technology.
Yes, TI platforms or services are the sexy beasts of IT right now, but unless you’re careful, you’ll pay too much for capabilities you might not need.
In his article, “Enterprise Scenarios for Threat Intelligence Tools,” Ed Tittel describes the security exposure that does not require full-featured TI protection. I’ve turned the criteria around to provide a profile of companies that might benefit from TI tools and services.
Company Profile 1: These Companies Should Consider TI Tools or Services
This is an external look at a company that would benefit from TI technology.
Has high-profile Internet presence
Is a large, very well-known organization
Has a website that provides interactivity (especially financial) as well as information.
Has physical, financial or information assets that attackers find attractive.
Draws attention to itself for political or social reasons
Anyone or more of these characteristics make an organization an attractive target.
Company Profile 2: Use Cases of Attractive TI Targets
Here’s another profile, which emphasizes different use cases.
In its market overview report, Gartner mentions that a TI platform would be most effective in solving security problems in:
Threat detection and prevention, anti-phishing, incident response, and fraud and threat analytics cases.
Larger, enterprise security teams.
Industry-wide, intelligence-sharing initiatives.
Teams of support service providers, such as managed security service vendors.
In-house, intelligence-driven security initiatives.
Situations that emphasize changing TI raw data into useful, effective security processes.
7. Make sure that your expectations are realistic.
Caution! TI Isn’t Simple!
§ In recent research from an SC Media survey:
43 percent of respondents expect TI to offer an “early warning of new threats and tactics” immediately, with minimal effort.
§ Thorough TI analysis also requires additional data format standardization, correlation, and contextualization.
The relative immaturity of TI technology presents several problems. One problem is that users often misunderstand TI capabilities. This causes sky-high expectations, which are difficult to meet.
Some of the hype surrounding threat intelligence might make it seem that you all you need to do is flip a switch, and you’ve “done threat intelligence.”
Survey respondents believed that TI involved just aggregating indicators of compromise (IoC) feeds, sending them to a SIEM and matching external and internal data.
Not so fast! There’s a lot more to TI analysis than “set it and forget it.” You can automate many or all the routine manual TI tasks. But humans will always be needed to analyze and prioritize information that machine learning serves up.
8. Recognize the gap between ideal and real TI capabilities.
TI platforms and services have evolved to protect the growing amount of data generated by a variety of internal and external sources, such as system logs and threat intelligence feeds.
A Marvelous Ideal…
Ideally, customers should expect their TI platform or service to:
Import structured and unstructured raw data from multiple sources.
Automatically analyze, correlate, and pivot on TI raw data.
Enrich and add context to feed data.
Automatically analyze data for threat indicators and relationships.
Integrate analyzed data with TI tools and methods. Full-featured TIPs distribute and integrate clean data with other network tools including SIEMs, internal ticketing systems, firewalls, intrusion detection systems.
Use visualization software to make complex relationships easier to understand.
Enable quick, accurate responses to breach identification or predictions.
That means in terms of business value, TI platforms and services should be able to:
Reduce security operations costs.
Reducing effort and cost of IT security pros finding, collecting and prioritizing data.
Reduce the analyst time, effort and cost of analyzing threat data.
Reduce the risk and effort of your SO processes.
Streamline the process of identifying the most relevant IoCs.
Make TI data handling part of routine security operations processes.
Reduce attack response time and resources.
Provide organization- and process-specific IoCs about new attacks.
Detect the indicators of known, sophisticated attacks.
Reduce the risk and costs of lost productivity, revenue, and brand reputation.
Help security teams find and neutralize attacks that cause the most damage to an organization.
…Is Brought Down to Earth
The reality is a bit different. In industry surveys taken in 2016, TI platform customers mentioned these problems:
TI programs require too much time and effort to set up and use. This is the opposite of the set it up and walk away approach that many users expect.
Data overload. 69% of respondents indicated that there’s way too much TI data to analyze, understand and convert to useful, customized security processes.
Integrating TI platforms with other security technologies and tools. Integrating new tools in an existing security infrastructure is often inefficient, partly because the tools have no standard data formats.
Data analysts and IT security pros don’t always see eye to eye. In theory, data analysts are on board to interpret TI data so that it can improve security ops. But survey respondents mention that analysts and IT security pros have different methods and priorities.
Not enough data. Yes, security ops involve tsunamis of data. But, 71% of organizations fail to keep more than three months of historical event logs online. This is not nearly enough historical information to operate high-volume data handling methods of TI.
Conclusion
So far, TI operations can save the effort and cost of routine TI tasks, but hopes that TI operations will be an end-to-end, automated dream is exactly that—a hope for the future.
What’s an IT manager to do? There are good products and services in the market, but getting the best and most appropriate TI offerings for each business requires planning, thought and resources.
NEXT TIME: We help answer your “OK, now what?” question with a detailed checklist. It’s filled with helpful ideas, essential resources, and planning strategies designed for TI newbies or managers looking for a better way to “do TI.”
