Perhaps one of the more controversial subjects in the field of cyber security is the topic of threat intelligence (variously referred to as “cyber threat intelligence” (CTI) or more simply “intelligence”) and its function within the security operations cycle. The body of knowledge on threat intelligence is both vast and voluminous; strangely, however, outside of some key concepts which occur frequently in the study of intelligence, generally, much of this body of knowledge is either disconnected or even contradictory. As such, many are unsure of exactly how to perceive threat intelligence, let alone how to integrate it into their security operations. However, before the topic of intelligence can be discussed, it is important to define what intelligence isn’t.
Recently, an article was published which began its discussion on the topic of intelligence by first defining what intelligence is not. Such a beginning can be valuable as it can dispel some of the conceptual clutter which threat intelligence has accumulated over the years. Therefore, it makes sense to begin the definition of threat intelligence through negation:
Another challenging aspect to threat intelligence is that it is often marketed to consumers in a variety of different layouts, many of which have few, if any, overlaps. Generally, commercially available threat intelligence is offered in two primary formats:
An important point to consider with both intelligence reporting and threat feeds is that frequently neither is tailored to an organization. Indeed, they are typically designed to be applicable across multiple clients. As such, most will not be purveyors of actual threat intelligence(which implies analysis and validation within an environment), but rather threat data or information. This is not to discount the value these solutions provide, as many are very valuable, rather it is to make clear that these solutions themselves do not provide intelligence but serve as inputs to an organization’s intelligence program.
With a firm grasp on understanding of what threat intelligence is not, and the distinction of what commercially available intelligence reporting and threat feeds provide, the definition of what threat intelligence is becomes clearer. The Merriam-Webster Dictionary defines intelligence as the “information concerning an enemy or possible enemy or an area,” as well as “the act of understanding.” Both of these definitions are valuable, as they simultaneously capture the inputs of intelligence (the former definition), as well as the goal of intelligence (the latter definition). However, what often remains unclear to external observers is the transitional ground between the two definitions.
The process by which an organization moves from the former definition to the latter one is often referred to as “the intelligence cycle.” The intelligence cycle is a process which has wide adoption within many government agencies responsible for collection and analysis of intelligence, and while the descriptions for the cycle can vary, the core of the process remains unchanged for several decades and includes six key steps, namely: definition and planning, collection, processing, analysis and production, dissemination, and feedback. An important point to mention is that while the intelligence cycle is often laid out as a cyclical process, multiple phases of the intelligence cycle can be concurrent, and multiple iterations of the intelligence cycle can be running depending upon the requirements of an organization and the size of the intelligence team.
Interestingly, one of the often-overlooked stages of the intelligence cycle is the first step, specifically defining intelligence requirements, and designing a collection plan. The process for establishing intelligence requirements (often referred to as “priority intelligence requirements” (PIR)) cannot be done in a vacuum, however, and requires detailed discussions with the intelligence consumers and stakeholders to identify their priorities, often in the form of questions needing answers. The more specific the question, the more specific the answers will be. Examples could include:
These questions are key, as they guide intelligence professionals in what they will collect, how they will collect it, and who or what they will target. Without these questions, intelligence teams may resort to collecting for the sake of collection, a waste of valuable resources and time.
Once intelligence requirements have been clearly defined, a plan is developed with which to meet those intelligence requirements. This plan, often referred to as an intelligence collection plan or ICP, has no set format, but must include the following information:
It should be noted that many organizations, in the development of an ICP, often look to external sources for collection, such as:
These are all valid sources within an ICP, however an area of collection that often is overshadowed or overlooked altogether is internal data sources, especially existing incident reporting. These incidents, on their own, may reveal little about the actor at play, however in aggregate, incident reporting is often far more indicative of existing threats and previous targeting than any external source. After all, unless an actor is trying to sell access to an organization’s environment within the underground, there may be very few, or no, external sources that inform you of a breach.
Intelligence collectors, once tasked, will begin the process of gathering relevant raw (or unanalyzed) data from the sources identified in the ICP. It is important to note that intelligence collectors do not have to be analysts — indeed, they often aren’t within the broader intelligence community. Instead, their job is to gather as much raw data as possible to support the PIRs, making sure that they record not only the events of interest but to capture the additional context surrounding those events. This contextualization is critical when that data begins to be processed (normalized).
Collection can often be a messy affair. Data, as it is collected, is captured in various raw formats: untranslated forum posts, open source reporting, threat data obtained from sharing groups, malware samples, and anything else that could potentially support the defined PIRs. While some organizations have advocated for the storage of such raw data in so-called threat intelligence platforms (TIP), this can often result in significant extraneous data, which can, in turn, lead to false correlations. Instead, storing the data in its original format, often using a search and indexing engine, can be more valuable, as it allows analysts access to the raw data as it was collected.
After collection has occurred — or indeed while it is happening — the next phase is that of processing. During this phase, data and information is “processed,” or normalized, from its raw formats into more usable formats. This might include (but is by no means limited to) translating and transliterating foreign languages, de-obfuscating encoded data, or reverse engineering malware. Again, there is no set format for this data, however the processed data must be traceable to the raw intelligence. This last step is crucial, as the analyst must be able to verify the normalized data, and should any challenges or failures be experienced, validation must be able to occur.
The analysis and production phase of the intelligence cycle is where the processed data and information is provided to analysts in order to answer the fundamental questions: who, what, when, where, how, why, as well as establishing the connective tissue between the collected events and how they answer the questions laid out in the PIR. Each assessment (that is a non-obvious result of the analysis of the processed data and information) made during the analysis is based upon the processed data and information and should use either estimative language (i.e. won’t, unlikely, possible, likely, very likely) or a numerical percentage (0–100%) to convey the certainty of the assessment being true. These assessments also form the basis for one of the most important Key Performance Indicators (KPI) for threat intelligence: the number of assessments made, against the number of which are true, false, or undetermined.
Once the assessments have been made and all the PIRs have been met, the analysts will prepare reporting based on the audience. Reporting destined for line analysts in a security operations center will vary drastically from reporting bound for various C-level executives, as each audience will have extremely different requirements.
The dissemination phase is relatively straightforward: once the audience-specific reporting is completed, the reports are then delivered to the intelligence consumers and stakeholders. While dissemination typically precedes the feedback, it is important to note that another aspect to dissemination, which is often overlooked, needs to occur and that is actioning of the data. This means that the intelligence reporting, if it is to be called successful, will result in direct action — or indeed perhaps direct inaction — by the consumers and stakeholders. When action (or deliberate inaction) does not occur, this should trigger additional review by the intelligence team during the feedback phase.
The final phase in the intelligence cycle is perhaps one of the most critical components: feedback. In this phase, not only is the intelligence team engaged to provide feedback on how the process was conducted, but the intelligence consumers’ and stakeholders’ feedback is also sought to determine if the reporting met the PIR and the results were satisfactory.
While threat intelligence remains a somewhat controversial topic in cyber security, and its role in the security operations cycle is not always well understood, threat intelligence is capable of providing immense value to all levels within an organization, from the C-level executives, to the individual analysts. However, it is critical that organizations consider that when implementing commercially available “threat intelligence” that they consider how it will serve to augment their intelligence program as well as add value to their security operations cycle.
For more on threat intelligence, read Soupy’s blog, The Trouble With Threat Intelligence Today.