Cyber security seems to have a strong infatuation for cycles. It seems like every discipline in cyber has at least one.
- Threat Intelligence? Check.
- Security Analysis? Check.
- Incident Handling? Check.
- Digital Forensics and Incident Response? Double Check.
But this focus on cycles isn’t without purpose. Cycles, at a high level, show that the disciplines themselves are repeatable. Those cycles also drive, a high level, the processes and procedures for those disciplines. This helps to establish that the discipline is also rigorous.
Threat hunting should be no exception to this pattern. This is because a hunt, on its own, is valuable to organizations. But, the true value of threat hunting comes from hunt teams repeating hunts over time. This repetition provides organizations confidence that the protection provided is consistent and thorough.
Putting a New Spin on an Old(er) Threat Hunting Loop
So to prove an organization’s threat hunting is both consistent and repeatable, and thus more reliable, there should be a cycle. Now, organizations have, in the past, proposed previous threat hunting cycles. They have borne various names including the Threat Hunting Loop and The Threat Hunting Lifecycle. These models have all had various merits, but they all had some shortcomings as well. At Cyborg Security we have tried to address these issues. We have also taken into some considerations specific to structured threat hunting. This has resulted in what we call a threat hunting cycle, or more formally a Threat Hunting Loop for Structured Hunting.
Introducing The Threat Hunting Loop for Structured Hunting
Let’s walk through the individual steps…
Hypothesis
Like every journey begins with a single step, so too does the threat hunting loop begin with a hypothesis. This step is crucial as it forms the core question the hunt seeks to answer. This step shouldn’t take you back to high school science class as it doesn’t need to be a true hypothesis. Instead, it should be a formal statement for investigation. For structured hunting this could look like
Attackers are actively exploiting CVE-2021-26855. This vulnerability affects the organization. The actors are deploying web shells on compromised Exchange servers. The hunt will look for activity characteristic of web shells on a server.
The hypothesis stage though, can be useful for unstructured hunting as well, such as
Attackers have concealed their command and control (C2) traffic in encrypted TLS/SSL. The organization doesn’t have TLS decryption. But, through analysis techniques such as
- volumetric analysis,
- frequency analysis, and
- statistical analysis
We can detect anomalous behavior on the network.
The hypothesis could also be a more general area for investigation, such as
Malware often uses manipulated user agent strings (UAS) to “tag” its covert C2 communications. It does so usually with small changes to the UAS. Parse, output, and identify all UAS across department(s). Compare the observed UAS and identify outliers.
Hunt teams can generate their hypotheses from a variety of sources. Threat intel, known vulnerabilities, and previous incidents are all great sources to draw from. The most important source for threat hunting hypotheses though, are the skills and experience of the hunt team.
Requirements
The next step in the Threat Hunting Loop for Structured Hunting is developing requirements. These requirements are the data needed to prove or disprove the hypothesis. These requirements will be quite obvious in the beginning. For example, to observe user agent strings requires netflow metadata. But as an organization matures in their hunting, their requirements will identify limitations. In this case it could be that:
- Certain ports or protocols are not recorded for netflow metadata,
- Netflow data is only kept for 2 days across the organizations, or
- A netflow sensor is improperly placed outside of a network segment.
Identifying these limitations is one of the bigger hidden ROIs for threat hunting, as it will highlight blind spots
Hunt teams will need to adapt to these limitations, to carry out the hunt. But hunt teams should also record the limitations as a part of the hunt plan. Those limitations are also incorporated into the Feedback phase of the cycle. This ensures ongoing and incremental improvement across an environment.
Plan
Next in The Threat Hunting Loop for Structured Hunting is the Plan phase. For every hunt, hunt teams must develop a formal plan. Hunt teams should write out the Hunt Plan and include all the relevant details, including
- the hypothesis – from the Hypothesis stage,
- the technological and operations requirements – from the Requirements phase,
- support from external teams that the hunters need,
- what actions hunters will carry out during the hunt,
- how hunters will confirm their findings,
- agreed escalation paths for incidents discovered during the hunt,
- points for improvement taken throughout the hunt, and
- appropriate sign-offs for privacy and risk managers.
While there is no formal format for a hunt plan, it should be clear and as simple and straightforward as possible. This is because the hunt plan should become a living document. This is because it will serve as the coordination plan for the current hunt. But it will also serve as a playbook and runbook for future hunts.
Hunt
With the logistical phases out away, it is time to look to the operational phases. The next phase in the Threat Hunting Loop for Structured Hunting is the Hunt phase. This phase encompasses the actual execution of the hunt.
This phase will vary depending upon the hunt. Depending upon the hunt plan, data sources, hunting tactics and techniques, and challenges will all differ. Findings, especially of what works and what didn’t, should also find their way into the hunt plan. This ensures that hunt teams don’t continually re-invent the wheel.
Another important point in the Hunt phase is to have a plan for what to do when hunters uncover malicious activity. If the organization has an Incident Response Plan, it should be followed. It is important that hunters hand off the investigation, and evidence, in a proper manner.
Enrich
Organizations often face challenges with threat hunting. This is because threat hunting can be often be an uncertain discipline. But, this uncertainty can be offset by ensuring organizations see ongoing benefit from threat hunting. One of the best methods of developing that ongoing benefit is in the Enrichment phase. In this phase of the Threat Hunting Loop for Structured Hunting hunters will analyze their findings.
The goal of this phase is to review the findings and develop new detection content. This detection content serves to “fill the gap” that the malware or actor used to get into the environment. This detection content is then transitioned to the SOC. This ensures that, moving forward, the organization remains defended. It also ensures that the SOC will handle future detections, not bogging down hunters
Detection content is not the only output of the Enrich phase. Documentation, including
- triage methodologies,
- analysis techniques, and
- suggested remediations
- false positives
This documentation provides security teams incredible value. It also contributes to a more efficient SOC process, and better organizational defense. This allows organizations to very quickly realize the ROI threat hunting can provide.
Feedback
The final phase in the Threat Hunting Loop for Structured Hunting is the Feedback phase. This phase is often overlooked in less mature hunt teams. But, the feedback phase is crucial for organizations seeking to mature their threat hunting.
An important consideration for the Feedback phase is who will provide feedback. Hunt teams will often discuss feedback amongst themselves, but more input is always valuable. Feedback can (and should be) be sought from the support teams and the consumers of hunt team findings. This ensures all parties can identify the strengths to be preserved and weaknesses to improve.
Closing the Threat Hunting Loop for Structured Hunting
The value that threat hunting provides to an organization can be immense. Many people will point to specific tactical successes by their hunt teams as evidence of that. But, the operational and strategic value threat hunting can provide is the ongoing defense it provides. To realize this benefit, it is critical that threat hunting is both rigorous and repeatable. Establishing and adhering to a formalized cycle can help hunt teams ensure that their hunts remain consistent.
