Threat hunting is all about finding the unknown, and in order to find it you first need to be able see what you are hunting. There needs to be something to search through, and that something typically manifests itself as system log data. Without logging in place, threat hunting is an impossible endeavour, and without the right logs, hunting can prove nearly impossible.
Organizations that are establishing or evolving their threat hunting practices need to be deliberate in which logs they collect (and how long they retain them). In order to help maturing programs set themselves up for future threat hunting success, our team recently put together thoughts on the most important log sources for successful future hunts. These sources need to come both from the network level and from endpoints, though today we will be examining the best network log sources to support successful hunts.
DNS
One of the most critical network log sources is DNS. DNS logs will reveal name resolution that occurs in the environment, and depending upon the logging, who made that resolution. This can enable hunters to identify simple name resolution, but it can also serve as a vital source for anomalous domains (think high entropy domains such as those produced by DGAs).
However, there’s also a higher-level value for hunters to examine DNS logs and that is that many attackers use DNS as a covert channel for communication. Attackers sometimes leverage DNS because many organizations often don’t log it due to the volume of log data that DNS activity produces — and the sophisticated adversaries know that. This means that activity like command and control communication could be waiting for threat hunters to discover.
NETFLOW
Netflow data comes from network traffic analyzers like Zeek (formerly Bro), which generates metadata around each session in your network. Metadata can be extremely useful information for piecing together network activity — in fact the leaks some years back from Edward Snowden revealed that the National Security Agency (NSA) views metadata collection of various networks as one of its most useful intelligence tools.
For threat hunters on their own network, metadata can be important for various reasons. For example, HTTP header data can be used to be find command and control activity. And hunters can use additional hints in areas like referrer fields and user agent strings to find symptoms of abuse. Ultimately, metadata helps provide a better fidelity in the picture they’re piecing together about an adversary on the network.
PROXY
Proxy logs are also important because even if you don’t necessarily have DNS logging enabled, some of the same information can be gleaned from a proxy server.
The logging will depend upon the appliance, however it is likely going to give you (at a minimum) the date, time, size, the internal host making the request, and what they requested. One thing to keep in mind is that when threat hunters are trying to dig into the network, log data overlap can be a very valuable resource. It allows threat hunters correlate and collate activity from different sources, while enriching it from specific data from the source.
FIREWALL
Firewall data is some of the most fundamental of network log sources for threat hunting. The data itself is relatively simplistic, but the conclusions that can be drawn from that data can be exceptionally valuable. Firewall data can reveal abnormally large file transfers, volume and frequency of communication by host, and important events such as failed sequential connection attempts.
Firewall data is also very useful as a data source for various unstructured hunting techniques, such as stacking ephemeral ports, or grouping and clustering different communication patterns.
TLS/SSL CERTIFICATE MONITORING
While TLS/SSL certificate monitoring is not a common log source, its value to threat hunting cannot be overstated.
Hunters should consider the source of the certificate:
Was it self-signed?
Was it generated using a free service?
Was the certificate issued from a reputable source?
However, there are other aspects of certificates that can be leveraged for hunting, especially the metadata for the certificates. For instance, hunters could identify certificates created using email addresses from their organization, for your IP addresses outside their netblocks, or for certificates that look just a bit too much like your organization. This type of hunting is also very valuable to discover those pesky “shadow IT” services.
VPC FLOW LOGS FOR CLOUD PROVIDERS
Finally, Virtual Private Cloud (VPC) flow logs continue to grow in importance for threat hunting. When organizations operate cloud environments, threat hunters will need to be able to examine network flows between clouds or between clouds and endpoints. Having that visibility in an era of increasing cloud dependence is very critical.
VPC data will vary depending upon the provider because each does their logging in a different way — they’ll all be slightly different, but usually include similar types of data.
Visibility is key in threat hunting, and network visibility is crucial to enable successful hunting. At the end of the day, if an attacker is in your environment, they still have to be able to communicate in and out of the environment and between different assets on the network. They’ll inevitably leave clues along the way. These sources should offer teams a starting point for places to find them.
Keep reading on the topic: Network Content and You: Why Logs Matter in the Age of TLS/SSL.
