Cyber Threat Hunting - The Hunted Become The Hunters

Written in conjunction with Crucial Academy

Threat Hunting. The term sounds cool and currently is one of those buzzwords making its way round the internet, cyber community and LinkedIn. It is a niche role within cyber security that is seeing a surge in demand but what exactly is it?

That’s what we’re going to dig into in this article.

Don’t worry, it’ll be a whistle-stop tour that you’ll have done within your lunch break/coffee break/exceptionally long toilet break!

Firstly though, let’s get a real handle on what threat hunting is, so it doesn’t get confusing when we reference it later. It’s one of those phrases that people will debate the minutiae of till the cows come home but mainly encompasses the process of proactively and aggressively going after cybersecurity threat actors and eliminating them as early as possible in the Cyber Kill Chain, thus in the process making your network environment safer by eliminating the threat sooner.

Reference: https://www.lockheedmartin.com/en-us/capabilities/cyber/cyber-kill-chain.html

Above is a visual representation of Lockheed Martin's Cyber Kill Chain framework that helps threat hunters "think like an attacker" by breaking up the process of a cyber-attack into actionable steps. Roughly the steps in the Cyber Kill Chain are Reconnaissance, Weaponisation, Delivery, Exploitation, Installation, Command and Control and then finally Actions on Objective (achieving their initial aims).

Threat Hunting is all about working out what your attacker’s tactics, techniques and procedures (TTPs) are and recognising them within your own environment as quickly and effectively as you possibly can. Essentially, threat hunting is the art of;

Knowing your enemy > Recognising their techniques and patterns> Finding them > Eliminating all presence of them from the network environment as early as possible

History and Context

Cyber threat hunting, especially in its current form has roots both in commercial cybersecurity SOC work and in military intelligence analyst circles and both have significantly different approaches to the art and science of threat hunting. Military analysts look at it as a process of proactively identifying, investigating and eliminating indicators of compromise before they impact important aspects of national infrastructure.

Reference: https://www.eucom.mil/media-library/article/36144/synergy-among-nations

Indicator of Compromise - At the simplest level, an Indicator of Compromise (IOC) is a piece of forensic data that can potentially alert you to malicious activity on a system – a red flag, in essence.

Common IOCs include: Unusual geography in logins (outside of designated holidays), changes in privileged account activities, large amounts of database read in short succession, registry changes, unusual ports being active and odd DNS traffic patterns (usually indicative of beaconing to a command and control server).

However, civilian Security Operation Centre (SOC) analysts view it differently. They see threat hunting as both reacting to IOCs generated by security incident and event management systems (SIEMs) and intrusion detection and prevention systems (IDS/IPS), technology driven, and a highly investigative process based off alerts and systems picking up on potential malicious activity. It can also be a proactive approach rather than reactive in that Hunters can scour networks and endpoints looking for the tell-tale signs of an intrusion that the alerting systems have missed.

How to Threat Hunt

They’re both technically right and both the longer-term goal of proactively going after and looking for IOCs and reactively investigating the feedback given from your perimeter security hardware (your firewalls/IPSs/security assessment software) will end up making your network a safer place in the end.

Now we know what threat hunting is, how do you go about doing it?

Well, you start off with the Preparation stage of the hunt. Usually, this would involve getting hold of previous penetration test results and risk assessment reports of the organisation and then working out what the highest-priority assets are. These are going to be your first prime targets for hunting for threats and IOCs. You then gather the information that you and your hunt team need (e.g. IP addresses, hostnames, usual running processes and their names/aliases etc) to help you establish a baseline of what normal is.

This baselining is important to the overall threat hunting process, as most malicious activity detection efforts boil down to identifying anomalies and outliers from statistically normal activity. If your Admin is logging in twice a day from an international destination when your head office is in Huddersfield, then it’s fairly simple to spot that as an anomalous activity and look further into it as a potential IOC. Unfortunately, it might not be as cut-and-dry as that.

Is the admin: a) outsourced and working out there legitimately? b) on a remote working agreement and thus totally justified in logging in remotely to work? Or has his account indeed been compromised? You just don’t know, if you haven’t worked out what normal is for the network environment. Preparation is really important for a successful hunt.

Lastly, the hunter is going to want to configure and deploy some sensor software to pick up and alert you to any anomalous activity. The sensors use a database of known malicious IOCs, TTPs and behaviours from both vendor feeds and developed bespoke from within the company then alert the hunters when a match is found.

Next, we get to put our deerstalker on and start the Investigation phase of our threat hunt. This step of the process has its own little sub-steps, which we’ll go into.

First you scope out your investigation, deciding what exactly you’re going to investigate about each of your targets identified in your preparation stage. Recently run and currently running processes and program activity would be an excellent place to start. If your web server seems to have an awful lot of PowerShell and cmd.exe running from the web root directory, that’s not normal at all and would warrant further investigation.

The other one of the investigation sub-steps is Gathering and Analysing Information. This is where your sensor software comes into its own, as it can gather the information required in your scope for you and potentially inform the direction of the hunt.

Starting from the assumption that your target environment has already been broken into by an attacker, you look for the traces that an attacker would have left or is leaving by their activity. You want to look for common exploit code, artefacts and implants left behind by a lazy adversary, get hold of memory dumps to look for odd instructions being pushed onto the stack (anomalous activity coming up again) and looking for those deviations from the baseline we established in the first phase.

An important skill and phase of the hunt that can be sprung onto you at any point is re-prioritising the hunt when you are provided with new information (usually by your software) to potentially catch and foil the attacker in the process. For example, you are mid-way through investigating your odd web root directory activity from earlier and working out whether definitelynotareverseshell.exe should be there or not, when your sensor software tells you that a large amount of database reads are occurring right now in really quick succession from your finance department. That’s a classic sign of a high-value target having been compromised in some way and data exfiltration being attempted.

Data Exfiltration is usually the end goal of most attacks and is the process of removing unauthorized data from the victim’s system back to the attackers. Common exfiltrated data includes credit card numbers, account details, company confidential documentation (financials etc.)

Attackers want to use channels to exfiltrate data that arouse as little suspicion as possible and would appear normal on an enterprise network – ICMP and DNS being popular choices by advanced attackers.

So, what do you do? You’re already investigating the potentially weird activity on the web server. Do you ditch that and start immediately working on the database alert? It could be nothing and a waste of valuable investigative time, or you could potentially thwart a cyber-attack in progress and gain valuable intelligence on your attacker. This is what re-prioritising your hunt means in practice.

So, let’s say you decided to chase the database alert and found that it was indeed data exfiltration in progress, with an encrypted channel to an external IP address being found on further investigation.

This leads you onto the next phase of the hunt: Adversary Removal. This is where you take decisive action to thwart the would-be cyber attacker and remove them and traces of their activity from your network and collect intelligence for future hunts. How do we go about doing this? First you would go looking at what exactly the active service being used to steal data was and suspending the thread, hopefully slowing down or stopping that service being used.

What is a thread? - A set of instructions that can be executed independently of other code and usually equate to a specific set of tasks - move this here, put this there, do this action etc.

If that doesn’t work you could try killing the used service or process altogether, but you have to strike a balance and justify whether shutting down a critical service for a while is worth the pay-off of potentially thwarting further activity. That’s just a decision you’ll have to make in the moment based on your company’s processes and your own intuition as a threat hunter.

Your last resort is isolating the affected server from the wider network in an attempt to stop the attacker from pivoting onto other machines and compromising more machines. This obviously takes an important server offline but also cuts the exfiltration channel off to the external IP address we found in our investigation earlier.

The suspicious activity stops, and you pat yourself on the back for a successful cyber-attack detected and stopped. You go grab yourself a coffee and wait for your blood pressure to drop back into healthy territory. Now what?

Arguably the most important even though it is the last stage – Reporting and Intelligence Collection.

Reporting and Summary

Now the job is done and your caffeine addiction back on track, your attention can turn to writing up your findings.

Exactly what form that takes and what software you use to write it is usually down to your individual company’s choice. But for the most part, threat intelligence reports are a description of what exactly tipped you off in the first place (the IOC), the traces you followed to find the attacker, the TTPs the attacker was using and the remediation steps taken or recommended by yourself.

This provides, after a series of reports, a bank of massively useful threat intelligence that can be used to educate and train other analysts and be fed back into your sensor software.

So now you know what threat hunting is, why you should do it and roughly how you’d go about it conducting it.

If it sounds like something you’d like to give a go (maybe even do for a living) then Crucial Academy run a CREST Practitioner Threat Intelligence Analyst course on this very subject at their Brighton Academy. You can learn more and apply for one of their courses at https://www.crucialacademy.co.uk/training-services.

Or if you’re ex-military, they offer free courses for veterans or HM Forces personnel in their resettlement period at https://www.crucialacademy.co.uk/military-programme.