Building a High-Performing Hunt Team
Alan Orlikoski has over 15 years of experience in both the private and public sectors of the IT industry. He has designed and implemented defense solutions for government and Fortune 100 companies. He has more recently participated on teams tasked to assess and advise Fortune 100 clients, with a focus on maturing an organization’s ability to more quickly and effectively detect, respond to, and contain targeted attacks.
- A good hunter is a “jack of all trades” and is able to cover alert monitoring, incident response, signature development, and intelligence gathering. They don’t have to be the best in every one of those categories, but they need to know the functions.
- Senior hunters should guide junior analysts to search for specific Tactics, Techniques, and Procedures (TTPs). This expands the talent base of the hunt team and takes pressure off senior analysts.
- Every hunt should begin and end within an 8-12 hour shift. This allows SOCs to gather metrics on how successful their methodologies are.
Cyber adversaries adapt at breakneck speed, and often have the capabilities to easily slip by network defenses. Analysts are finding that they have to go beyond reacting to events, and actively hunt through networks that have been compromised. However, building up a hunt team can present a significant challenge for SOCs which are just starting out. Who should you be hiring? How should you organize your team? What technology should be using? In a recent Threat Hunter Spotlight interview, Square’s Alan Orlikoski (@AlanOrlikoski) gave a breakdown of some of the best methods to build and develop your hunt team to bring it to peak performance.
Threat Hunting Academy’s ongoing Hunter Spotlight series features conversations with top-level threat hunters to discuss a range of topics, from spotting adversary tactics, techniques, and procedures to leading hunt teams. Each interview is loosely based around their “Threat Hunter Profile” which can be found on the Sqrrl blog. The original interview is available here.
Question (Q): What is the breakdown of roles between different analysts?
Alan Orlikoski (AO): I’ll break down SOC functionality into some buckets. So you’ve got alert monitoring, incident response, signature development and intelligence gathering– four types of people. And that represents the majority of your SOC staff. You may not have teams for all of those groups, but there’s someone on your team who’s doing that function.
What I’ve tried to say and promote is that good hunters are capable of doing any of those roles. They may not be the best at it. They may not be your best at any one category. But they’re the person who can go out and do all of them. And then in addition to that, you’re looking for people who are highly creative because we spent 15, 20 years teaching the security industry that we are “responsive only.” So I wouldn’t say they come from any of those buckets individually. [Instead] these people are going to be the people who are typically your go-to when things go south.
Q: How do you make sure that you’re allocating the right resources in terms of personnel to the right tasks?
AO: So the way that I advocate doing that is when you have your hunting team, you’re going to have those people are typically your most over-utilized. I’ll take a mix of people from all of those categories, and anybody who can touch two or three of the four, and give them a chance on the hunting team. But what I like to do is I like to have that senior person sit down with the hunting team and come up with what I call “hunting activities.” These are things where I want to look for “X,” and “X” is directly related to some attacker life-cycle model. You’ve got the kill-chain model. You’ve got the attacker life-cycle model. Some sort of diagram. And so if I look for this thing, it maps to one of those phases on pick your model.
When you do that as a team, so that the people who haven’t had exposure to all of that start learning what’s available. And when I’ve taught teams how to hunt, by day three, everybody has been able to switch their mindsets and started coming up with ways to look for TTPs that they know they can’t alert on. That teaches every person who’s part of that group so much about your network and makes them all better hunters. So that takes the load off of the senior person because they can set out the roadmap that says “go look for this thing, and here’s how I want you to look for it.” And for the junior members, it lets them have a starting point and add back into that hunting methodology. So you now have this cyclical thing. And you can pull people in and out of the hunting team from those different functions, and it makes everybody stronger.
Q: What works best for communicating that value of hunting activities to management/C-Suite level? And what are some of the things that you recommend for a manager or analyst to take note of to keep track of?
AO: We have this idea of hunting activities and hunting methodologies that all map to, say, the kill-chain model. Well, you have this nice pretty picture of a kill-chain model that you can show the senior management that they’ll understand with very few technical terms. So how do we make our hunt show value to them? The methodology that I like to do is to put a constraint on the hunting team: every hunt that they do [and] every hunting methodology that they use has to finish in one shift.
When you do that, the hunt only ends in one of three ways:
- You found something malicious. In that case, it kicks off an incident response.
- You found something bad that wasn’t malicious. We’re talking policy violations. Say, something kind of ridiculous, but somebody putting Neo Legends on a cash register. The company doesn’t want that. So that’s still providing value to the company even though it’s not a security incident.
- The last thing is you can do is “I found nothing.” And the output of that is going and re-validating your methodology to make sure that you’re looking for what — you really are looking for what you’re saying you’re looking for. And you’re not fooling yourself into saying, “Well, it’s not here,” but you’re not looking for the right data
If you only have three outcomes and each hunting methodology lasts one shift, you can now start building metrics across everybody on your hunt team. How many hunts are they doing in a day? Which hunting methodology did they use? From there you get “here is the value we’re providing to our company, here’s how many of these outcomes went into these buckets.” And when you do that, not only can you say “here’s how valuable my team is,” you can also say [for example] “I have nothing that’s going to detect lateral movement”– so you can provide metrics that say “I haven’t done a hunt to detect lateral movement because I don’t have tools or dSet featured imageataset XYZ. If you provide me X number of dollars, I can do this thing and start filling in that section of the kill chain.”
Q: What advice do you have, just for those that coordinate hunt operations, or analysts who have to deal with being coordinated?
AO: The best feedback I can have is from a manager level, don’t let the technology and the analysts drive your conversations with leadership. Leadership wants to know the value, and the technical people want to know how they’re going to do X, Y, and Z with whatever technology is available. So don’t get caught up in trying to explain the tech. Talk to the value, and talk to the return on investment. And from an analyst perspective, try to make everything repeatable and in the constant feedback loop of getting better. And then don’t keep hunts on the book that can be automated. Farm those out and start doing new, better, faster, stronger hunts based on different TTPs.