Under Pressure. How will your cybersecurity team do?

Under Pressure. How will your cybersecurity team do?


Under Pressure. How will your cybersecurity team do?

(Queue the song “Under Pressure” by the British rock band Queen and singer David Bowie [i])

“Under pressure, you don’t rise to the occasion, you sink to the level of your training”

~ Anonymous Navy SEAL

Under Pressure

Nothing could be truer than the quote above, often attributed to an anonymous Navy SEAL. When things get real, your training kicks in. Training is not just filling your head with stuff, but actually performing it. Try. Fail. Learn. Get it right. Perfect it. And doing it again and again. The better the training, the better the students learn. This truism is the bedrock of high-performing, effective teams everywhere.

Small Teams

Somehow the business world hasn’t taken this to heart yet. As cybersecurity threats have escalated, the business world’s search for an effective solution has evolved. After a period of denial, the great hope was that some AI-powered “black box” would solve all cybersecurity concerns without having to do anything. That didn’t work. Next, let’s outsource to a cyber insurance firm. The only problem is that it’s pricey, and you don’t control the process. The insurance company does, and they aren’t always on the same team as you. So, we’re left with one solution—an in-house or hybrid human-based solution, probably a small group of folks charged with the impossible. Stop any and every attack, 24x7x365 from any source—script kiddie or advanced persistent threat (APT). It’s got to be 100%, every time. There might be some pressure building there.  

The Challenge

Here’s where the challenge comes in. You see, the people on your incident response team, as defined in your IR policy and procedures (if you have one), most likely have never been hands-on with a complex incident (If they had, you probably couldn’t afford to keep them). They may have studied cases, taken classes, read tons of materials, and have an alphabet soup of certifications. But they probably have never executed your Incident Response Plan. They’ve never seen what the adversary’s tactics, techniques, and procedures (TTPs) look like in your technology stack. Do you have sufficient visibility? Is your logging up to snuff? So, how will your team perform in a high-pressure situation? How about with no sleep for 48 hours? Where are the gaps? You need to know. Your company is on the line.

Train Like the Champs

How do you overcome this? You train. And then train some more. This type of training is called Adversary Emulation or Purple Teaming. Regardless, the concept is to step through a targeted attack using real TTPs but without all the dangers of a real attack. Team members are divided into two groups, a Red (Offensive) Team, and a Blue (Defender) Team. PacketWatch team members are on both teams and provide the technical resources to emulate the attack. At each step, Red Team and Blue Team members get together to:

  1. Review the actions that occurred
  2. Analyze the result of those actions
  3. Determine the effectiveness of the current controls
  4. Identify the gaps
  5. Recommend changes
  6. Discuss other lessons learned

Custom Active Security Engagement

The PacketWatch team can fashion engagements tailored to your firm’s specific needs. Whether you need to test tools and visibility, your incident response capabilities, the effectiveness of specific controls around groups of assets, your defenses against a particular targeted threat, or a combination thereof, PacketWatch’s Active Security Team will build an effective engagement for you.

With an Active Security Engagement, you can:

  • Validate your security controls and incident response processes against the tactics of real threat actors representing the most significant risk to your industry vertical.
  • See and experience how real attacker tactics and exploits appear in your security tools. Identify gaps and assess the capabilities and maturity of your team in realistic scenarios.
  • Improve your organization’s readiness for detecting and responding to the next attack. This hands-on exercise is a better experience than just reading a white paper.

Why PacketWatch?

The better the instructor, the better the team learns. PacketWatch is a team of elite experts from a wide range of backgrounds, including the military, government, law enforcement, commercial enterprise, and the intelligence community. We respond to hundreds of complex breaches each year. Knowing and countering adversary tradecraft bolsters our effectiveness in quickly identifying and eliminating threats. We bring that real-world experience to bear for you and your team. That makes us the best for delivering this type of engagement for you. Planning, rehearsing, and testing with a high-performing team is key to ensuring your team’s success.

Ultimately, it’s all about the quality of your team’s training. That determines the outcome. Enable their success with a PacketWatch Active Security engagement.

Give us a call or Contact Us to give your team hands-on experience defending complex attack scenarios.

[i] “Under Pressure” by the British rock band Queen and singer David Bowie was originally released as a single in October 1981.

Why Wait for An Alert?

Why Wait for An Alert?


Why Wait for An Alert?

Is this Threat Hunting?

In a recent scan of marketing literature from other security vendors, nearly every piece I read claimed that they will provide you with “threat hunting” services – one even claiming they did 24x7x365. Really? Better double-check that SOW or service description before signing and ask yourself, “What am I really getting?”

Let’s look at what “threat hunting” actually is and compare. Gartner says this about threat hunting (emphasis added):  


To hunt for security threats means to look for traces of attackers, past and present, in the IT environment. Organizations that employ threat hunting use an analyst-centric process to uncover hidden, advanced threats missed by automated, preventative, and detective controls. The practice is distinct from threat detection, which relies heavily on rules and algorithms.[i]

Automated Threat Detection?

In reality, many of these vendors are selling “threat detection” rather than “threat hunting.” They changed the name of their managed security operations center (SOC) services to use the new marketing buzzword. It’s 24x7x365 because it’s just an automated detection service. Their “analyst” (a Tier 1 SOC guy) waits for an automated alert and then works to adjudicate the alert, likely escalating it to another more senior “analyst” before concluding its relevance and sending it back to you. They only have data from the sources you provided. How’s that any different than the managed SOC services they sold last year? It doesn’t sound like the definition Gartner set forth to me.   

In that same article, Gartner says:


While threat hunting includes the use of various tools and processes, people are at the core. These rare IT security professionals are highly and uniquely skilled, are known as threat hunters, and the best ones have a combination of systems, security, data analysis, and creative thinking skills. [ii]

Hunt Before the Alert

Note Gartner’s focus on highly skilled, creatively thinking humans. Preferably experienced ones that have responded to all types of security incidents. These are real analysts looking for an intruder before any alerts are generated. They want different tools to expand the context of what they see and allow them to conclusively adjudicate a potential threat (not just an alert). They make and test hypotheses based on current threat intelligence. Ideally, you’d want a dedicated analyst that has direct knowledge of your unique IT environment. Not a random pod of folks. These real threat hunters are “rare” it says. They are probably not working in the graveyard shift at a SOC.

Real Managed Threat Hunting

PacketWatch offers a real managed threat hunting service. Our team of elite experts is from a wide range of backgrounds, including the military, government, law enforcement, commercial enterprise, and the intelligence community. They hunt and respond to incidents using the proprietary PacketWatch platform. They are creative thinkers honed with skills from responding to all types of security incidents across the globe. They work one-on-one with you and your team to further your security program. They are equipped to “uncover hidden, advanced threats missed by automated, preventative and detective controls.” They aren’t waiting for an alert to act. That sounds more like what Gartner meant when they defined the term.

Next Steps

So, if you are considering hiring a team for Threat Hunting:

  1. Ask to meet the analyst assigned to your account
  2. Read the Statement of Work (SOW)
  3. Measure them against the Gartner standard
  4. Make a wise decision

Give us a call or Contact Us to meet some of these rare, highly skilled, creatively thinking humans.

[i] Gartner. “How to Hunt for Security Threats
[ii] Ibid.

The Packet Never Lies

The Packet Never Lies


The Packet Never Lies

The Black Box

“Self-learning, autonomous artificial intelligence (AI) security solution.”

That’s the marketing double-speak to sell you a “black box” that supposedly teaches itself about your computer network. Then, it autonomously spots bad stuff happening and takes actions to fix it.

It sees all things, knows all things, and is never wrong. You don’t need to do anything. Plus, it has an awesome graphical user interface (GUI).

Well, not exactly. You see, it may learn bad behaviors and think they’re good. It has to be trained properly. It sounds like something from a movie I once saw.


HAL 9000 from 2001: A Space Odyssey

The Packet Never Lies

Customer Story

In the not-too-distant past, a client called us to help assess their security posture as part of a merger and acquisition (M&A) transaction. The company is part of an international manufacturing organization. They had a large footprint of internet of things (IoT) manufacturing devices and a network interconnected with global supply chain partners. The internal security team was pretty confident. They greeted us with folded arms. After all, they’ve had one of those “black boxes” monitoring their environment for several years. There is no way we would find anything.

We deployed our PacketWatch sensors and began to collect data. Right away, we detected an older “bot” operating on one of the computer controllers for an IoT device. “No way,” they said. It’s simply not possible. And they dismissed our claim. We showed them in our PacketWatch analysis GUI, but they still didn’t believe us. So, we pulled a packet capture (PCAP) of the device’s traffic over the previous 48-hour period and showed the internal team the results.

The packets never lie. Sure enough. There it was.

Wireshark was brought in to definitively prove our claim. We had all the packets, including the payloads showing repeated malicious activity: inbound commands and outbound responses. If we only had net flow data, they probably would have wiggled off the hook claiming some ambiguity in interpreting the data. But we had the whole enchilada—full packet capture history. If we didn’t have historical data, a quick fix would have caused the bot to “disappear” before additional forensics could be run. But there’s no wiggle-room when you have the actual packets. In fairness, we asked the black box for its opinion, but it wasn’t capable of responding.

That meeting prompted an angry call from the client’s Chief Information Security Officer (CISO) to the black box vendor demanding answers. He had staked his reputation on the black box. The vendor went back and reviewed their application history. They couldn’t say exactly how, but the “bot” traffic was detected and somehow whitelisted. What? It had been whitelisted several years prior. So, a detection went without action, and the black box had “learned” it was OK. Oops. How do you un-learn that?

As a result of our findings, the client had to send an embarrassing incident disclosure to their supply-chain partners. The client’s CEO was angry, too. He had blown his budget on all this black box stuff the CISO guaranteed would work. The truth was that an entry-level human analyst with the proper tools would have found the bot easily for roughly the same spend.

Would this be a resume-generating event for the CISO?

Fortunately, not. As we worked together on the balance of the assignment, we continued to show our value.

  1. We were able to troubleshoot an application configuration error by providing them with the session negotiation packets. We showed them exactly where the handshake was failing. None of their other tools could.
  2. We also showed them some misconfigured DNS entries creating daily internal DNS storms.
  3. Our threat hunters showed them several design vulnerabilities (i.e., clear text credentials) that needed attention.

Again, the packets never lie. The CISO was now a hero for bringing us in.

Humans are Necessary

The point here is not to disparage the black box but to convince people that experienced “humans” are necessary to the security process. A black box can automate the detection of threats, but the only sure way to adjudicate a threat is for a human to go back to review the actual packets. If you don’t have the packets, that’s a problem. If your team lacks network visibility at the packet level and/or needs help figuring out exactly what your black box solution has been doing all these years, please call us. We’d love to help you out.
Happy Anniversary, Still WannaCry.

Happy Anniversary, Still WannaCry.


Happy Anniversary, Still WannaCry.

Happy Anniversary, Still WannaCry.

On Friday afternoon, May 12th, 2017, we started to hear about WannaCry ransomware which would ultimately impact over 250,000 computers worldwide. WannaCry, Eternal Blue, Shadow Brokers, and Server Message Block (SMB) exploits seem so long ago. What have we learned in those 5 years? Not enough, apparently.

Although the patch that would protect against WannaCry was issued by Microsoft on March 14th of that same year, it seems many organizations didn’t get around to installing it in time. Exploiting an SMB vulnerability efficiently abused by the NSA for years and then leaked to the public by the Shadow Brokers certainly caught people off guard. Ransomware has only accelerated from there. Some say we may have reached “peak ransomware” last year.

All these years later, organizations are still struggling to patch vulnerabilities before exploits take advantage of them. Or, unfortunately, they patch after the exploit has been utilized and never checked to see. You’re lucky to have one day to patch a critical vulnerability nowadays.  

Over the past 5 years, businesses have purchased more security tools and bought cyber insurance policies in the hopes of mitigating costs associated with their accumulated technical debt. Gone is the thought that some “black box” artificial intelligence (AI) machine can solve all of your security problems without you having to do anything.

Cyber insurers have lost their rears and are pushing back — jacking premiums, cutting coverages, and low-balling recovery efforts. Insurers also started asking more questions in a vain attempt to better underwrite risks. Too late.

Even the SEC has stepped up its efforts to force companies to better disclose their cyber practices and risks to investors. Upstream supply chain partners are asking what you do to mitigate risks. It’s not just “are you doing something” any longer. It’s now “Are you doing the right things?”

So, what are the right things?

  1. You need to practice good cyber hygiene; identify and patch vulnerabilities; have a strong, resilient infrastructure; verified security controls; and well-rehearsed IR plans.
  2. You need an experienced security team with visibility over the network and hosts (not just hosts). Logs are great but come after the fact.
  3. You need battle-hardened humans that actually hunt for badness in your network before it gets you. Not some automated detection tool whose “check-engine light” goes on when the oil is low. You need real people looking for signs that other real people are exploiting weaknesses in your systems.
  4. You need to test your controls and procedures to see how effective they actually are.
With all due respect to your IT team, they just won’t measure up against an adversary funded by a foreign government.

  • What does targeted adversary activity look like in your environment?
  • Has your team ever experienced that?

Businesspeople are sick of the slick marketing hype from security vendors making exaggerated promises. Organizations need real solutions from seriously experienced people.

I hope that by the next anniversary of WannaCry, more people will have implemented the elements described above with internal resources or with a team like ours—hunting every day to identify and close security gaps.

If you need help, we are here and ready.