Threat Hunting with Wire Data: User Agent Hunting

The threat hunting discipline continues to evolve in the security industry as organizations seek to become more proactive at finding threats to their infrastructure and engaging in what his fast becoming a “hand-to-hand” combat situation in IT security. At the time of this article there where over 7100 job postings on linked in with the terms “Threat Hunting” or “Threat Intelligence”. The deluge of data that we have undertaken in the last ten years has resulted in significant noise and difficulty finding context within the myriad of disparate data sources that exist in an enterprise. In this post, I want to discuss how we can “cut out the middle-man” in our “quest for context” gaining much needed agility, speed and making our investments in mash-up technologies such as SIEMs, big data lakes and innovations like Sqrrl more efficient and less work.

The case for wire data:
I tend to over use the title above but I think there is a significant case for leveraging wire data to gain context. In the case of this specific hunt, I am going to search specifically for non-standard user agents that I note on my network. UA hunting is relatively common and while most hackers worth their salt will change the user agent name to hide there is still a large number of solutions that do not, specifically IoT devices, of which millions will be purchased in the next few weeks that could be arbitrarily connected to your networks. The case for using wire data here is that, YES, you can log user agent in your web logs and parse them out, write them to SIEM or database then query them to find those user agents that you consider to be actionable. This is the “Middle Man” I am trying to eliminate. You may need to collect several terabytes of logs that need to be indexed (not always free) and stored (not always fast). Leveraging the ExtraHop wire data analytics platform, we are parsing non-standard User-Agents directly off the wire (no “middle-man”) and bring context to the surface within milliseconds. This data can now be send to your back end SIEM, data lake or Sqrrl instance delivering a much higher (term I like to use) “intelligence yield” as, in addition to populating dashboards, integrating with orchestration API’s and creating alerts and emails, you line your threat intelligence coffers with BETTER DATA! Better SIEM, better data-lake, better back end systems.

Below is a dashboard created specifically for the UA Hunt inspired by the threat hunting project that includes the following metrics:

  • Total number of unique user-agents per host (If your server has 7 unique user agents…it might have some malware or unauthorized software on it)
  • Unique IPs by non-standard User Agent.
  • Python and PowerShell user-agents (POSH is rapidly becoming the weapon of choice for a number of bad actors, if they use it to phone home….BUSTED!)
  • Geo-coded INGRESS and EGRESS (Example: if you don’t do business with Belarus, maybe you should look into the POSH user agent connected to it…)

Click Image

Drilling Down:
You can also drill down into the specific conversations that were observed to get an idea of the client/server involved in the conversation as well as the request/response bytes (how big a file was sent) and URI.

Click Image

Clicking the bull’s eye on the left will take you to the packets that can then be downloaded and analyzed as well.

Click Image


The “Data problem” is a big one, solving it requires re-thinking how/where we do analytics. Back end SIEMs and data lakes are in desperate need of better data while not everything will send Syslog messages or Netflow data the common thread in all of the devices we are trying to secure is wire data. If it has an IP address, we can monitor it and log it. Threading wire data analytics into your security strategy will add significant agility, shutter speed and a higher intelligence yield making your entire security practice more effective. This is just one example of threat hunting with Wire Data and ExtraHop. Future posts will include the following hunts:

  • Detecting lateral movement via Explicit Credentials
  • Beacon Detection
  • Dynamic DNS
  • URI Analysis
  • Command and Control Detection
  • External RDP
  • Webshells
  • Rogue Listeners

Threat hunting with wire data brings an entirely new data source to threat hunters and practitioners. Leveraging ExtraHop’s open solution that includes integration with REST API’s will make solving the “data problem” much easier and save you the time, cost and effort of parsing through terabytes and even petabytes of data to gain context.

Thanks for reading!

John Smith
Security Engineer, ExtraHop Networks.

Leave a Reply