Cisco is honored to be a companion of the Black Hat NOC (Community Operations Middle), and this was our seventh 12 months supporting Black Hat Asia. Cisco is the Official Cell System Administration, Malware Evaluation and DNS (Area Identify Service) Supplier.
We work with different official suppliers to deliver the {hardware}, software program and engineers to construct and safe the community, for our joint buyer: Black Hat.
- Arista: Community Gear
- Corelight: Community Analytics and Detection
- MyRepublic: Broadband
- NetWitness: Menace Detection & Response, Identification
- Palo Alto Networks: Community Safety Platform
The first mission within the NOC is community resilience. The companions additionally present built-in safety, visibility and automation, a SOC (Safety Operations Middle) contained in the NOC.
On screens outdoors the NOC had been displayed companion dashboards for the attendees to view the quantity and safety of the community visitors.
It All Began with Malware
Cisco joined the Black Hat NOC in 2016, when requested to offer automated malware evaluation with Thread Grid. The Cisco contributions to the community and safety operations advanced, with the wants of the shopper, to incorporate extra parts of the Cisco Safety Cloud.
The NOC leaders allowed Cisco (and the opposite NOC companions) to herald extra software program to make our inside work extra environment friendly and have higher visibility; nevertheless, Cisco isn’t the official supplier for Prolonged Detection & Response, Community Detection & Response or Collaboration.
- Breach Safety Suite
- Cisco XDR: Menace Searching / Menace Intelligence Enrichment / Government dashboards / Automation with Webex
- Cisco XDR Analytics (Previously Safe Cloud Analytics / Stealthwatch Cloud): community visitors visibility and menace detection
- Cisco Webex: Incident notification and staff collaboration
The Cisco XDR Command Middle dashboard tiles made it straightforward to see the standing of every of the related Cisco Safety applied sciences, and the standing of ThousandEyes brokers.
When the companions deploy to every convention, we arrange a world class community and safety operations heart in three days. Our aim stays community up time and creating higher built-in visibility and automation. Black Hat has the choose of the safety business instruments and no firm can sponsor/purchase their manner into the NOC. It’s invitation solely, with the intention of variety in companions, and an expectation of full collaboration.
As a NOC staff comprised of many applied sciences and corporations, we’re repeatedly innovating and integrating, to offer an general SOC cybersecurity structure resolution. We look ahead to persevering with the work with companion Palo Alto Networks, for additional automation at Black Hat USA 2024.
Beneath are the Cisco XDR integrations for Black Hat Asia, empowering analysts to analyze Indicators of Compromise (IOC) in a short time, with one search.
We admire alphaMountain.ai, Pulsedive and Recorded Future donating full licenses to Cisco, to be used within the Black Hat Asia 2024 NOC.
An instance of that is an investigation of a doubtlessly malicious exercise on the twond day of Coaching. An IP deal with was recognized by NetWitness for attainable geolocation leakage.
Investigation of the IP correlated the syslog sightings from the companion applied sciences within the NetWitness logs, with menace intelligence from Pulsedive, Recorded Future, alphaMountain and others.
Reviewing the DNS logs and the main points of the packet seize in each Corelight and NetWitness, it was confirmed no geolocation information was leaked and it was a part of a Coaching course. The exercise would have been blocked in a manufacturing setting.
A core built-in workflow within the Black Hat NOC is NetWitness and Corelight sending suspicious recordsdata to Safe Malware Analytics. Over 4,900 samples had been submitted.
The NOC analysts additionally used Malware Analytics to analyze suspicious domains, with out the chance of an infection. Fairly than going to the web site on a company or Black Hat property, we had been capable of work together with the web site within the glovebox, together with downloading and putting in the web site payload.
Detonating recordsdata or searching web sites in Safe Malware Analytics protects the analysts from unintentional an infection.
We noticed a collection of comparable (however totally different hash values) exploit kits downloaded on the primary day within the Enterprise Corridor. The downloads had been on the convention Wi-Fi and never in a Coaching course, so the occasion needed to be investigated to substantiate there was not an assault on the attendees. Working with the Corelight staff, the NOC responders parsed the visitors and confirmed it was a Seize the Flag occasion, which continued into the final day of the convention.
Menace Hunters’ Story, by Aditya Raghavan and Shaun Coulter
Within the Black Hat Asia 2024 NOC, Shaun staffed the morning shifts, and Aditya the afternoon shifts, as menace hunters centered on the Cisco XDR and Safe Malware Analytics consoles. Mornings had been normally fairly chill. Nevertheless, and for some heretofore unknown (espresso associated?) purpose, the exercise ramped up within the afternoon on most days, main Aditya to a spot of “concerned pleasure”, and Shaun to a spot of tormented jealousy :D. With dogged dedication each hunters spent their time reviewing alerts, actions, and carried out IOC scans utilizing XDR Examine. They reviewed submitted samples and community logs for indicators of intrusion or suspicious exercise.
Utilizing Safe Malware analytics, they dissected malware samples, analyzed phishing campaigns, and scrutinized community visitors patterns for anomalies. Quite a few alerts flagged as spikes in visitors from surprising sources, unusual locations and odd variants of malicious code popped up a number of occasions, initiating thorough investigations. Normally, they traced the anomaly to a licensed Black Hat Coaching or Briefing supply and closed such circumstances as “Black Hat Constructive”; that means you wouldn’t enable this in your manufacturing community, however for Black Hat, it’s enterprise as traditional. Since Black Hat is a convention designed for studying about offensive safety, these malware samples are anticipated, and marked as such.
Fortunately or unthankfully, because the system tuning was accomplished, most alerts raised had been as above and anticipated or truly ‘close to misses’ – gadgets that warrant investigation however didn’t prolong to impactful behaviours, as we had been capable of cease them in time.
On the primary day of Briefings, as Shaun is dutifully poring via the console of Safe Malware Analytics, in walks Aditya to alleviate the shift. Greetings apart, Shaun shortly pivots over excitedly “Brother, I wish to present you a few fascinating issues.” Aditya’s curiosity is piqued, and Shaun opens a brand new dashboard exhibiting one of many just lately launched options of Cisco XDR – MITRE ATT&CK ® Protection Map.
This new functionality shortly shows all of the techniques and strategies within the MITRE ATT&CK® matrix for which Cisco XDR has detections/protection. Along with the XDR Native, detections from Safe Endpoint and Safe Malware Analytics are additionally used to derive the protection map making it a holistic view. This view permits the consumer to visualise the detections of XDR natively, in addition to the built-in options and determine the scope of protection and importantly, map out the gaps for future consideration. Due to the Cisco Talos staff, all options throughout the Cisco Breach Safety Suite are mapped immediately and this might be rolled out to incorporate different suites and options, together with 3rd occasion integrations, quickly.
As our menace hunters geek out on the behind-the-scenes stuff on XDR, Jessica politely calls out “Adi. Shaun. Guys, there may be some new exercise on Umbrella. Are you able to look into it?” Nudged again to actuality, our menace hunters get to work – discovering needles within the stack of needles at Black Hat because it was rightly put by Grifter! Speaking about that, the brand new exercise seems to be a question for a website categorized as a Command & Management (C&C) area. Let’s dig into it.
A fast look into Umbrella Exercise Search exhibits the newest visitors exercise matching the C&C class that was allowed. Increasing the main points pane, we are able to see the area being queried and the id of the endpoint issuing the question which seems to be from the ‘Hacking Enterprises 2024 Pink Crew’. That could be a professional Coaching class at Black Hat Asia 2024. We pivot over to Umbrella Examine and see the explanation for this area being categorized as C&C and its indicators.
Let’s head over to XDR and question this observable towards all of the built-in options for extra intel. We shortly get a visible related graph and tabulated occasions on all of the related intel. The combination with NetWitness Logs gives us with occasions associated to that area, in addition to populating the graph with these relationships, together with the Umbrella occasion which was the supply for this hunt.
Trying on the proof, this turned out to be one other needle! Nothing untoward right here, we categorized this as a ‘Black Hat Constructive’ and moved on. Because the afternoon shift winds down, the staff is discussing potential locations for dinner and there may be all the time dessert to look ahead to on the finish. Aditya and Ryan had been pining for wealthy ice cream and House Finest Dessert seems to be the proper resolution for the ask. Within the NOC, the proper resolution is sort of all the time teamwork with all our companions.
One such occasion was when a Corelight hunter picked up a spike of visitors to uncommon locations. These look like DNS queries to a bunch of C&C domains. We shortly delve into Umbrella exhibiting us all of the domains being queried in a brief window and most of them being Malware and/or C&C categorized. This seems to be a system both being compromised or somebody deliberately doing a take a look at / recon for these domains.
Let’s examine a few of these domains in XDR. We will see numerous crimson icons on this visualization! The truth is, each queried area is classed as Malicious and identified to host different malicious content material. This doesn’t look anticipated for certain and that places the intentional take a look at / recon idea to relaxation shortly. Ben Reardon, the hunter from Corelight, places it succinctly “This field is pwned six methods to Sunday!” What else can we discover about this technique then?
Trying on the DHCP logs for the IP deal with, the Corelight hunter was capable of pinpoint the machine MAC deal with and hostname, which resembled a reputation. A brief Google search later, we now have a possible machine proprietor and the truth that he was delivering a session at Black Hat in one of many rooms subsequent door! A brief dialog with the particular person after his session ensued, the place the NOC leads suggested the NOC’s findings on his compromised system. He was grateful for the discovering and reached out for added context. This one turned out to be a ‘True Constructive.’
The next day, the staff has zeroed in on Turkish meals for the night. Ryan halts Shaun as he departs on the finish of his shift and calls for his lodge title and room quantity. “I’m gonna come knock at your door and wake you up tonight, man. I imply it. No day is just too lengthy. I used to do my shifts on three hours of sleep. Now, let’s go!” Ryan is deadpan severe. That’s what we thought whereas investigating our subsequent potential malware discovering.
One other occasion on the Umbrella console involves our consideration and this time it’s a question for a website categorized as Malware. The supply endpoint is shortly recognized from the Identification and Umbrella examine tells us this area is a part of the Malware block listing. In a traditional manufacturing community, this might ideally be blocked.
Black Hat isn’t your regular manufacturing community, and it attracts every kind of safety folks. And that’s precisely what it turned out to be this time. The Nationwide College of Singapore has a gaggle organizing common seize the flag (CTF) occasions and is working the same get-together at Black Hat. Go NUS Greyhats!
Actions involving malware what can be blocked on a company community should be allowed, throughout the confines of Black Hat Code of Conduct.
Community Observability with ThousandEyes, by Adam Kilgore and Patrick Yong
Deploying ThousandEyes at Black Hat is a rigorous course of involving numerous {hardware} (some proven under), configuration, testing, troubleshooting, and working across the convention heart.
Along with our typical deployment duties, we carried out a number of enhancements to the service. These enhancements included an overhaul of the dashboards to indicate granular information for every convention room, alongside mixture information for all the convention; and higher labeling and group of deployed brokers.
The ThousandEyes dashboard was projected on the big display within the NOC, for alerting on any community points, previous to experiences from customers.
On the troubleshooting aspect, we improved our log evaluation and assortment strategies and arrange centralized monitoring of wi-fi information. These efforts contributed to enhancements in visibility and agent uptime all through the convention.
In the course of the preliminary two days of Coaching classes at Black Hat, ThousandEyes brokers confirmed solely minor deviations from baseline because the Coaching classes got here on-line. Because the Coaching classes continued, efficiency was secure, with solely uncommon alerts for minor degraded throughput or average latency spikes. On Thursday, all of the two-day Coaching classes had been closed, and the convention shifted in the direction of Briefings, alongside two four-day Coaching classes that ran for the convention’s size. With begin of Briefings and opening the Enterprise Corridor, headcounts drastically elevated. ThousandEyes noticed degraded efficiency on the community, primarily within the giant convention rooms internet hosting the Briefings. The under picture exhibits a take a look at outcome from the Hibiscus 3610 ballroom:
The community path above exhibits heavy latency on the primary hyperlink to the default gateway, compounded by one other excessive latency hyperlink outdoors the convention community. A breakdown of connectivity for the above path is proven under:
The throughput quantity above is vital to this investigation. The Entry Factors (APs) for the Hibiscus 3610 ballroom had a mean throughput of round 174 Mbps. Reviewing AP logs, we discovered that 92 customers had been related to the identical AP from which the take a look at was run. Dividing the 174 Mbps by 92 provides a mean throughput consistent with the 1.7 Mbps proven above, so the poor connectivity was pushed by oversaturation of consumer connections on this space.
The Hibiscus 3610 room and different brokers in a close-by hallway persistently had the worst connection among the many convention rooms, as proven by our agent polling outcomes.
Whereas there have been limitations within the quantity of bandwidth obtainable for the convention on the whole, the information above suggests extra of the obtainable AP and bandwidth sources ought to be allotted to the Hibiscus 3610 ballroom and adjoining hallways for future convention topologies, which was shared with our Community Gear companion.
Meraki Techniques Supervisor, by Paul Fidler and Connor Loughlin
Our eighth deployment of Meraki Techniques Supervisor because the official Cell Gadgets Administration platform went very easily, and we launched a brand new caching operation to replace iOS units on the native community, for velocity and effectivity. Going into the occasion, we deliberate for the next kinds of units and functions:
- iPhone Lead Scanning Gadgets
- iPads for Registration
- iPads for Session Scanning
We registered the units prematurely of the convention. Upon arrival, we turned every machine on.
Then we ensured Location Companies enabled, all the time on.
As a substitute of utilizing a mass deployment expertise, like Apple’s Automated System Enrollment, the iOS units are “ready” utilizing Apple Configurator. This consists of importing a Wi-Fi profile to the units as a part of that course of. In Las Vegas, this Wi-Fi profile wasn’t set to auto be a part of the Wi-Fi, leading to the necessity to manually change this on 1,000 units. Moreover, 200 units weren’t reset or ready, so we had these to reimage as nicely.
Black Hat Asia was totally different. We took the teachings from Black Hat USA 2023 and coordinated with the contractor to organize the units. Now, should you’ve ever used Apple Configurator, there’s a number of steps wanted to organize a tool. Nevertheless, these could be mixed right into a Blueprint.
For Black Hat Asia this included:
- Wi-Fi profile
- Enrollment, together with supervision
- Whether or not to permit USB pairing
- Setup Assistant pane skipping
In Meraki Techniques Supervisor, we managed the functions by the assigned use, designated by Tags. Once we got here in on the primary morning of the Briefings, three iPhones wanted to be modified from lead scanning within the Enterprise Corridor, to Session Scanning for the Keynote, so the attendees might fill the corridor quicker. Reconfiguring was so simple as updating the Tags on every machine. Moments later, they had been prepared for the brand new mission…which was vital because the Keynote room stuffed and needed to go to an overflow room.
We additionally had been capable of affirm the bodily location of every machine if wiping was required as a consequence of loss or theft.
When it was time for the attendees to register, they only displayed their QR code from their private telephone, as obtained in e-mail from Black Hat. Their badge was immediately printed, with all private particulars secured.
This goes with out saying, however the iOS units (Registration, Lead Seize and Session Scanning) do have entry to non-public info. To make sure the safety of the information, units are wiped on the finish of the convention, which could be accomplished remotely via Meraki Techniques Supervisor.
Content material Caching
One of many greatest issues affecting the iOS units in Black Hat USA 2023 was the fast have to each replace the iOS machine’s OS as a consequence of a patch to repair a zero-day vulnerability and to replace the Black Hat iOS app on the units. There have been lots of of units, so this was a problem for every to obtain and set up. So, I took the initiative into trying into Apple’s Content material Caching service constructed into macOS.
Now, simply to be clear, this wasn’t caching EVERYTHING… Simply Apple App retailer updates and OS updates.
That is turned on withing System Setting and begins working instantly.
I’m not going to get into the weeds of setting this up, as a result of there’s a lot to plan for. However, I’d counsel that you simply begin right here. The setting I did change was:
Location and Jailbreak detection
One factor that we haven’t spoken about in a while is Jailbreak detection and Location. There are numerous components that we get again from a tool, however two of them, Location and Jailbreak should be retrieved from a tool utilizing a supplemental utility: On this case, the Meraki Techniques Supervisor agent.
HOWEVER, these can solely be retrieved from the machine if the applying is working within the background. If the machine has been rebooted, or the applying terminated, then we don’t get something.
One of many different painful, however comprehensible, features of MDM is that you would be able to’t launch an utility distant on a cellular machine…. However you’ll be able to!
On each Android and iOS, there’s a functionality referred to as Kiosk or Single App mode: Use circumstances for this are usually unattended units, like in eating places, or scanning units like supply drivers, and many others. And when sending the command to the machine to enter kiosk mode will launch the applying. You can too ship a command to take away kiosk mode from the machine too. The wonderful thing about this final level is that the applying stays in focus and open!
So, the opposite functionality that utilizing Meraki Techniques Supervisor provides us is the flexibility to schedule settings. Due to this fact, we are able to activate kiosk mode in the midst of the night time and take away it an hour later.
To make sure that this doesn’t influence the registration employees, we are able to go one step additional: after we’ve launched Meraki Techniques Supervisor, an hour later we are able to relaunch the registration utility, Swapcard Go.
Systematic ThousandEyes Agent Deployment
ThousandEyes has been a success at Black Hat. At an occasion the place understanding instantly the place points lie within the community and past to make sure an incredible convention is paramount, the visibility ThousandEyes provides is unimaginable. Provided that, and the complexity of the community right here, and provided that we now have a Mac Mini deployed for caching software program updates, as we’re utilizing Meraki Techniques Supervisor (SM) for different functions, I believed I’d take the chance to deploy the ThousandEyes Agent utilizing SM.
The opposite purpose is that, while we now have a substantial quantity of cloud and enterprise brokers, we had no endpoint brokers deployed. Nevertheless, issues are by no means that straightforward with software program deployment, primarily as a result of you’ll want to provision / configure software program as soon as deployed. On cellular units, that is easy, both utilizing settings payloads, or through the use of Managed Appe Config to configure an app.
On desktop, utilizing MDM, we are able to usually use issues like Managed Plists to do the identical factor, however the TE agent does NOT help this. As soon as put in, we should name the agent with a string.
So, to realize all this, we are able to bundle the agent and command right into a bundle utilizing a command line utility on the Mac referred to as PKGBUILD (extra particulars right here).
I additionally used a information I’d written for the Meraki Neighborhood, obtainable right here.
Info of be aware:
The Postflight:
#!/bin/bash
# this title will change with every model of the agent
installer -pkg /tmp/Endpoint Agent-x64-1.193.1.pkg -target /
/Functions/ThousandEyes Endpoint Agent.app/Contents/MacOS/te-agent –register “YOURUNIQUESTRING”
exit 0
The command to construct the bundle utilizing PKGBUILD
Extra particulars right here or watch the video.
Repurposing of Gadgets for the following present
We had been requested if there was something we might do to go away the units as they had been for the following present. After cautious consideration, we determined that we might go away the units in a state that was amenable to everybody. The foremost requirement was leaving the Swapcard Go app on the machine. However, because the app is provisioned for every present, it’s fairly the method to take away configuration after which re-add it….
So, the opposite factor to notice is the choices that we now have when putting in (and eradicating) an utility on a managed iOS machine:
Take away with MDM is the fascinating one, because it permits us to, reasonably than WIPING the machine on the finish of the present, to take away administration, together with any apps and settings, and their corresponding information.
The downside with that is that this was by no means a requirement at first of the present. So, we now want a course of in a selected order to facilitate this…. As that is for under a handful of units:
- Deprovision the app from units by unscoping the applying in Meraki Techniques Supervisor
- Wait to see this command has accomplished throughout all units
- Reprovision the app utilizing MDM once more, however with this being a brand new app set up, it should enable the OS to maintain the app in situ after an unenrollment
- Wait till accomplished
- Unenroll the machine
Area Identify Service Statistics, by Christian Clasen
Since 2018, we now have been monitoring DNS stats on the Black Hat Asia conferences.
The historic DNS requests are within the chart under.
With over 18.2M DNS requests made, we had probably the most so far at an Asia present. We made visibility developments on the earlier 12 months’s Asia convention. Previous to Asia 2023, we had been permitting attendees to make use of their chosen DNS resolvers over our assigned inside Umbrella Digital Home equipment. In coordination with Palo Alto Networks (the convention Firewall supplier), we started intercepting and redirecting DNS queries for different resolvers, to pressure decision via the Umbrella gear. Whereas that is solely efficient for plain-text DNS queries and never encrypted protocols like DNS over HTTPS, it never-the-less dramatically elevated visibility as evidenced by the numbers within the accompanying charts.
The Exercise quantity view from Umbrella provides a top-level stage look of actions by class, which we are able to drill into for deeper menace looking. On development with the earlier Black Hat Asia occasions, the highest Safety classes had been Malware and Newly Seen Domains.
In a real-world setting, of the 18.2M requests that Umbrella noticed, over 2,000 of them would have been blocked by our default safety insurance policies. Nevertheless, since it is a place for studying, we usually let the whole lot fly.
We additionally observe the Apps utilizing DNS, utilizing App Discovery.
- 2024: 4,327 apps
- 2023: 1,162 apps
- 2022: 2,286 apps
App Discovery in Umbrella provides us a fast snapshot of the cloud apps in use on the present. Not surprisingly, Generative AI (Synthetic Intelligence) has exploded over the earlier 12 months as a prime utility.
Umbrella additionally identifies dangerous cloud functions. Ought to the necessity come up, we are able to block any utility through DNS, equivalent to Generative AI apps, Wi-Fi Analyzers, or the rest that has suspicious undertones.
Once more, this isn’t one thing we’d usually do on our Common Wi-Fi community, however there are exceptions. For instance, once in a while, an attendee will study a cool hack in one of many Black Hat programs or within the Arsenal lounge AND attempt to use mentioned hack on the convention itself. That’s clearly a ‘no-no’ and, in lots of circumstances, very unlawful. If issues go too far, we’ll take the suitable motion.
In the course of the convention NOC Report, the NOC leaders additionally report of the Prime Classes seen at Black Hat.
General, we’re immensely happy with the collaborative efforts made right here at Black Hat Asia, by each the Cisco staff and all of the companions within the NOC.
Black Hat USA can be in August 2024, in Las Vegas. Christian Clasen will lead the Cisco staff within the NOC, so comply with his weblog to see if what occurs in Vegas, stays in Vegas.
Acknowledgments
Thanks to the Cisco NOC staff:
- Cisco Safety: Christian Clasen, Shaun Coulter, Aditya Raghavan, Adam Kilgore, Patrick Yong and Ryan Maclennan
- Meraki Techniques Supervisor: Paul Fidler and Connor Loughlin
- Extra Assist and Experience: Adi Sankar, Robert Harris, Jordan Chapian, Junsong Zhao, Vadim Ivlev and Ajit Thyagarajan
Additionally, to our NOC companions NetWitness (particularly Iain Davidson and Alessandro Zatti), Palo Alto Networks (particularly James Holland and Jason Reverri), Corelight (particularly Mark Overholser and Eldon Koyle), Arista Networks (particularly Jonathan Smith), MyRepublic and all the Black Hat / Informa Tech employees (particularly Grifter ‘Neil Wyler’, Bart Stump, Steve Fink, James Pope, Michael Spicer, Jess Jung and Steve Oldenbourg).
About Black Hat
Black Hat is the cybersecurity business’s most established and in-depth safety occasion collection. Based in 1997, these annual, multi-day occasions present attendees with the newest in cybersecurity analysis, improvement, and tendencies. Pushed by the wants of the neighborhood, Black Hat occasions showcase content material straight from the neighborhood via Briefings shows, Trainings programs, Summits, and extra. Because the occasion collection the place all profession ranges and educational disciplines convene to collaborate, community, and focus on the cybersecurity matters that matter most to them, attendees can discover Black Hat occasions in the USA, Canada, Europe, Center East and Africa, and Asia. For extra info, please go to www.blackhat.com. See the press launch for Black Hat Asia 2024.
We’d love to listen to what you suppose. Ask a Query, Remark Beneath, and Keep Linked with Cisco Safety on social!
Cisco Safety Social Channels
Share:
