In the realm of Sales Engineering, there’s a nuanced aspect that defies standard definition. It hinges on seasoned experience, unyielding determination, and the capacity to truly hear a customer’s voice and respond to their needs. Success in this realm demands adept discovery skills, adept negotiation both internally and externally, and the knack for viewing a customer’s needs through both tactical and strategic lenses. In mid-2021, I undertook a project that epitomized these demands, armed with neither a set timeline for completion nor substantial resources—save for one directive: proceed without additional expenditure.
The Challenge:
In late 2020, Tulane University inked a 3-year pact with AT&T for Managed Threat Detection and Response (MTDR). However, over six months into the deployment, progress languished, and the university’s CIO grew increasingly frustrated by the lack of headway. Despite sending logs from three sensors into the platform daily, the control node remained non-responsive. My mission was clear: rectify the situation.
Several stumbling blocks marred the MTDR deployment over the preceding three months. Here’s a breakdown of the key issues:
Sensor Overload: The Palo Alto PA7050 Firewall’s sensors struggled to ingest logs effectively, leading to overload and subsequent crashes due to the sheer volume of incoming data.
Resource Overutilization: The sensor overload cascaded to the control node, causing resource overutilization. This, in turn, hampered the processing speed of data as it traversed from the premise-based sensor to the cloud-based control node.
Delayed Threat Model Workshop (TMW): Initially stalled by contractual hitches, the TMW—a vital precursor to deployment—was delayed. However, it has since been successfully conducted, marking a pivotal step forward in the resolution process.
In navigating these challenges, my approach merged technical expertise with proactive problem-solving, ensuring a swift and effective resolution to Tulane University’s MTDR woes.
The Solution
Upon the completion of the TMW, I collaborated closely with both the deployment team and the customer to realign the deployment process in accordance with the TMW findings. Dividing the deployment into a multi-phase endeavor, we prioritized assets based on their criticality, ranging from indispensable to less crucial items slated for migration into the monitoring platform.
Our first move was to migrate the PA7050, aiming to mitigate controller overload. Working in tandem with the customer’s firewall engineers, I provided guidance on segmenting logging by type and subnets, ensuring that only logs pertinent to the assets under monitoring were brought into the system.
Subsequently, we transitioned critical servers, load balancers, and pivotal switches into the platform. Remarkably, the log volume from these assets paled in comparison to the PA7050, yet it propelled the customer just beyond the 90% utilization threshold for the contracted storage capacity. Recognizing the value of actionable insights gleaned from the platform, the customer opted to upscale their storage capacity, amending the contract from 6TB to 20TB of monthly storage.
This transformative process not only left the customer thoroughly satisfied but also underscored the tangible enhancements in their security posture. With the revised contract inked, the deployment team diligently facilitated the migration of remaining assets into the platform. By our last interaction, the customer had achieved approximately 85% visibility across their network, servers, and endpoints, marking a significant milestone in their security journey.
