MIRcon: What the Cosmos can Teach us about Security

A few people have asked me what central theme and message stayed with me after last week’s MIRcon. I would say that Dr. Neil deGrasse Tyson’s keynote resonated with me and matched the central theme I felt during the conference; allow me to explain.

During his keynote, Dr. Tyson spoke to us about science, and, specifically, the scientific method, allowing us to objectively overcome our natural human biases. In other words, science is about forming a hypothesis, testing that hypothesis through accurate measurement, and reaching an objective conclusion based on the observed data. Security is the same, or at least it should be.

In the security domain, we don’t always take advantage of and apply the rich foundations of knowledge and expertise that exist in other domains. Often these other domains are far more mature than our own. I can think of no better example of this than science. For hundreds of years, scientists have used an agreed upon, methodical approach to advance the state of science. We can learn a lot from this conceptually and apply it to the security domain.

At MIRcon, the presentations I saw and the discussions I was fortunate enough to take part in indicated to me that we have begun to approach the security domain far more scientifically. Gone are the days of emoting and guessing – the problems are far too complex, the data too diverse, and the attackers too sophisticated. As a profession, we have begun to demand a far more scientific approach to the security domain than was historically the case.

The atmosphere at MIRcon was invigorating. Security professionals have tired of unsupported hypotheses – we are ready for a more formal approach. Today’s challenges require a more scientific way of thinking. We need to explicitly identify and enumerate the challenges we are facing in the field, hypothesize the solutions to those challenges, test those solutions through accurate measurement, and reach objective conclusions about the merits of those solutions.

Recommendations, beliefs, and hypotheses are in no shortage in our field. But are they accurate, do they solve security problems, and do they address the challenges of the day? The answer to those questions needs to be evaluated scientifically, rather than debated in the absence of accurately measured data.

Security has evolved from a niche profession to a mainstream one. As such, our work must stand up to the same rigor we would apply to any other profession. Any other approach would simply be unscientific.

 

Flying Blind

With all the news about data breaches lately, it’s not particularly surprising to wake up to headlines describing yet another one. What is perhaps a bit surprising, however, is the common theme that seems to exist in many of the breach stories. Time and time again, when organizations get breached, they find out the hard way that they don’t have the endpoint and network visibility they thought they did. The necessary data to perform the forensics required to reach an analytical conclusion is simply missing. Further, there is no way to remedy this situation – if the data was not properly recorded when it traversed the network or endpoint, there is simply no way to access it.

What are some of the reasons that data is not available come breach response time? Let’s take a look at a few of them.

  • Collection: One of the goals of a security program is to ensure that the necessary network and endpoint data are collected. Unfortunately, this is often a challenge for even the most mature of security programs. In some cases, organizations may not have their networks and endpoints properly instrumented for collection. In other cases, organizations may not be properly equipped to retain and expose for analysis the volume of data created by the network and endpoint instrumentation. Either way, when it comes time to investigate, the relevant data will not be available.
  • Visibility: More data doesn’t necessarily mean more visibility or coverage. There is an important distinction between the volume of the data and the portions of the organization that it provides visibility into. Some organizations may have portions of their networks or endpoints instrumented for collection, but not others. But what if the breach occurs in an area of the network or on an endpoint that is not included in the area of visibility? In those cases, unfortunately, data that is relevant to the breach investigation will not be available for forensics and analysis.
  • Retention: Another important dimension to consider is that of retention. In the absence of an infinite volume of storage, data cannot be retained forever. Today’s organizations generate incredible amounts of data from their collection efforts. Sometimes, the network and endpoints are properly instrumented in the appropriate places, but there is simply nowhere to put the volume of data that is generated. As the volume of data grows, either the retention period shrinks, or the storage capacity grows to compensate. It is not uncommon for the retention period to fall to 30 days, or even less. With mean-time-to-detection at a staggering 229 days, it is easy to see that 30, 60, or even 90 days of retention is simply inadequate when it comes time to perform forensics and analysis. Although the relevant data for the investigation may have existed at one time, if it isn’t present when we perform our investigation, it doesn’t help us much. This necessitates us getting a bit smarter about what data we retain. Our goal should be data that provides us maximum visibility into the network and endpoints, but at the minimal volume. Perhaps it sounds a bit radical to say, but the days of “collect everything” are gone – instead we find ourselves in an era of “collect the most relevant things”.
  • Analysis: Even if our collection, visibility, and retention are squared away, we may still encounter frustrations and limitations when performing incident response. Although we may have the data we need over the time period we need it for, we still need to be able to analyze it. If we are unable to extract the data rapidly from our forensic collection platforms, we will be unable to analyze it. Simply put, what goes in must come out. For example, say we need to search for the first appearance of a given Indicator of Compromise (IOC) over the entirety of our retention period. For this example, let’s assume our retention period is on the order of 12 months. If that query fails before completing or takes days to complete, it is of no value to incident response. Incident response demands answers in seconds or minutes, rather than hours or days.

Despite the steady stream of bad news regarding data breaches, there is some good news. By taking proactive steps, organizations can prepare themselves to perform rapid and efficient incident response when they become the victim of a breach. Among many details, it’s important for an organization to consider the points above when assessing its breach preparedness.

When POS Comes to Shove

In today’s blog post, FireEye examines the threats posed to retailers by crimeware, Point-of-Sale (POS) malware, and other threats. It is certainly a topic that is on the mind of many organizations and individuals these days. But with all the hype and buzz, what proactive steps can a CISO take to better defend his or her organization against these threats? There are many potential approaches that could be taken, but two foundational concepts that come to mind are:

  • Best practices and first principles
  • Continuous Security Monitoring (CSM)

Best practices and first principles are not rocket science, but they still rule the day. As discussed in additional detail in the FireEye blog post on BrutPOS best practices can go a long way towards helping an organization defend itself. First principles such as identity management, sensible permissions, adequate controls for remote logins, and others can help keep an organization from falling victim to the wide variety of threats that it faces today. CISOs can do their part by communicating their vision for assessing the weak links in the chain and strengthening them. It is an iterative process and one that will not be fully completed in a day, a week, or even a month. But the CISO that pushes and motivates his or her organization in this direction will be doing that same organization a great service. It is always better for the organization itself to find a weakness in its security posture than for the attackers to find it.

Despite our best efforts and intentions, however, intrusions and breaches will still inevitably occur. In those instances, our attention quickly turns from prevention to detection and response. Continuous Security Monitoring (CSM) is the formalized process through which we build and enhance our organizational capability to rapidly detect, analyze, contain, and remediate intrusions and breaches. After all, breaches happen, but what a CISO must truly be on the lookout for is the theft of sensitive, proprietary, or confidential data. The financial, legal, and PR damage caused by an intrusion of any scale can be minimized, but only if that intrusion is detected and responded to rapidly. Proactively enhancing the organization’s CSM capability allows a CISO to markedly improve the security posture of the organization.

As an example, consider the case of a Point-of-Sale (POS) malware sample entering an enterprise network. This will likely trip one or more alerts that will be sent to the organization’s work queue (I.e., SIEM, incident ticketing system, etc.).

The first challenge we encounter here is ensuring that this alert does not get overlooked or lost in the noise. This can be accomplished by ensuring that we methodically approach the process by which we develop content to generate alerts for the work queue. We want to ensure a high enough rate of true positives to false positives, or signal-to-noise ratio.

Next, we will need to ensure that an analyst vets, qualifies, and analyzes the relevant alert or alerts. We can ensure this occurs by following a rigorous, formalized incident response process at strategic and tactical levels, along with ensuring we adequately train our staff.

As the analyst reviews the alert, we will need to ensure that the appropriate contextual information in support of the alert can be retrieved quickly and easily. This requires visibility across the network, endpoint, and intelligence in order to enrich the alert data with supporting evidence that will allow us to draw a conclusion as to whether or not we have a compromise, along with the scope of that compromise.

Lastly, we will need to contain and remediate the intrusion. These steps ensure that we stop the POS malware’s progress dead in its tracks — before it can steal valuable and sensitive payment card information from our organization.

If this seems like the familiar people, process, and technology triad, there is good reason for that. We must remind ourselves that it is no one piece of malware or intrusion that lands us in trouble. Rather, it is not detecting and responding to that intrusion in a timely manner that causes the damage.

It is certainly not easy to be a CISO these days. The microscope and heat lamp seem continually focused upon those in the role. The good news is that through a combination of best practices and Continuous Security Monitoring, CISOs can take a proactive stance to defend and protect their organizations against the breaches of today and of tomorrow.

Apple OS X: Security Through Obscurity is becoming an Absurdity

Today’s blog on a new Mac malware is a reminder that attackers go where the money is. Apple usage within the enterprise is growing rapidly, with 52 percent of newly issued computers being Macs according to Forrester. Forrester also highlights that executives and manager level employees often the prime targets of advanced attackers ­ represent 41 percent of enterprise Apple users. And with more of the enterprise brain trust using the Mac platform, VIPs are a logical and rich target. And now we see attackers simply porting Windows malware for Mac. The moral for security teams? Today’s blog disproves the “security through obscurity” moniker traditionally associated with using Macs to stay safe. Security teams: gear up now.

How do you do that? Well, for starters, it is important to remember several fundamental security operations and incident response best practices that can help combat this and other threats:

  • Develop, continually improve, and follow a formal incident response process
  • Perform gap analysis to determine where “blind spots” in visibility may exist
  • Ensure proper network instrumentation to address any lack of network visibility
  • Ensure proper endpoint instrumentation across all operating system platforms
  • Leverage a rigorous content development process to create high fidelity alerting that produces a unified work queue with a high signal-to-noise ratio (ratio of true positives to false positives)
  • Practice Continuous Security Monitoring (CSM) to rapidly detect and respond to any potential breaches or intrusions
  • Strive for smooth operations to include ensuring that staff are adequately trained and equipped
  • Incorporate actionable intelligence
  • Participate actively in both formal and informal information sharing forums

There is no one silver bullet that will immediately quash all the risk presented by APT actors and other threats. Rather, as security leaders, we need to ensure that we put the people, process, and technology in place to properly manage the risk our organizations face on a daily basis. A formal, rigorous security operations and incident response program is a key component of this endeavor.

Network Forensics: Use Cases In the Enterprise

Network forensics is an important component of a successful security operations program. It is an important capability that provides a data of record for the incident responder and plays an important role in the daily security operations workflow. While the utility and importance of network forensics may be clear to the security professional, that value may be difficult to communicate to the business decision maker or executive. In this post, I’d like to discuss several business use cases for network forensics that may help communicate the value and business need for network forensics as an integral component of incident response.

Breach Response

When an organization discovers, or is notified of a breach, time becomes of the essence. The organization’s immediate focus becomes moving quickly from detection to containment. In order to make this move, the organization needs to answer a set of essential questions aimed at identifying the extent of the breach and the damage caused by it. This process is often called breach response and involves investigation, analysis, and forensics. Examples of some of the questions that will need to be answered are:

  • How long has this activity been going on (i.e., when did the intrusion begin)?
  • Is the activity still going on?
  • How many systems were affected?
  • What data was taken?
  • Was any sensitive, proprietary, or confidential information taken?

These and other relevant questions are designed to focus the organization on rapidly identifying the extent of the damage, both for containment purposes, but also to address potential public relations, legal, and privacy concerns. For example, consider the case where a law enforcement agency approaches a business (of any size) and informs them that they have been breached and have been observed communicating with a known drop site. The organization will have many questions, including, among many others, those listed above. An accurate, cohesive, lossless data of record is required to properly answer all of the necessary questions. Further, it’s not sufficient merely collect the data, but rather, it must be easy to precisely, incisively, and rapidly extract that data for analysis and forensics.

Hunting

On any network, there will be unusual or suspect activity from time to time. Sometimes, this unusual activity can be indicative of advanced threats and targeted activity. Many times, the threat actors are quite adept at keeping a low profile and executing actions on objectives subtly. For example, Advanced Persistent Threat (APT) actors may compromise an endpoint inside of an organization and slowly collect sensitive, proprietary, or confidential information to stage for subsequent exfiltration. While detecting this activity in an automated fashion would be ideal, this turns out to be very difficult in practice. As a result, if this activity is successfully detected within the organization (as opposed to via a third party), it is most often done so via hunting. Hunting is the activity through which skilled analysts use a variety of different analytical techniques to “slice and dice” the network traffic data in the “hunt” for this subtle malicious activity. The best analysts will want to issue targeted, precise, incisive queries designed to extract the proper forensics data rapidly, with minimal noise. This requires a network forensics capability that can rise to this challenge at enterprise network speeds and traffic volumes.

Metrics/Network Knowledge

Metrics provide important data points to the decision maker and executive. As a recent example, let us consider the many new Top Level Domains (TLDs) that have become available for use. Attackers have already leveraged some of these TLDs for malicious purposes, and this activity will undoubtedly continue to increase. This example begs the question: If a TLD serves no legitimate business purpose and can only expose the organization to risk, should it be blocked proactively? I believe the answer to this question is yes. But how can we ensure that a TLD serves no business purpose? This is where the metrics and network knowledge become so crucial. Business decisions should be based on facts, and facts come from an accurate, precise data of record – the network forensics data. This is merely one example of the many business questions that can be answered with metrics driven by network forensics data. When business decision makers or executives need answers, it is best to be able to provide answers based on ground truth.

Intelligence

When leveraged properly, actionable intelligence can provide additional enrichment and maturity to a security operations program, as well as aid in the improved detection of intrusions. Leveraging intelligence properly involves many details, but one of the most important details is that a reliable data of record exists. After various Indicators of Compromise (IOCs) are received and vetted, they should be leveraged against a reliable data of record in order to maximize their value. There are two time-based aspects here – historical and ongoing. We can run IOCs against our historical data to check for evidence of intrusions present on the network from the past on through the current day. In addition to that, we should also monitor for evidence of intrusions on an ongoing basis and raise an event to the alert queue when we see that evidence. These are both productive activities, assuming that an accurate, cohesive, lossless data of record exists for us to run the IOCs against. Further, it is not merely enough to collect the data – we need to be able to rapidly and surgically extract the data through targeted, precise, and incisive queries. For example, an organization may receive a daily feed of malicious command and control domain names from one or more of its intelligence sources. That data needs to be run against a corresponding data of record to find instances of command and control activity present on the organization’s network indicating that some systems are compromised. A scalable network forensics solution, one that can both record all of the network data at high speed as well as make that data and meta-data available for analysis, is required in order to properly leverage intelligence.

DNS/Passive DNS

Data from Domain Name System (DNS) queries and responses provide a wealth of insight into unusual or suspect activity that may be occurring on the network. For example, most users pointing their browsers at legitimate websites will request domain names that resolve to routable, public IP addresses. But what if a resource on the network repeatedly requests a domain name that resolves to private, non-routable IP address or one that has no resolution at all? Further, what if that domain name suddenly “comes alive” and begins resolving to a routable, public IP address and/or the resource begins exfiltrating data to that IP address? This is just one of many interesting applications of DNS query and response data. As interesting and crucial as this data is, many organizations struggle to maintain adequate visibility across their DNS infrastructure. There are many reasons why this is the case, but there is a solution. A network forensics platform collects layer 7 enriched meta-data for many application protocols, DNS among them. This provides an organization with a de facto DNS monitoring and passive DNS data collection system, without requiring the organization to invest in additional technology or hardware. It’s one of my favorite use cases for network forensics and likely one that will resonate with many readers. For smaller organizations, there is also the additional benefit of using using one network forensics technology for multiple purposes.

Intelligent Alerting

As the old saying goes, ask a stupid question, get a stupid answer. The modern attacker is intelligent and sophisticated. We would be naïve to think that we could identify a sly attacker’s subtle activity with dull, generic queries. If we want to find the intelligent attacker, we need to ask intelligent questions. Asking intelligent questions requires two fundamental components. The first required component is that the data be collected and its associated meta-data extracted and indexed for rapid search. The second required component is a robust query language that allows the analyst to ask incisive, targeted, precise, intelligent questions of the data. It’s likely not a surprise that a mature, scalable, powerful network forensics solution provides both of these required components. For example, say I am a mid-sized organization concerned by potential theft of intellectual property from my executives. With the right solution in place, I can precisely craft my alert logic so as to focus in on the specific employees, systems, data, and threats I am concerned with. In the absence of that capability, many organizations struggle to issue queries powerful enough to identify suspicious and malicious activity designed to behave subtly and fly under the radar.

There are many use cases for network forensics, but I’ve tried to list those that may help to reinforce the strong business need for network forensics. When the need arises to perform breach response or any of the other use cases listed above, the organization that has implemented a robust, scalable network forensics solution will fare better than the organization that has not. With the stakes so high these days, it would be a shame not to be prepared.

BrutPOS From a Security Practioner’s Perspective

Today, FireEye Labs posted a technical blog on the malware for a botnet that we call BrutPOS. With a lot of attention focused on data breaches in retail, BrutPOS gives us a chance to look retrospectively on the state of retail security.

The popular phrase “a chain is only as strong as its weakest link” has great relevance in the information security world. There are a large number of ways to compromise a business network, yet attackers are quite successful in this endeavor using fairly pedestrian methods of attack. This raises the important question: Why is this the case? Part of the answer lies in the fact that attackers don’t feel a great need to use particularly sophisticated attack methods. In other words, if attackers can succeed using fairly elementary attack methods, why should they work any harder? Let’s examine this principle through the example of the BrutPOS malware.

Most businesses use Microsoft’s Remote Desktop Protocol (RDP) as an integral part of their day-to-day business operations. RDP allows for remote login to Windows systems. This has many legitimate uses, such as an administrator logging on to a system remotely to update a software package. Like any legitimate service, attackers are also quite happy to leverage RDP for their own nefarious purposes. An example of this is the BrutPOS malware, the analysis of which was detailed in a FireEye blog post today. At a high level, the purpose of the BrutPOS malware is to compromise Point of Sale (POS) terminals through the use of the remote desktop protocol (RDP). The malware aims to steal payment card information from those compromised POS terminals. There is no need for the attackers to write a sophisticated protocol for their malware to log on to systems remotely – the RDP works quite nicely.

There are many approaches an organization can take to better manage the risk presented in the BrutPOS malware. One of those approaches is to go back to basics and remember some important foundational tenets of information security. This approach involves ensuring that authentication and authorization policies are sensible and enforced across the organization. For example, some simple steps an organization can take to improve its defenses against threats such as BrutPOS include (but are not limited to):

  • Not allowing administrative access to systems, with the exception of special administrative accounts for administrators
  • Locking out accounts after N number of incorrect login attempts
  • Not allowing RDP login by default on systems, but rather, granting it on an as needed basis
  • Limiting or eliminating the use of shared or group accounts
  • Monitoring authentication logs for repetitive failed login attempts to one system or multiple systems

As organizations look to continually improve their information security postures, it’s important to remember that foundational tenets are as valid as ever. We do need to ensure that we have a variety of defensive measures in our arsenal, but it’s important to remember that not all of them need be cutting edge. Sometimes, foundational best practices can provide us with straightforward approaches to mitigating risk posed by modern threats.