Skip to content

Selecting a Managed Security Services Providers: Selection Criteria

August 28, 2008

Using a Managed Security Services Provider (MSSP) for Intrusion Detection and Prevention (IDP) services can be a cost-effective choice when compared to a self-managed infrastructure. However, when comparing specific providers, price should be one of the last factors considered as it’s often a case of “you get what you pay for.” Other criteria become far more important once the decision to use an MSSP has been made.

Executive Summary

Intrusion Detection and Prevention (IDP) helps IT departments protect valuable information assets. Using a Managed Security Services Provider (MSSP) typically allows for this capability to be delivered in a more cost-effective manner.
This research note highlights service aspects other than price that should be evaluated when selecting an IDP MSSP. Coverage includes:
» Security operations centers.
» MSSP staffing.
» Service Level Agreements (SLAs). » Reporting capabilities.
When selecting an MSSP to provide IDP services, too much variability exists in the capabilities of the providers to make decisions based on cost alone.
Selection Criteria
When it comes to Intrusion Detection and Prevention (IDP), utilizing a Managed Security Services Provider (MSSP) is often a more cost-effective solution than a self-managed infrastructure. Though cost is initially important in determining whether or not to use an MSSP, once the decision has been made, pricing must become a secondary consideration. Most service providers use similar prices and pricing structures, so costs from a variety of MSSPs will generally be equivalent for equivalent services. Thus, the capabilities of the services in question must be evaluated when determining the appropriate vendor.

Key Selection Criteria

There are four factors that must be considered when evaluating MSSPs for the delivery of managed IDP: » Security Operations Centers (SOCs).
» Staffing levels and skills.
» Service Level Agreements (SLAs).
» Reporting infrastructure.

Though many more factors exist, these four will ultimately determine the viability of the offering and therefore influence the multitude of other factors.

Security Operations Centers

The SOC is the key to effective delivery of any managed security service. This facility is where all collected data is aggregated, processed, analyzed, and responded to. To achieve high levels of service, redundancy is required in the SOC and it is this redundancy that must be evaluated. An MSSP must have:
1. Redundant SOCs. In order to avoid loss of service through catastrophic failure at the SOC, it is important that the MSSP maintains two or more redundant SOCs. These facilities should each be capable of supporting the full capabilities of the provider and have sufficient capacity to support all processing for all clientele. To enhance resiliency in the event of catastrophe, the SOCs must be geographically dispersed, though as the distance between sites increases, the complexity of keeping data synchronized between the two can also increase.
2. Redundant network connections. Internet connectivity is essential to an MSSP – without it, connection cannot be established between the monitored network and the SOC. It is essential that multiple diverse connections to the Internet are provided and that these connections are all of
equivalent throughput capabilities. Furthermore, to enhance the reliability of communications, the Internet connections should be provided by different providers at a physical (not logical) level so that the failure of one carrier does not wipe out all communications channels.
3. Redundant power capabilities. To ensure that power is always available, the MSSP must take several steps. First, is the acquisition and implementation of an Uninterruptible Power Supply (UPS); in essence, a large battery that provides short-term backup power. This must be backed up with a generator of sufficient capability. Ideally, dual generators should be provisioned in case the primary fails. In terms of power delivery, connecting the SOC to dual power grids and then connecting each computer to dual power distribution units are both good architectural decisions

Staffing Levels and Skills

The second major differentiator that needs to be considered when evaluating MSSPs is the staff. Employees are the single biggest cost in a self-managed infrastructure so it is essential that the providers staffing is evaluated.
1. Number of staff. Management of an IDP system 24/7 requires a minimum of 5 individuals. This allows only for single coverage and provides no coverage for vacations, sick days, and so on. As these analysts are “shared” across multiple accounts, the need for multiple analysts per shift increases. Determining the number of analysts is essential to ensuring that the MSSP has sufficient staff to provide efficient round the clock coverage.
2. Skill levels of staff. Though not an overtly technically challenging job, being able to quickly and accurately interpret the alarms generated by the IDP system requires specialized skills. The level of training and certification of the security analysts is a strong indicator of the accuracy with which alerts will be assessed.
3. Tenure of staff. Monitoring the alerts for an IDP system can be alternately boring and frenetic and is almost always tedious. Off-setting and minimizing these factors can be a careful balancing act. Determining staff employment terms indicate how well these factors are managed and how focused analysts are likely to be.

Service Level Agreements

When it comes to service providers, promises are about as valuable as the paper on which they are printed. To ensure that service is delivered at the levels agreed to, SLAs are needed. However, not all SLAs are created equally.
1. Factors that are measured. Numerous factors exist that can be measured, but in the end there are just a few key points against which everything else is secondary. SLAs should be placed on the service, not the infrastructure supporting the service. Pay attention to the following:
» Communication rate between monitored facility and SOC. Alerts must be delivered to the SOC in a timely fashion.
» Response rate to valid alarms. Alerts must be processed and responded to in a timely fashion.
» Missed attacks. Penalties must be levied if successful attacks occur and no notification is provided.
If an MSSP can’t provide an SLA covering these factors, they likely aren’t being measured. If they aren’t being measured, how effective can the service be?
2. Calculating and applying penalties. Penalties are to be calculated any time the provider fails to meet the established SLA. By basing the SLA on service functionality and not technical capability, many exception and exclusion clauses can be avoided, resulting in more accurate availability measurements. Ensure that the penalties levied are sufficiently punitive to offset the increased risk should SLAs not be met. Further, penalties should be applied automatically, not by “opting- in,” and assured that they are applied to the subsequent billing cycle.

Reporting Infrastructure

In order to determine what happened during the monitoring period, the MSSP needs to provide access to some form of reporting infrastructure. The more versatile the reporting system, the easier it will be to extract information from it and realize the value of the system as implemented.
1. Report delivery mechanisms. In order to allow the greatest level of insight into the activities recorded by the IDP system, real-time reporting offers tremendous benefits. As such, real-time report access through a Web portal or equivalent is far preferable to paper reports delivered at fixed increments. The portals should maintain an inventory of “canned” reports as well as offer the ability to generate custom reports on the fly. Access to these report portals should require some form of secure authentication to limit access to this valuable security data.
2. Report types. A variety of report types should be made available as indicated above. At a minimum these should include:
» High-level “dashboard” reports. Ideally suited to senior management to provide an “at a glance” overview.
» Detailed threat analysis reports. Essential for security managers to understand what is being targeted and how.


1. Use the “Managed IDP Evaluation Questionnaire post” This document will outline questions to ask of MSSPs when evaluating potential partners.
2. Don’t focus only on price. While price is a determining factor in choosing to use an MSSP in the first place, pricing models tend to be similar between the different providers while the services offered tend to be dissimilar.
3. Look at infrastructure – the foundation of the MSSP’s abilities. Service interruptions are going to occur. The frequency of these interruptions and their severity when they occur is mitigated by the reliability and redundancy of the supporting infrastructure.
4. People manage the processes so make sure they’re up to the tasks. While infrastructure is how the service gets delivered, the people are really the service. As such, the more people and the deeper their knowledge, the better the service is likely to be.
5. Create a strong reporting infrastructure. It doesn’t matter how good a service is if the data about the threat activities is unavailable or undecipherable. Ensuring that reports are clear and timely will help demonstrate value to senior management.
6. SLAs say a lot about the provider – pick one that stands behind its word. Anyone can make promises about the level of service being delivered. Only the truly capable will back it up with meaningful, measurable SLAs.
Bottom Line
While variability does exist in the pricing for managed IDP services from MSSPs, it should not be a primary decision-making criterion. Technical infrastructure, staffing breadth and capabilities, reporting infrastructure, and the SLAs that support them are the differentiators that should be evaluated instead

One Comment leave one →
  1. August 28, 2008 8:12 pm

    Consider using a flywheel UPS for bridging to generator instead of batteries. They’re cleaner, more efficienct, and more reliable than battery based UPS.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: