Posted by: Eric Siegel
Software as a Service (SaaS) is tempting a lot of
enterprises, but I worry that the service level agreements (SLAs) for these
services may be incomplete.
It's important to define precisely what the service provider
means by availability and performance. Is it measured inside the provider's
server center? If so, then that's not particularly useful, unless your
salesreps and other employees typically have their meetings there!
What you should want is a set of metrics to the locations
where your staff does their work; e.g., inside your branch offices or on the
road, across the Internet. Make the SaaS provider responsible for ensuring that
communications is acceptable to the boundary of your office or to major
Internet access points.
You can set up measurements at your branch office boundary
router (to ensure that the provider isn't held responsible for problems inside
the branch office network) and measure round-trip latency, availability, and
possibly the home page download time for the SaaS provider from that point.
(Coupled with a server-room measurement from the provider of the SaaS service,
you will probably need only DNS resolution time to the SaaS site, availability of the SaaS servers, and round-trip TCP latency to the SaaS servers, unless the
SaaS design is such that multiple server rooms are involved.)
Be sure that both you and the SaaS provider can see the
metrics in real time and agree on the technology involved; the goal is to have
the provider notice that the SLA metrics are in trouble and fix the underlying
problem before a major outage or
performance crisis occurs.
If your SLAs are being violated a lot of the time, then it's
the service provider's responsibility to fix the communications between the
SaaS server room and your branch offices. They may need to install a dedicated
comms link, or a cache or WAN performance optimizer system, or a reworking of
some of the SaaS software or tuning of the caching headers. But it's their
problem, not yours, if you've written the SLA correctly.
And that extends to people on the road using the SaaS
package. The SaaS provider can't be expected to handle every lousy last-mile
comms link through a underpowered hotel firewall and a dial-up line made out of
50-year-old twisted pair. BUT the SaaS provider should, at least, provide
decent performance to major nodes in geographic regions where you have a
presence and on Internet service providers that your staff uses.
To keep track of over-the-Internet SaaS performance, you
could set up a few monitoring facilities with short, uncongested links to the Internet
service providers that your mobile staff use, or you could subscribe to a
public service that already has those links and the credibility with service
providers. (Examples are Neustar's Webmetrics
GlobalWatch Ecosystem Management, which includes a specialty service for monitoring
of major SaaS providers, as well as the standard Web transaction performance
monitoring services such as Compuware's Gomez
and Keynote Systems [where I used to
work!] These vendors can also provide on-site monitoring appliances for your branch offices, to evaluate SaaS performance there.)
If performance is poor, the SaaS provider should have some
serious talks with their Internet service provider, and possibly move to
another provider or insist on better peering with the providers that you, their
customer, use.
