Network and Telecom Strategies Blog

SaaS network diagnostics

Posted by: Eric Siegel

And while I'm on a SaaS chain-of-thought, one final discussion for the day: diagnostics.

What will you do when users at a branch office or out in the field call and complain about SaaS performance? Your operations center needs a structured checklist to follow to quickly manage the incident and isolate it to the responsible party -- a process known as "triage." (See, for example, the Burton Group report "A Framework for Network Incident Management.")

Here's where your SLA key performance indicators, or service level indicators (SLIs) can be useful. Your SaaS provider should be giving you on-line access to their performance as seen by users in their server room, and that's clearly the first checkpoint for the operations staff. If that's in trouble, well, the problem is easy to locate. But if that's OK, then the next step is to look at your service level indicators at your branch offices, especially the branch office that's reporting trouble. If that indicator is in trouble (and that indicator may be a full SaaS web page download time, or it may be only the TCP round-trip time and DNS lookup time for the SaaS provider), then, if you have a solid SLA, you should be able to send that evidence to the SaaS provider and insist on their opening a trouble ticket. It's not really your job to diagnose the problems between your branch office and the SaaS server room; it's the SaaS provider's job. And they probably have more leverage with their Internet service provider (ISP) than you do; they can even demand better peering with the ISP that your branch office is using.

Of course, it may turn out that it's your ISP having internal problems. In that case, you and the SaaS provider may need to explore new ways of connecting to the SaaS server site, through better peering at a point that doesn't encounter the congestion within your ISP, by a direct link to your major offices outside of the Internet, or by your moving to a different ISP for your own access (the threat of doing that might inspire your ISP to upgrade its systems).

Or it may be that it's your user who is having problems with his own browser or with the branch office's internal network; you'll know that because your service level indicators won't see any problems. In that case, it's an incident to be handled by your service desk; there's no need to file a trouble ticket with the SaaS provider. (And your relationship and credibility with your SaaS provider will improve when you're not filing minor reports that turn out to be inside your own system!)

WAN Performance Optimization for SaaS

Posted by: Eric Siegel

While I'm thinking about Software as a Service (SaaS) today, how about WAN Performance Optimization for SaaS?

Some SaaS websites are well-designed for the Web and perform well over poor bandwidth and high latency, but others don't, or they require the upload/download of large files. For those situations, WAN performance optimization can make the difference between usable and useless. (See, for example, my blog entry "Be a Hero! Cut costs to Zero!" for a discussion of optimization.)

Indeed, if you've created performance indicators in your service level agreement (SLA) with the SaaS provider, they may be more than happy to work with you to install WAN performance optimization between their server sites and your users. (See my blog entry "SaaS network performance SLAs.")

So. How should you install WAN optimization? There are a few things to keep in mind as you discuss this with your SaaS provider.

First, if your SaaS provider wants to use a double-ended appliance system (i.e., an optimizer appliance at the SaaS server site and an optimizer appliance at your site), how are you going to configure the device and integrate it into your branch offices? What happens if you have more than one SaaS provider? Will there be a tall, tottering stack of WAN optimizers from different vendors, and a maze of Ethernet cables plugging everything together? If the "appliance" at the branch office is a software application, how will you load it into your branch office server, and will it conflict with other software?

Second, if your SaaS provider wants to use a single-ended system (i.e., an optimizer appliance at the SaaS server site communicating with ActiveX, Java, or JavaScript running in a browser), then you have to be sure that your staffs' browsers will accept the plug-in, if necessary, and that it won't interfere with anything else.

Third, even if your SaaS provider doesn't want to install WAN optimization, you may still be able to improve performance with a branch office cache that will store commonly-downloaded files for use by all browsers in the office. Many branch office optimizers -- such as those that you might be using for communications with your central office -- can provide caches, as can Web traffic filtering devices, such as Blue Coat's ProxySG. Of course, there are also free caches, such as Squid; they can run on an inexpensive Unix or Windows box and make a big difference if you have a lot of users accessing the same set of Web files.

SaaS network performance SLAs

Posted by: Eric Siegel

Software as a Service (SaaS) is tempting a lot of enterprises, but I worry that the service level agreements (SLAs) for these services may be incomplete.

It's important to define precisely what the service provider means by availability and performance. Is it measured inside the provider's server center? If so, then that's not particularly useful, unless your salesreps and other employees typically have their meetings there!

What you should want is a set of metrics to the locations where your staff does their work; e.g., inside your branch offices or on the road, across the Internet. Make the SaaS provider responsible for ensuring that communications is acceptable to the boundary of your office or to major Internet access points.

You can set up measurements at your branch office boundary router (to ensure that the provider isn't held responsible for problems inside the branch office network) and measure round-trip latency, availability, and possibly the home page download time for the SaaS provider from that point. (Coupled with a server-room measurement from the provider of the SaaS service, you will probably need only DNS resolution time to the SaaS site, availability of the SaaS servers, and round-trip TCP latency to the SaaS servers, unless the SaaS design is such that multiple server rooms are involved.)

Be sure that both you and the SaaS provider can see the metrics in real time and agree on the technology involved; the goal is to have the provider notice that the SLA metrics are in trouble and fix the underlying problem before a major outage or performance crisis occurs.

If your SLAs are being violated a lot of the time, then it's the service provider's responsibility to fix the communications between the SaaS server room and your branch offices. They may need to install a dedicated comms link, or a cache or WAN performance optimizer system, or a reworking of some of the SaaS software or tuning of the caching headers. But it's their problem, not yours, if you've written the SLA correctly.

And that extends to people on the road using the SaaS package. The SaaS provider can't be expected to handle every lousy last-mile comms link through a underpowered hotel firewall and a dial-up line made out of 50-year-old twisted pair. BUT the SaaS provider should, at least, provide decent performance to major nodes in geographic regions where you have a presence and on Internet service providers that your staff uses.

To keep track of over-the-Internet SaaS performance, you could set up a few monitoring facilities with short, uncongested links to the Internet service providers that your mobile staff use, or you could subscribe to a public service that already has those links and the credibility with service providers. (Examples are Neustar's Webmetrics GlobalWatch Ecosystem Management, which includes a specialty service for monitoring of major SaaS providers, as well as the standard Web transaction performance monitoring services such as Compuware's Gomez and Keynote Systems [where I used to work!] These vendors can also provide on-site monitoring appliances for your branch offices, to evaluate SaaS performance there.)

If performance is poor, the SaaS provider should have some serious talks with their Internet service provider, and possibly move to another provider or insist on better peering with the providers that you, their customer, use.

BT + Cisco Deliver Cloud-Based Telephony

posted by: Mark Cortner

The viability of cloud-based UC took a significant step forward earlier this month with an announcement by BT and Cisco on the launch of a global cloud-based IPT service from BT’s Onevoice UCC portfolio. The hosted IPT solution will be the first global offering based on Cisco Hosted UC Services and is currently available in the UK. The offering will be available in the USA and EMEA in 2010 followed by Asia Pacific. BT is working with Cisco to build the virtualized infrastructure and leverage its cloud computing technologies to deliver a cloud-based UC service. The relationship between virtualization of telephony applications and cloud-based UC was highlighted in a previous blog post; BT and Cisco have taken virtualization to the next level to deliver a cloud-based UC solution including hosted IPT.

BT’s service offering will enable businesses to access services using a utility-based pricing model; the cloud-based services now include telephony, voice mail, unified messaging, and conferencing. The service is delivered over BT’s MPLS network service and can be blend the cloud-based telephony services with existing legacy TDM and IP voice infrastructures—enabling enterprises to migrate to cloud-based IPT services based on their business requirements and in consideration of existing investments. I look for BT to offer solid a SLA with these services, anticipate availability metrics between 99.9 to 99.99%, and to leverage the use of SIP-based connectivity introduced earlier this year. The ability to enable the cloud-based IPT solution to “integrate” with existing legacy TDM and IP voice infrastructure will be interesting to watch, and the terms “integrate” and “interoperate” often hold different meanings to service providers and customers. Furthermore, although large enterprise customers will appreciate the ability to support blended telephony solutions, they will also seek the ability to support hybrid unified messaging and collaborative application environments … the first step has been taken.

Congestion Going Into Network Branches = Packet Loss

Posted by: Eric Siegel

End of the month, and I've been going over the list of the "dialogues" I had with Burton Group clients to see if there's something that might be of general interest.

So I uncovered one that I participated in with my colleague Ken Agress of Burton Group's consulting organization, and it's interesting because I think that the topic of QoS congestion on links into remote branches isn't given enough consideration in QoS network design.

The basic symptom of the problem was that despite QoS on the routers, there was excessive packet loss on high-priority flows into the branches on a hub-and-spoke network design.

But that's because the server-room router was trying to push too much data into the network cloud for a specific branch office. The discards weren't occurring at the server room boundary router into the cloud, because the fat access link from the server room will easily accommodate lots of traffic to the branch, especially if the other branches aren't receiving much traffic at that time.

If you have a large bandwidth access link into a cloud, but just a small bandwidth access link out of the cloud to a branch office destination, the cloud will need to discard data packets on high-bandwidth flows or during bursts, because the cloud itself (i.e., the backbone routers) has minimal buffering.

True, a single TCP flow will self-adjust (by "self-clocking") to a bandwidth-restricted exit from the cloud because its acknowledgements will be delayed, but if there are a lot of parallel flows, or there are non-flow-controlled UDP flows, you'll be in trouble. TCP flows will get the "global synchronization problem," in which all TCP flows ramp up in bandwidth concurrently, lose packets concurrently, reset their flow rates downwards concurrently, and then repeat. RED or WRED or FRED mechanisms can help avoid global synchronization, but you'll still lose packets.

So, how should this be handled? If you own the cloud or can signal to the cloud's administrator (e.g., by setting DSCP fields or EXP tags on packets going into a MPLS cloud), then the cloud can pick which packets to discard if the exit bandwidth isn't sufficient at the branch office. And the tags will, we hope, tell the cloud to discard packets from low-importance flows. RED / WRED / FRED on those flows will then get those low-importance flows to cut their bandwidth use. Those low-importance flows will be losing packets and will be miserable, but the higher-importance flows will get through.

A better idea, we think, is to avoid letting the cloud do your packet discards -- especially if you don't actually own/administer the cloud directly. Don't give it more traffic than it can handle; use traffic shaping on subinterface definitions to ensure that you never feed more data into the cloud than can exit at the destination. You will probably do a better job of discriminating among flow types and of smoothing brief traffic spikes (e.g., your routers probably have larger buffers than the cloud's routers have).

For example, see the following Cisco documents:

href="http://www.cisco.com/en/US/docs/ios/qos/configuration/guide/reg_pkt_flow_shaping.html" target=_blank>Regulating Packet Flow Using Traffic Shaping

href="http://www.cisco.com/en/US/tech/tk543/tk545/technologies_tech_note09186a0080114326.shtml" target=_blank>Applying QoS Features to Ethernet Subinterfaces

Or you could install a Blue Coat PacketShaper in the server room's exit path to do the traffic shaping for both TCP and UDP. And the new PacketShapers can also do compression.

UC and Virtualization

posted by: Mark Cortner

The recent announcement from Avaya related to its new virtualized UC offering is an interesting development for the UC industry. The single-server UC solution is based on Avaya’s Aura architecture which was introduced earlier this year. In early 2008, Mitel announced a single-server solution that signified the first application of virtualization technology to enterprise real-time communications applications.

Avaya’s initial virtualization solution, Aura System Platform, is currently targeted to the midsize market with a maximum scalability of 2400 users. The solution includes the core components within Avaya’s Aura architecture—Communications Manager, Voice Messaging, SIP Enablement Services, Application Enablement Services, Utility Services, and Media Services deployed on a single server.

I fully expect more announcements soon from the UC industry that take advantage of virtualization for not only asynchronous applications but also real-time communications and collaboration applications. I also expect the scalability of virtualized UC solutions to rapidly increase; anticipate that single-server offerings will evolve to effectively address enterprise requirements within the next year … and one step closer to cloud-based UC.

UC and the Cloud: Microsoft BPOS

posted by: Mark Cortner

The industry focus on the delivering cloud-based UC solutions continued to expand this week as Microsoft announced the commercial availability of its Business Productivity Online Suite (BPOS) in 20 countries worldwide. Microsoft’s BPOS is delivered via subscription services hosted by Microsoft and consists of Exchange Hosted Services, SharePoint Online, Exchange Online, Office Live Meeting, and Office Communications Online. BPOS utilizes global datacenters located in the USA, Dublin, Singapore, Amsterdam, and Hong Kong to deliver services to customers located in Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Ireland, Italy, Japan, Netherlands, New Zealand, Norway, Portugal, Spain, Sweden, Switzerland, UK, and the USA.

With particular respect to UC, the Office Communications Online service includes business-class instant messaging (IM) and presence services, peer-to-peer voice and video services with integrated Office, Exchange, SharePoint user experiences.  The service is delivered with a 99.9% service level agreement and boasts 9 levels of data security and is CyberTrust certified.

In addition to formally announcing the commercial availability of BPOS, Microsoft announced that it has signed a reseller agreement and a go-to-market agreement with EDS, an HP company. The announcement estimated the revenue potential for both companies to exceed $3B in cloud and professional services.

The initial market traction for these services will likely be small to mid-sized businesses, although the potential for enterprises to leverage the software-as-a-service (SaaS) model to augment premises-based UC platform strategies will appeal to many organizations and will factor into future service delivery strategies. The focus for cloud-based UC services today is on non-real time communications applications; how suites such as BPOS can integrate with enterprise telephony applications and how closely they can replicate the functional experience of premises-based platforms will determine their viability with large enterprise customers.