Network and Telecom Strategies Blog

UC and Catalyst 2009: Closing Thoughts

posted by: Mark Cortner

I had the opportunity to interact with a broad spectrum of enterprises this week at Burton Group’s Catalyst Conference North America held this past week in San Diego, CA. The majority of my conversations with attendees focused on the topic of UC. I found that many of the discussions were in step with the types of conversations that I frequently with clients related to this subject: what are other enterprises planning and currently implementing to support UC and how should a business navigate the evolving competitive landscape fo UC solutions?

I was also interested to hear how a number of enterprises were giving strong consideration to pursuing approaches to UC that challenge the status quo – specifically the use of software or unified clients to replace the use of wired IP handsets and the interest in cloud-based UC services that can be used to augment (or replace) the premises-based systems and platforms positioned by the major vendors. I don’t predict a rapid shift to ether of these approaches but I believe it’s important to recognize that enterprises are wary that UC is “old wine in new bottles” and as a result may be more open to adopting a less traditional view toward endpoint strategies and service delivery models while transforming their communications and collaboration environments.

 

Building Applications for the WAN

Posted by: Eric Siegel

Our final session at Catalyst this year was a joint presentation between Network and Telecom Strategies and Richard Watson of Burton Group's Application Platform Strategies group. We talked about how programmers and network engineers can work together to improve application performance and problem diagnosis.

This session was based on the NTS report "Designing Applications for the WAN," which will be updated next week -- so go read it when the update comes out (it's out now!), but there are a two points that came out in the session that I think were rather surprising to us network folks and that should be emphasized here.

First, the applications developers are now using frameworks to enable them to develop applications more quickly. Those application frameworks completely hide ("abstract") the network. For example, an application developer will ask a "method" for a "data object." The developer doesn't even realize that the network is involved, or that there may be some SQL queries used by that method to create the data object. That's why when you have a crisis, and you put Wireshark or a Sniffer on the communications link and show the resulting trace to the developer, he or she has no idea what the heck is going on. I'll say it again: the developer didn't even realize an SQL query was generated by the method request, and has no idea whatsoever what that stuff in the query string, trapped by the trace, really means.

No wonder there's frustration and wasted time in diagnosis! The time has arrived for the network organization, the operations group, and the development organization to cooperate and purchase a diagnostic tool that will trace individual transactions through the production system for diagnosis. (An example is Quest Software's PerformaSure; there are others.) The programmers will understand what they're seeing, the network staff won't need to run around doing Wireshark or Sniffer traces for days on end, and the problem will be handled much more quickly.

And, as the second point, realize that performance is poor because of the "abstraction" of the network layer. The application framework has a lot of parameters that set timeouts, window sizes, buffer sizes, compression, whether to disable the Nagle Algorithm, etc. The programmers understand none of this, so they use the defaults. But the defaults aren't usually set by the vendor to ensure optimum performance; they're set to create the fewest trouble calls to the vendor regardless of the network characteristics. That may not be the optimum setting for your particular situation, and that's why the people who install and set up the framework should talk to the network staff to jointly decide how to set the networking parameters for the networks that the framework will be running over. That'll help quite a bit in creating good performance.

TCP stack Performance Tuning for Windows

Posted by: Eric Siegel

I held a Birds of a Feather (BoF) session on WAN performance optimization at Burton Group's Catalyst Conference this week in San Diego, and we had approximately 14 attendees, which was quite good. (Attendance at Catalyst is almost exactly the same as last year: approximately 1500 people! We're VERY happy.)

We started the BoF with a talk by Nick Fiekowsky of Wyeth; he also has a talk coming up on WAN optimization, but the BoF was a great opportunity for us to have a more unstructured discussion (it lasted over an hour) about some of the detailed technical issues.

Nick made some great new observations about TCP performance on Windows; they're very interesting and also seem to have a large impact on performance. So here, with his permission, are some of them:

First, as Nick says, "Latency doesn't end at the RJ-45." When you're computing the required buffer size (receive window, RWIN) for long fat networks (i.e., large bandwidth and large latency), we always think of just the latency on the path and the bandwidth. I hadn't previously thought about the time that the received packet spends inside the receiving station's TCP buffers, waiting to be processed and then waiting to be read by the application. But that's an important factor in the latency; after all, the RWIN can open until there's space in the receive buffer, and the receive buffer won't have space until the application has read something out of it!

So Nick looked at what actually happened, and he discovered that although the bandwidth delay product (and therefore the receive buffer size) was approximately half a megabyte for a particular path, there were still long pauses waiting for acknowledgements to be returned to the sender -- throughput was lower than it could be. He had to increase the receive buffer size to almost THREE MEGABYTES before the sniffer (actually, Wireshark) trace showed that the bandwidth was being fully utilized. Wow.

And that led to Nick's other major point: it's important to optimize Windows performance to keep it from stalling while processing those incoming packets:

• "Non-Paged Pool" (NPP), where the buffers are located, is a precious resource. Nick says that in a 32-bit Windows OS, the NPP = RAM/16. That's not very large, especially if you need large receive buffers. (Look at the Task Manager to see how much of the NPP is being used. Or you can just notice that the TCP stack is requesting a small RWIN when a new connection comes in, because it's under memory pressure.) The only way to increase that is to increase the total RAM in the processor. In the 64-bit Windows OS, the NPP allocation is handled much better, TCP auto-tuning in Vista, Windows 7 and Windows Server 2008 makes better use of NPP, and finally the 64-bit OS also makes the entire system run better because system processes are faster. So the 64-bit OS is a big improvement here.
• Defrag the hard drive to be sure that fragmentation isn't slowing down the processes and the stack.
• Use updated NIC drivers. Nick says that recently he's noticed that there's a new emphasis on performance in the drivers, possibly being driven by iSCSI and datacenter requirements. Also increase the number of transmit and receive descriptors in the driver configuration.

Nick had a lot of other recommendations, such as the big improvement made by ensuring that Web cache expiration tags are properly used, and I'll try to get to them in a later blog.

Meanwhile, thanks, Nick!

Backup for WAN performance optimizers at remote offices

Posted by: Eric Siegel

(This is the first of a few blog entries that I'm going to post to share some ideas that came out of the Birds of a Feather (BoF) session on WAN performance optimization at Burton Group's Catalyst Conference this week in San Diego.)

A primary advantage of WAN performance optimization appliances is that they cut down the bandwidth needs of a remote branch office -- usually by 50% or more. And they reduce response time to make remote file shares, etc., actually usable instead of being a miserable, irritating experience that destroys productivity.

But what happens when the optimizer breaks? Bandwidth requirements spike upwards, and response time instantly becomes lousy.

My suggestions in the past involved QoS in the branch office boundary router to ensure that critical traffic would still flow despite the worsened bandwidth restrictions, and recommendations that the enterprise make arrangements with vendors or resellers to have spares available in-country (to avoid delays in customs and in transit) and configuration information available on a USB flash drive or by downloading as an email attachment. There would be a day or so of worse-than-normal performance as the compression directories are rebuilt, and then things would be OK.

But Nick Fiekowsky, while sharing his slides at the BoF, had a great idea: use the new software client optimizers, such as Blue Coat's Proxy Client, Cisco's WAAS Mobile, Expand Network's Mobile Accelerator Client, and Riverbed's Steelhead Mobile to serve as emergency backups. These software optimizers run on the users' laptops and provide the function of the optimization appliance. They're not as good, because (except for the new Expand Networks Mobile Accelerator Client! More later...) they don't share their compression directories and caches. But they're a lot better than nothing, and they also provide the ability for the user to use acceleration from home or on the road. This is especially great if the software optimizer system licenses are inexpensive or can be turned on and off easily from the central server location, with billing only when they're used.

My updated recommendation is therefore to install the WAN accelerator vendor's mobile client in branch office desktops. If the office's WAN accelerator fails, the desktop clients will step in to reduce WAN traffic and provide an acceptable user experience. It is necessary to have enough license capacity to cover all the client devices that may become active when a single branch office's WAN accelerator fails.

Avaya + Nortel, Not Quite Yet

posted by: Mark Cortner

Last week, Nortel announced that it had entered into a “stalking horse” agreement with Avaya for Nortel’s enterprise solutions business unit for $475M. The stalking horse agreement essentially sets a minimum bid for the auction of the business unit and will be filed with the bankruptcy courts. The auction process allows other qualified bidders to submit higher or otherwise better offers over a 45 day period and then the bankruptcy courts in the United States and Canada will have final approval of the winning bid once the bidding process is complete.

So, at this point, Avaya has not “purchased” Nortel’s enterprise business unit but has rather committed to purchase the business for $475M pending competitive bids. What is the likelihood that Avaya will ultimately “win” Nortel’s enterprise business unit? The same process just recently played out with auction of Nortel’s CDMA and LTE wireless infrastructure businesses. Nokia Siemens Networks entered into a stalking horse agreement with Nortel for these assets in mid-June and today it was announced that Ericsson submitted the winning bid for these assets. The initial stalking horse agreement was for $650M and Ericsson’s winning bid was for $1.13B. The auction process increased the purchase price by roughly 75%.

I’m not sure Nortel’s enterprise business unit will achieve as high a valuation as the wireless infrastructure business. The stalking horse bid for the CDMA and LTE assets was approximately valued at 33% of 2008 revenues and the winning bid was just short of 60% of 2008 revenues. In contrast, Avaya’s stalking horse bid for Nortel’s enterprise business unit is currently valued at roughly 20% of 2008 revenues. Who are other likely potential bidders in the auction? Alcatel-Lucent and Siemens Enterprise Communications could be candidates. Both could benefit from Nortel’s installed base in North America, although I suspect that Nortel’s customers would be uneasy with either. Nortel expects the auction process to be very active; perhaps there are many others that will aggressively bid but I’m not convinced …

Mobile Unified Communication - part 2

Posted by: Paul DeBeasi

Introduction

The growth of mobile phone usage and the availability of desktop unified communication (UC) solutions has created an opportunity for vendors and service providers to introduce products and services that enable mobile users to access many of the same features that previously could only be accessed through a PC or fully-featured desktop IP telephone. These products and services enable enterprises to extend telephony features to their mobile users, while making mobile users more productive regardless of location. In part one of this blog we discussed the capabilities and issues with mobile UC products. In part two, we focus on mobile UC services.

PBX as a service

PBX as a service provides similar features as mobile extension products but are offered as hosted services that enable incoming call routing based on predefined rules. (Mobile extension products, discussed in part one, treat the mobile phone as if it were an enterprise phone extension and simply route calls to the mobile device.) Users sign up for the service, obtain telephone numbers that become their personal phone numbers, and then use web portals to manage how calls are routed.

PBX as a service solutions are telephony-system and phone independent, meaning that they can function in an enterprise environment regardless of the type of telephony system or the type of mobile phone. The value of this approach is that a customer can use a single number for all inbound calls, and the service then gives incoming callers instructions to ensure calls are routed properly to the mobile or desktop phone.

Solutions such as Grasshopper, RingCentral, VirtualPBX, and Onebox are targeted at small and medium sized businesses. They provide enterprise telephony features without the need to buy and maintain equipment and hire IT staff. Calls that use the service consume monthly minutes of use, similar to a mobile cellular plan. The Grasshopper’s high-end “Max” plan provides three toll free numbers, two local numbers and 10,000 minutes of use. This plan would enable 50 employees to consume 200 minutes of use every month.

Managed Mobile UC

Network operators have begun to offer a limited set of mobile UC managed services. Verizon’s Managed Unified Communication and Collaboration service is based on Cisco’s Unified Communications System. British Telecom (BT) has partnered with Cisco, Avaya, and OnRelay to offer several UC services. Orange’s Business Together service provides a managed Microsoft Office Communications Server plan.

Some operators provide special rate plans and call routing for mobile phones. AT&T’s OfficeReach solution can extend an enterprise dialing plan to mobile phones (e.g., four-digit dialing) and can also designate a geographical region (e.g., United States) as a single calling zone. Sprint’s Mobile Integration Service provides unlimited enterprise calling for mobile phones and can route international mobile calls through trunks off the premises-based PBX, saving per-minute international roaming costs.

Many of the network operators offer a mobile extension service (e.g., Verizon PBX mobile extension, Sprint mobile extension, and AT&T mobile extension services). These services extend the enterprises’s PBX capabilities in much the same way that PBX vendors extend their premises systems, except that the network operator manages the solution. The service is a natural extension for enterprises that already use one or more of the operator’s other managed services.

Mobile UC Issues

The challenge with mobile operator services is three-fold. First, the service is under the control of the network operator and many large enterprises will want to fully control a technology as critical as mobile UC. Secondly, recent Burton Group research found that many enterprises view network operators as unresponsive and inflexible. Lastly, many enterprises can save money by operating their own equipment because large enterprises often have the same economies of scale as network operators. 

Conclusion

Mobile unified communications (UC) products and services enable enterprises to extend the benefits of UC to mobile users. Mobile UC capabilities vary widely and are highly dependent on which product, service, and mobile phone the enterprise selects. Enterprises should carefully evaluate mobile UC products and services in order to make informed decisions as to which solutions are most appropriate for them.

Mobile Unified Communication - part 1

Posted by: Paul DeBeasi

Individuals increasingly work from nontraditional office environments and expect to use their mobile phones wherever they work. At the same time that the mobile workforce is growing, enterprises are deploying Internet Protocol (IP) Telephony (IPT) and unified communications (UC) solutions. IPT systems provide new capabilities to virtualize communications across the enterprise, breaking the linkage between a user and a single physical telephone. UC integrates many forms of communication such as email, voice, instant messaging, and presence across various hardware and software platforms and also integrates communication into business applications and processes. Many organizations have begun the process of integrating their various communications applications and services.

Isolated Mobile Users

Unfortunately, the mobile phone user still largely operates outside of this environment. For most people, the mobile telephone is simply a means to make and receive phone calls, nothing more.  Smartphones—such as the iPhone, BlackBerry Storm, or those based on Windows Mobile—may offer additional features such as mobile messaging and calendar synchronization, but they lack the ability to offer mobile users the same access to communications services offered by a desktop phone in their office. In effect, the mobile phone operates outside of the enterprise private branch exchange (PBX) or telephony service.

These mobile users must endure numerous inconveniences that include dependence upon two phones (desktop and mobile), two phone numbers, two voice mailboxes, and two contact directories. When a user is away from their desk, a call to the desktop phone can result in a missed call, voice mail, and caller frustration. Similarly, a call to the mobile phone when the user is busy can result in voice mail, but in a different voice mail system, frustrating both caller and user. In addition, mobile users do not have access to corporate phone directories and they must often use full 10-digit dialing when calling another employee who may only be down the hall.

Emergence of Mobile UC Products

Numerous vendors are introducing products to enable mobile users to access many of the same features and services that previously could only be accessed through a PC or fully-featured desktop IP telephone. The goal is to enable enterprises to extend telephony features to their mobile users, while making mobile users more productive regardless of location. Some products take advantage of the increasing intelligence of the mobile phone, while others insert themselves between the public switched telephone network (PSTN) and users’ various communications services.

Current product offerings in the mobile UC space are widely disparate. Some products provide a high level of integration between mobile phones and enterprise communications applications (such as presence management, voicemail, and conferencing systems), while others are simply designed to give users more options in how they receive incoming calls.

Three well-defined classes of products have emerged, though specific product capabilities within each class vary by vendor (and by mobile platform within a particular vendor’s solution). The three classes are:

• Mobile extension solutions that intercept incoming calls and route them to the preferred device. Mobile extension solutions treat the mobile device as if it were an enterprise phone extension. Examples include Avaya’s Extension to Cellular, Mitel’s Dynamic Extension, and Nortel’s Mobile Extension.
• Mobile client solutions that rely on phone-based clients to enable users to manage their personal communications services and also give the enterprise the ability to remotely manage phones. Mobile client solutions rely on an application server to act as a proxy between the client on the phone and the various enterprise communications applications. Examples include Siemens’ Openscape MobileConnect, Cicso’s Unified Mobile Communicator, and Avaya’s One-X Mobile.
• Mobile Wi-Fi solutions that rely on dual-mode phones (mobile cellular and WiFi) and phone-based clients to enable users to seamlessly roam between wireless LAN (WLAN) and cellular networks. Examples include Agito’s RoamAnywhere, and Divitas’ Mobile Unified Communication.

Mobile UC Issues

Perhaps the biggest issue is the limitations of the products themselves. Mobile extension features are limited to the routing of phone calls; they do not provide for a phone-based client that can access applications such as a visual menu of voicemail messages, nor do they give users the ability to change their presence information via the phone.

Mobile Client and Mobile Wi-Fi solutions generally provide a richer set of features, but capabilities among systems vary greatly and, perhaps more importantly, capabilities will vary based on both handset and service provider. For example, products like RIM’s Mobile Voice System Client only support BlackBerry devices.

What these limitations mean is that it is difficult for an enterprise to deliver a service that supports all handsets that employees are likely to have. Rather, the enterprise can only support a limited number of user devices, meaning that at some level, the enterprise will likely have to dictate policies for mobile telephones. This will either limit user choice or require that users with nonstandard mobile phones obtain new devices.

Conclusion

Mobile unified communication (UC) products offer enterprises the ability to integrate mobile devices with their Internet Protocol Telephony (IPT) and UC systems. These solutions give employees the ability to enjoy many of the same features on their mobile phones that previously were only available on desktop phones and softphones. Products vary greatly among vendors and mobile devices. Over time, mobile UC will become a standard feature of enterprise IPT/UC systems. Enterprises should carefully evaluate mobile UC products in order to make informed decisions as to which solutions to deploy.

Will netbooks replace laptops?

Posted by: Paul DeBeasi

I think that we are seeing the emergence of a new product category that expands the market for mobile computing (i.e., netbooks & laptops coexist) rather than a new product category that replaces the laptop. Netbooks are great for mobile computing and communication but their small disk size, limited internal memory, and relatively slow processing speeds make them underpowered for many users.

Cost conscious enterprise users, that do not have heavy computing needs, will buy netbooks. We will also see enterprises move to data center computing with virtualized operating systems on netbooks as a way to provide greater control over data leakage.

Lastly, mobile power users will own a laptop, netbook, and smartphone. For short trips, the laptop will remain in the office while the netbook/smartphone travels on the road. The netbook will be used for email, web browsing, light word processing and spreadsheet manipulating. The mobile phone will primarily be used voice, IM, email, twitter.

The interesting question is whether or not the smart phone becomes powerful enough that it can replace the netbook. This might occur if projected keyboards and projected screens become popular on the smartphone. However, in the next few years, I don’t see this happening.

Click here for more information on netbooks.

XML Compression

Posted by: Eric Siegel

XML compression came up recently in one of my conversations with a client. It's an interesting topic that a lot of us will need to deal with as XML replaces other forms of process-to-process messaging.

XML is inefficient in its use of bandwidth, a tradeoff that was made to gain ease of use and generality. However, that need for a lot of bandwidth can be a problem, especially on paths with restricted bandwidth or high error rates, such as mobile wireless communications.

Some vendors, such as Stampede and Accelenet (ViaSat/ICT), provide advanced compression solutions that can be extremely effective -- and they also improve the transport protocol, which can help on wireless links. These are quite different from classical "zip" compression. They're proprietary and there are no "standards" involved; you can't pair up compression from two different vendors and expect them to interoperate. However, in exchange for the proprietary nature of these things, you get great compression. These systems maintain compression dictionaries at both end of the path, substituting short pointers for data strings. You can also pre-load these dictionaries with XML phrases; e.g., Stampede's Web 2.0 Performance Series has special features for SOA, including pre-loading of XML compression dictionaries, persistent TCP connections despite the tendency of AJAX (asynchronous Java and XML) to disconnect and reconnect frequently, and extensive security that includes XML schema validation, message anomaly detection, and denial of service attack detection. Indeed, most WAN performance optimization vendors provide advanced compression solutions that can assist with XML.

An alternative is to use binary XML, such as the Efficient XML Interchange format (EXI), which is being specified and evaluated by the Efficient XML Interchange (EXI) Working Group of W3C and is available from vendors such as AgileDelta. AgileDelta says that they provide software that supports standard XML APIs and they can also provide HTTP and TCP Socket proxies to convert XML to Efficient XML (EXI) without any change whatsoever to the application. The software could be used for applications that you can get the vendor to write with EXI; the proxies could be used to convert EXI to standard XML for applications that you don't have control over.

A big question, of course, is whether EXI will become broadly used. Although the EXI standard hasn't been formally approved, it's just one step away from that approval (it's in "Last Call"). The wikipedia article on binary XML has some discussion of these standards -- both EXI and others.

Note, of course, that any encryption or non-standard compression (e.g., proprietary compression, or possibly, EXI) can interfere with network devices that are trying to do "deep packet inspection" of data flows to detect policy violations and attacks, or to route data flows depending on flow contents. And any flow conversion system (compression or protocol acceleration) must usually intercept both flow directions; that can be tricky if flows travel asymmetrically. So use of any encryption, interception, or compression device means you have to think about precisely where you're going to place that device.

My colleague Chris Haddad also says that edge CPU processing speed for XML is important; it can be a problem that binary XML can help. An article referenced by the wikipedia binary XML entry is interesting here: "i-Technology Viewpoint: The Performance Woe of Binary XML" by Jimmy Zhang in SOA World Magazine, August 5, 2008.

New Developments in Telepresence

posted by: Mark Cortner

The telepresence market appears to be maintaining its market momentum and acceptance given numerous vendor and carrier announcements over the last several weeks. The most visible infrastructure vendor in these announcements has been Cisco, a strong proponent of visual collaboration technologies. Cisco has been a driving force in the telepresence market and the combined positioning of telepresence to its enterprise and service provider customers is generated several interesting announcements.

Cisco recently announced that it had successfully demonstrated the industry’s first telepresence session across multiple carrier networks. The demonstration occurred coincident with Cisco Live 2009 and consisted of inter-carrier telepresence connections between seven sites on the AT&T, BT, and Tata Communications networks.  The demonstration verified the technical ability for a business using Cisco’s telepresence solution to utilize telepresence services to collaborate with any other business or location that is also utilizing Cisco’s telepresence solution.

The ability to initiate and reliably conduct a telepresence session across multiple carrier networks will resonate with large global enterprises, as they frequently utilize multiple carriers to provide network connectivity to their worldwide facilities. Furthermore, the ability to conduct an inter-carrier telepresence sessions enables business-to-business or inter-enterprise telepresence usage—effectively expanding the number of addressable telepresence suites for an enterprise and driving more utility out of investments in visual collaboration infrastructure and services. The demonstration by Cisco, AT&T, BT, and Tata Communications is a great first step; ultimately the telepresence infrastructure vendors and carriers need to demonstrate telepresence over multiple networks with connections different third-party telepresence systems – that’s interoperability.