Network and Telecom Strategies Blog

Cisco the Computer Company, Act III (updated!)

Posted by: Eric Siegel

Hey, it looks as if my predictions may be correct! Today the papers are full of articles about how Cisco is "becoming a computer company." (See, for example, The New York Times and The San Francisco Chronicle)

I've blogged before about this before, ("Cisco the Computer Company" and "Cisco the Computer Company, Act II"), but I've always thought they were going to restrict themselves to becoming the intelligent, coordinating backplane for other vendors' computer systems. As I said, they would be "re-creating the Tandem Computers (HP) NonStop backplane in a heterogeneous server environment."

The articles indicate that Cisco may be aiming to take over the whole hairy, smoking boule of silicon 'n software. I suppose that's what happens when you have lots of hungry salesreps to feed.

But I'm not so sure that's what they'd want to do.

Yes, Cisco has a huge opportunity to pull together the world of virtual systems, ensuring that corresponding processes are located close to each other to decrease latency, that messages among processes are easily traceable and measurable, that even non-Cisco equipment can be integrated into an heterogeneous system bound together by communications paths and inter-process messages, that systems can be managed as a whole.

But Cisco has never shown major leadership in systems management; would they be able to rise to the occasion?

And Cisco's own development of its own principal operating system, IOS, has been, uh, difficult. Yes, they've made some real advances in the past few years, but they would be competing against companies who have been focused on OS development for customers who are programmers, not OS development as a partially-hidden piece of an infrastructure, where software tangles can be somewhat hidden.

So I'm going to stick to my original idea: that Cisco is really aiming at becoming the intelligent, coordinating backplane, where programmers could drop an inter-process message into that backplane (for example, with a SOAP header giving the name of the destination process) and be sure that it would be delivered to that process exactly once.

===== 2 February 2009 UPDATE =====

OK, I see that the press is full of news about how Cisco is going to go into the development of virtual servers, so maybe I should look at this again. Cisco's CTO, Padmasree Warrior, has just posted a video about this as a follow up to her her blog entry on 19 Jan 09.

She says that Cisco sees an opportunity in "breaking down the silos between compute, storage, virtualization and network platforms" to create an integrated system, with the network as the platform.  She wants to break down the "islands of virtulization."

Padmasree says that "phase two, or unified fabric, is the phase which optimizes and extends data center technologies through consolidation of virtualization across the network, storage, and server application. Phase three, or unified computing, virtualizes the entire data center through a pre-integrated architecture..."

How would this be done technically? Well, whenever a new process is created in a VM, it could be packaged inside a software shell that would automatically provide a higher-level set of session-level protocol headers giving the guaranteed identity of the sender, the precise time the message is sent, and a unique transaction and message ID. As I said, we're back to the Tandem Computers NonStop backplane, with Cisco as the unifying part of the ecosystem. As Padmasree says, "the network is the platform."

Network and Systems Management: VMess

Posted by: Eric Siegel

OK, now I'm worrying about how we're going to manage VMs without losing our minds.

You see, I've been hearing great stories about how there are going to be huge racks of servers containing zillions of VMs: virtual servers, virtual switches, virtual routers, virtual firewalls, virtual performance optimization appliances, virtual everything! And the system will instantly adapt to changing workloads, spawning new, fully-configured servers with their complete environment, along with switches, routers, whatever.

And if a newly-spawned virtual server is too many virtual-router hops away from closely-coupled applications or processes, well, then, that new virtual server will be automatically re-provisioned and re-configured closer to where it should be for minimum latency!

Yes, the entire mass of virtual whatevers is going to be in constant, thrilling motion! Everything is going to be moving around, minute by minute! Total efficiency in hardware utilization! Minimum transport latency! Automation! I just can't wait! It's going to be just totally wonderful!

Until it breaks.

And when it does break (and it will), I don't want to be the miserable technician trying to figure out where everything was when that transaction failed, trying to track down precisely what subtle interaction of data flow, firewall, optimization device, VLAN, switch... you get the picture... was the source of the difficulty.

It'll be like yelling "STOP!" six hours too late at a frenzied set of hyperactive squirrels loaded on caffeine and springing among the trees while chattering at each other. Now, where were they six hours ago, when the problem occurred? All the actors in the drama will have melted into thin air. I'll bet the guilty one won't even be there any more; he appeared, messed something up, and then vanished into the virtual fabric, leaving just the rack behind.

What will the vendors give us? Probably only the assurance that the virtual servers and switches and routers will have virtual MIBs, just like the real ones. But I know where the real ones are, and I can put my hand on them and monitor them. What happens if a virtual switch appears, switches a gigabyte or two, and then vanishes -- poof! -- inside of a minute? Where did it go? What did it do while it was alive? We'll be MIB-less in the fabric, without any vision.

I can already see a rule I'd like to impose on this mess, which will tie in with the "triage-based incident management" discussed in my post "Network and Systems Management: Over the Edge." It's this:

"Just because a production group can create a maze of unmeasured real and virtual servers, network components, and interconnections that change from one minute to the next is no reason it should. Edsger Dijkstra wrote a famous letter to the editor titled 'GO TO Considered Harmful' (Communications of the ACM, August 1968) in which he made the case for more program structure and less intertwined process flow. The same applies to IT infrastructure." [This is from my new Burton Group paper, "Using Network Management Platforms for Triage-Based Incident Management."]

I'll bet we network folks are going to be stuck in the middle of this VMess, and a miserable place it will be. We'd better start figuring out how we're going to handle it when incidents occur. And I think that triage orientation, with clearly-defined diagnostic domains and rules to ensure that the virtual servers providing a service are all grouped within the same domain, where we can evaluate them as a group with active measurement techniques, is a good place to start.

Cisco the Computer Company, Act II

Posted by: Eric Siegel

Yes! Cisco is indeed becoming a computer company, as I originally blogged a year ago in "Cisco the Computer Company." (Go look at it!) A little while ago (sorry, late blog entry), I went to the annual Cisco analyst's conference, and it was full of further progress towards making the network into the backplane for a distributed multi-server computer system -- in effect, they're re-creating the Tandem Computers (HP) Integrity NonStop backplane in a heterogeneous server environment.

Just look at what they're doing or planning to do:

• Lossless, extremely-low-latency transport ("lossless Ethernet" and cut-through switching, similar to NonStop's ServerNet low-latency inter-processor buses with "wormhole" switching)
• Positive identification of network users and of individual transactions, along with accurate timestamps. (Great for system audit and diagnosis in a message-based distributed server system)
• Standard API for talking to the network, asking for specific services and learning the identities, locations, and more about other network users.

With a secure, reliable "backplane" of the Cisco network, other manufacturers' servers and appliances could create a mesh of message-based services. And Cisco would provide the common, platform-independent transport API, integrating them all into one high-performance, secure fabric and, at the same time, handling data compression, transport performance optimization, and the measurement and diagnostic taps and tools needed to trace the performance of individual transactions or other interactions through the production system.

The big question, of course, is how this integrates with OS efforts from the computer system vendors, such as VMware and Microsoft, and with transaction monitors. Where is the line between the reliable, secure, "deliver the message exactly once" transport, and the higher-level OS functions? I'm very interested to see how this evolves!

Cisco the Computer Company

posted by: Eric Siegel

Let's take a few moments and think about what Cisco is doing with its AON (Application-Oriented Networking).

Cisco has already sold a router to every organization, enterprise, man, woman, child, and household pet in the entire known universe. Now its hungry army of salespeople need new markets, and "moving up the protocol stack" into the world of applications is an obvious direction.

AON has the potential to be much more than a quick move into a few hot applications markets, however. There's some fascinating technological history and concepts behind Cisco's recent moves. It seems to me that Cisco is beginning to re-create the Tandem Computers message-based operating system, Guardian, in a heterogeneous world.

The Tandem Computers system, which is still very successful and is sold by HP as the "NonStop" computer system, is a loosely-coupled, linearly-scalable, fault-tolerant highly-integrated set of hardware, OS, transaction monitor, and SQL database. It was originally designed for transaction processing but is now expanding into other areas that can use massive scalability and fault-tolerance with a UNIX and SQL programmer interface. One of the keys to its success is its message-based OS kernel, which guarantees to deliver any message exactly one time to its destination and to return a reply exactly one time to the message's originator. The message destination is a process name (which can be an application process or a systems process, such as a database or communications link), and there can be dozens, or hundreds, of identical processes in different processors sharing the same name and therefore sharing the workload. The OS distributes the workload over the processes, and it also handles recovery from failed processes. If a process hasn't successfully returned a reply to a message's originator, the OS automatically ensures that another process with the same name gets the work and that there isn't any database corruption. (I worked at Tandem Computers many years ago, before it became part of Compaq and then part of HP.)

Let's see what Cisco is doing, and compare it to the NonStop systems.

First, we need a message-based set of standards for applications to talk to one another. What a surprise! It seems that XML and SOAP are already standards, and they're already widely adopted. They can be used to send messages, and they have standardized internal metadata that can be used to find the information and context for load balancing. A load balancer (or Cisco device!) can look for a few key internal message fields for choosing a server; there's no need to understand everything in the message.

With XML and SOAP, it's easy build a general-purpose message switch that looks for a few internal fields. Note the contrast with raw TCP/IP, which is a stream of unformatted data. Doing the same thing with raw TCP/IP would require that the organization build its own custom internal format and program its own message switches to handle that format.

Second, we need to load-balance among multiple members of the same class. This capability is already provided by WCCP Version 2 and other load-balancing systems, especially for context-free messages, which is what we're talking about here.

Third, we need to provide the ability to return the reply to the originator of the message. Again, a Cisco device could easily do that.

And fourth, we need to guarantee delivery exactly once. This is more complex, but not insurmountable. Using SOAP header fields, the Cisco devices could embed a guaranteed-unique message identifier that Cisco devices would use to check that each message is actually delivered once, to suppress unwanted duplicates, and to re-deliver messages in a controlled way if the initial server fails before generating a reply.

There's more that should be done, such as the integration of a SQL database and transaction commit/rollback capabilities, which would provide fault-tolerant updates of a database despite server failure. But, clearly, it's possible for Cisco to start building the equivalent of the HP NonStop kernel on Cisco equipment.

And look at the result: Cisco would own the infrastructure underlying a massive distributed-processing system, which could use processors and operating systems from many different vendors. They'd need to use XML and SOAP, but they're already doing that! There would be a requirement that applications be context free and that replies repeat the transaction identifier, but that's minor. And many systems are already heading in that direction.

Cisco would be perfectly positioned to provide more than guaranteed-delivery and load balancing. It could also provide QoS prioritization and classification, security authentication and encryption, and -- because it would then know the security keys -- it could also provide detailed measurement, diagnostic transaction tracing (with synchronized clocks and records of the transaction IDs, of course!), logging, WAN performance optimization, and XML message and protocol transformation.

Cisco wouldn't need to get involved in the complexities of the operating systems and applications, although it would be sensible to build, or standardize on, a distributed-architecture SQL database and transaction monitor. Cisco could leave applications processors, operating systems, and applications to existing vendors.

And so, after enumerating all of these thoughts, I asked John Chambers at the Cisco Networkers conference if he was remaking Cisco as a distributed-computer company.

His answer? "Yes."