Answering Terry Slattery

A couple of days ago, Brad Reese over at Network World's Cisco Subnet blogged about the VC3 release. The posting did a good job of talking about Vyatta and some of the advantages we have (along with a nice highlight of the Dear John web page).

Brad then asked Terry Slattery, CTO of Netcordia, to comment on Vyatta and the announcement. Terry did so in his blog at the Netcordia web site.

In response, I sent a note to both Brad and Terry with some of my thoughts on the points that Terry raised. Now, it's important to remember that both Brad and Terry are hardcore Cisco alums. Brad was instrumental in putting together the Cisco training and education program. Terry worked on some fundamental IOS features around things like the CLI. (Remember tab-completion in IOS? That was his.)

So, here was my response:

Dear Brad,

I recently read your article quoting some of Terry Slattery's blog post about Vyatta. Terry is clearly an educated skeptic, which is wise when it comes to networking. There are many good answers to Terry's questions, however, and I felt compelled to provide them in order to clear the air and help others who may be asking themselves the same thing.

First, let me say that often debates of this nature get turned into all-or-nothing contests where the participants want to suggest that their particular point of view will reign supreme and the opposing point of view is utterly bankrupt. I don't believe this, and I don't think Terry does either. Like it or not, the world is gray. When this debate is long over, neither Cisco nor Vyatta will have 100% market share. And Vyatta doesn't need 100% share in order to be a success to our customers and the market.

As a background to this, I'll say that Vyatta has customers in just about every market segment you can think of. A partial list includes small business, medium enterprise, hosting service providers, VoIP providers, wireless service providers, federal and municipal government, financial, educational, media, retail, manufacturing, legal, health care, and aerospace. Some of these customers are documented in case studies on the Vyatta web site. In certain cases, we're in branch offices, and other times at the cores of networks.

Okay, let's get to Terry's comments...

"For small remote site use, it may well be acceptable to use the Vyatta router, provided you don't also need a local switch and voice capability, which the ISR can provide."

Let's parse this out. First, it depends what you call "small." It's a frequent tactic in technology arguments to suggest that something would only work for somebody's grandmother, but not in "real life." (The proprietary operating system vendors used to claim this about Linux, for instance.) This is absolutely not the case with Vyatta.

While it's true that there are features in the ISR that are not (yet) present in Vyatta, not all customers need those. Many customers prefer to keep their voice infrastructure products, even if VoIP-based, separate than their data infrastructure products--the network still carries the traffic, but the products are non-integrated. This is not to suggest that integration is not valued by some, only that it is not valued by all. Vyatta is perfectly happy today addressing the part of the market that doesn't need all the ISR widgets and doodads. We'll add the widgets and doodads over time, too, and we'll have a better solution for that part of the market.

Second, I'm a bit flummoxed about the cost structure of the integrated switching modules for the ISR. Looking at the web site of a large online merchant, I can buy an NM-16ESW, 16-port Fast Ethernet switching module for the ISR, for $1076. A Cisco Catalyst 2960-24TT, 24-port Fast Ethernet switch with two Gigabit Ethernet uplinks, only costs $856. And the 2960 is a top-shelf option. If I'm willing to forgo the Cisco logo, I can get a Netgear JFS524, 24-port Fast Ethernet switch for only $99. While Cisco charges 10x the price of the Netgear for the ISR module, it makes up for it by giving you fewer ports... Okay, so that's not a fair comparison. How about, while Cisco charges 25% more for the NM-16ESW than the 2960, it makes up for it by giving you fewer ports.

"If your interface is a set of T1s or similar speed links, then software based forwarding will work well (higher speeds are possible, depending on the hardware you use, as demonstrated in the Tolly Group comparison). At higher speeds on bigger boxes, Cisco will win -- it's simply a game of moving packets between interfaces at the highest speed the hardware will enable. And Cisco has the hardware at the high end."

It's important to place Vyatta in context. A lot of people suggest that Vyatta is a toy because it can't route for the backbone of the Internet. It's true that Vyatta won't be used by a tier-1 carrier to handle the Internet backbone anytime soon, but this was never a claim Vyatta made. We have been very clear about our performance claims and have published a lot of data on our web site substantiating our claims. (As a side note, it's a lot harder to lie about performance claims when you're an open source vendor and people can simply download the code and test it for themselves. It's much easier to bluff on performance issues when testing a claim costs thousands of dollars to buy the device under test.)

It's also important to note that many Cisco mid-range routers are software-based. The whole ISR series, in fact, routes packets on a MIPS processor. The Cisco 7200 is also software-based.

At the end of the day, the proof is in the pudding, and Vyatta's pudding tastes pretty good. The Tolly test actually demonstrates that Vyatta works well even for large links, far exceeding T1 speeds. In the test, Vyatta showed a minimum forwarding performance at 64-byte packets of approximately 280 Mbps. This was double the performance of the 2821. The maximum performance was over 2 Gbps at larger packet sizes (greater than 1024-bytes). And that performance was limited in this case by the two Gigabit Ethernet ports. Had we added more ports, we could have gone even faster. We didn't do this because Cisco charges over $2,500 (street) for the 1-port Gigabit Ethernet card for the ISR series (HWIC-1GE). It simply wasn't cost effective to go further.

Finally, this test was performed with on a 2.8 GHz Celeron processor--something that is basically obsolete at this point in time. Faster processors will deliver better performance, at a consequent increase in cost.

This is not to suggest that ASIC-based forwarding doesn't have its place. Certainly, ASICs can provide a huge increase in speed (along with a huge decrease in flexibility). If you're trying to route traffic at terabits per second in the Internet core, you'll need ASICs to do it, no doubt about it. But many (most?) people aren't trying to do that. Which is one reason that Cisco 2800 ISRs outsell CRS-1s in volume.

So, to summarize, while ASICs are a great match for anything running at more than 10 Gbps, many networking systems operate at much lower speeds. This includes anything in the Cisco 1800 to Cisco 7200 range. If you're buying these products, Vyatta is a good match and can probably save you a lot of money. We have had customers replace every Cisco model in this range with a Vyatta solution and we're confident that we can solve many problems in this space.

"Some of the other features that I didn't see listed in about 30 minutes of poking around the Vyatta web site and forums: QoS, MPLS, and Netflow/sFlow/IPFIX."

You didn't need to poke around that long. Vyatta has a wiki page with a complete list of features that we don't yet support that people have told us they need: http://www.vyatta.com/twiki/bin/view/Community/TopEnhancements We're committed to being transparent about what we have and don't have and allowing the community to help prioritize where we go moving forward. On this page, you're welcome to vote for features you think are important.

So, yes, Vyatta doesn't (yet) include those features and a whole lot of other ones. That said, QoS is currently scheduled to be released in 1Q08. MPLS and Netflow have been highly requested but are currently unscheduled.

"The question in my mind is whether they will be able to implement the features that customers need in order to be competitive with the likes of Cisco and the other router vendors (don't forget about 3Com, Adtran, and the other smaller router players)."

Pulling a paraphrased quote from another day and time, "The question in my mind is whether Linux will be able to implement the features that customers need in order to be competitive with the likes of Windows, Solaris, HP-UX, AIX, and a bunch of other smaller UNIXes." I'd just submit by analogy that open source has proved itself very capable of competing with proprietary offerings over time. While you may be correct that Vyatta can't meet a particular customer need today, open source adapts quickly--it's sort of the Borg of the technology universe. Would you honestly want to bet against open source in the long run? If the answer is no, then perhaps the only debate is about timing, and that's where I'd remind you that customers, like the world, are not black and white. They all have different needs and desires.

"I'm curious just how big a network a single Vyatta can handle. Leave a comment if you know of any big ones."

Vyatta can handle fairly large networks. Try us. We have multiple customers running full tables with more than 10 BGP peers. In fact, given that Cisco often charges $5000 per GB of memory and Vyatta leverages the x86 ecosystem to provide a GB of memory for less than $100, Vyatta customers can handle large networks without breaking the bank. We have customers that have purchased more than 100 systems and we have established the partnerships with Hyperic and Alterpoint to help manage networks this large and larger.

"* The Tolly Group comparison was done with UDP packets, which would may not take advantage of a cache that the Cisco might use to improve the performance of TCP. It would be interesting to see the same test done using a set of TCP flows."

The Tolly test was a standard RFC 2544 performance test. In fact, most packet forwarding tests are run with UDP rather than TCP because it makes it easier to set up test equipment (a Spirent SmartBits in this case). I didn't realize that Cisco has problems with UDP. I don't think that using TCP would have changed the outcome, but perhaps it would. If somebody has data showing that Cisco has a UDP deficiency, we'd be happy to look at it.

Cheers,

-- Dave Roberts
Vice President of Strategy and Marketing
Vyatta, Inc.