It's all about ASICs, right?

Every so often, Vyatta gets queries from potential users asking about Vyatta technology. One of those came in to our info-line this morning, and it's a frequently-asked-question, so I answered it with a lengthy reply and I'm going to post the question and answer here as well.

Fundamentally, the question is about the special mojo that Cisco must put into its products, right?

I’ve worked with almost all OS’s you can find and most devices as well.

I’ve read the articles on the OpenSource vs. Cisco, and to be honest I’ve known for a while now that most linux/unix os’s with some tweaking can do the same as most Cisco routers/switches.

But I also know that a lot of development not only goes into the os(source) of the devices, but also the device hardware which is specifically developed for routing/switching.

I would like to know will normal proprietary over the counter hardware match up to the Cisco device hardware?

Brandon

Here's my lengthy answer. I might have given Brandon more than he was looking for, but I figured it would pay dividends if I also blogged it.

Brandon,

The answer to your question is complex, but let me take a shot at breaking it down. Apologies for the long response, but there simply isn't a quick answer to your question.

The simple answer is that it depends highly on which market segment you're looking at.

First, let's separate switches and other appliances (routers, firewalls, etc.). All switches are ASIC-based and are definitely tuned to process packets as quickly as possible. On the other hand, they don't really do much processing other than basic forwarding and possibly some queuing for QoS. The ASICs are tuned for the forwarding task but they can't do much else, and there is typically only a small, embedded processor handling the management interface. It's incapable of doing a lot of heavy-lifting. The operating system in a switch has little to nothing to do with its performance because it's just running the management interface. In spite of that, many switches run on a standard operating system. For instance, Extreme Networks switches run Linux. So, to sum this up, you aren't going to terminate a VPN on your switch, but if you want 48-ports of cost-effective, wire-speed Gigabit Ethernet forwarding performance, switches are just the thing.

Now, let's take a look at appliances. Many of the appliances that are out on the market are based on standard operating systems (Linux or FreeBSD) under the hood, and many are implemented using standard PC hardware. For instance, Fortinet UTM boxes, Juniper J-Series routers, and Riverbed WAN optimization boxes are all based on PCs. Fortinet runs Linux under the hood. Juniper's JunOS is based on FreeBSD. Riverbed uses Linux.

Cisco's IOS is based on a custom, embedded OS because Linux didn't exist back when it was first created and the Unix systems of the day were too resource hungry for the cost structure that they wanted to achieve. While Cisco's low-end hardware is not based on PC platforms, it is not based on much specialized hardware. The ISR series uses a chip from SiByte that contains 4x MIPS architecture cores. So even the ISR is really a software-based device. Further, these cores are underpowered today and are not as fast as off-the-shelf processors. If Cisco were to start over, with a clean slate, I think they would probably make completely different architectural choices. In fact, they are moving this direction, slowly. The recently announced Cisco ASR, while it uses custom silicon, runs IOS on Linux. When Juniper had the chance to start from a clean slate, they chose FreeBSD as the foundation for JunOS and used x86 chips as the CPU for all the Juniper route processor modules.

Now, what you say is definitely true in the high-end of the networking market. Cisco's CRS-1 and Juniper's M and T series routers all use ASICs to perform high-speed forwarding. In this sense, they are more like switches. As with switches, they are very fast, but the processing they perform is not very rich. That's okay for where they sit in the network. The core of the Internet needs to be fast; it doesn't need to be terminating VPNs, doing load balancing, or executing IDS code.

Other parts of the network are different (say the branch office, the edge of service provider networks, or in data centers in front of individual applications), which is why you find many of the appliances that sit there running on Linux/BSD on x86 processors.

So, to summarize, switches are definitely ASIC-based and the operating system has nothing to do with their performance. Some switches (Extreme), even run Linux. In the high-end of the routing space (Cisco GSR and CRS-1; Juniper M and T series), the insides are more like a switch than a router. Like a switch, the forwarding is all handled by ASICs. Still, JunOS is based on FreeBSD. Finally, in the mid-range of the appliance market (routers, firewalls, VPNs, load balancers, etc.), many devices are implemented using standard operating systems running on x86. This is the market place that Vyatta is addressing with our products.

Finally, I'd point out that even if it was true that Cisco's ISR family was implemented using special ASICs and secret networking mojo, the results speak for themselves. The fact is, Vyatta has outperformed the Cisco ISR and 7200 with standard Intel processors at a fraction of the cost. If those Cisco products were using ASICs, that evidentally didn't matter very much in the final analysis. All the custom silicon and engineering did was raise the costs to develop those products, costs that were then passed on to customers. You can find those test results in the Tolly reports here: http://www.vyatta.com/documentation/whitepapers.php

Cheers,

-- Dave

So there you have it. The fact is, whether something is ASIC-based or not should be irrelevant to a consumer. The only thing we should care about is the price/performance of a device in our network and whether it can do the job we need it to do. It could be powered by ASICs, general purpose processors running software, or hamsters running on a wheel, as long as it works.

Maybe the ASIC/networking-mojo myth is a mental crutch. If $8000 of server hardware and Vyatta software can outperform a $35,000 Cisco router, where's the secret mojo that you're supposedly paying for? And if it doesn't exist, why should you pay for it? If you're committed to buying overpriced systems, maybe you need to believe there is something magical about them to justify it to yourself.

Cisco rootkit

Network World has a nice story today about a researcher who has created a rootkit for Cisco gear. I bring this to your attention, not because I want to criticize Cisco for having security issues, but rather because so many people think that if they just buy Cisco they are safe from this sort of thing. Nothing could be further from the truth!

The myth goes something like this:

1. Everybody has bugs and therefore has the potential for security flaws.
2. Because my proprietary vendor keeps its source code closed, however, the bad guys can't see that code and are thus hampered in developing exploits.
3. My proprietary vendor keeps me safe by employing a large security team to constantly monitor its own development and fix any flaws that it uncovers.
4. If my proprietary vendor uncovers a security flaw, they will use their vast resources to inform me of that flaw with instructions as to what I should do to deal with the problem.

The first statement in that list is true. The rest are all false. The facts are:

• The bad guys already have Cisco's code. The reality here is that no code of any worth can be kept absolutely private. There are too many people at Cisco who have access to that code to keep it safe for long. Ditto with Juniper, Nortel, or anybody else. The same things have happened in the past to Microsoft. If your network security plan relies on the bad guys not having the code, it is fundamentally flawed.
• It's great to have a security team that is monitoring your own products for flaws, but it's better to have a large community that is monitoring your products for flaws. At a certain level, you have to question the conflict-of-interest of an internal security team. Are they really incentivized to release information about potential exploits? How quickly? If security is in conflict with other internal priorities, what wins? We have seen lots of vendors sit on critical bugs for months after they have been discovered and communicated to them.
• Finally, what happens when flaws are discovered? In Cisco's case, it sued a researcher trying to warn the world of potential security problems. Is your vendor really being forthcoming about issues, or are they trying to silence reasonable, serious discussions about security problems?

I have said it before and I'll say it again: There are two types of companies, those that have security issues and those that are lying. Open source tends to handle the exploits better (not perfectly!) when they occur by providing reliable information rapidly to the people who are in the best position to make use of that information.

Many people believe that security is increased when there is a free flow of information about systems. As the source code thefts make clear, you have to assume that the bad guys have the code. If the bad guys are left to work by themselves in a secret back room, trying to discover remote exploits, they will find them. Your only chance to stay ahead of that is to give the good guys all the information they need to find the exploits first. That can only be done when there is free access to the code.

Is a Cisco rootkit surprising? No, not if you have an accurate view of the world. It's only surprising if you were buying into the myth that your proprietary vendor was immune to that sort of thing. Is a Cisco rootkit necessarily a big problem? Again, no, because if you had an accurate view of the world you always knew something like this could be done and you'd be trying to make sure your systems were secure from the start.

Could somebody develop a Vyatta rootkit? Sure. The difference is that we'll tell you that's a possibility up front and we won't act surprised when it happens. In fact, given that Vyatta is based on Linux and there are many Linux rootkits floating around the ether, it's likely that one could be easily adapted to work with a Vyatta system. That's all the more reason to dispense with any other security myths you may be holding on to and get down to securing your systems.

News Flash: Your Vyatta system just got cheaper

Here's a great example of the power of Ecosystem Economics™.

Cnet News.com is reporting that Intel just cut the prices of some of its quad-core CPUs by 50%. Obviously, this doesn't translate into a 50% cut in system prices, but isn't it nice to know that you'll be getting more power tomorrow for a lower price?

Obviously, your Cisco system probably won't get any cheaper tomorrow, barring a random pricing action from Cisco. Bummer.

This posting isn't the follow-up article I promised you the other day when we were talking about the Grid of Openness™, but it was a timely highlight of an underlying point: open source + open hardware = Ecosystem Economics™. Ecosystem Economics™ immediately incorporates any pricing action into the whole market and everybody benefits.

Unless your networking vendor is sitting in the upper-right box in the Grid of Openness™, you'll never see benefits from things like this latest move from Intel. I'll try to expand on this point a bit more this week and we'll call it the follow up post I promised last time.

"Open" 'Cause We Say So

So the recent feeding frenzy related to "open" networking devices started me thinking. What, exactly, do we mean by open?

This is one of those times in the market where a bandwagon starts to develop and everybody is hopping on board. Marketers around the networking industry are suddenly rushing around with yellow PostIt™ notes to stick the word "open" onto anything and everything. You half expect commercials such as, "The Global Networking Conglomerate 3000, now with improved 'openness.'" All this labeling of everything open really begs the question of what we mean by openness. And more to the point, what do consumers want "open" to mean? The answer from most of the vendors making announcements lately seems to be, "It's 'open' because we say so."

Over the next few paragraphs, let's examine what "open" could possibly mean, and then we'll try to triangulate the positions of the various companies that have announced "open" products.

It seems to me that there are at least two dimensions for openness: hardware and software. There may be more, but those are certainly the most obvious and having two dimensions makes for a good graph. Along each dimension, there are (at least) four degrees of openness.

For instance, along the hardware dimension a company could have:

• Proprietary hardware -- This represents the most-closed hardware. Developing for this sort of platform requires an embedded development kit because the architecture is non-standard. Note that proprietary hardware may use standard components, such as x86 CPUs, but because of the way the hardware is designed, it's still an embedded system, not standard.
• x86 blade -- When proprietary companies want to "open up" the hardware, they often do so by adding an x86 blade to the system. While the "host" hardware system is still very proprietary, the blade uses a more standard PC architecture and may use more standard development tools.
• x86-based -- In this model, the hardware is completely standard, but is sold as a proprietary system. Many security and traffic management appliances use this model. The vendor buys a white-box PC, adds a branded label to it, and then loads their own software onto the system. While the hardware is completely standard under the hood, there is no suggestion from the vendor that the user is able to swap components or perform system upgrades using standard components.
• Commodity hardware -- This is the final model. The system uses completely open hardware and the supplier and customer both expect that users will be able to perform system upgrades with components from different vendors. The current x86 server market delivers this kind of hardware today.

Similarly, along the software dimension, the following four models can be seen:

• Proprietary software -- In this model, the software is completely closed. It's sold only in binary form. Users use the software and there is no ability to develop extensions without acquiring a proprietary license.
• Country-club API -- In this model, the company opens some APIs that allow external developers to interact with its otherwise-proprietary system in a programmatic way, but the API or SDK must be licensed from the company. Typically, the company tightly controls who is allowed to participate in the program and may charge a multi-thousand-dollar "program fee" to participate. This makes the program as exclusive as a country-club.
• Open API -- In this model, the API is completely open, with no strings attached in order to get it an use it. A public SDK, posted for free download from a web site would qualify here. The code you're interfacing with would still be closed-source, but at least you could get ahold of the SDK without paying any fees or being "approved." Note that you may still have to buy oodles of proprietary hardware in order to do any development, but at least the SDK is freely available.
• Open source -- This is obviously the most open. The source code is readily available and there are no fees to develop with the system.

Now, given all these definitions, we can make a chart that describes the landscape graphically:

I've also plotted some of the recent announcements according to how I think they stack up.

• Vyatta -- Vyatta first launched its product in July 2006. Vyatta has been open source and running on open, commodity hardware since the get-go. Want to download our software? You can do it from the web site. Want to download the source code? We have instructions in plain view. Want to run your Vyatta system on whatever hardware you want? That's fine by us, and we have even published a hardware compatibility list to help you choose something known to work well, but you're free to go off-menu as well.
• 3Com -- 3Com was early out of the gate, over a year ago (1Q07). 3Com announced an x86 blade for its routers and a country-club API software program, called OSN. OSN has a couple membership different levels, with the lowest level free to join, so it may be that OSN is walking the line between a country-club API and an open API.
• Juniper -- Next up was Juniper in December 2007 with the PSDP program. The PSDP was a country-club API program delivered on the same proprietary hardware they had shipped previously (no blades required). There are a couple of positions for Juniper on the chart because they have different product lines with different implementation techniques and different capabilities. From what I have read, the PSDP only applies to the high-end service provider routers; the Juniper J-series routers are essentially PCs with a proprietary software load. Juniper does change the connectors and form-factor of the add-in cards so they look proprietary, but they're just standard PCI hardware under the hood. The processor is a stock Intel x86 CPU.
• Riverbed -- In February 2008, Riverbed started making noise about opening up with its RiOS Services Platform (RSP). Riverbed is a good example of an appliance vendor using stock PC hardware with a proprietary software load. The RSP program puts Riverbed into the country-club API on x86-based hardware category on this chart.
• Cisco -- Finally, in April 2008, Cisco announced its Application eXtension Platform (AXP) program. This is another good example of a country-club API paired with an x86-based blade to plug into proprietary hardware. Of course, the AXP is only available on the ISR series; the rest of Cisco's product line remains locked up tighter than Fort Knox and finds itself down in the Proprietary/Proprietary category.

This post is already getting pretty long, so I'll cut it off here. The major takeaway of all this is that there are different degrees of "open" that are running about the networking market these days. With everybody using the same word, and being intentionally vague (and sometimes misleading), it's easy to confuse one "open" for another. But they're not created equal. Don't be afraid to ask a vendor why they think they're being particularly open. If you don't like the vague, "'cause we say so" answer you're likely to get initially, don't be afraid to press ahead. At least at Vyatta, we have no trouble answering that question. The other guys...? Well, who knows.

In a follow-on post, we'll discuss the implications of being more open. Are there really differences between an x86-blade with a country-club API versus open source software running on commodity hardware? The short answer is you betcha! See you next time.

Dumb and Dumber

When you're a big networking company and all your competitors are talking about open networking platforms, you have to do something... fast. Unfortunately, charging oodles of money for a low-performance x86 blade that you can stuff into your router seems to be the typical response. Hang with me for a moment and I'll explain.

Our story starts way back in January 2007 when 3Com announced its Open Services Networking initiative. At the time, 3Com said that it was "opening up" its routers by allowing you to run Linux on an x86-based blade that plugged into its systems. Since that time, 3Com has announced a few partners and applications that have been developed. Back in early 2007, most people yawned. Frankly, this was a pretty obvious innovation in the industry and hey, it was from 3Com, so who cares?

Next, Juniper got into the act when it announced the piss-dip (PSDP) on the first day of Cisco's yearly analyst conference in December 2007? The piss-dip, as you'll recall, is a program to allow a group of country-club ISVs to implement interesting functionality on top of Juniper's products using some nifty APIs. In return for a development fee and some legal paperwork, Juniper sends you a software development kit (SDK) and you're good to go. Notably, Juniper did not announced an overpriced x86-blade for its routers as part of the program. That may be because Juniper already sells overpriced x86-blades (they're called "Routing Engines" to make you feel more comfortable paying that much).

Now, Cisco couldn't take all that laying down. They had to respond. And fast. When asked at the analyst conference, they waved their hands and said, "...someday..." But this was embarrassing. Here we have nearly-dead 3Com and now arch-rival Juniper going where Cisco has never gone, and flaunting it in front of Cisco's not-nearly-skeptical-enough analyst corps. That's not good.

So, enter the Application eXtension Platform (AXP). Basically, Cisco aped 3Com's approach: with the AXP, you can pay wads of money for a low-performance x86-blade that plugs into your Integrated Services Router (ISR).

Let's look at the numbers. 3Com was trying to sell us a 1.4 GHz Pentium M, 1 GB RAM, 80 GB HDD system for over $3000 street price. Now we have Cisco trying to sell us a 1.4 GHz Pentium, 2 GB RAM, 160 GB HDD for over $6000 street (NME-522). Okay, so they did double the RAM and hard disk size. But in today's world, that's worth a grand total of about $79 (per CDW.com, 80 GB ($50) vs. 160 GB ($62) Seagate Barracuda SATA HDD, 1 GB ($77) vs. 2 GB ($144) Crucial PC3200 DRAM). Even at the low end of the three modules that Cisco announced, they're trying to charge $1700 for a 300 MHz Celeron (AIM-102)! Yup, you read that right, MHz, not GHz. Frankly, I didn't realize that you could still buy something that slow from Intel. I think that processor was completely obsolete nearly 10 years ago.

Now, realize that neither of these x86 blades is expandable in any way. If you don't like the performance or RAM or HDD size, you have no options. You can't upgrade them, short of buying a whole new module in Cisco's case. If you already bought the fastest one (NME-522), you're screwed. No expansion slots. No multi-core. No options. Bluntly, you're trapped in Cisco World™ and 3Com World™.

Does anybody else feel like we're watching the movie Dumb and Dumber here?

Of course, for both 3Com and Cisco, you also have to buy the router to plug these underpowered, overpriced x86 blades into. Presumably, you have already made that decision, so the $4000 to $15,000 of sunk cost shouldn't bother you.

At this point, I have to hand it to Juniper: the piss-dip looks pretty good when compared to these options. Juniper at least lets you run piss-dip applications on the Routing Engine you already paid for instead of charging you oodles more for another blade.

The point of this rant is simply that this is what you get from proprietary networking companies. Even when they serve up completely open technologies like Linux running on x86, it's going to be terribly expensive with lock in not far behind.

In contrast, Vyatta runs on standard x86 systems. You can buy those systems with Vyatta software preloaded, directly from Vyatta, or you can buy the hardware from your favorite hardware vendor and your software subscription from us. If you want a hybrid of the two approaches, that's fine with us, too. While Vyatta does mark up the hardware we sell, we try to keep that markup small and appropriate.

Importantly, with Vyatta, you aren't stuck with no options if you want to make a change to the system. Need to run faster? There are oodles of vendors with blazing multi-core systems available today. Want more memory? Fine, you can purchase it from just about anybody. Need a bigger hard drive? No problem. Want to add different applications to your system? It's pretty easy since Vyatta is Debian-compatible. Want to extend or hack the system? The source code is on the Internet and you can download it for free, without any legal paperwork.

The other guys will go on and on about their proprietary hardware. "You just can't do networking on standard x86 systems," they'll say. "You need our sooper-dooper ASICs to run fast, and well, you know how much those cost..."

But the fact is, it simply isn't true. With Vyatta and an IBM x3550 quad-core server, available for about $4000 or so, you can whip a $35,000 Cisco 7204/G2. With Vyatta and a $1000 Dell PowerEdge 860, you can demolish a Cisco 2821 ISR. Check out Vyatta's 3rd party testing if you don't believe me.

Once you're done doing that, you can use all those MIPS to run whatever applications you want, including many of the sorts of things that Cisco and 3Com would charge you for (remember that the x86 blades are just the hardware--you still have to buy applications from other vendors).

At the end of the day, the key point here is that the other guys charge you a lot of money to open up a closed system. And when you pay that money, you still find yourself stuck in an alternative reality called Cisco World™, Juniper World™ or 3Com World™.

Is that "open?" Not in the Real World™

Update: Okay, a commenter pointed out that I pulled the wrong prices for the Cisco AXP modules. I had incorrectly used the WAAS version of the NME-522. Apologies for that. It's the same hardware, but a different software load, and therefore a different price. Looks like list on the AXP version of the NME-522 is about $3500. More than 3Com, but reasonable given the doubling of memory and disk capacity. That said, I still stick with my main point that this is an expensive, underpowered PC with no flexibility, and that's after you purchase the router to plug it into. Rather than titling this post "Dumb and Dumber," maybe I'll have to change it to "Dumb and Dumb."

Kernel.org to be upgraded to FreeBSD 7.0

Such is the word...

Wow, whoda thunk it?

Some of the comments back pointed out other documents worth reading today, just for historical perspective. Some of my favorite RFCs include:

1. RFC 748
2. RFC 1149
3. RFC 1606
4. RFC 1924
5. RFC 2550
6. RFC 2795
7. RFC 3093
8. RFC 3514

I particularly like the last two as related to Vyatta's firewall implementation. We have had numerous requests for RFC 3514 support and are slotting it into a future release.

You can find a more complete list of interesting RFCs on Wikipedia.

Gotcha...

"The Patent Reform Act will harm the U.S. technology industry"

Steve Tobak posted an article about the upcoming Patent Reform Act over at C|Net. I had just mentioned intellectual property reform a couple of days ago in this previous blog entry.

Interestingly, I disagree with Steve's analysis. Steve says:

Let's instead just cut to the chase. In lay terms, the bill makes it easier to challenge issued patents and harder for patent holders to obtain compensation through the U.S. legal system.

Steve argues that because the US is shifting away from a production economy to an intellectual property and licensing economy, these "reforms" are bad for US business. Steve says:

In one corner are big technology companies such as Apple, Cisco, Dell, Google, HP, Intel, Microsoft, Oracle and SAP. These folks make a living selling products and services. They say that patent abuses in the current system are stifling innovation.

In the other corner are technology licensing companies such as 3M, Qualcomm, Rambus, Tessera, and biotech and pharmaceutical companies. They say the act will limit patent holder's rights and stifle innovation.

While each side claims the other limits innovation, the truth is that neither side cares about innovation; they are only concerned with their business model. That's not necessarily a bad thing, since a company's duty is primarily to its shareholders, but it does bear mentioning here.

Now, I have lived in the technology industry for quite some time. I have never worked at a large technology licensing company such as 3M, Qualcomm, Rambus, or in biomed or pharma. That probably biases my thinking.

On the other hand, I have about 10 patents, assigned to various companies I have worked for over the years. I have spent a reasonable amount of time dealing with the patent system.

The fact is, many patents are bogus (think Amazon's 1-Click patent). There, I said it. Many should not have been issued, either because they are so obvious to those knowledgeable in the art or because there was existing prior art. These bogus patents are a noose around the neck of the technology industry. They clog up the system and make it impossible to create almost anything without treading on somebody's patent without even knowing about it. With a patent term of 17 to 20 years, these bogus patents are in force for multiple product lifetimes. For perspective, 20 years is 10 to 13 turns of of the Moore's Law crank. Patents expiring now would have been issued back in 1990, before the explosive rise of the Internet (though the Internet was actually being used at that point, nobody outside of academic and tech circles had heard of it). If you don't think that stifles product innovation, you have never tried to innovate. I have lived with this environment all of my career. I have made decisions in the past to navigate around bogus patents, simply because the lawyers told me it was a lost cause to try to challenge them.

What's all the more infuriating about the current patent situation is that many of today's patents go against the original social contract surrounding patents. The original goal of the patent system was to get inventors to share their innovations for the common good. In return for a limited monopoly, you, Mr. Inventor, share your invention so that We, the public, can understand how you did it and can then innovate on top of it. Rather than stifling innovation, patents were supposed to drive it forward.

Unfortunately, many patents, even the ones that are legit, would have been created independently anyway. It's obviously a balance, but at least in the world I live in, I see patents getting in the way rather than helping me. I have never gone and looked at old patents to get new ideas for products. The only time an independent patent, one that I'm not working on filing myself, comes to my attention, it's because somebody is getting sued for infringing it. This tells me that we have lost the original goal that patents were supposed to foster.

Now, it's important to realize what the patent reform act doesn't do. It doesn't mean that patents are extinct. It also doesn't mean that bogus patents go away with a snap of the fingers. What it does do is allow for easier challenging of what appears to be a bogus patent. This may increase the cost of patent filings since more people could challenge the patent and you'd have to respond to it. Personally, I think this works. The costs should be weighted toward the party that has the most to gain from the granting of the patent. The downstream costs of litigating bogus patents (think not just lawyers but injunctions and product disruptions) are far higher than the cost of allowing patents to be challenged with greater frequency.

In short, I think the Patent Reform Act, while in no ways perfect (doesn't go far enough, IMO), is at least a step in the right direction. Admittedly, I don't work for a large intellectual property company or a biotech/pharma company. Perhaps I'd feel differently in that case, but from where I sit, if we really want to spur innovation, we should really overhaul the system even further than the Patent Reform Act.