Is IP The Most Cost Effective Choice For Your Business Communication Applications?

I read the article a moment ago, and it seems to me that the author is mentally equating IP with Internet.

For example, one of the suggestions is to use long haul gigabit ethernet fibre because "Ethernet allows for layer 2 switching versus IP based routing" and "From a private network perspective, this may be an ideal way of lowering overheads and improving latency and jitter issues."

If you examine the following report, you see that at least for the Cisco 6500 with Sup720:

- IPv4 layer 3 GbE latency is less than 12 1/4 microsecond avg, 16 1/2 us max

- worst case layer 2 forwarding is the same as worst case layer 3 forwarding (i.e., the limiting factor was the switch fabric speed at the worst- contention case)

- best case layer 3 forwarding is only 0.02 Mpps worse than the best case layer 2 forwarding

- layer 3 forwarding with a 10,001 entry ACL was the same as layer 3 forwarding with no ACL

Effectively, then, on that device, layer 2 and layer 3 performance are the same.

The speed of light in fibre is about 200,000 km/s, so for a (say) 100 km haul (the article talks about "long haul", which probably implies even longer distances), the latency is

100 km / 200000 km/s = 500 microseconds.

The Cisco 6500 is not exactly a box you could afford to put into every bedroom, but the Cisco 3750 (a layer 3 switch) is much less expensive. It's measured latency is 24 to 52 microseconds

latency of the 3750 is apparently not considered to be very good at all compared to competing models

Can a switch do better? Yes, you can get a lot better than 24 microseconds if you are going for a switch designed specifically for low latency, such as the Cisco Catalyst 4948. And the Cisco 2800 router series (which doesn't have nearly enough ommph to run gigabit) has an internal latency of about 3 microseconds plus serialization delay; that takes it into the

8 to 15 microsecond range at 100 Mb

What is this all telling us:

1) The choice of switch and model can make a noticable difference on absolute latency;

2) layer 3 switching vs layer 2 switching is not necessarily significantly slower (but check the specs and tests)

3) that as soon as you are into anything even remotely "long haul", absolute switch latencies are 1/50 to 1/20th of the link distance latencies, and so quickly dwarfed by what sound to be relatively small increases in distances

Where is the sweet spot where a layer 2 solution could be "an ideal solution" to lowering overheads and reducing latency, on a gigabit long-haul link? It seems to me that it -ought- to be pretty small. It would pretty much require:

- links at most in the 5 km range - gigabit switches in the consumer or SOHO price range -- unmanaged or barely managed, and probably with few QoS features (perhaps a single priority queue based upon ToS) - an application sensitive to a 20 microsecond change in latency - and yet an application that is *not* built on IP -- which requires either a good dose of (expensive) custom application building, or else holding at Netware IPX or Appletalk

With regard to this last point, note that the article describes itself in terms of whether *IP* is the most cost-effective solution, not in terms of whether *routed IP* is the most cost-effective solution. A "flat" IP network that happens to have a gigabit long-haul link in the middle of it does *not* meet the article's implication that it is *IP* that leads to the latencies, rather than IP *routing* (which, as I show above is not necessarily the case either).

I can but arrive at the conclusion that what the author is *really* trying to talk about is the use of public IP networks, which might be multihop and subject to considerable variation in latency and jitter. But that's not what the article says: the article is about *IP*, not about public networks.

and the really big asumption here is that you can get a "dedicated long haul Ethernet" (paraphrase).

yes - such things are available - no, not cheap.

the stuff we build at work uses SDH or MPLS core networks to provide what is effectively an emulated GigE end to end

dedicated links are the rule where the 2 sites are fed from a common PoP via fibre - in the UK that equates to up to maybe 50 Km apart.

there is no point for a telco to wrap the raw data in something else unless the cost of getting onto a common transport is offset by the economies of scale of using it (unless you get other advantages such as better management).

i have tested the Cat6k for something like this (only on IP, not layer 3) - jitter is down at the "who cares" level under normal operating conditions. the dominant issue for delay and latency happens when an interface is congested, and data build up in internal Qs can rise from the uSec of forwarding delay to mSec - the effect is basically limited by how much buffering you have on the box and how interfaces are configured.

however - congestion for an outbound port is nothing to do with whether the box is carrying IP, "just Ethernet" or anything else - it is a system issue to do with carrying packets within a network that may have choke points that congest.

in reality - protocols that are delay sensitive seem to be even more sensitive to jitter.

and i cannot change the distances involved, so latency is fixed anyway. So if the application doesnt work over a 1000 Km link, then the application is the broken bit.....

Sort of agreed - but this ignores the fact that there are scaling problems with Ethernet vs IP, especially when you add resilience into the mix.

you can run "flat" networks with 1000s of end points on a switched Ethernet - but i wouldnt want to try to run a IP telephony system over one, given the convergence time of 10s of seconds. Adding all the complications of a WAN to that mix is a waste of time.

and all packet networks have those problems anyway - Ethernet or IP.....