Layer 2 Problem

Feb 21, 2006 21 Replies
Layer 2 Problem open original image

Hi NG,



i have a strange problem in my network. Sometimes one device (maybe a workstation or printer or even a server) is unreachable by 50% of all other devices in the network.A Ping fails and a traceroute stops at the core router. They are in different ip segments but all in a broadcast domain. I have to do an "clear arp-cache" on my core router in order to get everything fine. What in hell could that be ? The log on the router shows nothing weird....



Thx in advance



Michael


More information about how your network is configured would be needed to make any educated guesses. Do you have any HSRP in the part of the network under consideration? Do you have NAT running in conjunction with HSRP-based load balancing perhaps? A combination like that could cause problems like this though that only one device becomes unreachable seems a little weird.

To see more about ARP interaction with HSRP+NAT issues look at

formatting link
for instance.

Anyway, this is only a wild guess since more detailed information is missing. Maybe some other posters will have other ideas.

Cisco da Gama

formatting link

Hi,

we do have a cisco 6000 with 4 msfc modules as core router. hsrp is configured, but no nat routing.

Michael

formatting link

Hi.

Has MLS been enabled on your MFC/SUPERVISOR?

Let me know

Bye

Carlo

Hi,

no, MLS is not enabled. Any idea ?? The Problem drives me crazy.... someone things about a spanning-tree loop....

thx

Capture the contents of the arp cache befre and after you do a clear.

Determine if the MAC address for the device that has the reachability problem is the same in both cases..

thats the point.... it IS the same mac adress. But obviously no one knows how to deliver the packet to the mac. maybe still a loop in the network ? but why do other clients reach that ip ?

a loop would typically affect a lot more than one IP address

so please provide the router & switch details:

  1. hardware models
  2. IOS / CATOS versions
  3. topology for a recent failure case - which switch and port is the unreachable device connected to, IP addresses htat can reach devices, IP addresses that cannot reaach device.

that will be a hell of information... the network has about 10 routers and over 20 switches....

Perhaps, but if you want to get to the pbottom of this you have to start characterizing the nature of the problem in more detail

How many subnets (routes) are there in your network?

Does this problem happen on every one of those subnets?

Any there any observable patterns whatsoever - time of day, etc, etc.

Have you kept a lot of every IP address for which this problem has occurred; if not highly recommend this be done.

How long has this problem being occuring ?

Recent router or switch upgrades, new switches added, topology change, sighnficant configuration changes ...

If you want to get assistance from this group, this type of info will be required.

There are about 10 subnets. The Problem was until now noticed on three of them. No observable patterns...different time, different subnets, different unreachable ips different clients who are able to reach that "unreachable" ip

yes, i wrote down all Ips and the time it occured. turned the debugging on, checked the log.... This appears now for 3-4 months, sometimes once a day, sometimes even 4-5 times.

as far as i know there were no significant changes in the topology or the configs

  1. When the problem occurs is it a single or multiple IP's that becomes unreachable?

  1. Per occurrence are these IPs on a single subnet or on all three of the subnets?

  2. Are each of the three subnets on differnet switches or the same switch ?

  1. Has the problem ever gone away without you have to clear the arp cache ?

  2. Please provide the switch models and IOS version for the devices supporting the three affected subnets

Hey...first of all, thx for your assistance ;-)

  1. It´s a single IP
  2. The unreachable IP is everytime on a single subnet (Class C), the clients who can/cannot reach it are on different subnets (think so, still have to check this)
  3. They are on different switches
  4. No, only a clear arp cache on the core router solved the Problem or what i testet if that ip belongs to for example to a printer, I disconnected it from the net, connected the cable to a laptop, gave it the same ip it was reachable. Plugged in the printer again >> reachable....
  5. Core routers (2 cat 6000 cross linked with 2 msfc each, HSRP configured) The switches : Mostly 2950, 35XX, one cat6000) I will provide specified info as soon as possible (Currently not in office)

For the IP that becomes unreachable, can it be pinged from other PCs that are connected to the SAME access switch ?

Are the access switches dual homed to each of the 6500 switches ?

The next time problem occcurs, from the PC that becomes unreachable, try the following ping tests:

  1. ping HSRP virutal IP address for subnet (default gateway)

  1. ping physical IP address of 6500 A

  2. ping phsical IP address of 6500 B

have to check...

most of the access switches are dual homed.

will post results asap

Additionally when problem occurs, capture the CAM or MAC-address table on the access switch as well as on both 6500 switches (from the switch sup cards). also capture the output of show standby

Please post "show cdp nei detail " from each of the three access switches on which the IP connectivity issue has occurred

Join the Discussion

Have something to add? Share your thoughts — no account required.

Didn't find your answer?

Ask the community — no account required