Cat 5500 Earl process is nearly 80% cpu usage

I have a problem since one day with an old Catalyst 5500 which is up since 384 days. Console or telnet access are very slow and snmp query are often timed out. No recent change in the network, nothing in the logs. The only thing I found is that the Earl process is nearly 80% cpu usage.

cat_c13> (enable) ps -c CPU usage information: Name CPU-Usage Invokations

--------------- --------- ----------- idle 4% 1 Statuspoll 0% 1 SynDiags 0% 1 SynConfig 3% 1 SWPoll64bCnt 0% 1 SL_TASK 0% 1 RedundantTask 0% 1 nvram_active 0% 1 gentcp_act 0% 1 THREAD 0% 1 telnet10 0% 1 ATM/RSM Support 0% 1 telnet12 0% 1 telnet14 1% 1 SecurityRx 0% 1 Earl 77% 1 Console 0% 1 telnetd 0% 1 llcSSTPFlood 0% 1 sntp_recd 0% 1 sntp_xmitd 0% 1 cdpd 0% 1 cdpdtimer 0% 1 SptTimer 2% 1 SptBpduRx 8% 1 VtpTimer 0% 1 VtpRx 0% 1 RMON AlarmTimer 0% 1 ProtocolTimer 0% 1 DTP_Rx 0% 1 PAgP_Rx 0% 1 PAgP_Group_Rx 0% 1 sptHelper 0% 1 sptTraps 0% 1 fsd_timer 0% 1 SysLogTask 0% 1 pinggateA 0% 1 snmpdm 0% 1

System Idle - Current: 1% High: 1% Low: 1% Average: 1%

cat_c13> (enable) sh sys PS1-Status PS2-Status Fan-Status Temp-Alarm Sys-Status Uptime d,h:m:s Logout

---------- ---------- ---------- ---------- ---------- --------------

--------- ok ok ok off ok 384,07:26:39

20 min

PS1-Type PS2-Type Modem Baud Traffic Peak Peak-Time

---------- ---------- ------- ----- ------- ----

------------------------- WS-C5508 WS-C5508 disable 9600 1% 16% Fri Nov 18 2005,

15:54:32

System Name System Location System Contact

------------------------ ------------------------

------------------------ cat_c13

cat_c13> (enable) sh cam count dyn Total Matching CAM Entries = 762

cat_c13> (enable) sh mod Mod Module-Name Ports Module-Type Model Serial-Num Status

--- ------------------- ----- --------------------- --------- ---------

-------

1 2 100BaseFX MM Supervis WS-X5506 006824425 ok 2 2 100BaseFX MM Supervis WS-X5506 007408712 standby 3 back_adm1 1 Route Switch WS-X5302 028993639 ok 4 24 10/100BaseTX Ethernet WS-X5224 007420093 ok 5 24 10BaseT Ethernet WS-X5013 006822219 ok 6 24 10/100BaseTX Ethernet WS-X5225R 011580347 ok 7 24 10/100BaseTX Ethernet WS-X5225R 012132342 ok 9 24 10/100BaseTX Ethernet WS-X5225R 011575143 ok 10 24 10/100BaseTX Ethernet WS-X5225R 012131907 ok

Mod MAC-Address(es) Hw Fw Sw

--- -------------------------------------- ------ ----------

-----------------

1 00-80-1c-23-70-00 to 00-80-1c-23-73-ff 1.0 2.2(2) 4.4(1) 2 00-80-1c-23-70-00 to 00-80-1c-23-73-ff 2.1 2.4(1) 4.4(1) 3 00-e0-1e-39-9e-30 to 00-e0-1e-39-9e-31 7.7 20.22 12.0(9)W5(17a) 4 00-e0-1e-47-d9-18 to 00-e0-1e-47-d9-2f 1.3 3.1(1) 4.4(1) 5 00-10-54-85-8d-b8 to 00-10-54-85-8d-cf 1.0 1.1 4.4(1) 6 00-50-a2-37-e6-f0 to 00-50-a2-37-e7-07 2.0 4.1(1) 4.4(1) 7 00-50-a2-f0-7d-a0 to 00-50-a2-f0-7d-b7 3.0 4.3(1) 4.4(1) 9 00-50-a2-37-e2-70 to 00-50-a2-37-e2-87 2.0 4.1(1) 4.4(1) 10 00-50-a2-f0-7d-d0 to 00-50-a2-f0-7d-e7 3.0 4.3(1) 4.4(1)

Mod Sub-Type Sub-Model Sub-Serial Sub-Hw

--- -------- --------- ---------- ------

1 EARL 1+ WS-F5511 0005448642 1.0 2 EARL 1+ WS-F5511 0006839294 1.0 cat_c13> (enable)

Has somebody experienced this problem and how can I troubleshout it ?

Thanks for your help.

Philippe

formatting link

Encoded Address Recognition Logic (EARL) is a centralized processing engine for learning and forwarding packets based upon MAC address on the Catalyst 5000 Supervisor Engines. The EARL stores the VLAN, MAC address, and port relationships. These relationships are used to make switching decisions in hardware.

Maybe you have problems with excessive CAM table activity?

Not too many however I would guess that there is a lot of software involvement in address learning on the 5500 so check that out first.

The original 4000 for example only did 30 addresses a second at first then speeded up to 100 a second and it is rumoured that an early Bay 100Mpbs switch only did 1 a second.

Bug?

formatting link