Help. My Cisco router keeps crashing.

Okay, well forget the comment about it wokring better for 12 hours after deleting the NAT statements. It still has problems.

So is the router actually reloading per the discription or is the service just degrading over time? If it is crashing please post the "show stack" and "show ver" from the router. If there is something else going on I may need to see a "show tech".

Here is the show stack and show ver. Ignore the register being 0x2142. I was just trying to see if I could get it to save the config. It seems like I also can't type in "no ip directed-broadcast" and get it to save it in any interface's config.

show stack Minimum process stacks: Free/Size Name 2532/4000 Init 3412/4000 RADIUS INITCONFIG 4300/5000 DHCP Client 1780/4000 FTP Read Process

Interrupt level stacks: Level Called Unused/Size Name 3 0 3000/3000 Serial interface state change interrupt 4 31044 2284/3000 Network interfaces 5 14871 2872/3000 Console Uart

----------------

Cisco Internetwork Operating System Software IOS (tm) 2500 Software (C2500-I-L), Version 12.1(2)T, RELEASE SOFTWARE (fc1) Copyright (c) 1986-2000 by cisco Systems, Inc. Compiled Tue 16-May-00 13:28 by ccai Image text-base: 0x03040E54, data-base: 0x00001000

ROM: System Bootstrap, Version 11.0(10c), SOFTWARE BOOTFLASH: 3000 Bootstrap Software (IGS-BOOT-R), Version 11.0(10c), RELEASE SOFTWARE (fc1)

TheOrchid uptime is 13 minutes System returned to ROM by reload System restarted at 11:19:59 BKK Thu Sep 29 2005 System image file is "flash:c2500-i-l.121-2.T.bin"

cisco 2500 (68030) processor (revision N) with 14336K/2048K bytes of memory. Processor board ID 11162735, with hardware revision 00000001 Bridging software. X.25 software, Version 3.0.0. Basic Rate ISDN software, Version 1.1.

1 Ethernet/IEEE 802.3 interface(s) 2 Serial network interface(s) 1 ISDN Basic Rate interface(s) 32K bytes of non-volatile configuration memory. 8192K bytes of processor board System flash partition 1 (Read ONLY) 8192K bytes of processor board System flash partition 2 (Read/Write)

Configuration register is 0x2142 (will be 0x2102 at next reload)

The router is not actually rebooting. The service just degrades over time. Everything slows down. The console is slow, packets get lost when pinging, etc. I had another 2501 router in there and it did the same thing. We have other sites with no problems. Only serial0 and ethernet0 are being used.

Unless you need it I suggest that removing the

no ip route-cache

from the interfaces would be a VERY good idea. no ip route-c turns off "fast" switching which is about 10 x faster than "process" switching.

"no ip directed-broadcast" is probably the "default" and as such does not appear in the config.

You need to get some basic cisco skills sorted out.

This looks like the router is just doing what it should, routing traffic.

sh proc cpu

show interface switching

May help to track down the issue.

sh int and look at the packet rates on the interfaces.

A 2500 is not a fast box and could for example be easily overwhelmed by LAN side broadcasts.

Turn ON fast switching and see how it goes.

A sh proc cpu (pasted below) tells me the process that is probably causing the issue - IP Input. I unplugged it from all but one PC on the internal network and it still has this problem. I'm not sure what to do about it though. Other sites have almost the exact same config with no problems. If I let the machine run, IP Input will keep rising to a higher and higher percentage.

Any suggestions?

CPU utilization for five seconds: 43%/2%; one minute: 39%; five minutes: 32% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 116 395 293 0.00% 0.00% 0.00% 0 Load Meter 2 21884 807 27117 0.00% 0.02% 0.13% 0 Exec 3 7416 438 16931 0.00% 0.38% 0.36% 0 Check heaps 4 4 1 4000 0.00% 0.00% 0.00% 0 Chunk Manager 5 0 1 0 0.00% 0.00% 0.00% 0 Pool Manager 6 0 2 0 0.00% 0.00% 0.00% 0 Timers 7 4 17 235 0.00% 0.00% 0.00% 0 Serial Backgroun 8 440 221 1990 0.00% 0.01% 0.00% 0 ARP Input 9 8 4 2000 0.00% 0.00% 0.00% 0 DDR Timers 10 0 2 0 0.00% 0.00% 0.00% 0 Dialer event 11 32 2 16000 0.00% 0.00% 0.00% 0 Entity MIB API 12 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect 13 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd 14 924 433 2133 0.00% 0.02% 0.02% 0 Net Background 15 12 11 1090 0.00% 0.00% 0.00% 0 Logger 16 316 1954 161 0.00% 0.00% 0.00% 0 TTY Background 17 240 1974 121 0.00% 0.01% 0.00% 0 Per-Second Jobs 18 4 1 4000 0.00% 0.00% 0.00% 0 ISDN From Driver 19 84 247 340 0.00% 0.00% 0.00% 0 Net Input 20 236 396 595 0.00% 0.03% 0.00% 0 Compute load avg 21 1652 34 48588 0.00% 0.08% 0.06% 0 Per-minute Jobs 22 338944 31434 10782 40.21% 35.52% 28.20% 0 IP Input 23 10492 274 38291 0.00% 0.47% 0.55% 0 CDP Protocol 24 0 1 0 0.00% 0.00% 0.00% 0 X.25 Encaps Mana 25 0 2 0 0.00% 0.00% 0.00% 0 Emulator 26 0 1 0 0.00% 0.00% 0.00% 0 PPP IP Add Route 27 1132 73 15506 0.00% 0.02% 0.01% 0 IP Background 28 260 34 7647 0.00% 0.00% 0.00% 0 Adj Manager 29 16 21 761 0.00% 0.00% 0.00% 0 TCP Timer 30 8 2 4000 0.00% 0.00% 0.00% 0 TCP Protocols 31 0 1 0 0.00% 0.00% 0.00% 0 Probe Input 32 0 1 0 0.00% 0.00% 0.00% 0 RARP Input 33 0 1 0 0.00% 0.00% 0.00% 0 HTTP Timer 34 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers 35 16 2 8000 0.00% 0.00% 0.00% 0 DHCPD Receive 36 1900 33 57575 0.00% 0.08% 0.07% 0 IP Cache Ager 37 0 1 0 0.00% 0.00% 0.00% 0 COPS 38 0 1 0 0.00% 0.00% 0.00% 0 PAD InCall 39 4 2 2000 0.00% 0.00% 0.00% 0 X.25 Background 40 4 1 4000 0.00% 0.00% 0.00% 0 ISDN Timer 41 1760 112 15714 0.15% 0.01% 0.11% 2 Virtual Exec 42 0 2 0 0.00% 0.00% 0.00% 0 Router Autoconf 43 0 1 0 0.00% 0.00% 0.00% 0 CallMIB Backgrou 44 0 1 0 0.00% 0.00% 0.00% 0 ISDNMIB Backgrou 45 4 1 4000 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro 46 4128 18543 222 0.31% 0.42% 0.31% 0 IP NAT Ager 47 496 2008 247 0.00% 0.02% 0.00% 0 NTP 49 8 17 470 0.00% 0.00% 0.00% 0 DHCPD Timer 50 140 561 249 0.00% 0.00% 0.00% 0 DHCPD Database

Please TURN ON FAST SWITCHING.

int xxx ip route-cache

It is on by default and it has been turned off on this box.

Your "problem" might well disappear.

IP Input is the process that does IP routing.

CPU utilization for five seconds: 43%/2%; one minute: 39%; five minutes: 32%

43%/2% means 43% total of which 2% is fast switching.

enable fast switching on your interfaces as discussed.

Otherwise:-

Please (when the cpu is high):- clear counters sh run wait say 5 mins sh int sh proc cpu sh int switching wait say 20 mins sh int sh proc cpu sh int switching

Post output.

Sorry to post so much info, but I still cannot figure out the problem. I turned on ip-route cache again. But the problem persists. So my config for the main interfaces now looks like this:

--------- interface Ethernet0 ip address 192.168.0.1 255.255.255.0 ip nat inside ! interface Serial0 description Connect to M2K-TOT ip address 204.57.130.54 255.255.255.252 ip nat outside no fair-queue

-------- I booted the machine and waited a few minutes and then ran the commands as described in the previous post. I'll wait 20 more and post again. You will see that the IP Input process keeps rising. It got to 91% at one point before I had to reboot.

--------

---***** SH RUN ****----- Current configuration: ! ! No configuration change since last restart ! version 12.1 service timestamps debug uptime service timestamps log uptime no service password-encryption service udp-small-servers service tcp-small-servers ! hostname TheOrchid ! enable secret 5 $1$X5U7$dntiuJpq9ln7g65X1XulQ1 enable password 1234 ! ! ! ! ! clock timezone BKK 7 ip subnet-zero ip ftp username Philt ip ftp password birdnest7 ip name-server 204.57.160.129 ip name-server 204.57.128.67 ! ip dhcp pool network1 network 192.168.0.0 255.255.255.0 domain-name m2khotspot dns-server 204.57.160.129 204.57.128.67 default-router 192.168.0.1 lease 30 ! partition flash 2 8 8 ! ! ! ! interface Ethernet0 ip address 192.168.0.1 255.255.255.0 ip nat inside ! interface Serial0 description Connect to M2K-TOT ip address 204.57.130.54 255.255.255.252 ip nat outside no fair-queue ! interface Serial1 no ip address no ip route-cache no ip mroute-cache shutdown ! interface BRI0 no ip address shutdown ! ip nat inside source list 1 interface Serial0 overload ip nat inside source list 7 interface Serial0 overload ip nat inside source static tcp 192.168.0.254 8989 204.57.130.54 8080 extendable ip nat outside source static tcp 204.57.130.54 8080 192.168.0.254 8989 extendabl e ip classless ip route 0.0.0.0 0.0.0.0 204.57.130.33 ip route 0.0.0.0 0.0.0.0 204.57.130.53 no ip http server ! access-list 1 permit 192.168.0.0 0.0.0.255 access-list 7 permit 192.168.0.0 0.0.0.255 ! line con 0 transport input none line aux 0 transport input all line vty 0 password 1234 logging synchronous login line vty 1 4 password 1234 login ! ntp clock-period 17179875 ntp server 204.57.160.2 end

-----**** SH INT ****------ BRI0 is administratively down, line protocol is down Hardware is BRI MTU 1500 bytes, BW 64 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Last input never, output never, output hang never Last clearing of "show interface" counters 00:01:46 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/16 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions BRI0:1 is administratively down, line protocol is down Hardware is BRI MTU 1500 bytes, BW 64 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input never, output never, output hang never Last clearing of "show interface" counters 00:01:49 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/16 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions BRI0:2 is administratively down, line protocol is down Hardware is BRI MTU 1500 bytes, BW 64 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input never, output never, output hang never Last clearing of "show interface" counters 00:02:10 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/16 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions Ethernet0 is up, line protocol is up Hardware is Lance, address is 0010.7b7f.d00d (bia 0010.7b7f.d00d) Internet address is 192.168.0.1/24 MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters 00:02:13 Queueing strategy: fifo Output queue 0/40, 0 drops; input queue 0/75, 0 drops 5 minute input rate 24000 bits/sec, 49 packets/sec 5 minute output rate 17000 bits/sec, 6 packets/sec 6859 packets input, 431784 bytes, 0 no buffer Received 42 broadcasts, 0 runts, 0 giants, 0 throttles 14 input errors, 0 CRC, 0 frame, 0 overrun, 14 ignored 0 input packets with dribble condition detected 776 packets output, 624846 bytes, 0 underruns 0 output errors, 1 collisions, 0 interface resets 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier 0 output buffer failures, 0 output buffers swapped out Serial0 is up, line protocol is up Hardware is HD64570 Description: Connect to M2K-TOT Internet address is 204.57.130.54/30 MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec, reliability 255/255, txload 3/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters 00:02:38 Queueing strategy: fifo Output queue 0/40, 0 drops; input queue 0/75, 0 drops 5 minute input rate 12000 bits/sec, 1 packets/sec 5 minute output rate 20000 bits/sec, 48 packets/sec 798 packets input, 613033 bytes, 0 no buffer Received 19 broadcasts, 0 runts, 0 giants, 0 throttles 1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 7990 packets output, 422202 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions DCD=up DSR=up DTR=up RTS=up CTS=up Serial1 is administratively down, line protocol is down Hardware is HD64570 MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input never, output never, output hang never Last clearing of "show interface" counters 00:02:40 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/256 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions DCD=down DSR=down DTR=down RTS=down CTS=down

-----***** SH PROC CPU *****--------

CPU utilization for five seconds: 30%/3%; one minute: 24%; five minutes: 16% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 24 76 315 0.00% 0.00% 0.00% 0 Load Meter 2 5964 150 39760 0.00% 0.00% 1.12% 2 Virtual Exec 3 1528 92 16608 0.00% 0.37% 0.26% 0 Check heaps 4 4 1 4000 0.00% 0.00% 0.00% 0 Chunk Manager 5 12 4 3000 0.00% 0.00% 0.00% 0 Pool Manager 6 0 2 0 0.00% 0.00% 0.00% 0 Timers 7 4 2 2000 0.00% 0.00% 0.00% 0 Serial Backgroun 8 228 90 2533 0.00% 0.03% 0.01% 0 ARP Input 9 0 3 0 0.00% 0.00% 0.00% 0 DDR Timers 10 4 2 2000 0.00% 0.00% 0.00% 0 Dialer event 11 28 2 14000 0.00% 0.00% 0.00% 0 Entity MIB API 12 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect 13 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd 14 188 120 1566 0.00% 0.01% 0.00% 0 Net Background 15 8 7 1142 0.00% 0.00% 0.00% 0 Logger 16 128 369 346 0.00% 0.00% 0.00% 0 TTY Background 17 48 388 123 0.00% 0.00% 0.00% 0 Per-Second Jobs 18 0 1 0 0.00% 0.00% 0.00% 0 ISDN From Driver 19 104 129 806 0.00% 0.00% 0.00% 0 Net Input 20 56 77 727 0.08% 0.01% 0.00% 0 Compute load avg 21 308 7 44000 0.00% 0.05% 0.03% 0 Per-minute Jobs 22 63224 5066 12480 26.62% 18.95% 11.77% 0 IP Input 23 2092 58 36068 0.00% 0.85% 0.42% 0 CDP Protocol 24 0 1 0 0.00% 0.00% 0.00% 0 X.25 Encaps Mana 25 4 2 2000 0.00% 0.00% 0.00% 0 Emulator 26 0 1 0 0.00% 0.00% 0.00% 0 PPP IP Add Route 27 204 22 9272 0.00% 0.04% 0.00% 0 IP Background 28 48 8 6000 0.00% 0.01% 0.00% 0 Adj Manager 29 32 15 2133 0.00% 0.00% 0.00% 0 TCP Timer 30 8 2 4000 0.00% 0.00% 0.00% 0 TCP Protocols 31 0 1 0 0.00% 0.00% 0.00% 0 Probe Input 32 0 1 0 0.00% 0.00% 0.00% 0 RARP Input 33 4 1 4000 0.00% 0.00% 0.00% 0 HTTP Timer 34 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers 35 16 2 8000 0.00% 0.00% 0.00% 0 DHCPD Receive 36 488 7 69714 0.00% 0.15% 0.08% 0 IP Cache Ager 37 0 1 0 0.00% 0.00% 0.00% 0 COPS 38 0 1 0 0.00% 0.00% 0.00% 0 PAD InCall 39 8 2 4000 0.00% 0.00% 0.00% 0 X.25 Background 40 0 1 0 0.00% 0.00% 0.00% 0 ISDN Timer 42 4 2 2000 0.00% 0.00% 0.00% 0 Router Autoconf 43 0 1 0 0.00% 0.00% 0.00% 0 CallMIB Backgrou 44 0 1 0 0.00% 0.00% 0.00% 0 ISDNMIB Backgrou 45 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro 46 1624 3982 407 0.65% 0.50% 0.31% 0 IP NAT Ager 47 120 398 301 0.00% 0.00% 0.00% 0 NTP 49 0 4 0 0.00% 0.00% 0.00% 0 DHCPD Timer 50 44 119 369 0.00% 0.00% 0.00% 0 DHCPD Database

-----***** SH INT SWITCHING *****--------

Interface BRI0 is disabled

Interface BRI0:1 is disabled

Interface BRI0:2 is disabled

Ethernet0 Throttle count 0 Drops RP 0 SP 0 SPD Flushes Fast 0 SSE 0 SPD Aggress Fast 0 SPD Priority Inputs 0 Drops 0

Protocol Path Pkts In Chars In Pkts Out Chars Out Other Process 0 0 42 2520 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 IP Process 20097 1248347 1067 113859 Cache misses 1124 Fast 949 118626 1054 1016786 Auton/SSE 0 0 0 0 ARP Process 90 5400 13 780 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 CDP Process 0 0 7 2203 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 Serial0 Connect to M2K-TOT Throttle count 0 Drops RP 0 SP 0 SPD Flushes Fast 0 SSE 0 SPD Aggress Fast 0 SPD Priority Inputs 0 Drops 0

Protocol Path Pkts In Chars In Pkts Out Chars Out Other Process 2 321 42 924 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 IP Process 966 84480 19367 1008945 Cache misses 42 Fast 1054 1004608 949 109136 Auton/SSE 0 0 0 0 CDP Process 7 2114 13 3774 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 Interface Serial1 is disabled

----------------

Okay after 20 minutes the IP INPUT process has risen to 58%. (Actually as we speak it is 72%) Any ideas?

---*** SHOW INT ***---- BRI0 is administratively down, line protocol is down Hardware is BRI MTU 1500 bytes, BW 64 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Last input never, output never, output hang never Last clearing of "show interface" counters 00:19:15 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/16 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions BRI0:1 is administratively down, line protocol is down Hardware is BRI MTU 1500 bytes, BW 64 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input never, output never, output hang never Last clearing of "show interface" counters 00:19:16 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/16 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions BRI0:2 is administratively down, line protocol is down Hardware is BRI MTU 1500 bytes, BW 64 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input never, output never, output hang never Last clearing of "show interface" counters 00:19:46 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/16 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions Ethernet0 is up, line protocol is up Hardware is Lance, address is 0010.7b7f.d00d (bia 0010.7b7f.d00d) Internet address is 192.168.0.1/24 MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters 00:19:48 Queueing strategy: fifo Output queue 0/40, 0 drops; input queue 50/75, 3 drops 5 minute input rate 27000 bits/sec, 55 packets/sec 5 minute output rate 56000 bits/sec, 8 packets/sec 62381 packets input, 4196425 bytes, 1 no buffer Received 479 broadcasts, 0 runts, 0 giants, 3 throttles 121 input errors, 0 CRC, 0 frame, 0 overrun, 121 ignored 0 input packets with dribble condition detected 8019 packets output, 4356946 bytes, 0 underruns 0 output errors, 10 collisions, 6 interface resets 0 babbles, 0 late collision, 10 deferred 0 lost carrier, 0 no carrier 0 output buffer failures, 0 output buffers swapped out Serial0 is up, line protocol is up Hardware is HD64570 Description: Connect to M2K-TOT Internet address is 204.57.130.54/30 MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec, reliability 255/255, txload 3/255, rxload 7/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters 00:20:07 Queueing strategy: fifo Output queue 0/40, 0 drops; input queue 0/75, 0 drops 5 minute input rate 48000 bits/sec, 4 packets/sec 5 minute output rate 22000 bits/sec, 54 packets/sec 7819 packets input, 4189807 bytes, 0 no buffer Received 148 broadcasts, 0 runts, 0 giants, 0 throttles 1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 62724 packets output, 3581057 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions DCD=up DSR=up DTR=up RTS=up CTS=up Serial1 is administratively down, line protocol is down Hardware is HD64570 MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input never, output never, output hang never Last clearing of "show interface" counters 00:20:08 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/0/256 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 0 packets input, 0 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out 0 carrier transitions DCD=down DSR=down DTR=down RTS=down CTS=down

---*** SHOW PROC CPU *****----

CPU utilization for five seconds: 66%/7%; one minute: 57%; five minutes: 46% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 152 255 596 0.00% 0.00% 0.00% 0 Load Meter 2 13752 256 53718 0.00% 0.70% 0.95% 2 Virtual Exec 3 5164 298 17328 0.00% 0.46% 0.37% 0 Check heaps 4 4 1 4000 0.00% 0.00% 0.00% 0 Chunk Manager 5 20 5 4000 0.00% 0.00% 0.00% 0 Pool Manager 6 0 2 0 0.00% 0.00% 0.00% 0 Timers 7 4 2 2000 0.00% 0.00% 0.00% 0 Serial Backgroun 8 576 243 2370 0.00% 0.03% 0.01% 0 ARP Input 9 0 3 0 0.00% 0.00% 0.00% 0 DDR Timers 10 4 2 2000 0.00% 0.00% 0.00% 0 Dialer event 11 28 2 14000 0.00% 0.00% 0.00% 0 Entity MIB API 12 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect 13 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd 14 428 257 1665 0.07% 0.02% 0.00% 0 Net Background 15 8 7 1142 0.00% 0.00% 0.00% 0 Logger 16 372 1197 310 0.00% 0.02% 0.00% 0 TTY Background 17 248 1217 203 0.00% 0.01% 0.00% 0 Per-Second Jobs 18 0 1 0 0.00% 0.00% 0.00% 0 ISDN From Driver 19 140 220 636 0.00% 0.00% 0.00% 0 Net Input 20 328 255 1286 0.00% 0.03% 0.01% 0 Compute load avg 21 1064 22 48363 0.00% 0.04% 0.05% 0 Per-minute Jobs 22 396452 13840 28645 58.48% 48.20% 39.19% 0 IP Input 23 8144 198 41131 0.00% 0.85% 0.67% 0 CDP Protocol 24 0 1 0 0.00% 0.00% 0.00% 0 X.25 Encaps Mana 25 4 2 2000 0.00% 0.00% 0.00% 0 Emulator 26 0 1 0 0.00% 0.00% 0.00% 0 PPP IP Add Route 27 596 40 14900 0.00% 0.04% 0.00% 0 IP Background 28 168 23 7304 0.00% 0.01% 0.00% 0 Adj Manager 29 40 17 2352 0.00% 0.00% 0.00% 0 TCP Timer 30 8 2 4000 0.00% 0.00% 0.00% 0 TCP Protocols 31 0 1 0 0.00% 0.00% 0.00% 0 Probe Input 32 0 1 0 0.00% 0.00% 0.00% 0 RARP Input 33 4 1 4000 0.00% 0.00% 0.00% 0 HTTP Timer 34 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers 35 16 2 8000 0.00% 0.00% 0.00% 0 DHCPD Receive 36 1724 22 78363 0.00% 0.16% 0.12% 0 IP Cache Ager 37 0 1 0 0.00% 0.00% 0.00% 0 COPS 38 0 1 0 0.00% 0.00% 0.00% 0 PAD InCall 39 8 2 4000 0.00% 0.00% 0.00% 0 X.25 Background 40 0 1 0 0.00% 0.00% 0.00% 0 ISDN Timer 42 4 2 2000 0.00% 0.00% 0.00% 0 Router Autoconf 43 0 1 0 0.00% 0.00% 0.00% 0 CallMIB Backgrou 44 0 1 0 0.00% 0.00% 0.00% 0 ISDNMIB Backgrou 45 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro 46 9864 12591 783 1.39% 0.02% 0.91% 0 IP NAT Ager 47 428 1314 325 0.00% 0.02% 0.00% 0 NTP 49 4 11 363 0.00% 0.00% 0.00% 0 DHCPD Timer 50 112 374 299 0.00% 0.00% 0.00% 0 DHCPD Database

---**** SH INT SWITCHING ****-----

Interface BRI0 is disabled

Interface BRI0:1 is disabled

Interface BRI0:2 is disabled

Ethernet0 Throttle count 4 Drops RP 4 SP 0 SPD Flushes Fast 0 SSE 0 SPD Aggress Fast 0 SPD Priority Inputs 0 Drops 0

Protocol Path Pkts In Chars In Pkts Out Chars Out Other Process 0 0 148 8880 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 IP Process 72873 4534176 4342 485961 Cache misses 1861 Fast 4789 670721 5066 4516427 Auton/SSE 0 0 0 0 ARP Process 279 16740 29 1740 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 CDP Process 0 0 25 7891 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 Serial0 Connect to M2K-TOT Throttle count 0 Drops RP 0 SP 0 SPD Flushes Fast 0 SSE 0 SPD Aggress Fast 0 SPD Priority Inputs 0 Drops 0

Protocol Path Pkts In Chars In Pkts Out Chars Out Other Process 2 321 148 3256 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 IP Process 4066 356978 71828 3733807 Cache misses 90 Fast 5066 4457101 4789 622831 Auton/SSE 0 0 0 0 CDP Process 25 7550 49 14250 Cache misses 0 Fast 0 0 0 0 Auton/SSE 0 0 0 0 Interface Serial1 is disabled

Okay, I think I may acutally have narrowed down the problem. When I unplug one segment of the LAN the "IP INPUT" process CPU utilization drops right down to 1% or 0%. This segment is connected by a very long CAT 5 line (maybe 60 meters?). There are also several computers on this segment. I am thinking there may be two causes. Either one of the computers has a virus that is designed to flood a network. Or two, the cable is so long that a lot of errors are coming through the line confusing the poor underpowered Cisco 2503. I am going to shut things off one by one now.

Thanks.

Greetings,

Are any of the hosts on the ethernet segment reporting WORSE than "normal" performance of any sort, or are things operating fairly "normal" to them?

After the Router has been running for about 15 mins, what does a "sh ip arp" display?

If your ARP table has a lot of incomplete ARP's, then yes, a machine on that segment possibly has an infection that is looking for victims, but that alone does not suggest a cause for the processor load because the traffic is not that high. However, if the ARP table looks fairly normal, then I might start to suspect a faulty transceiver or possibly some sort of Cabling/Etherswitch/Hub issue. I have seen a transceiver causing something like this, replacing it resolved the issue.

I note from a previous posting your Ethernet showed this -

Ethernet0 is up, line protocol is up Hardware is Lance, address is 0010.7b7f.d00d (bia 0010.7b7f.d00d) Internet address is 192.168.0.1/24 MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters 00:19:48 Queueing strategy: fifo Output queue 0/40, 0 drops; input queue 50/75, 3 drops These figures are pretty much "to be expected" values for everything EXCEPT the input queue looks a bit "busy" to me, compared to the traffic it is passing.

I hope this helps..............pk.

Ethernet0 is up, line protocol is up Last clearing of "show interface" counters 00:19:48 Output queue 0/40, 0 drops; input queue 50/75, 3 drops 5 minute input rate 27000 bits/sec, 55 packets/sec 5 minute output rate 56000 bits/sec, 8 packets/sec 62381 packets input, 4196425 bytes, 1 no buffer Received 479 broadcasts, 0 runts, 0 giants, 3 throttles 121 input errors, 0 CRC, 0 frame, 0 overrun, 121 ignored 8019 packets output, 4356946 bytes, 0 underruns 0 output errors, 10 collisions, 6 interface resets 0 babbles, 0 late collision, 10 deferred

As Peter says, the "input queue 50/75" i.e. 50 in queue with room for a total of 75 is anomalous.

It does not make sense given the other counters.

I suspect that the counters are mis-reporting. Even on Porcess switching the 2500 should manage MUCH more than

5 minute input rate 27000 bits/sec, 55 packets/sec or 62000 packets in 1200 seconds.

There is something weird here.

One last suggestion is to turn on ip route-cache same-interface !! IIRC

Also on some platforms sh controller displays additional error counters.

I would be interested to know the solution to this one.

Bug? Upgrade code?

The input errors on eth0 looks high to me. Could the NIC of any of your PCs by faulty, and hance flooding the segment with invalid frames? Unplug all, and then reconnect one by one, and perform your "20 minute test" again, to compare the "input errors" counter.

a 60 metre eth cable is within the allowable limit, but it may be advisable to swap it for another, or preferably move the PCs closer and connect with a shorter cable for the period of the tests, in case the cable has a fault.

Otherwise, my only suggestion (although, I don't think this is the case) is that the Ethernet0 interface is faulty - try another.

Cheers Paul