CPU is pretty much pegged (99%) and the 2 biggest culprits are "IP Input" at 52% and "Crypto PAS" at 30%. One, does anyone know exactly what specific function "Crypto PAS" is and two, does this look like just an overloaded router? The unit is servicing 3 bonded T1s with a lot of that traffic being VPN. Any suggestions?
TIA.
Juste somes ideas. Try to : show debug
If debugging is enabled, run : undebug all
Try : sh logging And see what level of logging is used
Try : sh int counter
Try : sh int | include errors see if you have errors => packet retransmitted + algorithm Crypto re-called... i don't know.
Best Regards Rahan
Is CEF enabled ?
CEF is enabled.
Misread the Crypto PAS... most of the load is IP Input.
Nate,
Your router is spending its time process switching packets instead of fast switching them.
There are many possible reasons for this ranging across configuration, the use of features that cannot be fast switched, denial of service attack, or the router may be subject to traffic that the original network design did not cater for.
Since the scope is so wide it would be best of you could post.
sh runn ! (recommend that you remove sensitive material) sh int ! (right now)
when the CPU is high clear counters wait a few minutes sh int
One common gotcha is that packets routed in and out the same interface are not fast switched by default. If your router is also doing local routing then you may see this.
int xxxx ip route-cac same-interface ?? From memory
As a first cut, your router should be able to handle a few T1s, there is something wrong.
snipped-for-privacy@hotmail.com wrote:
There are actually 2 2801s with bonded T1s running a tunnel between the two. I tried the ip route-cache statements but they didn't seem to change anything. Interestingly, the statements don't show up in the config on the one 2801 but they do on the other 2801 (slightly different IOS versions and different defaults?).
This is the partial config (specifics removed) from the 2801 with the pegged CPU and 3 T1s... the other router sits around 50% usage and has
2 T1s but basically has the same problem. Both have about the same amount of load of them but the 3 T1 router has the higher CPU usage (higher CPU load for load sharing the 3 T1s as opposed to load sharing 2?).Current configuration : 11700 bytes ! version 12.3 service timestamps debug datetime msec service timestamps log datetime msec no service password-encryption ! hostname ! boot-start-marker boot system flash c2801-advsecurityk9-mz.123-11.T8.bin boot-end-marker ! logging buffered 8192 debugging enable secret 5 ! clock timezone NewYork -5 clock summer-time NewYork date Apr 6 2003 2:00 Oct 26 2003 2:00 no network-clock-participate wic 1 no network-clock-participate wic 2 mmi polling-interval 60 no mmi auto-configure no mmi pvc mmi snmp-timeout 180 no aaa new-model ip subnet-zero ip cef ! ! ip inspect name DEFAULT100 icmp timeout 10 ip inspect name DEFAULT100 netshow ip inspect name DEFAULT100 cuseeme ip inspect name DEFAULT100 streamworks ip inspect name DEFAULT100 udp ip inspect name DEFAULT100 tcp ip inspect name DEFAULT100 ftp ip inspect name DEFAULT100 h323 ip inspect name DEFAULT100 sqlnet ip inspect name DEFAULT100 rcmd ip inspect name DEFAULT100 realaudio ip inspect name DEFAULT100 vdolive ip inspect name DEFAULT100 tftp ip inspect name DEFAULT100 rtsp ip inspect name DEFAULT100 http urlfilter ! ! ip ips notify SDEE ip ips po max-events 100 no ip domain lookup ip domain name ip urlfilter allow-mode on no ftp-server write-enable ! ! ! ! controller T1 0/1/0 framing esf linecode b8zs channel-group 0 timeslots 1-24 speed 64 ! controller T1 0/1/1 framing esf linecode b8zs channel-group 0 timeslots 1-24 speed 64 ! controller T1 0/2/0 framing esf linecode b8zs channel-group 0 timeslots 1-24 speed 64 ! controller T1 0/2/1 framing esf linecode b8zs channel-group 0 timeslots 1-24 speed 64 ! ! crypto isakmp policy 1 encr aes 256 authentication pre-share group 2 ! crypto isakmp policy 3 encr 3des authentication pre-share group 2 crypto isakmp key crypto isakmp key no crypto isakmp ccm ! ! crypto ipsec transform-set esp-aes-sha esp-aes 256 esp-sha-hmac crypto ipsec df-bit clear ! crypto map SDM_CMAP_1 1 ipsec-isakmp description Tunnel set peer set peer set transform-set esp-aes-sha match address 102 ! ! ! interface Tunnel1 no ip address ! interface Loopback0 ip address 1.1.1.1 255.255.255.0 ! interface Multilink1 description $FW_OUTSIDE$ ip address ip access-group 103 in ip verify unicast reverse-path ip inspect DEFAULT100 out ip nat outside ip virtual-reassembly ppp multilink ppp multilink group 1 crypto map SDM_CMAP_1 ! interface FastEthernet0/0 ip address ip access-group 100 in ip nat inside ip virtual-reassembly ip policy route-map portmap_bypass duplex auto speed auto ! interface FastEthernet0/1 description $ETH-LAN$ ip address duplex auto speed auto ! interface Serial0/1/0:0 no ip address encapsulation ppp ppp multilink ppp multilink group 1 ! interface Serial0/1/1:0 no ip address encapsulation ppp ppp multilink ppp multilink group 1 ! interface Serial0/2/0:0 no ip address encapsulation ppp ppp multilink ppp multilink group 1 ! interface Serial0/2/1:0 no ip address ! ip classless ip route ! ip http server ip http access-class 1 ip http authentication local ip http secure-server ip http timeout-policy idle 5 life 86400 requests 10000 ! ip access-list extended OUTBOUND remark SDM_ACL Category=2 remark IPSec Rule deny ip 10.0.0.0 0.0.0.255 192.168.0.0 0.0.0.255 permit ip any any remark SDM_ACL Category=2 remark IPSec Rule remark SDM_ACL Category=2 remark IPSec Rule remark SDM_ACL Category=2 remark IPSec Rule remark SDM_ACL Category=2 remark IPSec Rule ! access-list 1 remark Auto generated by SDM Management Access feature access-list 1 remark SDM_ACL Category=1 access-list 1 permit access-list 1 permit access-list 1 permit
disable-eadi route-map portmap_bypass permit 10 match ip address 110 set ip next-hop 1.1.1.2 ! route-map SDM_RMAP_1 permit 1 match ip address OUTBOUND ! route-map nonat permit 10 match ip address 120 ! ! ! control-plane ! ! line con 0 logging synchronous login local line aux 0 line vty 0 4 privilege level 15 login local transport input telnet transport output none ! end
what is the state of the AIM ?
post output of show crypto engine config
More info. I just compared the process/fast switching on the 2 routers... the one router is mostly fast switching while the other is not. The one that is not is the one that, although it seems to take the ip-route statements, they don't show up in the config. Any idea what could cause this?
Some "default" conditions do not show up in the config.
When CEF is off
"ip route-cache" is the default so it doesn't show
when CEF is on "ip route-cache cef" (somthing like that) is the default so "ip route-cache" does show??
Of maybe you have netflow.
Anyway the answer to the last question is in there somewhere.
You need to check that your policy routing configuration can be fast switched. IIRC some policy routing can and some can't.
Multi-chassis MPPP - wow. Never seen that in action before. Can that be Fast switched?
Check carefully for local traffic going in and out of the router. You can check the interface counters, what comes in must go out. Usually.
that is why I suggested posting
wait 'til CPU high clear counters wait 10 mins sh int
Make sure that CEF is on, on both routers.
The same interface thing that I mentioned earlier may be important with Multi-chassis MPPP.
Traffic comes in to the default gw (say) then MC-MPPP fires it back out the same interface.
An idea anyway.
Just turn it on and try it.
int xxx ip route-cache same-interface
Is MC-MPPP fast switched?
Assuming we're talking about the same thing, we're not using multi-chassis MPPP. The 2 routers are at different locations, pretty much configured the same, and linking their networks via VPN. One is using 2 bonded Ts and the other 3 Ts. Sorry about the misdirect.
I think you're right about the fast switching. The router with the CPU pegged is not fast switching like the other one and it may be the policy routing. I think policy routing is only being used on the pegged router but I need to check again.
Thanks for the input and I'll keep trying things until I (hopefully) figure out what's causing it.
Hi Nate,
You may wish to investigate Troubleshooting High CPU Utilization on Cisco Routers:
Brad Reese BradReese.Com - Cisco Technical Forums
Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.