Cisco 851/%SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed|
|
|
Bookmark this page:
 Yahoo!
 Google
 Windows Live
 del.icio.us
 digg
 Netscape
|
|
||||||||||
|
Posted by on December 26, 2008, 10:28 am
Please log in for more thread options
First time posting here. I have a new Cisco 851 router. Like clockwork, every 4 days I find the following in the SYSLOG on my main server: Dec 25 15:44:48 Cisco 000064: *Dec 25 09:54:01.476 PCTime: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x8011A088, alignment 32 Dec 25 15:44:48 Cisco 69: Pool: Processor Free: 79776 Cause: Memory fragmentation Dec 25 15:44:48 Cisco 70: Alternate Pool: None Free: 0 Cause: No Alternate pool Dec 25 15:44:48 Cisco 71: Dec 25 15:44:48 Cisco 72: -Process= "ARP Input", ipl= 0, pid= 8 Dec 25 15:44:48 Cisco 73: -Traceback= 0x8054EE44 0x800F750C 0x800FD13C 0x8011A08C 0x803A69B4 0x807036A4 0x808F7C30 0x808F8250 0x808F8EDC 0x803B2F54 0x800DDF38 0x800E11F8 Dec 25 15:44:49 Cisco 74: 000065: *Dec 25 09:54:01.476 PCTime: %FIB-3-NOMEM: Malloc Failure, disabling CEF Dec 25 15:44:49 Cisco 75: -Traceback= 0x8054EE44 0x803A69FC 0x807036A4 0x808F7C30 0x808F8250 0x808F8EDC 0x803B2F54 0x800DDF38 0x800E11F8 As you can see, I received a Christmas present yesterday. I've searched the Cisco support forums and I've googled this error, and I have not found any real solution. When this happens, the router is inac- cessible and has to be reloaded from the serial console or power-cycled. Can anyone here offer a clue or solution? I am no IOS power user either, so please reply with cogent instructions. Thanks. -- VAXman- A Bored Certified VMS Kernel Mode Hacker VAXman(at)TMESIS(dot)ORG "Well my son, life is like a beanstalk, isn't it?" | ||||||||||
|
Posted by bod43 on December 26, 2008, 6:51 pm
Please log in for more thread options > =A0from 0x8011A088, alignment 32
fragmentation
> Dec 25 15:44:48 Cisco 69: Pool: Processor =A0Free: 79776 =A0Cause: Memory= > Dec 25 15:44:48 Cisco 70: Alternate Pool: None =A0Free: 0 =A0Cause: No Al=
ternate pool
> Dec 25 15:44:48 Cisco 71:
0x8011A08C 0x803A69B4 0x807036A4 0x808F7C30 0x808F8250
> Dec 25 15:44:48 Cisco 72: -Process=3D "ARP Input", ipl=3D 0, pid=3D 8 > Dec 25 15:44:48 Cisco 73: -Traceback=3D 0x8054EE44 0x800F750C 0x800FD13C = > =A00x808F8EDC 0x803B2F54 0x800DDF38 0x800E11F8
EM: Malloc Failure, disabling CEF
> Dec 25 15:44:49 Cisco 74: 000065: *Dec 25 09:54:01.476 PCTime: %FIB-3-NOM= > Dec 25 15:44:49 Cisco 75: -Traceback=3D 0x8054EE44 0x803A69FC 0x807036A4 =
0x808F7C30 0x808F8250 0x808F8EDC 0x803B2F54 0x800DDF38
> =A00x800E11F8
c-
> > As you can see, I received a Christmas present yesterday. > > I've searched the Cisco support forums and I've googled this error, and I > have not found any real solution. =A0When this happens, the router is ina= > cessible and has to be reloaded from the serial console or power-cycled.
r,
> > Can anyone here offer a clue or solution? =A0I am no IOS power user eithe= > so please reply with cogent instructions.
> > Thanks. Obviously - the router has exhausted its memory. If you check with sh mem router#sh mem Head Total(b) Used(b) Free(b) Lowest(b) Largest(b) Processor 83099124 70676188 27469044 43207144 40316708 38171256 I/O 7400000 12582912 3925532 8657380 8530336 8537212 and worry about the Processor, free, lowest, largest and also compare how that changes in time against sh proc mem entries per process you may be able to work out which process is capturing all of the memory. Since you have already been notified that an ARP process is the one failing the allocation I would check that you are not doing proxy arp on your internet facing interface, or for that matter on th inside interface. int fa 4 ! If I recall correctly no ip proxy-arp If none of this makes any sense post sh proc mem over several intervals before the router crashes. | ||||||||||
|
Posted by on December 26, 2008, 7:35 pm
Please log in for more thread options >On 26 Dec, 15:28, VAXman- @SendSpamHere.ORG wrote:
>> Hi,
>> >> First time posting here. >> >> I have a new Cisco 851 router. =A0Like clockwork, every 4 days I find the >> following in the SYSLOG on my main server: >> >> Dec 25 15:44:48 Cisco 000064: *Dec 25 09:54:01.476 PCTime: %SYS-2-MALLOCF= >AIL: Memory allocation of 65536 bytes failed
>> =A0from 0x8011A088, alignment 32
>> Dec 25 15:44:48 Cisco 69: Pool: Processor =A0Free: 79776 =A0Cause: Memory= > fragmentation
>> Dec 25 15:44:48 Cisco 70: Alternate Pool: None =A0Free: 0 =A0Cause: No Al=
>ternate pool
>> Dec 25 15:44:48 Cisco 71:
>> Dec 25 15:44:48 Cisco 72: -Process=3D "ARP Input", ipl=3D 0, pid=3D 8 >> Dec 25 15:44:48 Cisco 73: -Traceback=3D 0x8054EE44 0x800F750C 0x800FD13C = >0x8011A08C 0x803A69B4 0x807036A4 0x808F7C30 0x808F8250
>> =A00x808F8EDC 0x803B2F54 0x800DDF38 0x800E11F8
>> Dec 25 15:44:49 Cisco 74: 000065: *Dec 25 09:54:01.476 PCTime: %FIB-3-NOM= >EM: Malloc Failure, disabling CEF
>> Dec 25 15:44:49 Cisco 75: -Traceback=3D 0x8054EE44 0x803A69FC 0x807036A4 =
>0x808F7C30 0x808F8250 0x808F8EDC 0x803B2F54 0x800DDF38
>> =A00x800E11F8
>> >> As you can see, I received a Christmas present yesterday. >> >> I've searched the Cisco support forums and I've googled this error, and I >> have not found any real solution. =A0When this happens, the router is ina= >c-
>> cessible and has to be reloaded from the serial console or power-cycled.
>> >> Can anyone here offer a clue or solution? =A0I am no IOS power user eithe= >r,
>> so please reply with cogent instructions.
>> >> Thanks. >
>Obviously - the router has exhausted its memory. > >If you check with sh mem >router#sh mem > Head Total(b) Used(b) Free(b) Lowest(b) >Largest(b) >Processor 83099124 70676188 27469044 43207144 >40316708 38171256 > I/O 7400000 12582912 3925532 8657380 >8530336 8537212 > >and worry about the Processor, free, lowest, largest >and also compare how that changes in time against >sh proc mem entries per process > >you may be able to work out which process is capturing all of the >memory. > >Since you have already been notified that an ARP >process is the one failing the allocation I would check that >you are not doing proxy arp on your internet facing >interface, or for that matter on th inside interface. > >int fa 4 ! If I recall correctly >no ip proxy-arp I have this on Interface FastEthernet4 as well as Vlan1. >If none of this makes any sense post
> >sh proc mem > >over several intervals before the router crashes. I'll try. I found one discussion about this and added some arp timeout values that should free up the cached translations, I hope. I set FastEthernet4 to 300 seconds and Vlan1 to 900 seconds. I'll be watching the memory over the next couple of days. Any other suggestions, I'm all ears at this point having pretty much ex- hausted other paths. -- VAXman- A Bored Certified VMS Kernel Mode Hacker VAXman(at)TMESIS(dot)ORG "Well my son, life is like a beanstalk, isn't it?" | ||||||||||
|
Posted by on December 26, 2008, 8:43 pm
Please log in for more thread options writes:
>>On 26 Dec, 15:28, VAXman- @SendSpamHere.ORG wrote:
>>> Hi,
>>> >>> First time posting here. >>> >>> I have a new Cisco 851 router. =A0Like clockwork, every 4 days I find the >>> following in the SYSLOG on my main server: >>> >>> Dec 25 15:44:48 Cisco 000064: *Dec 25 09:54:01.476 PCTime: %SYS-2-MALLOCF= >>AIL: Memory allocation of 65536 bytes failed
>>> =A0from 0x8011A088, alignment 32
>>> Dec 25 15:44:48 Cisco 69: Pool: Processor =A0Free: 79776 =A0Cause: Memory= >> fragmentation
>>> Dec 25 15:44:48 Cisco 70: Alternate Pool: None =A0Free: 0 =A0Cause: No Al=
>>ternate pool
>>> Dec 25 15:44:48 Cisco 71:
>>> Dec 25 15:44:48 Cisco 72: -Process=3D "ARP Input", ipl=3D 0, pid=3D 8 >>> Dec 25 15:44:48 Cisco 73: -Traceback=3D 0x8054EE44 0x800F750C 0x800FD13C = >>0x8011A08C 0x803A69B4 0x807036A4 0x808F7C30 0x808F8250
>>> =A00x808F8EDC 0x803B2F54 0x800DDF38 0x800E11F8
>>> Dec 25 15:44:49 Cisco 74: 000065: *Dec 25 09:54:01.476 PCTime: %FIB-3-NOM= >>EM: Malloc Failure, disabling CEF
>>> Dec 25 15:44:49 Cisco 75: -Traceback=3D 0x8054EE44 0x803A69FC 0x807036A4 =
>>0x808F7C30 0x808F8250 0x808F8EDC 0x803B2F54 0x800DDF38
>>> =A00x800E11F8
>>> >>> As you can see, I received a Christmas present yesterday. >>> >>> I've searched the Cisco support forums and I've googled this error, and I >>> have not found any real solution. =A0When this happens, the router is ina= >>c-
>>> cessible and has to be reloaded from the serial console or power-cycled.
>>> >>> Can anyone here offer a clue or solution? =A0I am no IOS power user eithe= >>r,
>>> so please reply with cogent instructions.
>>> >>> Thanks. >>
>>Obviously - the router has exhausted its memory. >> >>If you check with sh mem >>router#sh mem >> Head Total(b) Used(b) Free(b) Lowest(b) >>Largest(b) >>Processor 83099124 70676188 27469044 43207144 >>40316708 38171256 >> I/O 7400000 12582912 3925532 8657380 >>8530336 8537212 >> >>and worry about the Processor, free, lowest, largest >>and also compare how that changes in time against >>sh proc mem entries per process >> >>you may be able to work out which process is capturing all of the >>memory. >> >>Since you have already been notified that an ARP >>process is the one failing the allocation I would check that >>you are not doing proxy arp on your internet facing >>interface, or for that matter on th inside interface. >> >>int fa 4 ! If I recall correctly >>no ip proxy-arp >
>I have this on Interface FastEthernet4 as well as Vlan1. > > >>If none of this makes any sense post
>> >>sh proc mem >> >>over several intervals before the router crashes. >
>I'll try. > >I found one discussion about this and added some arp timeout values that >should free up the cached translations, I hope. I set FastEthernet4 to >300 seconds and Vlan1 to 900 seconds. I'll be watching the memory over >the next couple of days. > >Any other suggestions, I'm all ears at this point having pretty much ex- >hausted other paths. I found it. I removed: # ip route 0.0.0.0 0.0.0.0 FastEthernet4 and replaced it with: # ip route 0.0.0.0 0.0.0.0 24.xx.yy.zz (IP address up upstream router,) I removed the timeouts that I had installed too. # show process memory 8 has shown a rock steady memory use now at 7176 bytes. -- VAXman- A Bored Certified VMS Kernel Mode Hacker VAXman(at)TMESIS(dot)ORG "Well my son, life is like a beanstalk, isn't it?" | ||||||||||
|
Home Cabling GuideFinally, an instantly downloadable book that saves you thousands in home improvement dollars! Enjoy living in 21st century technology-advanced home while increasing its selling value and competitive advantage on the real estate market. Whether your cabling is for home office or high-tech leisure, you can wire your home yourself or learn "wirish" to speak with your cabling contractors in their language! Learn More |
>
> First time posting here.
>
> I have a new Cisco 851 router. =A0Like clockwork, every 4 days I find the
> following in the SYSLOG on my main server:
>
> Dec 25 15:44:48 Cisco 000064: *Dec 25 09:54:01.476 PCTime: %SYS-2-MALLOCF=