Anyone know how to Analyze "crashinfo" files from Cisco access points?

Hello ...

We have a number of Cisco AP350 wireless access points thoughout our network (as well as some AP1100s and AP1210s, but these aren't involved in the problem that's causing me to send this message).

One of them seems to have started periodically crashing and resetting this past Monday evening (2006/08/28, 21:57:51, according to the timestamp on the crashinfo file), and is presently apparently looping between functioning and not. I have 10 crashinfo files and an r0.rcore file from it, and would like to try and figure out what the problem is.

Can anyone point me to information regarding how these should be used for crash analysis?

The syslog from this device is presently cycling through the following messages. I think I'm going to disconnect the device at this point ...

Aug 30 12:40:21 ap-gm401-90 573: Aug 30 12:40:21: %DOT11-2-RADIO_FAILED: Interface Dot11Radio0, failed - cmd busy Aug 30 12:40:21 ap-gm401-90 574: -Traceback= 4DC04 4511C4 45110C 4532B0 6AA4DC 6AA778 6AF00C 4503EC 4507C8 450B1C 1529D8 157988 Aug 30 12:40:22 ap-gm401-90 575: Aug 30 12:40:21: %LINK-3-UPDOWN: Interface Dot11Radio0, changed state to down Aug 30 12:40:22 ap-gm401-90 576: Aug 30 12:40:22: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down Aug 30 12:40:46 ap-gm401-90 577: Aug 30 12:40:45: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset Aug 30 12:40:48 ap-gm401-90 578: Aug 30 12:40:47: %LINK-3-UPDOWN: Interface Dot11Radio0, changed state to up Aug 30 12:40:48 ap-gm401-90 579: Aug 30 12:40:48: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up

Any ideas would be greatly appreciated ...

Reply to
Sylvain Robitaille
Loading thread data ...

Reply to
gene martinez

Ther eare tools on the Cisco web site but I am pretty sure that you need a login.

"output interpreter"

formatting link

Reply to
Bod43

There is a traceback message amongst those you posted.

A traceback almost invariably means a bug.

Is the AP350 run IOS or VXworks ?

Do you have a Cisco support contract for your APs ?

Power cycle the AP if you have not already.

Reply to
Merv

Yes, and a number of them in the crashinfo files as well.

Thanks, I imagined as much. Incidentally, I turned the radio off on this unit, and it appears to have stopped crashing. I'm not sure whether that helps pinpoint the problem in hardware or software, but I find it interesting.

Cisco IOS Software, C350 Software (C350-K9W7-M), Version 12.3(8)JA, RELEASE SOFTWARE (fc2)

Yes, but I wanted to get an idea of what the problem is before submitting a problem report.

I did (via PoE), but it didn't help ...

Reply to
Sylvain Robitaille

I imagine I'll need to do that, but would like to have some idea of the problem first. Thanks, though.

Reply to
Sylvain Robitaille

Thanks for that pointer. We do have a support contract, and I'll see if downloading such tools is included. This might help ...

Reply to
Sylvain Robitaille
12.3(8)JA is under software advisory and has been replaced by 12.3(8)JA2.

One of the first things the Cisco TAC will ask you to do is to upgrade to 12.3(8)JA2

Do you have other AP350 running this release?

How long had the AP350 with the problem been running with 12.3(8)JA?

What release was it running before that ?

Reply to
Merv

Sigh. That doesn't surprise me if the new version is available; there is some possibility that it will resolve the problem, given the symptom strongly suggests "software bug".

Another 112 of them, yes.

Months. I don't recall exactly when I performed the upgrade (I do have that recorded somewhere, though I'm not at work as I write this), but it was late March.

Based on archived configuration files, I can say for sure that it was running IOS, but prior to upgrading all the access points I had numbers of access points running different versions of IOS, and some running different versions of VxWorks (if that doesn't suggest "inherited installation" to you, allow me to disclaim that "it was like that when I got it!"). I don't recall having made an exact list then of which access points were running what software versions (and have already wished I had ...)

Once I had a good grasp of the size of the installation, and the different models of equipment we have (AP350s, AP1100s, and AP1210s), with which software versions we were running, I undertook first to convert all VxWorks access points to IOS, then to upgrade all access points to the then-latest version of IOS. I've had very little trouble since (two access points that were routinely power-cycling prior to upgrades have still been exhibiting that behaviour since). The problem I'm reporting here is very sudden.

I'll have a look for 12.3(8)JA2, and peruse release notes ...

Thanks.

Reply to
Sylvain Robitaille

Where this dlink or some other vendor with "free" support that'd have merit. But if you've got that many cisco units you'd probably be better off calling them FIRST instead of waiting for the usenet rabble to chime in...

Reply to
Bill Kearney

For 12.3(8)JA2, take a close look at CSCsb00606

You might be better to drop back to 12.3(7)JA3 on the troubled unit.

Also note that the first maintenance release usually gets deferred ( read junked):

12.3.8-JA2

12.3.7-JA3

12.3.7-JA2 12.3.7-JA1

12.3.4-JA2

12.3.4-JA1 12.3.4-JA

12.3.2-JA6

12.3.2-JA5 12.3.2-JA2

I normally like to wait for the third or fourth maintenenace release depending on the technology.

Reply to
Merv

Hrmmm... Thanks for that.

Will have a look. Thanks again.

Reply to
Sylvain Robitaille

I may have the wrong end of the stick here, do you mean for example 12.3.2-JA4 is the fourth maint. rel?

I am sure that Merv has more Cisco experience than I have however, I do not like the idea of what amount to unplanned releases. ...JAx where x = 1, 2, 3, ... are I understand fixes that are applied to software that is supposed to be working.

I fancy releases that do not need so much fixing.

12.3.2-JA6 just makes my skin crawl.

As I say this is only a wild guess with absolutely no evidence either way. I just feel that it looks like a Friday afternoon product. Maybe it can be fixed, but why not just stay away if that is an option?

Reply to
Bod43

Well in the "old days" before IOS version "trains" the number in between the ( ) was the maintenance release - things were much simpler than.

12.3JA is known as a STED - specific technology early deployment release - and as far as I can tell the number after the JA sure looks and feels like a maintenance as one can tell by looking carefully at the release notes.
Reply to
Merv

~ Merv wrote: ~ ~ > For 12.3(8)JA2, take a close look at CSCsb00606 ~ ~ Hrmmm... Thanks for that.

CSCsb00606 was actually filed against 12.3(2)JA. It doesn't look like anything's happened with that one in a year. I don't suppose it would pertain to current IOS.

~ > You might be better to drop back to 12.3(7)JA3 on the troubled unit. ~ ~ Will have a look. Thanks again.

Both 12.3(7)JA3 and 12.3(8)JA2 are good solid releases. Btw, we just posted 12.3(8)JEA today, if you really want to run the latest and greatest.

That said, it's not clear to me (after scanning known bugs) whether the condition you encountered that triggered the radio failure is going to be addressed by upgrading software.

Regards,

Aaron

Reply to
Aaron Leonard

To original question - try this website. I know the proprietor is on holiday from Monday for a couple of weeks but might respond after this.

formatting link

All the best,

Martyn (you can tell him I advised you contact him if you like!).

Reply to
Martyn Thompson

I think that it is like this:-

a.b(cx)JAy

JA tells us that it is a special (required for for many purposes) Avoid specials of you can These releases are based on a mainstream release in this case a.b(cx).

a - Major release b - Minor release (c) - Maintenance release - in some trains can contain new features x - Re-release of a.b(c) [On which JA is based] y - Re-release of a.b(c)JA - changes not affecting base a.b(c)

x is always a lower case letter y is always a number

As I say, I worry about re-releases but have no direct IOS derived evidence on which to base it.

Reply to
Bod43

Those releases as you call them are maintenance release pure and simple.

Look at the release notes for 12.3(8)JA2 and you will see that DDTS listed that are resolved between it and the previosu release.

Reply to
Merv

In my experience, that's also been true of 12.3(8)JA, save for a small number of units that have been acting up (the one we're discussing here that failed only recently, and a couple of others that have been acting up since before the upgrade to 12.3(8)JA).

No, I'm more interested in running the same version across all our access points. Only when it becomes necessary to upgrade (due to serious bugs, or specific new features, or as I did earlier this year, to bring all our units up to the same version) do I start looking at upgrading to the most current version (or ${most_current}-1, if ${most_current} hasn't been around long enough to get a bit of a shakedown).

You're probably right. We've started discussing replacing that unit with a newer (AP1200) one. That's very likely what I'll do.

Reply to
Sylvain Robitaille

Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.