Option11c SLIPD problems (Calls Dropped)

I would put in CSU's to start (where I am telco won't even take you seriously with out them) I would also look at whatever you are using for the D-channel since you don't state what type of cards you are using it might be worth trying the D-channel card first.

First, do an ssck 0 in LD 60. See if you are tracking, or free run, and if there is tracking errors. You might just need to do a trck pck. Also, you might have a defective replacement clock controller, or PRI card. Is it TMDI or regular?

Next make sure that the clock controller is on the PRI on the correct slot..... Pref might be slot 1, while the CC is on the card in slot 2. You should be able to swap the PRI cards, and put the CC on the older PRI card. This would rule out (or in) the PRI card as being the source of the problem. If the PRI card isn't providing timing to the CC, you should have an error when you do a ssck 0. Make sure the CC is on the PRI card in the correct slot if you swap things.

If you disable the CC (dis cc 0), does the CC status light change on the PRI card?

Here's how things are supposed to work as a deep level: The PRI card extracts a timing reference from the incoming T1. It provides a clock (frequency) to the Clock controller. The clock controller uses on onboard computer to match it's oscillator to the extracted clock (phase lock). The clock controller takes the output of this oscillator, and provides it back out to the backplane of the switch. The switch synchs all the codecs, and DTI/PRI cards to this clock source that was ultimately derived from the incoming T1. The end result should be that the transmit timing on all the PRI cards is in synch with whatever T1 is used as the reference.....

So, the point of the problem could be no clock from the PRI card to the CC (CC error when ssck 0'd), clock controller can't interpret the clock it's given (CC error when ssck 0'd), CC can't control onboard oscillator (different CC error when ssck 0'd), CC locks, but timing doesn't make it to backplane of switch due to burned trace, line getting dragged down by somthing else damaged by lightning, but not replaced (CC stats good, green light on faceplate, and maybe you get DTC10x, etc) This would result in massive timing slips, which is what might be wrong. Generally, even if a switch is free-running, it doesn't slip so bad that it instantly kills the T1.

As far as your NI (smartjack) PBX with no CSU arrangement goes, the CSU does two things for you, so you should (and are legally (FCC) required to) have one, but sometimes you can get away without one. It gives the Telco something to loop back past their interface for testing, and subsequently placing blame if they can loop the NI, but they can't loop a CSU. It also provides a level of isolation (electrical) that you don't have. Depending on the NI,and it's configuration, you may actually be connected to the incoming cable pairs when the NI isn't looped. A CSU always has transformer isolation from input to output. A CSU - or lack thereof (the plain-jane variety) has nothing to do with timing slips. You might have framing hits, and BPV's, etc if the level out of the NI isn't close enough to DSX, but it appears you have a timing issue. Also, unless the whole CO were timing off frequency, and you had T1's the went two different places - one on frequency, the other not - you wouldn't see slips. If the CO was in free run, and you were tracking properly, your switch would just blindly follow - even if they were WAY (like 5 hz) off frequency.

Please report back.......

Bob,

This all started after a lightning strike took out one of the PRI Cards. We replaced the PRI card, Clock Controller & DChannel card. (The PRI Cards are NOT the TMDI version) (System has two PRI Cards w/2 DChannels)

USLec came out with their test equipment and it shows the incoming clocking frequency at 1544000 and the phone system frequency at 1543998 (2 Hz off). If the incoming frequency changes to 1544001 the phone frequency shifts also to 1543999 again the 2 Hz off.

The CC is currently in Slot 1. During testing we moved it to slot 2, Replaced with a different PRI card, CC card & DChannel card. We then swapped out the cabinet and power supply. Still slipping.

The only thing left I can think of might be the ssc card. Any ideas?

Thanks, StinkMan

Regardless of what the test set shows, the frequency from telco IS 1544000. Anything slightly different is just an inacuaracy in the test set timebase. That's why the switch - even though it's 2 Hz low - appears to follow. When you moved the CC to slot 2, did you change the reference data in LD 73 DDB?

Also, post the response you get when you do a ssck 0. The SSC isn't likely to be the cause unless... But, then again, it's possible.

With no DTI (actually, no Clock Controller) in the system, the SSC generates the timing clock used by the entire system.

The CC on the DTI/PRI card generates a more accurate (even before it is tracking a T1) clock than the oscillator on the SSC. When a CC (and the clock it generates) is present, the SSC is supposed to take that as an input, and redistribute that clock. If the input to the SSC (or the backplane is burned open) is damaged, it may never take that clock input. I haven't seen this happen, and I have seen a fair number of lightning damaged machines, but it's possible. One way to tell is to look at the transmit frequency of the second PRI while you disable CC, and PRI and remove the other card. If there isn't a significant change in frequency, and you've swapped the CC and PRI card with a good one, you are likely dealing with a SSC or backplane problem. The good news is, you can just move the daughterboard, (and Ibutton) to another SSC, with no reinstall necessary. Here's the catch though.... If you are running 25, or 3.0 (or up), make sure that the SSC has the correct firmware on it, or it won't come up.

If you have no errors when you do a ssck 0, and if you can disable the CC ,and light the light on the DTI card, it's likely that the issue is SSC, or backplane.

I'm trying to figure out what this frequency test proves about your incoming service (and I may be displaying my ignorance here). The whole point of D-channel Slips is that they are often random -- like bursts of noise. I once had to work on an Option 11c which had a PRI that went down every Friday afternoon from a burst of D-Channel Slips. I inherited the problem from another tech who had worked on it for months (at least he was supposed to be working on it). I kept getting bounced from the Circuit support side of (what was) Southwest Telecom and the carrier support side. "No problem here." "No problem here." Finally I told one lady, "Look, you're playing ping pong with me." So we got a three way call. They *finally* checked their logs in the CO, and sure enough they showed problems every Friday -- the same times the PRI went down. "Only a few seconds each time, though." "That's all it takes." They changed to a back-up circuit/path and the problem went away.

Have you checked your (and/or the LECs) CSU? I have had several of them cause PRI problems like this.

Bob,

There are no errors when you issue the ssck 0 command. When you disable the CC the led lights up on the PRI card. We have replaced/swapped the backplane with the second cabinet. This client is

2 hrs from our office. I've got a SSC card in another switch at the office and a cc,dchannel,pri combo being delivered on Thursday. I'm going to replace the PRI,CC,DChannnel one more time if that fails I'm planning on swapping the SSC out as my next troubleshooting step. If these actions don't resolve it I'm going to mandate a new complete circuit from USLec.

I greatly appreciate your lengthy and informative input. Even if I don't get the damn thing working I've definitely learned a few things here.

Thanks to all for their informative ideas and suggestions.

Thanks, StinkMan

RonB,

I had a similier issue with a client. Circuit went down every other Wednesday. Come to find out the phone cables outside where getting hit by the weed wacker when the lawn service crew came by.

Thanks... StinkMan

Good luck. I didn't realize there were answers further back in the thread -- from someone a lot more knowlegable on PRIs than I have.

I think that the test set has shown that the incoming carrier isn't way off frequency. If it were, that would be caused by a transport problem (HDSL, SONET, etc.), not the central office. Their CO switch is locked locally or remotely to an atomic clock, and they'd know they had a problem. The Option

11 system should track at least +/- 10Hz., provided that it isn't varying wildly, which would have shown up on the T-berd, or whatever they were using, and you'd know it if you saw that. In other words, the switch should be tracking.

When you moved the CC, did you change the DDB to match?

Do you ever get a DTC102 /103 out of the system that indicates it thinks it's tracking. and then losing it? That is the result of the CC processor believing it has achieved phase lock with the derived clock from the DTI / PRI card. This is kind of what is on the cable that would go from a PRI card to a clock controller in a large switch. Just a frequency that has been extracted from the incoming T1. The CC messes with the steering voltage line going into the VCTCXO(the metal can) on the CC, until the 16.384Mhz clock divided down aligns with the derived incoming bit rate. There's learning over time involved too, so if the system goes into free run, it's close enough for a while...

The math for the way that frequency fits into the big picture.

16.384Mhz / 2048 = 8000 Which is the PCM sample rate.

So, 8000 samples per second * 8 bits per sample = 64000 bits per second per timeslot.

64000 * 24 channels = 1536000 bits per second.

But, you've got framing to deal with, so...

8 bits per sample * 24 channels = 192 data bits + 1 framing bit = 193 bits per frame. 193 bits per frame * 8000 samples per second = Tada.... 1544000 bits per second.

This might not help you fix the problem, but it might help take away some of the mystery.

Slip D (deletion) means that your system clock is running slower than the incoming T1, which you have seen is true. This means that an incoming frame has to be thrown away periodically because the system only buffers 1 frame (192 bits). So, if you are in fact 2 Hz low, you would trash a frame every (193 / frequency error) or every 96 seconds. After 10 minutes, you'd be at a Slip D of about 6.

Close?

I saw someone else mention D-Channel slips. The fact that this is a PRI really has no bearing on the issue. The D-Channel can be really messed up, and drop, and drop calls as a result of timeouts, but it doesn't affect the T1 slip counter. And, there's all kinds of things that can mess with the D-Channel... But, that is a separate error counter .... plog dch x in 96.