This discussion came up on twitter with @amyengineer when discussing odd gateway bugs and I had a couple of requests to blog this out as this is just TOO classic to lose to time.

Let me set the stage: It was approx 2003/2004. I was taking a break from the usual Callmanager/Uni craziness and was assigned to install a CME implementation with the JUST real eased Cisco Unity Express 1.1. ( I had installed 1.0 a few weeks before and was VERY happy they had fixed that little browser logout issue!).

Nothing special or crazy about this install. No ACD functionality was in CME yet, and this SHOULD have been a slam dunk.

I’m doing the typical day2 support calls. Cleaning up some fax issues, etc… And then I get a call.

“The system is randomly dropping calls and it seems to be happening more to the woman. “

Of course, you can imagine my reaction ” Are you seriously trying to imply that my phone system is sexist?” (sarcasm) ” users.” (/sarcasm)

So after doing some more info gathering, I start to put together more pieces of the puzzle.
Symptoms
1) completely random
2) impossible to consistently reproduce ( I never got it to reproduce once in fact!)
3) Users can hear far-end. But far-end can not here users.
4) The majority of the incident reports were filled by… WOMAN.

Typical one-way audio call right? Except for that last one. This was getting interesting. But we were never able to reproduce it, so we didn’t take it very seriously since there was no proof. Yes, I said it. I’m a network engineer and users random anecdotal complaints aren’t proof.

So I arrive onsite about day 4 of the day2 support week, and I get summoned to CIOs office.

(scary music) dum dum dum dum (/scary music)

Now I had heard some stories about this guy. No nonsense, ruled the IT department with an iron hand. Super tight ship. I too have control issues, so I kinda like this guy. But there was one more thing I had learned in the safe confines of the sound proofed data center. They had a nickname for Mrs. CIO.

The Dragon Lady

It was not uncommon for employees to hear this woman emasculating here husband over the phone. They told me that on bad days, You could hear her screeching through the closed door… And she wasn’t on speaker phone.

Now apparently it was on of those days. And apparently the issue happened. And apparently Mrs Dragon Lady thought that Mr CIO had hung up on her in the middle of the verbal assault. Things are not looking good for Mr. CIO when he gets home. And he knows it.

Back to the summons

“Phone boy!” says Mr. CIO
“Yes sir?” I reply.
Mr CIO ” You are not leaving until you figure out what the he** is going on and it’s fixed”
Me” You’re the customer. I’ll do my best”
Mr CIO ” and then you’re calling my wife”

Huh?

Now i’ve got some skills, I had already been a desktop guy, server guy ( Vines, Windows, Novell, OS/2). I had done some as/400, even done some Avaya and Nortel PBX installs.

but I was NOT a couples therapist!

But of course the customer is always right, right? I was just trying to figure out how i’m going to put that activity into the timesheet system? Any one got the code for “psych consult”?

So, i’ve got TAC on the phone, in those days there weren’t THAT many good voice guys, and I had been working with most of them for a few years. Case has been opened for a few days, but we have no hard data points to bite into, so we’re in waiting mode.

So I call up my TAC engineer and explain to him my situation. He puts me on hold so he can tell the guys around him. We wait for the laughter to fall down to a dull roar and start getting to work.

10 hours, no food, and a LOT of coffee later. ( I was never allowed to leave the building ) We finally figure this out. As most of you voice guys ( and galls!) know; human speech is based on a known frequency range. Nyquest theorem states that we’re going to sample at twice the highest frequency range and of course, we picked approx 0-4000 which gets us to the 64k codec.

Now as we all know, woman USUALLY have a higher voice than their harrier male counterparts. And as any married man can attest to, the more excited they get, the higher it goes.

The issue? When a woman callers voice crossed a certain threshold, CME was interpreting this as the initial negotiation tone of a fax machine. You can see how this goes

Mrs. Dragon lady calls Mr CIO -> Gets mad at Mr CIO -> Voice Raises -> Fax detected -> Fax negotiation commences -> one-way audio occurs -> Mrs Dragon thinks he hung up on her -> creative vocabulary ensues – > divorce is imminent -> it’s 9pm, I haven’t eaten in 12 hours and i’m sitting alone in a cold data center with Mr. CIO watching me outside the glass.

Worst part: This was a previously resolved bug which R&D had accidentally reintroduced ( pre IOS code train streamlining) and we had already discounted this hours earlier since it had been “fixed”.

To jump to the end, I called Mrs Dragon and explained the situation. Emailed her the case notes and the bugtraq link. She was actually a very pleasant lady!

Case closed. marriage saved

Almost ten years later I still laugh every time I think of this. Goes to show that there is ALWAYS a reason that things go the way they they do. It is what it is. And when it’s not?

It’s because you didn’t understand it in the first place.