Friday, December 22, 2006
Magical Troubleshooting |4:43 AM| I may have already posted about this once before, I apologize if I have
At Dell, I used to work in the printer services department. I was the support line, help desk, whatever. I processed tickets, called people, talked them through shit. Occasionally, something would go wrong with an entirely different department, and it would somehow break our shit, and I'd have to call around to get it resolved.
Dell's ordering management system, DOMS, was older,waaaay older than any existing employees, other than Michael Dell and Kevin. And they certainly didn't write this shit.
It was so old, that no one actually knew how any of it worked. Certainly not the guys supporting it. They had procedures, and fixes, but no understanding of the underlying code/technology. It might as well have been magic, or a religion They were beyond useless. Here's the thing, you'd call in, ask questions, and get totally different answers each time. The only time I needed to call is when, for whatever reason, a printer in one building (Say, RR1) would start spitting out a quote from a guy in an entirely different building (Like, RR3) and it would just keep printing this crap, hundreds of times, no way to stop it.
I'd call DOMS support, and explain the problem, and of course they'd be dumbfounded, saying how that couldn't happen. Over the several calls I'd made to them, I'd collected keywords from whatever complete lunacy the previous tech had claimed was the problem.
When I called in, I'd explain the problem, then I'd start asking questions based around these magical words.
Me: "Okay, the last time this happened, the dude said it was related to...the cluster server. Have you heard of this, this cluster server?"
Him: "no"
Me: Alright, what about, the 'net...forwarding system?
Him: No.
Me: "What about the print relays?"
Him: "Oh! I know what that is, hold on....yeah there was an issue there!"
Me: Thanks.
The next time I'd call, it'd be the cluster, and the guy would have no idea what those previous guesses meant.
At Road Runner
Back when I worked tech support for Road Runner, there was this mythical thing/concept called "The Link"
It somehow connected the billing system (With which we interacted) and these super advanced other servers that the super lazy and angry RDC dealt with.
I don't know what the link was, or looked like, or even its location. Was it a cable? Was it several computers? A network? A monk? Every so often the link would be "down". No explanation. No reason. It would go back up some hours later.
Here's the really interesting part, there was this code box, 2 digits, the action codes. These codes would be sent over THE LINK and do stuff. Most of it we didn't understand. We did know one code did the remote reset on the cable modems, but all the rest of the codes had definitions like
27: Acct In P
35: Reset M
Shit like that.
Now, sometimes, when an account was pretty fucked up, you could perform "The ritual". No one really knew what it did, but it was a series of these action codes: Like, 07, 19, 27,43, 16, 00 And it would fix a lot of problems. It could in some cases cause some new ones, but usually it fixed things. There was debate as to the components and order of the ritual.
You could see it on people's desks, and you could see how different people influenced their neighbors, judging by the ritual on the Post-it notes above their desk. I am not kidding.
There were a couple of the short ones, a couple of really long groupings of these codes. One of my supervisors had a version that was, oh, 15 code groupings long. He swore by it.
I never did find out what we were doing, or what "The LINK" was.
Magic.
Magical Troubleshooting |4:43 AM| I may have already posted about this once before, I apologize if I have
At Dell, I used to work in the printer services department. I was the support line, help desk, whatever. I processed tickets, called people, talked them through shit. Occasionally, something would go wrong with an entirely different department, and it would somehow break our shit, and I'd have to call around to get it resolved.
Dell's ordering management system, DOMS, was older,waaaay older than any existing employees, other than Michael Dell and Kevin. And they certainly didn't write this shit.
It was so old, that no one actually knew how any of it worked. Certainly not the guys supporting it. They had procedures, and fixes, but no understanding of the underlying code/technology. It might as well have been magic, or a religion They were beyond useless. Here's the thing, you'd call in, ask questions, and get totally different answers each time. The only time I needed to call is when, for whatever reason, a printer in one building (Say, RR1) would start spitting out a quote from a guy in an entirely different building (Like, RR3) and it would just keep printing this crap, hundreds of times, no way to stop it.
I'd call DOMS support, and explain the problem, and of course they'd be dumbfounded, saying how that couldn't happen. Over the several calls I'd made to them, I'd collected keywords from whatever complete lunacy the previous tech had claimed was the problem.
When I called in, I'd explain the problem, then I'd start asking questions based around these magical words.
Me: "Okay, the last time this happened, the dude said it was related to...the cluster server. Have you heard of this, this cluster server?"
Him: "no"
Me: Alright, what about, the 'net...forwarding system?
Him: No.
Me: "What about the print relays?"
Him: "Oh! I know what that is, hold on....yeah there was an issue there!"
Me: Thanks.
The next time I'd call, it'd be the cluster, and the guy would have no idea what those previous guesses meant.
Back when I worked tech support for Road Runner, there was this mythical thing/concept called "The Link"
It somehow connected the billing system (With which we interacted) and these super advanced other servers that the super lazy and angry RDC dealt with.
I don't know what the link was, or looked like, or even its location. Was it a cable? Was it several computers? A network? A monk? Every so often the link would be "down". No explanation. No reason. It would go back up some hours later.
Here's the really interesting part, there was this code box, 2 digits, the action codes. These codes would be sent over THE LINK and do stuff. Most of it we didn't understand. We did know one code did the remote reset on the cable modems, but all the rest of the codes had definitions like
27: Acct In P
35: Reset M
Shit like that.
Now, sometimes, when an account was pretty fucked up, you could perform "The ritual". No one really knew what it did, but it was a series of these action codes: Like, 07, 19, 27,43, 16, 00 And it would fix a lot of problems. It could in some cases cause some new ones, but usually it fixed things. There was debate as to the components and order of the ritual.
You could see it on people's desks, and you could see how different people influenced their neighbors, judging by the ritual on the Post-it notes above their desk. I am not kidding.
There were a couple of the short ones, a couple of really long groupings of these codes. One of my supervisors had a version that was, oh, 15 code groupings long. He swore by it.
I never did find out what we were doing, or what "The LINK" was.
Magic.
3 Comments:
The bills could only get through if The Link could defeat Ganondorf by traveling across Hyrule to assemble the TriForce. Sometimes The Link would get eaten by monsters, or fall into lava, and that's why it didn't work.
By 
, at
9:40 AM
I unlocked the mysteries of the link before I quit. It was stupid.
By , at
1:22 PM
I worked at Dell. I used DOMS every day of my miserable life while I was there. I saw the DOMS servers and they're so old they're made out of fucking wood.
By , at
3:34 PM
View My Stats
-->
