[SEL] Atis outage - the details

[maytagtwin at aol.com]

Spencer,

Thank you.

Ron






-----Original Message-----
From: Spencer Yost <yostsw at atis.net>
To: Antique tractor email discussion group <at at lists.antique-tractor.com>; sel at lists.stationary-engine.com
Sent: Sun, Jan 16, 2011 9:10 pm
Subject: [SEL] Atis outage - the details


First - I appreciate all the "Thank you"s - They do mean a lot.
Here is the full story, for anyone who is interested in ugly technical 
etails.....
The server went down the evening of 1-3, but interestingly so did my 
ome computer (on the same power backup).   The power backup was fried, 
ut the house computer came back up when plugged into the wall.   The 
TIS server did not.   I replaced the ATIS server power supply about a 
ear ago, so I figured the cheap Chinese replacement couldn't handle 
hatever the power backup did.    So I replaced the power supply again, 
ut the server would only boot intermittently.    After a few of these 
ower cycles, it wouldn't even boot.   In the afternoon of 1-4 I took it 
o the local techs and a quick check of the motherboard confirmed the 
otherboard was dead.
To be honest, I don't know if the power backup fried the board, or a bad 
ower supply fried the power backup and the board.  In the end, it 
idn't matter.   It was dead.

hank Goodness data was OK, as disk was good.
I bought a new server on 1-5, installed the old drive along the new one, 
nd began the process of re-building.  It only took me a few days to 
nstall a new operating system, get my security, BIND and mail services 
unning.  But I ran into serious problems with the mailing list 
oftware(Mailman).     So I punted Mailman for the moment and worked on 
verything else.  Within another day I had web services up.   BTW - as a 
ip to all the other systems folks reading on:   The latest version of 
pache and PHP do NOT work together on Fedora 14 machines.  Just a word 
o the wise  (-:  Within another day I had the database software 
pgraded and that meant I could upgrade the bulletin board software and 
et that running.  So for all intents and purpose the system was up in 
ess than a week; except for Mailman.
While I luckily had a vacation day 1-4,  the rest of the time I am 
orking, and commuting, a total of 12 hours a day.  So by now, I am 
unning on about 4 hours a sleep a night.
The mailing list software was another matter.   The language Mailman 
ses is called python.  Unfortunately, they never updated the current 
ersion of Mailman to use the most current versions of python.   The 
revious versions of Mailman obviously didn't support the new pythons 
ither, so downgrading python was the only answer.  Unfortunately a lot 
f system utility, scripts, and other software use python, and many of 
hem make use of the newer features in the latest versions of the 
anguage.  So downgrading ended up not being a possibility.    I was 
tuck - no way to downgrade either software, and no upgrades that solved 
y problems were available.
So my only option was to run  parallel copies of python.   So I rounded 
p an old copy of python that was compatible with Mailman and built and 
nstalled that in a completely separate directory structure, outside of 
ormal system paths.   Then I modified all the headers, make files, 
mport statements, etc in Mailman to use that parallel copy.  At this 
oint, Mailman was working but required extensive additional tweaks 
nside a bunch of other utilities it includes.  In addition, there were 
ome re-configurations of both the old version of python and Mailman.  
his ended up taking me a few days.
In the end - the system is a BUNCH faster, very, very up to date, and 
hould serve us for years to come.

 think I am going to get more sleep now.....

pencer
______________________________________________
EL mailing list
EL at lists.stationary-engine.com
ttp://www.stationary-engine.com/mailman/listinfo/sel

=