As 2024 has arrived...

mclemore

Administrator
Staff member

Donor 15 years: 2010-2024
Joined
Apr 3, 2001
Messages
5,360
Reaction score
1,940
Location
Pasadena, California
2024 looks bright

We've been working on a new web site for quite some time now. While we aren't finished (and may never be -- it's a work in progress), we are looking to launch same within a week or so. The new platform will look largely the same, though it will be more mobile friendly. A primary benefit of the site however will be that the new modern design of it will make it easy to add new features moving forward. And it should be a lot more search engine (and user) friendly.

2023 ended a little rough...

We had some old Dell R620/R720 and R630/R640 that was time to cycle out. They were working fine, but it was long time to move out. We previously got rid of all our R610/R710 and most of the R620/R720 machines (including one just a few weeks earlier). So I ordered replacement hardware on Dec 15th.

Two days later, on Dec 17th (before new equip arrived let along was set up), one of the most important machines ran into trouble (an R730. An SSD failed to exist on a reboot. No problem... we've lost lots of drives over the years and machines keep going. However, this time, the other SSD on the machine got wacked at the same time and had -tons- of bad sector corruption (apparently not the SSD itself but the data on it). I didn't know we really had a problem until Monday. I ended up working on Monday on it, all Monday night, and late into Tuesday -- a full 40 hour work day w/o sleep. Fortunately we only had about a half day of downtime on the site and that I feel lucky for.

I stabilized everything, enjoyed Christmas, and then started working to set up new hardware in time for the New Year. Then disaster struck again, and this time, the last R620 had the same thing happen -- on reboot an SSD dropped off, and the rest of the raid was damaged -- although only minimally.

Everything is working well now:

We now longer operate any R620, R720, R630, or R730 Dell machines at all.

I've also given up on Dell hardware RAID with SSDs. In the old days, hardware RAID was the best choice, then bios raid, then maybe OS raid last. Now there are more and more reasons to list them in the opposite order. We are trying software raid now to see how we like it.

We've now gotten rid of almost all our Centos 7 and Centos 8 servers and now almost entirely on latest generation Red Hat type servers.

When the servers went down, the site was on installations that were corrupted. While no data were lost, at the end of the day, everything had to be manually set up from scratch. Everything from PHP to everything else on the servers are now running newer versions of tools. While we have worked to address almost all issues arising from this, some have slipped by, so it's been a game of wack-a-mole. Now It's also going to be mostly moot since as mentioned above the site is about to be replaced with a new version anyway.
 
Back
Top Bottom