I mean, seriously… a restart of apache was enough to cause this:
Yup, Apache is a huge memory hog. :(
So, we had the launch party of Fresh last night :D After hours of me shuffling lights around and other folk decorating etc, we eventually transformed the venue into something like this: (more photos) Now, we have 6 (at the … Continue reading
The cluster welcomes it’s third member! Caterpie was created from a need – the need to a Minecraft SMP server to be hosted *somewhere*. My previous plans for the third cluster machine haven’t gone out the window – those plans have just moved almost entirely to metapod :P
caterpie.helpmebot.org.uk is an Amazon Web Services Elastic Compute Cluster (EC2) node – based on the “free tier” of services. The free tier doesn’t allow much in the way of memory, so I’m not entirely sure how well this will run.
It may turn out that the whole thing doesn’t actually work, in which case I’ll scrap it, but it’s worth a try. :)
Apologies about the HUGE hole near the spawn point – I thought I’d play with the server commands and TNT… ohai bedrock. :D
So, stuff I’ve found since the Beta update:
* Squids! (pictured)
* I’m more likely to stumble across dungeons within minutes of starting a new game.
* There are different types of tree
* Tree leaves disappear when the tree’s trunk has been cut down
* I know more about redstone and logic gates than I thought I did
blastoise.helpmebot.org.uk was rebooted to enable a network configuration change to take effect (addition of a new network adaptor). Estimated downtime was 5 minutes.
However, the reboot was not without issues:
The only major issue here was the DNS resolver configuration disappearing. What actually happened was the contents of the file /etc/resolv.conf changed from this:
... to this:
Now, that's slightly concerning, because before this reboot, I'd never seen that message before. And, as this config file was now effectively completely empty, I had zero possibility of doing hostname lookups. As all the software was configured to use dbmaster.helpmebot.org.uk and dbslaveX.helpmebot.org.uk as their database config, they couldn’t actually connect, so I shut down Apache to limit the damage.
It turns out that there’s a program called resolvconf, which has a few files in /etc/resolv-conf/ which control the contents of that file. Indeed, most of the original config was in /etc/resolv-conf/resolv.conf.d/original, so it was a simple case of putting the config into the correct place. This seemed to do the trick:
After that, restarting Apache was all that was needed to bring the site back up to where it should be. Total public-facing downtime was about 13 minutes.
Starting nagios-hmb and puppet back up was an easier one to solve, at least until I got some worrying stuff when I tried to run an update of puppet manually...
It turns out that when everything networkish failed earlier, our friend puppetmasterd somehow changed it's hostname or something, and all the certificates became invalid. With a bit of thanks to Wikitech, I managed to re-certify all the stuff, though I also removed /etc/puppet/ssl and restarted everything to do with puppet too.
After that, everything seems fine - if not please let me know ASAP.
Don’t you just love reading those things where someone sues someone else over something which actually probably helps them?
Well, apparently, the author doesn’t want Amazon.com to sell her book. Nor does she want Wikipedia to have an article on her book. Nor does she want Urban Dictionary to have a listing on her book.
Even better, the evidence she presents for this doesn’t actually cover any Wikipedia page – more a Facebook page with snippets of information from Wikipedia.
How is she trying to getting her way? A lawsuit, riddled with grammatical errors (even worse than mine).
Throughout today, I’m moving this domain over to the new server, blastoise.
However, it’s not just a simple move as many of the other hosted domains have been. I’ve got a bit more work to do for this one – as such it’s going to be a bit up/down for a while as I break stuff.
Sorry for the problems, I’m just trying to make this better for everyone