Database bullet dodging and lessons learned
I dodged a very large (metaphorical) bullet yesterday. I’m not in the habit of trying to find bullets to dodge, but this one found me because it tuns out I didn’t plan as well as I thought I had.
It all started yesterday morning when a client asked me to make some batch updates to the data in his MySQL database. So, I wrote a script to make the required updates. Before running the script, I made a backup of the database in case something went wrong. Everything was good so far. I ran the script, checked the database to see if anything looked amiss and called it a night (oh yeah, this was like 11:00pm last night). This morning, I get an urgent text message from the client saying that the data didn’t look quite right. No problem I think, I made a backup copy of the database last night, I can fix this. But nooooo, that database backup wasn’t around any more. Why, you ask? Oh yeah, that. I saved it to the same location that the regular, nightly backup saves to. So by the time I needed the backup this morning, it was already gone. Luckily, the eventual fix was very simple once I figured out where my script had gone wrong. It could have been really ugly though.
Read more…
