Meltdown and Spectre

I’m a huge fan of comedian Brian Regan. I love that his humor is clean, and he is a genuinely funny guy. One of his older routines has a bit on watching fishing on TV. He asks his audience, “Do you ever watch fishing on TV for like 15 minutes and then just go: ‘Boy, I’d better get a life. I’m watching fishing. I’m not even fishing, I’m watching fishing. I’m too lazy to fish… I’m taping fishing, to watch again later!'” While it does seem silly to watch fishing instead of going out to do it, I really enjoy being able to watch fishing during the winter months. When you can’t be out in a boat, it’s nice to be able to remember the warmth of the sun and the kiss of fresh lake air on your skin, not to mention the thrill of a strong fish tugging from the end of your line. In addition to the sweet memories these shows can evoke, they can also teach new tactics that can be used to improve your techniques. It’s important to pay attention to what the professionals are doing in order to improve yourself.
As a DBA, we should also be paying attention to news in the industry. A few weeks ago we were met with news of a few security flaws that have the potential to cause big problems. These flaws, named Meltdown and Spectre, exploit a vulnerability in the processor, so they affect almost all computers. As a DBA you have at least two places to address this: in the server OS software and in SQL Server. Here is a Microsoft’s guidance for protecting SQL Server from these vulnerabilities. It’s important to note that Microsoft is advising us to evaluate the performance impact of these patches. They do fix the vulnerability, but they have the potential to degrade performance, at least a bit. Those that use certain extensibility mechanisms with SQL Server, such as linked servers or CLR, are most impacted by this issue. On the OS side, unfortunately as of the time of writing this blog we still don’t have OS patches for two version of Windows Server – 2008 and 2012. If you are using either of these versions, check back on this page often to find out when these patches become available, and get them applied.


SQL Server Patching Concerns

A wise man once said “Lures aren’t made to catch fish, they’re made to catch fishermen.” That wise man was my Pop, though I doubt he was the first one to say it. The fact is fishermen, not fish, are the ones buying the lures. This means if you are a lure manufacturer, you want something that fishermen will look at and think “wow, that HAS to work!” I love trying new lures, and think that giving the fish something they haven’t seen before is a good idea. However, you have to be careful that the lure was designed with the intention of catching fish, not just the eye of a naive fisherman. Lures don’t come with a guarantee, and if you buy a dud, too bad so sad.
You hope an expensive relational database management system would be different, but that isn’t always the case. Last week Microsoft released Service Pack 1 for SQL Server 2014. Eleven hours later, the download was disabled and this message appeared on the Microsoft SQL Server Blog:

Dear SQL Customers,

We have chosen to remove SQL Server 2014 Service Pack 1 (SP1) from the Microsoft Download Center due to a significant installation issue discovered shortly after release. We sincerely apologize for the inconvenience this has caused you.

We will fix the installation issue and release the Service Pack for production use in a few weeks. Although operationally equivalent, only the newly released SP1 will be serviced (security updates, cumulative updates, hotfixes). Therefore, it will be important that you replace the current version of SP1 (12.0.4050.0) with the upcoming one.

In the meantime, you can continue to test and use build 12.0.4050.0. If you are experiencing the issue below, please implement the steps described in the workaround section.

We will communicate further as we approach the release date and we appreciate your patience.

The issue is described like this:

Any SQL Server instance with SSISDB catalog enabled runs into the following installation error during SQL Server 2014 SP1 upgrade. The instance is then in an invalid and unusable state.

Error: 912, Severity: 21, State: 2.

Script level upgrade for database 'master' failed because upgrade step 'SSIS_hotfix_install.sql' encountered error 3602, state 251, severity 25. This is a serious error condition which might interfere with regular operation and the database will be taken offline. If the error happened during upgrade of the 'master' database, it will prevent the entire SQL Server instance from starting. Examine the previous errorlog entries for errors, take the appropriate corrective actions and re-start the database so that the script upgrade steps run to completion.

Error: 3417, Severity: 21, State: 3.

Cannot recover the master database. SQL Server is unable to run. Restore master from a full backup, repair it, or rebuild it. For more information about how to rebuild the master database, see SQL Server Books Online. SQL Server shutdown has been initiated

Ouch! These errors are not what a DBA wants to see when applying a Service Pack to a SQL Server instance. Microsoft also released a workaround to get the instance back up and running, but with a major release like a Service Pack, which only occurs every one to two years, I’d like to have had more extensive testing before the release.
The real lesson here is that changes should ALWAYS be made to test instances before they are rolled out to an environment that is actually being used. Secondly, and this is just my opinion, unless there is a significant reason to apply updates immediately, wait a little while. Updates that break things are often found to be problematic fairly quickly, and waiting a day or a week before applying these noncritical patches can end up saving time and stress in the long run, as I’m sure anyone who tried to install SQL Server 2014 SP1 would agree.