SQL Server Patching Concerns

A wise man once said “Lures aren’t made to catch fish, they’re made to catch fishermen.” That wise man was my Pop, though I doubt he was the first one to say it. The fact is fishermen, not fish, are the ones buying the lures. This means if you are a lure manufacturer, you want something that fishermen will look at and think “wow, that HAS to work!” I love trying new lures, and think that giving the fish something they haven’t seen before is a good idea. However, you have to be careful that the lure was designed with the intention of catching fish, not just the eye of a naive fisherman. Lures don’t come with a guarantee, and if you buy a dud, too bad so sad.
You hope an expensive relational database management system would be different, but that isn’t always the case. Last week Microsoft released Service Pack 1 for SQL Server 2014. Eleven hours later, the download was disabled and this message appeared on the Microsoft SQL Server Blog:

Dear SQL Customers,

We have chosen to remove SQL Server 2014 Service Pack 1 (SP1) from the Microsoft Download Center due to a significant installation issue discovered shortly after release. We sincerely apologize for the inconvenience this has caused you.

We will fix the installation issue and release the Service Pack for production use in a few weeks. Although operationally equivalent, only the newly released SP1 will be serviced (security updates, cumulative updates, hotfixes). Therefore, it will be important that you replace the current version of SP1 (12.0.4050.0) with the upcoming one.

In the meantime, you can continue to test and use build 12.0.4050.0. If you are experiencing the issue below, please implement the steps described in the workaround section.

We will communicate further as we approach the release date and we appreciate your patience.

The issue is described like this:

Any SQL Server instance with SSISDB catalog enabled runs into the following installation error during SQL Server 2014 SP1 upgrade. The instance is then in an invalid and unusable state.

Error: 912, Severity: 21, State: 2.

Script level upgrade for database 'master' failed because upgrade step 'SSIS_hotfix_install.sql' encountered error 3602, state 251, severity 25. This is a serious error condition which might interfere with regular operation and the database will be taken offline. If the error happened during upgrade of the 'master' database, it will prevent the entire SQL Server instance from starting. Examine the previous errorlog entries for errors, take the appropriate corrective actions and re-start the database so that the script upgrade steps run to completion.

Error: 3417, Severity: 21, State: 3.

Cannot recover the master database. SQL Server is unable to run. Restore master from a full backup, repair it, or rebuild it. For more information about how to rebuild the master database, see SQL Server Books Online. SQL Server shutdown has been initiated

Ouch! These errors are not what a DBA wants to see when applying a Service Pack to a SQL Server instance. Microsoft also released a workaround to get the instance back up and running, but with a major release like a Service Pack, which only occurs every one to two years, I’d like to have had more extensive testing before the release.
The real lesson here is that changes should ALWAYS be made to test instances before they are rolled out to an environment that is actually being used. Secondly, and this is just my opinion, unless there is a significant reason to apply updates immediately, wait a little while. Updates that break things are often found to be problematic fairly quickly, and waiting a day or a week before applying these noncritical patches can end up saving time and stress in the long run, as I’m sure anyone who tried to install SQL Server 2014 SP1 would agree.

Advertisements