My Mac suddenly started throwing SSL errors when connecting to various sites, like search.twitter.com or support.apple.com. The App Store application refused to load content, too. Long story short, MacOS Mavericks 2015-004 has a bug where an incorrect certificate named “VeriSign Class 3 Public Primary Certification Authority – G5” is placed on the user’s login keychain. The fix is to run Keychain Access and remove it. Note: remove the one in the login keychain, not the System Roots.
Update 2: 2015-004 doesn’t put the entry on the login keychain, something else put it there earlier. 2015-004 does seem to change something though that triggers the verification problem. The fix is still to remove the key from the login keychain, I just don’t know what put it there in the first place.
Update 3: turns out it’s Cyberduck that’s writing the certificate entries.
This error seems really serious to me. Macs that are affected can’t get new software updates. Also Chrome will refuse to load any websites with SSL certs signed by that VeriSign certificate, including Apple’s own sites. Safari will load the site but will display SSL errors. Apparently Chrome is more strict in enforcing SSL security.
(I thought it was particularly interesting that it was impossible to get Chrome to visit Twitter. Twitter only serves HTTPS, not HTTP. And they have HSTS enabled which means Chrome will refuse to load a page without a working SSL certificate. Well that all succeeded, but boy was that a bad experience.)
Here’s some links with more discussion: Ask Different, Security StackExchange, Apple forums. I exported the two Verizon certs that were on my login keychain that were the problem, there’s a zip file here along with some screenshots of failed SSL certs. (That file won’t be online forever.)
I seem to be hitting a serious bug like this in MacOS every couple of months. Along with some broken-by-design things like their SMB client and I really am tempted to try going back to a Windows desktop. Or maybe Linux, if it weren’t so damn ugly.
Update 1: thanks to Ned’s suggestion in the comments I tried figuring out how those VeriSign certificates got in my keychain in the first place. I still don’t know, but looking at backups in Time Machine I can see it was placed there sometime between 2015-01-17 and 2015-02-23 13:48 (the two backups I have from before and after.)
$ grep -i verisign */Macintosh\ HD/Users/nelson/Library/Keychains/login.keychain
The Mac was turned off from 2015-01-08 until 2015-01-17 and again for 2015-01-21 until 2015-02-23. It seems likely the VeriSign entry ended up on my keychain when I turned the Mac back on. But it would have happened quickly; I entered the house at 13:42 and it’s in the keychain at 13:48. And my network is very slow. It could also have happened sometime between 2015-01-17 and 2015-01-21.
FWIW there’s a bunch of updates that installed a few hours after the keychain update, 2015-02-24 04:00 AM, including Security Update 2015-001 and Safari. But the keychain entry precedes those. Between 2015-01-17 and 2015-02-23 I only have a bunch of updates from “Google Voice and Video” and one from Seil, a keyboard kernel hack. Also FWIW I’m not given to manually tampering with my keychain nor blindly approving requests to manipulate it.
Nelson's log 
That’s unfortunate that you have had so much trouble. But your experience was by no means universal: I have multiple instances of 10.8.x, 10.9.x, and 10.10.x systems and none of them had this problem. What is suspicious to me is why you (and some others) have the out-of-date (faulty?) VeriSign intermediate certs in your login keychain. None of my systems did or now do. I don’t know of any reason why Apple would be installing any certificates like that in your login keychain; Apple-provided roots and intermediates should be in your system keychain. Is there any way you can establish whether those certs were present in your keychain prior to installing the security update? If so, the most likely scenario to me is that some third-party product had previously installed those certs into your user keychain and they had been working until Apple updated the system keychain certs in this security update, presumably properly breaking the validation chain. Googling around a bit and looking at the links you’ve provided, I haven’t seen any thorough-enough analysis (in particular, whether the intermediate certs in the login keychain were present prior to the security update and/or whether they were changed by the update) that would either support or refute my speculation. Have you found any? The second-most likely scenario to me is that there *was* an Apple bug originally in the security update that installed the certs in the user keychain and there has been a subsequent stealth update to the security update so that current downloads of it do not exhibit this behavior but, again, it seems very odd that a security update would be fooling with certs in user login keychains. I guess if someone has a system that exhibited this problem and has Time Machine or other backups from before installation of the security update, they could check the older version of the login keychain for the presence of these certs.
Yeah it’s not a universal bug, but then it’s not just me either. I suppose it’d be interesting to sort out what made this one Mac the lucky one but it feels like wasted work unless someone at Apple were committing to do something about helping others. FWIW I didn’t install any keys manually that I’m aware of, but who knows if some other software did.
I have Time Machine backups. Do you have any idea how I’d go about looking at the state of the keychain from an older version? The Time Machine UI is all file oriented and I have no idea how Keychain stores data or whether it’s possible to point their GUI at old files.
Looked a little closer at my backups; the Verisign entry first showed up on my login keychain between 2015-01-17 and 2015-02-23. No idea how it got there though.
Yes, ~/Library/Keychains/login.keychain is the place to look. If I understand the sequence of events properly, could it be possible that the certs were installed sometime between 2015-01-17 (the date of the last preserved Time Machine backup) and 2015-01-21 (the date the Mac was powered off)? IIRC, TM only keeps weekly backups for dates older than a month as space permits so any intermediate backups between 01-17 and 01-21 have been deleted by TM.
Yeah, it’s certainly possible the certs were installed before 2015-01-21. Does MacOS keep some sort of audit / action log for keychain changes? I couldn’t find one and didn’t see anything relevant while grepping through /var/log.
I’m not aware of an audit log specifically for keychain changes; that seems like a good thing to have. One thing that *might* yield some clues would be to check /var/log/install.log: there may be a record there of packages installed via various mechanisms during that time period, one of which might have been the guilty party.
I tried looking in install.log but nothing jumped out at me. The System Information view of installs just shows a bunch of attempts to install “Google Voice and Video” and one installation of Seil. Just re-ran the Seil installer, it didn’t create a new key on my keychain (and I wouldn’t expect it to!). God knows what Google’s installer was doing running twice a day, that’s certainly strange, and I suppose it’s possible they did something dumb adding a key to the keychain but it seems a lillte unlikely.
There’s no reason to think whatever created the keychain entry also wrote to install.log, is there? Couldn’t any user application do that?
I think the most likely way to narrow this down would be to talk to other people with the bug and see what they have in common. But that’s a job for Apple, not me.
Well, assuming (as I do) that the faulty certs were not installed by Apple, one could argue that it’s not really Apple’s problem when people directly install or indirectly cause to be installed certificates in their user keychain that override default system certificate verification chains. That’s how OS X security policy has always worked and presumably works on Windows and any other systems that provide multilevel security policies, e.g. individual users can tailor their own account’s security policy and that overrides the default system security policy (as tailored by users with system admin privileges). I wouldn’t argue that myself since it’s clear that some number of people ended up with semi-broken systems as a result of these overlapping certs in their login chain but I think the main culprit here is the complexity of today’s PKI: it’s too easy to screw things up. That said, there’s certainly more that Apple and other vendors can do to make it easier to diagnose and perhaps avoid such problems.
“There’s no reason to think whatever created the keychain entry also wrote to install.log, is there? Couldn’t any user application do that?” Sure, it wouldn’t have to be due to something triggered by an app or package install. A persistent log of changes to system and user security policies, including installation and removal of certs, would be a really good improvement.
Points taken. Apple’s Keychain is a hell of a better solution for managing PKI than anything else I’ve seen. Still troubled how that cert ever got into my keychain; I normally pay attention to things like this. Given that other folks have the problem but not too many, I’m guessing it’s some uncommon but not unknown software.
It’s a terrible irony that Apple’s own site certificate is affected.
I think a bigger concern is why the heck is Google Chrome relying on anything on my computer to decide if a site is secure? FireFox doesn’t do that, and everything works perfectly. (I’m also suffering from this problem, I’ve deleted the “bad” root certs but am still unable to access twitter, paypal, and may other sites that are PERFECTLY SECURE on Google Chrome). Seems like a fatal design flaw to me.
Yeah, Firefox has chosen to maintain its own certificates while Chrome uses the system’s. I think it’s the same choice on Windows. I could argue either side of this.
I’m sorry the fix hasn’t worked for you. Do you have much left in your login keychain? Did you restart Chrome? The solutions above definitely fixed it for me, but maybe there’s something more complicated going on for some people.
I figured out what is putting the bad cert on the keychain: an older version of Cyberduck. Details at https://nelsonslog.wordpress.com/2015/04/28/cyberduck-is-responsible-for-my-bad-ssl-certificate/