linux and trusted certificates



Slightly unusual post for me - t's not really azure related (or indeed Oracle related from my past life) this is just talking about ssl certificates in general on linux machines - this particular example is for RHEL on Azure but actually thats largely irrelevant - the process would be pretty much the same on any platform in any hosting zone.

So in my example we had an intranet application hosted at https://blahblah.intranet.com - the linux server was calling that website but failing due to certificate chain validation errors - i.e. the certificate protecting the intranet site was issued by a vendor that the linux box did not trust.

The access was from a particular piece of software but it's easy enough to re-produce the problem just with curl by doing this:

curl -iv https://blahblah.intranet.com
* Rebuilt URL to: https://blahblah.intranet.com/
*   Trying x.x.x.x...
* Connected to blahblah.intranet.com (x.x.x.x) port 443 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* TLSv1.2 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, Server hello (2):
* SSL certificate problem: unable to get issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html


 The key bit being the line in orange - this is basically saying i don't trust the guy that issued the certificate so I'm not letting you go to the site - standard behaviour


Now for curl you can just add the -k option to say - i don't care about the problem just show me the site - fine for testing but not the 'right' way to sort this.


So how do we fix this properly - we need to load the certificate chain of trust above the certificate that is protecting the site into the linux machine so that it trusts the issuers - lets see how that is done.











The first thing we need to do is get hold of the issuing certificates - there is more than one way to do this - the way i did it is as follows:

1) open the site in a browser (i just did this from ie on my work laptop - yes i am still using it sometimes....)

2) click on the padlock icon that is present in the address bar - on the certificates window that pops up click the certification path option - you'll then get something looking like this





This nicely illustrates that this cert has a parent, this parent was issued by a trusted root - so for this example i need that whole chain trusted on the linux box. As you can see on windows it's happy with the certs and reports them as 'OK' as the chain is already added here as trusted.

3) Now we need to extract those certs  - to extract follow the screenshot below and save the file in DER format


That has to be done individually for the parent and grandparent - we don't need the lowest level cert of the site itself - all we need is the trust chain above it.

4) now just copy the files over to the linux box (making sure any transfer doesn't alter the file in any way - i used psftp to do mine.

Now we have the files on linux we need to do a bit of massaging to get the certificate in the format that is required for the next stage.

First part of that is to convert the der format to pem format - to do that we use openssl. To do that we do the following for each file

openssl x509 -inform der -in filename1fromwindows.cer -out filename1fromwindows.pem

The end result of that being 2 pem formatted files.

Now we need to combine them into one file

cat file1.pem >> file2.pem

That just sticks the entire content of file1.pem after the end of file2.pem - so now file2.pem now contains both certs in pem format.

If you open that file you'll see it's in plain text (garbled plain text) - but you'll be able to read the BEGIN/END cert lines you may be familiar with.

Now we have this we just need to make linux load it up and use it - to do this we just need to do the following:

cp file2.pem /usr/share/pki/ca-trust-source/anchors
cp file2.pem /etc/pki/ca-trust/source/anchors

(truthfully here I'm not sure which one of these is right - i had trouble getting it working and the file ended up in both these locations - one of them is correct but no harm in putting it in both...)

Once there we just need to run the following utility which puts the cert in the correct locations and makes linux aware of it.

update-ca-trust --extract

And thats it if we try the curl command now we see

curl -iv https://blahblah.intranet.com
* Rebuilt URL to: https://blahblah.intranet.com/
*   Trying x.x.x.x...
* Connected to blahblah.intranet.com (x.x.x.x) port 443 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* Server certificate:
*        subject: blahblah.intranet.com
*        start date: Oct 30 10:40:46 2017 GMT
*        expire date: Oct 30 10:50:00 2020 GMT
*        subjectAltName: blahblah.intranet.com matched
*        issuer: xxxxxxx
*        SSL certificate verify ok.


Thought I'd share this as it was very fiddly to get the steps exactly right - formatting of the files and location of things was very important and it's hard to debug when things are not working correctly.


Comments