Cannot make secure [TCP] on RHEL7, PXPlus 14 (Certificates problem)

[Loren Doornek]

We are having a problem making a secure [TCP] connection from one RedHat Linux server, but the same connection works elsewhere.  The problem server is new, and is running PXPlus 14. 

On the working server, we can open the [TCP] channel, then print the FIN(xx,"X509_Subject") to confirm that the certificate was found, as shown below:

-} open(1,tim=5)"[tcp]images.e-brandid.com;443;Secure"
-} print fin(1,"Secure")
1
-} print fin(1,"X509_Subject")
/C=US/postalCode=92626/ST=Costa Mesa/L=Costa Mesa/street=Building A/street=3185 Airway Ave/O=Brand ID/OU=IT/OU=Secure Link SSL/CN=images.e-brandid.com
-} write record(1)"test"
-} end

But, on the bad server, the connection opens, but the X509_Subject is blank, and writing to the channel results in an Error 15, as shown below:
-} open(1,tim=5)"[tcp]images.e-brandid.com;443;Secure"
-} print fin(1,"Secure")
1
-} print fin(1,"X509_Subject")
-}
-} write record(1)"test"
Error #15: Operating system command failed
Last IO to [tcp]images.e-brandid.com;443;Secure, channel 1
[TCP][Sockets]Error[0]:Success (5:<Unk>)

On the bad server, I can open secure connections to other https URL's, and they return a proper X509_Subject.  For example, a connection to the test Cybersource server is shown below:
-} open(1,tim=5)"[tcp]ics2wstesta.ic3.com;443;Secure"
-} print fin(1,"Secure")
1
-} print fin(1,"X509_Subject")
/C=US/ST=California/L=Foster City/O=VISA INTERNATIONAL SERVICE ASSOCIATION/CN=ics2wstesta.ic3.com
-}

The fact that I can open other secure connections leads me to believe that PXPlus is properly using the Linux certificate authorities, but nothing I've tried lets me connect to the problem URL. 

I was able to connect using curl, as shown below:

$ curl -i -v -X POST https://images.e-brandid.com:443/xml2/xmlreceiver.asmx   -H "Content-Type: text/xml"   -H "SOAPAction: https://images.e-brandid.com/xml2/XMLOrder" -d @/usr/common/test.xml

* About to connect() to images.e-brandid.com port 443 (#0)
*   Trying 64.79.171.67...
* Connected to images.e-brandid.com (64.79.171.67) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_RSA_WITH_3DES_EDE_CBC_SHA
* Server certificate:
*       subject: CN=images.e-brandid.com,OU=Secure Link SSL,OU=IT,O=Brand ID,STREET=3185 Airway Ave,STREET=Building A,L=Costa Mesa,ST=Costa Mesa,postalCode=92626,C=US
*       start date: May 24 00:00:00 2016 GMT
*       expire date: Aug 05 23:59:59 2019 GMT
*       common name: images.e-brandid.com
*       issuer: CN=Network Solutions OV Server CA 2,O=Network Solutions L.L.C.,L=Herndon,ST=VA,C=US

I was able to issue another command that retrieved the certificate chain from the server (attached file brandid.cer.txt), and I tried everything that I can find on Google to update the root CA's for RHEL and to get that specific certificate chain to be recognized, but nothing is working.  At this point, I'm at a loss as to what to do to make this work in PXPlus. 

Any suggestions?

[Devon Austen]

Hi Loren,

By default when you make the [tcp] connection to the https server PxPlus does not validate the certificate. Unless you are using PxPlus 2017+ and you had the option

CERTIFICATES= VALIDATE | TRUSTREQD

So validation of the certificate is not likely the issue.

I can't really guess what the issue actually is though. The results should be the same for both systems as they are talking to the same server. Here are some questions that may help lead to an answer.

1) Is the PxPlus where it works and the PxPlus where it doesn't work the same version?
2) Is the OS the same?
3) Is the OpenSSL version the same and are there updates available?
4) Is there internet security software that could be blocking things?

[Loren Doornek]

1. The 'good' server is on PXP12, but the 'bad' server is on PXP14.  Regardless, I've tried on other servers using PXP12, and it fails, so I don't think it's a PXP version issue.
2. OS is RHEL7 on both servers, with only slight differences in the sub-versions.
3. OpenSSL is 1.0.1 on the 'good' server.  The 'bad' server has a new version (1.0.2).  I'm hesitant to update the 'good' server since this would cause problems if the newer version of OpenSSL is the problem.
4. I don't believe security software would be the problem.  I can make the connection to the port, but just the cert is not being 'recognized' in PXP for some reason.

[Devon Austen]

The problem is not likely that PxPlus is not recognizing the certificate but that it did not get one. You have connected to the server from a PxPlus and looked at the certificate so we know it is good. This points to the server not liking the clients that do not work. Maybe the server is requiring clients to use a specific cipher that is supported by the good servers OpenSSL and not the bad servers OpenSSL?  This would not be based on OpenSSL version but how the OpenSSL was compiled.

[Loren Doornek]

Ah!  You may be onto something there!  I'd forgotten that many ciphers have been disabled in newer versions of OpenSSL.  I ran a script to list the ciphers on my bad server, and it doesn't include the cipher listed when I connected using curl (TLS_RSA_WITH_3DES_EDE_CBC_SHA), so possibly I need to add that cipher. I had to do that once, and have largely blocked the pain from my memory, but I'll start Googling to see how to do that again, and let you know if it works.

[Mike King]

Loren,

It should be noted that if running Linux/Unix we use the openssl libraries that comes with your OS thus we are limited to whatever ciphers/interfaces (SSLv2, v3, TLS1, TLS1.1, ...) are supported by the installed version of openssl.   Older Linux system often have older openssl libraries making them unable to connect to some newer system until the OS libraries are updated.

For Windows, we ship current openssl libraries with our product.  PxPlus v14 shipped with openssl version 1.0.2.7 (1.0.2g). 

[Devon Austen]

Hi Loren,

Glad I could help and I hope you can get it to work. You may want to see if enforcing NOTLSV1 and probably NOSSLV3 and NOSSLV2 on your [TCP] open fixes the issue that cipher seems to be associated with TLS v1.

[Loren Doornek]

The ciphers are the problem.  Or, more specifically, SSL3/TLSv1.  The server we are connecting to only offers a cipher using a SSL3/TLSv1 protocol.  SSL3/TLSv1 protocols are NOT included in the default builds of openssl after 1.0.2.f.  Our 'bad' server is a newly installed RHEL7, using openssl 1.0.2k, so SSL3/TLSv1 protocols are not supported, and thus we couldn't connect to the server.  Our 'good' server is using openssl 1.0.1e, so it still supports SSL3/TLSv1.  Those older protocols are not included by default because of the Heartbeat and Poodle vulnerabilities of a few years ago. 

FWIW, I wrote a small program to check what ciphers and protocols are accepted by a server that you're trying to connect to.  I'll post it here to hopefully save somebody all the hours that I spent trying to figure this out.

Thanks, Devon, for pointing me in the right direction!


0010 BEGIN
0020 INPUT "Enter HOST server IP/Name to connect to for testing ciphers: ",server$; IF NUL(server$) THEN END
0030 LET port$="443"; INPUT EDIT "Enter HOST port to connect to (default 443 for secure connection):",port$
0040 LET so$="Y"; INPUT EDIT "Display successful ciphers only (Y/N)? ",so$; LET so$=UCS(so$)
0050 LET sp$=server$+":"+port$
0060 OPEN (UNT,ERR=*NEXT)"<openssl ciphers"; LET cl=LFO
0070 IF NOT(cl) THEN END
0080 READ (cl,ERR=*NEXT)c$; LET cl$+=c$; GOTO *SAME
0090 CLOSE (cl)
0100 LET clc=POS(":"=cl$,1,0); IF NOT(clc) THEN END
0110 DIM d$[1:clc]; READ DATA FROM cl$,SEP=":" TO d${ALL}
0120 FOR i=1 TO clc
0130 LET cipher$=d$
0140 LET x$="< openssl s_client -cipher "+QUO+cipher$+QUO+" -connect "+sp$
0150 LET rr$=$$
0160 LET ct=0; OPEN (UNT,ERR=*NEXT)x$; LET ct=LFO
0170 IF NOT(ct) THEN PRINT "UNABLE: "+x$; GOTO 0250
0180 READ (ct,ERR=*NEXT)r$; LET rr$+=SEP+r$; GOTO *SAME
0190 CLOSE (ct)
0200 LET failed=POS("no peer certificate available"=rr$)
0210 LET p=POS("Protocol  :"=rr$),p$=$$; IF p THEN LET p$=MID(rr$,p),p$=MID(p$,1,POS(SEP=p$)-1)
0220 IF NOT(failed) THEN PRINT 'RED',"SUCCESS: "+cipher$+" "+p$,'BLACK'
0230 IF failed AND so$<>"Y" THEN PRINT "FAILED: "+cipher$+" "+p$
0240 WAIT 0
0250 NEXT