Client Certificate not being used? #1793

kceleslie · 2024-08-14T02:18:50Z

App build number: 2.5.0
Android version: Android 12
Device: Pixel 3A
Installation source: Google

I've got nginx configured as a reverse proxy. I'm also using HTTP instead of MQTT. I've configured it to use TLS with a self signed certificate and it requires a client certificate for it to accept the connection. I've uploaded the root CA for the TLS into android's store and i've also installed the client certificate. If i use google chrome on the device I get prompted for my certificate and am able to connect to the web interface.

When the client tries to connect i see logs in nginx, it always returns an HTTP 400. I've tried using curl to manually POST with and without a client cert. It works if i use the cert, but if i dont provide a certificate i receive the same HTTP 400 error. It looks to me that the app is not sending the client certificate with the request.

Do i need to create the client cert in a specific way? Attached is the log file from the android client. I also tried it on another phone. Not sure of the Android version but it was also installed via the play store.

growse · 2024-08-14T10:48:24Z

~~Misconfigured TLS wouldn't give you a 400 error, it wouldn't give you an HTTP error at all.

Can you get any detail from the response body or the server as to what the 400 response actually contains?~~

Turns out nginx gives a 400 error on TLS issues? This is surprising, I'd expect the connection to simply not succeed.

Will check the logs later.

jpmens · 2024-08-14T11:34:34Z

@kceleslie it's a bit late because we can't completely hide it, but I've edited your comment to remove the link to logs: they contain all manner of sensitive data!

I will make the logs accessible to @growse

Edit: oh, I can actually delete the comment revision (TIL) which I've now done.

kceleslie · 2024-08-15T01:16:31Z

@kceleslie it's a bit late because we can't completely hide it, but I've edited your comment to remove the link to logs: they contain all manner of sensitive data!

I will make the logs accessible to @growse

Edit: oh, I can actually delete the comment revision (TIL) which I've now done.

Oh man, thanks!

kceleslie · 2024-08-15T01:27:05Z

~~Misconfigured TLS wouldn't give you a 400 error, it wouldn't give you an HTTP error at all.

Can you get any detail from the response body or the server as to what the 400 response actually contains?~~

Turns out nginx gives a 400 error on TLS issues? This is surprising, I'd expect the connection to simply not succeed.

Will check the logs later.

Yeah, so if i curl POST without a cert i get a HTTP 400

curl -v -k -X POST https://xxxxx
* processing: https://xxxxx
*   Trying 192.168.1.169:443...
* Connected to xxxxx (192.168.1.169) port 443
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: xxxx
*  start date: Aug 11 23:14:10 2024 GMT
*  expire date: Sep 12 23:14:10 2025 GMT
*  issuer: xxxx
*  SSL certificate verify result: self-signed certificate in certificate chain (19), continuing anyway.
* using HTTP/1.1
> POST / HTTP/1.1
> Host: xxxxx
> User-Agent: curl/8.2.1
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 400 Bad Request
< Server: nginx
< Date: Thu, 15 Aug 2024 01:19:28 GMT
< Content-Type: text/html
< Content-Length: 230
< Connection: close
< 
<html>
<head><title>400 No required SSL certificate was sent</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>No required SSL certificate was sent</center>
<hr><center>nginx</center>
</body>
</html>
* Closing connection
* TLSv1.3 (IN), TLS alert, close notify (256):
* TLSv1.3 (OUT), TLS alert, close notify (256):

If i provide my cert i get HTTP 200

curl -v -k -X POST --cert k.crt --key k.key https://xxxxx
* processing: https://xxxxx    
*   Trying 192.168.1.169:443...                                                      
* Connected to xxxxx (192.168.1.169) port 443
* ALPN: offers h2,http/1.1                                                           
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):                                     
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):                             
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):                                                                                                                          
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):                        
* TLSv1.3 (OUT), TLS handshake, Certificate (11):                                                                                                                         
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):                                                                                                                         
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384                                                                                                                   
* ALPN: server accepted http/1.1                                                                                                                                          
* Server certificate:                                                                                                                                                     
*  subject: xxxx
*  start date: Aug 11 23:14:10 2024 GMT
*  expire date: Sep 12 23:14:10 2025 GMT                                                                                                                                  
*  issuer: xxxxx
*  SSL certificate verify result: self-signed certificate in certificate chain (19), continuing anyway.
* using HTTP/1.1         
> POST / HTTP/1.1        
> Host: xxxxx
> User-Agent: curl/8.2.1    
> Accept: */*                     
>                                                                                    
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):                                
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing   
< HTTP/1.1 200 OK         
< Server: nginx       
< Date: Thu, 15 Aug 2024 01:22:49 GMT                                                
< Content-Type: text/html                                                            
< Content-Length: 3207                                                                                                                                                    
< Connection: keep-alive                  
< Last-Modified: Tue, 23 Jul 2024 12:36:34 GMT
< Etag: "669fa3d2.3207"       
< Accept-Ranges: bytes                    
<                                                                                    
<!DOCTYPE html>                                                                      
<html lang="en-US">                                                                  
  <head>                                                                             
.....

jpmens · 2024-08-15T12:48:27Z

Do i need to create the client cert in a specific way?

Please show us (without divulging the domain name) how you created both the CA and the client certificate.

kceleslie · 2024-08-15T23:13:12Z

I did it with a GUI. I'll work on testing/documenting with openssl

growse · 2024-08-16T07:17:46Z

I did it with a GUI. I'll work on testing/documenting with openssl

Try mkcert before you fall into the trap that is "trying to do what you meant to do with openssl".

kceleslie · 2024-08-17T02:21:25Z

I'm used to working with openssl. Here is what i did.

Creating the CA

$ openssl req -x509 -sha256 -days 10000 -nodes -newkey rsa:4096 -keyout rootCA.key -out rootCA.crt 
# output
Country Name (2 letter code) [XX]:US          
State or Province Name (full name) []:MO
Locality Name (eg, city) [Default City]:MO
Organization Name (eg, company) [Default Company Ltd]:US
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:FW
Email Address []:
$ mkdir demoCA
$ echo '01' > demoCA/serial; touch demoCA/index.txt; mkdir demoCA/newcerts

Create client key and csr request

$ openssl genrsa -out device_cert_key_filename.key 2048
$ openssl req -new -key device_cert_key_filename.key -out device_cert_csr_filename.csr
#output/answers
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:MO
Locality Name (eg, city) [Default City]:MO
Organization Name (eg, company) [Default Company Ltd]:US
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:Client
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Sign cert with rootCA

$ openssl x509 -req -in Client.csr -CA rootCA.crt -CAkey rootCA.key  -CAcreateserial -out client.pem -days 500 -sha256
Certificate request self-signature ok
subject=C = US, ST = MO, L = MO, O = US, CN = Client

Create PFX

$ openssl pkcs12 -export -out client.pfx -inkey Client.key -in client.pem 
Enter Export Password:
Verifying - Enter Export Password:

nginix config

server {                                                                                     
    listen 443 ssl;                 
    server_name mydomain                                                                                              
    error_log /var/log/nginx/error.log                                                                                                   
    access_log /var/log/nginx/access.log;                     
               
    ssl_certificate /etc/nginx/conf.d/ownrecorder.crt;                                                                                                
    ssl_certificate_key /etc/nginx/conf.d/ownrecorder.key;                                                                                         
    # client auth                                    
    ssl_client_certificate /etc/nginx/conf.d/rootCA.pem;                                                                                        
    ssl_verify_client on;                                                                                                                              
    location / {                                                                                                
        #proxy_set_header X-Forwarded-Host $host:$server_port;                                                                     
        #proxy_set_header X-Forwarded-Server $host;                                                                           
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;                                                                         
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_pass http://ownrecorder:8083;
        #proxy_http_version 1.1;
        #proxy_set_header Upgrade $http_upgrade;
        #proxy_set_header Connection "upgrade";
        #proxy_buffering off;
    }
}

jpmens · 2024-08-17T14:38:43Z

That looks sane to me, at least I see nothing glaringly wrong with either the certificates nor the nginx config.

@growse it really appears to be that the cert isn't sent off. I believe this to be the relevant (slightly redacted) portion of OP's initial log:

2024-08-13 21:08:53.162 D [DefaultDispatcher-worker-2] HttpMessageProcessorEndpoint: HTTP response received: Response{protocol=http/1.1, code=400, message=Bad Request, url=https://xxxxx/pub}
2024-08-13 21:08:53.162 E [DefaultDispatcher-worker-2] HttpMessageProcessorEndpoint: HTTP request failed. Status: 400
2024-08-13 21:08:53.163 E [DefaultDispatcher-worker-1] MessageProcessor$onMessageDeliveryFailed: Message delivery failed. queueLength: 2, message=[MessageLocation id=4b07d8fa ts=2024-08-14T02:08:12Z,lat=<redacted>,long=<redacted>,created_at=2024-08-14T02:08:51.708Z,trigger=DEFAULT]
2024-08-13 21:08:53.164 D [DefaultDispatcher-worker-2] HttpMessageProcessorEndpoint: Execute call failed
org.owntracks.android.net.OutgoingMessageSendingException: java.lang.Exception: HTTP request failed. Status: 400
        at org.owntracks.android.net.http.HttpMessageProcessorEndpoint.sendMessage(SourceFile:339)
        at org.owntracks.android.services.MessageProcessor.sendAvailableMessages(SourceFile:216)
        at org.owntracks.android.services.MessageProcessor.access$sendAvailableMessages(Unknown Source:0)
        at org.owntracks.android.services.MessageProcessor$sendAvailableMessages$1.invokeSuspend(Unknown Source:11)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(Unknown Source:8)
        at kotlinx.coroutines.DispatchedTask.run(Unknown Source:98)
        at androidx.work.Worker$2.run(SourceFile:14)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Unknown Source:2)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(SourceFile:96)
Caused by: java.lang.Exception: HTTP request failed. Status: 400
        at org.owntracks.android.net.http.HttpMessageProcessorEndpoint.sendMessage(SourceFile:259)
        ... 8 more

kceleslie · 2024-08-26T02:02:29Z

I tested the same steps on an IOS device, got the same HTTP 400 error.

I'm thinking that I might have done something wrong. Not sure if it's on the nginx side or on the certificate side.

ckrey · 2024-08-26T06:29:32Z

iOS does support TLS client certificates in MQTT mode only (https://owntracks.org/booklet/features/tlscert/#client-certificates)

growse · 2024-08-26T13:43:49Z

I've had the world's briefest glances at the implementation, and the HTTP handler uses the same socket handler (and therefore TLS code) as the MQTT one. So this certainly warrants closer inspection, and it's not just "forgot to implement it lol".

growse · 2024-08-27T17:01:27Z

Can reproduce. Weird issue.

Wasn't helped by the TLS client cert / CA UI being hidden in HTTP mode, but that's fixed now.

growse · 2024-09-08T15:36:06Z

Think I got this one. My local nginx with:

        server {
                listen 8901 default_server ssl;
                server_name localhost;
                ssl_certificate /tls/cert.pem;
                ssl_certificate_key /tls/key.pem;
                ssl_client_certificate /tls/ca.crt;
                ssl_verify_client on;
                location / {
                        proxy_pass http://recorder;
                        proxy_http_version 1.1;
                }
        }

seems to accept requests and return responses with a client cert set. You can either try and build off master or wait for the 2.5.2 to test the fix?

kceleslie · 2024-09-09T00:55:27Z

I'll wait. Thanks!

growse · 2024-09-14T21:51:46Z

2.5.2 is out on the play store. Let me know if it fixes!

growse · 2024-09-23T19:22:57Z

@kceleslie did this fix?

kceleslie · 2024-09-24T10:10:31Z

Yep, seems to work for me

growse added the bug label Sep 8, 2024

growse added the needs-more-info A little more info requested from the submitter label Sep 23, 2024

kceleslie closed this as completed Sep 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Client Certificate not being used? #1793

Client Certificate not being used? #1793

kceleslie commented Aug 14, 2024 •

edited by jpmens

Loading

growse commented Aug 14, 2024 •

edited

Loading

jpmens commented Aug 14, 2024 •

edited

Loading

kceleslie commented Aug 15, 2024

kceleslie commented Aug 15, 2024

jpmens commented Aug 15, 2024

kceleslie commented Aug 15, 2024

growse commented Aug 16, 2024

kceleslie commented Aug 17, 2024

jpmens commented Aug 17, 2024

kceleslie commented Aug 26, 2024

ckrey commented Aug 26, 2024

growse commented Aug 26, 2024

growse commented Aug 27, 2024

growse commented Sep 8, 2024

kceleslie commented Sep 9, 2024

growse commented Sep 14, 2024

growse commented Sep 23, 2024

kceleslie commented Sep 24, 2024

Client Certificate not being used? #1793

Client Certificate not being used? #1793

Comments

kceleslie commented Aug 14, 2024 • edited by jpmens Loading

growse commented Aug 14, 2024 • edited Loading

jpmens commented Aug 14, 2024 • edited Loading

kceleslie commented Aug 15, 2024

kceleslie commented Aug 15, 2024

jpmens commented Aug 15, 2024

kceleslie commented Aug 15, 2024

growse commented Aug 16, 2024

kceleslie commented Aug 17, 2024

jpmens commented Aug 17, 2024

kceleslie commented Aug 26, 2024

ckrey commented Aug 26, 2024

growse commented Aug 26, 2024

growse commented Aug 27, 2024

growse commented Sep 8, 2024

kceleslie commented Sep 9, 2024

growse commented Sep 14, 2024

growse commented Sep 23, 2024

kceleslie commented Sep 24, 2024

kceleslie commented Aug 14, 2024 •

edited by jpmens

Loading

growse commented Aug 14, 2024 •

edited

Loading

jpmens commented Aug 14, 2024 •

edited

Loading