TLS Mutual Authentication (TLS MA or more commonly nowadays mTLS) is increasingly being leveraged for securing links between applications (see the section of Istio’s documentation on security as an example).
It provides a much more robust and secure solution than static shared credentials:
- Shared credentials are susceptible to brute-force attacks.
- Shared credentials should be periodically rotated. In practice, this is almost never the case as the process is particularly painful and implies the rotation of the credentials at the server and client level simultaneously. In constrast, mTLS allows the client or the server to rotate their public/private key pair independently.
- When done correctly, the client private key never leaves the client infrastructure and is thus harder to steal.
That said, mTLS is admittedly harder to setup and has some pre-requisites: having deployed a Certificate Authority (CA), having a good operation tooling for the deployment and maintenance of those secrets and more importantly having devs and ops understand how the pieces fits together.
I was involved recently in a design discussion on how to secure an edge gateway facing multiple third-parties, relying on mTLS to authenticate connections from remote services. I was painfully reminded how hard it is, even for seasoned technical or security specialists, to reason about mTLS and what it means in terms of authorization for applications. For most people, mTLS is a very different beast than all other authentication methods. Spoiler: it’s not (that much).
Some important properties of certificates in a Public Key Infrastructure
X.509 Certificates are basically a bunch of metadatas, notably:
- The public RSA/DSA/EC public key associated with the private key the service holds
- A period of validity (NotBefore and NotAfter)
- The identity and a signature of the CA that issued the certificate
- Informations on how to use this certificate (Constraints)
- A Subject attribute
This Subject attribute has the form of a X.500 Distinguished Name (for example: CN=HR System,OU=HR,O=ACME). The only guarantee CA provides is that all certificates with a given Subject attribute are emitted to the same person or application.
Quoting RFC 5280 on X.509 certificates:
Where it is non-empty, the subject field MUST contain an X.500 distinguished name (DN). The DN MUST be unique for each subject entity certified by the one CA as defined by the issuer field. A CA MAY issue more than one certificate with the same DN to the same subject entity.
This happens usually when:
- A renewal of the certificate happened. Usually the previous certificate is expired or revoked.
- The application that represents this Subject is deployed in HA. Each service has its own private key and certificate.
All other attributes in the certificate (notably subjectAltName) have usually no unicity guarantees.
Also, the DN of the Subject attribute is arbitrary. It can be as simple as CN=HR System or as complex as CN=HR System,OU=Global HR,O=ACME,C=FR. All certificate authorities have different policies for naming.
It’s important to be aware that in no way the CA guarantees that the CN attribute in the DN is unique. Two certificates emitted by the CA to two different parties can share the same CN, for example CN=HR System,OU=Global HR,O=ACME,C=FR and CN=HR System,OU=Other HR Service,O=ACME,C=FR.
mTLS Basics
I won’t dive here into the details of mTLS. For a refresh on mTLS, see this article for example.
In summary, mTLS implies two things:
- the server exposes a non-expired server certificate, signed by a CA that the client trusts, that references the FQDN of the server in the subjectAltName attribute (not the CN of the certificate)
- the client has a non-expired client certificate, signed by a CA the server trusts for authentication.
That’s basically it. It should be noted here that most of the configuration is done at the level of the middleware hosting the code (an Application Server or Web Server). The code is generally unaware of how the TLS configuration is done.
mTLS with a single trusted CA
Let’s imagine a scenario where a service exposes two APIs, A and B. API A can be accessed by client 1 and API B can only be accessed by client 2. We absolutely do not want client 1 to access API B and inversely. Both clients use certificates emitted by the same Certificate Authority.
This is the easy road. In this scenario, the Subject DN of the client certificate is the identity the service can rely on for authorization on its API.
It’s important though to rely on the whole DN and not only the CN as it may not be unique at the whole CA level, as seen previously. It particularly holds true if the client uses a certificate emitted by a public CA.
mTLS with multiple trusted CA
This is there where things usually go wrong.
Let’s take the previous scenario, but now each client uses a client certificate emitted by two different certificate authorities. This is a frequent setup if you expose your services to partners through an edge gateway.
Depending on the technology used for implementing the service, you either:
- trust globally all the CAs on all your services. This is the case for Java, where the trust is global at the JVM level.
- create something similar to Virtual Hosts in Apache or Nginx, where you trust only one CA and do authorization in this compartment.
Global trust
In the first case, if you rely solely on the DN of the Subject to authenticate the clients, it is possible for an attacker to generate a certificate on the second CA with the same DN and access your service. This is bad.
This is not uncommon. Let’s look at the documentation of authorizations in Apache Kafka:
By default, the name of the principal identified by a TLS/SSL certificate is the DN (X.500 Distinguished Name) of that certificate (also known as the Subject), which uses the form CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown. You can use ssl.principal.mapping.rules to translate the DN to a more manageable principal name.
If your Kafka cluster is trusting more than one CA, you’re potentially vulnerable. In this context, the usage of mapping rules to extract an identifier from the DN adds another layer of danger.
In general, you can be tempted to check the format of the DN or trusting too much the issuance process of the CA. For example: thinking that the issuance process of the CA ensures that you’re necessarily authorized by ACME corporation to request certificates with a subject containing O=ACME.
Chances are that it is not the case.
Virtual Hosts
In the second case, you’re at first sight back to the single CA scenario, but this strategy will be probably hard to maintain as you add other CAs.
This strategy will also have impacts on how you expose your services to partners, as the server needs to determine on which Virtual Host it needs to route the incoming request:
- Use a different FQDN for each Virtual Host (if all your clients supports Server Name Indication / SNI).
- Use a different port for each Virtual Host
- Use a different IP for each Virtual Host (and as a consequence a different FQDN)
This also couples a lot of your applicative security to the infrastructure configuration, which is never a good idea.
A simple universal solution
People familiar with Radius or Kerberos will probably remember fondly the notion of realms.
mTLS can (and must) be considered similarly:
- Trusting a CA on your service basically means trusting the realm that is managed by this CA. This realm is defined by the Subject DN of this CA, which will appear in each certificate emitted by it (Issuer attribute). It may still be a good idea to ensure that you’re not trusting two CAs with the same Subject (though it’s unlikely).
- The client identity is determined by the combination of the Subject DN and the Issuer DN of its certificate, for example CN=HR System,OU=HR,O=ACME,C=FR@O=ACME,C=FR. This identifier is guaranteed to be unique across all certificates emitted by the CAs you trust.
This approach is simple, works independently of the number of CAs you trust in your application and makes no hypothesis on the format of the Subject DN or Issuer DN. It still allows for independent renewal of the client certificate. By following this principle, trusting globally all CAs also have a low security impact and allows us to simplify the configuration of the middleware.
If the application in the examples is an edge gateway and fine-grained authorization is done in backend-services, it’s equally important to propagate those two attributes.
The inherent complexity of mTLS makes us easily forget the basic properties of authentication and authorization: the user must prove its identity, its identifier must be unique at the system level and authorization must be based on this unique identifier.
Hope this post is helpful. Comments ? Hit me on Twitter.