first commit
This commit is contained in:
198
keycloak-hybrid-x509-spi/README.md
Normal file
198
keycloak-hybrid-x509-spi/README.md
Normal file
@@ -0,0 +1,198 @@
|
||||
# Keycloak Hybrid X509 Certificate Lookup SPI
|
||||
|
||||
Custom `x509cert-lookup` provider that lets a single Keycloak instance authenticate
|
||||
x509/CAC users arriving via **two paths at once**:
|
||||
|
||||
1. **F5 path (header):** F5 BigIP does L7 break-and-inspect, validates the user's cert,
|
||||
re-encrypts to Keycloak (through the L4-passthrough NLB), and forwards the user cert
|
||||
in an HTTP header — like the built-in `nginx` provider handles today.
|
||||
2. **Direct path (TLS):** TGW/brokerage traffic that bypasses the F5 performs mTLS
|
||||
directly with Keycloak (which terminates TLS itself behind the passthrough NLB with
|
||||
`https-client-auth=request`), and the cert is read straight off the handshake.
|
||||
|
||||
The critical addition over stock Keycloak: **the header is only honored when the request
|
||||
provably came from the F5** (by the F5's own TLS client cert fingerprint/DN, or source IP).
|
||||
Without this, any client on the TGW could forge the header and impersonate any user.
|
||||
|
||||
Per-request decision logic:
|
||||
|
||||
| Cert header | Peer is trusted proxy | `trust-mode` | Result |
|
||||
|---|---|---|---|
|
||||
| present | yes | any | header cert used |
|
||||
| present | no | `log` | header cert used + **loud warning** |
|
||||
| present | no | `enforce` | header **ignored**; TLS peer cert used if present |
|
||||
| absent | yes | any | no user cert (the F5's own cert is never a user) |
|
||||
| absent | no | any | TLS peer cert used if present |
|
||||
|
||||
No changes are needed to your realm's X509 browser authentication flow — the
|
||||
authenticator receives the cert the same way regardless of source.
|
||||
|
||||
---
|
||||
|
||||
## 1. Build
|
||||
|
||||
```bash
|
||||
mvn package # or: podman run --rm -v $PWD:/src -w /src maven:3.9-eclipse-temurin-17 mvn package
|
||||
```
|
||||
|
||||
Produces `target/keycloak-hybrid-x509-spi-1.0.0.jar` (a few KB — contains only these
|
||||
classes; all Keycloak APIs are `provided`). Set `<keycloak.version>` in `pom.xml` to match
|
||||
your deployed version (any 24.x–26.x works; the APIs used are stable across that range).
|
||||
|
||||
## 2. Deploy (custom image)
|
||||
|
||||
Add to your existing Keycloak Dockerfile:
|
||||
|
||||
```dockerfile
|
||||
FROM quay.io/keycloak/keycloak:26.0 AS builder # match your current base/tag
|
||||
COPY keycloak-hybrid-x509-spi-1.0.0.jar /opt/keycloak/providers/
|
||||
|
||||
# https-client-auth is a BUILD-time option in Quarkus Keycloak — it must be present
|
||||
# when kc.sh build runs, not just at start.
|
||||
ENV KC_HTTPS_CLIENT_AUTH=request
|
||||
# ...plus whatever build options your image already sets (KC_DB, KC_FEATURES, ...)
|
||||
RUN /opt/keycloak/bin/kc.sh build
|
||||
|
||||
FROM quay.io/keycloak/keycloak:26.0
|
||||
COPY --from=builder /opt/keycloak/ /opt/keycloak/
|
||||
ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]
|
||||
CMD ["start", "--optimized"]
|
||||
```
|
||||
|
||||
If your image doesn't use `--optimized`, dropping the jar into `/opt/keycloak/providers/`
|
||||
and setting `KC_HTTPS_CLIENT_AUTH=request` at runtime is enough — Keycloak re-augments on
|
||||
start. **`request` means the client cert is OPTIONAL**: connections without one (OIDC app
|
||||
backchannels, admin console, the F5 before it's given a cert) proceed exactly as before.
|
||||
Do NOT use `required`.
|
||||
|
||||
## 3. Runtime configuration (env vars on the Keycloak pod)
|
||||
|
||||
```bash
|
||||
# Switch the lookup provider from nginx to hybrid
|
||||
KC_SPI_X509CERT_LOOKUP_PROVIDER=hybrid
|
||||
|
||||
# Header carrying the user cert — COPY THE VALUE from your existing
|
||||
# KC_SPI_X509CERT_LOOKUP_NGINX_SSL_CLIENT_CERT (default "ssl-client-cert")
|
||||
KC_SPI_X509CERT_LOOKUP_HYBRID_SSL_CLIENT_CERT=ssl-client-cert
|
||||
|
||||
# Day-one: permissive mode (headers accepted from anyone, like today, but everything logged)
|
||||
KC_SPI_X509CERT_LOOKUP_HYBRID_TRUST_MODE=log
|
||||
|
||||
# After harvesting values from the logs (see rollout plan), add trust rules:
|
||||
#KC_SPI_X509CERT_LOOKUP_HYBRID_TRUSTED_PROXY_CERT_SHA256=<sha256hex>[,<sha256hex-next-rotation>]
|
||||
#KC_SPI_X509CERT_LOOKUP_HYBRID_TRUSTED_PROXY_CIDRS=10.x.y.0/28
|
||||
#KC_SPI_X509CERT_LOOKUP_HYBRID_TRUSTED_PROXY_SUBJECT_DN=CN=f5.internal.example,OU=Infra,O=YourOrg
|
||||
# ...then flip:
|
||||
#KC_SPI_X509CERT_LOOKUP_HYBRID_TRUST_MODE=enforce
|
||||
```
|
||||
|
||||
> Keycloak 26 prefers a double-dash format (`KC_SPI_X509CERT_LOOKUP__HYBRID__TRUST_MODE`);
|
||||
> the single-dash form above works on 24–26 (26 logs a deprecation warning). Use one style
|
||||
> consistently.
|
||||
|
||||
All options (prefix `spi-x509cert-lookup-hybrid-`):
|
||||
|
||||
| Option | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `ssl-client-cert` | `ssl-client-cert` | Header with the user cert (PEM, URL-encoded PEM, or base64 DER — auto-detected) |
|
||||
| `ssl-cert-chain-prefix` | `ssl-cert-chain` | Optional chain headers `<prefix>-0..n` |
|
||||
| `certificate-chain-length` | `1` | Max chain headers to read |
|
||||
| `trust-mode` | `log` | `log` = honor all headers, warn on untrusted; `enforce` = ignore headers from untrusted peers |
|
||||
| `trusted-proxy-cert-sha256` | — | Comma-separated SHA-256 fingerprints of the F5's client cert(s). **Strongest check.** List old+new during rotations. |
|
||||
| `trusted-proxy-subject-dn` | — | `\|`-separated exact subject DNs (RFC2253). Safe because TLS already validated the chain. Survives rotation. |
|
||||
| `trusted-proxy-cidrs` | — | Comma-separated CIDRs/IPs for the F5's egress addresses. Weakest; requires NLB client-IP preservation. |
|
||||
| `header-cert-enabled` | `true` | Master switch for the header path |
|
||||
| `direct-cert-enabled` | `true` | Master switch for the direct-TLS path |
|
||||
| `rebuild-chain-from-truststore` | `true` | Rebuild issuer chain for header certs from the Keycloak truststore (matches nginx-provider behavior) |
|
||||
| `truststore-chain-depth` | `4` | Max issuers appended during rebuild |
|
||||
| `verbose` | `true` | Per-request INFO logging of the decision (peer fingerprint, source IP, path taken). Set `false` after rollout. |
|
||||
|
||||
**Truststore:** your existing truststore (already working for header-cert validation) must
|
||||
also be reachable by the TLS layer so Keycloak can request/validate browser certs on the
|
||||
direct path. On KC 25+ point `KC_TRUSTSTORE_PATHS` at your CA bundle; on KC 24 use the
|
||||
`spi-truststore-file-*` options you likely already have. The CAs listed there are also what
|
||||
browsers use to filter the cert-picker dialog.
|
||||
|
||||
## 4. Rollout / test plan (one 30-min cycle each, in order)
|
||||
|
||||
**Cycle 0 — no deploy, info gathering.** From the current deployment grab:
|
||||
`KC_SPI_X509CERT_LOOKUP_NGINX_SSL_CLIENT_CERT` (header name), truststore config, base image
|
||||
tag, and whether `start --optimized` is used. Ask the F5 team to start on the one-pager
|
||||
(section 6) in parallel.
|
||||
|
||||
**Cycle 1 — deploy in log mode (zero behavior change expected).**
|
||||
Jar + `KC_HTTPS_CLIENT_AUTH=request` + provider=hybrid + `trust-mode=log`.
|
||||
Verify: existing F5 CAC login still works. Then grep logs for `x509-hybrid`:
|
||||
- startup line shows the parsed config;
|
||||
- each F5 login logs `remoteAddr=` (→ your CIDR value) and `peer=` (→ `no-tls-client-cert`
|
||||
until the F5 presents one);
|
||||
- `header cert decoded using strategy '...'` (DEBUG) confirms the F5's encoding.
|
||||
If a TGW-side test client exists already, hit Keycloak directly: expect a browser cert
|
||||
prompt and `using DIRECT TLS peer cert` in the logs.
|
||||
|
||||
**Cycle 2 — F5 presents its client cert.** After the F5 change, each F5 login logs
|
||||
`peer=subject=[...] sha256=<fingerprint>` — that fingerprint is your
|
||||
`trusted-proxy-cert-sha256` value. Confirm logins still work (`request` mode tolerates the
|
||||
new cert automatically, provided the F5 cert's CA is in the truststore).
|
||||
|
||||
**Cycle 3 — enforce.** Set the trust rules + `trust-mode=enforce`. Verify: F5 login works,
|
||||
direct TGW login works, and the negative test — from a TGW-side box,
|
||||
`curl -k https://keycloak.../realms/<realm>/account -H "ssl-client-cert: <any user PEM>"` —
|
||||
produces `IGNORED cert header from untrusted peer` in the logs and no authentication.
|
||||
|
||||
**Cycle 4 — quiet down.** `verbose=false`, optionally keep DEBUG off. Done.
|
||||
|
||||
Rollback at any cycle: set `KC_SPI_X509CERT_LOOKUP_PROVIDER=nginx` and restart — the
|
||||
built-in provider and all its config are untouched by this deployment.
|
||||
|
||||
## 5. Security notes
|
||||
|
||||
- **Never** leave `trust-mode=log` long-term once TGW routes are open: it preserves the
|
||||
legacy trust-any-header behavior. The provider logs a warning at startup to this effect.
|
||||
- Fingerprint pinning breaks when the F5 rotates its client cert — list the next cert's
|
||||
fingerprint alongside the current one before rotation, or rely on `subject-dn` (which
|
||||
survives rotation and is chain-validated by the TLS layer in `request` mode).
|
||||
- Source-IP trust requires the NLB target group to have **client IP preservation** enabled
|
||||
(targets registered by instance ID have it on by default; by IP it's configurable). The
|
||||
`remoteAddr=` log line from Cycle 1 tells you definitively what Keycloak sees.
|
||||
- The F5 pre-screens users against an allowlist and (presumably) checks revocation; the
|
||||
direct path has no such screen. Enable **OCSP/CRL revocation checking** in the realm's
|
||||
X509 authenticator config (Authentication → your browser x509 flow → config) if it isn't
|
||||
already on, and confirm the cert-to-user mapping attribute is strict enough that only
|
||||
provisioned brokerage users resolve.
|
||||
- `https-client-auth=request` sends a TLS CertificateRequest on every connection. Browsers
|
||||
without a matching cert and server-to-server OIDC clients simply continue certless; users
|
||||
hitting Keycloak directly get the platform cert-picker (expected CAC behavior).
|
||||
|
||||
## 6. One-pager for the F5 team
|
||||
|
||||
> Keycloak's listener will request (not require) a TLS client certificate. We need the
|
||||
> BigIP virtual server that fronts Keycloak to authenticate itself on its **server-side
|
||||
> (re-encrypt) SSL profile**:
|
||||
> 1. Issue a client certificate for the BigIP from an internal CA (or reuse an existing
|
||||
> device cert). Any subject is fine, e.g. `CN=bigip-keycloak-proxy,OU=Infra,O=<org>`.
|
||||
> 2. On the **Server SSL profile** used for the Keycloak pool: set this certificate/key so
|
||||
> the BigIP presents it during the TLS handshake with the backend.
|
||||
> 3. Send us: the certificate's SHA-256 fingerprint
|
||||
> (`openssl x509 -in cert.pem -noout -fingerprint -sha256`), its exact subject DN, the
|
||||
> issuing CA chain (PEM), and the self-IP/SNAT addresses the BigIP uses toward the
|
||||
> Keycloak NLB.
|
||||
> 4. No iRule/header changes needed — keep injecting the client cert header exactly as today.
|
||||
> Timing note: this can be deployed before or after our Keycloak change; the Keycloak side
|
||||
> is backward compatible either way (`https-client-auth=request` is optional-cert).
|
||||
|
||||
## 7. Troubleshooting via logs (category `com.example.keycloak.x509`)
|
||||
|
||||
| Log line | Meaning |
|
||||
|---|---|
|
||||
| `initialized. trust-mode=...` | Config as parsed at startup — check this first |
|
||||
| `using HEADER cert from trusted proxy` | F5 path, healthy (enforced) |
|
||||
| `honoring cert header from UNTRUSTED peer` | Log-mode: would fail in enforce — fix trust rules before flipping |
|
||||
| `IGNORED cert header from untrusted peer` | Enforce-mode rejection: forgery attempt, or your F5 trust rule is wrong/stale |
|
||||
| `using DIRECT TLS peer cert` | TGW path, healthy |
|
||||
| `trusted proxy connection without usable cert header` | F5 request with no/blank header (e.g. health checks) — normal |
|
||||
| `failed to parse cert header` | Unexpected header encoding — the logged prefix shows what arrived |
|
||||
| `no certificate found` | Certless request (normal for non-CAC flows) |
|
||||
|
||||
Set `KC_LOG_LEVEL=INFO,com.example.keycloak.x509:debug` during rollout for
|
||||
decode-strategy and chain-rebuild details.
|
||||
Reference in New Issue
Block a user