ESXi Syslog over TLS to HAProxy

ESXi can ship syslog over TLS, but the documentation is scattered and the gotchas aren’t obvious. This is the working guide for sending ESXi logs encrypted to HAProxy’s log-forward, which can then be forwarded to your system of analysis.

Why TLS for Syslog?

Plain syslog (UDP 514) is unencrypted and unauthenticated. Anyone on the network can sniff it or inject fake logs. For a homelab this might be “good enough,” but if you’re already running an internal CA and HAProxy with TLS, there’s no reason not to encrypt the log traffic too.

ESXi supports TLS natively via the ssl:// protocol — no agents or extra software required.

The Remote Host Format

ESXi’s syslog remote host string follows this format:

proto://hostname|ipv4|'['ipv6']'[:port][?formatter=value[&framing=value]]

Multiple hosts can be comma-separated for redundancy.

Configuration

1
2
3
4

esxcli system syslog config set \
  --loghost="ssl://syslog.example.com:1514?formatter=RFC_5424&framing=octet_counting"

esxcli system syslog reload

Verify:

1

esxcli system syslog config get

Output should show your loghost string under Remote Host.

Why RFC 5424 and Octet Counting?

RFC 5424 is the modern syslog format with structured data, precise timestamps, and proper hostname/app-name fields. RFC 3164 (BSD syslog) is legacy and ambiguous — parsers have to guess where fields start and end.

Octet counting prefixes each message with its byte length, so the receiver knows exactly where one message ends and the next begins. The alternative (non_transparent) uses newlines as delimiters, which breaks if your log messages contain newlines (and they will).

For any modern log pipeline, RFC 5424 + octet counting is the right choice.

Prerequisites

1. DNS Record

Your syslog hostname needs to resolve from ESXi:

syslog.example.com  →  <ip_address>  (HAProxy bind IP)

Test from ESXi shell:

1

nslookup syslog.example.com

2. NTP (Critical)

TLS certificate validation checks timestamps. If ESXi’s clock is skewed, the TLS handshake fails with certificate errors.

1
2

esxcli system ntp set --server=pool.ntp.org
esxcli system ntp start

Verify time is correct:

1
2

date
esxcli system time get

3. Internal CA Root Certificate

If your HAProxy uses certs from an internal CA (like step-ca), ESXi needs to trust that CA.

ESXi enforces certificate validation via the advanced setting Syslog.global.certificate.checkSSLCerts (enabled by default). When enabled, the syslog TLS handshake will fail unless the CA that signed the remote certificate is in ESXi’s trust store. Don’t disable this — instead, import your CA certificate.

Copy your CA root cert (or full chain) to ESXi and append it to the trusted store:

1
2
3
4
5
6
7
8

# Copy your CA cert to ESXi
scp root_ca.crt root@esxi:/tmp/

# SSH to ESXi and back up the existing store
cp -p /etc/vmware/ssl/castore.pem /etc/vmware/ssl/castore.pem.bak

# Append your CA cert to the trust store
cat /tmp/root_ca.crt >> /etc/vmware/ssl/castore.pem

The castore.pem file is ESXi’s trusted CA bundle. Your root cert gets appended to the existing CAs. If your syslog server uses an intermediate CA, append the full chain (root + intermediate).

Note: This change may not survive ESXi upgrades. Document it and re-apply after major updates.

4. HAProxy Log-Forward

HAProxy needs a log-forward section listening on 1514 with TLS:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

log-forward syslog_1514
  bind <ip_address>:1514 ssl crt /etc/haproxy/ssl/syslog.pem interface ens192
  log backend@cribl_rrb_1514 local0

backend cribl_rrb_1514
  mode log
  balance roundrobin
  server cribl_1514-01 tcp@<worker_1>:9519 ssl verify none log-bufsize 131072 check
  server cribl_1514-02 tcp@<worker_2>:9519 ssl verify none log-bufsize 131072 check
  source <ip_address>

The syslog.pem file should contain the private key and certificate chain for your syslog hostname. The log directive forwards received messages to a backend log ring that round-robins to your downstream receivers (e.g., Cribl).

Testing TLS Connectivity

Before configuring syslog, verify TLS works from ESXi:

1

openssl s_client -connect syslog.example.com:1514 -CAfile /etc/vmware/ssl/castore.pem

You should see:

Verify return code: 0 (ok)

If you see certificate errors: - Check the cert’s CN/SAN includes syslog.example.com - Verify the CA root is in castore.pem - Confirm NTP is running and time is correct

Traffic Flow

ESXi
  │
  │ ssl://<haproxy-host>:1514?formatter=RFC_5424&framing=octet_counting
  │ 
  ▼
HAProxy (<haproxy_host>:1514)
  │
  │ log-forward syslog_1514
  │
  ▼
Cribl (<cribl_host>:9519)
  │
  │ Processing pipeline
  │
  ▼
Splunk / S3 / wherever

Troubleshooting

No Logs Arriving

1. Check ESXi syslog status: esxcli system syslog config get

2. Test connectivity: nc -zv syslog.example.com 1514

3. Check HAProxy stats for the syslog backend — are connections coming in?

4. Firewall on ESXi: esxcli network firewall ruleset set --ruleset-id=syslog --enabled=true esxcli network firewall refresh

TLS Handshake Failures

Check /var/log/syslog on ESXi (yes, local syslog logs its own failures):

1

tail -f /var/log/syslog | grep -i ssl

Common causes: - Clock skew (NTP not running) - CA not trusted (root cert missing from castore.pem) - Certificate hostname mismatch - Expired certificate

Logs Arriving But Malformed

If logs arrive but look garbled in Cribl/Splunk: - Confirm HAProxy and the downstream receiver both expect RFC 5424 - Check framing matches (octet counting vs newline-delimited) - Some receivers need explicit configuration for RFC 5424 parsing

Multiple Log Hosts

For redundancy, comma-separate multiple destinations:

1
2

esxcli system syslog config set \
  --loghost="ssl://syslog1.example.com:1514?formatter=RFC_5424&framing=octet_counting,ssl://syslog2.example.com:1514?formatter=RFC_5424&framing=octet_counting"

ESXi will send to all listed hosts simultaneously. This provides redundancy but doubles the log traffic on the network.

Automating with Host Profiles

For multiple ESXi hosts, configure syslog in a Host Profile and apply it cluster-wide. The syslog settings live under:

Networking → Network Coredump and Syslog → Syslog

Set the remote host string there, and all hosts in the cluster inherit the config.

Summary

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Set NTP
esxcli system ntp set --server=pool.ntp.org
esxcli system ntp start

# Add internal CA (if using one)
cat /tmp/root_ca.crt >> /etc/vmware/ssl/castore.pem

# Configure syslog over TLS
esxcli system syslog config set \
  --loghost="ssl://syslog.example.com:1514?formatter=RFC_5424&framing=octet_counting"
esxcli system syslog reload

# Verify
esxcli system syslog config get
openssl s_client -connect syslog.example.com:1514 -CAfile /etc/vmware/ssl/castore.pem

Once it’s working, ESXi logs flow encrypted to your log pipeline with proper RFC 5424 formatting. No agents, no extra software, just native ESXi syslog with TLS.