[Day 28] Cisco ISE Mastery Training: Syslog & Live Logs for Wired Authentication Debugging

---

Introduction

When a wired user “can’t get online,” your two most powerful sources of truth are:

• Network device syslogs (switch/WLC/edge device): they report port up/down, EAP/DOT1X events, RADIUS connectivity errors, CoA acceptance, VLAN changes, interface errors, and local authorization events.
• Cisco ISE Live Logs: ISE’s policy engine view showing Authentication & Authorization decisions, identity source checked, matched policy rules, returned RADIUS attributes (VLAN, dACL, Filter-ID), and textual failure reasons.

Used together and correlated by time, MAC, username, port, syslogs + ISE Live Logs let you pinpoint root causes quickly — whether it’s a credential issue, identity source failure, RADIUS reachability, EAP/TLS cert chain problem, NAD (switch) attribute mapping, or a failing CoA.

This lesson teaches you how to set up syslog collection, how to use ISE Live Logs effectively, how to capture packet evidence, and a repeatable correlation workflow that scales from lab to production.

---

Problem Statement

Real-world pain points that logs solve:

• Authentication failures that produce no visible client error (supplicant silent).
• Intermittent failures (works sometimes — hard to reproduce).
• “ISE shows Accept but switch didn’t apply VLAN/dACL” — who’s wrong?
• CoA not changing session attributes after policy change.
• RADIUS packets lost/blocked by network devices / NAT.
• Certificate or AD lookup problems that only appear in the server logs.

Without consistent, timestamped logs from both the NAD and ISE, engineers waste hours guessing. Correlation is the antidote.

---

Solution Overview

This module shows you how to:

• Configure a central syslog server (rsyslog) to collect switch/WLC syslogs.
• Configure switches to forward rich auth events to the syslog server.
• Use ISE Live Logs to find exact RADIUS/GUID/EAP flows and the returned attributes.
• Capture EAPOL and RADIUS packet traces to verify TLS and attribute content.
• Correlate records (timestamp → MAC → username → port → RADIUS transaction ID) to get the full picture.
• Automate or manually forward ISE logs to SIEMs for long-term forensic analysis.

We’ll walk you through a lab build and then show common troubleshooting scenarios and how to resolve them using the logs.

---

Sample Lab Topology

Platform: VMware Workstation or EVE-NG
Nodes:

• Cisco ISE VM — 10.10.10.50 (PAN/PSN or single node)
• Catalyst access switch (IOS XE) — 10.10.10.2
• rsyslog server (Ubuntu) — 10.10.10.100 — collects syslogs from switches and ISE (optional)
• AD/DC — 10.10.10.10 (identity)
• Windows 10 client (802.1X supplicant) — connects to Gi1/0/10
• Printer (MAB) — Gi1/0/11

[rsyslog 10.10.10.100]

| [AD 10.10.10.10]

Use this lab to practice: (A) syslog capture of switch auth events, (B) ISE Live Logs correlation, (C) packet capture verification.

---

5. Step-by-Step GUI & CLI Configuration Guide (numbered)

NOTE: menu names in ISE can vary by release. Where exact labels differ, I call out the common alternatives. Replace IPs & secrets with your lab values.

A — Prepare infrastructure (NTP, DNS)

1. Ensure NTP synced across ISE, switches, syslog server, and AD.
   • Switch: show ntp status
   • ISE: Admin GUI → Administration → System → NTP (or CLI show clock)
     Why: accurate timestamps are essential for correlation.

---

B — Configure rsyslog server (Ubuntu example)

Install & enable rsyslog to accept remote syslog on UDP/TCP 514:

# Ubuntu
sudo apt update
sudo apt install rsyslog -y

# enable reception of remote logs
sudo sed -i 's/^#module(load="imudp")/module(load="imudp")/' /etc/rsyslog.conf
sudo sed -i 's/^#input(type="imudp" port="514")/input(type="imudp" port="514")/' /etc/rsyslog.conf
sudo sed -i 's/^#module(load="imtcp")/module(load="imtcp")/' /etc/rsyslog.conf
sudo sed -i 's/^#input(type="imtcp" port="514")/input(type="imtcp" port="514")/' /etc/rsyslog.conf

# create per-host log dir (optional)
sudo mkdir -p /var/log/remote
sudo chmod 755 /var/log/remote
# add file template (example)
cat <<'EOF' | sudo tee /etc/rsyslog.d/30-remote.conf
$template RemoteFormat,"/var/log/remote/%HOSTNAME%.log"
*.* ?RemoteFormat
& ~
EOF

sudo systemctl restart rsyslog
sudo ss -lntu | grep 514

Validation: On rsyslog server run sudo tail -F /var/log/remote/<switch-hostname>.log (or /var/log/syslog) and watch for messages once NAD sends logs.

---

C — Configure Catalyst switch to send syslog + time format + source

conf t
 service timestamps log datetime msec
 logging host 10.10.10.100 transport udp port 514
 logging trap informational
 logging source-interface Vlan1
 logging facility local7
exit
write memory
show logging

Validation (switch CLI):

• show logging — verify configured logging host(s) and trap level.
• On syslog server: tail -f /var/log/remote/<switch>.log and ensure lines arrive.

Tip: Use logging host 10.10.10.100 transport tcp port 514 if you prefer TCP (more reliable for SIEM).

---

D — Configure ISE to forward logs to SIEM (optional, useful for long-term correlation)

ISE can forward logs to remote syslog/SIEM. Exact menu may vary by version. Typical path:

GUI Steps (ISE):

1. Admin GUI → Administration → System → Logging (or: Administration → System → Logging → Remote Logging Targets).
   [Screenshot: ISE Admin → System → Logging → Add Remote Logging Target]
2. Add a Remote Logging Target:
   • Name: Syslog-SIEM
   • Host/IP: 10.10.10.100
   • Protocol: TCP/UDP, Port 514 (or your SIEM port)
   • Log Types: choose RADIUS, Policy, System, Authentication (or “All”)
   • Save & apply.

Validate: check syslog server for records coming from ISE; check ISE GUI for success status (some releases show ‘logging target status’).

If you don’t see the option, check ISE docs for your release — remote logging placement varies across versions.

---

E — Use ISE Live Logs to find the auth event

GUI Steps (ISE Live Logs):

1. ISE Admin GUI → Operations → RADIUS → Live Logs.
   [Screenshot placeholder: ISE Operations → RADIUS → Live Logs]
2. Filter by: Time Range, Username, MAC Address (Calling-Station-Id), NAS-IP-Address (the switch mgmt IP), or Port.
3. Open the session entry for the event — expand Authentication and Authorization details. Look for:
   • Authentication Method (dot1x / MAB)
   • Identity Source used (AD / Internal)
   • Matched Policy Rule (click the matched rule)
   • Authorization Result (Profile name)
   • Returned attributes: Tunnel-Private-Group-ID (VLAN), Filter-Id, Cisco-AVPair, dACL name
   • Failure reason (ISE provides a textual reason in most cases)
     [Screenshot placeholder: Live Logs session details showing VLAN attribute]

Interpretation tips:

• If Live Logs shows Access-Accept with VLAN attribute but switch shows different VLAN — check trunk allowed VLANs or switch attribute mapping.
• If Live Logs shows Access-Reject with AD error — check AD connectivity or credentials.

---

F — Configure detailed device logging on IOS for radius/802.1x

Use for short windows only — debugging is CPU/IO heavy.

debug dot1x all        ! shows dot1x events
debug aaa authentication   ! AAA debug
debug radius            ! RADIUS debug
terminal monitor        ! show debug output on SSH/console

Reproduce auth failure, collect debug output to a file (or copy). Stop debug quickly:

no debug all
undebug all

Validation: debug output will show RADIUS Access-Request/Access-Accept, EAPOL exchange steps, and reasons for failure.

---

G — Packet captures for EAPOL (switch) and RADIUS (ISE or tap)

Capture on switch (IOS built-in capture):

monitor capture CAP interface GigabitEthernet1/0/10 both
monitor capture CAP start
# Reproduce auth
monitor capture CAP stop
monitor capture CAP export tftp://10.10.10.100/gi10.pcap

Capture RADIUS on ISE (shell):
If you have shell access (lab only):

# Example: capture RADIUS traffic (UDP 1812/1813) on interface mgmt
sudo tcpdump -i eth0 -s0 -w /tmp/radius.pcap udp port 1812 or udp port 1813
# or examine logs: tail -f /var/log/messages (ISE logging location depends on version)

Open captures in Wireshark and filter:

• EAPOL: eapol
• RADIUS: radius
• Look for attributes: Tunnel-Private-Group-ID, Filter-Id, Cisco-AVPair.

---

H — Correlation workflow (step-by-step)

1. Start with the user: get username or MAC and exact timestamp (or narrow time window).
2. Search ISE Live Logs for that username/MAC/time → open the session → note: timestamp, NAS-IP (switch), Access-Request ID, Auth method, Identity Source, AuthZ profile and attributes.
3. Search syslog server for entries from the switch’s hostname/IP at that timestamp: port link events, 802.1X events, RADIUS reachability messages (e.g., RADIUS server 10.10.10.50 unreachable), or authorization application logs.
   • sudo grep 'Gi1/0/10' /var/log/remote/<switch>.log | grep '2025-08-11 10:1' (example)
4. Check switch CLI for show authentication sessions interface Gi1/0/10 details — verify what the switch thinks is the user/MAC, method, VLAN, and applied profile.
5. Check packet captures (EAPOL + RADIUS) to see if Access-Accept contained the attributes and if the switch applied them.
6. If mismatch: e.g., ISE sent VLAN but switch didn’t apply — inspect trunk allowed VLANs, and NAD capability (dACL vs Filter-ID), or vendor-specific attribute support.
7. If ISE shows no RADIUS request: verify show radius statistics on switch, show logging for UDP drops, and check intermediate devices for blocked UDP 1812/1813.
8. If intermittent: examine show memory/CPU on ISE PSN, and show processes cpu on switch; also review syslog history for transient errors.

---

I — Common log entries and what they mean (examples)

Example syslog lines (generic formats — vendor variations exist)

• DOT1X-5-AUTHEN_SUCCESS: Interface Gi1/0/10 Authenticated user <user@domain> method=dot1x
  → 802.1X succeeded; check ISE authz attributes next.
• RADIUS-3-NAS_UNREACHABLE: RADIUS server 10.10.10.50 unreachable
  → switch could not reach ISE; check network, firewall, or ISE service.
• DOT1X-5-PORT-STATUS: Interface Gi1/0/11 changed state to unauthorized
  → port not authorized; look for EAP failure or lack of Access-Accept.
• ISE Live Logs: Access-Accept returned Tunnel-Private-Group-ID = 20
  → ISE instructed VLAN 20; check switch accepted attribute.

---

FAQs — Syslog & Live Logs for Wired Auth Debugging

1. Q — Where do I find the “reason” for an authentication failure?
A: First place: ISE Live Logs (Operations → RADIUS → Live Logs) — open the session; ISE provides a textual failure reason (e.g., MSCHAPv2 failure, AD error, certificate error). If ISE logs indicate success but user cannot access, check switch syslog for local authorization errors or VLAN mismatches.

---

2. Q — Should I use UDP or TCP for syslog from switches to SIEM?
A: Use TCP (or TLS) for production SIEMs because it’s more reliable and avoids message loss. UDP is fine for labs but may drop messages.

---

3. Q — ISE Live Logs show Access-Accept but switch didn’t change VLAN. Why?
A: Common causes: VLAN not configured/allowed on trunk, NAD doesn’t support the attribute used (dACL vs Filter-ID vs VLAN attribute naming), or the switch applied a different authorization profile due to multiple matching rules. Check show authentication sessions and show ip interface trunk.

---

4. Q — How can I capture EAPOL frames if I don’t have a SPAN port?
A: Use the switch built-in monitor capture feature (IOS) to capture frames on the access interface and export via TFTP, or capture on the client PC using a mirrored port / USB Ethernet adapter in promiscuous mode.

---

5. Q — What timestamps should I use for correlation?
A: Use NTP-synced timestamps (ISO format) down to seconds/milliseconds. Correlate logs +/- 30 seconds for transient delays; if RADIUS timeouts exist expect up to several seconds for retries.

---

6. Q — Live Logs show EAP timeouts — how long does the switch wait before MAB fallback?
A: It depends on your switch EAP timeout settings; typical default is several seconds (varies by platform). You can configure order/timeouts with authentication timers and authentication order dot1x mab.

---

7. Q — How do I verify CoA reached the switch?
A: On the switch, check logs for inbound CoA messages (show logging or syslog) or use debug radius to see the disconnect/coa message processing. Also use ISE’s Live Logs → session → Disconnect/CoA action — check ISE logs for success/failure.

---

8. Q — Should I forward ISE logs to my SIEM or query Live Logs in GUI?
A: Both. Live Logs are excellent for quick triage; SIEM/central syslog is essential for long-term forensic analysis, alerting, and cross-system correlation (e.g., firewall, endpoint telemetry).

---

9. Q — What are the minimal syslog messages I should rely on from a switch for auth debugging?
A: Port link up/down, DOT1X auth success/failure, RADIUS unreachable or retries, VLAN assignment events, CoA acceptance/rejection. Ensure logging trap informational or lower to capture all these.

---

10. Q — My syslog messages show truncated content — how do I prevent truncation?
A: Enable service timestamps log datetime msec on switches and ensure your syslog server is configured to accept long messages. Some devices compress or truncate long messages — use TCP/TLS for better fidelity.

---

YouTube Link

For more in-depth Cisco ISE Mastery Training, subscribe to my YouTube channel Network Journey and join my instructor-led classes for hands-on, real-world ISE experience

---

Closing Notes (key takeaways)

• Always NTP-sync all devices (critical).
• Collect both sides: ISE Live Logs + network syslogs. Correlate by timestamp, port, MAC, username.
• Use packet captures (EAPOL + RADIUS) to confirm TLS handshakes and exact RADIUS attributes.
• Forward ISE logs to SIEM for long-term analysis and automated alerting.
• Keep a short-run debug window on network devices — debug is useful but CPU-heavy.
• Document the fix and update runbooks (e.g., if CoA needed firewall ports opened, add to runbook).

---

Fast-Track to Cisco ISE Mastery Pro

Want to master ISE troubleshooting end-to-end — from Live Logs to syslog correlation and packet captures — in real labs and live coaching?
I run a focused 4-month, Instructor-Led CCIE Security & Cisco ISE Mastery program with hands-on labs, troubleshooting playbooks, and 1:1 mentor time. You’ll get lab packs (EVE-NG/VMware), live troubleshooting scenarios like this one, and a ready-to-deploy runbook for your team.

Seats are limited — view course outline & enroll:
https://course.networkjourney.com/ccie-security/

Enroll Now & Future‑Proof Your Career
Email: info@networkjourney.com
WhatsApp / Call: +91 97395 21088

---