[Day #47 PyATS Series] Logging & Debug Outputs Parsing Automation Using pyATS for Cisco [Python for Network Engineer]

• Trainer Sagar Dhawan
• Posted on September 6, 2025
• No Comments

[Day #47 PyATS Series] Logging & Debug Outputs Parsing Automation Using pyATS for Cisco [Python for Network Engineer]

Introduction on the Key Points

When managing Cisco networks, logging and debugging outputs provide the most critical visibility into network events. However, parsing through thousands of syslog messages or debugging outputs manually is time-consuming and error-prone.

This is where pyATS comes in — Cisco’s Python-based test automation framework. Using pyATS, network engineers can automate the collection, parsing, and validation of syslog/debug messages in a structured manner.

In this masterclass article, we’ll build a step-by-step pyATS automation workflow that:

• Collects syslog/debug logs from Cisco routers and switches.
• Parses log lines into structured Python dictionaries using Genie parsers.
• Correlates log messages with events (interface flaps, OSPF neighborship resets, ACL denies).
• Validates log consistency across multiple devices.
• Produces GUI dashboards (ELK/Splunk/Kibana) for visualization.

If you are a Python for Network Engineer learner, this post will be your complete workbook on automating Cisco log parsing.

---

Topology Overview

Our test topology is a Cisco IOS-XE lab consisting of:

• 2x Core Routers (CSR1000v)
• 2x Access Switches (IOS-XE)
• 1x Syslog Server (Linux with rsyslog + Kibana dashboard)
• 1x pyATS Controller (automation host)

Topology Diagram

• All devices send logs to the Syslog server.
• pyATS connects via SSH to devices for on-demand debug collection.
• Syslog server provides GUI validation in Kibana.

---

Topology & Communications

• Management network: 10.0.1.0/24 (pyATS + Syslog server + device mgmt).
• Syslog UDP Port: 514.
• pyATS communicates via SSH (Netmiko/Genie harness) to Cisco devices.
• Logs are exported to JSON/CSV for further reporting.

---

Workflow Script

Here is the pyATS job file (log_parse_job.py):

from genie.testbed import load
import re

# Load the testbed
testbed = load('testbed.yml')

# Connect to devices
for device in testbed.devices.values():
    device.connect()

# Step 1: Collect syslog and debug outputs
for device in testbed.devices.values():
    print(f"\n--- Collecting logs from {device.name} ---")
    logs = device.execute("show logging")
    with open(f"{device.name}_logs.txt", "w") as f:
        f.write(logs)

# Step 2: Parse syslog messages
pattern = r"%(\S+)-(\d+)-(\S+): (.*)"
for device in testbed.devices.values():
    with open(f"{device.name}_logs.txt") as f:
        for line in f:
            match = re.search(pattern, line)
            if match:
                facility, severity, mnemonic, message = match.groups()
                print({
                    "device": device.name,
                    "facility": facility,
                    "severity": severity,
                    "mnemonic": mnemonic,
                    "message": message
                })

---

Explanation by Line

1. Load testbed → imports the YAML file that defines device credentials.
2. device.connect() → establishes SSH sessions to all devices.
3. show logging → retrieves current syslog buffer from the device.
4. Regex parser (%FACILITY-SEVERITY-MNEMONIC: MESSAGE) → extracts structured fields.
5. Write logs to file → ensures logs are saved for historical comparison.
6. Print parsed JSON → converts raw logs into structured output for automation pipelines.

---

testbed.yml Example

testbed:
  name: logging_lab
  credentials:
    default:
      username: admin
      password: Cisco123

devices:
  CSR1:
    os: iosxe
    type: router
    connections:
      cli:
        protocol: ssh
        ip: 10.0.1.11
  CSR2:
    os: iosxe
    type: router
    connections:
      cli:
        protocol: ssh
        ip: 10.0.1.12
  SW1:
    os: iosxe
    type: switch
    connections:
      cli:
        protocol: ssh
        ip: 10.0.1.21
  SW2:
    os: iosxe
    type: switch
    connections:
      cli:
        protocol: ssh
        ip: 10.0.1.22

---

Post-Validation CLI Screenshots (Real Expected Output)

Example 1: show logging (Cisco IOS-XE)

*Mar  1 00:02:12.071: %LINK-3-UPDOWN: Interface GigabitEthernet1, changed state to up
*Mar  1 00:02:13.234: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1, changed state to up
*Mar  1 00:05:02.999: %SYS-5-CONFIG_I: Configured from console by admin

Example 2: pyATS Parsed Output

{
  "device": "CSR1",
  "facility": "LINK",
  "severity": "3",
  "mnemonic": "UPDOWN",
  "message": "Interface GigabitEthernet1, changed state to up"
}

---

FAQs – Logging & Debug Outputs Parsing Automation with pyATS

Q1. Why is parsing logging and debug outputs important in network automation?

Answer:
Raw debug or syslog messages are unstructured text that humans can read, but automation tools need structured data. Parsing these logs allows engineers to:

• Extract root cause events (e.g., interface flaps, routing protocol errors).
• Detect patterns across multiple devices.
• Feed parsed data into dashboards or alerting systems.
  With pyATS, you can convert unstructured logs into structured JSON, making troubleshooting automated and repeatable instead of manual.

---

Q2. How does pyATS parse unstructured log outputs from Cisco devices?

Answer:
pyATS uses Genie parsers and custom regex-based extractors. For example:

• Command: show logging → Genie converts logs into structured dictionaries.
• Debug outputs (like debug ip ospf events) can be parsed with custom pyATS parsers written in Python.
  This ensures that instead of grepping through long text, you can run Python queries like:

for log in logs['events']:
    if "DOWN" in log['message']:
        print("Interface down detected:", log['timestamp'])

---

Q3. What common issues can be detected automatically from debug/log parsing?

Answer:
Automation helps detect:

1. Interface flaps (link up/down messages).
2. EIGRP/OSPF/BGP neighbor loss in debugs.
3. Authentication failures (AAA logs).
4. ACL or firewall drops recorded in syslogs.
5. High CPU/memory warnings.

Instead of manually scrolling, pyATS workflows flag only anomalies relevant to the engineer.

---

Q4. How can pyATS integrate log parsing with proactive alerts?

Answer:
Parsed logs can be fed into:

• Slack/Webex alerts using Python libraries.
• ELK/Prometheus dashboards for visualization.
• Email notifications when critical events (like %LINEPROTO-5-UPDOWN) occur.
  This turns log parsing into a real-time monitoring system instead of a static troubleshooting method.

---

Q5. Can pyATS differentiate between informational, warning, and error logs?

Answer:
Yes. pyATS parsing scripts can classify logs based on:

• Syslog severity levels (0 = emergency, 7 = debug).
• Keyword matches (%LINK-3-UPDOWN → Critical, %SYS-5-CONFIG_I → Info).
• Device-specific error codes.

This ensures that engineers don’t get alert fatigue and only receive actionable events.

---

Q6. How does pyATS help when dealing with high-volume debug outputs?

Answer:
Debugs can generate thousands of lines per second. pyATS can:

• Apply regex filters to capture only relevant lines.
• Stream logs into a structured file (JSON/CSV).
• Compare events before vs after a change to validate impact.
  For example, before a config push, you can capture logs, run automation, and then detect whether new errors appeared.

---

Q7. Can pyATS logging/debug parsers be customized for vendor-specific formats?

Answer:
Yes. While Genie parsers cover Cisco IOS-XE/XR/NX-OS, you can build custom pyATS plugins to parse:

• Arista EOS logs.
• Palo Alto system logs.
• FortiGate event logs.
  This makes pyATS vendor-agnostic, letting engineers monitor multi-vendor environments with the same automation framework.

---

Q8. What’s the difference between CLI scraping logs vs. pyATS parsing logs?

Answer:

• CLI scraping (manual/expect scripts): Reads raw text → needs regex per command → brittle if CLI changes.
• pyATS parsing: Converts CLI/debug into Python dictionaries automatically → stable, reusable, and readable.

This means instead of brittle regex-heavy scripts, you write clean Python validations like:

assert "LINK-3-UPDOWN" not in logs['critical'], "Interface flap detected!"

---

---

YouTube Link

Watch the Complete Python for Network Engineer: Logging & debug outputs parsing automation Using pyATS for Cisco [Python for Network Engineer] Lab Demo & Explanation on our channel:

---

Join Our Training

Mastering log parsing with pyATS is only one piece of automation. Imagine scaling this skill to VLAN audits, BGP checks, NETCONF APIs, and config drift detection across Cisco, Arista, Palo Alto, and Fortinet.

I run a focused 3-month, instructor-led program on:

• Python for Network Engineer (beginner → advanced)
• Ansible for automation
• API integrations (REST, NETCONF, gNMI)
• Cisco DevNet exam preparation

Check Course Outline Here

If you’re serious about moving beyond CLI and becoming a true network automation engineer, this course is your gateway.

Enroll Now & Future‑Proof Your Career
Email: info@networkjourney.com
WhatsApp / Call: +91 97395 21088

---