[Day #99 PyATS Series] Building Your Own pyATS Testing Framework (Plugin-Based) Using pyATS for Cisco [Python for Network Engineer]

Introduction: Key Concepts

Welcome to Day #99 of the 101 Days of pyATS (Vendor-Agnostic) series. Today, I will dive into building your own pyATS testing framework using a plugin-based architecture. For a professional Python for Network Engineer, understanding how to design modular, reusable, and extendable testing frameworks is critical.

With this approach, you can:

• Create custom plugins for device validation (interfaces, routing, services).
• Ensure multi-vendor compatibility across Cisco, Arista, Palo Alto, and FortiGate.
• Integrate GUI and CLI validation.
• Automate test execution, reporting, and post-validation checks.
• Build a framework that can be extended and reused across multiple projects.

This masterclass focuses on production-ready frameworks, including step-by-step scripts, explanations, testbed examples, and real-world CLI outputs.

Topology Overview

Our framework is designed to validate a multi-vendor lab network consisting of:

• Core Layer: Cisco ISR routers
• Distribution Layer: Cisco Catalyst and Arista switches
• Edge Layer: Palo Alto and FortiGate firewalls
• Management Node: pyATS automation server running Python scripts

Topology diagram:

This lab topology allows us to simulate real-world testing environments, making our plugin-based framework reusable and vendor-agnostic.

Topology & Communications

• Device Communication: SSH (CLI), REST APIs for devices supporting automation APIs.
• Framework Architecture:
  • Core engine: pyATS execution logic
  • Plugins: Individual Python modules for specific validations (interfaces, BGP, OSPF, VLANs, licenses)
  • Reports: JSON/HTML outputs
• Parallelization: pyATS pcall allows simultaneous execution across devices.
• Validation Flow: GUI snapshots, CLI parsing, structured reporting.

Workflow Script

The workflow is modular:

1. Load testbed.
2. Load plugins dynamically.
3. Execute plugin validation on target devices.
4. Collect structured outputs.
5. Generate comprehensive reports.

Core Framework Script

import importlib
import os
from genie.testbed import load
from pyats.async_ import pcall

# Load testbed
testbed = load('testbed.yml')

# Discover plugins
PLUGIN_DIR = 'plugins'
plugins = [f.replace('.py','') for f in os.listdir(PLUGIN_DIR) if f.endswith('.py')]

def run_plugin(device_name, plugin_name):
    device = testbed.devices[device_name]
    device.connect()
    plugin_module = importlib.import_module(f'plugins.{plugin_name}')
    result = plugin_module.run(device)
    device.disconnect()
    return {device_name: result}

# Execute all plugins on all devices
for plugin in plugins:
    results = pcall(run_plugin, *[(dev.name, plugin) for dev in testbed.devices.values()])
    # Save results per plugin
    import json
    with open(f'{plugin}_results.json', 'w') as f:
        json.dump(results, f, indent=2)
    print(f'{plugin} executed successfully.')

Sample Plugin: Interface Validation (plugins/interface_check.py)

def run(device):
    """Check interfaces for operational state and speed/duplex"""
    interfaces = device.parse('show interfaces')
    report = {}
    for intf, details in interfaces.items():
        report[intf] = {
            'status': details['status'],
            'admin_status': details['admin_status'],
            'speed': details.get('speed'),
            'duplex': details.get('duplex')
        }
    return report

Explanation by Line

• testbed = load(‘testbed.yml’): Loads multi-vendor devices.
• importlib.import_module(…): Dynamically imports plugins.
• pcall(…): Executes validation across devices in parallel.
• json.dump(…): Stores structured output for reporting.
• plugins/interface_check.py: Modular, reusable plugin for interface validation.

The plugin-based design allows easy extension by creating new modules without modifying the core engine.

testbed.yml Example

testbed:
  name: MultiVendorLab
  devices:
    core1:
      type: router
      os: iosxe
      connections:
        cli:
          protocol: ssh
          ip: 10.10.10.1
          username: admin
          password: Cisco123
    dist1:
      type: switch
      os: iosxe
      connections:
        cli:
          protocol: ssh
          ip: 10.10.20.1
          username: admin
          password: Cisco123
    arista1:
      type: switch
      os: eos
      connections:
        cli:
          protocol: ssh
          ip: 10.10.30.1
          username: admin
          password: Arista123
    palo1:
      type: firewall
      os: panos
      connections:
        cli:
          protocol: ssh
          ip: 10.10.40.1
          username: admin
          password: Palo123
    forti1:
      type: firewall
      os: fortios
      connections:
        cli:
          protocol: ssh
          ip: 10.10.50.1
          username: admin
          password: Forti123

Post-Validation CLI (Expected Outputs)

Example: Interface Plugin Results

core1> show interfaces
Interface    Status  Admin   Speed  Duplex
Gi0/0        up      up      1000   full
Gi0/1        up      up      1000   full

Example: BGP Plugin Results

core1> show bgp summary
Neighbor     AS     State
10.10.20.2   65002  established

All results are collected in JSON, allowing you to generate Markdown/PDF documentation.

FAQs

Q1: What is a plugin-based pyATS testing framework?
A1: It is a modular automation architecture where each test, validation, or workflow is developed as a self-contained plugin. This allows network engineers to reuse, extend, and share individual components, creating a scalable and maintainable testing ecosystem.

Q2: Why should I build a plugin-based framework instead of scripting individual tests?
A2: Traditional scripts are often hard-coded and inflexible. A plugin-based framework provides:

• Reusability: Individual tests can be invoked across multiple topologies.
• Scalability: New tests can be added without rewriting the core framework.
• Consistency: Standardized validation and reporting across multi-vendor networks.
• Integration: Easy integration with CI/CD pipelines or GitOps workflows.

Q3: How are plugins structured in a pyATS testing framework?
A3: Typically, each plugin contains:

• Test logic (Python functions using pyATS/Genie parsers)
• Metadata (plugin name, supported vendors, version)
• Configuration parameters (device lists, VLANs, interfaces)
• Reporting hooks (logs, JSON/HTML output)
  This modular approach decouples tests from core orchestration, simplifying maintenance.

Q4: Can a plugin-based framework support multi-vendor testing?
A4: Yes. Using Genie parsers and pyATS abstractions, each plugin can handle vendor-specific nuances while maintaining consistent output structure, allowing you to validate Cisco, Arista, Juniper, and other devices in the same framework.

Q5: How does the framework handle dependencies between plugins?
A5: Plugins can define preconditions and postconditions, enabling the framework to:

• Ensure pre-requisite tests (e.g., interface validation) run first.
• Skip dependent tests if critical conditions fail.
• Execute post-validation cleanup or reporting automatically.

Q6: How are results captured and reported in a plugin-based framework?
A6: The framework can aggregate results from multiple plugins into:

• CLI/log outputs for quick validation
• Structured JSON for programmatic consumption
• HTML/PDF reports for management visibility
  This allows a single source of truth for network validation.

Q7: Can this framework be integrated with version control and CI/CD pipelines?
A7: Absolutely. Each plugin can reside in Git repositories, enabling versioning, review, and testing. Integration with GitHub Actions, Jenkins, or GitLab CI allows automatic execution of validation tests after configuration changes or pull requests, supporting GitOps for network automation.

Q8: What are the benefits for network engineers and teams?
A8: A plugin-based pyATS framework:

• Accelerates multi-vendor network validation
• Reduces human errors in repetitive tests
• Encourages automation best practices
• Provides reusable assets for new topologies or projects
• Enhances training, documentation, and team collaboration

YouTube Link

Watch the Complete Python for Network Engineer: Building Your Own pyATS Testing Framework (Plugin-Based) Using pyATS for Cisco [Python for Network Engineer] Lab Demo & Explanation on our channel:

Join Our Training

Become a master Python for Network Engineer by building production-ready, plugin-based pyATS frameworks. Enroll in Trainer Sagar Dhawan’s 3-month instructor-led course and learn:

• Full automation frameworks
• Multi-vendor network validation
• CI/CD integration for network automation
• Real-world lab simulations and plugin development

Join the course here

Take control of multi-vendor automation and start building scalable, modular frameworks today.

Enroll Now & Future‑Proof Your Career
Email: info@networkjourney.com
WhatsApp / Call: +91 97395 21088