[Day #14 PyATS Series] Using pyATS “Jobs” for Multi-Device Tests with Cisco [Python for Network Engineer]

Introduction

As networks grow in scale and complexity, the ability to automate multi-device testing becomes a critical skill for any Python for Network Engineer practitioner. Cisco’s pyATS framework introduces the concept of Jobs, which allow engineers to orchestrate multiple testscripts and devices simultaneously. Instead of running individual scripts manually, a job file manages test execution, parallelization, logging, and result aggregation—making large-scale automation both manageable and efficient.

In this article, part of our 101 Days of pyATS (Vendor-Agnostic) series, we will walk you through building and executing a pyATS job file that runs health checks on multiple Cisco devices at once. You’ll learn how to:

• Define a pyATS job for multi-device orchestration
• Execute testscripts across multiple routers simultaneously
• Generate and analyze consolidated reports

By the end, you’ll be able to scale your automation scripts for enterprise-grade deployments. For those aiming to master this and other advanced automation skills, Trainer Sagar Dhawan’s 3-month instructor-led training provides hands-on experience with pyATS, Python, Ansible, and APIs.

Topology Overview

In this scenario, we have:

• Routers: 3 Cisco IOS XE routers (R1, R2, R3)
• Management Host: Running pyATS job execution
• Goal: Run a health check testsuite across all routers simultaneously

Diagram

Topology & Communications

• Protocol: SSH
• Transport: pyATS Testbed YAML file
• Tests Performed: Reachability, interface checks, device uptime
• Execution: Single job triggers multiple testscripts across all devices, leveraging parallel execution for efficiency.

Workflow Script

We will create:

1. A healthcheck.py testsuite (re-usable from previous lessons)
2. A job.py file that orchestrates running this script across all devices

healthcheck.py (Testscript)

from pyats import aetest
from genie.testbed import load
import logging

log = logging.getLogger(__name__)

class CommonSetup(aetest.CommonSetup):
    @aetest.subsection
    def connect(self, testbed):
        self.parent.parameters['devices'] = []
        for name, device in testbed.devices.items():
            device.connect(log_stdout=False)
            self.parent.parameters['devices'].append(device)
            log.info(f'Connected to {name}')

class VerifyReachability(aetest.Testcase):
    @aetest.test
    def ping_test(self, devices):
        for device in devices:
            result = device.ping('8.8.8.8')
            assert 'Success' in result, f'Ping failed on {device.name}'

class VerifyInterfaces(aetest.Testcase):
    @aetest.test
    def check_interfaces(self, devices):
        for device in devices:
            intf_info = device.learn('interface')
            for name, data in intf_info.info.items():
                assert data['enabled'] == True, f'{name} down on {device.name}'
                assert data['oper_status'] == 'up', f'{name} operationally down on {device.name}'

class VerifyUptime(aetest.Testcase):
    @aetest.test
    def uptime_check(self, devices):
        for device in devices:
            facts = device.learn('platform')
            uptime = facts.info['uptime']
            log.info(f'{device.name} uptime: {uptime}')
            assert uptime > 0, f'Uptime zero on {device.name}'

class CommonCleanup(aetest.CommonCleanup):
    @aetest.subsection
    def disconnect(self, devices):
        for device in devices:
            device.disconnect()

job.py (Job File)

from pyats.easypy import run

# Entry point for easypy

def main(runtime):
    # Run the health check script with the testbed
    run(testscript='healthcheck.py', testbed='testbed.yml')

Explanation by Line

• healthcheck.py: Defines tests for reachability, interfaces, and uptime for all devices in the testbed.
• CommonSetup: Connects to each device and stores them in a list for reuse in tests.
• Job File: job.py acts as an orchestrator, invoking healthcheck.py using easypy run.
• Parallel Execution: pyATS automatically handles multiple devices concurrently when using easypy jobs.
• Logs and Reports: Easypy generates structured reports for each device and test executed.

testbed.yml Example

testbed:
  name: Cisco_Lab
  devices:
    R1:
      os: iosxe
      type: router
      connections:
        cli:
          protocol: ssh
          ip: 192.168.1.101
      credentials:
        default:
          username: admin
          password: admin123
    R2:
      os: iosxe
      type: router
      connections:
        cli:
          protocol: ssh
          ip: 192.168.1.102
      credentials:
        default:
          username: admin
          password: admin123
    R3:
      os: iosxe
      type: router
      connections:
        cli:
          protocol: ssh
          ip: 192.168.1.103
      credentials:
        default:
          username: admin
          password: admin123

Post-validation CLI Screenshots (Expected Output)

When executing:

easypy job.py

You should see:

+----------------------------------------------------------------------------+
|                              Job Execution Report                          |
+----------------------------------------------------------------------------+
Healthcheck.VerifyReachability .......... PASSED
Healthcheck.VerifyInterfaces ............ PASSED
Healthcheck.VerifyUptime ................. PASSED
Total Devices: 3   Total Tests: 9   Passed: 9   Failed: 0
+----------------------------------------------------------------------------+

This confirms that all devices passed the health checks in parallel.

FAQs

Q1: What is a pyATS Job and how is it different from a Testscript?

A: A pyATS Job is a Python file that orchestrates one or more test scripts, defines testbed loading, passes runtime parameters, and controls how tests are executed in sequence or in parallel. Unlike a single testscript (which contains only test logic), a Job acts as the execution controller allowing you to run multiple testcases on multiple devices in a coordinated fashion.

Q2: Why should I use Jobs for multi-device testing?

A: Jobs make it easier to run large-scale automated tests across 10, 100, or even 1000+ devices without manually launching scripts. They enable:

• Parallel execution of test scripts
• Dynamic testbed loading
• Passing variables across scripts
• Scheduling recurring tests
• Centralized reporting
  This is essential for large enterprise or data center environments where multiple Cisco routers, switches, or firewalls need to be validated simultaneously.

Q3: Can a single Job run tests on different Cisco OS types (IOS, IOS-XE, NX-OS, IOS-XR)?

A: Yes. pyATS Jobs are vendor and OS-agnostic. You can define testcases for each OS type and include them in the same job. The testbed YAML maps devices to their respective OS, allowing testcases to handle platform-specific logic dynamically.

Q4: How do I run a pyATS Job with multiple test scripts?

A: You create a Job file that imports multiple testscript modules and specifies which tests to execute. Example:

pyats run job multi_device_job.py --testbed testbed.yaml

This will load the testbed, establish connections to all devices, and execute the included testcases in parallel or serially based on job configuration.

Q5: Can I use Jobs to schedule compliance checks or nightly regression tests?

A: Yes. Jobs can be integrated with CI/CD tools like Jenkins, GitLab CI, or Cron to schedule compliance checks, nightly health checks, or regression tests. This ensures continuous validation of your Cisco network and faster detection of configuration drift or outages.

Q6: How does pyATS Job handle failures during multi-device execution?

A: If a device fails to connect or a testcase fails, pyATS continues running remaining tests while marking failures in the final report. Jobs also support stop_on_failure options to control whether the job halts or continues on errors, providing flexibility for production testing.

Q7: Can pyATS Jobs generate a combined report for all devices tested?

A: Absolutely. One of the key advantages of using Jobs is consolidated reporting. The final HTML, JSON, or xUnit report aggregates results from all testcases and devices, making it easy to review network health at a glance and share results with teams or management.

YouTube Link

Watch the Complete Python for Network Engineer: Using pyATS “Jobs” for Multi-Device Tests with Cisco [Python for Network Engineer] Lab Demo & Explanation on our channel:

Join Our Training

Managing multi-device tests using pyATS jobs is an essential skill for modern network engineers. To gain hands-on experience in creating scalable automation frameworks, join Trainer Sagar Dhawan’s 3-month instructor-led Python/Ansible/API course. This program is designed to help you become an expert in Python for Network Engineer workflows.

Click here to join the training and start your journey towards advanced network automation.

Enroll Now & Future‑Proof Your Career
Email: info@networkjourney.com
WhatsApp / Call: +91 97395 21088