第309篇：Ansible Tower/AWX——自动化调度

关键词：Ansible Tower、AWX、Web界面、任务调度、RBAC、执行审计、Webhook、凭证管理

一、引言

Ansible Playbook在命令行下运行十分高效，但当自动化规模扩大后，多个运维团队共享同一套Playbook时，纯命令行方式暴露出明显短板：无权限管控、无执行审计、无可视化调度、无集中式日志。

Ansible Tower是Red Hat提供的企业级Ansible管理平台（商业版），AWX是其开源社区替代版，两者功能高度一致。它们为Playbook提供了Web界面、RBAC权限模型、定时调度、Webhook触发、执行仪表盘和审计日志，是Ansible从"工程师自用工具"进化为"团队级平台"的关键一步。

二、AWX 安装与架构

2.1 安装方式对比

2.2 Docker Compose 快速安装

# 克隆AWX安装仓库
git clone https://github.com/ansible/awx.git
cd awx/installer

# 编辑inventory，配置安装参数
# 关键配置项：
# admin_password          - Web界面管理员密码
# admin_user              - Web界面管理员用户名
# postgres_password       - 数据库密码
# secret_key              - 加密密钥（必须随机生成）
# host_port               - Web访问端口（默认80）

# 生成随机密钥
python3 -c "import secrets; print(secrets.token_urlsafe(30))"

# 执行安装
ansible-playbook -i inventory install.yml

2.3 架构组件

三、核心概念与配置

3.1 核心概念映射

3.2 配置 Inventory

在AWX Web界面或通过API创建Inventory：

# 通过API导入Inventory
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/inventories/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Production-Network",
    "organization": 1,
    "kind": "tower"
  }'

# 导入主机
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/inventories/1/hosts/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "core-sw-01",
    "variables": {
      "ansible_host": "192.168.1.1",
      "ansible_network_os": "ansible.netcommon.vyos",
      "ansible_connection": "ansible.netcommon.network_cli",
      "ansible_user": "admin",
      "device_role": "core"
    }
  }'

3.3 配置 Project

# 创建Git项目源
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/projects/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Network-Automation",
    "organization": 1,
    "scm_type": "git",
    "scm_url": "https://github.com/company/network-automation.git",
    "scm_branch": "main",
    "scm_clean": true,
    "scm_track_submodules": false
  }'

# 更新项目（拉取最新代码）
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/projects/1/update/

3.4 配置 Credential

# SSH凭据
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/credentials/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Network-SSH-Creds",
    "organization": 1,
    "credential_type": 5,
    "inputs": {
      "username": "admin",
      "ssh_key_data": "-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----"
    }
  }'

# 网络凭据（Netmiko密码）
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/credentials/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Network-Device-Password",
    "organization": 1,
    "credential_type": 19,
    "inputs": {
      "username": "admin",
      "password": "H@wa123!",
      "authorize": true,
      "authorize_password": "En@ble123!"
    }
  }'

# Vault密码
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/credentials/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Ansible-Vault",
    "organization": 1,
    "credential_type": 3,
    "inputs": {
      "vault_password": "MyVaultPass2025!"
    }
  }'

四、Job Template 与执行

4.1 创建 Job Template

# 创建Job Template
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/job_templates/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Deploy-VLANs-to-Core",
    "description": "批量部署VLAN配置到核心交换机",
    "job_type": "run",
    "inventory": 1,
    "project": 1,
    "playbook": "playbooks/deploy_vlans.yaml",
    "credentials": [2, 3],
    "extra_vars": "{\"target_group\": \"core\"}",
    "verbosity": 1,
    "limit": "",
    "forks": 5
  }'

4.2 执行 Job Template

# 立即执行
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/job_templates/1/launch/
# 返回: {"id": 42}

# 查看执行状态
curl -k -u admin:password \
  https://awx-server/api/v2/jobs/42/

# 查看执行日志
curl -k -u admin:password \
  https://awx-server/api/v2/jobs/42/stdout/

4.3 Survey：运行时参数输入

# 在Job Template中配置Survey
survey_spec:
  name: "VLAN Deployment Survey"
  description: "选择要部署的VLAN和目标设备组"
  spec:
    - question_name: "Target Group"
      question_description: "选择目标设备分组"
      required: true
      type: "multiplechoice"
      variable: "target_group"
      choices: |
        core
        aggregation
        access
        all
      default: "core"

    - question_name: "VLAN IDs"
      question_description: "要创建的VLAN ID（逗号分隔）"
      required: true
      type: "text"
      variable: "vlan_ids"
      default: "10,20,30"

    - question_name: "Dry Run Mode"
      question_description: "是否仅预览不实际下发"
      required: false
      type: "boolean"
      variable: "dry_run"
      default: false

# Playbook中使用Survey变量
---
- name: 基于Survey参数部署VLAN
  hosts: "{{ target_group }}"
  gather_facts: no

  vars:
    vlan_list: "{{ vlan_ids | split(',') }}"

  tasks:
    - name: 部署VLAN配置
      ansible.netcommon.config:
        lines:
          - "vlan {{ item }}"
          - "description AUTO_DEPLOYED_{{ item }}"
      loop: "{{ vlan_list }}"
      when: dry_run | bool | default(false) | not

    - name: 预览模式（Dry Run）
      ansible.builtin.debug:
        msg: "DRY RUN: 将在 {{ inventory_hostname }} 上创建 VLAN {{ item }}"
      loop: "{{ vlan_list }}"
      when: dry_run | bool | default(false)

五、调度与通知

5.1 定时任务（Schedule）

# 创建定时任务：每天凌晨2点执行配置备份
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/job_templates/1/schedules/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Daily-Backup-at-02:00",
    "rrule": "DTSTART:20250101T020000Z RRULE:FREQ=DAILY",
    "description": "每天凌晨2点执行全网配置备份"
  }'

# 创建周期性巡检：每6小时执行
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/job_templates/2/schedules/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Health-Check-Every-6H",
    "rrule": "DTSTART:20250101T000000Z RRULE:FREQ=HOURLY;INTERVAL=6"
  }'

常见rrule格式： | 频率 | rrule | |------|-------| | 每天 | FREQ=DAILY | | 每周 | FREQ=WEEKLY | | 每月 | FREQ=MONTHLY | | 每小时 | FREQ=HOURLY | | 每N小时 | FREQ=HOURLY;INTERVAL=N |

5.2 通知（Notification）

# 创建邮件通知模板
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/notification_templates/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Email-On-Failure",
    "organization": 1,
    "notification_type": "email",
    "avatars": {
      "email": {
        "email_host": "smtp.company.com",
        "email_host_user": "ansible@company.com",
        "email_host_password": "smtp_password",
        "email_port": 587,
        "email_use_tls": true,
        "email_send_to": [
          "netops-team@company.com"
        ]
      }
    }
  }'

# 将通知绑定到Job Template（失败时发送）
curl -k -u admin:password \
  -X PATCH https://awx-server/api/v2/job_templates/1/ \
  -H 'Content-Type: application/json' \
  -d '{
    "notifications": {
      "error": [1]
    }
  }'

5.3 Webhook 触发

# 配置Webhook，供外部系统（如GitLab CI）触发
# AWX自动生成Webhook URL：
# POST https://awx-server/api/v2/job_templates/1/webhook_token/xxxx/launch/

# 示例：GitLab CI Pipeline触发Ansible部署
# .gitlab-ci.yml
deploy_network:
  stage: deploy
  script:
    - |
      curl -X POST \
        https://awx-server/api/v2/job_templates/1/webhook_token/abc123/launch/ \
        -H "Content-Type: application/json" \
        -d '{"extra_vars": {"deploy_version": "'$CI_COMMIT_SHA'"}}'

六、RBAC 权限管理

6.1 角色层级

6.2 内置角色

6.3 API 分配角色

# 查询用户
curl -k -u admin:password https://awx-server/api/v2/users/

# 为用户分配角色
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/job_templates/1/access_list/ \
  -H 'Content-Type: application/json' \
  -d '{
    "username": "netops-zhangsan",
    "role_field": "execute_role"
  }'

# 为团队（Team）分配角色
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/inventories/1/access_list/ \
  -H 'Content-Type: application/json' \
  -d '{
    "teamname": "Network-Operations",
    "role_field": "read_role"
  }'

七、Workflow：多任务编排

7.1 编排场景

网络变更通常涉及多个步骤：备份 → 部署 → 验证 → 通知。Workflow将多个Job Template串联执行：

# 创建Workflow Job Template
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/workflow_job_templates/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Safe-Network-Change-Workflow",
    "description": "安全变更流程：备份→部署→验证→通知",
    "organization": 1,
    "inventory": 1,
    "survey_enabled": true
  }'

# 添加Workflow节点
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/workflow_job_templates/1/workflow_nodes/ \
  -H 'Content-Type: application/json' \
  -d '{
    "identifier": "step1_backup",
    "unified_job_template": 5,
    "success_nodes": ["step2_deploy"]
  }'

# 第二步：部署
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/workflow_job_templates/1/workflow_nodes/ \
  -H 'Content-Type: application/json' \
  -d '{
    "identifier": "step2_deploy",
    "unified_job_template": 6,
    "success_nodes": ["step3_verify"],
    "failure_nodes": ["step4_rollback"]
  }'

# 第三步：验证
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/workflow_job_templates/1/workflow_nodes/ \
  -H 'Content-Type: application/json' \
  -d '{
    "identifier": "step3_verify",
    "unified_job_template": 7,
    "success_nodes": ["step5_notify_success"]
  }'

# 第四步：回滚（失败时执行）
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/workflow_job_templates/1/workflow_nodes/ \
  -H 'Content-Type: application/json' \
  -d '{
    "identifier": "step4_rollback",
    "unified_job_template": 8
  }'

# 第五步：通知成功
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/workflow_job_templates/1/workflow_nodes/ \
  -H 'Content-Type: application/json' \
  -d '{
    "identifier": "step5_notify_success",
    "unified_job_template": 9
  }'

7.2 Workflow 执行流程

八、故障案例：AWX并发执行导致设备锁死

8.1 故障现象

运维团队在AWX上同时触发了5个Job Template，每个Template操作10台设备（共50台）。结果30台设备的SSH连接被拒绝，AWX返回大量Connection refused错误。

8.2 根因分析

1. AWX的forks参数默认设为10，每个Job Template同时连接10台设备
2. 5个Job Template并行运行，瞬时并发SSH连接数达到50
3. 部分老旧交换机SSH并发连接上限为5，超出限制的连接被拒绝
4. AWX重试机制在短时间内连续重试，加剧了设备的SSH压力

8.3 解决方案

# 方案1：降低Job Template的forks
# 在Job Template中设置 forks=3（每台设备串行+少量并行）

# 方案2：使用Semaphore限制全局并发
# AWX Semaphore 机制（类似信号量）
# 创建 Semaphore
curl -k -u admin:password \
  -X POST https://awx-server/api/v2/semaphores/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "network-device-limit",
    "organization": 1,
    "max_slots": 10,      # 全局最多10个并发设备操作
    "description": "限制网络自动化任务的全局并发数"
  }'

# 在Job Template中绑定Semaphore
# 这样即使同时触发5个Job，实际并发不超过10个设备

# 方案3：Schedule错峰执行
# 将5个Job的Schedule错开时间，避免同时触发

八（续）、高级主题

8.1 AWX 项目管理最佳实践

# projects/network-automation/ 目录结构
network-automation/
├── group_vars/
│   ├── core.yaml          # 核心设备通用变量
│   ├── aggregation.yaml   # 汇聚设备通用变量
│   └── access.yaml        # 接入设备通用变量
├── inventories/
│   ├── production/        # 生产环境清单
│   │   ├── hosts.yaml
│   │   └── group_vars/
│   └── staging/           # 测试环境清单
├── roles/
│   ├── vlan_management/   # VLAN 管理角色
│   ├── interface_config/  # 接口配置角色
│   ├── bgp_config/        # BGP 配置角色
│   └── config_backup/     # 配置备份角色
├── playbooks/
│   ├── deploy_vlans.yaml
│   ├── backup_all.yaml
│   ├── verify_bgp.yaml
│   └── emergency_rollback.yaml
├── templates/
│   ├── bgp.j2
│   ├── interface.j2
│   └── vlan.j2
└── ansible.cfg            # 项目级 Ansible 配置

8.2 AWX 与 CI/CD 集成流水线

# .gitlab-ci.yml - 网络自动化 CI/CD 流水线
stages:
  - validate
  - stage-deploy
  - verify
  - production-deploy
  - post-verify

# 阶段1: 验证 Playbook 语法
validate_playbook:
  stage: validate
  script:
    - ansible-playbook --syntax-check playbooks/deploy_vlans.yaml
    - yamllint playbooks/ roles/
  only:
    - merge_requests

# 阶段2: 在测试环境通过 AWX 执行
stage_deploy:
  stage: stage-deploy
  script:
    - |
      curl -k -u "$AWX_USER:$AWX_TOKEN" \
        -X POST $AWX_URL/api/v2/job_templates/staging-vlan-deploy/launch/ \
        -H 'Content-Type: application/json' \
        -d "{\"extra_vars\": {\"deploy_version\": \"$CI_COMMIT_SHA\"}}"
  only:
    - main

# 阶段3: 自动验证部署结果
verify_stage:
  stage: verify
  script:
    - |
      # 等待 AWX 任务完成
      sleep 30
      curl -k -u "$AWX_USER:$AWX_TOKEN" \
        $AWX_URL/api/v2/job_templates/staging-vlan-deploy/last_job/ | jq '.status'
  only:
    - main

# 阶段4: 生产部署（手动触发）
production_deploy:
  stage: production-deploy
  script:
    - |
      curl -k -u "$AWX_USER:$AWX_TOKEN" \
        -X POST $AWX_URL/api/v2/job_templates/prod-vlan-deploy/launch/ \
        -H 'Content-Type: application/json' \
        -d "{\"extra_vars\": {\"deploy_version\": \"$CI_COMMIT_SHA\"}}"
  when: manual
  only:
    - main

# 阶段5: 生产环境后验证
post_verify:
  stage: post-verify
  script:
    - |
      curl -k -u "$AWX_USER:$AWX_TOKEN" \
        -X POST $AWX_URL/api/v2/job_templates/prod-verify/launch/
  only:
    - main

8.3 AWX 执行审计与合规报告

# scripts/awx_audit_report.py
import requests
from datetime import datetime, timedelta
import csv

class AWXAuditReporter:
    """AWX 执行审计报表生成器"""

    def __init__(self, awx_url, token):
        self.base_url = awx_url.rstrip("/")
        self.session = requests.Session()
        self.session.headers.update({
            "Authorization": f"Bearer {token}",
            "Accept": "application/json"
        })
        self.session.verify = False

    def get_jobs_since(self, days=30):
        """获取指定时间段内的所有执行记录"""
        since = (datetime.utcnow() - timedelta(days=days)).strftime("%Y-%m-%dT%H:%M:%SZ")
        params = {"order_by": "-created", "created__gte": since}
        jobs = []
        url = f"{self.base_url}/api/v2/jobs/"

        while url:
            resp = self.session.get(url, params=params)
            data = resp.json()
            jobs.extend(data["results"])
            url = data.get("next")
        return jobs

    def generate_audit_csv(self, jobs, output_file="audit_report.csv"):
        """生成审计 CSV 报表"""
        with open(output_file, "w", newline="", encoding="utf-8") as f:
            writer = csv.writer(f)
            writer.writerow([
                "执行时间", "Job Template", "Playbook", "状态",
                "执行人", "变更设备数", "成功数", "失败数", "跳过数", "耗时(秒)"
            ])
            for job in jobs:
                writer.writerow([
                    job.get("created", ""),
                    job.get("job_template_name", ""),
                    job.get("playbook", ""),
                    job.get("status", ""),
                    job.get("uniform_execution_environment", {}).get("name", ""),
                    job.get("summary_fields", {}).get("hosts", {}).get("total", 0),
                    job.get("summary_fields", {}).get("hosts", {}).get("ok", 0),
                    job.get("summary_fields", {}).get("hosts", {}).get("failures", 0),
                    job.get("summary_fields", {}).get("hosts", {}).get("skipped", 0),
                    round(job.get("ended", "") - job.get("started", ""), 2) if job.get("ended") and job.get("started") else ""
                ])
        return output_file

    def generate_compliance_summary(self, jobs):
        """生成合规摘要"""
        total = len(jobs)
        success = sum(1 for j in jobs if j.get("status") == "successful")
        failed = sum(1 for j in jobs if j.get("status") == "failed")
        cancelled = sum(1 for j in jobs if j.get("status") == "canceled")

        # 按 Job Template 统计
        template_stats = {}
        for job in jobs:
            tpl = job.get("job_template_name", "unknown")
            if tpl not in template_stats:
                template_stats[tpl] = {"total": 0, "success": 0, "failed": 0}
            template_stats[tpl]["total"] += 1
            if job.get("status") == "successful":
                template_stats[tpl]["success"] += 1
            elif job.get("status") == "failed":
                template_stats[tpl]["failed"] += 1

        return {
            "total_jobs": total,
            "success_rate": f"{success/total*100:.1f}%" if total else "N/A",
            "success": success,
            "failed": failed,
            "cancelled": cancelled,
            "template_stats": template_stats
        }

# 使用示例
reporter = AWXAuditReporter(
    "https://awx.company.com",
    "your-awx-token"
)

jobs = reporter.get_jobs_since(days=30)
reporter.generate_audit_csv(jobs)
summary = reporter.generate_compliance_summary(jobs)
print(f"近30天执行: {summary['total_jobs']} 次, "
      f"成功率: {summary['success_rate']}")

8.4 AWX 高可用部署架构

九、总结

思考题

1. AWX的Workflow和Ansible的import_playbook有什么区别？什么场景下应该优先使用Workflow？
2. 如果需要在AWX中实现"灰度发布"（先执行10%设备、等待确认后再执行剩余90%），如何利用Workflow或Survey实现？
3. 对比Ansible CLI与AWX平台：在权限管理、执行审计、任务调度三个维度上，AWX分别解决了什么CLI无法解决的问题？

下篇预告

第310篇：YANG 数据模型基础。从CLI走向模型驱动编程，学习如何用结构化数据模型描述网络配置与状态，为NETCONF/RESTCONF奠定理论基础。