第355篇：多云互联网络架构设计与实施

关键词：多云互联、云专线、SD-WAN、跨云通信、Direct Connect

一、案例背景

某大型企业采用了多云策略，业务分布在华为云、阿里云和AWS三个云平台。各云之间的数据同步、应用互联和统一管理成为亟待解决的问题。现网通过公网通信存在时延高、安全性差、成本高等问题。

现状： - 华为云：承载核心业务系统（ERP、CRM） - 阿里云：承载前端应用和CDN - AWS：承载海外业务和大数据分析 - 互联方式：公网TCP通信 - 痛点：跨云时延>100ms、带宽不可控、安全性弱

目标： - 跨云时延<20ms（国内云之间） - 专线带宽每方向10Gbps - 统一的网络管理平台 - 自动故障切换

二、需求分析与方案设计

2.1 多云互联方案对比

2.2 目标架构

2.3 互联拓扑设计

三、技术细节与实施过程

3.1 华为云Direct Connect配置

# 华为云VPC配置
# 1. 创建云专线（通过华为云控制台）
# 2. 创建虚拟网关（VGW）
# 3. 创建虚拟接口（VI）

# 本地CE设备配置（华为CE6865）
[Huawei] interface 100GE 1/0/1
[Huawei-100GE1/0/1] ip address 10.255.0.1 255.255.255.252
[Huawei-100GE1/0/1] quit

# BGP配置（与华为云建立eBGP）
[Huawei] bgp 65001
[Huawei-bgp-65001] router-id 10.0.0.1
[Huawei-bgp-65001] peer 10.255.0.2 as-number 64512
[Huawei-bgp-65001] peer 10.255.0.2 connect-interface 100GE1/0/1
[Huawei-bgp-65001] peer 10.255.0.2 ebgp-max-hop 3
[Huawei-bgp-65001] peer 10.255.0.2 bfd enable
[Huawei-bgp-65001] quit

# 发布本地路由到华为云
[Huawei] ip route-static 0.0.0.0 0.0.0.0 10.255.0.2
[Huawei] bgp 65001
[Huawei-bgp-65001] import-route static
[Huawei-bgp-65001] quit

3.2 SD-WAN Hub配置

# 总部SD-WAN Hub配置（华为AR6140H）
[Huawei] sdwan
[Huawei-sdwan] local-system-id 0000000000000001
[Huawei-sdwan] local-system-name HQ-HUB

# 配置到华为云的SD-WAN隧道
[Huawei-sdwan] sdwan peer 0000000000000002
[Huawei-sdwan-peer] peer-ip 10.255.0.2
[Huawei-sdwan-peer] pre-shared-key cipher Huawei@123
[Huawei-sdwan-peer] ikev2-proposal SDWAN-IKEV2
[Huawei-sdwan-peer] ipsec-profile SDWAN-IPSEC
[Huawei-sdwan-peer] sla-path prefer-low-delay
[Huawei-sdwan-peer] quit

# 配置到阿里云的SD-WAN隧道
[Huawei-sdwan] sdwan peer 0000000000000003
[Huawei-sdwan-peer] peer-ip 10.255.1.2
[Huawei-sdwan-peer] pre-shared-key cipher Huawei@123
[Huawei-sdwan-peer] ikev2-proposal SDWAN-IKEV2
[Huawei-sdwan-peer] ipsec-profile SDWAN-IPSEC
[Huawei-sdwan-peer] sla-path prefer-low-delay
[Huawei-sdwan-peer] quit

# SD-WAN策略
[Huawei-sdwan] sdwan rule 10
[Huawei-sdwan-rule] match destination-ip 10.0.0.0 8
[Huawei-sdwan-rule] apply peer 0000000000000002  # 华为云流量
[Huawei-sdwan-rule] quit

[Huawei-sdwan] sdwan rule 20
[Huawei-sdwan-rule] match destination-ip 172.16.0.0 12
[Huawei-sdwan-rule] apply peer 0000000000000003  # 阿里云流量
[Huawei-sdwan-rule] quit

3.3 路由策略设计

# 路由策略：基于目的地址的流量引导
[Huawei] route-policy CLOUD_ROUTING permit node 10
[Huawei-route-policy-CLOUD_ROUTING] if-match ip-address prefix-list HUAWEI_CLOUD
[Huawei-route-policy-CLOUD_ROUTING] apply next-hop 10.255.0.2
[Huawei-route-policy-CLOUD_ROUTING] quit

[Huawei] route-policy CLOUD_ROUTING permit node 20
[Huawei-route-policy-CLOUD_ROUTING] if-match ip-address prefix-list ALIYUN
[Huawei-route-policy-CLOUD_ROUTING] apply next-hop 10.255.1.2
[Huawei-route-policy-CLOUD_ROUTING] quit

[Huawei] route-policy CLOUD_ROUTING permit node 30
[Huawei-route-policy-CLOUD_ROUTING] if-match ip-address prefix-list AWS
[Huawei-route-policy-CLOUD_ROUTING] apply next-hop 10.255.2.2
[Huawei-route-policy-CLOUD_ROUTING] quit

# Prefix-List定义
[Huawei] ip prefix-list HUAWEI_CLOUD permit 10.0.0.0 8
[Huawei] ip prefix-list ALIYUN permit 172.16.0.0 12
[Huawei] ip prefix-list AWS permit 192.168.0.0 16

3.4 跨云DNS解析设计

# 跨云DNS设计：使用智能DNS根据源地址返回最优云地址
[Huawei] dns view HUAWEI_CLOUD
[Huawei-dns-view-HUAWEI_CLOUD] dns loadbalance source-ip
[Huawei-dns-view-HUAWEI_CLOUD] dns record app.example.com A 10.0.1.100  # 华为云地址
[Huawei-dns-view-HUAWEI_CLOUD] quit

[Huawei] dns view ALIYUN
[Huawei-dns-view-ALIYUN] dns loadbalance source-ip
[Huawei-dns-view-ALIYUN] dns record app.example.com A 172.16.1.100  # 阿里云地址
[Huawei-dns-view-ALIYUN] quit

# DNS视图绑定ACL（根据源IP选择视图）
[Huawei] acl number 2000
[Huawei-acl-basic-2000] rule 10 permit source 10.0.0.0 0.255.255.255
[Huawei-acl-basic-2000] rule 20 permit source 172.16.0.0 0.15.255.255
[Huawei-acl-basic-2000] quit

[Huawei] dns view apply acl 2000 view HUAWEI_CLOUD

3.5 跨云监控与告警

# NTP时间同步（跨云监控时间一致性）
[Huawei] ntp-service unicast-server 10.255.0.2  # 华为云NTP
[Huawei] ntp-service unicast-server 10.255.1.2  # 阿里云NTP（备用）

# Telemetry流式遥测（实时采集跨云指标）
[Huawei] telemetry iana
[Huawei-telemetry-iana] subscription CLOUD_MONITOR
[Huawei-telemetry-iana-subscription-CLOUD_MONITOR] sensor-group SG_CLOUD
[Huawei-telemetry-iana-subscription-CLOUD_MONITOR] stream grpc
[Huawei-telemetry-iana-subscription-CLOUD_MONITOR] destination 10.0.0.50  # 监控服务器
[Huawei-telemetry-iana-subscription-CLOUD_MONITOR] sample-interval 5000  # 5秒采集
[Huawei-telemetry-iana-subscription-CLOUD_MONITOR] encoding-encoded-junos
[Huawei-telemetry-iana-subscription-CLOUD_MONITOR] heartbeat-interval 60000
[Huawei-telemetry-iana-subscription-CLOUD_MONITOR] quit
[Huawei-telemetry-iana] quit

# 定义采集的传感器组
[Huawei] telemetry iana
[Huawei-telemetry-iana] sensor-group SG_CLOUD
[Huawei-telemetry-iana-sensor-group-SG_CLOUD] data-filter /interfaces-state/interface/statistics
[Huawei-telemetry-iana-sensor-group-SG_CLOUD] data-filter /bgp-state/bgpe
[Huawei-telemetry-iana-sensor-group-SG_CLOUD] quit
[Huawei-telemetry-iana] quit

四、实战故障排查案例

4.1 案例：跨云BGP路由黑洞排查

故障时间线：

故障现象： 华为云ERP系统（10.0.1.0/24）无法访问阿里云前端应用（172.16.1.0/24），但公网访问正常。跨云专线链路状态正常，BGP邻居UP。

排查过程（完整步骤）：

# 步骤1：检查专线物理链路
<Huawei> display interface 100GE 1/0/1 brief
Interface       PHY   Protocol  InUtilization   OutUtilization
100GE1/0/1      up    up        0.50%           0.30%
# → 物理链路正常 ✓

# 步骤2：检查BGP邻居状态
<Huawei> display bgp peer 10.255.0.2 verbose
  BGP peer is 10.255.0.2,  external AS 64512
  BGP version 4, Router ID 99.99.0.1
  Remote AS 64512, Local AS 65001
  Link Type: external
  State: Established  ← BGP会话UP
  Uptime: 15d 6h
  Message statistics (Bytes):
    Sent: 125,600     Received: 128,300
    Update: 25         Notification: 0
  Address Family IPv4 Unicast:
    State: Active
    Routes received: 15  ← 只收到15条路由（异常，应>50条）
    Routes sent: 48

# 步骤3：检查收到的BGP路由
<Huawei> display bgp routing-table peer 10.255.0.2
Total Number of Routes: 15
Network            NextHop            MED     LocPref    PrefVal    Path/Ogn
172.16.2.0/24      10.255.0.2         0       100        0          ?
172.16.3.0/24      10.250.2           0       100        0          ?
...
# → 缺少172.16.1.0/24网段（前端应用所在网段）

# 步骤4：检查出方向Route-Policy
<Huawei> display current-configuration configuration route-policy ALIYUN_IMPORT
# 发现问题：node 10 仅放行了172.16.2.0/23，未包含172.16.1.0/24
[Huawei] display current-configuration configuration route-policy ALIYUN_IMPORT
 route-policy ALIYUN_IMPORT permit node 10
  if-match ip address prefix-list ALIYUN_ALLOW
 route-policy ALIYUN_IMPORT deny node 20

[Huawei] display ip prefix-list ALIYUN_ALLOW
 IPNet-to-Net Prefixlist: ALIYUN_ALLOW
   seq 5 deny   172.16.1.0 24   ← 172.16.1.0/24被deny了！
   seq 10 permit 172.16.2.0 23
   seq 15 permit 172.16.4.0 22

# 步骤5：根因分析
# → 上次变更中，有人误将172.16.1.0/24添加到deny列表
# → 变更原因：误认为该网段已迁移至华为云

# 步骤6：修复配置
[Huawei] ip prefix-list ALIYUN_ALLOW
[Huawei-ip-prefix-ALIYUN_ALLOW] undo seq 5
[Huawei-ip-prefix-ALIYUN_ALLOW] quit

# 步骤7：验证路由学习
<Huawei> display bgp routing-table peer 10.255.0.2 | include 172.16.1.0
172.16.1.0/24      10.255.0.2         0       100        0          ?
# → 路由已学习到 ✓

# 步骤8：验证端到端连通性
<Huawei> ping -a 10.0.1.1 -c 5 172.16.1.100
PING 172.16.1.100: 56 data bytes
Reply from 172.16.1.100: bytes=56 Sequence=1 ttl=62 time=4 ms
Reply from 172.16.1.100: bytes=56 Sequence=2 ttl=62 time=3 ms
Reply from 172.16.1.100: bytes=56 Sequence=3 ttl=62 time=4 ms
Reply from 172.16.1.100: bytes=56 Sequence=4 ttl=62 time=3 ms
Reply from 172.16.1.100: bytes=56 Sequence=5 ttl=62 time=4 ms
--- 172.16.1.100 ping statistics ---
  5 packets transmitted, 5 received, 0% packet loss

根因分析： 变更管理系统不完善，Prefix-List修改未走变更审批流程。工程师A在清理"过期"网段时，误将仍在使用的172.16.1.0/24网段加入deny列表。

预防措施： 1. 所有跨云路由策略变更必须走变更管理流程（RFC审批） 2. 变更前后使用display bgp routing-table peer x.x.x.x对比路由数量 3. 建立自动化监控：当BGP收到的路由数低于阈值（如<40）时立即告警

4.2 跨云性能测试

4.3 成本分析

4.4 跨云流量矩阵

4.5 多云互联拓扑（含Azure扩展）

当企业引入Azure作为第四个云环境时，需要重新评估互联拓扑。全互联方案链路数呈平方级增长（N个云需要N*(N-1)/2条专线），Hub-Spoke成为更经济的选择：

4.6 云间互联方案详细对比

成本对比（以华为云↔阿里云为例）：

4.7 多云网络健康检查Python脚本

#!/usr/bin/env python3
"""
多云网络健康检查脚本 - 跨云连通性/时延/丢包/路由一致性检查
支持：华为云/阿里云/AWS/Azure多云环境
"""
import subprocess
import json
import time
from datetime import datetime
from dataclasses import dataclass, asdict
from typing import Dict, List, Optional


@dataclass
class CloudEndpoint:
    """云端点定义"""
    name: str
    cloud: str          # huawei/aliyun/aws/azure
    ip: str             # 测试IP（通常为VPC内测试服务器）
    gateway_ip: str     # 云网关互联IP
    expected_latency: int  # 期望时延(μs)


@dataclass
class HealthCheckResult:
    """健康检查结果"""
    source: str
    destination: str
    timestamp: str
    ping_latency_ms: Optional[float]
    ping_packet_loss: Optional[float]
    bgp_session: str    # UP/DOWN/UNKNOWN
    route_reachable: bool
    status: str         # HEALTHY/DEGRADED/CRITICAL
    details: str = ""


class MultiCloudHealthChecker:
    def __init__(self):
        self.endpoints = {}
        self.results = []

    def add_endpoint(self, ep: CloudEndpoint):
        self.endpoints[ep.name] = ep

    def ping_test(self, source_host: str, dest_ip: str,
                   count: int = 5) -> Dict:
        """执行Ping测试"""
        try:
            cmd = f"ping -c {count} -W 2 {dest_ip}"
            result = subprocess.run(cmd, shell=True, capture_output=True,
                                    text=True, timeout=30)
            output = result.stdout

            # 解析时延
            latency_line = [l for l in output.split('\n')
                          if 'rtt' in l or 'round-trip' in l]
            if latency_line:
                parts = latency_line[0].split('=')
                if len(parts) >= 2:
                    latencies = parts[1].split('/')
                    avg_latency = float(latencies[2]) if len(latencies) > 2 else 0
                else:
                    avg_latency = None
            else:
                avg_latency = None

            # 解析丢包率
            loss_line = [l for l in output.split('\n') if 'packet loss' in l]
            packet_loss = 0.0
            if loss_line:
                loss_part = loss_line[0].split('%')[0].split()[-1]
                packet_loss = float(loss_part)

            return {'latency_ms': avg_latency, 'packet_loss': packet_loss}

        except subprocess.TimeoutExpired:
            return {'latency_ms': None, 'packet_loss': 100.0}
        except Exception as e:
            return {'latency_ms': None, 'packet_loss': 100.0, 'error': str(e)}

    def check_bgp_session(self, host: str, peer_ip: str) -> str:
        """检查BGP会话状态（通过SSH到设备执行display命令）"""
        # 实际场景使用paramiko/netmiko SSH到设备
        # 此处简化为SSH命令执行
        try:
            cmd = f"ssh admin@{host} 'display bgp peer {peer_ip} | include State'"
            result = subprocess.run(cmd, shell=True, capture_output=True,
                                    text=True, timeout=15)
            if 'Established' in result.stdout or 'Up' in result.stdout:
                return 'UP'
            return 'DOWN'
        except Exception:
            return 'UNKNOWN'

    def check_route_consistency(self, local_host: str,
                                  remote_host: str,
                                  test_prefix: str = '10.0.0.0/8') -> bool:
        """检查路由一致性：双向路由都存在"""
        # 本地→远端路由检查
        local_ping = self.ping_test(local_host,
                                     self.endpoints[remote_host].ip)
        # 远端→本地路由检查（需要在远端执行ping）
        # 简化：假设远端也能ping回本地
        return local_ping['packet_loss'] < 100

    def run_full_check(self, local_test_host: str) -> List[HealthCheckResult]:
        """执行全量健康检查"""
        for name, ep in self.endpoints.items():
            # Ping测试
            ping_result = self.ping_test(local_test_host, ep.ip)

            # 判断状态
            if ping_result['packet_loss'] >= 100:
                status = 'CRITICAL'
                details = f"完全不可达"
            elif ping_result['packet_loss'] > 0:
                status = 'DEGRADED'
                details = f"丢包率={ping_result['packet_loss']}%"
            elif (ping_result['latency_ms'] and
                  ping_result['latency_ms'] > ep.expected_latency / 1000 * 2):
                status = 'DEGRADED'
                details = (f"时延超标: {ping_result['latency_ms']}ms "
                          f"> {ep.expected_latency/1000}ms")
            else:
                status = 'HEALTHY'
                details = f"时延={ping_result['latency_ms']}ms, 无丢包"

            result = HealthCheckResult(
                source='HQ-TestHost',
                destination=name,
                timestamp=datetime.now().isoformat(),
                ping_latency_ms=ping_result.get('latency_ms'),
                ping_packet_loss=ping_result.get('packet_loss'),
                bgp_session='UP',  # 实际应从设备获取
                route_reachable=ping_result['packet_loss'] < 100,
                status=status,
                details=details
            )
            self.results.append(result)

        return self.results

    def generate_report(self) -> Dict:
        """生成健康检查报告"""
        healthy = sum(1 for r in self.results if r.status == 'HEALTHY')
        degraded = sum(1 for r in self.results if r.status == 'DEGRADED')
        critical = sum(1 for r in self.results if r.status == 'CRITICAL')

        report = {
            'check_time': datetime.now().isoformat(),
            'summary': {
                'total_endpoints': len(self.results),
                'healthy': healthy,
                'degraded': degraded,
                'critical': critical,
                'overall_status': ('HEALTHY' if critical == 0 and degraded == 0
                                  else 'DEGRADED' if critical == 0
                                  else 'CRITICAL')
            },
            'details': [asdict(r) for r in self.results]
        }
        return report


# 使用示例
if __name__ == '__main__':
    checker = MultiCloudHealthChecker()

    # 定义各云端点
    checker.add_endpoint(CloudEndpoint('HuaweiCloud', 'huawei',
                                        '10.0.1.100', '10.255.0.2', 5000))
    checker.add_endpoint(CloudEndpoint('AliYun', 'aliyun',
                                        '172.16.1.100', '10.255.1.2', 10000))
    checker.add_endpoint(CloudEndpoint('AWS', 'aws',
                                        '192.168.1.100', '10.255.2.2', 150000))
    checker.add_endpoint(CloudEndpoint('Azure', 'azure',
                                        '10.10.1.100', '10.255.3.2', 40000))

    # 执行检查
    results = checker.run_full_check('10.0.0.50')  # 本地测试主机IP
    report = checker.generate_report()

    print(f"多云健康检查报告:")
    print(f"  总体状态: {report['summary']['overall_status']}")
    print(f"  健康: {report['summary']['healthy']}, "
          f"降级: {report['summary']['degraded']}, "
          f"严重: {report['summary']['critical']}")
    for detail in report['details']:
        icon = '✓' if detail['status'] == 'HEALTHY' else '⚠' if detail['status'] == 'DEGRADED' else '✗'
        print(f"  {icon} {detail['destination']}: {detail['details']}")

4.8 混合云SD-WAN架构设计详解

SD-WAN在混合云环境中扮演"智能连接器"角色，将本地DC、多云环境、分支机构统一纳入一张可编程的广域网。

SD-WAN分层架构：

SD-WAN隧道架构：

智能选路策略（Service-Level Routing）：

# SD-WAN SLA策略配置示例
[Huawei-sdwan] sla-path group 10
[Huawei-sdwan-sla-group-10] sla-path prefer-low-delay threshold 20
[Huawei-sdwan-sla-group-10] sla-path prefer-high-bandwidth threshold 100
[Huawei-sdwan-sla-group-10] sla-path loss-threshold 1
[Huawei-sdwan-sla-group-10] quit

# 应用级路由策略
[Huawei-sdwan] application-rule 10
[Huawei-sdwan-app-rule-10] match application-name "SAP-R3"
[Huawei-sdwan-app-rule-10] action sla-path-group 10  # SAP走低时延路径
[Huawei-sdwan-app-rule-10] action tunnel-encapsulation ipsec
[Huawei-sdwan-app-rule-10] quit

[Huawei-sdwan] application-rule 20
[Huawei-sdwan-app-rule-20] match application-name "Backup"
[Huawei-sdwan-app-rule-20] action sla-path-group 20  # 备份走高带宽路径
[Huawei-sdwan-app-rule-20] quit

# 故障切换策略
[Huawei-sdwan] failover-policy
[Huawei-sdwan-failover] detection-method icmp
[Huawei-sdwan-failover] detection-interval 3
[Huawei-sdwan-failover] failover-threshold 3  # 连续3次检测失败即切换
[Huawei-sdwan-failover] switch-back-delay 300  # 恢复后延迟5分钟再切换回

4.9 多云安全管理方案

跨云安全架构要点： 1. 纵深防御：每朵云内部部署安全组/ACL，云间部署IPsec隧道，总部部署下一代防火墙，形成三层防护 2. 统一身份：使用LDAP/AD作为统一认证源，各云平台对接同一目录服务 3. 集中审计：通过Syslog/NetFlow将各云安全日志汇聚到统一SIAG平台 4. 最小权限：跨云访问遵循最小权限原则，使用Prefix-List精确控制路由发布范围

五、经验总结

思考

1. 多云互联中，如果使用不同的云厂商（如华为云+AWS+Azure），专线互联的兼容性如何保证？
2. 当企业有5个以上的云环境时，全互联专线方案的成本会急剧上升，有哪些替代方案？
3. 多云网络中，DNS解析如何做到跨云的最优路由选择？

下篇预告：第356篇：算网融合场景下的网络规划案例