openstack-k8s-operators · hjensas · May 26, 2026
diff --git a/scenarios/networking-lab/devstack-ceos-vlan-netconf/README.md b/scenarios/networking-lab/devstack-ceos-vlan-netconf/README.md
@@ -0,0 +1,30 @@
+# Devstack with Arista cEOS VLAN Trunking (NETCONF OpenConfig)
+
+Single-switch topology with 1 Arista cEOS switch, 1 Devstack node, 2 Ironic nodes, and 1 controller.
+Uses networking-generic-switch with the `netconf_openconfig` driver for NETCONF-based switch management.
+
+## Topology
+
+![Topology Diagram](topology-diagram.svg)
+
+Management: `192.168.32.0/24` | VLANs: 100-105 | MTU: 1500
+
+## Deployment
+
+Deploy the scenario:
+
+```bash
+ansible-playbook -e @scenarios/networking-lab/devstack-ceos-vlan-netconf/bootstrap_vars.yml -e os_cloud=<cloud-name> bootstrap_devstack.yml
+```
+
+## Accessing
+
+Access the switch and devstack nodes via SSH from the controller.
+
+```bash
+# Switch
+ssh admin@switch.stack.lab
+
+# Devstack
+ssh stack@devstack.stack.lab
+```
diff --git a/scenarios/networking-lab/devstack-ceos-vlan-netconf/TROUBLESHOOTING.md b/scenarios/networking-lab/devstack-ceos-vlan-netconf/TROUBLESHOOTING.md
@@ -0,0 +1,189 @@
+# Troubleshooting Network Connectivity
+
+## Access Methods
+
+- **Arista cEOS Switch**: `ssh admin@switch.stack.lab` (password: `admin`)
+- **Switch Container Host**: `ssh zuul@switch-host.stack.lab`
+- **Devstack**: `ssh stack@devstack.stack.lab`
+
+## Verify Network Topology
+
+### Arista cEOS Switch VLAN Configuration
+```bash
+# Check VLAN configuration
+show vlan
+
+# Check interface status
+show interfaces status
+
+# Check interface switchport configuration
+show interfaces switchport
+
+# Check trunk port configuration
+show interfaces Ethernet1 switchport
+
+# Check specific interface details
+show interfaces Ethernet1
+show interfaces Ethernet2
+show interfaces Ethernet3
+```
+
+### Devstack VLAN Interfaces
+```bash
+# Check trunk and bridge interfaces
+ip link show | grep -E "trunk0|br-ex"
+```
+
+## Check Interface Statistics
+
+### On Arista cEOS Switch
+```bash
+# Check all interface counters
+show interfaces counters
+
+# Check specific interface counters
+show interfaces Ethernet1 counters
+show interfaces Ethernet2 counters
+show interfaces Ethernet3 counters
+
+# Check for errors
+show interfaces counters errors
+
+# Check interface status and statistics
+show interfaces Ethernet1
+show interfaces Ethernet2
+show interfaces Ethernet3
+
+# Look for:
+# - RX/TX errors
+# - RX/TX dropped packets
+# - CRC errors
+# - Collisions
+```
+
+### On Devstack
+```bash
+# Check trunk interface
+ip -s link show trunk0
+
+# Check bridge interfaces
+ip -s link show br-ex
+ip -s link show br-ex.103
+
+# Look for dropped or error counters
+```
+
+## Verify VLAN Configuration
+
+### Check VLANs on Arista Switch
+```bash
+# Show all VLANs
+show vlan
+
+# Show VLAN brief summary
+show vlan brief
+
+# Show which interfaces are in which VLANs
+show vlan id 100
+show vlan id 103
+show vlan id 104
+show vlan id 105
+```
+
+### Check Trunk Configuration
+```bash
+# Verify trunk allowed VLANs
+show interfaces Ethernet1 trunk
+
+# Check spanning-tree status (should show portfast enabled)
+show spanning-tree interface Ethernet1
+```
+
+## NETCONF Troubleshooting
+
+### Verify NETCONF Connectivity
+```bash
+# Test NETCONF from devstack to switch (port 830)
+ssh -s -p 830 admin@switch.stack.lab netconf
+
+# Check NETCONF management API on switch
+show management api netconf
+```
+
+### Check networking-generic-switch Logs
+```bash
+# On devstack, check neutron-server logs for NGS activity
+sudo journalctl -u devstack@q-svc -f | grep -i generic
+```
+
+## Capture Network Traffic
+
+### Basic Packet Capture
+
+**On Devstack:**
+```bash
+# Watch traffic on VLAN 103
+sudo tcpdump -i trunk0 -evnn vlan 103
+
+# Filter by port (e.g., Ironic API on port 80)
+sudo tcpdump -i trunk0 -evnn vlan 103 and port 80
+
+# More verbose output with packet contents
+sudo tcpdump -i trunk0 -evvvnXX vlan 103
+```
+
+**On Arista cEOS Switch (using bash from EOS):**
+```bash
+# Enter bash shell from EOS
+bash
+
+# Capture on baremetal-facing port
+sudo tcpdump -i eth3 -evnn  # Ethernet2 (ironic0)
+sudo tcpdump -i eth4 -evnn  # Ethernet3 (ironic1)
+
+# Capture on trunk to devstack
+sudo tcpdump -i eth2 -evnn vlan 103  # Ethernet1 (trunk)
+
+# Exit bash when done
+exit
+```
+
+## Common Issues
+
+### VLANs Not Configured on Switch
+If networking-generic-switch is not configuring VLANs:
+```bash
+# Verify NGS can connect
+ssh admin@switch.stack.lab "show version"
+
+# Check that management API is enabled
+show management api http-commands
+
+# Verify VLANs exist
+show vlan
+
+# Check interface configuration
+show running-config interfaces
+```
+
+### Interface Not Passing Traffic
+```bash
+# Verify interface is up
+show interfaces status | include Ethernet
+
+# Check for errors
+show interfaces counters errors
+
+# Verify spanning-tree is not blocking
+show spanning-tree
+```
+
+### Access Port Not in Correct VLAN
+```bash
+# Check access VLAN assignment
+show interfaces Ethernet2 switchport
+show interfaces Ethernet3 switchport
+
+# Verify the port is in access mode
+show running-config interfaces Ethernet2
+```
diff --git a/scenarios/networking-lab/devstack-ceos-vlan-netconf/automation-vars.yml b/scenarios/networking-lab/devstack-ceos-vlan-netconf/automation-vars.yml
@@ -0,0 +1,65 @@
+---
+stages:
+  - name: Enroll nodes in devstack ironic
+    documentation: >-
+      Registers physical baremetal nodes with the Ironic service in the DevStack
+      deployment using the node definitions from ironic_nodes.yaml. This creates
+      Ironic node records with BMC access credentials, hardware profiles, and port
+      configurations for networking-generic-switch integration.
+    shell: |
+      set -xe -o pipefail
+
+      NODES_FILE=/home/zuul/data/ironic_nodes.yaml
+
+      # Enroll the nodes
+      openstack --os-cloud devstack-admin baremetal create "$NODES_FILE"
+
+      echo "Nodes enrolled successfully"
+      openstack --os-cloud devstack-admin baremetal node list
+
+  - name: Wait for ironic nodes to reach enroll state
+    documentation: >-
+      Monitors node state transition to 'enroll' status, indicating that Ironic
+      has successfully registered the nodes and validated basic BMC connectivity.
+      This is the first state in the baremetal provisioning lifecycle.
+    shell: |
+      set -xe -o pipefail
+
+      counter=0
+      max_retries=60
+      sleep_interval=5
+
+      echo "Waiting for all nodes to reach 'enroll' state..."
+
+      until ! openstack --os-cloud devstack-admin baremetal node list -f value -c "Provisioning State" | grep -v "enroll"; do
+        ((counter++))
+        if (( counter > max_retries )); then
+          echo "ERROR: Timeout waiting for nodes to reach enroll state"
+          openstack --os-cloud devstack-admin baremetal node list
+          exit 1
+        fi
+        echo "Attempt $counter/$max_retries - waiting ${sleep_interval}s..."
+        sleep ${sleep_interval}
+      done
+
+      echo "All nodes successfully reached enroll state"
+      openstack --os-cloud devstack-admin baremetal node list
+
+  - name: Manage nodes
+    documentation: >-
+      Transitions nodes from 'enroll' to 'manageable' state. This validates
+      basic hardware connectivity and prepares nodes for further operations.
+    shell: |
+      set -x -o pipefail
+
+      # Get list of node UUIDs
+      node_uuids=$(openstack --os-cloud devstack-admin baremetal node list -f value -c UUID)
+
+      # Manage each node with --wait (300 second timeout)
+      for uuid in $node_uuids; do
+        echo "Managing node: $uuid"
+        openstack --os-cloud devstack-admin baremetal node manage --wait 300 $uuid
+      done
+
+      echo "All nodes successfully reached manageable state"
+      openstack --os-cloud devstack-admin baremetal node list
diff --git a/scenarios/networking-lab/devstack-ceos-vlan-netconf/bootstrap_vars.yml b/scenarios/networking-lab/devstack-ceos-vlan-netconf/bootstrap_vars.yml
@@ -0,0 +1,46 @@
+---
+# Bootstrap configuration for devstack-ceos-vlan-netconf scenario
+
+# OpenStack cloud configuration
+os_cloud: default
+os_floating_network: public
+os_router_external_network: public
+
+# Scenario configuration
+scenario: devstack-ceos-vlan-netconf
+scenario_dir: scenarios/networking-lab
+stack_template_path: "{{ scenario_dir }}/{{ scenario }}/heat_template.yaml"
+automation_vars_file: "{{ scenario_dir }}/{{ scenario }}/automation-vars.yml"
+
+# DNS and NTP
+ntp_servers: []
+dns_servers:
+  - 8.8.8.8
+  - 8.8.4.4
+
+# Stack naming
+stack_name: "hs-{{ scenario | replace('/', '-') }}-{{ zuul.build[:8] | default('no-zuul') }}"
+
+# Stack parameters
+stack_parameters:
+  dns_servers: "{{ dns_servers }}"
+  ntp_servers: "{{ ntp_servers }}"
+  controller_ssh_pub_key: "{{ controller_ssh_pub_key | default('') }}"
+  dataplane_ssh_pub_key: "{{ dataplane_ssh_pub_key | default('') }}"
+  router_external_network: "{{ os_router_external_network | default('public') }}"
+  floating_ip_network: "{{ os_floating_network | default('public') }}"
+  controller_params:
+    image: hotstack-controller
+    flavor: hotstack.small
+  devstack_params:
+    image: ubuntu-noble-server
+    flavor: hotstack.large
+  switch_params:
+    image: hotstack-ceos
+    flavor: hotstack.medium
+  ironic_params:
+    image: CentOS-Stream-GenericCloud-9
+    flavor: hotstack.medium
+
+# Controller role configuration
+controller_install_openstack_client: true