Skip to content

Commit f953262

Browse files
Vadims06Vadims06
committed
IS-IS Watcher, addressed MR comments
1 parent 46c21f1 commit f953262

12 files changed

Lines changed: 289 additions & 1414 deletions

File tree

‎docs/telemetry/isis-watcher.md‎

Lines changed: 138 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,100 @@
11
# IS-IS watcher. Tracking IS-IS topology changes in Real-Time
22

3-
![IS-IS watcher containerlab](container_lab.drawio.png)
4-
This lab consists of 6 FRR routers and a single IS-IS Watcher. Each router is pre-configured for being in IS-IS domain with different network type. Topology changes are printed in a text file only (which is enough for testing), for getting logs exported to ELK or Topolograph (to see network changes on a map) start `docker-compose` files and follow instructions on main README.
3+
![IS-IS watcher containerlab](container_lab.drawio.png)
4+
This lab consists of 6 FRR routers and a single IS-IS Watcher. Each router is pre-configured to be part of an IS-IS domain with different network types. Any topology changes detected by the IS-IS Watcher are logged in the file `watcher/watcher.log`. The logging capabilities can be enhanced with tools like ELK or Topolograph, enabling features such as searching changes by time and exporting data to message brokers, Zabbix, and more. For further details, refer to the Links page.
55

66
### IS-IS Topology Watcher
7-
IS-IS Watcher is a monitoring tool of IS-IS topology changes for network engineers. It works via passively listening to IS-IS control plane messages through a specially established IS-IS adjacency between IS-IS Watcher and one of the network device. The tool logs IS-IS events into a static file, which can be exported by Logstash to **Elastic Stack (ELK)**, **Zabbix**, **WebHooks** and **Topolograph** monitoring dashboard for keeping the history of events, alerting, instant notification.
7+
IS-IS Watcher is a monitoring tool of IS-IS topology changes for network engineers. It works via passively listening to IS-IS control plane messages through a specially established IS-IS adjacency between IS-IS Watcher and one of the network device. *It assists in analyzing incidents by determining the precise time and location of events, as well as their distribution across the network in a retrospective manner.*
88

99
#### Detected network events:
1010
* IS-IS neighbor adjacency Up/Down
1111
* IS-IS link cost changes
1212
* IS-IS networks appearance/disappearance from the topology
13+
* IS-IS TE attributes:
14+
* Administrative Group (color, resource class)
15+
* Maximum Link Bandwidth
16+
* Maximum Reservable Link Bandwidth
17+
* Unreserved Bandwidth
18+
* Traffic Engineering Default Metric
1319

1420
### Supported IS-IS TLV
15-
| TLV name | TLV |
16-
|----------------------------------|-----|
17-
| IS Reachability | 2 |
18-
| Extended IS Reachability (new) | 22 |
19-
| IPv4 Internal Reachability (old) | 128 |
20-
| IPv4 External Reachability (old) | 130 |
21-
| Extended IPv4 Reachability (new) | 135 |
22-
| IPv6 Reachability | 236 |
21+
| TLV name | TLV | subTLV |
22+
|----------------------------------|-----|------------------------|
23+
| IS Reachability | 2 | |
24+
| Extended IS Reachability (new) | 22 | 3,6,8,9,10,11,12,13,18 |
25+
| IPv4 Internal Reachability (old) | 128 | |
26+
| IPv4 External Reachability (old) | 130 | |
27+
| Extended IPv4 Reachability (new) | 135 | |
28+
| IPv6 Reachability | 236 | |
2329

2430
## Quickstart
2531

2632
1. [Install](https://containerlab.srlinux.dev/install/) containerlab.
27-
2. Create a `br-dr` linux bridge (to emulate broadcast network):
33+
2. Run the script to prepare environment:
2834

2935
```
30-
sudo brctl addbr br-dr
31-
sudo ip link set up dev br-dr
36+
./prepare.sh
3237
```
3338
3439
3. Start the lab
3540
```
3641
sudo clab deploy --topo frr01.clab.yml
3742
```
3843
39-
4. Start watching logs
44+
4. Check that Watcher is ready (usually it requires 10-15sec to be ready). Proceed to the next step once Watcher is ready.
45+
```
46+
$ sudo docker logs clab-frr01-isis-watcher
47+
```
48+
Expected output:
49+
```
50+
lsdb_output:Area lab:
51+
IS-IS Level-1 link-state database:
52+
LSP ID PduLen SeqNumber Chksum Holdtime ATT/P/OL
53+
0100.1001.0001.00-00 * 222 0x00000002 0x9611 1144 1/0/0
54+
Protocols Supp
55+
ISIS LSDB has been received
56+
Sniffing packets on interface: eth1
57+
```
58+
59+
5. Start watching logs
4060
```
4161
sudo tail -f watcher/watcher.log
4262
```
4363
44-
5. Change IS-IS settings on lab' routers. Connect to a router
64+
6. Change IS-IS settings on lab' routers. Connect to a router in another terminal
65+
```
66+
sudo docker exec -it clab-frr01-router6 vtysh
67+
```
68+
Change metric on the interface
69+
```
70+
router6# conf t
71+
router6(config)# int eth1
72+
router6(config-if)# isis metric 66
73+
```
74+
75+
Add new stub network
76+
```
77+
router6(config-if)# ip address 10.10.36.6/24
78+
```
79+
80+
Remove another one
81+
```
82+
router6(config-if)# no ip address 3ffe::192:168:36:6/127
83+
```
84+
85+
Change unreserve bandwidth TE attribute on a link
86+
```
87+
router6(config-if)# link-params
88+
router6(config-link-params)# unrsv-bw 1 9+06
89+
```
90+
assign `admin group` on a link
91+
```
92+
router6(config-link-params)# admin-grp 0xaa
93+
```
94+
95+
Shutdown adjancency
4596
```
46-
sudo docker exec -it clab-frr01-router2 vtysh
97+
router6(config-if)# shutdown
4798
```
4899
49100
### IS-IS Watcher logs location
@@ -52,10 +103,75 @@ Available under `watcher` folder. To see them:
52103
sudo tail -f watcher/watcher.log
53104
```
54105
55-
Note:
56-
log file should have `systemd-network:systemd-journal` ownership
106+
### Logs sample 1
107+
```
108+
2023-01-01T00:00:00Z,demo-watcher,1,host,0200.1001.0002,down,0200.1001.0003,01Jan2023_00h00m00s_7_hosts,49.0002,12345
109+
```
110+
111+
* `2023-01-01T00:00:00Z` - event timestamp
112+
* `demo-watcher` - name of watcher
113+
* `1` - IS-IS level
114+
* `host` - event name: `host`, `network`, `metric`, `temetric`
115+
* `0200.1001.0002` - event object. Watcher detected an event related to `0200.1001.0002` host
116+
* `down` - event status: `down`, `up`, `changed`
117+
* `0200.1001.0003` - event detected by this node.
118+
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
119+
* `49.0002` - area number
120+
* `12345` - AS number
121+
* `10.1.23.3` - Local IP address of detected node `0200.1001.0003` (available in 2.0.15)
122+
* `10.1.23.2` - Remote IP address relative to the detected node `0200.1001.0003` (available in 2.0.16)
123+
*Summary: `0200.1001.0003` on the interface with `10.1.23.3` IP address detected that `0200.1001.0002` host went down at `2023-01-01T00:00:00Z` in IS-IS level 1 in area 49.0002 in AS 12345*
57124
58-
> **Note**
59-
> [IS-IS Watcher](https://github.com/Vadims06/isiswatcher) - IS-IS topology tracker
60-
> This lab is based on simple FRR for building topology based on frr routers, more information about it is available here: https://www.brianlinkletter.com/2021/05/use-containerlab-to-emulate-open-source-routers/
125+
### Logs sample 2
126+
```
127+
2023-01-01T00:00:00Z,isis-watcher,2,metric,4ffe::192:168:23:2/127,changed,old_cost:10,new_cost:12,0200.1001.0002,stub,0200.1001.0002,01Jan2023_00h00m00s_7_hosts,49.0002,12345,external,1
128+
```
129+
130+
* `2023-01-01T00:00:00Z` - event timestamp
131+
* `isis-watcher` - name of watcher
132+
* `2` - IS-IS level
133+
* `metric` - event name: `host`, `network`, `metric`, `temetric`
134+
* `4ffe::192:168:23:2/127` - event object. Watcher detected an event related to 4ffe::192:168:23:2/127` subnet
135+
* `changed` - event status: `down`, `up`, `changed`
136+
* `10` - old cost
137+
* `12` - new cost
138+
* `0200.1001.0002` - event detected by this node.
139+
* `stub` - subnet type
140+
* `0200.1001.0002` - since it's a stub network it has router id of terminated node.
141+
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
142+
* `49.0002` - area number
143+
* `12345` - AS number
144+
* `external` - subnet type internal|external
145+
* `1` - subnet ext type 1|2. 0 for internal subnets
146+
*Summary: `0200.1001.0002` detected that metric of `4ffe::192:168:23:2/127` stub network changed from `10` to `12` at `2023-01-01T00:00:00Z` in IS-IS level 2*
147+
148+
### Logs sample 3. TE
149+
```
150+
2024-12-29T13:20:50.398Z,
151+
isis-watcher,1,temetric,0200.1001.0002,changed,0_17_19_20_21_22_26_29_30,1000000000,1000000000,1000000008_1000000016_1000000024_1000000032_1000000040_1000000048_1000000056,11223344,0200.1001.0003,2024-07-28T18:03:05Z,49.0001,01Jan2023_00h00m00s_7_hosts,10.1.23.3,10.1.23.2
152+
```
153+
* `2023-01-01T00:00:00Z` - event timestamp
154+
* `demo-watcher` - name of watcher
155+
* `1` - IS-IS level
156+
* `temetric` - event name: `host`, `network`, `metric`, `temetric`
157+
* `0200.1001.0002` - event object. Watcher detected an event related to `0200.1001.0002` host
158+
* `changed` - event status: `down`, `up`, `changed`
159+
* `0_17_19_20_21_22_26_29_30` - 0, 17, 19, 20, 21, 22, 26, 29, 30 admin groups
160+
* `1000000000` - Maximum Link Bandwidth (Sub-TLV 9) bits per sec
161+
* `1000000000` - Maximum Reservable Link Bandwidth (Sub-TLV 10) bits per sec
162+
* `1000000008_1000000016_1000000024_1000000032_1000000040_1000000048_1000000056` - Unreserved Bandwidth (Sub-TLV 11) for priority 0,..7 bits per sec
163+
* `11223344` - Traffic Engineering Default Metric (Sub-TLV 18)
164+
* `0200.1001.0003` - event detected by this node.
165+
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
166+
* `49.0002` - area number
167+
* `12345` - AS number
168+
* `10.1.23.3` - Local IP address of detected node `0200.1001.0003` (available in 2.0.15)
169+
* `10.1.23.2` - Remote IP address relative to the detected node `0200.1001.0003` (available in 2.0.16)
170+
171+
172+
173+
> **Note**
174+
log file should have `systemd-network:systemd-journal` ownership
61175
176+
### Links
177+
* [IS-IS Watcher](https://github.com/Vadims06/isiswatcher) - IS-IS topology tracker

‎labs/isis-watcher/README.md‎

Lines changed: 106 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,31 @@
11
# IS-IS watcher. Tracking IS-IS topology changes in Real-Time
22

3-
![IS-IS watcher containerlab](container_lab.drawio.png)
4-
This lab consists of 6 FRR routers and a single IS-IS Watcher. Each router is pre-configured for being in IS-IS domain with different network type. Topology changes are printed in a text file only (which is enough for testing), for getting logs exported to ELK or Topolograph (to see network changes on a map) start `docker-compose` files and follow instructions on main README.
3+
![IS-IS watcher containerlab](container_lab.drawio.png)
4+
This lab consists of 6 FRR routers and a single IS-IS Watcher. Each router is pre-configured to be part of an IS-IS domain with different network types. Any topology changes detected by the IS-IS Watcher are logged in the file `watcher/watcher.log`. The logging capabilities can be enhanced with tools like ELK or Topolograph, enabling features such as searching changes by time and exporting data to message brokers, Zabbix, and more. For further details, refer to the Links page.
55

66
### IS-IS Topology Watcher
7-
IS-IS Watcher is a monitoring tool of IS-IS topology changes for network engineers. It works via passively listening to IS-IS control plane messages through a specially established IS-IS adjacency between IS-IS Watcher and one of the network device. The tool logs IS-IS events into a static file, which can be exported by Logstash to **Elastic Stack (ELK)**, **Zabbix**, **WebHooks** and **Topolograph** monitoring dashboard for keeping the history of events, alerting, instant notification.
7+
IS-IS Watcher is a monitoring tool of IS-IS topology changes for network engineers. It works via passively listening to IS-IS control plane messages through a specially established IS-IS adjacency between IS-IS Watcher and one of the network device. *It assists in analyzing incidents by determining the precise time and location of events, as well as their distribution across the network in a retrospective manner.*
88

99
#### Detected network events:
1010
* IS-IS neighbor adjacency Up/Down
1111
* IS-IS link cost changes
1212
* IS-IS networks appearance/disappearance from the topology
13+
* IS-IS TE attributes:
14+
* Administrative Group (color, resource class)
15+
* Maximum Link Bandwidth
16+
* Maximum Reservable Link Bandwidth
17+
* Unreserved Bandwidth
18+
* Traffic Engineering Default Metric
1319

1420
### Supported IS-IS TLV
15-
| TLV name | TLV |
16-
|----------------------------------|-----|
17-
| IS Reachability | 2 |
18-
| Extended IS Reachability (new) | 22 |
19-
| IPv4 Internal Reachability (old) | 128 |
20-
| IPv4 External Reachability (old) | 130 |
21-
| Extended IPv4 Reachability (new) | 135 |
22-
| IPv6 Reachability | 236 |
21+
| TLV name | TLV | subTLV |
22+
|----------------------------------|-----|------------------------|
23+
| IS Reachability | 2 | |
24+
| Extended IS Reachability (new) | 22 | 3,6,8,9,10,11,12,13,18 |
25+
| IPv4 Internal Reachability (old) | 128 | |
26+
| IPv4 External Reachability (old) | 130 | |
27+
| Extended IPv4 Reachability (new) | 135 | |
28+
| IPv6 Reachability | 236 | |
2329

2430
## Quickstart
2531

@@ -35,14 +41,60 @@ IS-IS Watcher is a monitoring tool of IS-IS topology changes for network enginee
3541
sudo clab deploy --topo frr01.clab.yml
3642
```
3743
38-
4. Start watching logs
44+
4. Check that Watcher is ready (usually it requires 10-15sec to be ready). Proceed to the next step once Watcher is ready.
45+
```
46+
$ sudo docker logs clab-frr01-isis-watcher
47+
```
48+
Expected output:
49+
```
50+
lsdb_output:Area lab:
51+
IS-IS Level-1 link-state database:
52+
LSP ID PduLen SeqNumber Chksum Holdtime ATT/P/OL
53+
0100.1001.0001.00-00 * 222 0x00000002 0x9611 1144 1/0/0
54+
Protocols Supp
55+
ISIS LSDB has been received
56+
Sniffing packets on interface: eth1
57+
```
58+
59+
5. Start watching logs
3960
```
4061
sudo tail -f watcher/watcher.log
4162
```
4263
43-
5. Change IS-IS settings on lab' routers. Connect to a router
64+
6. Change IS-IS settings on lab' routers. Connect to a router in another terminal
65+
```
66+
sudo docker exec -it clab-frr01-router6 vtysh
67+
```
68+
Change metric on the interface
69+
```
70+
router6# conf t
71+
router6(config)# int eth1
72+
router6(config-if)# isis metric 66
73+
```
74+
75+
Add new stub network
76+
```
77+
router6(config-if)# ip address 10.10.36.6/24
78+
```
79+
80+
Remove another one
81+
```
82+
router6(config-if)# no ip address 3ffe::192:168:36:6/127
83+
```
84+
85+
Change unreserve bandwidth TE attribute on a link
4486
```
45-
sudo docker exec -it clab-frr01-router2 vtysh
87+
router6(config-if)# link-params
88+
router6(config-link-params)# unrsv-bw 1 9+06
89+
```
90+
assign `admin group` on a link
91+
```
92+
router6(config-link-params)# admin-grp 0xaa
93+
```
94+
95+
Shutdown adjancency
96+
```
97+
router6(config-if)# shutdown
4698
```
4799
48100
### IS-IS Watcher logs location
@@ -53,28 +105,32 @@ sudo tail -f watcher/watcher.log
53105
54106
### Logs sample 1
55107
```
56-
2023-01-01T00:00:00Z,demo-watcher,1,host,0200.1001.0002,down,0200.1001.0003,01Jan2023_00h00m00s_7_hosts
108+
2023-01-01T00:00:00Z,demo-watcher,1,host,0200.1001.0002,down,0200.1001.0003,01Jan2023_00h00m00s_7_hosts,49.0002,12345
57109
```
58110
59111
* `2023-01-01T00:00:00Z` - event timestamp
60112
* `demo-watcher` - name of watcher
61113
* `1` - IS-IS level
62-
* `host` - event name: `host`, `network`, `metric`
114+
* `host` - event name: `host`, `network`, `metric`, `temetric`
63115
* `0200.1001.0002` - event object. Watcher detected an event related to `0200.1001.0002` host
64116
* `down` - event status: `down`, `up`, `changed`
65117
* `0200.1001.0003` - event detected by this node.
66118
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
67-
*Summary: `0200.1001.0003` detected that `0200.1001.0002` host went down at `2023-01-01T00:00:00Z` in IS-IS level 1*
119+
* `49.0002` - area number
120+
* `12345` - AS number
121+
* `10.1.23.3` - Local IP address of detected node `0200.1001.0003` (available in 2.0.15)
122+
* `10.1.23.2` - Remote IP address relative to the detected node `0200.1001.0003` (available in 2.0.16)
123+
*Summary: `0200.1001.0003` on the interface with `10.1.23.3` IP address detected that `0200.1001.0002` host went down at `2023-01-01T00:00:00Z` in IS-IS level 1 in area 49.0002 in AS 12345*
68124
69125
### Logs sample 2
70126
```
71-
2023-01-01T00:00:00Z,isis-watcher,2,metric,4ffe::192:168:23:2/127,changed,old_cost:10,new_cost:12,0200.1001.0002,stub,0200.1001.0002,01Jan2023_00h00m00s_7_hosts
127+
2023-01-01T00:00:00Z,isis-watcher,2,metric,4ffe::192:168:23:2/127,changed,old_cost:10,new_cost:12,0200.1001.0002,stub,0200.1001.0002,01Jan2023_00h00m00s_7_hosts,49.0002,12345,external,1
72128
```
73129
74130
* `2023-01-01T00:00:00Z` - event timestamp
75131
* `isis-watcher` - name of watcher
76132
* `2` - IS-IS level
77-
* `metric` - event name: `host`, `network`, `metric`
133+
* `metric` - event name: `host`, `network`, `metric`, `temetric`
78134
* `4ffe::192:168:23:2/127` - event object. Watcher detected an event related to 4ffe::192:168:23:2/127` subnet
79135
* `changed` - event status: `down`, `up`, `changed`
80136
* `10` - old cost
@@ -83,12 +139,39 @@ sudo tail -f watcher/watcher.log
83139
* `stub` - subnet type
84140
* `0200.1001.0002` - since it's a stub network it has router id of terminated node.
85141
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
142+
* `49.0002` - area number
143+
* `12345` - AS number
144+
* `external` - subnet type internal|external
145+
* `1` - subnet ext type 1|2. 0 for internal subnets
86146
*Summary: `0200.1001.0002` detected that metric of `4ffe::192:168:23:2/127` stub network changed from `10` to `12` at `2023-01-01T00:00:00Z` in IS-IS level 2*
87147
148+
### Logs sample 3. TE
149+
```
150+
2024-12-29T13:20:50.398Z,
151+
isis-watcher,1,temetric,0200.1001.0002,changed,0_17_19_20_21_22_26_29_30,1000000000,1000000000,1000000008_1000000016_1000000024_1000000032_1000000040_1000000048_1000000056,11223344,0200.1001.0003,2024-07-28T18:03:05Z,49.0001,01Jan2023_00h00m00s_7_hosts,10.1.23.3,10.1.23.2
152+
```
153+
* `2023-01-01T00:00:00Z` - event timestamp
154+
* `demo-watcher` - name of watcher
155+
* `1` - IS-IS level
156+
* `temetric` - event name: `host`, `network`, `metric`, `temetric`
157+
* `0200.1001.0002` - event object. Watcher detected an event related to `0200.1001.0002` host
158+
* `changed` - event status: `down`, `up`, `changed`
159+
* `0_17_19_20_21_22_26_29_30` - 0, 17, 19, 20, 21, 22, 26, 29, 30 admin groups
160+
* `1000000000` - Maximum Link Bandwidth (Sub-TLV 9) bits per sec
161+
* `1000000000` - Maximum Reservable Link Bandwidth (Sub-TLV 10) bits per sec
162+
* `1000000008_1000000016_1000000024_1000000032_1000000040_1000000048_1000000056` - Unreserved Bandwidth (Sub-TLV 11) for priority 0,..7 bits per sec
163+
* `11223344` - Traffic Engineering Default Metric (Sub-TLV 18)
164+
* `0200.1001.0003` - event detected by this node.
165+
* `01Jan2023_00h00m00s_7_hosts` - name of graph in Topolograph dashboard
166+
* `49.0002` - area number
167+
* `12345` - AS number
168+
* `10.1.23.3` - Local IP address of detected node `0200.1001.0003` (available in 2.0.15)
169+
* `10.1.23.2` - Remote IP address relative to the detected node `0200.1001.0003` (available in 2.0.16)
170+
88171
89-
Note:
90-
log file should have `systemd-network:systemd-journal` ownership
91172
92-
> **Note**
93-
> This lab is based on simple FRR for building topology based on frr routers, more information about it is available here: https://www.brianlinkletter.com/2021/05/use-containerlab-to-emulate-open-source-routers/
173+
> **Note**
174+
log file should have `systemd-network:systemd-journal` ownership
94175
176+
### Links
177+
* [IS-IS Watcher](https://github.com/Vadims06/isiswatcher) - IS-IS topology tracker

0 commit comments

Comments
 (0)