Presentation is loading. Please wait.

Presentation is loading. Please wait.

AT&T M320 NGRE 64bit MIS RR TEST PLAN – 13.2 RBU AT&T CST Team July 2013.

Similar presentations


Presentation on theme: "AT&T M320 NGRE 64bit MIS RR TEST PLAN – 13.2 RBU AT&T CST Team July 2013."— Presentation transcript:

1 AT&T M320 NGRE 64bit MIS RR TEST PLAN – 13.2 RBU AT&T CST Team July 2013

2 2 Copyright © 2009 Juniper Networks, Inc. www.juniper.net REVISION HISTORY Version Number OwnerDateRelease Targeted Version Notes (Changes Made) V1Sharon7/7/201313.2R2Initial Draft Version for 64bit MIS RR V2Sharon7/12/201313.2R2Add Revision History Slide V3Sharon7/18/201313.2R2Update test plan per review feedback from ATT -Hardware updated,both DUTs use 64bit RE - scaling number update - test cases update V4Sharon7/29/201313.2R2Update last two scaling from 15 paths to 20 paths (slide 13) V5Sharon8/14/201313.2R2Update slide 31 to add a few more test cases V6Sharon9/6/201313.2R2Update test case for SNMP polling per doc received from AT&T (add slide 32)

3 3 Copyright © 2009 Juniper Networks, Inc. www.juniper.net OBJECTIVE  Measure CPU and memory usage of M320 RR with NGRE 64bit RPD at different scaling setup  64bit Junos, rpd accessible memory increased from 2G to 3G  64bit rpd, rpd can access the full 16G of available memory  Stability Testing for NGRE M320 RR at scaling setup  BGP Convergence Performance Measurement  Find the scaling capacity of NGRE M320 RR with 64bit rpd

4 4 Copyright © 2009 Juniper Networks, Inc. www.juniper.net OVERVIEW  Test Topology  Network Setup  Hardware Details  Network Configuration  Scaling Number  Test Methodology  Test Cases  Caveat

5 5 Copyright © 2009 Juniper Networks, Inc. www.juniper.net 15 Virtual Junos as RRs M320 MIS RR Test Topology M320 RE-A-1800x2 M320 RE-A-1800x2 M320 RE-A -1800x2 M320 RE-A -1800x2 RR1 RR15 P1 64bit M320 MIS-RR Test Topology P3 P4 ge-0/0/0 ge-3/0/0 ge-0/0/0 ge-2/0/0 …... M10i MIS-PE M10i MIS-PE M10i MIS-PE M10i MIS-PE

6 6 Copyright © 2009 Juniper Networks, Inc. www.juniper.net 15 Virtual Junos as RRs …… 15 Virtual Junos as RRs …… M320 MIS-RR BGP Sessions M320 RE-A-1800x2 M320 RE-A-1800x2 M320 RE-A -1800x2 M320 RE-A -1800x2 RR1 RR15 Internet Feed Routes Simulated RRs eBGP Peers Simulated PEs iBGP Peers iBGP eBGP MPLS/LDP Core iBGP eBGP M320 MIS-RR BGP Sessions iBGP eBGP iBGP Additional Routes iBGP M10i MIS-PE M10i MIS-PE M10i MIS-PE M10i MIS-PE P Router iBGP

7 7 Copyright © 2009 Juniper Networks, Inc. www.juniper.net M320 MIS-RR Network Setup DUT M320 with RE-A-1800x2 (64bit) DUT M320 with RE-A-1800x2 (64bit) Network Setup  Internet feed with IPv4 routes to P router  IXIA insert addition IPv4/v6 routes to P router  P router establish eBGP peers with RR1 to RR15  RR1 to RR15 iBGP sessions to DUTs reflecting IPv4/IPv6 routes (15 paths per each route)  IXIA ports simulate PEs to establish iBGP sessions (350)  IXIA ports simulate RRs to establish eBGP sessions (30)  Two M10i as MIS PEs  Each M320(RR) is in unique cluster, iBGP Peers between two DUT RRs Network Setup  Internet feed with IPv4 routes to P router  IXIA insert addition IPv4/v6 routes to P router  P router establish eBGP peers with RR1 to RR15  RR1 to RR15 iBGP sessions to DUTs reflecting IPv4/IPv6 routes (15 paths per each route)  IXIA ports simulate PEs to establish iBGP sessions (350)  IXIA ports simulate RRs to establish eBGP sessions (30)  Two M10i as MIS PEs  Each M320(RR) is in unique cluster, iBGP Peers between two DUT RRs

8 8 Copyright © 2009 Juniper Networks, Inc. www.juniper.net M320 MIS-RR Network Setup -- Hardware Core Network MX240 and MX960 – Simulate core network Core Network MX240 and MX960 – Simulate core network RR Service Network  M320 (64bit)  RE: RE-A-1800x2-16G (Dual RE)  FPCs: M320 E2-FPC Type 1 (2)  PICs: PB-4GE-TYPE1-SFP-IQ2 (2)  Software: 13.2  M320 (64bit)  RE: RE-A-1800x2-16G (Dual RE)  FPCs: M320 E3-FPC Type 1 (2)  PICs: PB-4GE-TYPE1-SFP-IQ2 PB-4GE-TYPE1-SFP-IQ2E  Software:13.2  Virtual Junos and IXIA RT for RRs  VJ Software: 12.1 RR Service Network  M320 (64bit)  RE: RE-A-1800x2-16G (Dual RE)  FPCs: M320 E2-FPC Type 1 (2)  PICs: PB-4GE-TYPE1-SFP-IQ2 (2)  Software: 13.2  M320 (64bit)  RE: RE-A-1800x2-16G (Dual RE)  FPCs: M320 E3-FPC Type 1 (2)  PICs: PB-4GE-TYPE1-SFP-IQ2 PB-4GE-TYPE1-SFP-IQ2E  Software:13.2  Virtual Junos and IXIA RT for RRs  VJ Software: 12.1 MIS Network IXIA RT: Simulate PE routers to advertise IPv4/IPv6 routes to RR for the scaling Two M10i as Real MIS PEs MIS Network IXIA RT: Simulate PE routers to advertise IPv4/IPv6 routes to RR for the scaling Two M10i as Real MIS PEs

9 9 Copyright © 2009 Juniper Networks, Inc. www.juniper.net OSPF non-zero area 10.10.34.9 Uplink Metric 15000 SPF timer 50ms Prefix-export-limit 10 Rib-group OSPF-RIBGP OSPF non-zero area 10.10.34.9 Uplink Metric 15000 SPF timer 50ms Prefix-export-limit 10 Rib-group OSPF-RIBGP Customer Model Network Design -- Configuration BGP ASN 7018 Local address (lo0.0) keep none Path-selection external-router-id Advertise-inactive mtu-discovery Precision-timers Log-updown Tcp-mss 4096 BGP ASN 7018 Local address (lo0.0) keep none Path-selection external-router-id Advertise-inactive mtu-discovery Precision-timers Log-updown Tcp-mss 4096 MPLS: IPv6 tunneling LDP: Not Enabled PIM: Not Enabled BFD: Not Enabled MPLS: IPv6 tunneling LDP: Not Enabled PIM: Not Enabled BFD: Not Enabled Uplink to Core No QoS MTU: 9192 Filter: None on Interface Firewall Filter Applied to lo0.0 (PROTECTHOST) Uplink to Core No QoS MTU: 9192 Filter: None on Interface Firewall Filter Applied to lo0.0 (PROTECTHOST) Chassis GRES enabled Redundancy NSR not enabled Router ID Lo0.0 IPv4 address Chassis GRES enabled Redundancy NSR not enabled Router ID Lo0.0 IPv4 address Management Functions Services – SSH, telnet, ftp, Enable path MTU discovery Enable SNMP Management Functions Services – SSH, telnet, ftp, Enable path MTU discovery Enable SNMP

10 10 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Customer Model Network Design -- BGP Configuration BGP Global Properties (DUT)  Local-AS 7018  keep none  Path-selection external-router-id  Advertise-inactive  MTU-discovery  Precision-timers  Log-updown  Tcp-mss 4096 BGP Global Properties (DUT)  Local-AS 7018  keep none  Path-selection external-router-id  Advertise-inactive  MTU-discovery  Precision-timers  Log-updown  Tcp-mss 4096 IBGP Peer Groups (DUT)  IBGP sessions to PE Clients  Divided into 22 groups  IBGP sessions to RR Peers  IBGP sessions to BGPMon  Family inet and inet6 for each group  Export policy to export aggregate routes IBGP Peer Groups (DUT)  IBGP sessions to PE Clients  Divided into 22 groups  IBGP sessions to RR Peers  IBGP sessions to BGPMon  Family inet and inet6 for each group  Export policy to export aggregate routes EBGP Peer Groups (DUT) DUT EBGP group to RRs in other regions Family inet and inet6 Next-hop unchanged EBGP Peer Groups (DUT) DUT EBGP group to RRs in other regions Family inet and inet6 Next-hop unchanged EBGP Session between P router and RR (VJ) Family inet and inet6 Next-hop changed for the routes to RRs EBGP Session between P router and RR (VJ) Family inet and inet6 Next-hop changed for the routes to RRs

11 11 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Customer Model Network Design -- 64bit RPDBGP Configuration Config knob to enable 64bit RPD In order to use 64bit RPD, need to turn on 64bit rpd knob by configuring the following: system { processes { routing { auto-64-bit } } Or system { processes { routing { force-64-bit } } Note: 1. Configure either knob to enable 64bit rpd, auto-64-bit only take effect when RE has 16G memory 2. When commit the change, 64bit RPD takes effect, bgp serssions will bounce. Config knob to enable 64bit RPD In order to use 64bit RPD, need to turn on 64bit rpd knob by configuring the following: system { processes { routing { auto-64-bit } } Or system { processes { routing { force-64-bit } } Note: 1. Configure either knob to enable 64bit rpd, auto-64-bit only take effect when RE has 16G memory 2. When commit the change, 64bit RPD takes effect, bgp serssions will bounce.

12 12 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Scaling Number – Baseline Scaling Baseline Scaling Number (1A) OSPF 50000 routes BGP Number of Paths: 10 Number of IPv4 routes: 702,000 Number of IPv6 routes: 22,000 Total IPv4 Paths: 7,020,000 Total IPv6 Paths: 220,000 Total BGP groups: 25 Total BGP peers: 400 (355 PEs, 15 RRs(IBGP), 30 RRs (EBGP)) IPv4 Static Routes: 1000 IPv6 Static Routes: 1000 Multicast Routes: 100 Multicast Paths: 1000 The setup will first be configured with this baseline scaling number. RPD memory usage and RE kernel memory usage will be measured. Baseline Scaling Number (1A) OSPF 50000 routes BGP Number of Paths: 10 Number of IPv4 routes: 702,000 Number of IPv6 routes: 22,000 Total IPv4 Paths: 7,020,000 Total IPv6 Paths: 220,000 Total BGP groups: 25 Total BGP peers: 400 (355 PEs, 15 RRs(IBGP), 30 RRs (EBGP)) IPv4 Static Routes: 1000 IPv6 Static Routes: 1000 Multicast Routes: 100 Multicast Paths: 1000 The setup will first be configured with this baseline scaling number. RPD memory usage and RE kernel memory usage will be measured.

13 13 Copyright © 2009 Juniper Networks, Inc. www.juniper.net SCALING NUMBER TABLE (2A) WITH 64BIT RPD No. of Paths Add. ipv4 routes Add. ipv6 routes Total ipv4 BGP routes Total ipv6 BGP routes Total ipv4 paths Total ipv6 paths BGP peer groups 64bit RPD Converg. time (average) 64bit RE Memory Utilization 64bit RPD Memory Usage 64bit RPD Memory Utilization (min.) % MB% 10none 702,00022,000 7,020,000 220,00025 10 10,000none802,00022,0008,020,000220,00025 10 none10,000702,00032,0007,020,000320,00025 10 36,000 10,000738,00032,0007,380,000320,00025 10 72,000 20,000774,00042,0007,740,000420,00025 10 144,000 40,000846,00062,0008,460,000620,00025 10 216,000 80,000918,000102,0009,180,0001,020,00025 10 275,000 160,000977,000182,0009,770,0001,820,00025 10 350,000 170,0001,052,000192,00010,520,0001,920,00025 20 none 702,00022,00014,040,000440,00025 20 216,000 80,000918,000102,00018,360,0002,040,00025

14 14 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Scaling Capacity – Melt Down Scaling Test Start point scaling for melt-down scaling testing - 3A OSPF 50000 routes BGP Number of Paths: 15 Number of IPv4 routes: 702,000 Number of IPv6 routes: 22,000 Total IPv4 Paths: 7,020,000 Total IPv6 Paths: 220,000 Total BGP groups: 25 Total BGP peers: 400 (355 PEs, 15 RRs(IBGP), 30 RRs (EBGP)) IPv4 Static Routes: 1000 IPv6 Static Routes: 1000 Multicast Routes: 100 Multicast Paths: 1000 Start point scaling for melt-down scaling testing - 3A OSPF 50000 routes BGP Number of Paths: 15 Number of IPv4 routes: 702,000 Number of IPv6 routes: 22,000 Total IPv4 Paths: 7,020,000 Total IPv6 Paths: 220,000 Total BGP groups: 25 Total BGP peers: 400 (355 PEs, 15 RRs(IBGP), 30 RRs (EBGP)) IPv4 Static Routes: 1000 IPv6 Static Routes: 1000 Multicast Routes: 100 Multicast Paths: 1000 Melt-down Scaling Testing  Setup DUT with start point scaling number(3A), increase the number of IPv4 routes, find out the maximum number of IPv4 routes which will break the DUT RR  Setup DUT with start point scaling number(3A), increase the total number of paths, find out the maximum number of IPv4 paths which will break the DUT RR Melt-down Scaling Testing  Setup DUT with start point scaling number(3A), increase the number of IPv4 routes, find out the maximum number of IPv4 routes which will break the DUT RR  Setup DUT with start point scaling number(3A), increase the total number of paths, find out the maximum number of IPv4 paths which will break the DUT RR

15 15 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Scaling Number Applied to DUTs  Apply baseline scaling to both DUT with NGRE  Increase the scaling numbers (shown on the table) to measure the resource usage and bgp performance  For operation trigger test and stability testing, use the scaling number 8 on the table(2A) for the testing.  For melt-down scaling testing, start from the baseline scaling (3A) and then increase the number of the routes or the number of paths to find the melt-down point of scaling number  Apply baseline scaling to both DUT with NGRE  Increase the scaling numbers (shown on the table) to measure the resource usage and bgp performance  For operation trigger test and stability testing, use the scaling number 8 on the table(2A) for the testing.  For melt-down scaling testing, start from the baseline scaling (3A) and then increase the number of the routes or the number of paths to find the melt-down point of scaling number

16 16 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Methodology Use customer model network and Focus on RR-DUT (M320s) Background noise apply to DUT to simulate real network environment Resource Measurements  RE and PFE resource monitored and measured during the test execution cycle Test Execution  Only one event/trigger/failure on the setup ( no simultaneous failures/triggers/events)  One iteration of event/trigger/failure executed during all test case execution  Selected event/triggers/failures will be executed ‘n’ iterations. Value of ‘n’ depends on availability of resources and schedule. For 13.2, n is set to 5  DUT resource measurements (CPU usage, memory consumption etc) will be taken  BGP convergence performance will be measured  Melt-down scaling will be tested by increasing the routes or paths

17 17 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Summary  Steady State Checklist and Resource Measurements  Soak Test (12 hours) and Halt Test (24 to 36 hours)  Background Noise to Simulate Real Network Environment  Operation Triggers Test ( Stability and Robustness Test) – Hardware Operations – Protocol Operations – Process restart – Network Failure operations  BGP Convergence Performance  Memory Leak Test (Valgrind on RPD )  Security Test: MU Dynamics  Scaling limit of DUT

18 18 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Steady State Check List  Check all the configured interfaces, protocols up on each router  All IFLs are up  All OSPF neighbors are at full state on all routers  Expected iBGP peers are established (DUT)  Expect eBGP peers are established (DUT)  Check all routing tables (inet.0, inet.6 routing tables)  Check DUT scaling  Check DUT has the required scaling number  Monitor KRT queu status, syslog message and core dump  Check any anomalies on syslog (syslog message, chassisd log, ksyncd log) and check if any core dumped.  Check KRT queue status  Record RE and PFE resource  RPD process memory usage  Kernel memory usage  RE memory usage  RE CPU usage  Jtree memory usage

19 19 Copyright © 2009 Juniper Networks, Inc. www.juniper.net BACKGROUND NOISE  From RT port, routes flapping to DUT RR periodically (2% routes from RR every 10 minutes, internet peer flapping every 60 minutes)  From RT port, portion of bgp sessions to DUT RR flapping periodically (10 iBGP Peers randomly flapped at interval of 15 seconds every 10 minutes)  From RT port, send traffic to DUT RR lo0 to stress control plane  SNMP polling  CLI queries  Configure provision for bgp peer add/delete

20 20 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases Test Case 1 Upgrade and Steady State Check Test Case 2 Hardware Operation Triggers Test Case 3 Protocol Operation Triggers Test Case 4 Process Operation Triggers Test Case 5 Network Failure Operation Test Case 6 BGP Convergence Performance Test Case 7 Customer Network Operation Test Case 8 HALT Test Test Case 9 Memory Leak Test Test Case 10 Security Test Test Case 11 Operation Trigger Iteration Test Test Case 12 Melt-down Scaling Number

21 21 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases1 – Upgrade and Steady State Check Test Case 1.1 Upgrade from 11.4S3.2 to 13.2R2  Setup Baseline Scaling for DUTs with 11.4S3.2 – Load 11.4S3.2 image on both DUTs – Apply baseline scaling 1A to both DUTs – Verify all BGP sessions are up and convergent as expected.  Upgrade DUTs to 13.2R2 image and enable 64bit rpd – Upgrade both REs on each DUT to 13.2R2 image – Configure 64bit rpd knob and commit – Make sure DUT 64bit rpd running

22 22 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases1 – Upgrade and Steady State Check Test Case 1.2 Steady State Check and Soak Test on 13.2  Steady State Test Measurements – All BGP peers up and convergent – RE and PFE resources will be measured on DUT RR when the network is at steady state – Duration for monitoring the resources: 60 Minutes  Soak Test Measurements – Soak test will be conducted for a fixed period of 12 hours – Background routes flapping and BGP peer flapping – Resources on RE, PFE, will be monitored and measured on DUT RR during the soak test – Steady state will be checked before and after soak test

23 23 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 1 – Upgrade and Steady State Check Test Case 1.3 Apply test scalings and measure resource on 64bit M320  Setup test scaling – Increase the scaling number to each number from the scaling number table(2A) – Apply the scaling to both DUTs  Steady State Test Measurements – All BGP peers are up with new scaling – RE and PFE resources will be measured on DUT RR at steady state after upgrade is complete – Duration for monitoring the resources: 60 Minutes  Measure BGP Convergence and RE Resources ‒ Measure BGP convergence time for each scaling ‒ Measure rpd memory usage and RE resource usage for each scaling ‒ Measure BGP convergence time for 32bit rpd when scaling number is supported for 32bit rpd for comparision

24 24 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 2 – Hardware Operation Triggers Run One Trigger at a time RE/PFE resource measurement will be taken DUT recovers and no cores Steady state check passes  Test Case 2.1 FPC offline/online  Test Case 2.2 PIC Offline/online  Test Case 2.3 FPC Restart  Test Case 2.4 RE Switchover  Test Case 2.5 SIB Failure/Redundancy  Test Case 2.6 Deactivate/activate interface

25 25 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 3 – Protocol Operation Triggers Run One Trigger at a time RE/PFE resource measurement will be taken DUT recovers and no cores Steady state check passes  Test Case 3.1 Deactivate/Activate OSPF  Test Case 3.2 Deactivate/Activate BGP  Test Case 3.3 Disable/Enable OSPF  Test Case 3.4 Disable/Enable BGP  Test Case 3.5 Clear OSPF neighbor  Test Case 3.6 Clear BGP neighbor – Test Case 3.6.1 Clear all BGP neighbor – Test Case 3.6.2 Clear BGP neighbor to RR (Virtual Junos RR) – Test Case 3.6.3 Clear BGP neighbor to PE – Test Case 3.6.4 Clear BGP neighbor to RR in other cluster

26 26 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 4 – Process Operation Triggers Run One Trigger at a time RE/PFE resource measurement will be taken DUT will recover after trigger and no cores Steady State Check passes  Test Case 4.1 rpd process restart  Test Case 4.2 chassisd process restart  Test Case 4.3 dcd process restart

27 27 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 5 – Network Failure Operation  Network Failure Testing Network failure triggers will be tested without doing any CLI configuration/operational command on routers. One trigger at a time on the network  Network Failure Test Cases Test Case 5.1 Node Failure a. One M320 RR failed, make sure no impact on DUT (64bit RR) b. One P router failed, make sure DUT (64bit RR) will be recovered c. One VRR restarted, make sure no impact on DUTs Test case 5.2 Link Failure One of uplink on DUT (64bit RR) failed, make sure DUT will be able to be convergent Test Case 5.3 Control Plane/RE Failure One RE on DUT (64bit RR) failed, make sure RE switchover and all BGP sessions established and convergent

28 28 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 5 – Network Failure Operation  Network Failure Test Cases (Cont’d) Test Case 5.4 Data/Forwarding Plane Failure Test case 5.4.1 FPC Failed One FPC failed, make sure DUT will update the routes and convergent Test case 5.4.2 PIC Failed One PIC failed, make sure DUT will update the routes and convergent Test case 5.5 Routes flapping Routes flapping from Internet feed and RT port happened on network, make sure DUT routes updated accordingly Test case 5.6 Upgrade RR peer Upgrade the other RR peered with DUT, check the impact on DUT RR

29 29 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 6 – BGP Convergence Performance Measure BGP Convergence time with different triggers Test Case 6.1 BGP Convergence Time after Restart RPD - Restart routing - Measure BGP convergence time - Monitor RE/PFE resource usage Test Case 6.2 BGP Convergence Time after RE Switchover - RE Switchover - Measure BGP convergence time - Monitor RE/PFE resource usage

30 30 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 6 – BGP Convergence Performance Test Case 6.3 BGP Convergence Time after Restart FPC - Restart two FPCs - Measure BGP convergence time - Monitor RE/PFE resource usage Test Case 6.4 BGP Convergence Time after clear BGP neighbor - Clear BGP neighbor - Measure BGP convergence time - Monitor RE/PFE resource usage Test Case 6.5 BGP Convergence Time after clear BGP session - Clear BGP session to one of RR - Measure BGP convergence time - Monitor RE/PFE resource usage

31 31 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 7 – Customer Network Operation  Test case 7.1 Customer Network Operation  Run customer network operations SNMP queries, CLI and Provisioning commands on DUT for the duration of 2 hours.  Check for any memory leaks and core-dumps.  RE/PFE resource measurements will be taken during the test execution  Check if any BGP session flap on both RR DUT and RR client  Test case 7.2 BGP policy activate/deactivate provision  Apply BGP import/export policy to filter bgp routes  Check routes updated accordingly and no core seen.  Test case 7.3 Add BGP groups  Split existing bgp peers into more bgp groups to increase the number of bgp groups  Commit the config change and make sure no core seen and bgp peers are up.  Test case 7.4 Merge multiple BGP groups to one  Merge two or three bgp groups to one group, commit the change.  make sure bgp peer re-established and no rpd core seen,

32 32 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 7 – Customer Network Operation  Test case 7.5 SNMP Fault Management Trap and MIB  Run SNMP trap and SNMP MIB polling test per document received from AT&T (trimmed from document M320 MIS SNMP MIBS TRAPS SYSLOGS CLI.doc for items applied to MIS-RR).  Test Case 7.6 SNMP Performance MIBs test  Test Case 7.7 Other Pollable MIBs  Test Case 7.8 Cli troubleshooting Commands SNMP MIBs/CLI commands tested is in this document M320 MIS-RR SNMP MIBS TRAPS SYSLOGS CLI.doc

33 33 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 8 – HALT Test HALT TEST  Periodic network events on DUT for 24 or 36 hours  Monitor for cores and error messages during the test  Verify the steady state check list at the end of the test  Measure RE and PFE resource periodically Note: HALT Test will run the following triggers randomly Restart FPCFPC offline/onlinePIC offline/online RE switchoverdeactivate/activate interface Restart routingRestart chassisRestart dcd clear ospf neighborclear bgp neighbordeactivate/activate bgp deactivate/activate ospfdisable/enable ospfdisable/enable bgp

34 34 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 9 – Memory Leak Test Valgrind on RPD  Run valgrind on RPD on DUT  Periodic routes flapping and BGP peer flapping  Run test for 8 hours  Check any memory leak on RPD Note: Need to reduce scaling for valgrind testing Will try baseline first, if not working, reduce scaling down

35 35 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 10 – Security Test  Security Test – MU Dynamics Test for OSPF, BGP and TCP syn attack Scaling need to be reduced for this test. Test with baseline scaling first, if not working, scale down.

36 36 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Cases 11 – OPERATION TRIGGER ITERATION TEST Operation Trigger Iteration Test Iterate the triggers 5 times, make sure DUT will be recovered after the trigger. Test Case 11.1 FPC offline/online Offline/online FPC 5 times, wait for 5 minutes between the triggers, Make sure DUT will be recovered. No cores happen. Test Case 11.2 FPC Restart Restart FPC 5 times, wait for 5 minutes between the triggers, Make sure DUT will be recovered. No cores happen. Test Case 11.3 RE Switchover RE Switchover 5 times, wait for 30 minutes between the triggers, make sure DUT operates and no cores. Test Case 11.4 PIC Offline/Online PIC offline/online 5 times, wait for 5 minutes between the triggers, make sure DUT recovered and no cores.

37 37 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Case 12 - Melt Down Scaling Number Test Case 12.1 Found maximum number of IPv4 routes Test Setup and Procedure 1. Setup 64bit M320 with scaling number 3A 2. Increase IPv4 routes from IXIA emulated PEs (single path routes) 3. Wait for DUT to steady state 4. Monitor memory usage 5. Repeat Step 2-5 until DUT memory usage reach to 85%

38 38 Copyright © 2009 Juniper Networks, Inc. www.juniper.net Test Case 12 – Melt Down Scaling Number Test Case 12.2 Found maximum number of IPv4 paths Test Setup and Procedure 1. Setup network with scaling number 3A 2. Increase IPv4 routes reflected from RRs (Increase IPv4 paths) 3. Wait for DUT to steady state 4. Monitor memory usage 5. Repeat Step 2-5 until DUT memory usage to 85%

39 39 Copyright © 2009 Juniper Networks, Inc. www.juniper.net BGP CONVERGENCE COMPARISON BETWEEN 32BIT AND 64BIT No. of Paths Add. ipv4 routes Add. ipv6 routes Total ipv4 BGP routes Total ipv6 BGP routes Total ipv4 paths Total ipv6 paths BGP peer groups 32bit/64bit RPD Converg. time (average) 32bit/64bit RE Memory Utilization 32bit/64bit RPD Memory Usage 32bit/64bit RPD Memory Utilization (min.) % MB% 10none 702,00022,000 7,020,000 220,00025 10 10,000none802,00022,0008,020,000220,00025 10 none10,000702,00032,0007,020,000320,00025 10 36,000 10,000738,00032,0007,380,000320,00025 10 72,000 20,000774,00042,0007,740,000420,00025 10 144,000 40,000846,00062,0008,460,000620,00025 10 216,000 80,000918,000102,0009,180,0001,020,00025 10 275,000 160,000977,000182,0009,770,0001,820,00025 10 350,000 170,0001,052,000192,00010,520,0001,920,00025 20 none 702,00022,00014,040,000440,00025 20 216,000 80,000918,000102,00018,360,0002,040,00025

40


Download ppt "AT&T M320 NGRE 64bit MIS RR TEST PLAN – 13.2 RBU AT&T CST Team July 2013."

Similar presentations


Ads by Google