1. CJK NGN Test-bed Activities Phase I & II Testing Result
Taesang Choi
On behalf of CJK NGN Test-bed Ad-hoc Group

2. Topics Terms of Reference Testing Schedule Testing Scenarios
Testing Items
Testing Network & Service Configurations
Testing Activities
Testing Results
Next Steps

3. Terms of Reference

4. TOR: NGN Testbed Objectives
Interconnecting CJK NGN Testbeds and building a common NGN technology testing infrastructure
Feasibility testing prior to NGN commercial deployment
Interoperability testing for Standard-based NGN components developed by CJK
Providing NGN service development platform especially focusing on inter-country issues
Reflection of testing results into corresponding standardization activities periodically
Cooperation with other international NGN related testbeds and their activities such as MOONv6, Plugtests, GMI, etc. (Note: This issue will be revisited in the future meeting)

5. TOR: Work Plan Phase 1 (March 2005 ~ March 2006): Preparation Stage
Pick up the study items which include our target services for the evaluations on the test bed
Check the participants who are interested in the studies again
Plan the schedule of the implementation and evaluation of the test bed
Prepare the testbed in each SDO
Establish each administration’s NOC for each SDO’s testbed
Agree on first and second set of target services and components and test scenarios
Build test case suits for the agreed target services including a specific milestone
Phase II (April 2006 ~ December 2007): First Interoperability Testing Stage
Construct CJK NGN testbed
common operation and management infrastructure
Test agreed first set of target services, especially focusing on transport functionality such as QoS, adimission control, traffic monitoring, etc.
Especially main interest is in inter-domain/inter-operator issues
Phase III (January 2008~ December 2010): Second Interoperability Testing Stage
Test agreed second set of target services, especially focusing on application/service functionality and its binding with transport functionality
Test other advanced NGN services yet to be defined

6. TOR: Action Plan for Phase I
Define Objectives of the CJK interoperability tests for NGN (4th Meeting)
Define Roadmap (4th Meeting)
Define Network Architecture (Physical, Transport and Service Topologies and underlying technologies (e.g., MPLS, etc.)) of the testbed both intra- and inter-domain scope (4th Meeting)
In Intra-domain case, each administration decides its own network (e.g., KOREN or APII in case of Korea) and enhances its functionalities to conform to NGN
Define Implementation Agreements for inter-domain connectivity (5th Meeting)
Define target services, target components, test scenarios and test case suits (5th Meeting)
Recruit participants for equipment and services at each testbed site (5th Meeting)
Establish a dedicated reflector for detailed discussion

7. Testing Schedule

8. Testing Schedule Phase I: 2006 3rd/4th Quarter
Network Connectivity
Senario1 only (Feb. 26 ~ Mar. 2)
First Report/Demo during 9th CJK NGN Meeting (April 2007)
Phase II: st/2nd Quarter
Scenario 2 ~ 4 (Feb. 26 ~ Mar. 2)
Second Report/Demo during 9th CJK NGN Meeting (April 2007)
Phase III: rd/4th ~ st/2nd Quarter
Scenario 5 ~ 7
Testing details will further be defined depending on the technology availability status
Third Report/Demo 11th CJK NGN Meeting

9. Testing Scenarios

10. Test Scenarios
Scenario 1: Single Call Server within a single NGN Network Domain without PSTN
Scenario 2: Call Servers across multiple IP Network Domains without PSTN
Scenario 3: Single Call Server within a single NGN Network Domain with PSTN
Scenario 4: Call Servers across multiple NGN Network Domains with PSTN
Scenario 5: Call Servers across multiple NGN Network Domains with PSTN & Value Added Services
This scenario adds Value Added Services to scenario 4 using Media Servers and Application Servers
Scenario 6: Performance Evaluation
Scenario 7: Call Servers across multiple NGN Network Domains with PSTN, Value Added Services and RACF Involvement

11. Scenario 1: Single Call Server within a single IP Network Domain without PSTN
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-T),H.323
SIP(-T),H.323 AG SG SG AG
Enterprise
Access
Enterprise
Access
Residential
Access
Residential
Access
Country 1
Country 2

12. Scenario 1 - Testing Cases
Basic Call with the following combinations
Access Gateway – Access Gateway
SIP phone – Access Gateway
SIP phone – SIP phone
Subscriber Gateway – Access Gateway
Subscriber Gateway – SIP phone
Subscriber Gateway – Subscriber Gateway
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

13. Scenario 2: Call Servers across multiple IP Network Domains without PSTN
SIP-T
H.323
BICC
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-T),H.323
SIP(-T),H.323
Enterprise Access
Enterprise Access SG AG AG
Residential Access
Residential Access
Country 1
Country 2

14. Scenario 2 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-T or BICC
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

15. Scenario 3: Single Call Server within a single NGN Network Domain with PSTN
IP Network Domain SG
PSTN
Sigtran (SCTP/M3UA)
Call Server
MGCP H.248/Megaco
Class4/5 SW TG
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-T),H.323
SIP(-T),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

16. Scenario 3 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Access Gateway – Class4/5 SW
Subscriber Gateway – Class4/5 SW
SIP Phone – Class4/5 SW
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIGTRAN
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

17. Scenario 4: Call Servers across multiple NGN Network Domains with PSTN
IP Network Domain 1
Class4/5 SW
IP Network Domain 2 SG SG
PSTN TG
Call Server 2
Call Server 1 TG
SIP-T
H.323
BICC
SIP
H.323
MGCP
H.248/Megaco*
SIP(-T),H.323
SIP
H.323
MGCP
H.248/Megaco*
SIP(-T),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

18. Scenario 4 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Access Gateway – Class4/5 SW
Subscriber Gateway – Class4/5 SW
SIP Phone – Class4/5 SW
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-T or BICC
SIGTRAN
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

19. Scenario 5: Call Servers across multiple NGN Network Domains with Value Added Services & PSTN
Back-End Servers
Back-End Servers MS MS
SIP
Application Server
SIP
Application Server
Class4/5 SW
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco SG SG
SIP(-T)
SIP(-T)
PSTN TG TG
IP Network
Domain 1
IP Network
Domain 2
Call
Server 2
Call
Server 1
SIP-T
H.323
BICC
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-T),H.323
SIP(-T),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

20. Scenario 5 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Access Gateway – Class4/5 SW
Subscriber Gateway – Class4/5 SW
SIP Phone – Class4/5 SW
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-T or BICC
SIGTRAN
Application Servers Feature Testing
SIP Application Server Features: IP conferencing, Voice mail, etc.
Media Server Features: Generic conference bridge, Voice mail media processing, etc.
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

21. Scenario 6 – Performance Evaluation
Network Performance Testing cases
E2E Call Setup Delay
Call Completion Rate
Each Segment & E2E IPTD/IPDV/IPLR/Path unavailability
Call Traffic Accounting
Harmonization Testing of Service QoS class differences among CJK
User Terminal Performance Testing cases
Various User Terminal Performance Testing

22. Scenario 7: Call Servers across multiple NGN Network Domains with Value Added Services, PSTN, and RACF Involvement
Back-End Servers
Back-End Servers MS MS
SIP
Application Server
SIP
Application Server
Class4/5 SW
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco SG SG
SIP(-T)
SIP(-T)
PSTN TG TG
IP Network
Domain 2
IP Network
Domain 1
SIP-T
H.323
BICC
Call
Server 2
Call
Server 1
RACF
RACF
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-T),H.323
SIP(-T),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

23. Scenario 7 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 7
SIP phone – RACF
Access Gateway – RACF
Subscriber Gateway – RACF
Application Server - RACF
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-T or BICC
SIGTRAN Rq
Application Servers Feature Testing
SIP Application Server Features: IP conferencing, Voice mail, etc.
Media Server Features: Generic conference bridge, Voice mail media processing, etc.
RACF Feature Testing
Resource Admission Control Performance
NAT/Gate Control Performance
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

24. Testing Items

25. Detailed Test Items Number Service Combination Test case
Basic voice call
MGCP IAD–H.248 AG or IAD
Not Available by ETRI
Normal call
Abnormal call- the called party is busy
Abnormal call- no answer
Abnormal call- the number does’t exist
MGCP IAD–SIP phone
Available only for SIP phone in ETRI side
SIP phone–H.248 AG or IAD

26. Detailed Test Items PSTN user – SIP phone Available by ETRI
Normal call
Abnormal call- the called party is busy
Abnormal call- no answer
Abnormal call- the number doesn’t exist
Multimedia call
SIP phone – SIP phone
Fax
IAD/AG–IAD/AG
Not Available by ETRI
Including G.711 and T.38

27. Testing Network & Service Configurations

28. RTNet’s CS Lab Configuration
SCP
SoftX3000
SIP
IOSS
FTP/FTAM
SIP/MGCP
/H.248/H.323 U -
NICA
SIP
SIGTRAN
Lanswitch
INAP
USAU
SIGTRAN
MGCP
MRS6100
STP
ISUP
IP Network
SG7000
ISUP
SNMP
STUN
UMSN2000 FE
PSTN
Eudemon E1
Lanswitch
Lanswitch
H.248
H.323
SIP
MGCP
UMG 8900+UA5000
POTS / BRI/ADSL
POTS / BRI/ADSL …
VedioPhone
OpenEye
SoftPhone
IAD

29. ETRI’s CS Lab Network Configuration
Japan
CERNET
155M POS
Internet
Border
Router
2G Ethernet
Support Facility
GBE
GBE, 100BT, 10BT
Management/
Monitoring System,
GW, SIP phones, …
Meeting Room
100BT
SBC
100BT(VLAN)
L2 스위치
100BT
100BT
100BT
100BT
100BT
AS/MS CS
SG/TG
RACF
Packet
Monitor
E1 TDM
100BT
PSTN
모델망
Signaling Link
Bearer Link

30. Testing Services, Venders & Terminals
Interworking services:
voice and video
Interworking venders:
CATR (Huawei) and ETRI (Xener)
Interworking terminals:
ETRI SIP phone and CATR SIP, MGCP IAD and PSTN terminal(H.248 TG)
25 test items

31. Testing Activities

32. Testing Processes and Schedules
Started at Sept.25, 2006
Four Phases
Network connection: Sept.25-30, 2006
Data configuration: Jan.30-Feb.1, 2007
Test: Feb.26-March 2
Test: March 5-9

33. Failed Testing Example 1

34. Failed Testing Example 2

35. Successful Testing: Normal

36. Successful Testing: Busy

37. Testing Results

38. Testing Results

39. Testing Results

40. Testing Results

41. Next Steps

42. Upcoming Testing Phases
Phase I: rd/4th Quarter (Completed)
Network Connectivity
Scenario 2 and 4 only (2 CS and 2 domains)
Status Report during 8th CJK NGN Meeting (January 2007)
Phase II: st/2nd Quarter (Completed)
Scenario 1 and 3, (simpler version of 2 and 4, single domain)
Status Report at the 12th CJK NGN Meeting (March 2008)
Phase III: st/2nd Quarter
Performance Evaluation of RTP/RTCP-based MoIP （Multimedia（Voice/Audio/Video) over IP） service
Network connective for Performance evaluation and Performance Monitoring Scenarios 6-1 and 6-2
Status report at the 13th CJK NGN Meeting in or 10
Phase IV: rd ~ nd Quarter
Performance Evaluation of RTP/RTCP-based MoIP and IPTV services
Performance Monitoring Scenarios 6-3 and 6-4
Initial RACF Interoperability testing : Scenario 7
Testing of RACF and RTP/RTCP-based MPM Interactions
IPTV Interoperability testing
Status report at the 14th and 15th CJK NGN Meetings
Goal of the CJK NGN Test-bed

43. Scenario 6 is under Study Now
“Scenario 6: Performance Evaluation with RTP/RTCP-based Measurement extensions” could be fractionated as follows:
Scenario 6-1: Performance evaluation of Network Segments
Measure Network Performance Metrics among CJK independent from services
Measured results can be cross referenced by subsequent tests for accuracy
Periodic and constant measurement
Measurement intervals and other details to be defined
IPTD, IPDV, IPLR of C-K, C-J, J-K, J-C, K-C, K-J
Scenario 6-2: Performance evaluation of MoIP service among CPEs
Measure RTP/RTCP performance of MoIP（Multimedia（Voice/Audio/Video) over IP） service based on CS and/or IMS
CPE-to-CPE performance evaluation
Scenario 6-3: Performance evaluation of MoIP service with two MPMs
Measure RTP/RTCP performance of MoIP service based on CS and/or IMS
CPE – MPM1 – MPM2 – CPE performance evaluation
Scenario 6-4: Performance evaluation of MoIP service with three MPMs
MPMs send/append evaluated performances
CPE – MPM1 – MPM2 – MPM3 – CPE performance evaluation

44. CPE terminals and other equipment
Scenarios 6 & 7 Details
Scenario No.
IP Network
Service Platform
Services
CPE terminals and other equipment
Evaluations
Countries
Report
Metrics
Connections
6-1
C-K: R&D Network
C-J: Commercial Network
K-J: Commercial Network
SIP
RTP 　　　　Streams
(MoIP:
Multimedia over IP)
Voice/
Audio/
Video)
>>IPTV
CPE (VoIP/Video): J (Soft/Hard), K, C
Call Server: K, C, J（Emulator)
MPM: J (Soft/Hard), K
Y.1543 －
Local test
C,J,K
Two MPMs　　
C-J, K-J, C-K
Three MPMs
C-J-K
13th CJK meeting
6-2
RTCP (CPE-CPE)
CPE-CPE
6-3
RTCP (CPE-MPM1-MPM2-CPE)
CPE-MPM
6-4
RTCP (CPE-MPM1-MPM2-MPM3-CPE) 7
SIP/
IMS-SIP
RACF: K, J（1st: ACF, 　　　　　　2nd: RCF)
MPM-RACF
14th or 15th CJK meeting

45. Next Steps
ETRI will upgrade our CS platform so that other tests can be available including PSTN interworking, IAD, AG support, etc.
QoS consideration for transport
Preparation for Phase 3 testing
What about IMS-based Interop testing?
Need budget support for more meaningful tests
Collaboration opportunities with other Testing Programs? (In Korea, we have BcN test-bed. What about others?)

46. Thanks You CJK NGN Test-bed Ad-hoc Group