1. 2008. 4. 2 CJK NGN Test-bed Ad-hoc Group
CJK NGN Test-bed Activities
Phase I & II Testing Result
CJK NGN Test-bed Ad-hoc Group

2. Topics Terms of Reference Testing Schedule Testing Scenarios
Testing Items
Testing Network & Service Configurations
Testing Activities
Testing Results
Next Steps

3. Terms of Reference

4. TOR: NGN Test-bed Objectives
Interconnecting CJK NGN Test-beds and building a common NGN technology testing infrastructure
Feasibility testing prior to NGN commercial deployment
Interoperability testing for Standard-based NGN components developed by CJK
Providing NGN service development platform especially focusing on inter-country issues
Reflection of testing results into corresponding standardization activities periodically
Cooperation with other international NGN related testbeds and their activities such as MOONv6, Plugtests, GMI, etc.

5. TOR: Work Plan Phase 1 (March 2005 ~ March 2006): Preparation Stage
Pick up the study items which include our target services for the evaluations on the test-bed
Check the participants who are interested in the studies again
Plan the schedule of the implementation and evaluation of the test-bed
Prepare the test-bed in each SDO
Establish each administration’s NOC for each SDO’s test-bed
Agree on first and second set of target services and components and test scenarios
Build test case suits for the agreed target services including a specific milestone
Phase II (April 2006 ~ December 2007): First Interoperability Testing Stage
Construct CJK NGN test-bed
common operation and management infrastructure
Test agreed first set of target services, especially focusing on transport functionality such as QoS, adimission control, traffic monitoring, etc.
Especially main interest is in inter-domain/inter-operator issues
Phase III (January 2008~ December 2010): Second Interoperability Testing Stage
Test agreed second set of target services, especially focusing on application/service functionality and its binding with transport functionality
Test other advanced NGN services yet to be defined
NOTE: Phase III and later phases are being updated. Refer to Phases III and IV plans.

6. TOR: Action Plan for Phase I
Define Objectives of the CJK interoperability tests for NGN (4th Meeting)
Define Roadmap (4th Meeting)
Define Network Architecture (Physical, Transport and Service Topologies and underlying technologies (e.g., MPLS, etc.)) of the testbed both intra- and inter-domain scope (4th Meeting)
In Intra-domain case, each administration decides its own network (e.g., KOREN or APII in case of Korea) and enhances its functionalities to conform to NGN
Define Implementation Agreements for inter-domain connectivity (5th Meeting)
Define target services, target components, test scenarios and test case suits (5th Meeting)
Recruit participants for equipment and services at each testbed site (5th Meeting)
Establish a dedicated reflector for detailed discussion

7. Testing Schedule

8. Testing Schedule Phase I: 2006 3rd/4th Quarter
Network Connectivity
Senario1 only (Feb. 26 ~ Mar. 2)
Conducted by CATR and ETRI
First Report/Demo during 9th CJK NGN Meeting (April 2007)
Phase II: st/2nd Quarter
Scenario 2 ~ 5 (Feb. 26 ~ Mar. 2)
Second Report/Demo during 9th CJK NGN Meeting (April 2007)
Phase III: th ~ st/2nd Quarter
Scenario 6 ~ 7
Testing details will further be defined depending on the technology availability status
CATR, KDDI, NTT, and ETRI will participate in the testing
Third Report/Demo 13th CJK NGN Meeting
NOTE: Phase III and later phases are being updated. Refer to Phases III and IV plans.

9. Testing Scenarios

10. Test Scenarios
Scenario 1: Single Call Server within a single NGN Network Domain without PSTN
Scenario 2: Call Servers across multiple IP Network Domains without PSTN
Scenario 3: Single Call Server within a single NGN Network Domain with PSTN
Scenario 4: Call Servers across multiple NGN Network Domains with PSTN
Scenario 5: Call Servers across multiple NGN Network Domains with PSTN & Value Added Services
This scenario adds Value Added Services to scenario 4 using Media Servers and Application Servers
Scenario 6: Performance Evaluation
Scenario 7: Call Servers across multiple NGN Network Domains with PSTN, Value Added Services and RACF Involvement

11. Scenario 1: Single Call Server within a single IP Network Domain without PSTN
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-I),H.323
SIP(-I),H.323 AG SG SG AG
Enterprise
Access
Enterprise
Access
Residential
Access
Residential
Access
Country 1
Country 2

12. Scenario 1 - Testing Cases
Basic Call with the following combinations
Access Gateway – Access Gateway
SIP phone – Access Gateway
SIP phone – SIP phone
Subscriber Gateway – Access Gateway
Subscriber Gateway – SIP phone
Subscriber Gateway – Subscriber Gateway
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

13. Scenario 2: Call Servers across multiple IP Network Domains without PSTN
SIP-I
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-I),H.323
SIP(-I),H.323
Enterprise Access
Enterprise Access SG AG AG
Residential Access
Residential Access
Country 1
Country 2

14. Scenario 2 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-I
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

15. Scenario 3: Single Call Server within a single NGN Network Domain with PSTN
IP Network Domain SG
PSTN
Sigtran (SCTP/M3UA)
Call Server
MGCP H.248/Megaco
Class4/5 SW TG
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-I),H.323
SIP(-I),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

16. Scenario 3 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Access Gateway – Class4/5 SW
Subscriber Gateway – Class4/5 SW
SIP Phone – Class4/5 SW
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIGTRAN
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

17. Scenario 4: Call Servers across multiple NGN Network Domains with PSTN
IP Network Domain 1
Class4/5 SW
IP Network Domain 2 SG SG
PSTN TG
Call Server 2
Call Server 1 TG
SIP-I
SIP
H.323
MGCP
H.248/Megaco*
SIP(-I),H.323
SIP
H.323
MGCP
H.248/Megaco*
SIP(-I),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

18. Scenario 4 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Access Gateway – Class4/5 SW
Subscriber Gateway – Class4/5 SW
SIP Phone – Class4/5 SW
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-I
SIGTRAN
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

19. Scenario 5: Call Servers across multiple NGN Network Domains with Value Added Services & PSTN
Back-End Servers
Back-End Servers MS MS
SIP
Application Server
SIP
Application Server
Class4/5 SW
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco SG SG
SIP(-)
SIP(-I)
PSTN TG TG
IP Network
Domain 1
IP Network
Domain 2
Call
Server 2
Call
Server 1
SIP-I
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-I),H.323
SIP(-I),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

20. Scenario 5 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 1
Access Gateway – Class4/5 SW
Subscriber Gateway – Class4/5 SW
SIP Phone – Class4/5 SW
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-I
SIGTRAN
Application Servers Feature Testing
SIP Application Server Features: IP conferencing, Voice mail, etc.
Media Server Features: Generic conference bridge, Voice mail media processing, etc.
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

21. Scenario 6 – Performance Evaluation
Network Performance Testing cases
E2E Call Setup Delay
Call Completion Rate
Each Segment & E2E IPTD/IPDV/IPLR/Path unavailability
Call Traffic Accounting
Harmonization Testing of Service QoS class differences among CJK
User Terminal Performance Testing cases
Various User Terminal Performance Testing

22. Scenario 7: Call Servers across multiple NGN Network Domains with Value Added Services, PSTN, and RACF Involvement
Back-End Servers
Back-End Servers MS MS
SIP
Application Server
SIP
Application Server
Class4/5 SW
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco
Diameter
RADIUS
Proprietary
SIP
MGCP
H.248/Megaco SG SG
SIP(-I)
SIP(-I)
PSTN TG TG
IP Network
Domain 2
IP Network
Domain 1
Call
Server 2
Call
Server 1
SIP-I
RACF
RACF
SIP
H.323
MGCP
H.248/Megaco*
SIP
H.323
MGCP
H.248/Megaco*
SIP(-I),H.323
SIP(-I),H.323
Enterprise Access
Enterprise Access SG SG AG AG
Residential Access
Residential Access
Country 1
Country 2

23. Scenario 7 - Testing Cases
Basic Call with the following combinations
Same cases with scenario 7
SIP phone – RACF
Access Gateway – RACF
Subscriber Gateway – RACF
Application Server - RACF
Protocols to test
SIP
H.323
MGCP
H.248/Megaco
SIP-I
SIGTRAN Rq
Application Servers Feature Testing
SIP Application Server Features: IP conferencing, Voice mail, etc.
Media Server Features: Generic conference bridge, Voice mail media processing, etc.
RACF Feature Testing
Resource Admission Control Performance
NAT/Gate Control Performance
Testing Residential & Business Features
Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference
Centrex, Private numbering plan, DDI/DDO

24. Testing Items

25. Detailed Test Items Number Service Combination Test case
Basic voice call
MGCP IAD–H.248 AG or IAD
Normal call
Abnormal call- the called party is busy
Abnormal call- no answer
Abnormal call- the number does’t exist
MGCP IAD–SIP phone
SIP phone–H.248 AG or IAD

26. Detailed Test Items PSTN user – SIP phone
Normal call
Abnormal call- the called party is busy
Abnormal call- no answer
Abnormal call- the number doesn’t exist
Multimedia call
SIP phone – SIP phone
Fax
IAD/AG–IAD/AG
Including G.711 and T.38

27. Testing Item Description Ex. 1
TEST NUMBER:：1
TITLE：Basic Call between SIP user and AG user
SUBTITLE：normal call
PRE-TEST CONDITIONS：
1、SIP user 1 registered on SS1; AG registered on SS2；User 2 is one of the AG’s Subscribers.
2、The protocol between SS1 and user1 is SIP；The protocol between user2 and SS2 is H.248
TEST CONFIGURATION：SIP user 1－SS1－SS2－AG－user 2
TEST STEP：
1、User 1 calls user2，user2’s phone rings；
2、User2 answers the call；
3、User1 hangs up first,then user2 hangs up,too.

28. Testing Item Description Ex. 1
EXPECTED SIGNAL SEQUENCE:：
Call-setup flow：
SIP user1 sends an INVITE to SS1 to setup a session，SS1 responses 100 Trying.SS1 forwards the INVITE request to SS2.
SS2 sends an Add request to creat a new context in AG .AG sends back a Reply message with the RTP port and the stream descriptor.
SS2 sends an Modify to AG ,AG sends the ringtone to user2 and monitors user2 hook-off.
SS2 sends a 180Ringing response to SS1,SS1 forwards this message to SS2.
User2 answers the call ,AG sends a Notify to SS2 and SS2 sends back a Reply.
SS2 sends a 200 OK response to SS1 and SS1 forwards to user1.
User1 sends back ACK .
SS2 sends AG a Modify command to stop ring tone and monitor user2 hook-on.

29. Testing Item Description Ex. 1
Call-release flow：
Caller releases the call, user1 sends a BYE to SS1,and SS1 sends BYE to SS2.
SS2 sends a Modify command to AG to send a busy tone to user2.
SS2 sends a 200 OK response to SS1.SS1 sends 200 ok to user1.
User2 hungs up,and AG sends Notify to AG,then SS2 sends Subtract (TDM) + Subtract (RTP) to AG to delete the context.
SS2 sends Modify to AG to monitor hook-oof.

30. Testing Item Description Ex. 2
TEST NUMBER:：2
TITLE：Basic Call between SIP user and AG user
SUBTITLE：abnormal call——the called party is busy
PRE-TEST CONDITIONS：
1、SIP user1 registered on SS1; AG registered on SS2；User2 is one of the AG’s Subscribers.
2、The protocol between SS1 and user1 is SIP；The protocol between user2 and SS2 is H.248
3、User2 is busy.
TEST CONFIGURATION：SIP user1－SS1－SS2－AG－user2
TEST STEP：
1、User1 calls user2；
2、User1 hears the busytone or voice information；
3、User1 hangs up.

31. Testing Item Description Ex. 2
EXPECTED SIGNAL SEQUENCE:：
SIP user1 sends INVITE to SS1 to setup a session，SS1 responses 100 Trying.SS1 forwards the INVITE request to SS2.
SS2 knows that the called party is busy now, then SS2 sends a 486 response to SS1.
SS1 forwards the 486 to user1.

32. Testing Item Description Ex. 3
TEST NUMBER:：17
TITLE：Basic Call between SIP users
SUBTITLE： normal call
PRE-TEST CONDITIONS：
1、SIP user 1 registered on SS1; SIP user 2 registered on SS2.
TEST CONFIGURATION：SIP1－SS1－SS2－SIP2
TEST STEP：
1、User1 calls user2，user2’s phone rings；
2、User2 answers the call；
3、User1 hangs up first,then user2 hangs up,too.

33. Testing Item Description Ex. 3
EXPECTED SIGNAL SEQUENCE:：
(1) SIP user1 sends an INVITE request to SS1 to setup a session，SS1 responses 100 Trying.SS1 forwards the INVITE request to SS2 then to user2.
(6) SIP user1 rings .User 2 sends a 180Ringing response to SS2,SS2 forwards this message to SS1 then to user1.
(9) User2 answers the call ,User 2 sends a 200 OK response to SS2 then to User1.
(10) User1 sends ACK back.
User1 and user2 setups a multimedia session.

34. Testing Network & Service Configurations

35. CATR’s CS Lab Configuration
SCP
SoftX3000
SIP
IOSS
FTP/FTAM
SIP/MGCP
/H.248/H.323 U -
NICA
SIP
SIGTRAN
Lanswitch
INAP
USAU
SIGTRAN
MGCP
MRS6100
STP
ISUP
IP Network
SG7000
ISUP
SNMP
STUN
UMSN2000 FE
PSTN
Eudemon E1
Lanswitch
Lanswitch
H.248
H.323
SIP
MGCP
UMG 8900+UA5000
POTS / BRI/ADSL
POTS / BRI/ADSL …
VedioPhone
OpenEye
SoftPhone
IAD

36. ETRI’s CS Lab Network Configuration
Japan
CERNET
155M POS
Internet
Border
Router
2G Ethernet
Support Facility
GBE
Management/
Monitoring System,
GW, SIP phones, …
GBE, 100BT, 10BT
Meeting Room
100BT
SBC
100BT(VLAN)
L2 Switch
100BT
100BT
100BT
100BT
100BT
AS/MS CS
SG/TG
RACF
Packet
Monitor
E1 TDM
100BT
PSTN
Network
Signaling Link
Bearer Link

37. Testing Services, Venders & Terminals
Interworking services:
voice and video
Interworking venders:
CATR (Huawei) and ETRI (Xener)
Interworking terminals:
ETRI SIP phone and CATR SIP, MGCP IAD and PSTN terminal(H.248 TG)
25 test items

38. Testing Activities

39. Testing Processes and Schedules
Started at Sept.25, 2006
Four Phases
Network connection: Sept.25-30, 2006
Data configuration: Jan.30-Feb.1, 2007
Test: Feb.26-March 2
Test: March 5-9

40. Testing Result Examples
The following several testing result examples provide a few failed cases and successful cases
Ethereal was used to capture the SIP messages exchanged in each Call Server site
The failures were fixed during the testing process
The failures were two kinds: intentional (i.e., abnormal) and non-intentional
The former were used to check the correctness of the system behavior. For example, a call attempt was used to non-existing callee
The latter were mainly for the message format mismatches and/or IP address misuses

41. Failed Testing Example 1
This example shows the receiving Call Server doesn’t understand the SIP Invite request message

42. Failed Testing Example 2
This example shows call attempt to non-existing callee

43. Successful Testing: Normal

44. Successful Testing: Busy

45. Testing Results

46. Testing Results

47. Testing Results

48. Testing Results

49. Lessons Learned from Testing
First successful interconnection of CK NGN Test-beds and build of a common NGN technology testing infrastructure
Successful interoperability testing for CJK-driven ITU-T Standard-based NGN components and services (i.e., call server)
Provided NGN service development platform especially focusing on inter-country issues
First attempt to bridge the standard and implementation by the reflection of testing results into corresponding standardization activities
Stimulation of further test-bed activities, for example, RTP/RTCP-based Performance Measurement testing

50. Next Steps
NOTE: Information here is at April Further updates will be made.

51. Upcoming Testing Phases
Phase I: rd/4th Quarter (Completed)
Network Connectivity
Scenario 2 and 4 only (2 CS and 2 domains)
Phase II: st/2nd Quarter (Completed)
Scenario 1 and 3, (simpler version of 2 and 4, single domain)
Phases I & II have been assumed to be achieved their objectives originally planned such as testing the interoperability issues of the SIP signaling. We decided to revise the scenarios of phases III & IV and pursuit them as the high priority study among CJK considering the current NGN requirements.
CJK NGN Test-bed
Phase III: st/2nd Quarter
Performance Evaluation of RTP/RTCP-based MoIP (Multimedia
Voice/Audio/ Video) over IP) service
Network connective for Performance evaluation and Performance
Monitoring Scenarios 6-1 and 6-2
Phase IV: rd ~ nd Quarter
Performance Evaluation of RTP/RTCP-based MoIP and IPTV services
Performance Monitoring Scenarios 6-3 and 6-4
Initial RACF Interoperability testing : Scenario 7
Testing of RACF and RTP/RTCP-based MPM Interactions
IPTV Interoperability testing

52. Test Scenario 6 & 7 in Phase III & IV
Performance Evaluation with RTP/RTCP-based Measurement extensions
and initial RACF interoperability testing.
Scenario 6-1: Performance evaluation of Network Segments
Scenario 6-2: Performance evaluation of MoIP service among CPEs
Local testing of MPM (Provided by KDDI R&D Labs.)
CPE-to-CPE performance evaluation
Scenario 6-3: Performance evaluation of MoIP service with two MPMs
CPE – MPM1 – MPM2 – CPE performance evaluation
Scenario 6-4: Performance evaluation of MoIP service with three MPMs
CPE – MPM1 – MPM2 – MPM3 – CPE performance evaluation
Scenario 7: Testing of RACF and RTP/RTCP-based MPM Interactions

53. CPE terminals and other equipment
Scenarios 6 & 7 Details
Scenario No.
IP Network
Service Platform
Services
CPE terminals and other equipment
Evaluations
Countries
Report
Metrics
Connections
6-1
C-K: R&D Network
C-J: Commercial Network
K-J: Commercial Network
SIP
RTP 　　　　Streams
(MoIP:
Multimedia over IP)
Voice/
Audio/
Video)
>>IPTV
CPE (VoIP/Video): J (Soft/Hard), K, C
Call Server: K, C, J（Emulator)
MPM: J (Soft/Hard), K
Y.1543 －
Local test
C,J,K
Two MPMs　　
C-J, K-J, C-K
Three MPMs
C-J-K
13th CJK meeting
6-2
RTCP (CPE-CPE)
CPE-CPE
6-3
RTCP (CPE-MPM1-MPM2-CPE)
CPE-MPM
6-4
RTCP (CPE-MPM1-MPM2-MPM3-CPE) 7
SIP/
IMS-SIP
RACF: K, J（1st: ACF, 　　　　　　2nd: RCF)
MPM-RACF
14th or 15th CJK meeting

54. Thanks You CJK NGN Test-bed Ad-hoc Group