OPTICAL NETWORKING STANDARDS: A COMPREHENSIVE GUIDE
OPTICAL NETWORKING STANDARDS: A COMPREHENSIVE GUIDE
Edited by Khurram Kazi
Springer
Khurram Kazi, Ph.D
[email protected]
Optical Networking Standards: A Comprehensive Guide
Library of Congress Control Number: 2006921777 ISBN 0-387-24062-4 ISBN 978-0-387-24062-6
e-ISBN 0-387-24063-2
Printed on acid-free paper. "The materials in Chapters 10 and 12 have been reproduced by Springer with the permission of Cisco Systems, Inc. COPYRIGHT © 2006 CISCO SYSTEMS, INC. ALL RIGHTS RESERVED." © 2006 Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science-HBusiness Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if the are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. 9 8 7 6 5 4 3 2 1 springer.com
Dedication
This book is dedicated to my wife Sameema, my family and friends and all the folks who have spent countless hours developing networking standards
Contents
Foreword
xix
Preface
xxi
Acknowledgements
xxiii
About the Authors
xxv
CHAPTER 1 OVERVIEW 1.1. Optical Transport Network Infrastructure 7.7.7 Functional Modeling Specification Technique 1.1.2 Multiservice Optical Transport Network Infrastructure 1.1.3 Global Optical Transport Network Timing 1.2. Carriage of Services over Transport Networks 7.2.7 Ethernet Services Architecture and Definitions 1.2.2 Storage Area Services Over SONET 1.3. Control and Management of Optical Transport Networks 1.4. Intra-Network Element Communication and Component-centric Standards 1.4.1 Intra-Network Element Communication 1.4.2 Optical Interfaces 1.4.3 High-Speed Serial Interconnects 1.5. Standards Development Process
1 1 2 3 6 7 7 10 10 11 11 12 12 13
vni PARTI Optical Transport Network Infrastructure
15
CHAPTER 2 ARCHITECTURE OF TRANSPORT NETWORKS
17
2.1. Introduction. 2.2. Transport Functional Modeling 2.2.1 Basic Concepts 2.2.2 Functionality 2.2.3 Connections and Points 2.2.4 Connection Dimension Model. 2.2.5 Sublayers and Function Decomposition 2.2.6 Examples 2.2.7 Equipment Packaging 2.2.8 Application Examples 2.2.9 Equipment Control 2.2.10 Equipment Supervisory Process 2.2.11 Modeling Connectionless Layer Networks 2.2.12 Summary 2.3. Notes 2.4. References
17 18 20 29 31 32 35 36 39 40 50 53 60 61 61 62
CHAPTER 3 INTERFACES FOR OPTICAL TRANSPORT NETWORKS 3.1. Introduction. 3.2. OTN Standards 3.3. Standardized Interfaces 3.4. Forward Error Correction 3.4.1 Theoretical Description 3.4.2 Coding Gain. 3.5. Tandem Connection Monitoring 3.6. OTN Hierarchy Overview 3.7. OTN G.709 Frame Structure 3.8. G.709 Overhead Bytes: In-Depth Analysis and Processing 3.8.1 OPUk Overhead Bytes and Client Mapping Structure 3.8.2 Similarly Valued/Formatted Fields within G.709 Frame 3.8.3 ODUk Overhead and Processing 3.8.4 Tandem Connection Monitoring (TCM) 3.9. OTUk Overhead and Processing 3.9.1 Scrambling 3.9.2 Frame Alignment Overhead 3.9.3 Section Monitoring Byte Descriptions 3.9.4 General Communication Channel 0 (GCCO) 3.10. ODUk Multiplexing 3.10.1 Multiplexing Data Rates 3.10.2 4 XODUl to 0DU2 Multiplexing 3.10.3 0DU1/0DU2 to 0DU3 Multiplexing 3.10.4 Summary
63 63 64 66 67 68 70 73 76 79 81 82 88 90 95 98 99 100 101 104 104 105 107 112 117
Contents 3.11. References
ix 117
CHAPTER 4 MULTIPLEX STRUCTURES OF THE OPTICAL TRANSPORT NETWORK 4.1. Introduction 4.2. The Situation in the Previous Century 4,2.1. SDH structure details 4.3. The Evolution of the Bandwidth 4.4. New Clients 4.5. Virtual Concatenation 4.5.1. Differential Delay 4.5.2. Pay load Distribution and Reconstruction 4.5.3. Additional Benefits 4.5.4. Restrictions 4.5.5. VCATDetails 4.6. Link Capacity Adjustment Scheme (LCAS) 4.6.1. Link Capacity Increase 4.6.2. Link Capacity Decrease (Planned) 4.6.3. Temporary Link Capacity Decrease 4.6.4. LCAS Details 4.1. Advantages of Using VCAT LCAS and GFP 4.8. Implementers Guide for VCAT and LCAS 4.8.1. Detection of Differential Delay 4.8.2. Compensation of Differential Delay 4.8.3. Structure and Management of Differential Delay Buffers 4.8.4. Differential Delay Buffer Overview 4.8.5. Alignment within a VCG 4.8.6. Sizing the Delay Buffers 4.8.7. Processing Time 4.8.8. Controlling Distribution/Reconstruction Order 4.8.9. Member Status 4.9. References
119 119 120 120 127 130 131 131 133 136 136 137 140 140 140 141 141 144 144 144 145 146 147 148 149 149 150 151 152
CHAPTER 5 GENERIC FRAMING PROCEDURE (GFP) 5.1. Introduction 5.2. Background 5.2.1 Packet Transport on Public Networks 5.2.2 Other Traffic Adaptation Approaches 5.2.3 Other Design Considerations 5.3. Formats and Procedures 5.3.1 GFP Frame Formats 5.3.2 GFP Control Frames 5.3.3 Client-Independent Procedures 5.3.4 Client-Dependent Procedures 5.4. Implementation Considerations 5.4.1 Virtual Framer Management
153 153 155 155 156 157 15 8 159 164 164 166 171 171
5.4.2 Scrambler Options 5.5. Performance 5.5.1 Probability of GFP Frame Delineation Loss (FDL) 5.5.2 Probability of False Frame Synchronization (FFS) 5.5.3 Probability of Frame Unavailability (FUA) 5.5.4 Frame Acquisition Delay 5.5.5 Scrambler Resynchronization Delay 5.5.6 Link Efficiency 5.6. Applications 5.6.1 Ethernet Private Lines 5.6.2 Virtual Leased Lines 5.6.3 Packet Rings 5.7. Future Directions 5.8. References CHAPTER 6 SYNCHRONIZATION OF OPTICAL NETWORKS 6.1. The Field of Network Synchronization Engineering 6.LI Introduction 6.2. Background on Timing, Synchronization, and Jitter 6.2.1 Basics of Digital Transmission, Timing Jitter, and A lignment Jitter 6.2.2 Jitter Tolerance, Transfer, Generation, and Network Limit 6.2.3 Mapping and Multiplexing 6.2.4 Pointer Adjustments 6.2.5 Timing Signal Imperfections 6.2.6 Characterization of Timing Performance 6.2.7 Wander Network Limits and Wander Performance 6.3. Roadmap of Current ITU-T Recommendations on Timing, and Jitter, For OTN, SDH, and PDH 6.4. Timing and Jitter Requirements for SONET/SDH and OTN 6.4.1 SEC and ODC Frequency Accuracy, Clock Modes, Pull-in and Pull-out/Hold-in Ranges 6.4.2 STM-N and OTUk Jitter Network Limit and Tolerance, STM-N Regenerator and ODCr Jitter Generation and Transfer, and STM-N and OTUk Jitter Accumulation 6.4.3 Jitter and Wander Accumulation for PDH Clients of SDH Networks and SDH Clients of OTN 6.5. Reliable Distribution of Synchronization 6.5.1 The Need for Synchronization 6.5.2 Synchronization A reas 6.5.3 Reference Duplication and Reference Selection 6.5.4 Synchronization Status Messages 6.5.5 Satellite Timing 6.5.6 Synchronization Network Engineering 6.6. Conclusions and Closing Remarks 6.6.1 Conclusions 6.6.2 Closing Remarks
172 174 174 175 J 76 179 182 182 184 184 185 186 187 187
189 189 189 191 191 196 200 203 206 209 212 214 216 218
219 227 233 234 235 241 243 248 249 250 250 251
Contents 6.7. Notes 6.8. References
xi 252 254
CHAPTER 7 SYNCHRONIZATION ARCHITECTURES FOR SONET/SDH SYSTEMS AND NETWORKS 7.1. Synchronization Concepts 7.2. Timing Traceability 7.2.7 Source Traceability 7.3. Synchronization Distribution 7.4. Network Element (NE) Architecture 7.4.1 Timing Engine (TE) Functions 7.4.2 Timing Distributor (TD) Functions 7.4.3 Network Element System A rchitecture 7.4.4 Small Network Element A rchitecture 7.4.5 Medium Network Element A rchitecture 7.4.6 Large Network Element A rchitecture 7.5. External Timing Configurations 7.5.1 Direct-Source Timing Method. 7.5.2 Bridged-Source Timing Method 7.5.3 Line/External Timing Method 7.5.4 Mult Timing Method 7.6. Clock Backup Modes and Implications 7.7. Synchronization Guidelines 7.8. Notes 7.9. References
257 257 261 262 266 268 269 270 2 75 2 76 2 77 2 78 279 280 281 282 285 286 292 293 294
CHAPTER 8 NETWORK SURVIVABILITY 8.1. Introduction 8.2. Network Survivability Techniques 8.3. Survivability Offered by Protection 8.3.1 Network Objectives 8.3.2 Protection Switching Architectures 8.3.3 Protection Switching Parameters 8.3.4 Protection Switching Classes 8.3.5 Hold-off Timer 8.3.6 Protection Switching Trigger Criteria 8.3.7Null Signal 8.3.8 Automatic Protection Switching (APS) Protocol 8.3.9 Examples 8.3.10 Optical Transport Networks (OTN) Survivability 8.4. Survivability Offered by Restoration 8.4.1 Network Restoration Techniques 8.4.2 Restoration time 8.4.3 Interoperability 8.5. Link Capacity Adjustment Scheme (LCAS) 8.6. Multilayer Survivability
295 295 295 296 297 297 303 306 309 310 310 310 312 313 314 315 315 316 317 318
xu 8.7. References PART 2 Services Offered over Transport Networks CHAPTER 9 METRO ETHERNET OVERVIEW AND ARCHITECTURE 9.1. Metro Ethernet Demand and Requirements 9.1.1 Network Resiliency 9.1.2 Trajfic and Performance Management 9.1.3 Circuit Emulation Services 9.2. Metro Ethernet Forum Charter 9.3. Metro Ethernet Network (MEN) Architecture 9.3.1 MEN Reference Model 9.3.2 MEN Layer Network Model 9.3.3 MEN Reference Points 9.3.4 MEN Architectural Components 9.3.5 MEN Layer Relationship to the Architecture Model Components 9.4. References CHAPTER 10 ETHERNET SERVICES OVER METRO ETHERNET NETWORKS 10.1. Introduction 10.2. Services Model 10.2.1 Customer Edge View 10.2.2 User Network Interface 10.2.3 Service Frame 10.2.4 Ethernet Virtual Connection 10.2.5 Identifying an EVC at a UNI 10.3. Service Features 10.3.1 CE-VLAN ID Preservation 10.3.2 All-to-One Bundling Map 10.3.3 Service Multiplexing 10.3.4 Feature Constraints 10.3.5 E-Line and E-LAN Service 10.3.6 Class of Service 10.3.7 Bandwidth Profiles 10.3.8 Layer 2 Control Protocols 10.4. Conclusion and Future Work 10.5. Appendix A: Ethernet Basics 10.5.1 Ethernet Physical Layers 10.5.2 Ethernet Media Access Control Layer 10.5.3 Ethernet VLANs 10.6. Notes 10.7. References
319
321 323 323 323 324 324 325 326 326 327 329 334 337 341
343 343 343 344 344 345 346 348 349 350 350 352 355 356 356 359 365 367 367 368 368 370 371 372
Contents
xiii
CHAPTER 11 ETHERNET SERVICES OVER PUBLIC WAN 11.1. Introduction 77.7.7 Why Ethernet over the public WAN? 11.1.2 Organization of the chapter 11.1.3 Related standards activity 11.1.4 Definition of some technical terms in this chapter 11.2. Service Types and Characteristics 77.2.7 Ethernet connection (EC) attributes 11.2.2 Ethernet Private Line (EPL) service 11.2.3 Ethernet virtual private line service (EVPL) 11.2.4 Ethernet private LAN (EPLAN) service 11.2.5 Ethernet virtual private LAN service 11.3. Transport Network Models In Support of Ethernet Connectivity Services 11.4. Ethernet Client Interfaces 7 7. '^. 7 Multiplexed access 11.4.2 VLAN mapping. 11.4.3 Bundling 11.4.4 Bandwidth profile 11.4.5 Layer 2 Control Protocol processing 11.4.6 Summary of UNI Service Attributes for Different Services 11.5. Ethernet Transport Network To Network Interface (NNI) 11.6.0AM 11.7. Protection and Restoration 11.7.1 Service Protection or Restoration Provided by the Transport Network 11.7.2 Service Restoration at Layer 2 11.8. Conclusion 11.9. Notes 11.10. References
420 421 421 422 423
CHAPTER 12 ETHERNET SERVICES OVER MPLS NETWORKS 12.1. Virtual Private Networks 72.7.7 Traditional Layer 2 Virtual Private Networks 12.1.2 Classification of VPNs 12.1.3 Multiservice Converged Packet Switched Backbone \1.1. L2VPNS over MPLS Backbone 72.2.7 L2VPNs Architecture Generic Components 12.3. Metro Ethernet Services 72. J. 7 Ethernet Virtual Connection (EVC) 12.3.2 E-Line Service 12.3.3 E-LAN Service 12.4. Metro Ethernet Services Over MpPLS 72.^.7 Emulation of E-Line Services using VPWS
425 425 425 426 427 428 428 436 436 436 436 437 438
373 373 373 3 75 377 378 379 381 387 388 389 391 392 401 402 404 404 404 404 405 405 411 419
XIV
12.4.2 E-Line Service Emulation Walk-Through Example 12.4.3 Emulation ofE-LAN Services using VPLS 12.4.4 E-LAN Service Emulation Walk-Through Example 12.5. Importance of VPLS for Metro Ethernet Services 12.6. Summary 12.7. Appendix A: MPLS Basics 12.7.1 Forwarding Equivalence Class 12.7.2 Labels 12.7.3 Label Encoding 12.7.4 Label Switched Router (LSR) 12.7.5 Label Stack Operations — Imposition, Disposition, Swapping 12.7.6 MPLS Control Plane 12.7.7 MPLS Forwarding Plane 12.7.8 Label Switched Path (LSP) 12.7.9 Benefits of MPLS Technology 12.8. References
441 443 448 449 450 451 451 451 452 453 453 453 454 454 455 455
CHAPTER 13 METRO ETHERNET CIRCUIT EMULATION SERVICES 13.1. Metro Ethernet Circuit Emulation Services 13.1.1 Circuit Emulation Service Definition, 13.1.2 Circuit Emulation Service Framework, 13.2. References
457 457 457 466 496
CHAPTER 14 METRO ETHERNET NETWORK RESILIENCY AND TRAFFIC MANAGEMENT 14.1. Metro Ethernet Network Resiliency 14.1.1 Introduction 14.1.2 Protection Terminology 14.1.3 Discussion of Terminology 14.1.4 Protection Reference Model 14.1.5 Requirements for Ethernet Services protection mechanisms 14.1.6 Framework for Protection in the Metro Ethernet 14.2. Metro Ethernet Traffic and Performance Management 14.2.1 Ethernet Traffic Management Overview 14.3. References
497 497 497 499 504 505 516 521 524 524 526
CHAPTER 15 SONET SERVICES FOR STORAGE AREA NETWORKS
527
15.1. Data Growth 15.2. Storage Networking 15.3. Storage Area Networks 15.3.1 Factors Driving SAN Extension 15.3.2 Fibre Channel: The Storage Protocol of Choice 15.4. Distance Extension Requirements 15.5. Distance Extension Alternatives 75.5.7 Legacy Private Line
527 528 5 31 532 534 536 538 539
Contents 15.5.2 WDM, 15.5.3 Storage over IP 15.5.4 SONET/SDH 15.6. SONET - An Ideal Distance Extension Protocol 15,6,1 Making SONET Fit - The Role of Standards 15.7. Summary 15.8. References PART 3 Control and Management of Transport Networks CHAPTER 16 ARCHITECTING THE AUTOMATICALLY SWITCHED TRANSPORT NETWORK 16.1. Introduction 16.2. Network Requirements (G.807) 16.2.1 Architectural Context 16.2.2 Call and Connection Control 16.2.3 Business and Operational Aspects 16.2.4 Reference Points and Domains 16.2.5 Architecture Principles 16.2.6 Supporting Functions and Requirements 16.2.7 Signaling Communications Network Requirements 16.2.8 Support for Transport Network Survivability 16.3. Architecture (G.8080) 16.3.1 The Control Plane View of the Transport Network, 16.3.2 Identifying Components 16.3.3 General Component Properties and Special Components 16.3.4 Component Overview 16.3.5 Interlay er Modeling 16.3.6 Distribution models 16.3.7 An Example of Components in A ction, 16.3.8 Identifier Spaces 16.3.9 Restoration A rchitecture 16.4. Signaling Communications Network Architecture (G.7712) 16.4.1 Signaling Methods 16.4.2 Delivery of Control Plane Messages 16.4.3 DCN Topologies 16.4.4 DCN Reliability Considerations 16.4.5 DCN Security Considerations 16.5. Service Activation Process Elements 16.6. Discovery (G.7714) 16.6.1 Discovery and Connectivity Verification 16.6.2 Discovery A rchitecture 16.6.3 Types of Discovery 16.6.4 Discovery Considerations across Administrative Boundaries 16.7. Routing (G.7715 and G.7715.1) 16,7,1 Requirements
xv 539 540 541 542 544 547 548
549
551 551 553 554 555 559 562 564 567 570 571 571 576 578 580 580 583 585 586 588 593 595 596 597 599 602 603 603 604 605 606 607 611 611 611
XVI
16.72 A rchitecture 16.73 Hierarchy in Routing 16.7.4 Routing Information Exchange 16.8. Signaling (G.7713) 16.8.1 Call and Connection Management Operations 16.8.2 Basic Call and Connection Control Sequences 16.8.3 Signaling Attributes 16.8.4 Signaling Application Example 16.9. Control Plane Management 16.10. Protocol Analysis 16.10.1 Analysis Approach 16.10.2 Requirements Implications on Protocol Solutions 16.11. Methods and Protocols — Discovery 16.11.1 Layer Adjacency Discovery Methods 16.12. Methods and Protocols — Signaling 16.12.1 G. 7713.1 PNNI Signaling 16.12.2 G,7713,2 GMPLS RSVP-TE Signaling 16.12.3 G.7713.3 GMPLS CR-LDP 16.12.4 Interoperability and Interworking 16.13. METHODS AND PROTOCOLS — ROUTING 16.14. Signaling Communications Network — Mechanisms (G.7712) 16.15. Futures 16.16. Acknowledgements 16.17. References PART 4 Intra-Network Elements and Component-Centric Standards CHAPTER 17 INTRA-NETWORK ELEMENTS COMMUNICATION 17.1. Introduction 17.2. Requirement Placed on the Network Elements by the Network 17.3. Network Element Design and Interface Architecture 17.3.1 Packet Based Network Elements 17.3.2 TDM Based Network Elements 17.3.3 Hybrid (TDM + Cell/Packet Based) Network Element Architecture 17.4. 2.5 Gbits/s Systems 17.4.1 SPI-3 Signal Descriptions 17.5. 10 Gbits/s Systems 17.5.1 System Framer Interface-4 Phase 1 (SFI~4 Phase 1) 17.5.2 SPI-4 Phase 1 (Oc-192 System Packet Interface) 17.5.3 System Framer Interface-4 Phase 2 (SFI-4 Phase 2) 17.6. SPI-4 Phase 2 (Oc-192 System Packet Interface) 17.7. 40 Gbits/s Systems 17.7.1 Serdes Framer Interface-5 (Sfi-5) 17.7.2 SPI-5 (Oc-768 System Packet Interface) 17.7.3 TFI-5 (TDM Fabric to Framer Interface)
615 619 621 626 627 628 630 631 633 637 63 7 639 640 640 643 643 644 648 649 651 652 653 655 655
659
661 661 662 664 665 666 667 668 669 672 672 674 677 679 681 682 685 687
Contents 17.8. Acknowledgements 17.9. References
xvii 688 688
CHAPTER 18 ITU OPTICAL INTERFACE STANDARDS 18.1. Introduction 18.2. ITU Optical Interface Standards 18.2.1 Historical Perspective 18.2.2 Transverse versus Longitudinal Compatibility 18.2.3 Overview of Optical Fiber Types and Associated Recommendations 18.2.4 Overview of Optical Interface Recommendations 18.2.5 Application Code Terminology Related To Distance 18.2.6 Power Budget Design Considerations and Limitations 18.3. Optical Interface Implementations 18.3.1 General 18.3.2 140 Mbit/s - 2.5 Gbit/s Technology 18.3.3 10 Gbit/s Technology 18.3.4 40 Gbit/s Technology 18.4. Considerations on Optical Fault and Degradation Detection 18.4.1 General 18.4.2 Faults in Conventional Transmitters and Receivers 18.4.3 Faults in Optically Amplified Systems 18.5. Notes 18.6. Acknowledgments 18.7. References
703 706 709 710 712 712 713 721 729 729 729 729 731 732 733 733
CHAPTER 19 HIGH-SPEED SERIAL INTERCONNECT 19.1. Introduction 19.1.1 Chip-Chip Interconnect 19.1.2 Backplane Interconnect 19.2. High-Speed Interconnect System Architecture 19.2.1 Topologies 19.2.2 Printed Circuit Board (PCB) Interconnects 19.3. Compliance Test Methodology 19.3.1 Eye Mask 19.3.2 Jitter modeling conventions for high-speed interfaces 19.3.3 Bathtub curve analysis ofjitter 19.4. Interconnect Extension Using De-Emphasis and Equalization 19.4.1 De-emphasis at the Transmitter 19.4.2 Equalization at the Receiver 19.4.3 Usage Models 19.5. Standards-Based High-Speed Interconnect 19.5.1 OIF Sxl-5 19.5.2 OIF TFI-5 19.5.3 IEEE® 802.3ae™ Clause 47, XAUI 19.5.4 Backplane Ethernet
735 735 736 736 737 737 738 742 742 744 746 748 749 754 756 758 758 759 759 760
691 691 692 692 700
XVIU
19.5.5 Summary of Standards-Based High-Speed Interconnect 19.6. Higher and Higher Speeds 19.7. Summary 19.8. Notes 19.9. References
PARTS Standards Development Process
760 762 764 764 764
765
CHAPTER 20 STANDARDS DEVELOPMENT PROCESS 20.1. Introduction 20.2. The International Telecommunication Union (ITU) 20.2.1 Hierarchy 20.2.2 Membership 20.2.3 Standards Development 20.3. Technology-Specific Industry Forums 20.3.1 Message 20.3.2 What is involved? Election/hierarchy 20.3.3 The History behind Standards Groups: Why join? 20.3.4 Membership 20.3.5 Reality of human nature 20.3.6 Teamwork 20.4. Conclusion
767 767 768 768 772 772 776 776 777 779 780 781 782 783
INDEX
785
FOREWORD
Khurram Kazi SMSC
''O mankind! We have created you from a single (pair) of a male and female, and have made you into nations and tribes, so that you may know each other,..'' [Quran 49.13]. When one ponders over how we get to know each other; certain thoughts come to mind. As we venture outside our own region or domain, we tend to follow certain protocols that allow us to communicate with each other. Those protocols have diverse flavors; for example, the first thing we try is to communicate in a common language that both parties understand. If that fails, we use gestures or sign language or even resort to drawing pictures to get our message across. In short we find a common ground or similar footing which to build our communication platform on even though we may come from such diverse cultures and background. Just as we have diversity in mankind, we have disparate, ever-evolving communications networks. These networks are evolving towards providing seamless connectivity between different platforms and applications so that they cater to our insatiable need to communicate with each other in many different ways. Evolutionary technologies, including Dense Wavelength Division Multiplexing (DWDM), advances in optics/optoelectronics, highly versatile electronic integrated circuits, and control and management software have provided an excellent foundation for present-day networks to emerge into robust and scalable networks with ever-increasing intrinsic intelligence. These advances have been enabled by the relentless activities taking place within scores of technical standards committees, industry fora and consortia across the globe. In this comprehensive volume, we seek to give an overview of the converged multiservice optical transport networking development activities occurring within standards development
XX
organizations and industry fora including the International Telecommunication Union, ITU-T, Internet Engineering Task Force (IETF), Institute of Electrical and Electronics Engineers (IEEE), Metro Ethernet Forum (MEF) and Optical Internetworking Forum (OIF). Some of the issues these bodies are addressing are: • Multiservice and data-optimized SONET/SDH and OTN transport infrastructure • Ethernet and MPLS in converged transport networks spanning the enterprise, access, and core network realms • Flexible and efficient support for a diverse set of services over existing and emerging next-generation transport network architectures • Enhanced service provisioning, enabling more dynamic and responsive connection services, via automatically switched transport networks • Equipment functional block specifications that enable multivendor interoperability without constraining equipment implementation • Physical-layer specifications that enable multi-carrier network deployments and interconnection of equipments among multiple vendors • Network and equipment management specifications encompassing FCAPS (fault, connection, administration, performance, security management) that assure common behavior and minimize the need for human intervention, reducing operational and management expenses • Timing and synchronization in global transport networks • Backplane and component specifications encompassing optical, electrical, and mechanical characteristics that impact, e.g., optoelectronic modules. Very Large Scale Integrated devices, and backplanes utilized in existing and next-generation equipment
PREFACE
In the late 1980s and 1990s I was exposed to ANSI, ITU-T, IEEE and ATM forum standards and implementation agreements while developing ASICs and systems for Tl, SONET/SDH, ATM, and Ethernet applications. During the architecture, design, and verification, I had to go through the standards documents to ensure that my designs were standards compliant. While designing I was always on the lookout for a comprehensive source that could give me a broad prospective on all the relevant standards dealing with optical networking. Suffice it to say I did not find a single book that comprehensively covered the work being done at the major standards bodies and I ended up going through quite a few books, standards documents and technical papers. The development of the bigger picture proved to be quite useful in my design process as I started to understand which components my ASICs would be interfacing with, what systems and services features these ASICs were going to be providing etc. In short, I started to understand the design partitioning and hierarchical abstractions; from ASICs, standard VLSI products, and optoelectronic components to systems and networks architecture. With the desire to share these thoughts with the rest of the networking community, I embarked upon this project. With the help of Eve Varma, I was able to assemble a world-renowned team of over twentyfive leading contributors and editors of the standards from networking powerhouses such as Tellabs, PMC-Sierra, Nortel, Marconi, Lucent Technologies, Cisco, Ciena, British Telecom, Atrica, AMCC, and Agere Systems, as well as independent consultants, who have come together to make this work a reality. From our collective efforts we have put together Optical Networking Standards: A Comprehensive Guide, which provides a single-source reference work for the specifications of networks at all levels: from components (optics, optoelectronics, and VLSI
XXll
devices) through systems to global transport networks infrastructure, their management, and the services they offer. While going through the standards documents, especially the designers and implementers who ensure standards-compliant products should keep in mind that generally standards documents are not easy to read. There are several reasons behind the statement. Typically, the process of defining the requirements for specific sets of functionalities kickstarts the development of a particular standard or suite of standards. During the early phases of the development of standards technologies or services, the norm is to define a generic architecture. Details are subsequently added either to the same document or to different ones to get into specifics. Generally speaking, every effort is made to ensure that the standards are written in such a way that they are technology and implementation independent. This approach results in careful usage of the language that at times makes it difficult to read. From my personal experiences as a designer, I felt anguish while going through these documents. However, persistent reading made things clearer. One lesson that I learned is that one needs to spend the time going through the documents with patience and full concentration, along with having discussion with the colleagues, to fully appreciate the subtleties of the recommendations. It is always helpful if one develops some background information base prior to going through standards documents. Every effort is made in this book to give a reader some background information so that going through the respective recommendations is not as painful as when one starts to read them cold. The chapters are written such that they can be read as standalone chapters or can be combined to get a better understanding of the different aspects of optical networking standards. It should be kept in mind that the reader should always use the actual standards documents, implementation agreements, or RFCs as the definitive source of information.
Acknowledgements
I would like to thank ITU-T, Metro Ethernet Forum and Optical Internetworking Forum for allowing us to use appropriate information from their respective standards and the implementation agreements.
Contributing Authors
Ghani Abbas has spent over fifteen years in the SDH and Optical Networks business, initially with GPT and later with Marconi Communications, U.K. He is currently international standards manager in the Network Engineering and Technology department. He previously held various engineering development and management posts. He is currently the rapporteur for ITUT SGI5 Q9, which develops standards for transport equipment and network protection and restoration. He is an active member of OIF, ETSI, ITU SGI 3 and SGI5. Ghani received a B.Sc (Honour) degree from Manchester University in Electrical Engineering and a Ph.D degree in Electronics from Southampton University, U.K. P. Stephan Bedrosian is a Distinguished Member of Technical Staff in the Standards and Advanced Technology organization at Agere Systems. He has worked in the research, design, and development of synchronization systems, networks, and devices spanning the last two decades. At Bell Laboratories, his focus was on telecommunications synchronization systems, including the design and development of building integrated timing supplies (BITS). At Lucent Technologies, he was involved in the design and development of both telecom and datacom synchronizations systems and subsystems for use in SONET/SDH, xDSL and ATM networks. At Agere Systems, he is very involved with the development of computer timing devices as well as standardization of packet timing protocols. He has published several articles, including "Timing Synchronization Speeds Network Access," and holds several synchronization-related patents and patents pending. Mr. Bedrosian holds a Bachelor of Science in Electrical
XXVI
Engineering from Worcester Polytechnic Institute and a Master of Science in Electrical Engineering from Georgia Institute of Technology. Nan Chen Nan Chen is the Vice President of Marketing at Strix Systems (www.StrixSystems.com), a leading provider of mesh wireless Ethernet solutions enabling rapid networking without wires. Mr. Chen is also the President of the Metro Ethernet Forum (www.MetroEthernetForum.org), a worldwide standards organization for Carrier-class Ethernet networks and services. Before Strix, Nan Chen was the Vice President of Marketing at Atrica Inc. (www.Atrica.com), where he successfully drove Ethernet's metro vision in the industry and its wide adoption in carriers networks worldwide. Prior to joining Atrica, Mr. Chen was the Director of Product Management and Product Marketing at Force 10 Networks while serving as a founding member of the Board of Directors of the 10 Gigabit Ethernet Alliance (10 GEA). Mr. Chen also spent four years at Nortel/Bay Networks/SynOptics. While serving as a Director of Technology at Nortel Technology Center, Mr. Chen drove Nortel 10 Gigabit Ethernet strategy and served as a founding member of the IEEE 802.3ae Task Force for development of 10 Gigabit Ethernet standards. Mr. Chen holds two MS degrees from the University of Arizona and a B.S. degree from Beijing University, China where he also was a record holder in pole vault. Carmine Daloia (
[email protected]) is a senior communications engineer and consultant in Washington Group International. He holds an M.S. degree in Electrical Engineering from Columbia University and a B.S. degree in Electrical Engineering from The Cooper Union, New York. He has expertise on transport and data network architecture, planning, and design covering a wide range of technologies, including SONET/SDH, OTN, ATM, MPLS, and IP. Beginning in 1995, he worked at Telcordia Technologies as a senior communication engineer, where he was responsible for SONET and ATM network design and planning projects and led the development of various Generic Requirements (GR) documents, including the SONET UPSR, SONET BLSR, and DWDM OADM GRs. While at Telcordia he represented both Telcordia and the Regional Bell Operating Companies within national and global standards. He joined Lucent's Optical Networking Group in June 2000, where he continued his standards activities by contributing to the development of the OTN architecture, equipment, and protection specifications as well as ASON specifications, and was editor of G.7712 "Architecture and Specification of the Data Communications Network." He joined Metro Tech Consulting Services in September 2003, where he provided network planning consultation to New York City Transit
Contributing Authors
xxvii
(NYCT) for the future Second Avenue Subway line communications network. Mimi Dannhardt is a Consultant who received her M.S. degree in Electrical Engineering from Virgina Tech. In her career, she has designed numerous networking and telecommunications chips for ATM, SDH, PDH and Ethernet over Sonet applications. Tracy Dupree is a public relations professional who has worked in the telecom and networking industries for over a decade. Ms. Dupree operated her own consulting agency for several years, where she worked with the Metro Ethernet Forum, among other clients. She has been employed with a variety of communications companies including Tekelec, Nortel Networks and currently is employed at Alcatel. Geoffrey Garner received an S.B. degree in Physics from M.I.T. in 1976, S.M. degrees in Nuclear Engineering and Mechanical Engineering from M.I.T. in 1978, and a Ph.D. in Mechanical Engineering from M.I.T. in 1985. He is currently a consultant in telecommunications, specializing in network timing, jitter, and synchronization; network performance and quality of service; systems engineering; and standards development. Since 2003 he has worked on a variety of projects, including simulation of network level jitter and wander performance, development of a simulator for Optical Burst Switching Network performance, and development of new standards for carrying time-sensitive traffic over Residential Ethernet. Prior to his work as a consultant, he was a Distinguished Member of Technical Staff in the Transport Network Architecture Department of Lucent Technologies. Beginning in 1992, his work at AT&T and then Lucent included the development of international and national standards for jitter and synchronization performance and transmission error performance of OTN and SONET/SDH networks, and for Quality of Service of ATM networks. He was the Rapporteur of the Transmission Error Performance Question in ITU-T SG 13 from 2001 to 2004, and the Editor for the ITU-T Recommendation specifying jitter and wander in the Optical Transport Network (G.8251) in SG 15. He joined AT&T in 1985, went with Lucent Technologies upon its divestiture from AT&T in 1996, and became a consultant in 2003. Steven Scott Gorshe is a Principal Engineer with PMC-Sierra's Product Research Group. He received his B.S.E.E. (University of Idaho) and M.S.E.E. and Ph.D. (Oregon State University) in 1979, 1982, and 2002, respectively. He has been involved in applied research and the development
XXVlll
of transmission and access system architectures and ASICs since 1982, including over five years at GTE and over 12 years with NEC America, where he became Chief Architect for NEC Eluminant Technologies. His current work at PMC-Sierra involves technology development for applications-specific standard product ICs, including those for Ethernet WAN transport over telecommunications networks. Dr. Gorshe is a Senior Member of the IEEE and Co-Editor for the regular Broadband Access series and guest editor for multiple Feature Topics in the IEEE Communications Magazine. He has also been involved in telecommunications network standards continuously since 1985 and serves as Senior Editor for OPTSX (formerly TlXl, responsible for North American SONET and optical network interface standards); technical editor for multiple standards within the SONET series; and a technical editor for multiple ITU-T Recommendations including G.7041 (GFP), G.8011.1 (Ethernet Private Line Service), and G.7043 (Virtual Concatenation of PDH Signals). Areas in which he has made key contributions include architectures for multiple generations of SONET/SDH equipment and much of the transparent GFP protocol. He is a recipient of the Committee Tl Alvin Lai Outstanding Achievement Award for his standards contributions. He has 27 patents issued or pending and multiple published papers. Adam Healey is a Distinguished Member of Technical Staff at Agere Systems and is responsible for the definition of subsystems and components required for access and enterprise networks. Adam joined Lucent Microelectronics / Agere Systems in 2000. Prior to joining Agere Systems, he worked for seven years at the Interoperability Lab at University of New Hampshire where he developed many of the test procedures and systems used to verify interoperability, performance, and comphance to standards of 10, 100, and 1000 Mb/s electrical and optical links. Adam is a member of IEEE and contributes to the development of international standards as a member of the IEEE 802.3 working group. He currently serves as chair of the IEEE P802.3ap Backplane Ethernet Task Force. He received a B.S. and M.S. in Electrical Engineering from the University of New Hampshire. Huub van Helvoort is a Standards Consultant at Huawei Technologies Co., Ltd. In 1977 he received his MSEE degree at the Technical University in Eindhoven, the Netherlands. In his 26-year career he has collected extensive experience in public switching systems and ISDN, PDH, and SDH technology. He represents Huawei Technologies in the standards bodies: ITU-T (SG15) and ANSI (T1X1.5) and is the editor of several ITU-T recommendations. He is a senior member of the IEEE. He can be contacted at tel: +31 36 5315076; e-mail:
[email protected]
Contributing Authors
xxix
Enrique Hernandez-Valencia is a Consulting Member of the Technical Staff at Lucent Technologies' Bell Laboratories. He received his B.Sc. degree in Electrical Engineering from the Universidad Simon Bolivar, Caracas, Venezuela, and his M.Sc. and Ph.D. degrees in Electrical Engineering from the California Institute of Technology, Pasadena, California. He has over 15 years of experience in the design and development of systems architectures and protocols for high-speed communications networks. Dr. Hernandez-Valencia is a Bell Labs Fellow and a member of the Institute of Electrical and Electronics Engineers, Association for Computing Machinery, and Sigma Xi societies. Iftekhar Hussain is a technical leader with the Internet Technologies Division at Cisco Systems. For the past several years, Iftekhar has been involved in the design of high availability related aspects of IP/MPLS networks. He brings extensive industry experience to the subject of networking and telecommunication, including switching, traffic management, and voice delivery over packet switched networks. Dr. Hussain's current interests are in the area of IP/MPLS networks, network security, and mobile wireless architectures. He holds a Ph.D. degree in electrical and computer engineering from the University of California, Davis. Nevin Jones is a Consulting Member of Technical Staff (CMTS) with the Advanced Technology and Standards Development Group of Agere Systems (formerly Lucent Microelectronics). He has worked for approximately 20 years in the field of communications engineering at AT&T Bell Laboratories, Lucent Technologies and Agere Systems. He continues to work in a multidisciplinary communications engineering capacity and encompassing switching and transport networks modeling and planning, systems engineering, and software and hardware development of integrated circuits for PDH and SONET/SDH systems. His primary applied research activities currently include optical networking systems and architectures, client signal adaptation protocols, and integrated circuits physical-layer interface specifications for backplanes and chip-to-chip interconnects. He is an active contributor and member at ATIS OPTXS, PTSC, Optical Internetworking Forum (OIF), ITU-T SG-15 & SGI-3, IETF, and several other industry standards fora. He holds a B.S.E.E. (SUNY), MS (CUNY), and Ph.D. (CUNY). Khurram Kazi has over 19 years of industrial hands-on expertise in the computing, data and telecommunication networking field. He is a senior systems architect at SMSC working on trusted computing platforms. Prior to
XXX
SMSC, he concentrated on the architectural studies and designs of 40+-Gbps Next Generation Secure Transport Network, where he studyed the detailed trade-off analyses of crucial optical networking methodologies, devising new techniques, and recommending implementation methods within the intended architecture of the global network, including implementations of network elements and custom ASICs. Prior to this, he developed numerous ASICs for IP switching, SONET, Ethernet, ATM, and PDH applications. His extensive ASICs and systems work experience ranges from mid-size to venture-backed startup companies (Safenet-Mykotronx, General DataComm, TranSwitch, and Zagros Networks) and world-class research organizations like Bell Laboratories, Lucent Technologies. His work has resulted in over a dozen published papers and conference tutorials on optical components to ASICs to Optical Networks. Khurram received his B.S. from University of Bridgeport, M.S. and Ph.D. from the department of Electrical and Systems Engineering at the University of Connecticut. He can be reached at
[email protected]. Bob Klessig is a Director of Engineering at Cisco Systems. He is the Vice President of the Metro Ethernet Forum, where he is also the Co-Chair of the Technical Committee and a member of the Board of Directors. Before joining Cisco, Dr. Klessig was a founder of Telseon, an early competitive metro Ethernet service provider in North America. Before Telseon he was with 3Com, where he developed and helped execute the corporate ATM strategy. At 3Com he was the lead representative to the ATM Forum Technical Committee, where he focused on standards for data networking with ATM. He has held lead positions at Bellcore and Bell Laboratories. While at Bellcore he led the conception, design, and specification of Switched Multi-megabit Data Service (SMDS), the first high-speed metropolitan area data service. His Bellcore responsibilities also included leading RBOC participation and serving as Vice Chair of the IEEE 802.6 committee that wrote the IEEE Standard for Metropolitan Area Networks. Dr Klessig has a Ph.D. in Electrical Engineering and Computer Sciences from the University of California at Berkeley and is a co-author of the book SMDS: Wide Area Data Networking with Switched Multi-Megabit Data Service. Gert Manhoudt received a Masters Degree in Electrical Engineering from Delft University of Technology in 1986 in the area of Integrated Optics. He then worked for 17 years for Lucent Technologies and its predecessors and has a broad experience in optical transport networking as a hardware designer and systems engineer and in technical marketing. He has been active in the development and marketing of SDH/SONET equipment, specializing in optics, high-speed electronics, synchronization, and network performance aspects. He represented Lucent Technologies in ITU-T and
Contributing Authors
xxxi
ETSI in the synchronization working groups; in this capacity he was editor of two ETSI documents, namely, ETS 300462 parts 3 and 5. He has been one of the pioneers involved with the packet over SDH/SONET technology, as a responsible system engineer for the implementation of Ethernet transport and protocols over SDH/SONET networks. He has contributed to papers and studies on the evolution of packet-based transport in today's public networks. Since 2003 he worked as a network consultant for AimSys, www.aimsys.nl, a startup company that designs and manufactures equipment for metro optical networks. Alan McGuire is a Principal Engineer in the Network Technology Centre of BT. He leads a multidisciplinary team working on next generation transport networks. Alan is also active in the standards arena, where he has made numerous contributions and acted as editor of numerous ITU Recommendations concerned with network architecture, control and management, and optical networking. He graduated from the University of St. Andrews in 1987 with a first in Physics and received an M.Sc. in Medical Physics one year later from the University of Aberdeen. Alan is a member of the IEEE and the Institute of Physics and is a Chartered Physicist. George W. Newsome is currently a Systems Engineering Consultant whose primary work includes Control Plane management, architecture, and global standards. In previous assignments, George was a significant contributor to the ASON architecture standards, and has both created and managed the development of Network Element software. He has been involved with functional modeling since its inception in the ITU and has also worked on information modeling for network management. Mr. Newsome is a Chartered Engineer, Member of the lEE, and Senior Member of the IEEE, and holds a B.Sc. degree in Electrical Engineering from University College, London. Lyndon Ong is currently Director of Network Control Architecture in the CTO organization of Ciena Corporation, a supplier of intelligent optical and data transport equipment. Dr. Ong joined Ciena in 2001 after working at Nortel Networks, Bay Networks, and Bellcore/Telcordia. He received his doctoral degree in Electrical Engineering from Columbia University in 1991. He has had a long career in the area of control protocols, starting with the original team defining Signaling System 7 standards for North America, then working on ATM networking, IP QoS and transport protocols, and finally working on the optical control plane. Dr. Ong has chaired groups in ITU-T Standards, currently chairs the Signaling Transport WG in IETF, and is the editor of the OIF E-NNI Signaling Implementation Agreement.
XXXll
Richard Orgias currently works as a product marketing manager in the Broadband Networks business at Nortel. Mr. Orgias has worked in a number of different business units in various roles spanning operations, finance, and marketing since joining Nortel in 1992. Prior to his current role, in which he has marketing responsibility for Nortel's broadband access solutions, Mr. Orgias had responsibility for marketing Nortel's optical storage connectivity solutions, which included DWDM-based solutions as well as solutions based on SONET and Ethernet. Mr. Orgias received a B.Sc. degree from McMaster University in 1985 and also holds Master of Science and MBA degrees from McMaster. Mr. Orgias resides in Alpharetta, Georgia, with his wife and two children. Jonathan Sadler is a Staff Engineer in the Advanced Technologies group at Tellabs. With over 20 years of data communications experience as a protocol implementer, network element designer, carrier network operations manager, and carrier network planner, Jonathan brings a broad set of knowledge and experiences to the design and analysis of carrier network technology. Currently, he is involved in the development of technologies to provide the efficient transport of packet oriented services in Carrier Networks. He is the Chairman of the Optical Internetworking Fomm's Architecture and Signaling Working Group and an active participant in the IETF and ITU. Jonathan studied Computer Science at the University of Wisconsin - Madison. Stephen Shew is an Architect in the Optical Networks group at Nortel Networks. In his career at Nortel, Stephen has participated in the development and specification of distributed routing protocols, traffic engineering optimization tools, ATM PNNI, and MPLS signaling. His standards involvement has included the ATM Forum and IETF. He is currently participating in ITU-T Study Group 15 and OIF, and contributes to the architecture and protocols for the Automatic Switched Optical Network (ASON). Stephen received his Bachelor of Computer Science from Carleton University and M.Sc. from the University of Toronto. Peter J. J. Stassar holds a Masters Degree in Electrical Engineering from the Technical University in Eindhoven, the Netherlands. Between 1980 and 2003 he has been employed at Lucent Technologies Bell Labs, Hilversum, the Netherlands, lastly as senior technical consultant for optical technology and strategy in the R&D organization of Lucent's Optical Networking Group. He has well over 20 years of working experience in virtually all aspects of optical transmission, ranging from research to development, manufacturing, and deployment, in particular on SDH/SONET and FTTH
Contributing Authors
xxxiii
optical technologies. Since 1989 he played a key role in ITU SG15s activities on optical interface specifications. He is currently engaged as Product Manager for FTTH products at Genexis BV, Eindhoven, the Netherlands, and furthermore he is representing the interests of Finisar Corporation in ITU SO 15 on the field of optical interface technologies. Stephen J. Trowbridge received his Ph.D. in Computer Science from the University of Colorado at Boulder. He has worked for Bell Laboratories since 1977, and is currently a member of the Bell Laboratories Standards and Intellectual Property Department. He is a vice-chairman of ITU-T Study Group 15 (Optical and other Transport Network Infrastructures), where he chairs Working Party 3 of Study Group 15 (Optical transport network structure), which is responsible for standards related to PDH, SDH, and OTN Transport, Transport Equipment, Frame Formats, Network Synchronization, Equipment Management, and Automatically Switched Optical Networks (ASON). He chairs Working Party 3 of the ITU-T Telecommunication Standards Advisory Group (TSAG) on Electronic Working Methods and Publication Policy. He is also vice-chairman of the Optical Heirarchal Interfaces (OHI) subcommittee of the ATIS Optical Transport and Synchronization Committee (OPTXS). Eve L. Varma is a Technical Manager, Lucent Technologies, part of Lucent's Optical Networking Group. Ms. Varma has 26 years of research experience within the telecommunications industry. Her primary research areas are currently focused upon the automatically switched transport network (ASON/GMPLS) and converged multiservice transport networks. She is actively engaged in supporting the development of the associated specifications within global standards and industry fora spanning ITU-T, ATIS, and OIF. From 1995 to 1997, she led the team responsible for designing and prototyping a distributed optical network management system as part of the Multiwavelength Optical NETworking (MONET) Consortium (partially supported by DARPA). Previous research experience includes characterization, analysis, and development of transmission jitter requirements; systems engineering and standards for SDH/SONET and OTN transport and network management applications; as well as associated enabling technology and methodology development/assessment. Ms. Varma has been an active contributor to global standards since 1984 and has coauthored two books. Achieving Global Information Networking, Artech House (1999), and Jitter in Digital Transmission Systems, Artech House (1989). She holds an M.A. degree in Physics from the City University of New York. Ms. Varma is a 2004 Bell Labs fellow.
XXXIV
Tim Walker received his BS EE from Rensselaer Polytechnic Institute and his MS EE from University of Illinois at Urbana-Champaign. He is a member of IEEE, Tau Beta Pi, and Eta Kappa Nu. Mr. Walker is currently a Systems Engineer at Applied Micro Circuits Corporation (AMCC) in Andover, Massachusetts. Prior to AMCC he spent 16 years at Bell Labs/Lucent in various Hardware and Software design positions. Before that, he was an RF Engineer for four years at Teradyne. His main areas of expertise are OTN (G.709), SONET/SDH, and PON. He has presented numerous proposals to Standards bodies (ITU SG15 and TlXl),
Chapter 1 OVERVIEW
Khurram Kazi SMSC
The late 1990s heralded the need for optical transport networks to make a transition from catering to mostly voice traffic to converged voice and multiservice data-centric traffic. This transition led to several innovative standardized solutions, developed primarily within the International Telecommunication Union, (ITU-T) to develop and more efficiently leverage new and existing transport infrastructures to support the delivery of conventional and emerging services. Associated with this trend has been the cross-fertilization of concepts and technologies between traditionally data-oriented standards organizations, such as the Institute of Electrical and Electronics Engineers (IEEE) and Internet Engineering Task Force (IETF), and the ITU-T. Further coupled with this cross-fertilization has been an industry focus upon defining the multitude of services that such networks can offer, as evidenced by diligent work efforts in industry forums such as the Metro Ethernet Forum (MEF). Industry forums, including the MEF, Optical Internetworking Forum (OIF), and TeleManagement Forum (TMF), currently play important roles related to the realization and deployment of new technologies. Last but not least, due to the large number of component vendors whose standard products are used in the design of the network elements that provide these diverse set of services, the need for standardized backplane and component interfaces has never been greater. For example, the OIF and IEEE facilitate development and deployment of optical and electrical component/backplane technologies. The book is organized into five categories: 1. Optical Transport Network infrastructure 2. Services offered over Transport Networks
Chapter 1 3. Control and management of Transport Networks 4. Intra-Network Elements and component-centric standards 5. Standards development process The chapter distribution is illustrated in Figure 1-1.
f
Chapters * coverage of the Standards related to the respective categories Optical Transport Network Infrastructure: Chapters 2-8
Services offered over public and private Transport Networks: Chapters 9-15 L J
Control and Management of the Transport Networks: Chapter 16
Network Elements and components: Chapters 17-19
Standards development process: Chapter 20
\
/ Figure l-l. Categorical distribution of chapters
1.1.
OPTICAL TRANSPORT NETWORK INFRASTRUCTURE
1.1.1
Functional Modeling Specification Technique
Optical transport networks have evolved into intelligent and complex networks that provide global connectivity and increasingly diversified services. Such networks need to interoperate between numerous carriers and service providers to ensure seamless connectivity across the globe. This requirement leads to the daunting task of representing and specifying equipment and network functionality and behavior for transport networks in a coherent and implementation-independent manner. ITU-T took the lead in defining a functional modeling specification technique that can be used as the requirements' capture and analysis tool. Chapter 2 describes the fundamental concepts of the functional modeling approach used in multiservice optical transport networks. In reviewing the basics, it describes how the complexity of the network functions can be reduced by a "divide and conquer" approach using the concepts of layers and partitions. The simplicity and versatility of the functional modeling scheme is illustrated by demonstrating how a handful of symbols can be used to represent various networking functions, be they for the network level or for detailed descriptions of functions within the networking gear (even down to the ASIC level). To illustrate its usage, several examples are provided. For
Overview
3
example, functions like the mapping of a Plesiochronous Digital Hierarchy (PDH) client signal onto a Synchronous Optical Networks (SONETySynchronous Digital Hierarchy (SDH) server signal or Add Drop Multiplexing (ADM) equipment characteristics are shown in simple diagrams. Numerous other examples are given in illustrating the usage of the functional modeling techniques. The fundamental concepts developed in this chapter are also used in subsequent chapters where services, management, and control of the multiservice transport networks are described. For the digital design engineer, this platform can be compared to the US Department of Defense's Very High Speed Integrated Circuit (VHSIC) project that resulted in the development of VHSIC Hardware Description Language (VHDL). The intent in creating VHDL was to use it for writing the specifications of electronic devices and systems in a standard format. However, since then the scope of VHDL has expanded significantly, it is now used for designing, simulating and synthesis of Application Specific Integrated Circuits (ASICs). Similarly, the functional modeling techniques developed by ITU-T may be utilized in the design and development process of hardware and software for multiservice transport networks.
1.1.2
Multiservice Optical Transport Network Infrastructure
Since the wide deployment of the Internet, the transport networks that once catered to low bandwidth Time Division Multiplexing (TDM) centric services required evolutionary features to simultaneously support TDM and multiservice data-centric traffic (variable length packets or frames) that had "best effort delivery" or "time and jitter sensitive" characteristics. In the late 1990s, based upon the tremendous growth in data traffic, ITU-T experts determined that there was a need to come up with a new set of standards that was optimized for this ultra-high-growth and data-driven capacity demand. Based upon a set of requirements predicated upon carrier globalization, support for gigabit-level bandwidth granularity for scaling and managing multiterabit networks, and leverage of dense wavelength-division multiple (DWDM) technology, work started on a next-generation optical transport network. This infrastructure was intended to complement the SONET/SDH infrastructure, which was designed around a mux structure starting from VT1.5/VC12 that filled the payload with voice-centric 64 Kb/s traffic. Leveraging lessons learned in the establishment of SONET/SDH standards were also developed.
4 1.1.2.1
Chapter 1 Optical Transport Hierarchy
Chapter 3 describes in detail two key ITU-T standards, G.709 and G.798, that describe the implementation of Optical Transport Hierarchy (OTH). It goes through the rationales and requirements behind the development of a suite of standards. For example, it covers topics including its role as an efficient transport networking layer that supports very high rate services at 2.5, 10, and 40 Gb/s; strong Forward Error Correction (FEC); support of multiple nested and overlapping Tandem Connection Monitoring (TCM) layers; the management of frequency slots Optical Channels, (OChs); single or multiple A,s) instead of time slots (STS-ls, etc.). Furthermore, it extensively covers the details of the G.709 frame structure, how various client signals can be mapped onto it, and the maintenance and management of networked DWDM signals. Prior to the bursting of the telecom bubble, it was widely anticipated that the SONET/SDH signals would be become clients of the OTN. However, in the time frame during which the OTN standardization effort came to fmition, the market need for support of Gbitlevel networking did not materialize, and OTN deployment has been slower than anticipated. Currently, one of the major driving factors in its deployment is the availability of strong Forward Error Correction (FEC) functionality at high speeds. This factor will become even more pervasive as we see the deployment of 40 Gb/s line rates due to the power savings and lower Bit Error Rate (BER) features of the strong FEC. 1.1.2.2
Data and TDM-Friendly Next-Generation SONET/SDH
Leveraging the worldwide investment in SONET/SDH network infrastructure and equipment to support the delivery of new services continues to be an important consideration for carriers. The advantages of this leveraging include incurring only incremental investments for deploying new services and enabling generation of revenues from delivery of new services via standardized mappings into SONET/SDH (e.g., transport of non-SONET/SDH bit-rate payloads). As the saying goes, "Necessity is the mother of all inventions" and this truth led to simple and elegant solutions that changed the status of "legacy SONET/SDH" to "the next big thing." Recent ITU-T standards added the efficient multiservice data-centric transport capabilities to the Time Division Multiplexed voice-centric SONET/SDH networks without requiring a "forklift" upgrade. There were two main issues that needed to be addressed in achieving the desired features: 1) efficient utilization of bandwidth in transporting data traffic and 2) seamless encapsulation of any data centric traffic into SONET/SDH pipes.
Overview
5
Chapters 4 and 5 provide extensive details of the underlying technologies that breathed new life into SONET/SDH. From the networking point of view, a majority of the data-centric traffic across the globe originates in the form of Ethernet frames. Almost everyone who uses a networked computer knows something about Ethernet and feels comfortable with it. This situation led to the huge momentum behind the evolution of the Ethernet that resulted in its emerging from the enclosing scope of local area networks along with high-speed Internet, IP Virtual Path Networks, IP VPN, video distribution, and varied other services. This emergence created a requirement for the efficient transport of Ethernet services over the SONET/SDH and OTN transport network infrastructures. Chapter 4 describes the details of the ITU-T's work on Virtual Concatenation (VCAT) and the Link Capacity Adjustment Scheme (LCAS) to ensure that future networking needs are met by adding data-centric capabilities into the existing Time Division Multiplexed, TDM, transport structures. The concept behind VCAT is that the primitive payload containers that carry the client data can be virtually "glued" or concatenated to build the desired bandwidth pipe. This outcome is achieved by using pointers in the overhead field of the containers. Theoretically speaking, bandwidth can be added or deleted when desired with the lowest granularity of approximately 1.5 Mb/s (VT 1.5 or VC-11). However, there are no restrictions on changing the desired bandwidth by any container size (e.g., VT-x, VC-n, STS-n, or STM-n). The Link Capacity Adjustment Scheme defines the protocol of how bandwidth can dynamically be changed. In essence, one can say that VCAT in conjunction with LCAS allows networks operators to provide bandwidth on demand without disrupting service while effectively utilizing the SONET/SDH pipes. A nimble and efficient encapsulation protocol known as Generic Framing Procedure (GFP) that supports fixed and variable-length frames/packets over transport networks was also recently standardized within the ITU-T (post2000). Chapter 5 describes how diverse set of protocols can be efficiently mapped into a single transport protocol (GFP) without having to go through multiple layers of encapsulation before being transported over TDM-based core networks. As an example, native IP traffic, Ethernet, Fibre Channel, and Storage Area Networking (SAN) protocols, along with others, can be mapped into GFP with minimal overhead. In GFP there are three main modes of the client frames/packets adaptation: Frame Mapped, Transparent Mapped, and Asynchronous Transparent Mapped. The Frame Mapped adaptation mode is mostly utilized in encapsulating data traffic like Ethernet, IP, MPLS etc. In this mode, the entire client data frame/packet has to be buffered locally first, prior to the encapsulation. This process causes delay
6
Chapter 1
and may not be suitable for time-sensitive traffic. Transparent Mapped modes of the GFP provide low jitter and delay by encapsulating the client signal in fixed-length encoded GFP frames without having to wait for the entire client frame/packet to be buffered.
1.1.3
Global Optical Transport Network Timing
As the saying goes, "Timing is everything." This phrase is an exact fit for transport networking, since we know that timing and synchronization, among the network elements that make up the global transport network infrastructure play a very critical role in ensuring the proper operation and performance of these networks. Chapter 6 gives a comprehensive overview of the timing and synchronization techniques utilized within optical transport network infrastructures. It covers the fundamental concepts of jitter and wander and their relationship with network synchronization. These basic topics lead into the jitter, wander, and synchronization requirements for SONET/SDH networks and the differences between these requirements and those for OTN infrastructures. Moreover, topics including client signal mapping into SONET/SDH or OTN signals, and the accommodation of timing variation to ensure seamless operation, are extensively covered. The architectural solutions to accommodate effects due to jitter and wander within the components/blocks, may these be Clock and Data Recovery (CDR) or SONET/SDH or OTN framers, are provided. The fundamental concepts covered in Chapter 6 have to some extent a high degree of abstraction, since they are described in mathematical terms. To illustrate those points explicitly, it is prudent to follow the discussion with some technology specific concepts and examples. Chapter 7 complements the content of Chapter 6 by adressing the issues within the SONET/SDH systems and networks. It starts out by reviewing why synchronization is needed, how it is administered, and why tracing the timing source is important. Numerous methods of timing extraction mechanisms utilized by the network elements are shown. Chapter 6 also reviews the hierarchical timing distribution across the network. Several architectural examples of timing distribution within the various types of network elements are also presented. 1.1.3.1
Transport Network Survivability
A very important aspect of providing uninterrupted networking services is to ensure connectivity, while maintaining an acceptable level of service quality, under conditions of natural or man-made disasters. This topic is extensively addressed in Chapter 8 under the category of network
Overview
7
survivability. There are two predominant techniques utiHzed for achieving survivable networks: (1) network protection and (2) network restoration. Network protection imphes that there are dedicated resourses that have already been deployed as backups and that "take over" when the primary resources have failed. Network restoration refers to the use of backup resources from a pool of resources, as opposed to dedicated backup resources. Chapter 8 starts by providing the objectives of the protection schemes and covers the details of the major five protection architectures namely, (1) 1+1, (2) l:n, (3) m:n, (4) (1:1)", and (5) rings. Protection switching parameters, protection switching classes, protection switching trigger criteria, automatic protection switching protocol, and examples are discussed. Topics in survivability through restoration implemented by either preplanned or on-the-fly routing with centralized route-calculation techniques and distributed techniques are also covered.
1,2.
CARRIAGE OF SERVICES OVER TRANSPORT NETWORKS
The MEF, ITU-T, and IETF have taken the lead in defining technical specifications, standards and RFCs for extending the reach of Ethernet from its traditional LAN environment to metro- and wide-area networks. The intent of the effort by these standards and industry forums is to ensure that enhanced Ethernet-based services can be deployed reliably and are scalable at lower capital and operating expenses.
1.2.1
Ethernet Services Architecture and Definitions
In describing Ethernet services, it is prudent to describe reference models that can be used in defining terms and services. Chapter 9 lays the foundation for describing the reference models used in defining Ethernet services over metro area networks. These services are defined in such a way that they can be carried over any of the prevailing transport technologies like IEEE 802.3 PHY, IEEE 802.1 bridged networks, SDH VC-nA^C-n-Xc, SONET STS-n/STS-4n-Xc, ATM VC, OTN ODUk, PDH DSl/El, MPLS, dark fiber, etc., or possibly different future networks. 1.2.1.1
Ethernet Services over Metro Networks
Traditionally, services are defined in observable terms, with clear demarcation points between the subscriber and the service provider's
8
Chapter 1
equipment. The subscriber equipment is referred to as the Customer Edge (CE) at which observable service-level parameters are defined that become the basis for a Service Level Agreement (SLA) between the subscriber and the service provider. The physical demarcation point between the Service Provider and a single subscriber is termed a User to Network Interface {UNI) across which such SLAs are made. Beyond the UNI, the types of technology and the architecture inside the metro and wide-area networks are invisible to the subscriber. This transparency allows the services to be defined and observed from UNI to UNI. Moreover, the definition of the services allows the service providers to offer metro- and wide-area Ethernet services to over 100 million existing devices capable of using the services. Chapter 10 describes the service definitions and characteristics of Ethernet Virtual Connection (EVC) as defined by MEF. Details of point-topoint, point-to-multipoint, and multipoint-to-multipoint EVCs enabled by VLAN tags are provided, along with details of how these can be used in offering E-Line and E-LAN services. Traffic and performance management is an integral part of ensuring that the SLAs are met for such services. Traffic policing becomes essential in monitoring compliance with the SLAs and is based on parameters like Committed Information Rate (CIR), Committed Burst Rate Size (CBS), Excess Information Rate (EIR), and Coupling Flag (CF). The performance-monitoring parameters such as frame delay, frame delay variation, frame loss can be used in defining different classes of service. The means whereby these parameters are used in service delivery and performance monitoring/assurance are also covered. 1.2.1.2
Ethernet Services over Public Wide-Area Networks
In Chapter 11, we see that ITU-T has taken its lead from the work done by the MEF and extended the scope by defining the Ethernet services over public wide-area transport networks. ITU-T uses traffic management and performance parameters consistent and complementary with MEF in defining its services standards. In addition to the services description, it covers the transport network models that support the Ethernet connectivity. These include Ethernet private line services that leverage the existing connection-oriented, circuit-switched TDM networks. Several service scenarios that provide diverse sets of applications are provided. Chapter 11 goes on to describe the Ethernet-based user-to-network interfaces (UNIs) and Network-to-Network interfaces (NNIs). To facilitate reliable Ethernet services, Ethernet operations, administration and management, and survivability are also discussed. It should be noted that the ITU-T's datacentric transport technologies like Virtual Concatenation (VCAT), Generic Framing Procedure (GFP), and Link Capacity Adjustment Scheme, (LCAS)
Overview
9
have become the enablers of Ethernet services over SONET/SDH and OTN networks. 1.2.1.3
Ethernet Services over MPLS Networks
Leveraging the extensive work done on Muhiprotocol Label Switching (MPLS) networks, along with its expertise on packet-based networks, IETF has developed RFCs that are the enablers for providing Ethernet services by using MPLS networks. Chapter 12 details Ethernet services over MPLS networks. It starts by describing the fundamental concepts and architectures of layer 2 virtual private networks (VPN) over an MPLS backbone as it sets the stage for the Ethernet services over MPLS. E-Line and E-LAN functions are subsequently discussed, as well as how virtual private wire and virtual private LAN services can be offered. A walk-through example of E-Line emulation over MPLS is given to clarify the concepts and outline the steps needed to provide the service. 1.2.1.4
Circuit Emulation over Metro Ethernet
Traditional voice and other TDM services have been the core of our communication needs and over time have been offered on different technology platforms as these evolve. Ethernet as a service platform has gained significant momentum over the past couple of years. Carriers worldwide are deploying metro Ethernet networks to cater to the everdemanding business customers' requirements for faster and cheaper data and voice transport. Moreover, these carriers are finding increased demand for their existing lucrative TDM traffic, may it be PBX trunks or private line services. Chapter 13 describes the details of MEF's recommendations regarding Ethernet circuit emulation services, such as N x 64 kbit/s, Tl, El, T3, E3, OC-3, and OC-12, across a Metropolitan Ethernet Network. It provides numerous service reference models that can be used in implementing TDM services. 1.2.1.5
Metro Ethernet Network Resiliency
We are seeing tremendous momentum behind Ethernet services that are being offered by carriers to businesses. The availability of the network services to the business is very critical, as is the Quality of Service (QoS). The combination of service availability and QoS has become a crucial aspect of the Service Level Agreement (SLA) between the user and the service provider. Service availability is tightly coupled with network resiliency.
10
Chapter 1
Chapter 14 covers the metro Ethernet network resiliency as recommended by the MEF. Topics like protection types, failure types, resource selection, and event timings are covered. Chapter 14 also discusses timing issues along with service-level specification commitments. Protection reference models are described such that consistent descriptions of protection capabilities are applied on these services across various transmission technologies, network topologies, and policies. This approach enables the description of protection of services in the ETH layer (Ethernet Line Service, Ethernet LAN service, etc.). The models lend themselves nicely to a definition of requirements for the Ethernet services protection mechanism. This definition subsequently provides the framework for the protection in the metro Ethernet and the implementation of the protection schemes.
1.2.2
Storage Area Services over SONET
The data-centric upgrades to SONET/SDH networks and key developments in the Fibre Channel, the widely used protocol for storage-area networks, have provided a strong platform for connecting storage-area networks across town or across the globe. Chapter 15 provides an overview of the trends for tremendous data growth across the globe and the need to access stored data at remote sites. It briefly covers the basics of the storagearea networks and the different technologies that are used therein. It reviews the stringent requirements that storage networks place on wide-area networks when they are connected over wider distances, as well as the need for network resiliency, scalability, and performance. It gives various options for how storage data can be moved across various sites using WDM, storage over IP, Fibre Channel over IP, and SONET/SDH. It makes a strong case for why SONET/SDH (using VCAT and GFP), along with the work being done at the ANSI Technical Committee on Fibre Channel Back Bone (FC-BB-3), provides a good, strong platform for extending the storage-area networks across long distances.
1.3.
CONTROL AND MANAGEMENT OF OPTICAL TRANSPORT NETWORKS
Chapter 16 provides a treatise on the activities of the ITU-T in specifying the architecture and requirements for automatic switched optical networks (ASON), concentrating on defining the critical functions and the required components for the optical control plane. Since the 2000 time frame, the ITU-T has been engaged in developing a suite of Recommendations
Overview
11
encompassing control plane architecture, auto-discovery, signaling, routing, and control plane management requirements and specifications related to optical transport networks. The optical control plane enables rapid establishment of connection services across heterogeneous networks by supporting intelligence that enables transport networks to be dynamically managed for traffic engineering and bandwidth-on-demand applications, particularly in the areas of QoS, connection management, and traffic restoration after network failures. To achieve this goal, it was considered essential to first establish the general networking requirements related to the service control and management functions that are essential elements of the solution. These requirements included the fundamental concepts of separation of call and connection control; relationships among the management, control, and transport planes; and establishment of a flexible control-plane component architecture. One of the goals of this work is to assure that the optical control plane may be gracefully deployed into existing and new transport network infrastructures, and varied network management environments, in an evolutionary manner. This process would allow networks operators to harness the advances in optical transport networking technologies while ensuring that the existing deployed infrastructure is not rendered obsolete. The ultimate goal of the ASON suite of Recommendations is to enable automated cross-domain connection management supporting multivendor and multicarrier interoperability on a global scale. The methodology for the development of the ASON suite of Recommendations involves a foundation of protocol-neutral specifications that facilitate common requirements and behaviors for various technology and implementation options, hence lending themselves nicely to future growth. Chapter 16 also describes the relationships among the various standards and industry forums involved in the development of control plane specifications (IETF, ITU-T, OIF, ATM Forum), including utilization of associated protocol specifications (GMPLS, PNNI).
1.4.
INTRA-NETWORK ELEMENT COMMUNICATION AND COMPONENTCENTRIC STANDARDS
1.4.1
Intra-Network Element Communication
As we have seen in the earlier chapters, we review diverse sets of networking functions and multinetworking services that are presently being
12
Chapter 1
offered or will be offered in the near future. In building the networks and offering such services, numerous different types of network elements are used. Functions within network elements are primarily implemented on optoelectronic modules and ASICs. Chapter 17 starts with architectural examples of packet- and TDM-based network elements that can be used in providing multiservices. Architectural blocks described in the network element architectures can be mapped to ASICs or standard VLSI offered by semiconductor firms or developed in-house by the system vendor. OIF developed a number of implementation agreements that allowed the ASICs or standard VLSI products from different firms to communicate and interoperate with each other. Agreements such as serializer/deserializer (SERDES) to Framer interface or system packet interface operating at different rates are covered.
1.4.2
Optical Interfaces
Chapter 18 covers the diverse sets of topics on the optical interface standards developed by ITU-T. It starts by giving a history and rationale behind the evolution of optical interface standards relating to PDH, SONET/SDH, DWDM, OTN, CWDM, and all optical networks (AON). This chapter covers the general concepts and reference models, along with illustrative examples, so that the reader can subsequently get further details from the relevant standards documents. This approach was deemed necessary due to the large number of standards and the intricate details each respective standard provides. Chapter 18 gives an overview of optical fiber types and optical interface recommendations. It reviews the power budget design considerations and the limitations to overcome worst-case scenarios. Uses of certain coding schemes to achieve the required bit error rates are also covered. Subsequently, examples of discrete and integrated optoelectronic solutions related to operating speeds of 140 Mb/s - 10 Gb/s are highlighted. Finally, the chapter covers the illusive topic (from the standardization point of view) of faults and degradation detection in the optical transmitters, detectors, and amplifiers.
1.4.3
High-speed serial interconnects
In highly integrated network elements with ever-increasing port speeds and port card densities, the pressure is on to reduce printed circuit board (PCB) traces, layer count, and routing complexity. This situation has lead to the usage of serial interconnects using serializer/deserializer devices commonly know as SERDES. The high-speed interconnects operating at
Overview
13
Gb/s rates pose some interesting challenges. Chapter 19 discusses the highspeed serial interconnects that are used in the communication between devices within the same card along with card-to-card communication using the backplane architecture. It reviews the architectural considerations and signal-integrity design challenges. The discussion on topologies and the effects of material loss, layer connection, and the environment lead into the topic of de-emphasis and equalization—a powerful method in achieving highly reliable and robust chip-to-chip interconnects solution. The chapter subsequently gives an overview of the work of OIF, IEEE, and PCI Industrial Computer Manufacturers Group (PICMG) on interconnect standards. Finally, it considers some challenges and possible solutions for 6 Gb/s and 10 Gb/s applications that are anticipated in the near term.
1.5.
STANDARDS DEVELOPMENT PROCESS
The standards development process within the networking field has a long progressive history where innovative technologies are shaped in providing value-added products and services. Over the course of many decades, numerous standards organizations and fora have been working diligently to provide innovative standards and recommendations that have shaped the ever-evolving networking field. To appreciate this work, one needs to understand the behind-the-scene dynamics of what makes these standards bodies and fora "tick." Chapter 20 provides a snapshot of the practices and procedures that are used within ITU-T in developing global networking standards. In this chapter, practices in ITU-T and fora such as MEF and OIF are discussed. It is interesting to note that how the cultures in these organizations develop based on the policies and practices. For example, in the ITU-T, the approval of Recommendations requires unanimous agreements. This very fundamental premise developed a culture at ITU-T where civility and a spirit of cooperation prevail even against a background of fierce competition in the marketplace. However, within industry fora, the culture is quite different, since majority vote (of a certain percentage) governs who "wins." This chapter highlights some interesting insights into the whole recommendations/standards development process that are useful in understanding the "systems." It also highlights the "behind the scenes" hard work of personnel who make such venues possible so that we can appreciate their efforts in making such meetings and gatherings successful.
This page intentionally blank
PARTI Optical Transport Network Infrastructure
This page intentionally blank
Chapter 2 ARCHITECTURE OF TRANSPORT NETWORKS The Functional Modeling Approach^ Eve Varma* and Carmine Daloia** "^Lucent Technologies, "^"^Washington Group International
2.1.
INTRODUCTION
Transport networking has steadily grown more complex as a consequence of more sophisticated customer needs, the convergence of data and transport networking, and conditions imposed by external market and regulatory forces. In the evolution of embedded core transport infrastructures or in building new core transport networks, efficient cost-effective transport capacity expansion, network reliability, flexible and dynamic bandwidth management, and quality-assured service management are of paramount importance to service providers. Given the wide range of technology choices, there is a trend for networks to employ heterogeneous technology equipment. Whereas in the past, transport networks only supported plesiochronous digital hierarchy (PDH) equipment, current networks may utilize equipment employing various technologies, including SONET/SDH, DWDM/OTN, IP/MPLS, Ethernet, and ATM. Current technologies and hierarchies are being designed to facilitate interoperation of equipment produced by different manufacturers, a process that further widens the competitive aspects of equipment purchase. Equipment suppliers may support a number of operators, possibly within a single nation, and may be presented with a number of different equipment specifications. At best, this situation leads to duplication of effort, with several, often very comprehensive, specifications relating to the same piece of equipment. In many cases, especially in a period of standards evolution, the specifications each require slightly different functionality, which may reduce competition and increase the price an operator must pay.
18
Chapter 2
From an operator's perspective, the necessity to fully specify particular equipment in order to avoid confusion and misinterpretation by a number of different manufacturers has led to increased specification complexity, which can make it difficult to judge among competing suppliers. Adoption of a common methodology for describing such equipment is therefore intended to simplify the specification process, to prevent misunderstanding, and to ensure fair competition. It should also present a set of common basic equipment requirements, facilitating inter-operation of multivendor equipment and driving down costs to both the operator and the end user [1]. Spurred by the above factors, motivation arose for establishment of standardized model-based approaches to • Enable description of the generic characteristics of networks, using a common language, at a level that can transcend technology and physical architecture choices; • Provide a view of functions or entities that may be distributed among many types of equipment; and • Concurrently specify transport and management functionality. Accomplishing the above allows us to • Design and plan networks prior to investments, including selection of the most appropriate types of equipment, to support telecommunications services; and • Facilitate development of new transport services and their associated management. As discussed above, the transport network is a large, complex network with various components, and a network model with well-defined functional entities is essential for its design and management. Within this chapter, we introduce and describe transport functional modeling standards, which provide the foundation for equipment control and management.
2.2.
TRANSPORT FUNCTIONAL MODELING
Transport functional modeling can be thought of as a requirements capture and analysis tool. Its objective is to describe the information transfer capability of transport networks in a manner that is independent of networking technology and to provide a set of "tools" for describing, in a common, consistent manner, the technology-specific transport functionality contained within a complex network. It enables • A flexible description of transport network and equipment functional architectures; • A means to identify functional similarities and differences in heterogeneous technology architectures;
A rch itecture of Transport Networks
19
•
A means to derive equipment functional architectures that are traceable to and reflective of the transport network requirements; and • Formation of the basis for a rigorous and consistent relationship between these functional architectures and their associated management specifications. ITU-T Recommendation G.805 [2] was the first transport functional modeling specification developed, and was specifically designed to address the connection-related characteristics of transport networks and equipment. It has been used to provide the methodology and basic concepts that are the foundation for other ITU-T Recommendations for technology-specific network architectures, including: • Synchronous Digital Hierarchy - G.803 [3], the functional architecture of SDH networks - G.783 [4], the functional architecture of SDH equipment - G.841 [5], the SDH network protection functional architecture - G.842 [6], SDH protection architecture interworking • Optical Transport Networking - G.872 [7], the functional architecture of Optical Transport Networks (OTN) - G.798 [8], the functional architecture of OTN equipment - G.873.1 [9], OTN Linear protection The above specifications address connection-oriented networks. In connection-oriented networks, a connection must be set up within the data plane by either the management plane or the control plane prior to the transfer of information across the network. The connection setup process includes the routing process, which determines an appropriate path through the network, and a resource allocation process, which assigns network resources along the calculated path to support the connection. The focus of this chapter is on connection-oriented networks. In addition to connection-oriented networks, connectionless networks are also being deployed in service provider networks. In connectionless networks, data grams are transferred through the network without any prior negotiation of routes or resources. The data gram itself contains sufficient address information for network nodes to route the data gram from its source to its destination. As connectionless networks such as IP and Ethernet have become more heavily deployed within service provider networks in conjunction with the increase in IP and Ethernet service offerings, service providers and equipment suppliers within the ITU-T saw a need to develop a functional modeling specification, namely. Recommendation G.809 [10], designed to address connectionless networks much in the same way Recommendation
20
Chapter 2
G.805 addressed connection-oriented networks. It has been used to provide the methodology and basic concepts that are the foundation for other ITU-T Recommendations for technology-specific connectionless network architectures, including G.8010 [11], the functional architecture of Ethernet networks,
2.2.1
Basic Concepts
The G.805-based modeling approach has allowed us to analyze the transport network and to identify generic functionality that is independent of implementation technology. This outcome has provided a means to describe network functionality in an abstract way in terms of a small number of architectural components, which include topological components, transport entities, transport processing functions, and reference points. These are typically defined by the function they perform in terms of transformations applied to the signal or by the relationships they describe between other architectural components. In general, these functions act on a signal presented at one or more inputs and present processed signals (i.e., transformed signals) at one or more outputs, and are defined and characterized by the information processed between their inputs and outputs. Architectural components may also be associated together in particular ways to form the equipment from which real networks are constmcted. Patterns and structure in the network can be rapidly obscured in a cloud of complex relationships. From a connection perspective, two separate concepts are involved: topology and function. The topology of a network is essentially the set of relationships between nodes (which will later be seen as subnetworks), and it defines the available connectivity. Application of this concept simplifies network description by keeping logical connections distinct from their actual routing in the network and the resources that physically support them. Thus, the logical pattern of interconnection of elements in the network is established without concern for the associated signal processing functions, and this outcome allows an operator to easily establish connections as required. On the other hand, the concept of function refers to how signals are transformed during their passage through the network versus how elements of the network are interconnected. Recommendation G.805 provides elements that support the modeling of both topological and functional concepts. Within the topology domain, the two fundamental concepts that relate to the organization of the network are layering 2inA partitioning.
A rch itecture of Transport Networks 2.2.1.1
21
Layering
We have already introduced the concept of topology, which allows us to separate logical connections from the physical routes and resources used in their carriage. This logical separation is well represented by the client/server paradigm, where the client refers to the signal being carried and the server refers to the entity providing its carriage; i.e., client signals are transported by servers. To utilize this paradigm, we consider the client and server to be two layers, where the client layer is supported by the server layer. The client/server paradigm is recursive, in that any particular server layer could itself be considered a client of another server layer. If we elaborate this paradigm, a network can be represented in terms of a stack of client/server relationships (a stack of layers). It's also useful to note that server layers are relatively more permanent than their clients. This outcome follows from the observation that a server connection must exist both before and after a client connection carried by that server. Layering therefore enables decomposition of a transport network into a number of independent transport layer networks, and this independence provides the required separation between its logical topology and physical routes and resources. In particular, the process for setting up connections becomes layer independent. Network management may also be simplified because each layer's properties can be handled in the same way (e.g., each layer can be assigned a quality of service, monitored for its performance independent of the other layers, and assigned an identification to help in fault isolation). A layer is defined (characterized) in terms of its set of signal properties, which form what is called the characteristic information of the layer (e.g., 2.048 Mb/s and its format). These properties are chosen in such a way that any access points having the same characteristic information can be interconnected. This term emphasizes the abstract properties of the stream in order to avoid the connotations of a physical signal in a medium, though the properties most often chosen tend to be related to the way a particular stream is represented, e.g., the rate and format at which information is transported. Conventionally, lower-order client layer networks use transport services provided by underlying higher-order server layer networks. The notion of higher- and lower-order layers follows the assumption that a server has a higher capacity than its client does, but this terminology is sometimes confusing because higher order layers are conventionally drawn at the bottom of the page. The complete set of access points in the layer that can be associated for the purpose of transferring information defines the boundary of a layer network.
22
Chapter 2
For purposes of clarification, we provide an example of layering utilizing PDH (e.g., a DS3 client) and SONET/SDH. The SONET [12] and SDH [13] standards define a hierarchy of signal layer networks (see Figure 2-1), as do the PDH and FDM standards that preceded them. Each layer network requires the services of a higher-order layer network to perform the required transport functions. We will discuss the exact location of the layer boundaries in more detail later in this chapter. For this example, we describe the primary signal layer networks below: • The logical client signal layer represents the logical DS3 signal, i.e., the DS3 signal rate and format, irrespective of physical media characteristics (e.g., line coding, etc.). • The logical SONET STS-1 / SDH VC-3 path layer network deals with the transport of the DS3 client signal (which may be considered as a "service"). The main function of the path layer network is to provide endto-end supervision capabilities for the signal, which traverses a series of SDH Multiplex Sections. Additionally, the layer maps its client into the format required by the SONET Line layer network, on whose services it relies. • The logical SONET Line or SDH Multiplex Section layer network deals with the reliable transport of path layer network payload and its overhead across the physical medium. The main functions of this layer network are to provide alignment (e.g., frequency or phase) and multiplexing for the path layer network. It relies on the services provided by the SONET Section/SDH Regenerator Section layer network. • The logical SONET Section/SDH Regenerator Section layer network deals with the transport of an STS-N/STM-N frame across the physical medium, and uses the services of the physical layer network to form the physical transport. Functions in this layer network include framing, scrambling, section error monitoring, etc. • The Physical Media Layer network (photonic or electrical), identified as either the STM-N Optical Section (OSn) or STM-1 Electrical Section (ESI), deals with the transport of bits across the physical medium. For example, in the case of photonic media, issues dealt with at this layer network might include optical pulse shape, power levels, and wavelength. This layer is required whenever equipment is to be represented; i.e., physical equipment description is incomplete without provision of physical interfaces. Thus, using the client/server model recursively, a logical DS3 signal would act as the client layer network while being transported by a server logical STS-1 A^C-3 path layer network, the logical STS-1/VC-3 path layer network would be the client layer network to the server logical SONET Line/SDH Multiplex Section layer network, etc.
23
Architecture of Transport Networks Lower-Order Layers ."f Logical DS3 Signal
DS3 payload mapping
Logical Line Layer
STS-1 Path overhead insertion
Alignment of STS-1 Path payload
Logical DS3 Signal
Logical STS Path Layer
DS? payload mapping
Mapping and multiplexing into logical STS-N frame
Logical Section Layer
Line overhead generation
Section overhead generation for STS-N frame
Physical Media Layer
Higher-Order layers
Logical DS3 Client Layer
Conversion into OC-N Physical Interlace
Logical Multiplex Section Layer
Logical DS3 Client Layer
VC-3 Path overhead insertion
Alignment of VC-3 Path payload
Logical Regenerator Section Laver Physical Media Laver
Logical VC-3 Path Layer
Mapping and multiplexing into logical STM-N frame
Line overhead generation
Section overhead generation for STM-N frame
Conversion into STM-N Physical Intertace
5
Figure 2-1. SONET/SDH signal hierarchy examples: DS3 client carried on an OC-N/STM-N signal
The architecture of the Optical Transport Network (OTN), specified within G.872, has layering characteristics analogous to those for SDH. This similarity should not be surprising, as the OTN was similarly developed to provide fully featured transport networking functionality optimized for highcapacity path networking in a multidomain environment. The defined OTN layers are • Optical Channel (OCh) layer that supports end-to-end networking of optical channels for transparently conveying digital client information • Optical Multiplex Section (OMS) layer that provides functionality for networking of a multiwavelength optical signal • Optical Transmission Section (OTS) layer that provides functionality for transmission of optical signals on optical media Recommendation G.872 specifies maintenance requirements for each of the defined OTN layers listed above. During the development of G.709 [14], it was realized that only digital techniques were available to meet the continuity, connectivity, and signal-quality supervision requirements specified in G.872 for the OCh layer. The use of digital techniques within the OTN was not considered to be a serious limitation for the following reasons: • The scope of G.872 is limited to the support of digital client signals.
24
Chapter 2
•
Due to limitations in the current optical technology, it is not possible to build a worldwide optical network (i.e., 3R regeneration of the optical channel is required after a certain distance). • 3R regeneration will be used at domain boundaries to decouple the domains with respect to optical signal impairments. Therefore, G.709 specifies an implementation of the OCh utilizing a digital framed signal with digital overhead. The use of a digital framed signal to implement the OCh allowed for the use of Forward Error Correction to enhance performance within the OTN. Recommendation G.709 therefore defines two additional digital layer networks, the Optical Channel Data Unit (ODU) layer network, and the Optical Channel Transport Unit (OTU) layer network. Characteristics of the OTN will be elaborated in Chapter 3. 2.2.1.2
Partitioning
As discussed earlier, the concept of layering helps us manage the complexity created by the presence of different types of characteristic information in current networks, which utilize multiple technologies supporting a wide range of bandwidths. However, even within a single layer, complexity is introduced by the presence of many different network nodes and the connections between them. In order to manage this complexity, we introduce the partitioning concept, which also uses the principle of recursion to tailor the amount of detail that needs to be understood at any particular time according to the need of the viewer. Partitioning refers to the division of layer networks into separate subnetworks that are interconnected by links representing the available transport capacity between them. The role of the subnetwork is to describe flexibility of connection, with no notion of distance being traversed, where traversal of distance is the role of the link. Subnetworks may be delimited according to a wide range of criteria, including those related to network infrastructure, network services, administrative and/or management responsibility, or even geography. Just as a layer network is bounded by access points that can be associated with each other, a subnetwork is bounded by ports that can be associated with each other. (It is important to note that while an access point can only be associated with one layer network, a port may be a member of one or more subnetworks.) Just as layers enable the management of each layer to be similar, so does partitioning allow the management of each partition to be similar. If we consider that a layer network is actually the largest possible subnetwork bounded by access points, it should not be surprising that subnetworks themselves can also be recursively partitioned into sets of still
Architecture of Transport Networks
25
smaller subnetworks and interconnecting links until the last level of recursion is reached (i.e., a fabric in an equipment). Figure 2-2 below illustrates recursive partitioning of a layer network, focusing upon illustrating the principle of partitioning versus the reasons for creating each partition. As each level of partition is created, it is important to understand that the original set of ports around the largest subnetwork neither increases nor decreases in number. The inner subnetworks are intentionally drawn touching the outer subnetworks to indicate that the ports are members of all the touching subnetworks. As more partitions are created, the inner links that become exposed have their own ports on the inner subnetworks. An interesting concept is that at any particular level of partitioning, subnetworks can be considered as a graph whose vertices are the subnetworks and whose edges are the links. In this view, subnetworks provide for flexible connectivity, while links bridge physical distance.
Figure 2-2. Recursive partitioning of a layer network
As might be expected, the rationale for employing a recursive description of subnetworks also applies to links. Recalling that a link can represent available transport capacity between a pair of subnetworks, link connections have been defined within G.805 as representing the smallest granularity capacity (supported on a server layer) that can be allocated on a link. Thus, a link may be considered as composed of (partitioned into) a bundle of link connections. However, the concept of link partitioning can be further
Chapter 2
26
extended; specifically, we can consider partitioning a link into a set of links of equivalent aggregate capacity (illustrated in Figure 2-3 below).
Links with capacities of x1, x2, x3... respectivelvj
Link with a capacity of y (y>x1+x2+x3...)
Figure 2-3. Partitioning a link into a set of links
This type of link partitioning allows us to assign server capacity to several links, rather than to just one. It thus allows us to assign server capacity to several subnetworks, which is necessary for modeling the sharing of a common server layer by several networks. This link-partitioning concept is particularly relevant to the modeling of variable capacity technology networks. From a terminology perspective, links that have been partitioned into a bundle of smaller links in parallel may be considered as compound and component links, respectively (Figure 2-4).
Figure 2-4. Parallel partitioning of link into links
Links may also be serially partitioned into an arrangement of linksubnetwork-link, illustrated in Figure 2-5; such links may be designated as serial compound and component links, respectively.
11
Architecture of Transport Networks
Serial-compound link
Figure 2-5. Serial partitioning of a link
The concepts of layering and partitioning are brought together in Figure 2-6, which illustrates a "vertical" arrangement of the layering example described earlier. As illustrated, each layer may be thought of in terms of a layer network, which can be "horizontally" partitioned into subnetworks to reflect infrastructure or equipment organization, such as self-healing rings, or to reflect convenient management or administrative boundaries. As mentioned earlier, the same network can be partitioned differently for different purposes. For example, the partitioning for connection management of various services, network administration, and maintenance may all be different.
Logical DS3 Client Layer Logical f STS-1 \^ Path Layer
tvc-3 y Logical (
Line
)^ Layer
^' Multiplex Section ^ Logical I
Section
| Layer
Regen, Section Physical Media Layer Layering View
Partitioning View
Figure 2-6. Illustration of layering and partitioning
We will close this section with an example of how partitioning enables a network management application to abstract the topology of a layer network (Figure 2-7), which is particularly relevant to the connection management
28
Chapter 2
domain. This builds off the example provided in Figure 2-2, and provides a more detailed view of the final stage of partitioning of this particular network to set up a connection from access point A to access point B.
Network connection Layer network
A-
-^B
I, Link
Access point
Subnetwork
Figure 2-7. Enabling abstraction of the topology of a layer network
The topology of the layer network is modeled as an interconnected set of links and subnetworks. The connection management domain utilizes the abstraction of the layer network topology to determine the appropriate set of links and subnetworks required to support a connection between two points. Once the set of links and subnetworks is selected, the transport resources (i.e., the link connections and subnetwork connections) are reserved to support the connection. This description of connection management is applicable to a single layer network, and the processes described are applied one layer at a time, which correctly models the connection management in real networks. As discussed previously, the complexity due to the presence of different types of characteristic information and technologies in current networks is managed using the concept of layering. Real networks are therefore modeled as multiple layer networks, each layer network defined by its characteristic information. The general process for setting up connections is similar for each layer network. As noted before, server layers are relatively more permanent than their clients, since a server connection must exist both before and after a client connection carried by that server. As a consequence, the connection management process must be completed first within the server layer to ensure that a server layer trail exists and is ready to support the client layer connection. The creation of the server layer trail results in new transport
Architecture of Transport Networks
29
resources becoming available within the client layer to support the client layer connection requests.
2.2.2
Functionality
While we have discussed the network topology dimension, and seen how complexity can be reduced by introducing the concepts of layers and partitions, we have not yet said anything about the functionality needed to actually transport a signal across a network. We have shown how network layers represent client/server relationships; we may consider the functionality involved in transporting signals to be the implementation of these client/server relationships. This functionality is provided by the same three transport processing functions in each layer, namely, adaptation, termination, and connection functions. The fundamental components of transport processing functionally are known as "atomic" or "elementary" functions, and are related to a single layer of the signal hierarchy (or layer network). We will later see that "atomic" does not mean that the function could not be further decomposed, but that we choose not to decompose the function at this particular time (e.g., it is not necessary from the particular layer perspective). There are, however, rules for composition (and decomposition) of atomic functions. Transport processing functions have been identified and grouped into classes corresponding to adaptation and termination. As signal transport is directional, these functions have a source, which originates the signal, and a sink, which receives the signal. Source functions apply a transformation to the signal, and sink functions remove that transformation. Source and sink functions thus occur in pairs within a layer, and are bounded by ports, which represent the function inputs and outputs. These ports are actually the same ports that we have described as bounding subnetworks and link connections in our partitioning topology model. Transport processing functions are described in more detail below: • Adaptation Function: An atomic function that passes a collection of information between layer networks by changing the way in which the collection of information is represented into a form that is suitable for the server layer. The adaptation source function is responsible for several key processes: - Client encoding: The adaptation source adapts a data stream to the server characteristics. - Client labeling: The adaptation source "labels" each client so that the corresponding adaptation sink can correctly identify it. This process enables clients to be multiplexed; however, the means by which this is done is very technology specific.
Chapter 2
30
-
Client alignment: Adaptation sources align the client signal with capacity in the server layer, while adaptation sinks remove the effects of alignment. While the actual process is technology dependent, in time division multiplexed (TDM) systems the buffering of the signal is commonly required. • Trail Termination Function: An atomic function within a layer network where information concerning the integrity and supervision of adapted information may be generated and added or extracted and analyzed. While this function's full title is trail termination function, a common abbreviation is just termination function. The termination source is concerned with transforming signals so that they can be monitored for signal quality. This frequently involves the addition of components to the signal for the purposes of monitoring, frequently called overhead. The termination sink monitors the signal quality and removes any overhead. It is this overhead removal function that gives the function its name, i.e., overhead termination. Overhead can be provided via insertion of additional capacity or, alternatively, via usage of already available, but unused, capacity. While not a transport processing function, there is a third function in common use, known as the connection function. • Connection Function: An atomic function within a layer, which, if connectivity exists, relays a collection of items of information between groups of atomic functions. It does not modify the members of this collection of items of information, although it may terminate any switching protocol information and act upon it. Any connectivity restrictions between inputs and outputs are defined. We note that the connection function is actually the same topological component as the subnetwork and has the same properties. These atomic functions are represented using a set of symbols, shown in Figure 2-8, which constitute part of a shorthand diagrammatic notation that will be used for specification purposes. The intent is to simplify technical descriptions via a common set of symbols and naming conventions.
Connection Function Adaptation Function
°
Ports
Figure 2-8. Graphical representation of "atomic" functions
Architecture of Transport Networks
2.23
31
Connections and Points
In Section 2.2.2 we have seen that network layer functionality, including the client/server relationship, may be described in terms of a set of elementary functions. The client/server relationship itself is most precisely defined as the association between layer networks that is performed by an adaptation function. In fact, these elementary functions can be connected to describe the complete layer behavior, with associated rules describing allowable combinations. Functions are interconnected by considering their ports to be bound together, where a binding between two ports is called a reference point (or just point). This convention makes it possible to illustrate relationships between functions without having to explicitly cite which port is involved. Subnetworks allow flexible bindings between their ports, and the binding of two such ports is called a subnetwork connection. The most commonly used bindings and reference points are described below and are illustrated in Figure 2-9:
Sink
Source k / Layer Y/Z Adaptation Source
Layer Y/Z \ Adaptation Sinl<
AP 8)
AP Layer Y Termination Source
TCP
SNC
CP
Layer Y Termination Sinl<
CP Layer Y Connection
Layer Y Connection
Layer XA' Adaptation Source
SNC
TCP
Layer XA' Adaptation Sink
AP
AP
Figure 2-9. Illustration of various bindings and reference points
Access Point: Binding of an adaptation source output port to a termination source input port, or an adaptation sink input port to a termination sink output port, is called an access point (AP). This binding is never flexible and can therefore never be partitioned, so it is of relatively little interest. Access points are frequently omitted in functional model diagrams. (An access group is defined as a group of co-located access points, together with their associated trail termination functions.)
32
Chapter 2
•
Any binding involving a termination source output port or sink input port is called a termination connection point (TCP). The termination source output port may be bound to an adaptation source input port or a connection function input port. The termination sink input port may be bound to an adaptation sink output port or a connection function output port. • Any binding of an adaptation source input port to a connection function output port, or of an adaptation sink output port to a connection function input port, is called a connection point (CP). The preceding discussions imply that layers have no "thickness" and are simply planes representing the location of all the connection points in the particular layer. Adaptation and termination functions are located between layers, with inputs and outputs in different layers. These "vertical" relationships are usually statically configured, while the "horizontal" relationships are usually more dynamic. While this view leads to the least ambiguity in models, layers are conventionally considered to have thickness, and the adaptation and termination functions are assigned to either the client or the server layer (Figure 2-10). This convention has more to do with establishing who is responsible for what than with creating good modeling constructs.
p.p
Client Layer Network
Server Layer Network
Figure 2-JO. Allocation of atomic functions to network layers
2.2.4
Connection Dimension Model
We have already introduced the concept of a connection as representing an entity that transports information transparently without any integrity control. Several kinds of connections may be identified, depending on the layer and partition traversed by the connection. Some of these we have
A rch itecture of Transport Networks
33
informally introduced, or inferred, earlier in the chapter. They are depicted in Figure 2-11 and more formally defined below: • Trail: We have seen that access points bound a layer network. These access points are bound to the input and output ports of trail termination functions. This association between connection termination points is called a trail, and it provides an end-to-end connection that offers an automatic means to check the quality of the transport. • Network Connection: A network connection represents an association between output and input ports of trail termination functions that transfers information across a layer network without ensuring its integrity. From our earlier discussion of partitioning layer networks, a network connection is composed of contiguous subnetwork connections and/or link connections. • Link: A link represents the capacity between two subnetworks, two access groups, or one subnetwork and one access group. The granularity of this capacity depends on the implementation technology. Links are both providers and consumers of capacity. A link can be decomposed into several links of lower capacity, each serving different subnetworks or capacity consumers. • Link Connection - A link connection transfers information transparently across a link and is delimited by ports that represent the fixed relation between the ends of the link. These ports are the connection ports associated with an adaptation function. • Subnetwork Connection: A transport entity that transfers information across a subnetwork. It is formed by the flexible association of ports on the boundary of the subnetwork. This definition is more specific than that in the G.805 definition, which defines a subnetwork connection as an association between reference points. (The fixed bindings characteristic of trail connection points and connection points may also be thought of as subnetwork connections, which are very often called degenerate subnetwork connections. The subnetwork, or reference point, containing such a connection is very often called a degenerate subnetwork.) In summary, a trail may convey information for several clients of a layer network through the application of multiplexing and transcoding capabilities at the layer network boundary. Existence of a trail in one layer provides any client in that layer with a potential for information transfer between the access points characterizing the extremities of that trail. The client/server relationship, more precisely defined as the association between layer networks that is performed by an adaptation function, allows the link connection in the client layer network to be transported over a trail in the server layer network. The usage of the bandwidth contained in a link is flexible, even if its route may be fixed. Except for the case of definite stable
Chapter 2
34
capacity between points that characterize the cable infrastructure, transport services usually involve temporary associations between points. Thus, to allow transport resource reuse, a network needs flexibility (reflected in the subnetwork concept). The potential for flexibility across a subnetwork is governed by an associated policy. The subnetworks give the flexibility, and links give the fixed transport capabilities between subnetworks. Again, as noted earlier, when we refer to fixed infrastructure, we do not mean that this infrastructure is inflexible; rather, we mean that such possible flexibility is not exercised during the time of the connection we are considering. Links do not change during the time it takes to set up a network connection; neither do the allocated link connections during the duration of the network connection of which they are a part. In general, the higher the order of the link, the more fixed the link tends to be (and vice versa). The usage of the above terminology, and associated relationships, is illustrated in Figure 2-11 below, which shows all the relationships (no other arrangements are possible) between ports, reference points, and connections. These restrictions effectively specify a description language.
Client Layer Network
AP CP SNC TCP
Adaptation Function Access Point Connection Point Subnetworit Connection Termination Connection Point Trail Termination Function
Figure 2-11. Illustration of terminology and relationships
Figure 2-11 uses the above concepts to show how a client layer trail may be transported by a server layer signal. Here, the client trail is first terminated, then transported through a subnetwork via a subnetwork connection, and adapted for transport across a server layer trail involving server layer subnetwork and link connections. This model allows us to characterize network functionality in a technology-independent manner. In Section 2.2.5, we will provide some examples illustrating application of
Architecture of Transport Networks these principles to specific technologies.
2.2.5
networking examples involving
35 various
Sublayers and Function Decomposition
The functions described so far are considered to be atomic at the current level of interest. As with the topological concepts we are now familiar with, these atomic functions can be decomposed to reveal internal detail when it is necessary. Conversely, more detailed layers can be collapsed to reduce the level of visible detail. The goal, as with the topology models, is to reduce the number of items being dealt with at a given level of interest. Expansion of the adaptation function or termination function (see Figure 2-12) may expand a layer to show more detail. Expansion of the adaptation function allows more detailed specification of the adaptation necessary to create the server layer characteristic information, while expansion of the termination function allows more detailed specification of the termination of the server layer. These techniques have been used to specify greater levels of detail in equipment, new monitoring arrangements in existing layers, and fault recovery arrangements for existing layers, as well as completely new server networks. For completeness. Figure 2-12 also depicts the expansion of the connection point, though this is simply the inclusion of additional resources in the connection.
Figure 2-12. Expansion of layers
The converse of expanding layers is, of course, collapsing layers (see Figure 2-13). Layers are often collapsed when there are no flexibility points between them and it is not necessary to fully understand the details of every layer. This is most often done in equipment, though it is possible to collapse layers simply to reduce the amount of detail in a drawing.
Chapter 2
36 When constrained to be in same component
for useful layers internal to a piece of equipment
Figure 2-13. Simplification vs.flexibility:collapsing layers
2.2.6
Examples
Let us first consider how we would model the transport of a PDH DS-3 client signal onto an STM-N server signal (Figure 2-14). Here, the logical DS3 client signal is adapted for transport onto a VC-3 trail via the VC-3/DS3 adaptation function, the VC-3 path overhead is provided by the VC-3 trail termination function, and the VC-3 client signal is then adapted for transport on a Multiplex Section trail (frequency or phase alignment and multiplexing) via the Multiplex Section adaptation function. Finally, the STM-N Regenerator Section overhead is provided by the Regenerator Section termination function. We note also that it is possible to stop the recursive descent through client/server associations at any arbitrary point. This makes it possible to separate the concerns of the different layer networks, enabling focus on the layer network(s) of interest for any particular purpose. For example. Figure 2-15 only describes associations from the DS3 client through the VC-3 trail and network connections, whereas Figure 2-14 shows the remainder of the recursion to the section layers in this example scenario. The technology and distribution-independent aspects of the functional modeling approach provide a highly flexible tool to accommodate mixed technologies and various possible functional distributions.
Architecture of Transport Networks DS3 Client Signal
37 DS3 Client Signal
DS3 Path Link Connection
—oVC-3/DS3 Adaptation AP
—e
VC-3 Trail
VC-3 Trail Termination \ [ F
yC.:3 Network Conn^^^^^^^^^^^ VC-3 LC .<;;;^^C7SNC^ N C ^ ..^
TCP
•4)
yC:3 LC.
cpSTM-N MS/VC-3 Adaptation
STM-N -MS Trail. _
AP STM-N MS Trail Termination
Y
TCP
(J>
STM-N ...MSNC...
Y
STM-N RS/MS Adaptation AP
STT/
STM-N MSNC
•0
STM-N RS Trail
RS Trail.
STM-N RS Trail Termination X T T > / \TT/ Y STM-N TCP 0 •^•5..NC...,
STM-N RSNC
Figure 2-14. DS3 client conveyed on an SDH VC-3/STM-N server signal
DS3 Path Link Connection
VC-3/DS3 Adaptation
DS3 aient Signal
—^
VC-3 Trail AP
VC-3 Termination TCP
VC-3 Network Connection
(J) TCP
Figure 2-15. DS3 client conveyed on an SDH VC-3 server signal
We will next examine how to model the carriage of an STM-N client signal on an OTM-n.m server signal (Figure 2-16). The STM-N client signal is treated as a constant bit rate (CBR) signal within a certain bit rate range. Here, the logical CBR client signal is adapted for transport onto an ODUkP trail via the ODUkP/CBRx-a adaptation function. The CBR client signal may be either asynchronously or synchronously mapped into the ODUkP server signal. In this example, an asynchronous mapping is supported. The ODUkP path overhead is provided by the ODUkP trail termination function. The ODUkP client signal is then adapted into an OTUk server trail via the OTUk/ODUk adaptation function (synchronous mapping of the ODUk frame signal into the OTUk frame signal). The OTUk section overhead is provided by the OTUk trail termination function. The OTUk client signal is then adapted into an OCh server trail (forward error correction, scrambling, and clock recovery) via the OCh/OTUk adaptation function. The adapted
Chapter 2
38
signal is then conditioned for transport across the optical medium, and the OCh path nonassociated overhead is provided by the OCh trail termination function. The OCh signal is adapted into an OMS server trail (wavelength assignment and wavelength division multiplexing) via the OMS/OCh adaptation function. The OMS nonassociated overhead is provided by the OMS trail termination function. The OMS signal is adapted into an OTS server trail via the OTS/OMS adaptation function. The OTS nonassociated overhead is provided by the OTS trail termination function. The OTS trail termination function also maps the logical OTM Overhead Signal supporting the nonassociated overhead into the Optical Supervisory Channel and combines the OSC with the OTS payload signal to form the OTSn characteristic information.
CBRx Client Signal ODUkP/CBRx-a Adaptation
CBRx Client Signal
CBRx Path Link Connection
o\
—e
A PPiiH-I'^y
AP ODUkP Trail Termination
ODUk Network Connection
\rp/ TCP
\n/
m
OTUk/ODUk Adaptation
OTUk Trail Termination TCP OCh/OTUk Adaptation OCh Trail Termination TCP OMS/OCh Adaptation
OMS Trail Termination TCP OTS/OMS Adaptation
OTS Trail Termination
Vt^
T
VA-7 OTMmn \ n /
..°I?££ I
\ n /
I--
Figure 2-16. STM-N client conveyed on an OTM-n.m server signal
A rch itecture of Transport Networks
2.2.7
39
Equipment Packaging
We have seen that the topological model of layers and partitions, as well as the interlayer functions, do not specify the packaging of functions into telecommunications equipment. Equipment packaging is the realm where layers, partitions, and functions all come together. We have already seen how partitions can be forced by some physical boundary. Equipment provides such a boundary; therefore, equipment content is either driven by partitioning decisions or certain partitioning decisions are forced by equipment content decisions. Unlike the network model, which can support logical reference points at any layer, equipment is obviously constrained to provide only physical interfaces. Equipment therefore encapsulates some common element of the layer, partition, and functional models. It is clear that larger partitions, which are of interest from a network-level perspective, are not usually wholly contained in equipment. However, as we have discussed earlier, all partitions are bounded by ports and, since adaptation and termination functions are present only in source/sink pairs, it is clear that any network layer can usually have only one end terminating in any particular equipment. Layer functions also present ports to both client and server layers. Thus, the modeling component that is common from both a network- and equipment-level perspective is the port. The intersection of the network partition and network layers inside an individual equipment takes place at these ports; i.e., the equipment encapsulates the ports of a partition and one end of one or more layers. (As a corollary, layers and partitions that are fully contained in an individual equipment are internal matters and are generally not of interest to a network. When the equipment allows some flexibility of internal connections, as is generally the case in current equipment, the equipment may be considered to contain an internal flexible subnetwork, which is defined by the ports available for connection (represented as logical resources). Because equipment only provides physical interfaces, all reference points are located inside equipment and are therefore inaccessible. This property allows the functional description of the equipment to be independent of the implementation chosen. Returning to our example of a DS3 client conveyed on an SDH STM-N signal, we see that Figure 2-14 describes the complete set of functional associations between the client DS3 signal and the logical STM-N signal without ever once referring to any physical equipment. Figure 2-17 below shows a possible equipment functional partitioning, i.e., a typical organization of functions into equipment, to support transport of the DS3 client signal across an STM-N transport network. Specifically, what is shown is a DS3 connection supported by a VC-3 trail that is terminated by STM-N multiplexers and traverses an intervening cross-connect system with
Chapter 2
40
STM-N interfaces and an internal VC-3 matrix. Due to the restriction that equipment can only present physical interfaces, we first complete the model by adding DS3 physical interfaces and ensuring that the STM-N section layers are physical layers. DS3 Client Signal
DS3 Client Signal
""^ \^
\
V
<
/ /
u^>.^ pain Linx L oiinecnon
< ,
K
1.,, v '
y
VC-3SN(: VC-3 LC
VC-3 LC
.. <" CP
Y TCP ( )
^
^^•^
\ — f
rcp ( )
STM-N RS Trail
\
A /
A
X
XX
STM-N Physical Media Frail
\
AP f
\
'(A
A /
AP (D
AP a)
\
A
•
4
APX
TCP d)
•
AP
STM-N
/
\
1
AP (i
\
i
T
STM-N RS NC
AP (D
^ S
)
STM-N RS Trail
/
AP (4 STM-N RS NC
STM-N MS NC 4
Y
X
STM-N MS Trail
TCP (p \
)
C P
T T
STM-N MS NC
4 ........ •
^
. . 1.
STM-N MS Trail
AP (p
)
r~^ i!7^ ^ r \ \T77 L r\ 7 \
V(:-3 Trail
\ T T /
TCP (
>
r
11
)
1
r
Figure 2-17. Possible equipment functional packaging
2.2.8
Application Examples
SONET/SDH technology has introduced new network topologies within the transport domain, one of the most notable being the ring. A ring is composed of a set of equipment called add/drop multiplexers (ADMs) that are intercormected in a loop configuration. These ADMs allow traffic to enter and to exit the ring. The main advantage of the ring topology is survivability: the ring offers traffic two different ways of passing from its ingress to its egress (see Figure 2-18).
Architecture of Transport Networks
41 Example: traffic egress A-Bf^
S-A
ADM Example: traffic ingress
^
^ . /
fiber I / '
\ A ^
^ ^AOMI ,
A0M
0
A
••
\flber2
/
J
ADM: Add-Drop Multipl ex
Figure 2-18. Example of ring architecture
Specifically, the ring's provision of diversely routed fibers allows traffic to utilize an alternative route when its usual route is no longer working. The allocation of preassigned capacity between nodes, in conjunction with automatic protection switching facilities, enables protection of traffic upon detection of a failure. SONET/SDH ring protection architectures significantly improve transport service quality by reducing unavailable time. Ring protection schemes can provide both trail and subnetwork connection protection. The simplest scheme is 1+1 (one working and one protection transport entity) subnetwork connection protection (SNCP) on a physical ring. As illustrated in Figure 2-19 [16], the input traffic in this case is broadcast over two routes (one being the normal working route and the second one being the protection route). As an example, consider the failurefree state for a path from node B to node A. In this case, node B bridges a SONET/SDH path layer signal destined for node A onto both working and protection routes around the ring (fibers 1 and 2, respectively). At node A, the signals from both of these routes are continuously monitored for path layer defects, and the better-quality signal is selected.
A-B B-A Selector—>
Bridge
-^^
fiber 1 / /
1*f \
\
••*"*v. Working
Protection\
^
I!A*\^ Sdector \
Bridge
^
\flber2
Figure 2-19. l-fl SNCP in a physical ring — failure-free state
Chapter 2
42
In the event of a failure, a selector switches to the standby route at the ring output if the active route fails. This process is illustrated in Figure 2-20 [16] for the case of a failure between nodes B and A [16].
Bridge
^
Selector — •
...^^ ^"•^v^
fiber 1
i Workirrg
Failure State
-^-'V*^
Bndge
Protection \
l>ov
\
C
B-AI Sel»ctor
\fiber2
Figure 2-20. 1+1 SNCP in a physical ring — failure state
Obviously, the traffic on the ring has a single characteristic information, such as VC-12, and uses the other layers defined for SDH to support these VC-12 connections. We will use this simple example to show how such a network can be modeled from several perspectives, specifically: • Topological architecture from a network-level perspective; and • Associated transport functions from an equipment-level perspective. To simplify the example, we will consider the case of VC-12 client traffic being carried by VC-4 server trails. This example could be easily extended by considering the servers of the VC-4 link connections and/or the clients of the VC-12 trails. From the service provider's perspective, the ring may be represented as a VC-12 subnetwork with an associated VC-12 subnetwork connection (illustrated in Figure 2-21).
VC-12 subnetwork
VC-12 subnetwork connection
Figure 2-21. Representation of a ring as a VC-12 subnetwork
A given VC-12 subnetwork connection may exit the ring at any individual equipment; thus, the assignment between the input and the output ports needs to be flexible. This flexibility may be supported by the use of the partitioning concept. The previous VC-12 subnetwork may be partitioned
Architecture of Transport Networks
43
into three subnetworks (i.e., equipment fabrics, one in each add/drop multiplex), with the two consecutive ones connected via VC-12 link connections. This level of partitioning is reflected in Figure 2-22.
VC-12 subnetwork
Figure 2-22. Further partitioning of VC-12 subnetwork
In fact, all VC-12 link connections are served by a VC-4 trail established between the two topologically adjacent equipment fabrics by a process in the server layer, as illustrated in Figure 2-23.
VC-12 subnetwork connection
VC-12 link connection
Figure 2-23. Illustration of VC-12 client/VC-4 server relationship
From an equipment perspective, we can now use the network-level model to represent the transport function characteristics that will be implemented in the ADM equipment. If we do so, we see that the equipment characteristics for one ADM may then be expressed as illustrated in Figure 2-24. As mentioned earlier, equipment encapsulates some common element of the layer, partition, and functional models, and is constrained to provide only physical interfaces.
Chapter 2
44
L
2.048 Mb/s
S12/E12 ^
\
S4/S12
^ ^
^
1
S12/E12
^X^
S4/S12
. ^
VC-4 network connection " ^
MS/S4
^
^v^
cJ>
MS/S4
;X"
cp ^MS
RS/MS
RS/MS
OS/RS
CP OS
• / ^
^ ^
OS/RS
^x-^
.OS
STM-N optical signal
Figure 2-24. ADM equipment characteristics
If we consider the direction from ingress to egress of the ring, illustrated previously in Figure 2-19, we can model the protection scheme in a straightforward manner. The specific application example for a 1+1 SNCP ring is illustrated in Figure 2-25. The selector connection function is flexible and is driven by trail signal fail (TSF) signals derived from the S12m_TT_Sk termination points that are reading the S12 layer characteristic information. This arrangement is known as nonintrusive monitoring because, while the layer overhead is read to provide signal quality information, the layer is not in fact terminated. We note that these connection functions model the "bridge" and "selector" previously depicted in Figure 2-19. In this example we illustrated how the model can be used to represent a SONET/SDH ring architecture as well as SDH ring ADM equipment. This example shows how such a model provides a language that links service description, equipment functionality, and equipment management.
45
Architecture of Transport Networks Uni-directional representation
Protected sub-network connection Bridge
Selector
;TSF
<^o>
u^tvv.Qrki;piiDectiQ.a___
(Q^
Figure 2-25. Subnetwork connection protection using nonintrusive monitoring
In the next example, we illustrate a service provider offering STM-64 switched connection services via an OTN mesh network, focusing our attention on the transport plane as opposed to the control plane, which supports the signaling necessary to automatically setup the connections within the transport plane. The STM-64 service provides flexible connectivity of STM-64 Regenerator Section (RS64) connections. In this example, the term STM-64 is used to refer to the STM-64 RS64 layer network. We illustrate how such a network can be modeled from several perspectives, specifically: • Topological architecture from the STM-64 RS64 network level perspective; • Topological architecture from the 0DU2 network level perspective; and • Associated transport functions from an equipment-level perspective. The customer connects to the service provider's network within the transport plane via an STM-64 physical interface (see Figure 2-26). The service provider provides an STM-64 switched connection service via the combination of a transport plane that supports the flexible connectivity of STM-64 connections and a control plane that provides dynamic routing and signaling capabilities to determine a path for the STM-64 connection and assign resources within the network to support the connection.
Chapter 2
46
Service Provider Network
SDH
xc
SDH XC
UNI
L Nl SDH XC
STM-64 Physical Interface
Figure 2-26. STM-64 switched connection service
The customer, via UNI signaling within the control plane, requests STM64 connections between a set of endpoints across the service provider's STM-64 network. The customer is not aware of, nor does it care about, the technologies and architecture used by the service provider to support such a service. From the customer's perspective, the topological architecture of the service provider's network can be modeled as an STM-64 subnetwork and multiple STM-64 access links (see Figure 2-27). The STM-64 subnetwork provides the flexible connectivity within the transport plane, and the STM64 access links provide the fixed connectivity between the customer and the service provider's network. In using the model to describe the topological architecture, we can clearly see that such signaling provides coordination for connection management between two partitions of the STM-64 layer network.
STM-64 Link
!
[1 | ( ( \
UNI
STM-64 Subnetwork
\^^^
fUNI
STM-64 Link End
Figure 2-27. Topological architecture of the STM-64 layer network
The customer requests an STM-64 connection originating and terminating at specific link ends. The connection is subsequently setup across the STM-64 access links and the STM-64 subnetwork (see Figure 2-
Architecture of Transport Networks
47
28). The connection is partitioned into two STM-64 link connections and one STM-64 subnetwork connection.
STM-64 Link Connection
STM-64 Link Connection
Ai
Figure 2-28. Partitioning of the STM-64 connection
Across the access hnks, the STM-64 hnk connections are supported via the Optical Section 64 (OS64) server trail, as described in Section 2.2.1.1, thus supporting an STM-64 physical interface between the customer and service provider. Within the service provider's network, there is a need to monitor the quality of the STM-64 subnetwork connection as it is transported across the network. Therefore the service provider must transport the STM-64 subnetwork connection via a server layer that can provide the necessary monitoring capabilities. In this example, the service provider supports the STM-64 connection via an OTN. The STM-64 subnetwork connection is supported via an ODU2 server trail. The 0DU2 server trail allows the service provider to monitor the STM64 client as it is transported across its network. The topological architecture from the 0DU2 network level perspective can be modeled as an 0DU2 subnetwork associated with various access groups (see Figure 2-29).
0DU2 Link
a
^ ^
0DU2 Subnetwork
1) 1
1 ^^ 0DU2 Access Group
Figure 2-29. Topological architecture of ODU2 layer network
Chapter 2
48
The ODU2 subnetwork can be further partitioned into four smaller 0DU2 subnetworks corresponding to four OTN cross-connect fabrics, connected via ODU2 links (see Figure 2-30).
/ JL
ODU2LJnk
/
y "^
y^
0DU2 Subnetwork
\ 1^
0 D U 2 Link
\ ^ (K OOU2 H i l l Subnetwork
A\ III
(k 111
ODU2 m Subnetwork l l j
m •
1 V
\ .
(
Subnetwork
T
Figure 2-30. Partitioning of 0DU2 layer topological architecture
An 0DU2 network connection is set up that originates and terminates within ODU2 access groups, across the 0DU2 links and ODU2 subnetworks. The network connection is partitioned into three 0DU2 link connections and two ODU2 subnetwork connections (see Figure 2-31). ODU2 Subnetwork Connection
0DU2 Subnetwork Connection
Figure 2-31. Partitioning of 0DU2 network connection
Architecture of Transport Networks
49
Above, we modeled the topology of both the STM-64 service layer and the ODU2 layer networks. Notice that when describing the topological architecture from a network-level perspective, we focus on a particular layer network; in other words, we take a horizontal snapshot of the network. In modeling the transport functions from an equipment-level perspective, we take a vertical snapshot of the network, focusing on termination and adaptation of multiple layer networks. To illustrate the modeling of transport functions from an equipment-level perspective, we will focus on the OTN equipment used to support the ODU2 links and subnetworks modeled above. In this example, the service provider uses OTN line systems interconnected via OTN cross-connects to support the network architecture. Figure 2-32 illustrates the physical OTN network architecture, which is the physical equipment implementation of the network architecture models presented above.
OTN XC
SDH XC
OTM-n.m
OTN XC
OTN DWDM Line System
OTM-n.m
OTN XC
OTM-n.m
SDH XC
OTN XC
r~\ SDH XC
Figure 2-32. Physical OTN network architecture
The OTN equipment transport functions, specifically the OTN DWDM line systems and the OTN cross-connects, can be modeled as shown in Figure 2-33. The OTN DWDM line terminals at the edge of the service provider network terminate the OS64 layer network and map the CBRIOG layer network characteristic information into an ODU2P server layer. The 0DU2P layer network is subsequently mapped into the OTU2 layer network, which is supported via an OCh layer network. Multiple optical channels are multiplexed into the OMSn layer network, which is then supported via the OTSn layer network. The ODU2 subnetwork is implemented via the OTN
Chapter 2
50
cross-connect. The OTN cross-connect demultiplexes/multiplexes the 0DU2 connections from/to the OTM-n.m hne interface. The OTSn, OMSn, OCh, and OTU2 layers are terminated, and the ODU2 logical signal is crossconnected.
OTN DWDM Line System
OTN XC
OTN DWDM Line System
i\
Figure 2-33. Functional model of an OTN network
2.2.9
Equipment Control
The functional model of an individual equipment describes the way in which the equipment accepts, processes, and forwards the information contained in a signal. Transmission and equipment supervision processes are concerned with the management of the transmission resources in the network and involve the equipment functionality. Description of these processes requires a functional representation of equipment that is implementation independent. We assume that the manager of the transport equipment has no knowledge of the internal equipment implementation, so equipment faults are recognized as the unavailability of the affected functions. Most atomic functions monitor the signals they are processing for certain characteristics
A rchitecture of Transport Networks
51
and provide performance information or defect information based on those characteristics. Therefore, transmission supervision processing provides information about the external interface signals that are processed by equipment. Equipment supervision processing is concerned with the fault localization and repair of the equipment itself. Its purpose is to answer the classic question "Who to send where to repair what?" with a single replaceable unit at a single location. This desire to report only a single fault stems from the need to avoid overloading management systems and personnel with unnecessary information. It does not require a knowledge of the transmission network, other than that the equipment faults may have been categorized to indicate the severity (e.g., prompt, deferred, maintenance event information) of the fault. Just as we have seen that the network topology can be described recursively (e.g., subnetworks can be partitioned into smaller subnetworks), we can also describe equipment recursively. Specifically, equipment is considered to be built from containers (i.e., racks, which contain shelves, which contain plug-in units) where the limit of equipment recursion is a replaceable unit. As its name suggests, a replaceable unit is the smallest piece of equipment that can be replaced, and is usually a plug-in unit. It also represents the lowest level of granularity needed for equipment fault reporting. Since a manager has no knowledge of the internal details of equipment, it clearly makes no sense to burden the manager with fault reports about these details. This point is an important to keep firmly in mind when working with the functional model, because many functions are frequently packaged on the same replaceable unit, and it can be very tempting — but not useful — to consider reporting detailed fault information for each function. Figure 2-34 below illustrates atomic function information points, i.e., information exposed by a function that is not part of the payload or its standard overhead. Information points include management, timing, and remote points that generate management information (MI), timing information (TI), and remote information (RI), respectively. Timing and management points can connect to any atomic function. (An independent functional model should be developed for a timing information network. Though we will not describe TI further in this chapter, ETS 300 417-6-1 [17] and G.806 [15] contain timing atomic functions.) Further detail on function inputs and outputs is illustrated in Figure 235. Here the vertical flows represent the pay loads, designated as characteristic information (CI) and adapted information (AI).
Chapter 2
52 DS3 Client Signal
DS3 Client Signal - e
DS3 Path Link Connection
& •
VC-3/DS3 Adaptation
A /-MI
\
A /-MI
-\n/-RI
\
A
y^MI
STM-N RS Trail
STM-N RS Trail
MI-\ry^Rl
MI-Kry^
RI
MI - Management Information TI - Timing Information RI - Remote Information Figure 2-34. Atomic function information points
Figure 2-35. Atomic function inputs and outputs
MI-\ry^
A rch itecture of Transport Networks
53
2.2.10 Equipment Supervisory Process In this section, we introduce the basic concepts, terminology, and constructs necessary for utihzation of equipment supervisory process concepts. 2.2.10.1
Basic Concepts
Estabhshing network and equipment level functional models according to the principles described in Section 2.3.2 provides us the necessary foundation for understanding the concepts associated with equipment supervisory processes. In particular, we will address processing, which takes place within an atomic function to derive basic network management information from the signal, and its preparation for passage to the elementlevel processing that derives alarms and performance parameters. Processing within an atomic function encompasses such steps as detailed definition of the information to be made available to a management system and detailed definition of the parameters that must be provisionable by an operator or management system. Element-level processing refers to the correlation and analysis of information provided by several atomic functions to provide a reduced amount of higher-level information to the operator. This processing leads to alarms, performance reports, and lighting of indications on the equipment. In summary, this data is the most detailed available on client signals. The supervision process describes the way in which the actual occurrence of a disturbance or fault is analyzed, with the purpose of providing an appropriate indication of performance and/or detected fault condition to maintenance personnel. In general, performance monitoring involves the continuous collection, analysis, and reporting of performance data associated with a transmission entity. It refers to the set of functions and capabilities necessary for equipment to gather, store, threshold, and report performance data associated with its monitored transmission entities. These performance-related data elements are termed performance parameters. Performance-monitoring parameters can be generated in trail termination, adaptation, and connection functions. The actual signal parameters monitored are technology dependent. Performance parameters are normally gathered under in-service, nonfailure conditions and are typically accumulated (and stored) over predetermined accumulation periods. Performance history data is useful for verifying customer trouble reports and responding to alerts so as to quickly
Chapter 2
54
assess the recent performance of transport systems (determine the Quahty of Service) and to sectionahze the trouble or degradation (for example, to locate sources of intermittent errors). This history can also be used in performance assessment against long-term performance objectives [18-19]. The equipment management function performance monitoring process collects the events associated with trail, link, and protection performance parameters. It counts the events to derive the performance parameters, and stores these parameters for later retrieval. In general, fault (or alarm/status) monitoring is a process that tracks failure events so as to contribute to an understanding of the overall transmission performance of an entity. The information conveyed via alarm/status monitoring consists of a set of indications that are maintained by the equipment. The equipment sets and clears indications according to well-defined criteria based upon the occurrence and duration of specific events. Some events immediately lead to indications, while others must persist for a specified amount of time prior to the setting of an indication. Alarm and status indications are generally reported under failure events. Alarm/status monitoring and performance monitoring complement one another [20]. Fault-monitoring parameters can be generated in trail termination, adaptation, and connection functions. Each atomic function and performance monitoring process generates and delivers a fault cause to the Element Management Function (EMF) fault management process, and a performance indication to the EMF performance monitoring process. The EMF fault management process within equipment performs a persistency check on the fault causes before it declares a fault cause a failure. The failure is reported via an output failure report and by means of alarms (audible and visible indicators). These functions of the supervision process are illustrated in Figure 2-36 below.
1 Atomic Functions Fault & Performance Monitoring Process
7 , Fault Causes
Equipment Management Function Fault Management Process
Performance Indications
Equipment Management Function Performance Monitoring Process
Figure 2-36. Functions of the supervision process
A rch itecture of Transport Networks 2.2.10.2
55
Terminology and Constructs
The following terms are used to describe the supervision process: anomaly, defect, fault, fault cause, failure, consequent action, and alarm. Performance primitives are basic performance-related occurrences detected by monitoring the signal, and these impairment events give rise to various performance parameters. Primitives are grouped into categories of anomalies and defects. Performance parameters are derived from the processing of performance primitives, and the associated terms are defined below [18-19]: • Anomaly: A performance anomaly is a discrepancy between the actual and desired characteristics of an item. The desired characteristic may be expressed in the form of a specification. An anomaly may or may not affect the ability of an item to perform a required function. • Defect: A performance defect is a limited interruption in the ability of an item to perform a required function. It may or may not lead to maintenance action, depending on the results of additional analysis. Successive anomalies causing a decrease in the ability of an item to perform a required function are considered to be a defect. • Fault: A fault is the inability of a function to perform a required action. Faults do not include an inability due to preventive maintenance, lack of external resources, or planned actions. • Consequent action - The action taken in response to an anomaly, defect, or fault. • Fault cause: A single disturbance or fault may lead to the detection of multiple defects. A fault cause is the result of a correlation process that is intended to pinpoint the defect that is representative of the disturbance or the fault that is causing the problem. • Failure: Performance failures refer to the termination of an item's ability to perform a required function. In equipment, both local and remote failures can be observed. Local failures involve near-end signal failures, and remote failures are those that occur and are recognized elsewhere and are reported within the transmission system. (While SDH transports most indications in special channels embedded in the signal, this is not necessary for the working of the model.) • Alarm: Alarms are specific (human observable) types of notifications concerning detected failures (or abnormal conditions) usually giving an indication of the severity of the failure. Typically, alarms can be divided into unit level alarms, equipment level alarms, and central office/station alarms. Detected anomalies are processed as described below: • Anomalies are subjected to a check to identify defects;
Chapter 2
56 • • •
Certain defects initiate consequent actions; Defects are correlated to identify the probable fault cause; and Near-end and far-end defect and performance impairment indicators are counted.
2.2.1Q3
Utilization of Concepts
The atomic function fault expanded to show its intemal performance processes for the Figure 2-37. While the details shown are completely generic.
Far-end Anomaly
Consequent Action
To/From Atomic Function Fault Management
Integration of anomalies into defects
and performance process can be further workings. As an example, the fault and trail termination function are depicted in are technology specific, the basic functions
Far-end Defect
Far-end . Performance [determination process
Consequent ' action control Near-end I performance Idetermination process Near-end Defects
Near-end Anomalies
Integration of anomalies into| defects
Consequent Actions
ToEMF performance monitoring
[Correlation of defects into a tfault cause
To EMF fault management
Consequent action control
Figure 2-37. Fault and performance processes for Trail Termination Function
The EMF fault management process is illustrated in Figure 2-38 below.
Architecture of Transport Networks
57
Unit
Translation of failures into Unit Alarms
Fault Cause cZZZ
Translation of failures into Equipment Alarms (e.g., Severity)
Failure Integration fZZZ of fault causes into failures
Equipment Alarms
; ^:
Station Alarms
j i
Translation of failures into Station Alarms and alarm disconnect control
Fault management selective reporting control
j
Autonomous j Failure Reports ; to Mgmt. System ^ i
Figure 2-38. Element Management Function fault management process
Similarly, for the EMF performance monitoring process (Figure 2-39), the individual performance parameter outputs are fed to functions that determine near- and far-end performance according to appropriate standardized performance parameters (e.g., errored seconds) and are next forwarded to the performance monitoring history process. As a final step, performance management reports may be provided on a selective basis.
raw performance information filter process far-end defect second
derived performance parameters
•
registers and filter report threshold
filter _ process
j performance : mgmt. - ^ reports ; :
mgmt. system
Figure 2-39. Element Management Function performance monitoring process
58
Chapter 2
This section has dealt with equipment supervision from the perspective of processes that have been described in equipment standards. In real implementations, the need to reduce the amount of data sent to management systems and personnel leads to internal proprietary mechanisms that are not part of the standard. For example, most switch fabrics use some sort of internal continuity check to generate appropriate fault indications, yet are not part of any standard. In the language of functional modeling, these checks can be modeled as internal trails in exactly the same way as has been discussed for external trails. 2.2.10.4
Application Example
While the atomic function fault and performance monitoring processes are internal to the atomic function, the results of the processing are generally shown as function indication inputs and outputs. An example using the VC4 trail termination sink function is described below and illustrated in Figure 2-40. In addition to the signal inputs and outputs described in the preceding sections, the termination sink function is considered to have four sets of management-related interfaces :< • An input interface from the server layer that accepts indications about the health of the server layer. - These indications can be used to represent a failure or degradation in the server layer network. • An output interface to the client layer that provides indications about the health of the trail provided by this layer. - These indications can inform the next downstream adaptation function of signal fail or signal degrade condition of the associated trail. • An output interface carrying remote indications from the far end sink termination. - These indications can convey the defect status of, or the exact or truncated number of error detection code violations within, the characteristic information received by the trail termination sink function back to the network element that contains the characteristic information originating trail termination source function. It is used for single-ended monitoring of the entire trail (i.e., the two extremities). • An input/output interface carrying local management indications allowing for the provisioning of threshold data and any local control that may be provided. These indications: - Enable control (start and stop) of the trail termination fault monitoring reporting activities
Architecture of Transport Networks -
59
Enable control of the server signal failure (SSF) report, when detected, to the adaptation function - Provide the expected VC-4 trace identifier (the received path identifier has to match with the expected one to prevent misconnections) - Enable the report of remote defect indicator (RDI) and server signal failure (SSF) to the management interface. This feature allows monitoring of the far-end trail termination function defects from the near-end trail termination function. - Allow provisioning of the minimum number of errors to be detected by this function over a one-second time period required to declare the second as a bad second. The naming conventions for these management-related interfaces are provided below [19]: • Input interface to server layer — _CI_[SSF] • Output interface to client layer — _AI_[TSF, TSD] • Output interface carrying remote indications from the far end sink termination — _RI_[RDI,REI] • Input/output interface carrying local management information - _TT_SK_MI_[TPmode, SSF-reported, ExTI, RDI_reported, DEGTHR]; input (or output) signal where SSF = Server signal fail SD = Server signal degrade TSF = Trail signal fail TSD = Trail signal degrade RDI = Remote defect indication REI = Remote error indication TT_SK = Trail termination sink TPmode = Trail termination fault reporting control ExTI = Expected trace identifier DEGTHR = Degraded threshold
60
Chapter 2 S4_AI_[TSF,TSD]
S4_TT_SK_MI_[...]
<
\
/
• S4_RI_[RDI,REI]
S4_CI_[SSF] Figure 2-40. VC-4 Trail Termination Sink processing
2.2.11 Modeling Connectionless Layer Networks As the G.805 modeling approach allowed us to analyze the transport network and identify generic functionality that is independent of implementation within connection-oriented networks, G.809 does the same for connectionless networks. The same types of architectural components used to describe connection-oriented networks (such as topological components, transport entities transport processing functions, and reference points) are used to describe connectionless networks. The concepts of layering and partitioning that were fundamental in describing the topological aspects of a connection-oriented network are similarly used in describing connectionless networks. Whereas the concept of a connection is central to the description of connection-oriented networks, such a construct is inappropriate for the description of connectionless networks. Within connectionless networks, analogous to the connection is the concept of a flow, which is an aggregation of one or more traffic units or data grams with an element of common routing. The concept of a flow uses the principle of recursion in that a flow can contain another flow, until the limit of recursion is reached when a flow contains exactly one data gram. In substituting the concept of a flow for the connection, equivalent components within the connectionless network may be defined. The topology of connectionless layer networks is described by access groups, flow domains, and flow point pool links, which correspond to access groups, subnetworks, and links within connection-oriented networks. In addition.
A rchitecture of Transport Networks
61
several kinds of flows may be identified, depending on the layer and partition traversed by the flow. These are the link flow, flow domain flow, network flow, and a connectionless trail, which correspond to link connections, subnetwork connections, network connections, and trails within connection-oriented networks. Finally, a fundamental difference between connection-oriented networks and connectionless networks is the directionality of transmission that may be supported. Whereas connection-oriented networks may support either bidirectional or unidirectional transmission, connectionless networks always transfer data unidirectionally.
2.2.12 Summary The functional modeling method utilized within these dimensions is based on a recursive decomposition technique that decomposes the network into layer networks, layer networks into atomic functions, atomic functions into processes, and processes into detailed requirements. The created set of processes and atomic functions establish a kind of library of components, which can be used to describe many types of equipment. As such, there are no longer any mandatory requirements for particular equipment in standards. A requirement is mandatory for equipment only if the equipment includes an atomic function containing a process that is stated to support the requirement. It becomes the manufacturers' responsibility to select the proper subset of atomic functions within each of the network elements offered. (Some further reading material related to the usage of functional modeling in networks and equipment may be found in references [21-23].)
2.3. 1.
NOTES Portions of the material in this chapter appeared in or are updates of material originally published in Varma, E.L., and T. Stephant, et al, Achieving Global Information Networking, Artech House, Norwood, MA, 1999. Replicated material is reprinted here and updates made with permission of the publisher.
62
2.4. [I]
Chapter 2
REFERENCES
S. Brown, "A Functional Description of SDH Transmission Equipment," BT Technology Journal, Volume 14, No. 2, April 1996. [2] ITU-T Recommendation G.805, Generic functional architecture of transport networh, March 2000. [3] ITU-T Recommendation G.803, Architecture of transport networks based on the synchronous digital hierarchy (SDH), March 2000. [4] ITU-T Recommendation G.783, Characteristics of SDH Equipment Functional Bloclcs, February 2004. [5] ITU-T Recommendation 0.841, Types and characteristics of SDH network protection architectures, October 1998, Corrigendum 1, August 2002. [6] ITU-T Recommendation G.842, Interworking of SDH network protection architectures, April 1997. [7] ITU-T Recommendation G.872, Architecture of optical transport networks, November 2001 Amendment 1 December 2003, [8] ITU-T Recommendation G.798, Characteristics of optical transport network equipment functional blocks, January 2002, Amendment 1 June 2002 [9] ITU-T Recommendation G.873.1, "Optical Transport Network (OTN): Linear protection", March 2003 [10] ITU-T Recommendation G.809, Functional architecture of connectionless layer networks, March 2003. [II] ITU-T Recommendation G.8010, Architecture of Ethernet layer networks, February 2004. [12] ANSI T1.105 -1995, Synchronous Optical Network (SONET) — Basic Description including Multiplex Structure, Rates, and Formats. [13] ITU-T Recommendation G.707, Network node interface for the Synchronous Digital Hierarchy (SDH), December 2003. [14] ITU-T Recommendation G.709, Interfaces for the Optical Transport Network (OTN), February 2001. [15] ITU-T Recommendation G.806, Characteristics of Transport Equipment - Description Methodology and Generic Functionality, February 2004. [16] J. Manchester and P. Bonenfant, "Fiber optic network survivability: SONET/optical protection layer interworking," in Proc. ofNFOEC'96, Denver, CO, 1996. [17] ETSI EN 300 417-6-1, Generic requirements of transport functionality of equipment; Part 6-1: Synchronization layer functions, May 1999. [18] ANSI Tl .231-1997, Digital Hierarchy - Layer I In-Service Digital Transmission Performance Monitoring. [19] ITU-T Recommendation G.784, Synchronous digital hierarchy (SDH) management, July 1999. [20] ETSI EN 300 417-7-1, Generic requirements of transport functionality of equipment, Part 7-1: Equipment management and auxiliary layer functions, October 2000. [21] Mike Sexton, and Andy Reid, Transmission Networking — SONET and the Synchronous Digital Hierarchy, Artech House, 1992. [22] Mike Sexton, and Andy Reid, Broadband Networking: A TM, SDH, and SONET, Artech House, 1997. [23] Huub van Helvoort, SDH/SONET Explained in Functional Models, John Wiley & Sons, 2005.
Chapter 3 INTERFACES FOR OPTICAL TRANSPORT NETWORKS
Timothy P. Walker* and Khurram Kazi** ""AMCC, **SMSC
3A.
INTRODUCTION
The advancements in optics, photonics, electronics, software, and human expertise ensure the continual evolution of telecommunication and data networks. Over and over again, we have seen that present-day transport networks become tomorrow's access networks. We have witnessed Plesiochronous Digital Hierarchy (PDH) networks that once were the backbone of the network operators, with the widespread laying of the optical fiber, become the feeding data pipes to Synchronous Optical Networks or Synchronous Digital Hierarchy (SONET/SDH). The single optical channel transport system based on SONET/SDH standards proved to be very successful for its time. With the advent of Dense Wavelength Division Multiplexer Optical Add-Drop Multiplexer (DWDM-OADM), optical amplifiers, and Optical Cross-Connects (OXCs), we are witnessing a paradigm shift, where SONET/SDH networks are slowly becoming the feeding pipes to an "all-optical" transport network, better know as the Optical Transport Network (OTN). Since the telecom bubble burst, the rush to deploy OTN has been much slower that originally anticipated. With each paradigm shift, the standards committees are faced with the challenge of defining new standards that become the foundation or the blueprints [1] for the network service providers, equipment manufacturers, and users alike. Every effort is made to ensure that such standards are
64
Chapter 3
general enough so as not to be limited by the present technology yet provide enough information such that interoperability between different service providers or equipment manufacturers is seamless. International Telecommunication Union-Standardization Sector (ITU-T) has been leading the efforts in defining the optical networking standards. As we saw in Chapter 2, at the very outset of the development of OTN standards, it was realized that a framework or set of conventions had to be developed. ITU-T Rec. G.805 [2] established the rules and formulated the fundamental vocabulary that is used in capturing and analyzing the requirements of the transport networks.
3.2.
OTN STANDARDS
Many standards fall under the umbrella of OTN. In this chapter, we focus on Layer 1 standards and will not cover the physical or optical layers. Chapters 17 through 19 cover the relevant standards regarding the intranetwork element communication, physical electronic links, and optical properties. The Optical Transport Hierarchy (OTH) is a new transport technology for the OTN. It is based on the network architecture defined in ITU G.872 [3] Architecture for the Optical Transport Network (OTN). It defines an architecture that is composed of the Optical Channel (OCh), Optical Multiplex Section (OMS), and Optical Transmission Section (OTS). Functionality that is needed to make OTN work is also described in it. However, it may be interesting to note the decision made during G.872 development, as noted in Section 9.1/G.872 [3] : During the development of ITU-T Rec. G.709, (implementation of the Optical Channel Layer according to ITU-T Rec. G.872 requirements), it was realized that the only techniques presently available that could meet the requirements for associated OCh trace, as well as providing an accurate assessment of the quality of a digital client signal, were digital techniques.... For this reason ITU-T Rec. G. 709 chose to implement the Optical Channel by means of a digital framed signal with digital overhead that supports the management requirements for the OCh listed in clause 6. Furthermore this allows the use of Forward Error Correction for enhanced system performance. This results in the introduction of two digital layer networks, the ODU and OTU The intention is that all client signals would be mapped into the Optical Channel via the ODU and OTU layer networks.
The above statements can be attributed to the fact that the use of optical amplifiers and DWDM components significantly increased the transport capacity of the optical fiber. Hundreds of narrowly spaced wavelengths can
Interfaces for Optical Transport Networks
65
carry individual optical channels over ever-increasing distances. With these advancements came the problems associated with the "analog" transmission. Due to characteristics of the optical fiber and the related optical components, the optical signals eventually have to be regenerated. Moreover, it is possible that no single network operator will be the owner of the end-to-end path. Thus, appropriate management and control information for "signal handoff' needs to be passed from one operator's domain to another's without revealing the details of the two respective networks. This requirement resulted in the intelligent processing in the digital domain at naturally occurring 3R points (Retime, Reshape, and Reamplify) or at the domain boundaries. Another requirement that was placed on the next generation optical transport network is that it should be able to transport any client signal transparently. Figure 3-1 summarizes the requirements placed on the OTN [1].
-'—r
Ability to manage the network at the interfaces
Ability to accept ANY client signal Och Trail. FEC, TOM OH inserted
Och FEC, TOM OH terminated/inserted
Och Trail, FEC, TOM OH terminated
Optically tj-anspaireiiit;:
W:^gi^m^::-
OTN Client
Optically transparent subnetwork, bounded by feature enhanced electronics
3R Opto-Electronic OADM Processing Unit OLS: Optical Line System; OXC: Optical Gross Connect; OADM: Optical Add-Drop Mux; 3R: Retime, Reshape, and Reamplify
Figure 3-1. 3R points within Optical Transport Networks
Some characteristics that the standards committee members were keeping in mind during the evolution of OTN were to provide • An efficient transport networking layer that supports very high rate services at 2.5, 10 and 40 Gb/s • A strong Forward Error Correction (FEC) • An increased number of Tandem Connection Monitoring (TCM) layers
66
Chapter 3 • •
• • • •
Management of frequency slots (OCh's signals of multiple A.s) instead of time slots (STS-1, etc.) Service transparency for any client signal transport—for example, Ethernet, SONET/SDH, ATM, EP, Fibre Channel, Digital Video Broadcast, or proprietary data stream The shortest and most efficient physical layer stack for data services transport The gigabit-level bandwidth granularity required to scale and manage multiterabit networks A means to avoid very large numbers of fine granular pipes stressing network planning, administration, and management A means to select the frequencies such that they can be transmitted over existing transoceanic optical line systems with lOG wavelengths
It should be noted that unlike the SONET/SDH standard, which was designed around a mux structure starting from VT1.5/VC12 that filled the payload with voice-centric 64 Kb/s traffic, one of the major rationales behind the development of OTN standards was to solve the transport problems at higher speeds and not be restricted by any legacy work. The two relevant standards that cover the implementation of OTH are G.709 [4] and G.798 [5].
3.3.
STANDARDIZED INTERFACES
The optical transport network architecture as specified in ITU-T 0.872 defines two interface classes: • Interdomain interface (IrDI) • Intradomain interface (laDI) The OTN IrDI interfaces are defined with 3R processing at each end of the interface, i.e., the interface between Operators. It could also be thought of as the interface between different vendors within the same operator, as depicted in Figure 3-2.
Interfaces for Optical Transport Networks
67
User \ ii«
Figure 3-2. laDI vs. IrDI
The laDI is the interfaces within an operator or a vendor domain. G.709 apphes to information transferred across IrDI and laDL G.709 compHance at the IrDI by itself is not sufficient to guarantee a midspan meet, since it doesn't specify the electrical or optical interfaces. It is important to note that G.709 only defines logical interfaces.
3.4.
FORWARD ERROR CORRECTION
Forward error correction is a major feature of the OTN where the FEC check information is appended to the G.709 frame (making it "out of band" FEC), thus lending itself to strong FEC. The FEC defined for SONET/SDH networks uses undefined section overhead bytes to transport the FEC check information and hence is referred to as in-band FEC. It allows only a limited amount of FEC check information, which limits the performance of the FEC. For the OTN a Reed-Solomon 16-byte interleaved FEC scheme is defined, that uses 4 x 256 b3^es of check information per Optical channel Data Unit (ODU) frame. In addition, enhanced (proprietary) FEC schemes are explicitly allowed and are widely used, as described in ITU-T recommendation G.975.1 [6]. FEC has been proven to be effective in Optical Signal-to-Noise Ratio (OSNR) limited systems as well as in dispersion limited systems. As for nonlinear effects, reduction of the output power leads to OSNR limitations, against which FEC is useful. FEC is less effective against Polarization Mode Dispersion (PMD); however, G.709 defines a stronger FEC for OTN that can result in up to a 6.2 dB improvement in Signal to Noise Ratio (SNR). Another way of looking at this is as a transmission of a signal at a certain Bit Error Rate (BER) providing 6.2 dB less power than without such an FEC. The coding gain provided by the FEC can be used to •
Increase the maximum span length and/or the number of spans, resulting in an extended reach (note that this point assumes that other impairments
68
Chapter 3
such as chromatic and polarization mode dispersion are not becoming Hmiting factors) • Increase the number of Dense Wavelength Division Multiplexing (DWDM) channels in a DWDM system that is limited by the output power of the amplifiers by decreasing the power per channel and increasing the number of channels (note that changes in nonlinear effects due to reduced per-channel power have to be taken into account) • Relax the component parameters (e.g., launched power, eye mask, extinction ratio, noise figures, and filter isolation) for a given link and lower the component costs • Most importantly, make the FEC an enabler for transparent optical networks Transparent optical network elements like Optical Add Drop Multiplexers (OADMs) and Photonic Cross-Connects (PXCs) introduce significant optical impairments (e.g., attenuation). The number of transparent optical network elements that can be crossed by an optical path before 3R regeneration is needed is therefore strongly limited. With FEC, an optical path can cross more transparent optical network elements. This allows the evolution from point-to-point links to transparent, meshed optical networks with sufficient functionality.
3.4.1
Theoretical Description
G.709 FEC implements a Reed-Solomon RS(255,239) code. A ReedSolomon code is specified as RS(n,k) with ^--bit symbols, where n is the total number of symbols per codeword, k is the number of information symbols, and s is the size of a symbol. A codeword consists of data and parity, also known as check symbols, added to the data. The check symbols are extra redundant bytes used to detect and correct errors in a signal so that the original data can be recovered. For G.709: s = size of the symbol = 8 bits n = symbols per codeword = 255 bytes k = information symbols per codeword = 239 bytes A typical system is shown in Figure 3-3.
Interfaces for Optical Transport Networks interference Data source
Reed-Solomon encoder
> "^
69
i^
Communication Channel
J ^
Reed-Solomon decoder
Data sink
Figure 3-3. FEC block diagram
This setup means that the encoder takes k information symbols of s bits each and adds check symbols to make an ^-symbol codeword. There are n-k check symbols of s bits each. A Reed-Solomon decoder can correct up to t symbols that contain errors in a codeword, where 2t = n - k. Figure 3-4 shows a typical Reed-Solomon codeword:
n ^
Data
<
21
*
Parity
Figure 3-4. Reed-Solomon codeword
For the G.709 standard ITU recommended RS (255,239) code: 2/ = / 2 - ^ = 255-239=16 Hence, the decoder can correct any eight symbols in a codeword. ReedSolomon codes treat errors on a symbol basis; therefore, a symbol that contains all bits in error is as easy to detect and correct as a symbol that contains a single bit-error. That is why the Reed-Solomon code is particularly well suited to correcting burst errors (where a series of bits in the codeword are received in error by the decoder). Given a symbol size s, the maximum codeword length {n) for a Reed-Solomon code is n = T-\=255 Interleaving data from different codewords improves the efficiency of Reed-Solomon codes because the effect of burst errors is shared among many other codewords. Interleaving spreads the impact of a noise burst over multiple symbols, that come from several code words. As long as each deinterleaved codeword has fewer errors than it can correct, the interleaved
70
Chapter 3
group of codewords will be corrected. It is possible that some codewords will be corrected and some not if excessive errors are encountered. Interleaving actually integrates the error-correction powers of all of the codewords included in the interleaved group that is the depth of the interleaver. This allows a higher rate of code and channel efficiency and still protects against an occasional very long error. For example, if 64 codewords that can correct 8 errors each are interleaved, the interleaved group can correct almost any combination of symbol errors that total less than 512. It does not matter whether all 512 are in one long burst, appear as 512 onesymbol errors, or are in any combination in between. Both ITU-T G.709 and ITU-T G.975 specify interleaving as part of the transport frame to improve error-correction efficiency.
3.4.2
Coding Gain
The advantage of using FEC is that the probability of an error remaining in the decoded data is lower than the probability of an error if an FEC algorithm, such as Reed-Solomon, is not used. This outcome is coding gain in essence. Coding gain is the difference in Input SNR for a given Output BER. The Input SNR is measured either as the "Q factor" or as E\JNo or OSNR (). The "Net Coding Gain" takes into effect that there was a 7% rate expansion due to the FEC. What this means is that the data rate had to increase by 7% in order to transmit both the data and the FEC. 3.4.2.1
Coding Gain measured via Q Factor
The widely used technique of measuring coding gain is the Q-factor (quality factor) measurement. This technique estimates the OSNR at the optical amplifier or receiver by measuring BER vs. voltage threshold at voltage levels where BER can be accurately determined (see Figures 3-5 and 3-6). In reality, however, the Q-factor is derived from the measurement of the eye-pattern signal. It is defined as the ratio of peak-to-peak signal to total noise (conventionally electrical): Q = (jLli - |Llo)/(ao + Gi)
where |Lii and jLioare the mean signal levels of logic level 1 and logic level 0 Go and Gi are the respective standard deviations
Interfaces for Optical Transport Networks
X -x- :><
71 1^1, CJi
1^0, CFo
Figure 3-5. Eye diagram
Ui = logic level 1 average value Gi = logic level 1 noise standard deviation Uo = logic level 0 average value Go = logic level 0 noise standard deviation A system that requires an operating BER of 10'^^ has a Q-factor measurement of 18 dB without FEC. If RS (255, 239) FEC is employed, the Q-factor measurement decreases to 11.8 dB, yielding 6.2 dB of coding gain.
3.4.2.2
Coding Gain measured via Ey^/No
Another way to measure coding gain is with a plot of BER vs. EJNQ. E\J is the bit energy and can be described as signal power (S) times the bit time Tb- ^0 is the noise power spectral density and can be described as the noise power (AO divided by the bandwidth (W). Thus E^No is equal to SNR * (Bandwidth/Bit Rate). For a more thorough discussion, the reader is referred to [7]. Figure 3-7 shows what the BER out of the FEC decoder would be for a given input SNR (Eb/No). Thus to operate a system at 10"'^ BER, then one would need over 14 dB SNR without FEC or only 8.5 dB with FEC.
72
Chapter 3 BER vs Q for R-S 255 Code (t = 8)
[
\ BER in
1
1
k j
.
. No Overhead
\
1
: j 10
11
12 \
13
14
15
16
17
18
19
20
Q(dB)
Figure 3-6. BER vs. Q factor
0
r- -^^
"^
\ ^
L ncorre cted
X
v
7^
\= \
^ ^
'
"v—
\
.709
\
\
V—
F\
o""
\, \
2
3
4
5
6
7
;.86dE
LAJ 9
\
V A
6.2 dE 10
11
12
13
k
\ \
LAI
14
15
Figure 3-7. BER vs. E^NQ
3.4.2.3
Coding Gain measured via OSNR
Figure 3-8 shows the FEC net coding gain (NCG) of various FEC schemes. These are theoretical and real measurement results from running systems. Coding gain is the reduction of signal-to-noise ratio due to the use of the FEC at a reference BER. The Net Coding Gain (NCG) takes into
Interfaces for Optical Transport Networks
73
account the fact that the bandwidth extension needed for the FEC scheme is associated with increased noise in the receiver. For example, consider a reference BER of 10"^'. The SDH in-band FEC provides an NCG of 4 dB, the standard OTN FEC an NCG of 6.2 dB, and an enhanced FEC an NCG of 9.5 dB.
Unencoded
'"-^''''"^---rr^, ,-.-.:
10-' CO •3
%
10"'
^SDH in-band FEC
:
\
i
L
:
r
: : !
1 t I
•
}
• OTN EFEC (measured)
\ \
S.^A-'^T^ . - - ^
\^
OTN EFEC (theoretical)
\
^*^
1
10"
|
OTN Standard FEC
\
11 1
\
\
\ " NEGC @ BER 10
ll
1
8
10 O S N R (dB)
-''« .
\
12
FEC margins Figure 3-8. Coding gain measured via OSNR
3.5.
TANDEM CONNECTION MONITORING
SONET/SDH monitoring is divided into section, line, and path monitoring. A problem arises with the "carrier's carrier" situation, as shown in Figure 3-9, where it is required to monitor a segment of the path that passes another carrier network. Here Operator B is carrying the signal of Operator A. However, Operator A needs to monitor the signal as it passes through Operator B's network. This is how a "tandem connection" is defined. It is a layer between line monitoring and path monitoring. SONET/SDH was modified to allow a single tandem connection. G.709 allows six. TCMl is used by users to monitor the Quality of Service (QoS) that they see. TCM2 is used by Operator A to monitor his or her end-to-end QoS. TCM3 is used by the various domains for intradomain monitoring. Then TCM4 is used for protection monitoring by Operator B. There is no standard concerning which TCM is used by whom. The operators have to have an agreement so that they don't conflict. As we shall
Chapter 3
74
see later in the chapter, TCMs also support monitoring of ODUk (G.709 without the FEC) connections for one or more of the following network apphcations (outlined in ITU-T G.805 and ITU-T G.872):
Figure 3-9. Tandem Connection Monitoring
Optical UNI to UNI tandem connection monitoring; monitoring the ODUk connection through the public transport network (from public network ingress network termination to egress network termination) Optical NNI to NNI tandem connection monitoring; monitoring the ODUk connection through the network of a network operator (from operator network ingress network termination to egress network termination) Sublayer monitoring for linear 1+1, 1:1, and 1:« optical channel subnetwork connection protection switching, to determine the signal fail and signal degrade conditions Sublayer monitoring for optical channel shared protection ring (SPRing) protection switching to determine the signal fail and signal degrade conditions Monitoring an optical channel tandem connection for the purpose of detecting a signal fail or signal degrade condition in a switched optical channel connection, to initiate automatic restoration of the connection during fault and error conditions in the network Monitoring an optical channel tandem connection for, e.g., fault localization or verification of delivered quality of service
Interfaces for Optical Transport Networks
75
The number of monitored connections along an ODUk trail may vary between 0 and 6. Monitored connections can be nested, overlapping, and/or cascaded. Nesting and cascading are shown in Figure 3-10. Monitored connections A1-A2/B1-B2/C1-C2 and A1-A2/B3-B4 are nested, while B1-B2/B3-B4 is cascaded.
TCM OH Field not used
[ZD
TCMi OH Field in used
Figure 3-10. ODUk monitored connections
Overlapping monitored connections as shown in Figure 3-11 (B1-B2 and C1-C2) are also supported.
TCM6
TCM6
TCM6
TCM6
TCM6
TCM5
TCM5
TCM5
TCM5
TCM5
TCM4
TCM4
TCM4
TCM4
TCM4
TCM3
TCM3
ri:cM3:;
'tckr
TCM3
^^9
TCM2
TCM2
TCMl
TCMl
^B
TCM2 TCMI
^Ib
^
TCMl
TCMl
^
^ j ^
Al
Bl
CI
B2
C2
Cl-C 2 I J1-B2 A1-A2
1 TCM OH Field nc tused
1
^H
'^^r'H-
TC MiC IHFi eld inu secI
Figure 3-11. Overlapping of ODUk monitored connections
A2
Chapter 3
76
3.6.
OTN HIERARCHY OVERVIEW
The OTN allows multiple optical channels to be transported simultaneously, where each channel is nominally mapped to an individual wavelength. Each optical channel is transported in a typical digital frame structure that is made up of payload and overhead fields. As we will see shortly, the client data is mapped into the payload area and the networkrelated information is carried by the overhead section. The entire structure is partitioned into layers, as depicted in Figure 3-12 [8].
OCh Payload Unit OCh Data Unit OCh Transport Unit
Optical Cliannel Carrier Optical Multiplex Section Optical Transmission Section OTM Overhead Signal Optical Supervisory Channel Optical Physical Section
Figure 3-12. OTN layers and containment relationships
Figure 3-13 [8] shows the partitioning of the layers from a transport networks point of view. The OPUk, ODUk, and OTUk are in the electrical domain. The OPUk encapsulates the Client signal (e.g., SONET/SDH) and does any rate justification that is needed. It is analogous to the Path layer in SONET/SDH in that it is mapped at the source, demapped at the sink, and not modified by the network. The ODUk performs functions similar to these of the Line Overhead in SONET/SDH. The OTUk contains the FEC and performs functions similar to those of the Section Overhead in SONET/SDH. After the FECs are added, the signal is then sent to a Serializer/Deserializer (SerDes) to be converted to the optical domain. The
Interfaces for Optical Transport Networks
11
OCh is in the optical domain, which in the DWDM context can be envisioned as mapping of the OTUk to one of the wavelengths. Architectural demarcation points Optical Multiplex Section (OMS) can be mapped to the
OCh, OTUk 4 — • !
•<
OCh,OT^/k
OCh, OTUk
LT\
Client
H
i|><jfe"Y~ \
LT: OCADM: OCXC: 3R: R:
/LT
l
Line Terminal w/ optical channel multiplexing Optical Channel Add/Drop Multiplexer Optical Channel Cross-Connect O/E/O w/ Reamplification, Reshaping & Retiming and monitoring Repeater
OCXC
LT/
\LT
Figure 3-12. OTN layer network trails
Optical multiplexing function, and the Optical Transmission Section (OTS) is synonymous to the transport of the aggregated wavelengths, or "single signal," over the fiber. However, due to custom and diverse sets of implementations, the OMS and OTS are not yet defined within the ITU standards. For all practical purposes, only four layers are defined, as depicted in Figure 3-14.
Client Signals like SONET/SDH, Constant Bit Rate, Fibre Chatinel, IP, Ethernet, etc.
\ \/7 OPUk ODUk OTUk OCh
Figure 3-14. Defined OTN hierarchy
78
Chapter 3 <
B B
<
00 1
k2 BDI
r
NJ 1
00 BDIi
[en
Icnl H-j
M
1
"ft'o
1 1 ^
CO 1 ~~^^^^^^^ p T s
|~To
Pi
~"^^ 1
2^ 2^
r%. o ft f_. ho 1
C) i—>
m :
H"
u
u
•^
*">
00
m Di
m
^ J fc H
CO
PH
GO
w p^
o y. ^
00 ft
,S C
fe w 'o
-%ft ^ "in
Pu
o
o
r
__
H
Tf
(N
:^ u
\ ! o^ l~~|
H
pq
^ fi^
1 O
1
^
H H
<:
00
ft
•X3
g CD
o
ft
c
> TWj ^
_o
o a> CO U
ft
ifl
O '-CS •j3 cj
'o ft
*-
CO
IS
4^ • -
^
^
Q
Sec ^ H -B
T3
3 .f1
m
PQ PQ CO
00 ft
<
W Q
-d
U m PQ PQ 00
Ts I d =^ 0 0 ^ 3 ft
ft
00 H ft ft
K^
cj H:^
&
1 §
>
1^
O
;2:
ft
go iz:
ft
cd
§
'—' O
in c^
00
H
S
oo
^r
CO I-
i^
T)
! t< u
0
S
1
(S
ft 00
MD
ko
i 1 11 u1 .g •4:3
pHUk ClveiBiea
R
"S
•2
•S? 53
r-i
00
^ H
H"
1—1
D 00
H <
<
'S
(N
5 o
^
CO
•—'
<1
00
H
r—1
"IS CD CO
W
1 1
00
U 00
^
o U
§
"S 3
•4:3
(N
m
1 ^
(N
m
• * ~ |
Row Number
Figure 3-J5. OTN G.709 frame format
00 rt
0
0
rt
ft
U
3 ft
0
0
cli r o ^ H >' u W c> U 00
"-S 0
0
S
C So a "^=2 W CD c3 O O fii H <
O e^i H <
^
0 ft
0 a
0 •-E3
o
g
1 S)
lO
•S 1
!!> bri
O
3 ft ft ft
ft
X, 5 S W
<
ft
00 U ft U <1 ft
V X
ft
w
ft
Interfaces for Optical Transport Networks
79
3.7. OTN a 7 0 9 FRAME STRUCTURE Figure 3-15 shows the OTN frame format as defined in G.709. It consists of OTUk, ODUk, and OPUk overhead; an OPUk payload area; and an FEC field. It is the OPUk field where client signals like SONET/SDH, constant bit rate, Generic Framing Procedure (GFP), and encapsulated data (IP, Ethernet, Fibre Channel, etc.) to name a few, are mapped for transport. GFP will be covered in depth in Chapter 5. Figure 3-16 shows the mapping of the varied client signals onto the G.709 frame. One should keep in mind that the OPUk (Client signal) can only be of one type; e.g., it can either carry SONET/SDH (can also be considered CBR traffic) or ATM or CBR or GFP encapsulated traffic. The traffic type cannot be intermixed at the OPUk level. However, as we will see in Chapter 5, various traffic types can be encapsulated within GFP and can subsequently be transported as the "GFP" traffic type within the OPUk structure. The data rates were constructed so that they could transfer the SONET/SDH signal efficiently.
MPLS
SONET/SDH
t
ATM
CBR
IP
GbE
Ipibre Channell
3- -E GFP
W^^-^^n='.: ^^^-^i;r--r--r^
4 rows 4080 columns Figure 3-16. Client signal mapping onto the G.709 frame
The bit rates of OTUk, ODUk, and OPUk are shown Tables 3-1 to 3-4. In Table 3-4, the period is an approximated value that is rounded off to three digits.
80
Chapter 3
Table 3-1. OTU types and capacity (from ITU-T Rec. G.709)
OTU type
OTU nominal bit rate
OTU bit-rate tolerance
OTUl
255/238 X 2 488 320 kbit/s
±20 ppm
0TU2
255/237 X 9 953 280 kbit/s
OTU3
2 5 5 / 2 3 6 x 3 9 813 120 kbit/s
Table 3-2. ODU types and capacity (from ITU-T Rec.709) ODU nominal bit rate
ODU bit-rate tolerance
ODUl
239/238 X 2 488 320 kbit/s
±20 ppm
0DU2
239/237 X 9 953 280 kbit/s
0DU3
2 3 9 / 2 3 6 x 3 9 813 120 kbit/s
ODU type
Table 3-3. OPU types andcapacity (from ITU-T Rec. G.709) OPU type
OPU Payload nominal bit rate
OPUl
2 488 320 kbit/s
0PU2
238/237 X 9 953 280 kbit/s
0PU3
2 3 8 / 2 3 6 x 3 9 813 120 kbit/s
OPU payload bitrate tolerance
Table 3-4. OTUk/ODUk/OPUkframeperiods (from ITU-T Rec. G.709) OTU/ODU/OPUtype
Period (Note)
OTUl/ODUl/OPUl/OPUl-Xv
48.971 ^is
OTU2/ODU2/OPU2/OPU2-XV
12.191 fis
OTU3/ODU3/OPU3/OPU3-XV
3.035 ^s
±20 ppm
Interfaces for Optical Transport Networks
3.8.
81
G.709 OVERHEAD BYTES: IN-DEPTH ANALYSIS AND PROCESSING
In transport networks, the overhead b3^es are generally used for conveying performance, status, configuration, control, and management information regarding the transport network. This information can be relayed as instant snapshot values in single or multiple bytes or can span multiple frames in the form of messages. These bytes are passed between the network elements spanning their respective sections, domains, or paths. One of the characteristics of the G.709 frame structure is that, certain overhead fields use the same value or have the same content format and are transported over different overhead bytes. The information contained in these similarly valued overhead bytes is interpreted within the respective segments of the transport network. For example. Bit Interleaved Parity code (BIP-8) or Trail Trace Identifier (TTI) are such fields (and are elaborated in the subsequent section). To understand the context of the communication between different network nodes within the entire OTN, one needs to keep in mind the directionality of the information flow. Frequently used terms such as upstream node or downstream node need to be clarified. This concept can be illustrated by using the exchange of BIP-8 information between two different nodes, which do not have to be adjacent to each other. The transmitting node (Node A) sends the BIP-8 code to the downstream node (Node B). The receiving node (Node B) recalculates the BIP-8 and compares its calculated value with the value it received. If there are differences between the two values, appropriate information regarding the number of BIP-8 violations is sent to the upstream node (Node A). Figure 3-17 shows the directionality of the upstream and the downstream nodes. It should be kept in mind that the nodes depicted in Figure 3-17 do not have to be adjacent to each other. As illustrated in Figure 3-13, we can say that in the OTUk context, the source/sink node pair (downstream/upstream pair) will be the 3R point nodes, whereas in the ODUk context, they will be the LTs connected to the DXCs.
Chapter 3
82
Downstream Direction :
o
BIP-8 locally calculated and compared to received value
NodeB
If any BIP-8 violations are ""detected, information is sent "" back to the transmitting node ^T^TN
Upstream Direction Figure 3-17. Directionality of upstream and downstream nodes
3.8.1
OPUk Overhead Bytes and Client Mapping Structure
The OPUk {k = 1,2,3) frame structure is shown in Figure 3-18 (a). It is organized in an octet-based block frame structure with four rows and 3810 columns, with column 3824 being the last column of the OPUk field. The two main areas of the OPUk frame are (1) the OPUk overhead area and (2) the OPUk payload area. Columns 15 to 16 of the OPUk are dedicated to the OPUk overhead area. Columns 17 to 3824 of the OPUk are dedicated to the OPUk payload area. It should be noted that the OPUk column numbers are derived from the OPUk columns in the ODUk frame. OPUk OH information is added to the OPUk information payload to create an OPUk. This includes information to support the adaptation of client signals. The OPUk OH is terminated where the OPUk is assembled and disassembled.
Interfaces for Optical Transport Networks s^ Column Row FA
MFAS
OTUkOH
ODUkOH
15
16
RES
JC
17
RES
JC
RES
JC
PSI
NJO
PJO
15
16
17
RES
JC
RES
JC
RES
JC
PSI
NJO
83
18
Payload (Clicnf Signal) Slightly Slow!
o /
(a) PJO byte stuffing y Column FA
MFAS OTUkOH
ODUkOH
3824
18
Payload (Client Signal) Exact Match!
/
(b) Exact rate 17
1 FA
MFAS
OTUkOH
ODUk OH
RES
JC
RES
JC
RES
JC
18
Payload (Client Signal) Slightly Faster!
PSI
(c) Use additional byte for payload Figure 3-18. OPUk frame structure
3.8.1.1
OPUk Overhead description
Payload Structure Identifier (PSI): The 256-byte PSI signal is aligned with the ODUk multiframe (i.e., PSI [0] is present at ODUk multiframe position 0000 0000, PSI[1] at position 0000 0001, PSI [2] at position 0000 0010, etc.). PSI [0] contains a one-byte payload type. PSI [1] to PSI [255] are mapping and concatenation specific. Payload Type (PT): A one-byte payload type signal is defined in the PSI [0] byte of the payload structure identifier to indicate the composition of the OPUk signal. There are a number of payload types defined in Table 3-5. The virtual concatenated signal and the ODU multiplex structure are dealt with later in this document. Mapping SONET/SDH (CBR) into OPUk either synchronously or asynchronously is the most common mapping. Synchronous mapping is a subset of asynchronous mapping; thus we will only discuss asynchronous mapping.
Chapter 3
84 Table 3-5. Payl oad type code points (from ITU-T Rec. G.709) Interpretation
MSB
LSB
Hex code
1234
5678
(Note 1)
0000
0001
01
Experimental mapping (Note 3)
0000
0010
02
Asynchronous CBR mapping
1
0000
001 1
03
Bit synchronous CBR mapping
0000
0100
04
ATM mapping
0000
0101
05
GFP mapping
1 1
1
0000
Olio
06
Virtual concatenated signal (see Note 5)
0001
0000
10
Bit stream with octet timing mapping
1
0001
0001
11
Bit stream without octet timing mapping
1
0010
Olio
20
ODU multiplex structure
1 1 1 r
0101 Olio 1000 1111
0101
55
Not available (Note 2)
Olio
66
Not available (Note 2)
xxxx
80-8F
Reserved codes for proprietary use (Note 4)
1101
FD
NULL test signal mapping
1
1111
1110
FE
PRBS test signal mapping Not available (Note 2)
1111 FF 1 1111 1 Note 1 - There are 226 spare codes left for future international standardization. Note 2 - These values are excluded from the set of available code points. These bit patterns are present in ODUk maintenance signals. Note 3 - The value "01" is only to be used for experimental activities in cases where a mapping code is not defined in this table. Note 4 - These 16 code values will not be subject to standardization. Note 5 - For the payload type of the virtual concatenated signal, a dedicated payload type overhead (vcPT) is used.
3.8.1.2
Frequency Justification
Asynchronous mapping of a 2.5 Gb/s, lOGb/s, or 40 Gb/s constant bit rate (CBR) traffic signal into an OPUk (k = 1,2,3) may be performed as depicted in Figures 3-19 (b) and 3-19 (c). The maximum bit-rate tolerance between OPUk and the client signal clock that can be accommodated by this mapping scheme is ±65 ppm. With a bit-rate tolerance of ±20 ppm for the OPUk clock, the client signal's bit-rate tolerance can be ±45 ppm. If the client's frequency is out of range, then there aren't enough justification bytes in the OPUk overhead to make up the difference. The OPUk overhead for these mappings consists of a payload structure identifier (PSI) including the payload type (PT) and 255 bytes reserved for future international standardization (RES), three justification control (JC) bytes, one negative justification opportunity (NJO) byte, and three bytes reserved for future
Interfaces for Optical Transport Networks
85
international standardization (RES). The JC bytes consist of two bits for justification control and six bits reserved for future international standardization. The OPUk payload for these mappings consists of 4 x 3808 b)^es, including one positive justification opportunity (PJO) byte. The asynchronous and bit synchronous mapping processes generate the JC, NJO, and PJO bits according to Table 3-6 and Table 3-7, respectively. The demapping process interprets JC, NJO, and PJO according to Table 3-8. Majority vote (two out of three) is used to make the justification decision in the demapping process to protect against an error in one of the three JC signals.
Table 3-6. JC, NJO, and PJO generation by asynchronous mapping process (from ITU-T Rec. G.709)
JC [78]
NJO
00
justification byte
01
data byte
data byte data byte not generated
10 11
PJO
justification byte
justification byte
Table 3-7. JC, NJO, and PJO generation by bit synchronous mapping process (from ITU-T Rec. G.709)
JC [78] :
00
NJO justification byte
PJO data byte
01 10 11
not generated
Chapter 3
86 Table 3-8. JC, NJO, and PJO interpretation (from ITU-T Rec. G.709)
JC [78]
1
NJO
PJO
00
justification byte
data byte
01
data byte
data byte
10 (Note)
justification byte
data byte
11
justification byte
justification byte
One should keep in mind that a mapper circuit does not generate the code in Table 3-8. Due to bit errors, a demapper circuit might receive this code. Moreover, it should be noted that unlike SONET/SDH, there is no "Start of Payload" indication or pointer. 3.8.1.3 Mapping a CBR2G5 signal (e.g., STS-48/STM-16) into OPUl Groups of eight successive bits (not necessarily being a byte) of the CBR2G5 signal are mapped into a Data (D) byte of the OPUl, as depicted in Figure 3-19 (a). Once per OPUl frame, it is possible to perform either a positive or a negative justification action. 3.8.1.4 Mapping a CBRIOG signal (e.g., STS-192/STM-64) into OPU2 Groups of eight successive bits (not necessarily being a byte) of the CBRIOG signal are mapped into a Data (D) byte of the OPU2 and are shown in Figure 3-19 (b). A total of 64 fixed stuff (FS) bytes are added in columns 1905 to 1920. Once per 0PU2 frame, it is possible to perform either a positive or a negative justification action. 3.8.1.5
Mapping a CBR40G signal (e.g., STS-768/STM-256) into OPU3 Groups of eight successive bits (not necessarily being a byte) of the CBR40G signal are mapped into a data (D) byte of the OPU3, as illustrated in Figure 3-19 (c). A total of 128 fixed stuff (FS) bytes are added in columns 1265 to 1280 and in columns 2545 to 2560. Once per OPU3 frame, it is possible to perform either a positive or a negative justification action.
Interfaces for Optical Transport Networks
87
Payload (Client Signal) Column Row\ 15 1
16
17
RES
JC
RES
JC
RES
JC
PSl
NJO
3824
18
PJO
(a) Mapping of 2.5 Gb/s signal mapping
Payload (Client Signal) UOI umn
\ 1
15 RES
16
17
RES
JC
RES
JC
PSI
NJO
O
..
< > OS
CN
3824
OS
JC
o fe
4
i2 B Payload (Client stgnal)
Paylc ad (Client Signal)
o 5> PJO
(b) Mapping of 10 Gb/s signal mapping
Payload (Client Signal) ^ Column 15
o
16
17
RES
JC
RES
JC
RES
JC
PSI
NJO
oo
18
PayWad (Client Signal)
o
^
o ^
3824
Payload (Client Signal)
o fe O 2^
PJO
(c) Mapping of 40 Gb/s signal mapping Figure 3-19. Mapping of CBR signal on OPUk
Payload (Client Signal)
Chapter 3
3.8.2
Similarly Valued/Formatted Fields within G.709 Frame
3.8.2.1
BIP-8
One method of monitoring the performance of the network is to monitor the received bit errors. The error detection is performed by using Bit Interleaved Parity (BIP) check. ITU-T Rec. G.707 [9] defines BIP as follows: Bit Interleaved Parity-X (BIP-X) is a method of error monitoring. With even parity an X-bit code is generated by the transmitting equipment over a specified portion of the signal in such a manner that the first bit of the code provides even parity over the first bit of all X-bit sequences in the covered portion of the signal, the second bit provides even parity over the second bit of all X-bit sequences within the specified portion, etc. Even parity is generated by setting the BIP-X bits so that there is an even number of Is in each monitored partition of the signal. A monitored partition comprises all bits which are in the same bit position within the X-bit sequences in the covered portion of the signal. The covered portion includes the BIP-X. The BIP-8 is computed over the bits in the OPUk (columns 15 to 3824) area of frame /, and is inserted in the respective ODUk or the OTUk overhead field BIP-8 overhead locations of the fi-ame / + 2, as shown in Figure 3-20.
Xco lumn Number Number i
15
•^s^^j^^m
16 17 -
BIP-^ calculated over tiie cntijpe OPUk region
2 3 4
\%-<'/^-:i+ 1
i" i+2
fc^'^-'^f^M'f/. ^
Calculated BIP-8 inserted in the respective ODUk or OTUk OH field
Figure 3-20. BIP-8 computation
BlP-8
Interfaces for Optical Transport Networks
89
3.8.2.2 Trail Trace Identifier (TTI) A Trail Trace Identifier is a single byte that is used to transport a 64 byte message (similar in functionality to the JO in SONET/SDH). The message contains a source and destination identifier used for routing the OTU signal through the network. There are bytes allocated for operator-specific use also. The 64-byte message is aligned with the OTU multiframe and is transmitted four times per multiframe (since the multiframe extends over 256 frames). The TTI contains globally unique Access Point Identifiers (API), which are used to specify the Source Access Point Identifier (SAPI) and Destination Access Point Identifier (DAPI). The APIs contain information regarding the country of origin, network operator, and administrative details. Table 3-9 illustrates the format of the source and the destination access point identifiers. These consist of a three-character international segment (IS) and a twelve-character national segment (NS). The characters are coded according to the ITU-T T.50 International Reference Alphabet —7-bit coded character set for information exchange [10]. Table 3-9. Access Point Identifier structure (taken from ITU-T Rec. G.709) IS Character # NS Character #
1 1 2 1 3 CC
cc
CC
cc cc cc
4 ICC
5 1 6 1 7 1 8 1 9 1 10 1 11 1 12 1 13 1 14 1 15
ICC 1 ICC 1 ICC ICC ICC
1 1
UAPC UAPC UAPC UAPC UAPC 1 UAPC
The Country Code (CC) under the IS column is represented by three letters uppercase alphabet as per the ISO 3166 country code; for example, France is represented by FRA or United States of America by USA. The national segment consists of two subfields, namely, the ITU carrier code (ICC) followed by a Unique Access Point Code (UAPC). The ITU-T Telecommunication Standardization Board (TSB) assigns and maintains the ICC for the network operators/service providers. The ICC consists of 1-6 left-justified characters, alphabetic or leading alphabetic with trailing numeric. The UAPC is selected by the organization to which the CC and the ICC have been assigned. The selection of the UAPC has to be made such that its uniqueness is ensured and guaranteed. The UAPC consists of 6-11 characters with a trailing NUL character, thereby making it a 12-character national segment.
90
Chapter 3
3.8.3
ODUk Overhead and Processing
The ODUk {k=' 1,2,3) frame structure is shown in Figure 3-21. It is organized in an octet-based block frame structure with four rows and 3824 columns.
\ c^Column o l u Number l o w \ i1 RowX
14 15 16 17 . t Overhead
^
iSL ^
a :1^^^ SjPSjf'i
^
OPUk Payload (4 X 3808 bytes)
'Jsed of FA and OTUk Overhead
Figure 3-21. ODUk frame structure
The three main areas of the ODUk frame are • • •
The OTUk area The ODUk overhead area The OPUk area
Columns 1 to 14 of rows 2-4 are dedicated to the ODUk overhead area, columns 1 to 14 of row 1 are reserved for frame alignment and OTUkspecific overhead, and columns 15 to 3824 of the ODUk are dedicated to the OPUk area. ODUk OH information is added to the ODUk information payload to create an ODUk. This includes information for maintenance and operational functions to support optical channels. The ODUk OH consists of portions dedicated to the end-to-end ODUk path and to six levels of tandem connection monitoring. The ODUk path OH is terminated where the ODUk is assembled and disassembled. The TC OH is added and terminated at the source and sink of the corresponding tandem connections, respectively. The term ODUk may or may not include the bytes in row 1. If one talks about the ODUk rate, then the bytes in row 1 are included. However, if one talks about the ODUk OH, then the bytes in row 1 are not included. In the functional model (G.798), the ODUk is considered to include row 1, but with all the bytes in row 1 equal to zero (Sections 14 and 14.3.1.1 of [5]). The ODUk overhead bytes location is shown in Figure 3-22. Figures 3-23 and Figure 3-24 expand the PM and the TCMi overhead fields.
Interfaces for Optical Transport Networks
91
Column Num 3er
^
1 1 II 2 II 3 II 4 II 5 II 6 II 7 II 8 II ' I M O I I " II ' M l 13 II ! l l l 15 11..^. J illlliilitt^^^^ III lllllll 1 •i|iji|||
illl
Ci
s O
W
2
RES
3
TCM3
4
GCCl
ITCM ACT
TCM5
TCM6 TCM2
GCC2
TCM4
TCMl
PM
APS/PCC
RES: TCM: ACT: FTFL: PM: EXP: GCC: APS: PCC:
FTFL EXP
RES
Reserved for future international standardization Tandem Connection Monitoring Activation/deactivation control channel Fault Type «fe Fault Location reporting channel Path Monitoring Experimental General Communication Channel Automatic Protection Switching coordination channel Protection Communication Control channel
Figure 3-22. ODUk overhead
Path Monitoring Overhead TTI
BIP-8 ^
| ; I 2 I 5 I ¥ \J\ 6 1 7 ( 5 1 SAPI 15 16
DAPI 31 32
Operator Specific 63
1 ^^^
[eoj
SAPI: DAPI: TTI: BIP-8: BEI: BDI: STAT: PSI: PT:
STAT 1
Source Access Point Identifier Destination Access Point Identifier Trail Trace Identifier Bit Interleaved Parity—level 8 Backward Error Indication Backward Defect Indication Status Payload Structure Identifier Payload Type
Figure 3-23. ODUk path monitoring overhead (from ITU-T Rec. G.709)
Chapter 3
92 (TCMi) Overhead 2
TTIi
BIP-8i
|/|2|5|4 SAPI 15 16
DAPI 31 32
Operator Specific 63
BEIi/BIAEi
SAPI: DAPI: TTI: BIP-8: BEI: BDI: STAT: PSI: PT:
0 ICQJ
^|7|<9| STATi
Source Access Point Identifier Destination Access Point Identifier Trail Trace Identifier Bit Interleaved Parity—level 8 Backward Error Indication Backward Defect Indication Status Payload Structure Identifier Payload Type
Figure 3-24. ODUk tandem connection monitoring #i overhead (from ITU-T Rec. G.709)
3.8.3.1 Path Monitoring (PM) Byte Descriptions Path monitoring overhead bytes, as depicted in Figure 3-23, consist of a Trail Trace Identifier (TTI), a Bit Interleaved Parity (BIP-8), and a third byte that consists of Backward Error Indication and Backward Incoming Alignment Error (BEI/BIAE), Backward Defect Indicator (BDI), and Status (STAT) fields. The Trail Trace Identifier is a single byte used within the context of the path within the optical transport network, as depicted in Figure 3-13. The multi-frame message format of the TTI is elaborated in Section 3.8.2.2 and is illustrated in Figure 3-24. The Bit Interleaved Parity (BIP-8) is used in the PM region of the G.709 frame, as elaborated earher in Section 8.1 (Figure 3-18), and is one byte that is used for error detection. The BIP-8 byte is computed over the entire OPUk region and is inserted two frames later in the PM's BIP-8 field, as discussed in Section 3.8.2.1. Backvv^ard Defect Indication (BDI) is defined to convey the "Signal Fail" status, detected at the Path Terminating Sink Function, to the upstream
Interfaces for Optical Transport Networks
93
node. This signal is created by the consequent action of aBDI (G.798/14.2.1.2). The actual defect equations are as follows: RI_BDI = aBDI ^ CI_SSF or dAIS or dOCI or dLCK or dTIM It should be noted that dAIS, dOCI, dLCK, and dTIM are all detected at the PM layer. CI_SSF = AI_TSF at the TCM layer AI_TSF = aTSF ^ CI_SSF or (dAIS or dLTC or dOCI or dLCK or (dTIM and not TIMActDis)) and TCMCI_Mode == OPERATIONAL dAIS, dLTC, dOCI, dLCK, and dTIM are TCM defects CI_SSF = aSSF^ dAIS or dLOF or dLOM (0.798/123.1.2); dAIS here is the SM AIS RI: BDI: CI: SSF: AIS: OCI: LCK: TIM: AI: TSF: LOF: LOM:
3.8.3.2
Remote Information Backward Defect Indication Characteristic Information Server Signal Fail Alarm Indication Signal defect Open Connection Indication defect Locked defect TTI mismatch detection Adapted Information Trail Signal Fail Loss of Frame Loss of Multiframe
Backward Error Indication and Backward Incoming Alignment Error (BEI/BIAE)
The BEI/BIAE signal is used to convey in the upstream direction the count of interleaved-bit blocks that have been detected in error by the corresponding ODUk path monitoring sink using the BIP-8 code. This count has nine legal values, namely, 0-8 errors. The remaining seven possible values represented by these four bits can only result from some unrelated condition and are interpreted as zero errors, as shown in Table 3-10.
Chapter 3
94 Table 3-10. ODUk PM BEI interpretation (from ITU-T Rec. G.709)
ODUk PM BEI bits 1234
BIP violations
0000
0
0001
1
0010
2
0011
3
0100
4
0101
5
Olio
6
0111 1000 1001 to n i l
7
1 1 1 3.8.3.3
8 0
Path Monitoring Status (STAT)
The three most significant bits of the PM status octet indicate the presence of a maintenance signal, as illustrated in Table 3-11.
Table 3-11. ODUk PM status interpretation (from ITU-T Rec. G.709)
1
PM byte 3, bits 678
status
000
Reserved for future international standardization
001
Normal path signal
010
Reserved for future international standardization
on
Reserved for future international standardization
100
Reserved for future international standardization
|
101
Maintenance signal: ODUk-LCK
!
110
Maintenance signal: ODUk-OCI
111
Maintenance signal: ODUk-AIS
Interfaces for Optical Transport Networks
3.8.4
95
Tandem Connection Monitoring (TCM)
There are six TCMs. These can be nested or overlapping, as illustrated in Figures 3-10 and 3-11. Trail Trace Identifier: The TTI is a single byte used within the context of the respective tandem connection, as depicted in Figures 3-10 and 3-11. The multiframe message format of the TTI is elaborated in Section 3.8.2.2 and is illustrated in Figure 3-24. BIP-8: The same value that was calculated for the BIP-8 in the PM BEP8 field is also used in the TCMi BIP-8 field. The BIP-8 is only overwritten at the start of a tandem connection. Any existing TCM is not overwritten. The BIP-8 byte is computed over the entire OPUk region and is inserted two frames later in the respective TCMi's BIP-8 field, as discussed in Section 3.8.2.1. 3.8.4.1
Backward Defect Indication (BDI)
Backward Defect Indication (BDI) is defined to convey the "Signal Fail" status, detected at the Path Terminating Sink Function, to the upstream node. This signal is created by the consequent action of aBDI at the TCM level (G.798/14.5.1.1.2). The actual defect equations are as follows: RI_BDI = aBDI^
(CI_SSF or dAIS or dLTC or dOCI or dLCK or (dTIM and not TIMActDis)) and TCMCIJAode ^^TRANSPARENT) dAIS and dTIM are TCM defects CI_SSF = aSSF ^
dAIS or dWF or
dWM{G.79SA23A.2)
dAIS here is the SM AIS RI: BDI: CI: SSF: AIS: LTC: OCI: LCK: TIM: LOF: LOM:
Remote Information Backward Defect Indication Characteristic Information Server Signal Fail Alarm Indication Signal defect Loss of Tandem Connection Open Connection Indication defect Locked defect TTI mismatch detection Loss of Frame Loss of Multiframe
Chapter 3
96
3.8.4.2 Backward Error Indication and Backward Incoming Alignment Error (BEI/BIAE) The BEI/BIAE signal is used to convey, in the upstream direction, the count of interleaved-bit blocks that have been detected as being in error by the corresponding ODUk tandem connection monitoring sink using the BIP-8 code. It is also used to convey in the upstream direction an incoming alignment error (lAE) condition that is detected in the corresponding ODUk tandem connection monitoring sink in the lAE overhead. As shown in Table 3-12, during an lAE condition the code "1011" is inserted into the BEI/BIAE field and the error count is ignored. Otherwise, the error count (0-8) is inserted into the BEI/BIAE field. The remaining six possible values represented by these four bits can only result from some unrelated condition and are interpreted as zero errors and BIAE not active. Table 3-12. ODUk TCM BEI interpretation (from ITU-T Rec. G.709)
ODUk TCM BEI bits 1234
BIAE
BIP violations
0000
false
0
0001
false
1
0010
false
2
0011
false
3
0100
false
4
0101
false
5
Olio
false
6
0111
false
7
1000
false
8
1001,1010
false
0
1011
true
0
llOOtolUl
false
0
1
3.8.4.3
TCM Monitoring Status (STAT)
For each tandem connection monitoring field, three bits are defined as status bits (STAT). These indicate the presence of a maintenance signal, if there is an incoming alignment error at the source TCM, or if there is no source TCM active, as illustrated in Table 3-13.
Interfaces for Optical Transport Networks
97
Table 3-13. ODUk TCM status interpretation (from ITU-T Rec. G.709)
TCM byte 3, bits 678
[
3.8.4.4
Status
000
No source TC
001
In use without lAE
010
In use with lAE
on
Reserved for future international standardization
100
Reserved for future international standardization
101
Maintenance signal: ODUk-LCK
no
Maintenance signal: ODUk-OCI
111
Maintenance signal: ODUk-AIS
Tandem Connection Monitoring ACTivation/deactivation (TCM-ACT)
The definition of this term is for further study. 3.8.4.5
General Communication Channels (GCCl, GCC2)
The GCCl and the GCC2 bytes are primarily used in providing a general communication channel between the ODUk termination points. Within the context of the data communication network as defined and structured in G.7712A^.1703 [11], the GCC bytes can be used to provide Embedded Control Channel. These bytes can carry various operations, management, administration and provisioning information. For example, a connection table for a digital cross-connect or software upgrades can be sent via these bytes from a remote management station. 3.8.4.6
Automatic Protection Switching and Protection Communication Channel (APS/PCC)
Up to eight levels of nested APS/PCC signals may be present in this field. The APS/PCC bytes in a given frame are assigned to a dedicated level depending on the value of MFAS, as shown in Table 3-14:
Chapter 3
98
Table 3-14. Multiframe to allow separate APS/PCC for each monitoring level (from ITU-T Rec. G.709)
MFAS bit 678
APS/PCC channel applies to
:
000
ODUk Path
1
001
ODUkTCMl
010
0DUkTCM2
on
ODUkTCMS
100
0DUkTCM4
101
ODUkTCMS
110
0DUkTCM6
111
ODUk SNC/I APS
For linear protection schemes, the bit assignments for these bytes and the bit-oriented protocol are given in Recommendation G.873.1. Bit assignment and byte oriented protocol for ring protection schemes are for further study.
3.8.4.7
Fault IVpe and Fault Location reporting communication channel (FTFL)
FTFL has been defined in G.709 to aid in fault location. The actual implementation is still under discussion in the ITU-T and thus won't be discussed here.
3.9.
OTUK OVERHEAD AND PROCESSING
The OTUk {k= 1,2,3) frame structure is based on the ODUk frame structure and extends it with a forward error correction (FEC). A total of 256 columns are added to the ODUk frame for the FEC, and the overhead bytes in row 1, columns 8 to 14, of the ODUk overhead are used for OTUkspecific overhead, resulting in an octet-based block frame structure with four rows and 4080 columns. The bit rates of the OTUk signals are defined in Table 3-1. The OTUk FEC contains the Reed-Solomon RS(255,239) FEC codes. If no FEC is used, fixed stuff bytes (all-Os pattern) are inserted. The RS(255,239) FEC code is elaborated in detail in Annex A/G.709.
Interfaces for Optical Transport Networks
99
OTUk OH information is part of the OTUk signal structure and is depicted in Figure 3-25. It includes information for operational functions to support the transport via one or more optical channel connections. The OTUk OH is terminated where the OTUk signal is assembled and disassembled. ^Column Number
rrnrm TF m RES BEI
FA:
Frame Aligmnem
MFAS:Multiframe Aligimient Signal SM: Section Monitoring GCC: General Communication Charmel RES: Reserved for future international standardization
TTI: SAPI: DAPI BIP-8 BEI: BDI: BEI: lAE: BIAE
Trail Trace Identifier Source Access Point Identifier Destination Access Point Identifier Bit Interleaved Parity—level 8 Backward Error Indication Backward Defect Indication Backward Error Indicator Incoming Alignment Error Backward Incoming Aligrmient Error
s
Operator Specific
Figure 3-25. OTUk Overhead (from ITU-T G.709)
3.9.1 Scrambling The OTUk signal needs sufficient bit timing content to allow a clock to be recovered. A suitable bit pattern, which prevents a long sequence of Is or Os, is provided by using a scrambler without adding any additional overhead to the overall OTUk frame. The operation of the scrambler is functionally identical to that of a frame synchronous scrambler of sequence length 65535 operating at the OTUk rate. The generating polynomial i s l + x + x + x + X . Figure 3-26 shows a functional diagram of the frame synchronous scrambler.
r-^f
©^
&<
^y-
(-•ID Q[-WD Q W D QHHD Q W D Q W D Q W D Q|-^D Q W D Q|-HD QW"^ ^\M° QHH° ^\M° ^ W ^ Q\M'^ Q|—'-K+)—• I II II II II II II II II II II II II II II II I ^-^Q.Scrambled ^rir rWCLK I iMCLK I IWCLK iHcLK I iHcLK [WcLK I WcLK I iMCLK I jWCLK I jHCLK jWcLK iWcLK iMcLK iWcLK I WcLK I |W( Data Out OTUk Clock
I ^1 ^1 ^1 y^l ^1 ^1 ^1 y^l ^1 ^1 ^1 ^1 ^1 ^1 ^1
OTUk MSB of MFAS byte
Figure 3-26. Frame synchronous scrambler
Chapter 3
100
The scrambler is reset to "FFFF" (HEX) on the most significant bit of the byte following the last framing byte in the OTUk frame, i.e., the MSB of the MFAS byte. This bit, and all subsequent bits to be scrambled, are added modulo 2 to the output from the x position of the scrambler. The scrambler runs continuously throughout the complete OTUk frame. To ensure frame acquisition at the receiving end, the framing bytes (FAS) of the OTUk overhead are not scrambled. Scrambling is performed after the FEC check bytes computation and insertion into the OTUk signal.
3.9.2
Frame Alignment Overhead
3.9.2.1
Frame Alignment Signal (FAS)
A six-byte OTUk-FAS signal as shown in Figure 3-27 is defined in row 1, columns 1 to 6, of the OTUk overhead. OAl is "1111 0110." OA2 is "0010 1000."
FAS OH Byte I 1
••
4
"I
OAl
FAS OH Byte 2 8 1
=•
4 5 OAl
.1 8
FAS OH Byte 3 1 2 3 4 5 617 8 OAl
FAS OH Byte 5
FAS OH Byte 4
li
5 6 7 8 0A2
1
'
4 5 0A2
•'
FASOHByteS 8 1 2 3 4 5
"
1 8
0A2
Figure 3-27. Frame alignment signal overhead structure (from ITU-T G.709)
3.9.2.2
Multiframe Alignment Signal (MFAS)
Some of the OTUk and ODUk overhead signals span multiple OTUk/ODUk frames. A single Multiframe Alignment Signal (MFAS) byte is defined in row 1, column 7, of the OTUk/ODUk overhead, as depicted in Figure 3-25. The value of the MFAS byte will be incremented each OTUk/ODUk frame and provides as such a 256-frame multiframe. Individual OTUk/ODUk overhead signals use this central multiframe to lock their 2-frame, 4-frame, 8-frame, 16-frame, 32-frame, etc., multiframes to the principal frame (depicted in Figure 3-28).
Interfaces for Optical Transport Networks
•m \
101
MFASOHByte
3
4
5
6
7
8
Equivalent Decimal Value
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 1 1
0 0 1 1 0 0
0 1 0 1 0 1
0 1 2 3 4 5
1 1 1 1 0 0
1 1 1 1 0 0
1 1 1 1 0 0
1 1 1 1 0 0
1 1 1 1 0 0
1 1 1 1 0 0
0 0 1 1 0 0
0 1 0 1 0 1
252 253 254 255 0 1
Figure 3-28. Multiframe alignment signal overhead
3.9.3
Section Monitoring Byte Descriptions
The Section Monitoring overhead bytes within the OTUk consist of TTI, BIP-8, and the third byte, consisting of subfields of BEI/BIAE, BDI, AEI and RES. The Trail Trace Identifier is a single byte used within the context of the section monitoring within the optical transport network, as depicted in Figure 3-13. The multiframe message format of the TTI is elaborated in Section 8.2.2 and is illustrated in Figure 3-29. Bit Interleaved Parity (BIP-8), used in the PM region of the G.709 frame as elaborated earher in Section 3.8.1 (Figure 3-18), is one byte that is used for error detection. The BrP-8 byte is computed over the entire OPUk region and is inserted two frames later in the PM's BIP-8 field, as discussed in Section 3.8.2.1.
Chapter 3
102 Path Monitoring Overhead
JTTI
BIP-8
y
SAPI
DAPI
Operator Specific
nwkmnmmnm BEI/BIAElQ
TTI: SAPI: DAPI: BIP-8: BBI: BIAE: BDI: lAE: RES:
RES
Trail Trace Identifier Source Access Point Identifier Destination Access Point Identifier Bit Interleaved Parity—level 8 Backward Error Indication Backward Incoming Alignment Error Backward Defect Indication Incoming Alignment Error Reserved
63 Figure 3-29. OTUk section monitoring overhead (fi-om ITU-T Rec. G.709)
Note: The OPUk includes the Justification Bytes, and thus an OTN signal can not be retimed v^ithout demapping back to the client signal. 3.9.3.1
Backward Defect Indication (BDI)
The Backward Defect Indication is defined to convey the "Signal Fail" status, detected at the Section Terminating Sink Function, to the upstream node. This signal is created by the consequent action aBDI at the SM level (Section 13.2.1.2 of G.798). The actual defect equations are as follows: RI_BDI = aBDI = CIJSF or (dTIM and not TIMActDis) CI_SSF = aSSF = dAIS or dWF or ^ZOM (G.798.12.3.1.2) dAIS=OTUk-AIS(G.79S.6.2.63A) J77M = 0.798.6.2.2.1 RI: BDI: CI: SSF: AIS: LCK: TIM: LOF:
Remote Information Backward Defect Indication Characteristic Information Server Signal Fail Alarm Indication Signal defect Locked defect TTI Mismatch detection Loss of Frame
Interfaces for Optical Transport Networks LOM:
103
Loss of Multiframe
3.9.3.2 Backward Error Indication and Backward Incoming Alignment Error (BEI/BIAE) The BEI/BIAE signal is used to convey, in the upstream direction, the count of interleaved-bit blocks that have been detected in error by the corresponding OTUk section monitoring sink using the BIP-8 code. It is also used to convey, in the upstream direction, an incoming alignment error (lAE) condition that is detected in the corresponding OTUk section monitoring sink in the lAE overhead. During an lAE condition the code "1011" is inserted into the BEI/BIAE field and the error count is ignored. Otherwise, the error count (0-) is inserted into the BEI/BIAE field. The remaining six possible values represented by these four bits can only result from some unrelated condition and are interpreted as zero errors (Table 3-15) and BIAE not active. Table 3-15. OTUk SM BEI inteqjretation (from ITU-T Rec. G.709)
OTUk SM BEI/BIAE bits 1234
BIAE
BIP violations
0000
false
0
0001
false
1
0010
false
2
0011
false
3
0100
false
4
0101
false
5
Olio
false
6
0111
false
7
1000
false
1001,1010
false
8 0
1011
true
0
llOOtollll
false
0
104 3.9.3.3
Chapters Incoming Alignment Error (lAE)
A single-bit incoming alignment error (lAE) signal is defined to allow the ingress point to inform its peer egress point that an alignment error in the incoming signal has been detected. lAE is set to " 1 " to indicate a frame alignment error; otherwise, it is set to "0". The egress point may use this information to suppress the counting of bit errors, which may occur as a result of a frame phase change of the OTUk at the ingress of the section. G.798 shows an incoming alignment error being detected on the source side (Section 13.3.1.1 of G.798). The consequent action (AI_IAE) is then used to set the SM lAE bit (Section 13.2.1.1 of G.798). Practically, it is detected on the sink side and then passed to the source side. However, if one is going through an ODUk switch, then it would need to be detected on the source side.
3.9.4
General Communication Channel 0 (GCCO)
The GCCO bytes are primarily used in providing a general communication channel between the OTUk termination points. Within the context of the data communication network as defined and structured in G.7712/Y.1703 [11] the GCC bytes can be used to provide an Embedded Control Channel. These bytes can carry various operations, management, administration, and provisioning information. For example, a connection table for a digital cross-connect or software upgrades can be sent via these bytes from a remote management station.
3.10.
ODUK MULTIPLEXING
Multiplexing in the OTN domain, as defined in G.709, supports several lower-speed channels into higher-speed channels. For example, four ODUl's can be multiplexed to an ODU2, or up to sixteen ODUl's or four ODU2's can be multiplexed to an 0DU3. It is also possible to mix ODUl's and 0DU2's in an ODU3. For 0DU2 to 0DU3 multiplexing, there have to be two positive stuff opportunities. For ODUl to ODU3 multiplexing, there is a fixed stuff in column 119. Thus the stuffing for multiplexing is different from the stuffing for mapping. In order to understand this intricacy, it is necessary to examine the data rates.
Interfaces for Optical Transport Networks
105
3.10.1 Multiplexing Data Rates 3.10.1.1
ODUl to ODU2 Justification Rate
For the case of multiplexing ODUl to ODU2: From Table 3-2, we see that the ODUl rate is equal to 239/238 * OC48 +- 20 ppm = 2,498,775,126 +- 49,976 b/s We can put data in the "Fixed Stuff bits of the 0PU2 payload that is shown in Figure 3-20b. Thus the 0PU2 payload is now 238/239 * 0DU2 rate, or 238/239 * 239/237 * OC192 +- 20 ppm. The 0PU2 payload is time sliced for the four ODUl's. Thus each ODUl has 238/237 * OC48 +- 20 ppm = 2,498,819,241 +- 49,976 b/s. The worst-case frequency difference is then (2,498,819,241 + 49,976) - (2,498,775,126 - 49,976) = 144,067 b/s or 57.65 ppm Thus we have to account for a data-rate mismatch of 144,067 b/s by stuffing. The stuffing is done on a multiframe basis. Each timeslot is stuffed once per four frames. The stuffing rate is defined as (stuff bits/frame)/(bits/frame) * (data rate) = (8/4)/(3824*4*8)*(238/237*OC192) = 163,364 b/s = 65 ppm. Therefore, in the worst case, there are enough data bytes coming in to match the outgoing rate. 3.10.1.2
ODU2 to ODU3 Justification Rate
For the case of multiplexing ODU2 to 0DU3: From Table 3-2, it can be see that the ODU2 rate is equal to 239/237 * OC192 +- 20 ppm = 10,037,273,930 +- 200,745 b/s. We can put data in the "Fixed Stuff bits of the OPU3 payload that is shown in Figure 3-20c. Thus the OPU3 payload is now 238/239 * 0DU3 rate, or
106
Chapters
238/239 * 239/236 * OC768 +- 20 ppm. The OPU3 payload is time sliced for the four ODU2's. Thus each 0DU2 has 238/236 * OC192 +- 20 ppm = 10,037,629,830 +- 200,753 b/s The worst-case frequency difference is then (10,037,629,830 + 200,753) - (10,037,273,930 - 200,745) = lSl,2>n b/s or +75 ppm and (10,037,629,830 - 200,753) - (10,037,273,930 + 200,745) = -45,598 b/s or -4.5 ppm Thus we have to account for a data-rate mismatch of 757,398 b/s by stuffing. The stuffing is done on a multiframe basis. This is more than the +- 65 ppm that the normal scheme can accommodate. Thus, each timeslot has two positive stuff opportunities and one negative stuff opportunity per four frames. The stuffing rate is is defined as (stuff bits/frame) * (bits/sec)/(bits/frame) = (16/4)/(3824*4*8)*(238/236*OC768) = 1,312,452 b/s = 130 ppm Therefore, in the worst case, there are enough data bytes coming in to match the outgoing rate. 3.10.1.3
ODUl to ODU3 Justification Rate
For the case of multiplexing ODUl to ODU3, it can be seen from Table 3-2 that the ODUl rate is equal to 239/238 * OC48 +- 20 ppm = 2,498,775,126 +- 49,976 b/s We can put data in the "Fixed Stuff bits of the OPU3 payload that is shown in Figure 3-20c. Thus the OPU3 payload is now 238/239 * ODU3 rate, or 238/239 * 239/236 * OC768 +- 20 ppm
Interfaces for Optical Transport Networks
107
The OPU3 payload is time sliced for the 16 ODUl's. Column 119 of the time-sliced ODU3 is fixed stuff. An all-Os pattern is inserted in the fixed stuff bytes. Thus each ODUl has 237/239 * 239/236 * OC48 +- 20 ppm = 2,498,863,728 +- 49,977 b/s The worst-case frequency difference is then (2,498,863,728 + 49,977) - (2,498,775,126 - 49,976) = 188,555 b/s or 75.46 ppm Therefore, we have to account for a data-rate mismatch of 188,555 b/s by stuffing. The stuffing is done on a multiframe basis. Once again, each timeslot has two positive stuff opportunities and one negative stuff opportunity per 16 frames. The stuffing rate defined as (stuff bits/frame) * (bits/sec)/(bits/frame) = (16/16)/(3824*4*8)*(238/236*OC768) = 328,113 b/s = 130 ppm Therefore, in the worst case, there are enough data bytes coming in to match the outgoing rate.
3.10.2 4 X ODUl to ODU2 Multiplexing 3.10.2.1
4 X ODUl to ODU2 Multiplexing Structure
The 0PU2 is divided in a number of Tributary Slots (TSs) and these Tributary Slots are interleaved within the OPU2, as depicted in Figure 3-30. The bytes of an ODUl input are mapped into one of four OPU2 Tributary Slots. The bytes of the Justification Overhead to adapt the asjoichronous ODUl to the 0DU2 clock rate are mapped into the 0PU2 OH area. An 0PU2 Tributary Slot occupies 25% of the OPU2 Payload area. It is a structure with 952 columns by 4 rows. The four 0PU2 TSs are byte interleaved in the 0PU2 payload area. It is important to note that the ODUl frame repeats every four 0DU2 frames! One of the implications of this point is that the FAS bytes in the ODUl frame could cause false locking of the 0DU2 frame. This is not supposed to be a problem according to contributions to the ITU. However, the FAS bytes can't be removed because there is no standard on where the ODUl frame starts. Thus the ODUl FAS bytes are needed to frame the recovered ODUl signal. The OTUl OH (SM, GCCO, and RES) is set to all Os, as illustrated in Figure 3-31.
108
Chapter 3
r - CNJ CO - ^ CNJ O J (Nl C S
CO CO 00 00 00 CO CO CO
bits R o w \ 1 ''
1 2
0PU2 Paytoad (4x 3808 bytes)
11 3 4 1 2
OPU2 Payload (4x 3808 bytes)
00 3 4 1 2
0PU2 Payload (4x 3808 bytes)
01 3 4 1
OPU2 Payload (4x 3808 bytes)
2 3
10
4 1 2 1^
0PU2 Payload (4x 3808 bytes)
3 4 1 2
00
0PU2 Payload (4x 3808 bytes)
3 4
Figure 3-30. 0PU2 tributary slot allocation (from ITU-T Rec. G.709)
^Column Number 1 14 15 16 17 Fixed Stuff FA OH (an-Os) 2
KOW X
Number
3 4
ODUj Overhead
Figure 3-31. Extended ODUj frame structure (from ITU-T Rec. G.709)
3824
Interfaces for Optical Transport Networks 3.10.2.2
109
4 X ODUl to ODU2 Justification Structure
The Justification Overhead (JOH), consisting of Justification Control (JC) and Negative Justification Opportunity (NJO) signals of the four OPU2 TSs, is located in the overhead area, column 16 of rows 1 to 4. The JOH is assigned to the related tributary slots on a per-frame base. The JOH for a tributary slot is available once every four frames. A four-frame multiframe structure is used for this assignment. This multiframe structure is locked to bits 7 and 8 of the MFAS byte, as shown in Table 3-16 and Figure 3-32. The PJOl and PJ02 bytes are in the ODUl payload. Figure 3-32 shows how the bytes are distributed. It should be noted that there are two PJO bytes and only one NJO byte. This is because the timeslot provides more capacity than is needed. Table 3-16. 0PU2 Justification OH tributary slots (from ITU-T Rec. G.7Q9)
MFAS Bits 7 & 8
JOHTS
00
1
01
2
10
3
11
4
. Column Number
o 01
o
Q o
I O I
l§ I I jo
Pi
10 11
0000 o l Sf 0001
Q
§
I
ISI I
I Is I I I Is
ol
§I Pi I
Figure 3-32. OPUk Multiplex Overhead
I I
I
no 3.10.2.3
Chapters OPU2 Payload Structure Identifier (PSI)
Byte 0 is defined as the Payload Type and is equal to 0x20. Byte 1 is reserved. Bytes 2-17 are the "Multiplex Structure Identifier." Bytes 18-255 are reserved. A total of 239 bytes are reserved in the OPUk PSI for future international standardization. These bytes are located in PSI[1] and PSI[18] to [PSI255] of the OPUk overhead. These b)1;es are set to all zeros.
3.10.2.4
OPU2 Multiplex Structure Identifier (MSI)
The Multiplex Structure Identifier (MSI) overhead, which encodes the ODU multiplex structure in the OPU, is located in the mapping-specific area of the PSI signal (PSI[2]... PSI[17]). The MSI indicates the content of each tributary slot (TS) of an OPU2. The generic coding for each TS is shown in Figure 3-33. One byte is used for each TS. • Bits 1 and 2 indicate the ODU type transported in the TS. • Bits 3 to 8 indicate the tributary port of the ODU transported. This is of interest in the case of flexible assignment of ODUs to tributary slots (e.g., ODU2 into OPU3). In the case of fixed assignment, the tributary port number corresponds to the tributary slot number.
PSI[I+1] ODU Type
1
00: ODUl 01: 0DU2 10: 0DU3 11: RES
Tributary Port Number
Tributary Slot Number
1
00 0000 Tributary Port 1 00 0001 : Tributary Port 2 00 0010 : Tributary Port 3 00 0011 : Tributary Port 4 • • • • 00 n i l : Tributary Port 16 Figure 3-33. Generic MSI coding
For the four OPU2 tributary slots four bytes of the PSI are used, as shown in Figure 3-34.
Interfaces for Optical Transport Networks
• •
111
The ODU type is fixed ODUl The tributary port # indicates the port number of the ODUl that is being transported in this TS; the assignment of ports to tributary slots is fixed; the port number equals the tributary slot number
The remaining 12 bytes of the MSI field (PSI[6] to PSI[17]) are unused. They are set to 0 and ignored by the receiver.
PSI [2]
00
00 0000
Tributary Slot Number 1
PSI [3]
00
00 0001
Tributary Slot Number 2
PSI [4]
00
00 0010
Tributary Slot Number 3
PSI [5]
00
00 0011
Tributary Slot Number 4
Figure 3-34. 0PU2-MSI coding
3.10.2.5
Frequency Justification
The mapping of ODUl signals (with up to ±20 ppm bit-rate tolerance) into the ODU2 signal is performed as an asynchronous mapping. The 0PU2 signal for the multiplexed ODUl structure is created from a locally generated clock, which is independent of the ODUl client signals. The ODUl signal is extended with Frame Alignment Overhead and an all-Os pattern in the OTUl overhead field; The extended ODUl signal is adapted to the locally generated 0DU2 clock by means of an asynchronous mapping with a -1/0/+1/+2 positive/negative/zero (pnz) justification scheme. The asynchronous mapping process generates the JC, NJO, PJOl and PJ02, according to Table 3-16. The demapping process interprets JC, NJO, PJOl, and PJ02, according to Table 3-16. Majority vote (two out of three) is used to make the justification decision in the demapping process to protect against an error in one of the three JC signals. The value contained in NJO, PJOl, and PJ02 when they are used as justification bytes is all-Os. The receiver is required to ignore the value contained in these bytes whenever they are used as justification bytes.
Chapter 3
112
Note: based on the calculations for ODUl to 0PU2 mapping (see Section 3.3.10.1.1), there should never be a need to do a "Double Positive Justification." Table 3-17. JC, NJO, PJOl, and PJ02 generation and interpretation (from ITU-T Rec. G.709)
JC [7,8]
NJO
PJOl
PJ02
Interpretation
00
justification byte
data byte
data byte
no justification (0)
01
data byte
data byte
data byte
1 ^^
justification byte
justification byte
justification byte
(-1) double positive justification (+2)
11
justification byte
justification byte
data byte
positive justification
j negative justification
1
(+1)
3.10.3 ODU1/ODU2 to ODU3 Multiplexing 3.10.3.1
ODU1/ODU2 to ODU3 Multiplexing Structure
The OPU3 is divided into a number of Tributary Slots (TSs) and these Tributary Slots are interleaved within the 0PU3, as depicted in Figure 3-35. The bytes of an ODUl or 0DU2 input are mapped into one or four OPU3 Tributary Slots. The bytes of the Justification Overhead to adapt the asynchronous ODUl or 0DU2 to the 0DU3 clock rate are mapped into the 0PU3 OH area. An 0PU3 Tributary Slot occupies 6.25% of the 0PU3 payload area. It is a structure with 238 columns by 4 rows. The sixteen 0PU3 TSs are byte interleaved in the OPU3 Payload area. It is important to note that the ODUl frame repeats every sixteen ODU3 frames! One of the implications of this point is that the FAS bytes in the ODUl frame could cause false locking of the ODU3 firame. This is not supposed to be a problem according to contributions to the ITU. However, the FAS bytes can't be removed because there is no standard on where the ODUl frame starts. Thus the ODUl FAS bytes are needed to frame the recovered ODUl signal. The OTUl OH (SM, GCCO, and RES) is set to all Os and is illustrated in Figure 3-36.
Interfaces for Optical Transport Networks
113
Column MFAS bits Rovi^ 1 5678
OPUS Payload (4 X 3808 bytes)
1111
0PU3 Payload (4 X 3808 bytes)
0PU3 Payload (4 X 3808 bytes)
0001
0PU3 Payload (4 X 3808 bytes)
0PU3 Payload (4 X 3808 bytes)
1111
0PU3 Payload (4 X 3808 bytes)
Figure 3-35. 0PU3 tributary slot allocation (from ITU-T Rec. G.709)
\Co lu Column Number Row 14 15 Number IFAOH^ Fixed Sftifr (sH-Os)
ODUj Overhead
16 17 .
OPUj Area (4x3810 bytes)
Figure 3-36. Extended ODUj frame structure (from ITU-T Rec. G.709)
Chapter 3
114
3.10.3.2
ODU1/ODU2 to ODU3 Justification Structure
The Justification Overhead (JOH), consisting of Justification Control (JC) and Negative Justification Opportunity (NJO) signals of the 16 OPU3 TSs, is located in the overhead area, column 16 of rows 1 to 4. The JOH is assigned to the related tributary slots on a per-frame base. The JOH for a tributary slot is available once every 16 frames. A 16-frame multiframe structure is used for this assignment. This multiframe structure is locked to bits 5-8 of the MFAS byte, as shown in Table 3-18 and Figure 3-32. It should be noted that there are two PJO bytes and only one NJO byte. This is because the timeslot provides more capacity than is needed. Table 3-18. OPUS Justification OH tributary slots (from ITU-T Rec. G.709)
3.10.3.3
MFAS bits 5678
JOHTS
MFAS bits
0000
1
1000
9
0001
2
1001
10
0010
3
1010
11
0011
4
1011
12
0100
5
1100
13
0101
6
1101
0110
7
1110
14 15
0111
8
1111
16
JOHTS
5678
OPU3 Payload Structure Identifier (PSI)
Byte 0 is defined as the Payload Type and is equal to 0x20. Byte 1 is reserved. Bytes 2-17 are the "Multiplex Structure Identifier." Bytes 18-255 are reserved. A total of 239 bytes are reserved in the OPUk PSI for future international standardization. These bytes are located in PSI[1] and PSI[18] to PSI[255] of the OPUk overhead. These bytes are set to all zeros.
Interfaces for Optical Transport Networks 3.10.3.4
115
OPU3 Multiplex Structure Identifier (MSI)
The MSI overhead, which encodes the ODU multiplex structure in the OPU, is located in the mapping specific area of the PSI signal (PSI[2]... PSI[17]). The MSI indicates the content of each tributary slot (TS) of an OPUS. The generic coding for each TS is shown in Figure 3-33. One byte is used for each TS. • Bits 1 and 2 indicate the ODU tj^e transported in the TS. • Bits 3 to 8 indicate the tributary port of the ODU transported. This is of interest in the case of flexible assignment of ODUs to tributary slots (e.g., 0DU2 into 0PU3). In the case of fixed assignment the tributary port number corresponds to the tributary slot number. For the 16 0PU3 tributary slots, 16 bytes of the PSI are used, as shown in Figure 3-37. • The ODU type can be ODUl or ODU2. • The tributary port # indicates the port number of the ODU 1/2 that is being transported in the respective TS; for the case of 0DU2 a flexible assignment of tributary ports to tributary slots is possible, for the case of ODUl this assignment is fixed, and the port number equals the slot number. ODU2 tributary ports are numbered 1 to 4. 3.10.3.5
Frequency Justification
The mapping of ODU1/ODU2 signals (with up to ±20 ppm bit-rate tolerance) into the 0DU3 signal is performed as an asynchronous mapping. The OPU3 signal for the multiplexed 0DU1/0DU2 structure is created from a locally generated clock, which is independent of the 0DU1/0DU2 client signals. The 0DU1/0DU2 signal is extended with Frame Alignment Overhead and an all-Os pattern in the OTUl overhead field. The extended ODU1/ODU2 signal is adapted to the locally generated ODU3 clock by means of an asynchronous mapping with a -1/0/+1/+2 positive/negative/zero (pnz) justification scheme. The asynchronous mapping process generates the JC, NJO, PJOl and PJ02, according to Table 3-19. The demapping process interprets JC, NJO, PJOl, and PJ02, according to Table 3-19. Majority vote (two out of three) is used to make the justification decision in the demapping process to protect against an error in one of the three JC signals. The value contained in NJO, PJOl, and PJ02 when they are used as justification bytes is all-Os. The receiver is required to ignore the value contained in these bytes whenever they are used as justification bytes.
Chapter 3
116 1
2
3
4
5
6
7
8
PSI [2]
OPUType
Tributary Port Number
Tributary Slot Number 1
PSI [3]
OPUType
Tributary Port Number
1 Tributary Slot Number 2
PSI [4]
OPUType
Tributary Port Number
1 Tributary Slot Number 3
PSI [5]
OPUType
Tributary Port Number
Tributary Slot Number 4
PSI [6]
OPUType
Tributary Port Number
Tributary Slot Number 5
PSI [7]
OPUType
Tributary Port Number
Tributary Slot Number 6
PSI [8]
OPUType
Tributary Port Number
Tributary Slot Number 7
PSI [9]
OPUType
Tributary Port Number
Tributary Slot Number 8
PSI [10]
OPUType
Tributary Port Number
Tributary Slot Number 9
PSI [11]
OPUType
Tributary Port Number
Tributary Slot Number 10
PSI [12]
OPUType
Tributary Port Number
Tributary Slot Number 11
PSI [13]
OPUType
Tributary Port Number
Tributary Slot Number 12
PSI [14]
OPUType
Tributary Port Number
Tributary Slot Number 13
PSI [15]
OPUType
Tributary Port Number
Tributary Slot Number 14
PSI [16]
OPUType
Tributary Port Number
Tributary Slot Number 15
PSI [17]
OPUType
Tributary Port Number
Tributary Slot Number 16
Figure 3-37. 0PU3-MSI coding
Table 3-19 JC, NJO, PJOl and PJ02 generation and interpretation (from ITU-T Rec. G.709) JC [7,8]
NJO
PJOl
PJ02
Interpretation
00
justification byte
data byte
data b3^e
no justification (0)
01
data byte
data byte
data byte
negative justification (-1) double positive justification (+2)
10
justification byte
justification byte
justification byte
11
justification byte
justification byte
data byte
positive justification (+1)
Interfaces for Optical Transport Networks
117
3.10.4 Summary The ITU-T standards under the OTN umbrella provide a transport platform optimized to carry high-capacity data pipes (greater than or equal to 2.5 Gb/s) with minimal usage of the overhead bytes and much more relaxed timing than SONET/SDH. The OTN frame structure defined with G.709 is a nimble transport fi-ame that carries user payload efficiently and provides strong FEC capabilities. Moreover, it provides a comprehensive method of detecting and locating faults along with relaying the status of such faults to the equipment in the forward direction (downstream) and backward direction (upstream). It was realized at the outset that an Optical Fiber duct which contains numerous fiber strands that can carry hundreds of A.s, if cut/damaged can generate thousands of alarms signals for individual channels. Hence the network can be easily flooded with alarm status traffic. Precautions were taken to ensure that only minimal traffic relaying necessary alarm signals would be generated if such a disaster occurred. Intelligent diagnostic capabilities of pinpointing failures and possibly healing from such failures, without human intervention in most cases, make it a viable transport technology of choice.
3.11. [I]
REFERENCES
Alan McGuire and Paul Bonenfant, "Standards: The Blueprints for Optical Networking," IEEE Communication Magazine, February 1998. [2] ITU-T Recommendation G.805, Generic functional architecture of transport networks, 2001. [3] ITU-T Recommendation G.872, Architecture for the Optical Transport Network (OTN), 2001. [4] ITU-T Recommendation G.709, Interfaces for the optical transport network (OTN), 2001. [5] ITU-T Recommendation G.798, Characteristics of optical transport network hierarchy equipment functional block, 2002. [6] ITU-T Recommendation G.975.1, ''Forward error correction for submarine systems'' [7] B. Sklar, Digital Communications: Fundamentals and Applications'' 2^^ edition, Prentice-Hall, 2001. [8] Maarten Vissers, "Optical Transport Network and Optical Transport Module; Digital Wrapper," Beyond SONET/SDH conference, Paris, France, April 2001. [9] ITU-T Recommendation G.707, Network node interface for the Synchronous Digital Hierarchy (SDH), 200^. [10] ITU-T T.50, International Reference Alphabet—7-bit coded character set for information exchange. [II] ITU-T Recommendation G.7712Af.l703, Architecture and specification of data communication network, 2003.
This page intentionally blank
Chapter 4 MULTIPLEX STRUCTURES OF THE OPTICAL TRANSPORT NETWORK Evolution: voice to data transport and voice over data transport Huub van Helvoort and Mimi Dannhardt Independent Consultants
4.1.
INTRODUCTION
This chapter will describe the transported payload capabilities of the Optical Transport Network (OTN). Since the introduction of the OTN in 1990, the initial capacity of 155 Mbit/s, based on the transport capabilities of the existing Pleisiochronous Digital Hierarchy (PDH) network, has grown to the current 40 Gbit/s. Up until about six years ago, the payload of the OTN was mainly used to transport voice traffic, and the multiplexing structure of the OTN was based on this traffic. At that time a consensus emerged among major market researchers indicating that, while voice traffic will continue to grow at a moderate rate, the transport of data traffic will dominate most networks by the years 2002-2005, requiring more bandwidth than was anticipated. Some of the new applications emerging that require transport over Optical Transport Networks include • Local/MetroAVide Area Networks (LAN/MANAVAN) services (e.g. Ethernet) • Storage Area Networks (SAN) service (e.g. Fibre Channel, ESCON) • High Speed Internet • rPVPN • Video distribution As with more and more applications emerge in the very near future, it is likely that they will have data-oriented characteristics similar to those mentioned above.
120
Chapter 4
This chapter describes the adaptations made to the existing standards to facilitate this anticipated growth. To avoid confusion, where the text refers to SDH (Synchronous Digital Hierarchy), also the terms SONET (Synchronous Optical NETwork) and OTN (Optical Transport Network) are applicable.
4.2.
THE SITUATION IN THE PREVIOUS CENTURY
The initial SDH transport structures were optimized for traditional TDM voice-type applications. In order to transport the new data-centric applications in an efficient manner, these SDH structures needed to be enhanced. Given the large installed base of SDH equipment, any enhancement needs to operate in a manner transparent to deployed networks. The revolutionary shift of voice to data transport needs an evolutionary change in the SDH standards. All information sent over the SDH network is placed into containers. Each container has its own management area, called overhead, and the area used for the transport of the information, ov pay load. One or more containers can then be mapped into the payload area of a particular SDH frame to be transported over an optical fiber or multiplexed in a larger container. The containers are virtual in the sense that they are pointed to from the frame overhead but can be interchanged or moved without difficulty. They are also virtual because they are not permanently assigned but are only used during the time of a connection through the network for a specific client. The containers are referred to as VC-n, virtual container of multiplex order n, where n = 11, 12,2,3,4.
4.2.1.
SDH structure details
The structure of an SDH frame transported over the fiber also has an overhead area and a payload area. The overhead area is divided into a management area dedicated to the Multiplex Section overhead (MS-OH), i.e. the line between two multiplexers; and the Regenerator Section overhead (RS-OH), i.e. the section between a multiplexer and a regenerator or between two regenerators. A line can consist of one or more sections. Between the MS-OH and the RS-OH, an area is reserved for the pointers that will be used to locate the transported containers in the payload area. The payload area is also referred to as Synchronous Payload Envelope (SPE). Figure 4-1 depicts the structure of an SDH frame. It is a matrix structure consisting of 9 rows and 270 columns containing octets. The frame rate is 125 |as. The management overhead is located in the first 9 columns, followed by the payload area of 261 columns. The SONET equivalent is the OC3
Multiplex Structures of the Optical Transport Network
111
structure. The shaded area is referred to as the Administrative Unit Group of the first order: AUG-1. An AUG-1 is capable of containing one AU-4 or three AU-3s, The pointer information of the AU-n is placed in the pointer area.
1
91
261
1 RS-OH area 4
pointer area payload area MS-OH area
9
Figure 4-1. STM-1/0C3 structure
Because SONET is based on STS-1, the first-order structure is the STS-1, and its format is depicted in Figure 4-2. This structure, in the ITU-T, is referred to as STM-0. Here the overhead area is located in the first three columns and the payload area in the next 87 columns. An AU-3 will fit in the shaded area.
1 1
31
87
RS-OH
area 4 pointer
payload area MS-OH area 9
Figure 4-2. STS-l/STM-0 structure
The definitions and allocation of the overhead octets are provided in the next subsections.
Chapter 4
122
Before the SDH/SONET frame is transmitted, it is scrambled to prevent long sequences of Is or Os. A frame-synchronous scrambler of sequence length 127 is used with a generating polynomial of 1 + x^ + x^. The RS-OH octets in the first row will not be scrambled. 4,2.1.1.
Overhead bytes
Figures 4-3 and 4-4 illustrate the basic SDH/SONET frame overhead area in detail. The STM-1 overhead is shown in Figure 4-3 and the OCl overhead in Figure 4-4. 1 1 Al RS-OH 2 Bl 3 Dl pointers 4 HI 5 B2 6 D4 MS-OH 7 D7 8 DIO 9 SI
2 Al R R HI* B2 R R R Zl
3 Al R R HI* B2 R R R Zl
4 A2 El D2 H2 Kl D5 D8 Dll Z2
5 A2 R R H2* R R R R Z2
6 A2 R R H2* R R R R Ml
7 8 9 JO NU/ZO NU Fl NU NU R D3 R H3 H3 H3 K2 R R D6 R R D9 R R D12 R R E2 NU NU
Figure 4-3. STM-1/0C3 overhead byte allocation
RS-OH pointers
MS-OH
1 2 3 4 5 6 7 8 9
1 ^^^^ 1 ^1 1 ^1 HI B2
1 D4 1 D7 1 DIO 1 SI
A2 El D2 H2 Kl D5 D8 Dll Ml
Figure 4-4. OCl/STM-0 OH
JO
1
Fl
D3
1
H3 K2
D6
1
D9 D12
E2
1
1
1
Multiplex Structures of the Optical Transport Network
123
The Regenerator Section (RS) overhead bytes are as follows: Al, A2 — frame alignment octets, Al = 11110110, A2 = 00101000 JO — RS trace, a 1-, 16-, or 64-byte information field that can be used to uniquely identify the SDH/SONET signal Bl — BIP-8, bit interleaved even parity calculated over the previous scrambled frame, used for RS error monitoring El — engineering order wire for access at the regenerators, 64 kbit/s Fl — user communication channel, 64 kbit/s D1...D3 — DCCR the RS data communication channel, 192 kbit/s, used for Operation Alarming and Maintenance (OA&M) NU — octets for national use (the octets located in the first row are not scrambled and their values shall be chosen carefully) R — reserved for future use ZO — octet defined only in SONET for future growth (this octet is located in the first overhead row; it is not scrambled, and its value shall be chosen carefully) The Multiplex Section (MS) overhead bytes are as follows: •
4.2.1.2. •
B2 — BIP-(Nx24), bit interleaved even parity calculated over the previous scrambled frame, used for MS error monitoring Kl, K2 — APS, automatic protection switch channel + MS-RDI D4...D12 — DCCM, the MS data communication channel, 576 kbit/s, used for Operation Alarming and Maintenance (OA&M) SI — synchronization status Ml — remote error indication (REI), reflects the B2 error count to the far end for performance monitoring E2 — engineering order wire for access at the multiplex, 64 kbit/s NU — octets for national use R — reserved for future use Z1/Z2 — octets defined only in SONET for future growth Pointers HI, H2 — these two octets form a word that contains the AU-n or TU-3 pointer information, i.e. a four-bit New Data Flag (NDF) to indicate that a new pointer value is present, two S bits (ignored in pointer processing), and a ten-bit pointer. The pointer is able to accommodate differences, not only in the phases of the container and the SDH frame but also in the frame rates. Figure 4-5 shows the
124
Chapter 4 allocation of the pointer bytes.
ISTM-l column
1
2
3
4
5
6
7
8
""9"
AU-4 pointer
HI
Y
Y
H2
Z
Z
H3
H3
H3
13 AU-3 pointers
HI,
HI2 HI3 H2i H22 H23 H3| H32 H33
Y = "1001 xx:ir' (value x is ignored) Z = "llll n i l " Figure 4-5. Allocation of pointer bytes in an STM-1 frame
ll HI H2
H3 *'*^^«*.^,.,^^^H1H2 ^PTR i Jl
|2 HI H2
H3
[3 HI H2
H3 *^"*"^***^.,,,^^^H1H2 = negative justification
4 negative justification opportunity HI H2 ' H3 ^^^'^^^N,.,,^^^
H1H2 = PTR-1
[5 HI H2
H3 ^"'*'*
.i,^„^^HlH2: : positive justification
6 positive justific ation opportunity HI H2
H3
^"^'^*******^1H2: = PTR+1
Figure 4-6. AU-4 AU-3 TU-3 pointer function
1
Multiplex Structures of the Optical Transport Network
125
The pointer value designates the offset from the first byte after the H3 octet to the first octet of the transported container; see the first two frames in Figure 4-6. H3 — pointer action octet defined for frequency justification purposes. Its use depends on the H1/H2 pointer information. If it indicates a negative justification, the H3 byte is used to transport payload (frames 3 and 4 in Figure 4-6). If it indicates a positive justification, the byte immediately following the H3 octet is used for stuffing the payload (frames 5 and 6 in Figure 4-6). VC-n structure
4.2.1,3.
Figure 4-7 depicts the structure of an SDH VC-n container. It is a matrix structure consisting of 9 rows and m columns containing octets. A VC-4 has 261 columns and a VC-3 has 85 columns. The frame rate is 125 |Lis. The first column contains the path overhead. The next (m-1) columns are designated for the payload. A VC-4 plus its associated pointer is referred to as an AU-4. 1 .
1 Jl B3 C2 Gl F2 H4 F3 K3 9 Nl
m
payload area VC-4 m = 261 VC-3 m = 85
Figure 4-7. VC-4A^C-3 structure
Figure 4-8 shows the structure of a SONET STS-1 SPE. It is a matrix structure consisting of 9 rows and 87 columns containing octets. The frame rate is 125 |us. The first column contains the path overhead. The next 86 columns contain the payload and 2 columns fixed stuff. An STS-1 SPE (or a VC-3 with the additional fixed stuff bytes) plus its associated pointers are referred to as an AU-3.
Chapter 4
126
30.
59 . . . . 87
\n
B3 C2 Gl _^^ F2 H4 Z3 Z4 Nl
H c3 O
>>
C/D
T3
,^H
T3 O
>»
^
c^ T^
o
">, cd
a. ^H
cd
«4H
F/^wr^ ^-5. STS-1 SPEA^C-3^ structure
The Virtual Container overhead bytes are as follows: Jl — trail trace, a 16- or 64-byte information field that can be used to uniquely identify the SDH/SONET signal B3 — BIP-8 bit interleaved parity for path error monitoring C2 — signal label, indicating the use of the payload area Gl — path status: path RDI, path REI F2 — user communication channel, 64 kbit/s H4 — position and sequence indicator F3 — user communication channel, 64 kbit/s K3 — APS (bl.. .b4) + data channel (b7, b8) Nl — network operator octet, used in tandem connection monitoring Z3/Z4 — octets defined only in SONET for future growth 4.2,1 A.
Substructuring
The payload area of the VC-n structures can again be used to transport containers of smaller structure sizes: • The VC-4 payload area can contain three Tributary Unit Groups of order 3 (TUG-3). A TUG-3 structure is 9 rows by 86 columns; they are byte interleaved in columns 4.. .261 of the VC-4; columns 2 and 3 of the VC-4 contain fixed stuff. • A TUG-3 (STS-1) can contain a TU-3, i.e. a VC-3 (STS-1 SPE) and its associated pointer, or it can contain seven TUG-2s. • A TUG-2 can contain a TU-2 (VT6), i.e. a VC-2 (VT6 SPE) and its associated pointer, or it can contain two VT3s, three TU-12s, or fourTU-lls.
Multiplex Structures of the Optical Transport Network • • • •
4.3.
127
A TU-12 (VT2) consists of a VC-12 (VT2 SPE) and its associated pointer. A TU-11 (VT1.5) consists of a VC-11 (VT1.5 SPE) and its associated pointer. A VT3 consists of a VT3 SPE and its associated pointer. The VC-3 pay load area can contain seven TUG-2s. A TUG-2 structure is 9 rows by 12 columns; they are byte interleaved in columns 2... 85 of the VC-3.
THE EVOLUTION OF THE BANDWIDTH
To be able to satisfy the demand for transport of more information on the same optical link, higher-order multiplexers were defined. These multiplexers increased the bandwidth in steps of 4, i.e. the STM-N (N = 4, 16, 64, 256) were defined and similarly OCn (n = 3N). For super-rate signals requiring the full payload area of these new multiplexers, contiguous concatenated (CCAT) containers are defined that provide a contiguous payload area that also increases in steps of 4. These are referred to as VC-4-Xc, with X = 4, 16, 64, and 256. The STM-N structure consists of 9 rows by (N x 270) columns. The overhead is located in the first (N x 9) columns, followed by the payload area of (N x 261) columns. The shaded area in Figure 4-9 represents an AUG-N and it has a fixed phase relation with the STM-N frame. 1
(Nx9) 1
(Nx261)
1 RS-OHarea 1 pointer area MS-OHarea
Figure 4-9. STM-N/OCn structure, N = 1, 4, 16, 64, 256; n = 3N
•
An AUG-(4M) can contain one VC-4-(4M)c or four AUG-Ms (M = 1,4, 16,64).
The structure of the STM-N overhead has also been extended to provide room for all the extra pointers required to allow all possible combinations of transported containers. The B2 size has been increased as well to match the
128
Chapter 4
required accuracy of performance monitoring. The STM-N overhead structure is shown in Figure 4-10. 1 1 IAI RS-OH
2 3 pointers 4 5 6 MS-OH 7
Bl
pi HI B2
p4 p? pio
1 SI
2...3N Al R R HI B2 R R R Zl
3N+1 3N+1...6N 6N+1 6N+1...9N A2 A2 JO NU/ZO 1 El R Fl NU D2 R R D3 H2 H2 H3 H3 Kl R K2 R D5 R D6 R R R D8 D9 DU D12 R R Z2 Ml E2 NU 1
Figure 4-10- STM-N/OCn OH, N=l, 4, 16, 64, 256; n = 3N
Note — for the highest rates (N = 64, 256), this overhead structure is slightly modified, e.g. some of the Reserved octets are used for Forward Error Correction (FEC) or an additional communications channel DDDMX (9216kbit/s). The special H1/H2 value Y/Z "1001 x;cll 1111 1111" mentioned in Figure 4-5 is in fact the contiguous concatenation indicator and shall not be used as a pointer value. The structure of the VC-4~Xc is shown in Figure 4-11. The first column contains the same path overhead as a VC-4. The next (X-1) columns contain fixed stuff and the concatenated payload area is X times that of a VC-4. 1 1|jl B3 C2 Gl F2 H4 F3 K3 9 |NI
X 1
fixed stuff
(X X 260)
payload area
Figure 4-11. VC-4/STS-3c structure
Figure 4-12 provides an overview of the SDH multiplex structures that are currently defined.
Multiplex Structures of the Optical Transport Network
129
1 STM-1 1 1 STIVI-4 ||STM-16||STM-64||STM-256|| 1x1 1 x1 x1 x1 x1 |AUG-256| 1 1 I
|AUG-64|
'^'^
..} 1 AUG-16 1 "" 1
1 AUG-4 1 '^^ higher order multiplexes 1 1"'
1 AUG-1 1 1 1 xN f f
pointer processing muitiplexinq aligning nnapping
AH
x1
1x1 1 AU-4 |
•
|[_VC-4 1
x1
|AU-4-4C|
xl
|AU-4-16C|
xl
1
|AU-4-64C||AU-4-256C||
^ ^ 4 4 |VC-4-4c| |VC-4-16c| |VC-4-64c| |VC-4"256c|
r ; |TUG-3r1
f""^ -^ 1 TUG-2 1 -
Xl. COntinii^ii*^ nnnrs^kfans^kfinn
x4| X3, ' 1 x1 |TU-11||TU-12|| TU-2 II TU-3 |
+ .-*^4
t
t
|VC-11||VC-12|| VC-2 II VC-3 |
•
•
Tlfc
1 C-11 I I C-12| 1 C
i» • •3 1 1 C-4 1
» 1
(> 1) f < 1 C-4-4C 1 1 C-4-16c 1 1 C-4-64C | | C-4-256c | __.„.„:
•
,
,
'
1
Figure 4-12. SDH multiplexing structures
The initial SONET multiplex structure was based on STS-1 (AU-3), but to provide interoperability with SDH, the AU-4 based structure is used for the higher-order multiplex, as shown in Figure 4-13. Emerging client applications, with their specific payload sizes requiring transport over an SDH/SONET network with its own specific structure, are faced with the problem of a relatively limited choice of bandwidth, i.e. concatenation levels. In addition to limited choice, there is also the problem of transport bandwidth inefficiency because the contiguously concatenated containers provide more bandwidth but not necessarily the "right-size" bandwidth.
130
Chapter 4 I QC-3 irOC-12 II OC-48 || OC-192 || OC-768 JxT x1 x1 x1 x1 |AUG-256|
OC-1 x1
I AUG-64 x4 3 AUG-16| x4 AUG-4 AUG-1
I I pointer processing xN multiplexing f aligning f
x3 AU-3
3=
x1
_r x4
x4 higher order multiplexes
x1
x1
x1
x1
AU-4 I [717-4-40 I |AU>4-16C| |AU-4>64C] |AU-4>256C|
mapping SPE rsTsTl
| S T S - 3 c | fsTS-12c I [STS-48C | |STS-192^ |STS^768c|
1
,,,,,,111, -^'"7
1 TUG.2 '1
1 X4|
AO|
*
f
t
T
•
•
•
X2|
r
Xl
contiguous concatenation
\VT^.5\\ VT2 II VT3 || VT6 |
f
| V r i . 5 | | VT2II W 3 | l VT6ISP E |C-11||C-12||
•
<
| | C - 2 | | 0 3 | I 0 - 4 I |G-4-4c| |C-4-16c| |C-4-64c| |C-4-256c Figure 4-13. SONET multiplexing structures
4.4.
NEW CLIENTS
The original tributary bit rates chosen for SDH were intended for voice services. These rates have a coarse granularity, require duplicate network resources for protection, and are not a good match to LAN, MAN, WAN, or SAN bandwidths. Examples of currently supported "traditional" SDH/SONET bit rates are shown in Table 4-1, SDH El E3 VC-4 VC-4-4C VC-4-16c VC-4-64C
2 Mbit/s 34 Mbit/s 155 Mbit/s 622 Mbit/s 2.4 Gbit/s lOGbit/s
SONET DSl DS3 STS-3C SPE STS 12c SPE STS-48C SPE STS-192 SPE
1.5 Mbit/s 45 Mbit/s 155 Mbit/s 622 Mbit/s 2.4 Gbit/s 10 Gbit/s
Table 4-1. Traditional SDH/SONET bit rates
Multiplex Structures of the Optical Transport Network
131
Bit rates for LAN/MANAVAN services are typically lOMbit/s, but lOOMbit/s and even 1 Gbit/s are becoming more and more popular. Other services, e.g, SAN, may vary from a few Mbit/s to several hundreds of Mbit/s. In order to transport these data services via an SDH transport, network there is no match in the bandwidth granularity. In addition, operators like to be able to sell their bandwidth in small chunks. Standards organizations have taken several successful steps to resolve this issue.
4.5.
VIRTUAL CONCATENATION
The enhancement of SDH to support Virtual concatenation (VCAT) provides the necessary payload sizes to enable the transport of the emerging services most efficiently. First proposed in November 1999, virtual concatenation is now present in all transport standards; i.e. the ITU-T Recommendations G.707 [1], G.783 [2] for SDH and Recommendations G.709 [3], G.798 [4] for OTN, the ETSI standard EN 300 417-9-1, and the ANSI standard T 1.105 [6]. Virtual concatenation provides an efficiency of 95 - 100% by grouping a number (X) of Virtual Containers (VC-n), using the combined payload provided by the Virtual concatenated Container (VC-n-Xv) to match the required bandwidth featuring: •
•
•
No requirements on existing SDH nodes that transit VC-n's part of a VC-n-Xv Virtual concatenation Group (VCG). Only the termination points of a connection are required to be compatible with virtual concatenation. Compensation for the differential delays caused by the difference in optical path length because each VC-n in the VCG does not have to follow the same physical path through the network, i.e. no routing constraints for operators. This diverse routing capability provides a better network resource utilization. Identifying the individual members of a VCG in order to enable the reconstruction of the original payload at the receiving side. The payload is distributed over the individual members of the VCG at the sending side.
4.5.1.
Differential Delay
There are several causes for the appearance of differential delay in a network:
132
Chapter 4 A geographically large ring with VC-n's from the same VCG routed around the ring in different directions (see Figure 4-3 member p and member q) caused by the availability of the required VC-ns in each part of the ring. The experienced delay is caused by the physical length of the fiber, i.e. the propagation delay of 5 |Lis per 1 km, and the transfer delay of each individual network element, i.e. 1 - 30 |as per NE. Depending on the network size, the differential delay, i.e. the difference in time between the fastest and slowest member in a VCG, can be several milliseconds. The standards allow a maximum differential delay of 256 ms. working path member q
Figure 4-14. Example of a ring with diverse routing
•
Networks using path-protected VC-n's to meet the required service availability. Many installed networks with path protection do not support locking of the path protection switch across a group of VC-n's. If a fault occurs that impacts only one of several virtually concatenated VC-n's, e.g. an equipment failure, only the failed VC-n will be switched to a protection path, and it will therefore arrive at the receiving end with a different delay (see Figure 4-3, member p). Again, the delay is due mainly to fiber propagation delay. To be able to detect and to compensate for the differential delay experienced by the members of a VCG, the X individual virtual containers will be sent with an identical label value. A counter provides the values of this label. At the receive end of the path, the received containers are stored in a buffer so as to be able to realign the received bytes from the individual VC-n's using the label value. The size of the buffer will determine the maximum allowable differential delay. The standards require a minimum delay of 125 |as. In the standards, this label is referred to as the Multi-Frame Indicator (MFI). The methodology that can be used to measure and compensate the differential delay experienced is explained in Section 4.6 of this chapter.
Multiplex Structures of the Optical Transport Network
4.5.2.
133
Payload Distribution and Reconstruction
In order to be able to reconstruct the original payload that was distributed over the X individual members of a VCG, the order of distribution needs to be known at the receiving side. 1
X
Xxm
C-n-Xc
125 MS
m+1
1 \
• •
"D CO
•
> VC-n-Xv
sz Xm.
o > o 1 2 5 MS
VC-n#X 1 2 5 MS
VC-n#1 Figure 4-15. Contiguous to virtual mapping
In the VCAT standard, each member of a particular VCG is assigned a unique Sequence Number (SQ). These sequence numbers are consecutive starting at 0 and ending at X-1. The information stream to be transported is divided into bytes, and each consecutive byte is placed in the identical byte position of the payload area of the consecutive VC-n's of the VCG using their sequence number (see Figure 4-4). At the receiving end of the path through the network, the bytes are recovered from the payload area of the individual VC-n's and are output consecutively using the sequence number to reconstruct the original signal. The effective payload transport capacity of the initial SDH/SONET multiplexes is shown in Table 4-2.
Chapter 4
134 SDH
SONET
Payload Capacity
VC-11
VT1.5SPE
1.600 Mbit/s
VC-12
VT2 SPE
2.176 Mbit/s
VC-2
VT6 SPE
6.784 Mbit/s
VC-3
STS-1 SPE
49.536 Mbit/s
VC-4
STS-Sc SPE
149.760 Mbit/s
VC-4-4C
STS-12c SPE
599.040 Mbit/s
VC-4-16c
STS-48C SPE
2.396160 Gbit/s
Table 4-2. Effective payload capacities
After the introduction of virtual concatenation, the following additional payload bandwidth sizes are available: SDH
SONET
from
to
In steps of
VC-11 (1-64)
VT1.5(164)
1.6 Mbit/s
102.4 Mbit/s
1,6 Mbit/s
VC-12 (1-64)
VT2(164)
2.2 Mbit/s
139.3 Mbit/s
2.2 Mbit/s
VC-3 (1-256)
STS-1 (1256)
49 Mbit/s
12.7 Gbit/s
49 Mbit/s
VC-4 (1-256)
STS-3c(1256)
150 Mbit/s
38.3 Gbit/s
150 Mbit/s
Table 4-3. Link sizes provided by virtual concatenation
Table 4-4 shows some examples of the efficiencies that are achievable in the SDH/SONET network. From Table 4-4, it is readily seen that at the 10 Mbit/s, 25 Mbit/s, 100 Mbit/s, and 1 Gbit/s rates there is no non-concatenated group that even closely matches the bit rates. Only with virtual concatenation of SDH/SONET bit rates can efficiencies better than 90% be achieved. The 10 Mbit/s Ethernet could be carried in a VC-3, but this yields an efficiency of just 20%. Similarly, the 100 Mbit/s rate could be carried in a VC-4, yielding a miserly 66%. However, now that virtual concatenated VC11, VC-12, VC-2, VC-3, and VC-4 are available, these services can be carried with almost 100% efficiency.
Multiplex Structures of the Optical Transport Network
k^ ^
11
OS
00
ON
O
O
oo
ON
a\ o
o
1>
> 1
>
-^
> (N ON
^ > ^
H
H
>
^
S
(N
o o
o o
> \o >
(N
o o
ON
>
>
en
1
U
>
>
1
QJ
i
03
1
en CO
1
00
H
00
en en
1 1
(N
^ 1
u
o ^ 1 ^ 1
o
1
o
>
00
*
>
IT)
1
00
1
4
u
>
1 '1"^
CO
1
1
00
1
1
OO
1
00
SN
.2
O
O
S3
^
o
(J
1
c
a o U
>
en en
,o C«
en
CO
> 00
0^
1 1
1 1
>
ON
>
.2 CO
135
H
O
a o
U
o
00
o
o
^ ^
CO
1^
en U
>
> p
§2
00
C/5
^.1
^ Is o S to
O
PL,
^
^
o
•S
t
§
o^- 1
Table 4-4, SDH/SONET virtual concatenation efficiencies
136
Chapter 4
It is worth noting that if even a single VC12 of an STM-1 is used in an access situation, then a 100 Mbit/s data service cannot be carried unless it is transported in a VC-3-2v. Moreover, with the rapid expansion to carry data over SDH, the need for virtual concatenation at all VC-n levels has become increasingly desirable by operators whose transport equipment cannot handle contiguous concatenation. This is especially true for interworking between global SDH networks and American SONET networks.
4.5.3.
Additional Benefits
The main objective of virtual concatenation is to flexibly provide multiple right-sized channels over an SDH ring or network. Virtual concatenation uses the SDH VC-n payload area directly and therefore does not have the inefficiency of mapping into an asynchronous payload first. In addition, since VCAT is a byte-level inverse multiplexing technique, it has the characteristics of a right-sized bandwidth, with an improved granularity, a low delay, low jitter, efficient reuse of the protection bandwidth, and a high efficiency payload mapping. Virtual concatenation is not restricted to the situation where all the individual VC-n's are transported in a single multiplex section (i.e. within a single SDH signal structure). In fact, the real potential flexibility offered by virtual concatenation occurs when the individual VC-ns forming the logical group are diversely routed over a number of different SDH signals. The diverse routing capability enables the transport of client signals in situations where a single link does not have enough resources to transport the client signal as a contiguous payload. In addition, virtual concatenation provides the network operator with the ability to implement channels in an SDH network that are more appropriate for the new, increasingly router-based applications. The advantages of these channels are bandwidth granularity, right-sized capacity, efficient mapping into VC-n, traffic scalability, and channelized high-capacity SDH interfaces. Finally, virtual concatenated payload transport is transparent to intermediate SDH Network Elements (NEs) on the path between two ends of a channel. Therefore it can be cost-effectively deployed into an existing SDH network without the need to upgrade all the NEs.
4.5.4.
Restrictions
Even though there are many benefits to using VCAT, there are also some restrictions. The size of the transported bandwidth of a VCG is fixed and, if one or more of the virtual containers fail, the full payload is discarded. Once
Multiplex Structures of the Optical Transport Network
137
the operator has provisioned the size of a VCG, it cannot be changed without interrupting the carried signal. Data transport can have a variable requirement for bandwidth regarding the time of the day, or the day of the week. Both these issue are addressed by the extension of the Virtual concatenation standard, know as Link Capacity Adjustment Scheme (LCAS).
4.5-5.
VCAT Details
A distinction has to be made between VC-n-Xv (n=3,4) and VC-m-Xv (m= 11,12,2), VC-n-Xv uses the H4 byte for the Path Overhead (POH), and VC-m-Xv uses K5 bit 2 for the POH. To reserve enough room for future expansion, both H4 and K5 comprise a multiframe. For the VCAT POH, an MFI field and an SQ field are allocated in this multiframe. Figure 4-16 shows the higher-order VCAT multiframe using the H4 byte. The higher-order VCAT multiframe uses the MFI-1 in H4 bits [5...8] for alignment. Figure 4-17 shows the lower-order VCAT multiframe utilizing the K4/Z7 byte bit 2. The lower-order VCAT multiframe uses the Multi-Frame Alignment Signal (MFAS) in K4/Z7 byte bit 1 for alignment.
Chapter 4
138 H4 byte Bitl
Bit 2 Bit 3 Bit 4 Bits
Bit 6
Bit?
1st 2nd multimultiframe frame number number
Bits
MFLl (bits 1-4) SQ LSB (bits 5-8)
"^
1
1
1
1
P T5
1 ^
II
MFI-2 MSB (bits 1-4)
0
0
0
0 1
1
MFI-2 LSB (bits 5-8)
0
0
0
1
II
Reserved ("0000")
0
0
1
0
II
Reserved ("0000")
0
0
1
II
Reserved ("0000")
0
1
0
0
4
II
Reserved ("0000")
0 1
1
0
1
5
II
Reserved ("0000")
0
1
1
0
6
II
Reserved ("0000")
0
1
1
II
Reserved ("0000")
0
0
0
8
II
Reserved ("0000")
0
0
11
9
II
Reserved ("0000")
0
1
II
Reserved ("0000")
0
1
II
Reserved ("0000")
1
0
II
Reserved ("0000")
1
0
II
SQ MSB (bits 1-4)
1
1
1
SQ LSB (bits 5-8)
1
1
1 ^^ 1 11 ^ ^ 0 11 ^ ^ 13 11 0 11 ^ ^ 1 15
1
MFL2 MSB (bits 1-4)
0
0
0
~
n
11
1 11
1 11
2
^ ^
0
1I
0
Figure 4-16. Higher-order VCAT Overhead in the H4 byte
n+1
Multiplex Structures of the Optical Transport Network MFAS in K4/Z7 bit 1 Function
Value
VCAT OH in K4/Z7 bit 2 Function
139
Bitnumber
Value
lilii jll^^^^^^
jljjlljlj^^^^
iiliii
jljl^^^^^^^^^^^^^^^^
IB^^^^^^
111
Extended signal label Not used by VCAT
Fixed Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved
^B
B li^^^^^^^^^^
iill Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved
Figure 4-17. Lower-order VCAT Overhead in the K4/Z7 bit 2
10 11 12 13 14 15 16 17 19 20
21 22 23 24
25 26 27
28 29 30
31 32
140
4.6.
Chapter 4
LINK CAPACITY ADJUSTMENT SCHEME (LC AS)
LCAS provides a mechanism for a hitless increase or decrease of the payload area of a VCG that is transported through an SDH network. In addition, the scheme will automatically decrease the payload capacity if a member experiences a failure in the network, and will increase the payload capacity when the network fault is repaired. The scheme is applicable to every member of the virtual concatenation group. The LCAS standard, i.e. ITU-T recommendation G.7042/Y.1305 [5], defines the required states at the source side and the sink side of the VCG path as well as the control information exchanged between both the source side and the sink side of the VCG path to enable the flexible and hitless resizing of the virtual concatenated signal. The feature characteristic of VCAT links using LCAS is the capability to reduce the transported payload bandwidth in the event of a path failure. This is a good match for data traffic with a mixture of priority levels. The reason for this is that the loss of bandwidth will affect the lower-priority traffic first and should allow the higher-priority traffic to continue passing over the link. The change in payload bandwidth takes place within a few milliseconds, depending on the physical distance between the two ends of the link.
4.6.1.
Link Capacity Increase
To increase the available virtually concatenated payload bandwidth, an additional path has to be set up through the network via the TMN. Once this path has been established, the VC-n can be added to the virtually concatenated signal.
4.6.2.
Link Capacity Decrease (Planned)
In a manner similar to the case of an increase in virtually concatenated bandwidth, decreasing the available virtually concatenated bandwidth requires the VC-n to be deleted from the virtual concatenated signal. Once the VC-n has been taken off the virtual concatenated signal, the path through the network can be deleted via the TMN. Note that in both cases of increasing or decreasing the virtually concatenated signal, a mechanism for the source node and sink node to notify each other about a request for path size changes and the status of the constituent signal is required. This has been accomplished by using a control field embedded in the overhead (OH) that is allocated for the implementation of virtual concatenation.
Multiplex Structures of the Optical Transport Network
141
In addition to the above signaling mechanism, the requirement of "hitless" path resizing (increase/decrease) created a further need for the development of a synchronization protocol between the source and sink nodes,
4.6.3.
Temporary Link Capacity Decrease
A temporary link capacity decrease can also be used if one or more VCn's belonging to a virtually concatenated signal fail. This failure is reported to the source node, which, upon reception and validation of this failure, will proceed by not using the payload area of the failed VC-n for transport of user data. Until the signal failure clears, the available bandwidth for the user is decreased by the size of the payload area of the failed signal. If the failure clears, the sink node notifies the source node, and the recovered VC-n will hitlessly be added to the virtual concatenated signal. First proposed in June 2000, the generic definition of LCAS is now in the new ITU-T recommendation G.7042/Y.1305 [5]. The actual information fields used to convey the control information through the transport network are defined in their respective Recommendations, namely, G.707 [1] and G.783 [2] for SDH and G.709 [3] and G.798 [4] for OTN. ETSI and ANSI refer to ITU-T.
4.6.4.
LCAS Details
Because the LCAS process is an extension of VCAT, it reuses the VCAT path overhead (light shading in Figures 4-18 and 4-19), i.e. the MFI and SQ numbering, and uses reserved bytes and bits for the additional LCAS POH (dark shading in Figures 4-18 and 4-19). The VCAT multiframe is referred to as a control packet because it contains the information needed by the LCAS protocol to control the use of the payload of each member in the VCG. The additional LCAS POH consists of the following: • A four-bit CTRL field, used to convey the operational state, i.e. IDLE, ADD, NORM/ EOS (normal operation with End Of Sequence indicator), or DNU (Do Not Use), of the member from the transmit side to the receive side. The state FIXED indicates that the member does not utilize the LCAS protocol. • An eight-bit MST field, to report the status of each member at the receive side back to the transmit side. • The RS-Ack bit, to acknowledge that the receive side has detected a change in the sequence numbering of the VCG.
142
Chapter 4 •
The GID bit, which can be utilized to verify the connectivity of the VCG through the network. • The CRC bits (eight bits in H4, three bits in K4/Z7), calculated over the total control packet, used for immediate validation of the control packet. Figure 4-18 shows the allocation of the higher-order LCAS POH in the control packet utilizing the H4 byte multiframe. H4 byte
Bitl
Jcs
1st 2nd multi- multiBit 2 Bits Bit 4 Bits Bit 6 Bit? Bits frame frame number number MFI-1 (bits 1-4) 7
C6
C7
Cs
0
1
1
IIM, IM5
M2
M3
M4
1
0
0
Mfi
M7
Ms
1
0
0
11
1 0
0
0
1
0
1
0
1
0
1
1
1
0
0
1
1
0
11
13
1
1
1
0
14
1
1
1
II II II 1 1
^
8 9 10
k Reserved ("0000") Reserved ("0000") Reserved ("0000") SQ MSB (bits 1-4) SQ LSB (bits 58)
II MFI-2 MSB (bits 14) 1 Mn-2 LSB (bits 58) |cTi CTz CT3 CT4 II Reserved ("0000") 0 0 GID II Reserved ("0000")
1 ^ c,
1
1 11
^^
1 12
0
0
0
0
0
0
0
0
1
0
0
1
0
1
0
0
1
0
1 11 15 0 11 ^ 1 11 1 0 1 2 11 3 0 1 "1 1 11 5
Ci
C3
C4
0
1
1
0
||C5
Ce
C7
Cg
0
1
1
11
6 7
]MI
M2
M3
M4
1
0
0
0
8
n+1
Figure 4-18. Higher-order VCAT + LCAS Overhead in the H4 byte. Light Shading: reuse of the VCAT path overhead. Dark shading: use of reserved bytes and bits for the additional LCAS POH.
Figure 4-19 shows the allocation of the lower-order LCAS POH in the control packet utilizing the K4/Z7 byte bit 2 multiframe.
Multiplex Structures of the Optical Transport Network MFAS in K4/Z7 bit 1 Function
Value
VCAT + LCAS OH in K4/Z7 bit 2 Function
MFI (bits 15) LSB MSB
Multiframe alignment signal SQ (bits 16)
0 CTRL (bits 14)
Extended signal label Not used by VCAT
Fixed Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved Reserved
0 0 0 0 0 0 0 0 0 0 0 0 0
GID Reserved Reserved Reserved Reserved RS-Ack
LSB CTi CTa CT3 CT4
0 0 0 0 , Ml M2
MST(bitsl8)
M3 M4 M5 M6 M7
CRC(bitsl3)
Bit number
Value MSB
0
143
Ms Ci C2 C3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 3J
32
Figure 4-19. Lower-order VCAT + LCAS Overhead in the K4/Z7 byte bit 2. Light Shading: reuse of the VCAT path overhead. Dark shading: use of reserved bytes and bits for the additional LCAS POH.
1
144
4.7.
Chapter 4
ADVANTAGES OF USING VCAT LCAS AND GFP
Both GFP mapping modes GFP-F and GFP-T may use the Virtual concatenation techniques described above. Virtual concatenation allows matching of the transport bandwidth as closely as possible to the bandwidth required by the client signal. The actual bandwidth required by the client signal may be a fraction of the standard LAN bit rate and may not be constant. LCAS provides the flexibility of changing on demand the size of the transported payload. LCAS also provides the capability to adjust the required protection bandwidth to the availability requirements in steps of (1/Xy^ of the working bandwidth. One of the applications that may use GFP mapping is DP. One of the technologies that uses DP as a means of transport is Voice over IP (VoIP).
4.8.
IMFLEMENTERS GUIDE FOR VCAT AND LCAS
The VCAT functionality can be implemented in part by reusing existing devices that provide the individual VC-n termination points and access to the VC-n payload signal, and the H4 or K4/Z7 overhead bytes at the network or server-layer side and devices that provide access to the payload signal of the client signal to be transported by the VCG at the customer or client-layer side. The VCAT implementation is then required for the distribution and reconstruction of the transported payload and the VCAT specific overhead. Part of the reconstruction of the transported payload is the compensation of the differential delay. For the actual delay buffer, commercially available memory devices can be used, thereby providing the possibility of matching the memory size with the required or acceptable maximum differential delay. The LCAS protocol is described in ITU-T recommendation G.7042 [5] by using state machines specified in SDL diagrams. SDL is the Specification and Description Language described in ITU-T recommendation Z.IOO. While the individual state machines for each member are not complex, the interworking may be complicated.
4.8.1.
Detection of Differential Delay
The differential delay can be detected by comparing the value of the MFI fields in the VCAT overhead among all the members within the group. For VC-4-Xv and VC-3-Xv, the MFI is the combination of the MFI-2 field and the MFI-1 field into a 12-bit number. The differential delay is calculated (in frames) by subtracting the MFI of one member from another member using 2's complement math. The result
Multiplex Structures of the Optical Transport Network
145
is interpreted as a 2's complement number. When the process of differential delay detection is initiated, there is no a priori knowledge of the delays, and thus the result of this calculation may be either positive or negative. A positive result indicates that member 1 leads (has less delay than) member 2 while a negative result indicates that member 1 lags (has more delay than) member 2. With a 12-bit number and a frame rate of 125 |as, the maximum differential delay could be 512 ms. However, due to aliasing, this value cannot be used. For example, an actual differential delay of 384 ms between two members will be calculated as a difference of -128 ms using the 2's complement math. Thus the leading delay of 384 ms aliases to a lagging delay of 128 ms. To avoid aliasing, the maximum differential delay is defined to be less than half the maximum detection range. For VC-11/12/2, the MFI field is only 5 bits, but with a frame rate of 16 ms, the maximum detectable differential delay is also 512 ms. While the standard allows for a maximum of 256 ms of differential delay to be detected, most implementations will not support the maximum. With a fiber propagation delay of 5 ms per 1000 km, a signal sent around the earth (40,000 km) will experience a delay of 200 ms. Supporting a differential delay less than the maximum will also decrease the probability of aliasing. By using the MFI to determine the differential delay, the accuracy of the delay is related to the frame rate, e.g. 500 |us for VC-12 VCAT. The accuracy of the calculation can be improved when the byte number within the frame is tracked as well. Now the differential delay can be determined down to the byte level if desired, e.g. 3.6 |as for VC-12 VCAT. Another aspect of differential delay detection is to determine whether the differential delay exceeds the available buffer space. If the actual differential delay goes beyond the implemented buffer range, this error condition will be detected and reported, since the data will be corrupted if the delay buffers cannot be aligned properly,
4.8.2.
Compensation of Differential Delay
Realignment of the received VCs can only be accomplished by buffering the incoming data from the time the least-delayed VC arrives until the mostdelayed VC arrives. The required buffer size, i.e. the physical amount of memory, depends on the bit rate of the payload data, the maximum number of members in the VCG, the acceptable differential delay, and the buffer management scheme. The implemented maximum acceptable differential delay can be negotiated between the vendor and the operator and may depend on the network topology and/or the cost of the required buffer
146
Chapter 4
memory. The acceptable differential delay may be further limited by provisioning based on operator policy. Examples of the amount of memory required to compensate for the experienced differential delay are calculated as follows: •
VC-12 The VC-12 pay load container structure consists of 4 rows x 34 columns or 136 bytes repeated every 500 |us. The amount of buffer memory required is 272 bytes per ms per VC12. Example: 10 Mbit/s Ethernet transported in a VC-12-5v: The buffer memory required is 5 x 272 = 1360 bytes per ms differential delay.
•
VC-3 The VC-3 payload container structure consists of 9 rows x 85 columns or 765 bytes repeated every 125 ps. The amount of buffer memory required is 6120 bytes per ms per VC-3. Example: 100 Mbit/s Fast Ethernet transported in a VC-3-2v: The buffer memory required is 2 x 6120 = 12,240 bytes per ms differential delay.
•
VC-4 The VC-4 payload container structure consists of 9 rows x 260 columns or 2340 bytes repeated every 125 |LIS. The amount of buffer required is 18,720 bytes per ms per VC-4. Example: 1 Gbit/s Gigabit Ethernet transported in a VC-4-7v: The buffer memory required is 7 x 18720 = 131040 bytes per ms differential delay.
4.8.3.
Structure and Management of Differential Delay Buffers
The features that have to be supported and the flexibility of the implementation will determine the structure and management of the differential delay buffers. If the implementation is required to support VCAT without LCAS, there are simplifications from an implementation that must support LCAS. Table 4-5 below illustrates some of the delay buffer
Multiplex Structures of the Optical Transport Network
147
management differences between non-LCAS and LCAS implementations. That is, LCAS requires more dynamic and flexible buffer management than a non-LCAS implementation.
Feature Alignment required
Non-LCAS - at startup - after error recovery
Differential delay range determination
" at startup
Sequence number allocation
Fixed - at startup
LCAS - at startup - after error recovery - at member addition - at startup - after error recovery - at member addition - at member removal Variable - at startup - at member addition - at member removal
Table 4-5. Delay buffer management differences
4.8.4.
Differential Delay Buffer Overview
While the differential delay buffers can be conceptually thought of as FIFOs, they are generally implemented as circular buffers with read pointers and write pointers. As the member data arrives, it is written at the write pointer location. Once the differential delay is determined, the read pointer is set for each member link and the reconstruction of client data can begin. Depending upon the implementation, either the read pointers or the write pointers are synchronized across all members of the group. This synchronization step across the multiple delay buffers in the group favors the circular buffer architecture over a strict FIFO implementation. Once aligned to the MFI, the depth of the buffer, or the difference between the read pointer and write pointer, generally stays constant. There will normally be some jitter caused by pointer adjustments in the transport network and the presence of the SDH/SONET overhead bytes. Typically, the pointer adjustments will not occur at the same time and the overhead bytes will not be aligned. However, since all the member signals of a VCG are generated with the same clock, the long-term clock rate of the members will not diverge.
148
4.8.5.
Chapter 4
Alignment within a VCG
For VCAT, alignment of the members is required when the group is created. Realignment will be required after an error condition, since the routing through the network of one or more members in the group may have changed. Since the (re-) alignment occurs when data is not valid, the effect of the alignment process on the data transport is not critical. For VCAT with LCAS enabled, the alignment process is also active each time a member is added to the group. The alignment process should then check whether the calculated differential delay of the additional member is within the implemented boundaries (i.e. the available buffer size). The alignment process can start as soon as the control packet with the CTRL word ADD is received. The process should be ready when the control packet with the CTRL word NORM/EOS arrives and consequently can hitless increase the bandwidth. When adding a new member to an existing VCG, there are three possible scenarios: the new link may have similar delay, less delay, or more delay than the existing members in the group. If the new member experiences similar delay through the network, the delay of all the members of the VCG remains the same, and the new member and its associated delay buffer are added to the VCG. In this scenario, neither the differential delay range nor the propagation delay of the group will be modified. If the new member experiences less transport delay than the existing members in the VCG and the available delay buffer is large enough to accommodate the required delay compensation, the member is simply added to the group. This outcome increases the differential delay range of the group but not the propagation delay. If the new member experiences more propagation delay than the existing members in the VCG, then, to align the new member, delay has to be added to all the other members in the group (by increasing the size of each member's individual delay buffers). This addition increases both the differential delay range and the propagation delay of the group. These scenarios bring up bring up many interesting design points concerning propagation delay or latency. Depending upon the type of client traffic transported, there are trade-offs between latency and flexibility. Some applications are more sensitive to changes in the latency than to the latency itself. In this case, changing the latency of the group to accommodate a new member with more propagation delay that the existing members in the VCG may not be desired, and the new member may be refused by maintaining the fail condition, i.e. MST=FAIL. If fixed latency is a desired
Multiplex Structures of the Optical Transport Network
149
feature, the operator has to calculate and provision the worst-case latency that shall be used from the moment the group is initiated. Other applications are insensitive to changes in the latency; the implementation can even try to minimize the group latency. Change in the latency of a group, can either be slow, by increasing or decreasing the data playout rate slowly, or instantaneous, by stopping the data playout until the desired delay buffer depth is reached.
4.8.6.
Sizing the Delay Buffers
Depending upon the type of memory used, implementation can trade off simplicity for size of memory. If the memory chosen is very expensive, like ASIC internal memory or SRAM memory, the buffer sizes should be minimized. To achieve this result each member in the group will have a different amount of memory dedicated to its delay buffer. While this setup results in an optimal use of memory and a minimal memory size, the management of the delay buffers will become very complex and may limit the flexibility of LCAS groups. The simplest buffer structures are based on the maximum allowable propagation delay (determined by the vendor or buyer of the equipment). These buffers allow indexing based upon MFI and byte location within the member frame. The simplest buffer structures, however, are probably the least memory efficient, since memory efficiency is traded for simplicity and cheaper external DRAM could be used to support these simple structures.
4.8.7.
Processing Time
One feature of LCAS is that the source process controls the exact multiplexing order and timing of member additions and removals. This feature requires the sink process to react to the contents of the control packets at the very next multiframe boundary. The next multiframe boundary is located approximately 42 |Lis after the last (H4) byte of the HO VCAT control packet and approximately 125 |Lis after the last (K4) bit of the LO VCAT control packet. Although at first glance this may seem to be a lot of time, issues can arise if the implementation should have the capability of supporting very large VCAT groups and/or a large numbers of VCAT groups simultaneously. The major task that has to be performed within this time frame is to determine changes in the multiplexing order and to configure the data path accordingly to be able to switch to the new order at the beginning of the next multiframe. Because the CRC validates the content of a control packet, it is not useful to start interpreting MFI and SQ values before the CRC arrives. The standard
150
Chapter 4
requires that control packets with CRC failures be ignored. The CRC validation gives a faster response than the more common 3 or 5 times exact match validation. Ignoring a control packet that contains a change because it is erroneous could cause data corruption; however, the frequency of change within an LCAS group is fairly low, and the probability of data loss is even less. The implementation must resolve any inconsistencies within the VCAT group, such as duplicated or missing SQ numbers and duplicated or missing EOS control words. Since the probability of bit errors within the SDH/SONET network is low, the number of erroneous control packets is also low. The amount of processing time required to handle erroneous control packets is implementation dependent. Since the acceptable member state changes are limited and there are rules governing sequence number reassignments, some erroneous control packets may be reconstructed by analyzing the control information of the remaining members of the group. The question is, as always, whether the results justify the effort. A more common scenario is the total failure of a member trail. In this case, the member trail termination reports a signal fail (TSF) and the member is moved to the DNU state according to the LCAS procedure. The member remains in this DNU state until the trail failure is repaired. Since communication is lost with that member at the sink side, removal of the member from the group by the source process cannot be detected except by correlating the SQs of the remaining members.
4.8.8.
Controlling Distribution/Reconstruction Order
In the VCAT process, the client payload is distributed at the source side over all the individual member payload areas, octet by octet, in a round robin fashion. At the sink side, the client signal is reconstructed by interleaving the octets from all the member payload areas. It is essential that the order of distribution is also used for the reconstruction. That is the reason the source assigns sequence numbers to the members. Each member transports its assigned sequence number to the sink for the sink to use in the reconstruction process. A special case is introduced for the VCAT application with LCAS enabled. Here, only active members carry the distributed payload, i.e. members in the NORM/EOS state. A member can also be in the DNU state. While in the DNU state, the member retains its sequence number but will be skipped in the distribution/reconstruction process. When implementing LCAS, the DNU state could be handled by implementing an interleave sequence number. The interleave sequence
Multiplex Structures of the Optical Transport Network
151
number controls the distribution/reconstruction process. The interleave sequence number is assigned to a member based exclusively on the member status NORM/EOS. Table 4-6 contains an example. Member
1^
b
State NORM NORM Assigned SQ nr 0 1 1 Interleave SQ nr 0
c
d
e
f
DNU 2 n/a
NORM 3 2
DNU 4 n/a
EOS 5 3
g DNU 6 n/a
Table 4-6. Example of the interleave sequence number
4.8.9.
Member Status
The member status is the parameter to communicate the health and usability of the members from the sink side back to the source side of the VCG. The MST protocol uses one bit for each sequence number to indicate whether the member status is OK=0 or FAIL=1. The MST bits are transferred in an MST multiframe; its size is determined by the maximum of SQ numbers and is technology specific (e.g. 256 for HO SDH LCAS). Since not enough bits are present in a single control packet, the member status is serialized over multiple control packets. All members of the VCG transfer the MST multiframe. This ensures correct operation with just a single member in the reverse direction. Since all information is carried on every member, implementations that monitor just a single return member are allowed. The renumbering of the members when members are added or removed from the VCG complicates this simple protocol. Whenever a member is added to the group, the new member receives the next available higher sequence number. When a member is removed from the group, all members with a higher sequence number must be renumbered (decremented by one). Note that a member may have one sequence number while in the ADD state and a different sequence number when it transitions to the NORM/EOS state. When the sequence numbers of the members change due to member addition or removal, there exists a period of uncertainty during which it is not known whether the reported MST is for the old numbering or the new numbering. This period is caused by transmission propagation delays and processing delays. To control this period of uncertainty, the source will stop interpreting the received MST until the sink acknowledges that it detected a change in sequence numbering by toggling the RS-Ack bit. Once the toggling is detected by the source, it will assume that the received MSTs match the current numbering again.
152
Chapter 4
In certain situations, the sink side may not detect a resequence operation and consequently may never toggle the RS-Ack bit. One of these cases where the resequence may not be seen by the sink side is with the removal of a member in the DNU state that has the highest SQ number of the VCG. This is resolved in the standard by the definition of an RS-Ack timeout timer that is started when the source stops processing the MST due to a resequence. When the timer expires, the source will continue to process the MST and assumes the RS-Ack was lost or not sent.
4.9. REFERENCES [1] [2] [3] [4] [5] [6]
ITU-T Recommendation G.707/Y1322, Network Node Interface for the Synchronous Digital Hierarchy SDH, 2003. ITU-T Recommendation G.783, Characteristics of Synchronous Digital Hierarchy (SDH) Equipment functional blocks, 2003. ITU-T Recommendation G.709, Interfaces for the optical transport network (OTN), February 2001. ITU-T Recommendation G.798, Characteristics of OTN equipment functional blocks, November 2001. ITU-T Recommendation G.7042/Y. 1305, Link Capacity Adjustment Scheme (LCAS) for Virtual Concatenated Signals, 2003. ANSI American National Standard T1.105, Synchronous optical network (SONET) — Basic Description including Multiplex Structure, Rates and Formats.
Chapter 5 GENERIC FRAMING PROCEDURE (GFP) Enrique J. Hernandez-Valencia Lucent Technologies
5.1.
INTRODUCTION
The Generic Framing Procedure (GFP) is a new protocol recently standardized under ITU-T G.7041/Y.1303 [1] and ANSI Tl.105.02 [2] and designed to support variable- and fixed-length packet transport modes over a general-purpose bit or byte synchronous high-speed communications channel. GFP extends the HEC-based packet delineation mechanism used by other broadband applications such as ATM [3] to variable-length data transport applications. GFP exploits the ability of modern point-to-point transmission links to deliver the incoming information stream in a sequential and orderly fashion to greatly simplify data link layer synchronization and frame boundary delineation operations. Unlike packet delineation mechanisms based on the HDLC framing procedure [4, 5], GFP requires no special line encoding for the framed protocol data units (PDU), which substantially reduces processing logic requirements for the data link mapper/demappers. Unlike ATM, GFP delegates high-touch QoS management functions to the client layers, which further reduces operational overhead. The lower implementation complexity makes GFP particularly suitable for high-speed transmission links such as SONET/SDH [6, 7] pointto-point links, wavelength channels in an optical transport network [8], or even dark fiber applications [9]. For high data rate environments, GFP is a very attractive alternative to solutions such as ATM, Frame Relay [10], PPP/HDLC [11], PPP-over-SONET (PCS) [12], or X.85/X.86 [13, 14].
154
Chapters
TDM
PHY
PHY
PHY
PHY
STS/VTJJross-Connect Matrix
X
Virtual Concatenation/ LCAS
GFP Encapsulation/Framing
Packet Swjlch
Ports
PHY
PHY
PHY
PHY
Figure 5-1. High-level functional model of a hybrid Ethernet/TDM transport system
From a hybrid Packet/TDM system perspective, as illustrated in Figure 51, two aspects are of key relevance when discussing transport mechanisms for packet-oriented traffic over a TDM-based telecommunications infrastructure: (1) the data-link "adaptation" mechanism to transform the packet-based data flow into a bit/byte stream that preserves the native packet structure, and (2) the rate adaptation mechanism to map the resulting bit/byte stream into the SONET/SDH payload. For Ethernet transport, for instance, all solutions of practical interest (ATM, FR, POS, X.85/X.86, and GFP) perform data-link adaptation by reencapsulating the original protocol data unit (PDU) and then reframing the resulting PDU into a TDM-friendly cell/packet flow. It is this PDU "framing procedure" that largely differentiates the various solutions. The final rate-adaptation step is fairly similar across technologies. For constant-bit-rate-oriented traffic, the adaptation models over TDM-based telecommunications are based on the
Generic Framing Procedure (GFP)
155
principle of quantization of the incoming flow, with options for line code compression, if feasible. Rate adaptation typically requires fine-grained mapping of the adapted client signal into its new constant-bit-rate container to minimize additional signal jitter and wander. This chapter provides an overview of GFP. The chapter begins with a review of background information, related work associated with packet transport over public networks, and design factors influencing its development. Next follows a brief summary of current formats, procedures, and implementation considerations. The chapter closes with a discussion on performance and a sample look at applications.
5.2,
BACKGROUND
The immense popularity of the Internet and Internet Protocol- (IP-) based networks has created an explosion in the number of IP-based end systems, and consequently, in the aggregated IP traffic being carried over the public circuit-switched infrastructure. Most of this traffic originates in corporate LANs, which are today over 90% Ethernet based. While voice and private line traffic still account for a majority of traffic in most public network backbones today, it is widely expected that packet-oriented traffic, originating from IP end systems or native Ethernet transport applications, will dominate the public backbone traffic in the not-so-distant future. (Indeed, this is already the case for IP-centric Internet Service Providers [ISPs]). This increase in IP and native Ethernet traffic demands much higher access link rates than those in use today. It also demands data transport approaches that are compatible with future data-aware value-added services.
5,2,1
Packet Transport on Public Networks
Figure 5-2 illustrates various transport options for packet traffic over the public network infrastructure. A significant portion of IP traffic today is encapsulated in Frame Relay, PPP/HDLC, or POS, or isadapted to ATM for transport across a TDM-based core network. Currently, most Frame Relay and PPP line interfaces operate at DSl/El, DS3/E3, or 0C-3c/STM-l rates or less. The same is true of most line interfaces for IP edge routers, although OC-48C/STM-16 and OC-192c/STM-64 SONET/SDH interfaces are being deployed at an increasing rate, particularly in the core of metropolitan and wide-area networks. Ethernet and Storage Area Networking (SAN) protocols such as Fibre Channel, ESCON, and FICON have traditionally been transported over the public network infrastructure by means of proprietary (vendor-specific) solutions. Given the widespread availability of
156
Chapter 5
inexpensive 10/100/1000 Mbps Ethernet interfaces for CPE switches/routers, the growing need to improve data center/SAN interconnectivity, and the recent additions of Virtual LAN-based Virtual Private Networking (VPN) and QoS capabilities via IEEE 802.1Q/P, there is a renewed interest in a QoS-friendly, standard-based mechanism to transport Ethernet and SAN traffic directly over TDM networks. Voice (Applications)
Data (IP, IPX, MPLS, etc.)
SANs (Applications)
Video (Applications)
Ethernet* (MACS & PHYs)
Private Lines (Circuits)
RPR (MACS & PHYs)
o:
PPP (Services)
(Services)
ATM (Setvices)
HDLC I (Encapsulation) |
GFP (Encapsulation)
SONET/SDH (Physical Channels)
OTN (Physical Channels)
Fiber or WDM (Physical Channels) ' May also run directly on fiber
Figure 5-2. Transport options for voice, data, storage, and video traffic over a SONET/SDH network and Optical Channels via ATM, FR, HDLC, or GFP
5,2.2
Other Traffic Adaptation Approaches
While flag-based PDU delineation has been common in various framing and payload mapping standards, alternative approaches have been in use as well. For packet/cell delineation, ATM relies on implicit information about the packet length (fixed at 53 bytes) and the header CRC, rather than a flag, for the purpose of packet delineation. The header CRC match is used for initial link synchronization. The fixed packet (cell) size and the header CRC are used to verify link synchronization after initial link synchronization has been established. The fixed-length ATM PDUs significantly simplify processing at the data link receiver. There are also more recent examples of PDU delineation using implicit and explicit PDU length indicators. ATM Adaptation Layer, Type 2 (AAL-2) [15], packs variable-length PDUs into fixed-length ATM cells. A PDU length indicator is used to provide self-delineation once the packet boundary is identified. At the
Generic Framing Procedure (GFP)
157
beginning of each ATM cell, the length pointer helps regain frame delineation quickly. A similar mix of a PDU length indicator and boundary pointer is used for the downstream communication channel in Hybrid Fiber Coax (HFC) links using ADAPt+ [16]. In ATM, AAL-2, and HFC, the basic lower-layer framing exploits the fixed-length format of the PDUs at that protocol layer (ATM cells, downstream frames for HFC). The pointers and length indicators are then used to provide delineation of variable-length PDUs within the payloads of the lower-layer protocol. In absence of a lowerlayer framing with fixed-length frames, a delineation mechanism applicable to the variable length PDUs directly is needed.
5,2.3
Other Design Considerations
Although it would seem straightforward to extend the frame delineation and synchronization procedures used for ATM and other fixed-length PDUs to variable-length PDUs, that is not the case. Some of the issues that need to be addressed [17-18] are as follows: • For fixed-length PDUs, it is important to allow an efficient format for accessing and buffering the adapted PDUs. A header CRC could be used to identify a potential PDU boundary and then invoke the fixed PDU length to jump to the next frame boundary, and to verify that frame synchronization has been achieved. When the second header CRC check fails, because of the false start, the real PDU boundary may have been lost among the many bits/bytes jumped over. In ATM, the small cell size guarantees that, at most, 53 bytes are lost. Large variable-length PDUs make this test more difficult. A false match of the header CRC and a subsequent invocation of the wrong length indicator could waste a large number of opportunities to identify the true PDU boundary. Thus, straightforward extension of the procedure used for ATM may result in a much longer resynchronization interval for variable-length PDUs. • For small, fixed-length PDUs, a failed header CRC need not cause immediate loss of synchronization because the PDU length is specified implicitly. With variable-length PDUs, a failure in the header CRC makes the PDU length indicator itself suspect and causes immediate loss of synchronization. The error correction capability becomes very important for variable-length PDUs. • Small PDUs also imply that a single user can gain control over the link payload only for a very short period of time. PDU interleaving from multiple sources decreases the probability of successful attacks from malicious users trying to induce low bit transition density on the link. In large, variable-length PDUs, a single user gets access to the link for a
158
Chapter 5
much longer time period. Mechanisms to counter attacks aimed at creating very low bit transition density over the data link become critical. • Variable-length PDUs tend to have loose maximum size bounds (up to 64 Kbytes in IP), which make the resynchronization phase. The above protocol design issues are carefully addressed in GFP. The next sections presents the design choices made in the specification of the formats and procedures for GFP.
5,3.
FORMATS AND PROCEDURES
A high-level functional overview of GFP is presented in Figure 5-3. GFP consists of both client-independent and client-specific aspects. Common aspects of GFP apply to all GFP-adapted traffic and cover issues such as PDU delineation, data link synchronization, payload scrambling, client and control PDU multiplexing, and client-independent performance monitoring. Client-specific aspects of GFP cover issues such as mapping of the client PDU into the GFP payload and client-specific performance monitoring and OA&M.
Ethernet
1
HDLC/ PPP
RPR
IP/ MPLS
other Client Signals
ESCON
Framed Mapped
FICON
Fibre Channel
DVB ASI
Transparent Mapped
GFP Clu3nt-Specific Asf)ects GFP Common Aspects PDH Path
SONET/SDH Path
OTN ODUk Path
Figure 5-3, High-level functional model
Generic Framing Procedure (GFP)
5,3,1
159
GFP Frame Formats
The GFP frame format is designed to support both the multiplexing of multiprotocol PDUs as well as the multiplexing of a number of logical virtual links within the data link. Logical virtual links can be used to support different traffic streams with potentially different higher-layer protocols and with different QoS requirements. Two basic GFP frame formats are defined: GFP client frames and GFP control frames, as illustrated in Figure 5-4. GFP also supports a flexible (payload) header extension mechanism to facilitate the adaptation of diverse data clients. The GFP client and control frame formats are shown in Figure 5-5.
GFP Frames
Client Frames
Client Data Frames
Client Payload Transfer
Con irai Frames
Client Management Frames Client Resource Management
kJie
FnhiHh^i
Idle Time Fills
')A.iUyi F fatness
Unk OA&M
Figure 5-4. GFP frame types
5.3.1.1
GFP Client Data Frames
Client data frames provide the basic payload transport mechanism in GFP. As illustrated in Figure 5-5, client data frames are octet aligned and consist of a GFP Core Header and a GFP Payload Area.
160
Chapter 5
Client Data Frames 1
Core Header
// /
/
/
/
Payload Length LSB Core HEC MSB
/
Core HEC LSB
^''>^^ Payload Area
^' Order
^^''^
Payload Type LSB Type HEC MSB Type HEC LSB
CAI
^^^^'^ ^^—•^"'
CID
0-60 Bytes of
Spare
Extension Headers (Optional)
Extension HEC MSB
\\
Vari«ibl« UrHptN Packets \
PFI UP!
Payload Header
\
3\\ Transmission Byte
1
f
/ /
^^^^^
PTI
Payload Type MSB
t
Payload Inforitiation FfjHMlUfigth
^ ^\
f
1
// // / // // /
Payfoad Length MSB
Extension HEC LSB
^^---^'^
Header shown (others may apply)
Payload FCS MSB
Payload PCS
Payload FCS
\ Payload FCS
\\
\
\
Payload FCS LSB
0x00
(0xB6)
0x00
(OxAB)
0x00
(0x31)
Figure 5-5, GFP frame formats
5.3.1,1.1 GFP Core Header The Core Header supports the datalink management procedures in GFP. The Core Header length is fixed at four octets and consists of a PDU Length Indicator field and a Core Header Error Check field. The Core Header is always scrambled upon transmission (via an exclusive OR operation) with a well-known Barker-like pattern BAB31B0. PDU Length Indicator (PLI) Field: a two-octet field indicating the number of octets in the GFP Payload Area. It is used to extract the encapsulated PDU and to look for the next GFP frame boundary. Core HEC (cHEC) Field: a two-octet field containing an ISO CRC-16 to protect the integrity of the Core Header via single-bit error correction and multibit error detection. The cHEC sequence is calculated over the remaining octets of the Core Header. 5.3.1.1.2 GFP Payload Area The GFP Payload Area consists of all octets in the GFP frame after the GFP Core Header. This variable length area may include from 4 to 65,535 octets. It is intended to convey client layer specific protocol information. The GFP Payload Area consists of two common components: a Payload Header and a Payload Information field. A third optional component, the Payload FCS field, is also provided to protect the contents of the Payload Information field (Payload Headers are protected separately). The Payload Area is always scrambled upon transmission and descrambled upon reception via an ATM-like self-synchronous scrambler.
Generic Framing Procedure (GFP)
161
5.3.1.1.2.1 Payload Header The Payload Header is a variable-length area, 4 to 64 octets long, intended to support data link management procedures specific to the higherlayer client signal. The Payload Header contains two mandatory fields, namely, the Type field and the accompanying Type HEC (tHEC) field. The tHEC protects the integrity of the Type field. Optionally, the Payload Header may include an additional variable number of subfields, referred to as a group as the Extension Header. The Type field specifies the presence and format of the Extension Header. GFP Type Field: a mandatory two-octet field of the Payload Header that indicates the content and format of the GFP Payload Information. The Type field distinguishes between services in a multiservice environment. The Type field consists of a Payload Type Identifier (PTI), a Payload PCS Indicator (PFI), an Extension Header Identifier (EXI), and a User Payload Identifier (UPI), as shown in the top right corner of Figure 5-5. For Ethernet transport, for instance, PTI=0 (User Data), no Payload FCS (PFI=0), and the default Null Extension Header (EXI=0) are used. Type HEC (tHEC) Field: a two-octet field that contains an ISO CRC-16 sequence to protect the integrity of the Type field via single-bit error correction and multibit error detection. The Payload Header in GFP allows the support of multiple transport modes that may coexist within the same transport channel. Three adaptation modes are currently defined. The first mode, referred to as Frame-Mapped GFP (GFP-F), is optimized for packet switching environments where resource management functions are delegated to the native data clients. The Client-specific adaptation sublayer is fairly thin, supporting basic Layer 2 PDU encapsulation functions. This is the transport mode used for native IP, PPP, and Ethernet traffic. The second mode, referred to as Transparent GFP (GFP-T), is intended for delay-sensitive 8B/10B coded applications, where the goal is transport efficiency and transparency of the logical line code data. The Client-specific adaptation sublayer performs 8B/10B codeword recoding for data compression. This is the transport mode used for Fibre Channel, ESCON, and FICON traffic. The transport mode is indicated in the UPI field. The third adaptation mode. Asynchronous Transparent mapping, is a variation of GFP-T that supports selective client character removal to facilitate rate adaptation into a lower rate (compared with the native) transport channel. 5.3.1.1.2.2 GFP Extension Header The GFP Extension Header is a O-to-60-octet set of fields intended to support technology-specific data link information such as virtual link identifiers, source/destination addresses, port numbers. Class of Service, or
162
Chapter 5
extended header error control information. The type of Extension Header is indicated by the content of the EXI bits in the Type field. Three Extension Header Types are currently defined: Null Extension Header: the default extension header when the entire GPP payload is dedicated to a single service (as indicated by the UPI field). Linear Extension Header: a two-octet extension header that supports sharing of the GFP payload across multiple clients in a point-to-point configuration. Ring Extension Header: an 18-octet extension header (currently under study) that supports sharing of the GFP payload across multiple clients in a ring configuration. Extension HEC (eHEC) Field: a mandatory two-octet field that contains an ISO CRC-16 check sequence to protect the integrity of the contents of the Extension Header via single-bit error correction (optional) and multibit error detection. 5.3.1.1.2.3 Payload Information The Payload Information field contains the framed PDU. This variablelength field may include from 0 to 65,535 - X octets, where X is the size of the Payload Header. It may include an optional Payload PCS field. The client user/control PDU is always transferred into the GFP Payload Information field as an octet-aligned packet stream. The payload may be a single layer-2 MAC frame, via the frame-mapped GFP adaptation mode, or multiple layer-1 line codes, via the transparent-mapped GFP adaptation mode. 5.3.1.1.2.4 Payload Frame Check Sequence (FCS) The Payload Frame Check Sequence (FCS) is an optional, four-octet long, frame check sequence. It contains an HDLC-like CRC-32 check sequence that protects the contents of the GFP Payload Information field. A value of 1 in the PFI bit within the Type field indicates the presence of the Payload FCS field. Unless otherwise stated, corrupted GFP frames are passed to a client adaptation process for local handling according to clientspecific rules. 5.3.1.2
Client Management Frames
GFP provides a generic mechanism to propagate client-specific source adaptation information, such as performance monitoring and OA&M information, to end-systems. Currently, the only client-specific facility defined is a Client Signal Fail (CSF).
Generic Framing Procedure (GFP)
163
5.3.1.2.1 Client Signal Fail (CSF) CSF is a message that may be sent from the GFP source-adaptation process to the far-end GFP sink-adaptation process upon failure detection in the ingress client signal. Detection rules for client signal failure events are by definition client-specific. Figure 5-6 illustrates the use of CSF messages.
GFP Link
nana ^ ^ ^ --.^ I • Loss of Signal (LOS) ; f • Loss of Client Character Sync (LCS) S
Client Signal Fail: .LOS . LCS
- Loss of clock/frame - Running disparity violations
Figure 5-6. Example of Client Signal Fail usage in GFP
The CSF indication is a special type of GFP Client Frame consisting only of a Payload Header and no Payload Information field. The Payload Header consists of a Type field with its accompanying tHEC, and an Extension Header, if applicable to the encapsulated client signal. In the Type field, the PTI subfield is coded as Client Management, the PFI subfield is set to 0 (no Payload FCS), and the EXI subfield is set to the applicable Extension Header type. The UPI subfield is used to indicate the type of client signal failure. Two generic types of failure defects can be reported: • Loss of client signal (UPI=0) • Loss of client character synchronization (UPI=1) Upon failure detection, the GFP client-specific source adaptation process may send periodic far-end CSF indications. The GFP client-specific sink adaptation process should clear the defect condition either 1. after failing to receive a number of consecutive CSF indications (value of 3 is suggested), or 2. after receiving a valid GFP User Frame. The handling of incomplete GFP frames at the onset of a CSF event should be consistent with the GFP error-handling procedures.
164
5.3.2
Chapter 5
GFP Control Frames
GFP Control frames provide in-line link control mechanisms for GFP. This information is indicated via the lower values of the PLI field (0-3). Currently, only the GFP Idle Frame function is specified. The remaining PLI values are under consideration for dark fiber extensions. It is expected that such an in-band channel would use very small payload areas to minimize interactions with life FP Data frames. Note that it is not expected that such GFC control will have the same format as the GFP user frames, but at the very least they should incorporate a CRC-16 for the control message payload using the same generation procedure as for the cHEC computation. 5.3.2.1
GFP Idle Frame
The GFP Idle frame is a special four-octet GFP Control frame. It consists of only a GFP Core Header with the PLI and cHEC fields set to 0. The GFP Idle frame does not contain a Payload Area. It is intended as a filler frame for the GFP transmitter to facilitate the adaptation of the GFP octet stream to any given transport medium. The GFP Idle frame format is shown in the bottom right corner of Figure 5-5.
5.3.3
Client-Independent Procedures
5.3.3.1
GFP Frame Delineation
One important function in GFP is to identify the PDU boundary at the time of link initialization and also after packet delineation loss. The GFP receiver state machine is shown in Figure 5-7. Under normal conditions, the GFP receiver would be operating in the Sync state. The receiver examines the PLI field, validates the incoming HEC field, extracts the framed higher-layer PDU, and then rolls over to the next GFP Header. As soon as an uncorrectable error occurs in the GFP Header (that is, the HEC fails and more than a single bit error is detected), the receiver enters a Hunt state. It starts looking for the boundary to the next GFP PDU by moving one bit/byte at a time. Assuming that this bit/byte starts a new frame, the receiver checks the first four octets to see if they form a valid GFP Header (that is, a GFP Header where the HEC field checks out against the content of the PLI field). If the check succeeds, the receiver tentatively assumes that it has identified the frame boundary; otherwise, it shifts forward by one bit/byte and checks again. Boundary acquisition, and hence link resynchronization, is declared after the GFP receiver detects N consecutively correct GFP Headers. The GFP receiver can then return back
Generic Framing Procedure (GFP)
165
to the Sync state. The GFP frame delineation procedures are based on selfdelineation/self-synchronization principles illustrated in Figure 5-8.
Frame-by-Frame Core Header Correction Disabled
Frame-by-Frame Core Header Correction Enabled
2ndcHEC match
O
Pre-Sync State
Sync I State c:^
Noncorrectable Core Header Error
Correctable Core Header Error
NocHEC match
Figure 5-7. GFP state machine
rr Hun! Stale
Octet or Bit synchronous stream
CL
X
[_
Q-
O LU X o
o j
LU X o
Payload Area
•••
cHFrPnilJ '^' l'**'^ cHEC Fail I '^' '"*•='
i
PLI [c*CC
cn c
Valid
PLI Bytes
P U Bytes c i E C Matc^
1 t^
Paytoad Area
1
fcHECFaii' ""^ i****^!
J
Payfoad Area
3
Pre-Sync State
PLI Byies c HEC Mote b
Sync state
1•
Figure 5-8. Link synchronization and frame acquisition
The HEC-based frame delineation procedure permits sophisticated traffic engineering, flexible QoS aware routing, better partitioning of SONET/SDH bandwidth, and multiservice integration, either at the GFP layer (via Header Extension options with the GFP Payload Header) or via native Layer 2 (e.g., Ethernet) or Layer 3 (e.g., IP) mechanisms. Commercial component interconnect interfaces such as MII/GMII (IEEE 802.3) for the Ethernetrelated layers and SPI-3/SPI-4 (OIF) for the optical layer are readily available to facilitate the integration of system components, promote feature interoperability and decrease system development costs.
166 5.3.3.2
Chapter 5 Frame multiplexing
GFP Client and Control frames from multiple ports and multiple client types are multiplexed on a frame-by-frame basis. GFP does not impose any constraints on the choice of scheduling algorithms, since traffic-handling aspects tend to be client specific. In general, when there are no other GFP frames available for transmission, GFP Idle frames shall be inserted, thus providing a continuous stream of frames for mapping into an octet aligned physical layer. CMF, other than CSF, are to be sent opportunistically to minimize contention with client data frames. 5.3.3.3
Link Scrambler
Link-level scrambling is required to achieve adequate link transparency and bit density when data is mapped directly over the SONET-SPE or SDHAUG. Equivalent requirements are also anticipated for WDM-based solutions. A self-synchronous scrambler with generating polynomial 1 -I- x"^^ is specified. This scrambler is applied only to the GFP Payload Area in a manner similar to the link scrambler application for ATM over SONET/SDH.
5*3.4
Client-Dependent Procedures
The first step for a transport integration mechanism is a common signal convergence mechanism to map any native bit-stream into the transport channel and provide for signal rate adaptation and minimal OAM&P functions. The native adaptation mechanism provided by GFP is frame based and allows the segmentation of the physical channel into fixed- or variablesize containers or GFP frames. Three modes of client signal adaptation are provided with GFP: Frame Mapped, (Synchronous) Transparent Mapped, and Asynchronous Transparent Mapped modes. Table 5-1 summarizes the major client signals supported by each mode. Table 5-1. Summary of client signals supported by GFP Transparent Mapped Frame Mapped Ethernet MAC Gb Ethernet Fibre Channel HDLC-like/PPP FICON MAPOS ESCON RPR MAC FC-BBW2 AVB DSI MPLS
Async Transparent Mapped FC-BBW3
Generic Framing Procedure (GFP) 5.3.4.1
167
Frame-Mapped Mode (GFP-F)
The Frame-Mapped adaptation mode is a more flexible adaptation mode that is suitable for either full/subrate point-to-point or multipoint packetbased applications. Adaptation is accomplished by mapping upper-level PDUs (such as HDLC-like/PPP frames, IP/MPLS packets, or IEEE 802.3 MAC frames) into the variable-size GFP frames. The frame structure for mapping an Ethernet/IEEE 802.3 frame onto a GFP frame (assuming a Null Extension Header) is illustrated in Figure 5-9. Linear and Ring extension headers that support client-multiplexing functions in point-to-point or ring topologies are also defined. 1^
^1
ijihK-h i-rame
L r
Core ^ ^ Header ^ ^
Pavload Area
m
PCS
i|:|i:||iE||||
1
/Optional
|jj|:;|2i||t|||:|:; ||j|;i|i|i|||y;|i
4B\/tes
jl
Figure 5-9, Frame-Mapped adaptation mode
For applications where both the transport and bridging capabilities of Ethernet are integrated into the transport NEs, the Frame-Mapped mode is the preferred mode of adaptation, since the physical-layer aspects of both the SONET/SDH and Ethernet interfaces (layer 1) are segregated from the media access control (layer 2) aspects. Since the same mode of adaptation is applied to either point-to-point or multipoint configurations, service providers can deal with these two styles of application with the same provisioning and management procedures. Thus, for instance, if a customer wishes to migrate from a point-to-point transport service to a multipoint transport service, both these services can be delivered from the same service interface without further reconfiguration of the preexisting end-points. 5.3.4.2
Transparent-Mapped Mode (GFF-T)
The Transparent-Mapped adaptation mode (currently defined for 8B/10B encoded signals only) is particularly suitable for full-rate point-to-point applications requiring very low delay and delay jitter. Full-rate means that the entire capacity of the local physical interface is supported across the end-
Chapter 5
168
to-end path. Client adaptation is accomplished by preprocessing the incoming signal to remove the native link-layer codewords, postprocessing this raw data to the characteristics of the new transport media, and mapping the postprocessed data into fixed-size GFP frames, as illustrated in Figure 510. The detailed processing steps for 8B/10B encoded signals are discussed in Section 5.3.4.4. This mode is intended for applications that seek to emulate a native physical interface with very strict packet delay, loss, and throughput requirements (e.g.. Fibre Channel, FICON, and ESCON).
GFP-T Frame Payload Area
Core Header " ^ PU 2 Bytes
cHEC 2 Bytes
Payload Header
PCS
8X64B/65B + 16 Superblocks
4 Bytes
(Optional 4 Bytes
64B/65B # 1 1 64B/65B # 2 |
64B/65B Superblock (Flag bits carried in last octet of the super-block)
^ w
64B/65B # N - l [ 64B/65B # M | r l 1 72 ...| F8 1
^\.^^^ ^\.^
CRC-1$ MSB
1
CRC-16 I^B
1
1
CCL#1
1 CCI#1
1
N
CCL#n 1 CCI#n
1
DCI#1
1
64B/65B block (minus Flag bit)
DCI#8-a
Figure 5-10. Transparent-Mapped adaptation mode (GFP-T)
5.3.4.3
Asynchronous -Transparent Mapped Mode
The Asynchronous-Transparent adaptation mode (currently only defined for 8B/10B encoded signals) is a variant of GFP-T that is particularly suitable for full-rate point-to-point applications requiring a trade-off between low-to-medium delay and delay jitter versus bandwidth efficiency. As with GFP-T, signal adaptation is accomplished by mapping link-layer codewords into fixed-size GFP frames, as illustrated in Figure 5-7. But unlike GFP-T, this process requires that all signal components are adapted into a GFP frame. Thus, codewords associated with interframe fills may be removed, or entire codeword sequences related to link-control functions may be extracted, processed, and modified prior to their mapping to GFP. As such, GFP-A requires a certain level of client signal awareness but not complete processing of the native L2/L3 PDUs. This mode is intended for applications
Generic Framing Procedure (GFP)
169
that seek to emulate a native physical interface with packet delay, loss, and throughput requirements but that can take some level of delay, such as asynchronous Fibre Channel applications. 5.3,4.4
8B/10B Client Processing in GFP-T
GFP provides either full-rate or subrate adaptation for 8B/10B line coded signals, which is the prevalent line code in local area networks. These signals consist of a 10-bit character encoding either 8-bit data or control information. In GFP-T, an 8B/10B data codeword is decoded into its original 8-bit value, and an 8B/10B control codeword is decoded into special GFP-T control characters. The 8B/10B control codewords are mapped into one of the 16 possible 4-bit Control Code Indicators for the 8-bit control characters available in transparent GFP. r> Leadi Leading Bit 8 byte block 8 X 65B blocks = 520 bits
I I i I I
rr
O
Group 8 X 65B blocks
A
Rearrange Leading Bits at end of the block
^ ^ Generate & append CRG-16 ® check bits to form [536,520] superblock. ^ ^ n||| I 0 ™*-'/-^ Payload FCS (4 bytes) M-'
N X [536,520] Superblocks
Form GFP Payloads with N X [536,520] Superblocks. Append GFP Payload Header and (optional) pQg
0 Append Core Header and Scramble Payload Area with x43+i SSS (Core header not scrambled.)
Core Header (4 bytes)
Figure 5-11. GFP-T adaptation process
5.3.4.4.1 Generating GFP-T 64B/65B codes The decoded 8B/10B characters are mapped into a 64-bit/65-bit (64B/65B) block code. The components of the 64B/65B block code are illustrated in Figure 5-12. The leading bit of the 65-bit block, the Flag bit, indicates whether that block contains only 64B/65B 8-bit data characters or whether client control characters are also present in that block. (Flag bit = 0 indicates data octets only, and Flag bit = 1 indicates at least one control octet in the block). Client control characters, which are mapped into 8-bit
170
Chapter 5
64B/65B control characters, are located at the beginning of the 64-bit block payload if they are present in that block. The first bit of the 64B/65B control character contains a Last Control Character (LCC) flag bit, which indicates whether this control character is the last one in this block (LCC = 0) or whether there is another control character in the next octet (LCC = 1). The next three bits contain the Control Code Locator, which indicates the original location of the 8B/10B control code character within the sequence of the eight client characters contained in the block. The last four bits, the Control Code Indicator, give the four-bit representation of the 8B/10B control code character.
_D2_
_Da_
JM_
_Q5_
Dfi
n?
jQa_
_D4_
_D5_
JIL. _Qg_
_Da-.
-XLL n,hhh,C;
_Qi.
_D2_
J22_
JM-
Si5-
-DQ-
_Di-
_D2_
_Da_
_[M_
_D5_
Ql
_D2_
_Da_
_D±_
_Di_
_D2_
_xia_
O.fff.Ofi
_m_
_D2^
1 .hhh.C; n.nno.c:
1 hhh nd 1 nnn.C:^ n.ridd.Ci
1 hhh C. 1 -r.nr.,C?\ 1 dridc4 n.fififiCi
l.aaa.CI l.hhh.Ca
tcm.ca^ 1.rldd.C4l.RRR.Cf^
1,afiri,C1 Lhhh.O: i.GcaCa 1.rJf1fl,C4l.ftfiR.Cfl Lfff.Cfi n.fiflfl.C'
-D2-
_Di.
1,aaa,C1 l.hhh.ca 1 ,nnn,0/ 1,fif1ri,c4l,ftftR,0fl 1,fff,Cfi l.flfln.C -bit representation of the Ist control code's original position • bit representation of the 2nd control code's original position hhh s 3 CI s 4 Dl = 8
- bit representation of the 8th control code's original position 'bit representation of the Ah control code -bit representation of the /th data value In order of transmission
Figure 5-12. GFP-T 65B/64B code components
5.3.4.4,2 Adapting 64B/65B code blocks into GFP Superblocks To preserve the octet alignment of the GFP-T signal with the transport channel, the first step in the adaptation process is to group eight 64B/65B codes into a Superblock (Step 2 in Figure 5-11). The leading (Flag) bits of each of the eight 64B/65B codes are grouped together into a first trailing octet. The sixteen bits of the last two trailing octets are used for a CRC-16 error check over the bits of this Superblock (Step 3 in Figure 5-11). N 65B/64B codes (and associated CRC) are packed into a single GFP frame. Assuming no Payload FCS and a Null Extension header, the resulting GFP frame is [(A/^x ((65 x 8) + 16) + (8 x 8)] bits long, where N is the number of superblocks in the GFP frame. The value of A^ depends on the base rate of the client signal and on the transport channel capacity.
Generic Framing Procedure (GFP)
171
5.3.4.4.3 Error control with Transparent GFP The 16 error control bits in a Superblock contain a CRC-16 error check code over the 536 bits in that Superblock. If the GFP-T demapper detects an error, it should output either a lOB Error control character(s) or a 1 OB Unrecognized control character(s) in place of all the client characters contained in that Superblock. The generator polynomial for the CRC-16 is G(x) = x^^+ x^^ + x^^ + x^^ + x"^ + x^ + x^ + X + 1 with an initialization value of zero, where x^^ corresponds to the MSB and x^ to the LSB specially selected for this application.
5,4.
IMPLEMENTATION CONSIDERATIONS
All the procedures in the GFP state machine may be performed either on a bit-by-bit or byte-by-byte basis, making the equally suitable to bit and byte synchronous channels. If either 4-byte or 8-byte parallel processing is deemed necessary, then all the operations described here, including CRC computations, can be performed in parallel. The standard is open as to design choices for certain design parameters of the virtual framer and options for link scrambling.
5,4.1
Virtual Framer Management
The procedures described so far do not explicitly constrain the potential ways framers may be handled when returned to the Hunt state (after failing to produce A^-7 consecutive HEC matches), or when all the available framers are in use and a new potential GFP header is detected while in the Hunt state. There are a few implementation options available depending on the amount of link configuration information available to the receiver, the link synchronization performance objective, and the implementation complexity. Below we identify three simple implementation options that do not exploit the information conveyed in the assumed PLI fields. One design objective may be to maximize the chance to synchronize the link at the first available opportunity. The implementation (Option 1) could then allocate enough framers to guarantee, with a reasonably high probability, that one of the M framers will succeed in capturing the first incoming GFP frame boundary. For implementation simplicity, framers that fail to yield the proper GFP frame boundary would not be reused, and any further HEC field matches (beyond M) while in the Hunt state would be ignored. If all the framers fail to yield the proper GFP frame boundary in the first pass, then the resynchronization procedure must be restarted from scratch, A drawback in this approach is that the time to frame delineation
172
Chapters
could be large if the incoming GFP frame is missed, particularly in scenarios where the BER is high, since each of the failed framers may have been pointing up to 64 Kbytes into the incoming byte stream. When the receiver knows the Maximum Transmission Unit (MTU) for the link, the synchronization time can be further improved (valid PLI fields can only point up to MTU bytes into the incoming byte stream) at the expense of further implementation complexity. Reuse of failed framers can be exploited to further improve the link synchronization performance. Given the size of the HEC field and typical Internet traffic profiles, the chances of a large number of random HEC field matches in a PDU may be deemed rather unlikely. An alternative design philosophy (Option 2) may assume that a large number of active framers is indeed an indication that the GFP receiver is facing such an unlikely event. Thus, the receiver may just reset the ''oldest" of the framer when all available framers are in use and a new HEC field match is detected while the receiver is in the Hunt state. This approach has the advantage of decreasing the probability of "flushing" the GFP frame boundary with a hard reset of all framers. Alternatively, the framer with the farthest time to frame may also be a good candidate for reset. Yet this approach requires sorting according to the expected time to frame. We evaluate both approaches in Section 5.6.
5.4.2
Scrambler Options
The need for pay load scrambling was identified as an afterthought in the original POS specification. For expediency, the technical community selected an ATM-like self-synchronous scrambler with generating polynomial 1 + x'^^. Below are some considerations concerning the choice of a self-synchronous scrambler: The l+x"^^ scrambler is known to exhibit the poorest bit randomization property among all degree-43 polynomials [18]. The state self-synchronous scrambler is affected by the input data. It is relatively easy to generate input packets that yield a periodic bit pattern (with a period of 43 bits) on the link, even after applying the SONET/SDH scrambler. Although today's PDUs often contain as many as 1536 bytes, they can be as large as 64 Kbytes. The periodicity increases the probability of low bit transition density to 10% to 20% instead of the desired 50%. While the situation is not as serious as the one without the additional scrambler (where a zero transition density can be created relatively easily, as shown in Table 5-2), some transmission equipment may be sensitive to even this low transition density. The self-synchronous nature of the scrambler multiplies errors. In particular, the receiver perceives every bit error on the link as two bit errors (separated by 43 bits). Similar multiplication of errors occurs when there are
Generic Framing Procedure (GFP)
173
two or more errors on the link. Error multiplication interferes with the error correction in the header as well as in the payload. In some environments, the link payload may require a much more powerful FEC that would be required in the absence of error multiplication in order to achieve a given level of burst error protection. Error multiplication also affects the error detection capability of the PCS. In particular, the CRC-16 polynomial is inadequate in the presence of error multiplication, even with the maximum PDU size of 1536 bytes. Table 5-2. Probability distribution of bit transition density
TRANSITION DENSITY 0/43 2/43 4/43 6/43 8/43 8/43
EVENT PROBABILITY 2.0x10" 2.0x10'° 2.8 X 10"^ 1.0 xlO"^ 4.0x10"^ 4.3x10"'*
AVERAGE TIME PER EVENT AT 0C-12C 322 Years 177 Days 20 Hours 33 Minutes 1.2 Minutes 4.8 Seconds
Thus, one is driven to consider an alternative scrambler for GFP. Below we discuss a new independent set-reset scrambler, its operation, and the method for synchronizing the descrambler with the scrambler. All bits in every GFP PDU following the GFP header are scrambled using an independent scrambler with polynomial 1 + x + x^'' + x^^ + x'^^ The transmit scrambler and receive descrambler can be implemented using shift registers with 48 stages that are set to all-ones when the link is initialized. Each is refilled with all-one bits if the value in the shift register ever becomes all zeros. This scrambler, shown in Figure 5-13, is not reset at the beginning of each frame, as is the SONET/SDH x^+x^-i-1 scrambler, nor is it modified by the transmitted data, as is the ATM self-synchronous scrambler. Instead, the two ends are kept in synchronization using special GFP messages.
^
1 1
o<
rx.'^.* r^4
r^4
, W p=|| Y [ = f |
1 L-J ^"
1
DpR '
"
Dp7 1
"
1
=
r=fl 1 i
DoA 1 1
r
Figure 5-13. An independent setreset scrambler
1
D47
1i
r
:
174
Chapter 5
Each XOR is an exclusive-or gate, which is equivalent to a modulo-2 adder. Each Dn block is a D-type flip-flop clocked on the appropriate data clock rate. The scrambler is clocked once after transmission of each bit of GFP data, whether or not the transmitted bit is scrambled. When scrambling is enabled for a given octet, the OUT bit is exclusive-ored with the raw data bit to produce the transmitted bit. The scrambler and descrambler are kept synchronized by means of periodic GFP messages. Since the scrambler state exchange by GFP messages does not rely on the SONET/SDH structure or overhead bytes, the whole procedure can be used for GFP over WDM or any other core transport layer. To generate a scrambler state message, a snapshot is taken of the contents of D47 through Do at the point where the first scrambler state bit is sent. The receiver of a scrambler state message must run the CRC-16 payload check and execute the single bit header error correct algorithm over this message. If the CRC-16 detects multiple bit errors, then the message is dropped and is not processed further. Additional mechanisms are also provided to reduce the likelihood that a falsely corrected scrambler state message with multiple bit errors corrupts the running scrambler state.
5.5,
PERFORMANCE
Synchronization errors and delays affect the adequate performance of the transmission layer. In this section, we evaluate the extent of these performance impairments on GFP and compare the link efficiency with that of the HDLC-framed PDUs.
5.5.1
Probability of GFP Frame Delineation Loss (FDL)
GFP header integrity is key to the proper operation of GFP, and the HEC field ensures a very low error probability on the PLI field. Whenever the GFP header CRC fails, the value in the PLI field can no longer be trusted (even if the receiver is still synchronized). In those instances, the frame boundaries must be reacquired by entering the Hunt State. Since the header field is four octets long, the probability that the header contains an error is about 32p, where p is the link's Bit Error Rate (BER). Thus, the loss of synchronization would be an order p event in the absence of header error correction: too weak for gigabit rate operations. With all single bit errors corrected, the loss of synchronization becomes an order p^ event, a significant improvement in performance against random bit errors. Indeed, it is easy to show that for low p values, the probability of synchronization loss is given by
Generic Framing Procedure (GFP)
P(FDL) =. 1 - (1 - p + HpXl - pr
175
^^ ^ ^ ^ ^
(5.1)
where H is the size of the GFP core header in bits. This relation applies to both bit-synchronous and byte-synchronous links. Thus, from Eq. (5.1), for a BER of 10'^ the FDL is roughly once every 10^^ frames, while for a BER of 10"'^ P(FDL) is as low as once every 10^^ frames. The FDL events are even less common on links with lower BERs, as illustrated in Table 5-3. Table 5-3. FDL probability as a function of link BER
BER 10-^ 10"^ 10"^ 10"^^
P(FDL) 5x10-^^ 5x10"^"^ 5x10-1^ 5x10-^8
GFP is primarily designed for core networks where the fiber BERs are generally better than lO'^l Given a PDU size of 48 bytes and a BER of 10 ^ even at an OC-768 (40 Gbps) rate, GFP would lose synchronization only once every 224 days (on average) due to random bit errors. Frame resynchronization events will next be dominated by the frequency of burst errors, which in fiber transmission systems rarely occur as often as a few times each day. In the presence of burst errors, frame boundary resynchronization needs to take place, irrespective of whether the packet size is fixed or variable. Therefore the overall number of frame resynchronization events would be virtually the same for ATM and GFP. In the case of HDLC framing, a frame boundary loss occurs much more frequently because a single error in a flag or an error in a data octet causing it to look like a flag will result in frame boundary loss. In the case of HDLC, frame loss occurs as an order p event.
5.5,2
Probability of False Frame Synchronization (FFS)
False link synchronization events occur when A^ false HEC fields are matched in sequence in the incoming byte stream. This event requires A^ random CRC-16 matches before the GFP link is properly synchronized. With the header CRC-16 operating only in detection mode after a loss of link synchronization is declared, the probability of a random set of four octets passing the CRC-16 is q=2'^\ or ^=MTU/2^^ when the working MTU for the link is known. Then
176
Chapters PFS
(5.2)
The duration of this false synchronization event is bounded in time by, at most, one maximum-size PDU.
5.53
Probability of Frame Unavailability (FUA)
GFP allows multiple candidate HEC field matches to create up to M simultaneous checkers. Each such checker assumes the frame boundary has been identified. The receiver (virtually) jumps to the next assumed frame boundary and checks for a valid HEC field match. When all M checkers fail to capture the next incoming GFP header, the framer state machine must be reset and link resynchronization reinitiated. During such events, the framer state machine has become essentially unavailable. To quantify the link synchronization performance, one must thus quantify the probability of such events. Since the probability that the receiver fails to detect multiple header errors is quite small, the hunt process in GFP will typically start at the beginning of the current GFP payload. Subsequent resynchronization attempts, because of the framer unavailability events, would then start at a random point in some GFP PDU. Below we evaluate PFU for the two receiver implementation options described earlier, that is, either the first M HEC field matches are preserved (Option 1) or only the last M HEC field matches are preserved (Option 2). For the remainder of this discussion, it is assumed that N=2. 5.5.3.1
Option 1: Resynchronization reinitiated after all M active framers fail
In this scenario, resynchronization events are triggered after the last of the M spawned framer processes fails to capture the next incoming GFP PDU. The hunt process evolves along one of two disjoint paths. In the first path (FUAi), the search starts just at the beginning of the payload of the incoming GFP PDU, since the probability of undetected GFP header errors is very low. This search occurs any time the receiver is in the Sync state and multiple bit errors are detected. This path can only capture the next proper HEC field when less than M false HEC field matches are encountered after examining l-J bytes/bits, where / represents the size of the current GFP PDU. In the second path (FUA2), after the first path fails to produce a proper GFP PDU, the hunt process restarts at an arbitrary point within some GFP PDU. Any position within the payload of such GFP PDU is equally likely to
Generic Framing Procedure (GFP)
111
be the starting point of the search. Again, less than M false HEC field matches must be encountered within the residual frame length for the receiver to be in a position to capture the proper GFP header. For an arbitrary frame size distribution PB(1), the probability of next frame unavailability becomes
PFUJ = Y.PB(1)T.I
U
k=M\
1>M
PFU2-
^-
I
([^
^'^s^''^
1,>M l=M
5.5.3.2
(5.3)
J
'i-l D n \
^1
k=MKh
1/(1-'//-'^
Option 2: Reusing the oldest framer after the M-\-l HEC field matches while in the Hunt state
In this scenario, excessive HEC field matches while in the Hunt state do not trigger the reinitialization of the resynchronization process from scratch. Only the oldest framer is reset and reused to store the new HEC field match. The next GFP PDU becomes unavailable only if there are more than M HEC field matches between two consecutive GFP PDUs. Since there is at least one HEC field match in any GFP PDU of size /, the probability of next frame unavailability becomes
P(Ff/A) = X^fi(0£ 1>M
k=M
ViX-q)'-'-'
(5.4)
V ^ y
The computation of the frame unavailability probability so far ignores the impact of the links' BER. With the header CRC operating only in detection mode while in the Hunt state, the next GFP PDU boundary will also be missed whenever the next GFP header is in error. Assuming independent random bit errors, then PFU(BER) = P,,, + P{FUA){\ - P,,,)
(5.5)
where P^er is the uncorrected header error probability, P/^^^ =l-(l- p)^ , and P(FUA) is given by Eq. (5.3) in the case of Option 1, or Eq. (5.4) in the case of Option 2.
178
Chapter 5
Figure 5-14 and Figure 5-15 show P(FUA) as a function of the GFP PDU size and the number of framers over octet- and bit-synchronous channels, respectively. In these figures, Ml refers to P(FUA2) under Option 1, while M2 refers to Option 2. As expected. Option 2 gives slightly higher values for P(FUA), particularly for larger PDU sizes. P(FUA) values for bitsynchronous links are necessarily higher, as a larger number of events must be examined in between consecutive GFP PDUs.
64
128
256 384
512 1024 2048 3072 4096 8192 16384 32768 65535
GFP PDU (octet)
Figure 5-14. Probability of next frame unavailability as a function of the GFP PDU size and number of framers (octet-synchronous channel)
?5 1-E-04
128
256 384 512 1024 2048 3072 4096 8192 16384 32768 65535 GFP PDU size (octets)
Figure 5-15. Probability of next frame unavailability as a function of the GFP PDU size and number of framers (bit-synchronous channel)
Generic Framing Procedure (GFP) 5.5,4
179
Frame Acquisition Delay
For any given receiver implementation, the frame acquisition delay is a function of the packet size, the link BER, the number of framers, and the required number of consecutive HEC field matches. The most relevant measure of this impairment is the mean time to frame (MTTF), that is, the average time it takes to regain synchronization when a certain number, M, of framers is used in the Pre-Sync state. When M false matches create M checkers, the link resynchronization process is reinitiated. The process will start again after losing at least one GFP PDU for link-synchronization purposes. It is also possible that the true GFP frame boundary will be missed because of an error in the header. A checker with a true frame boundary may also revert to the Hunt state when an error occurs in one of the subsequent N-1 GFP headers. Therefore, the MTTF is a function of the probabilities calculated above and the BER. Assuming that the boundary search is initiated at a random point in the current GFP PDU, there are three main components to the boundary acquisition delay: 1. The time (tj) spent examining the subsequent bytes/bits after the initial HEC field failure, from the beginning of the current GFP payload to all but one byte/bit of the next candidate frame header. This event is a uniformly distributed event over the length of the GFP PDU. 2. The time (^2) spent examining the last complete GFP PDU prior to declaring link (re)-synchronization. This event is uniformly distributed over the length of the GFP PDU. 3. The time (tjhf) spent chasing after the false HEC field matches and restarting the link resynchronization process. This is essentially a geometrically distributed event with rate PFU2. Thus, for our receiver implementations, the MTTF can be expressed as MTTF={E{t,hE{t2)il-Pfui)+flE{t,hn4 = E{t^) + kh) + PpfAtfhM-Pfu2) For either Option 1 or 2, either E(t])=E(l)/2 or E(t])=E(l) depending on whether it relates to an initial synchronization or a resynchronization event, while E(t2)=E(l). E(tjhf) is just the mean value of the (uniformly) randomly matched PLI field, or E(tjhf)=L/2, where L is the largest possible GFP PDU. Figure 5-16 and Figure 5-17 show the MTTF as a function of the GFP PDU size and the number of framers over octet= and bit-synchronous channels, respectively. Option 2 yields lower MTTF values for GFP PDU
180
Chapter 5
sizes up to 4K octets as compared with Option 1, which reflects the impact of the missed frame acquisition opportunities once the framer state machine becomes unavailable. For larger GFP PDU sizes, the lower PFU values from Option 1 exhibit better MTTF performance.
•
2.00
I
t
•
Z) Q
1.75
1.50
t
T
f
64
128
256
1 I 7 384 512
T f T
1024 2048 3072 4096 8192 16384 32768 65535
GFP PDU size (octets)
Figure 5-16. MTTF as a function of the GFP PDU size and number of framers (octetsynchronous link)
2.00
3 Q Q_
1.75
1.50 384
512 1024 2048
3072 4096 8192 16384 32768 63536
GFP PDU size (octets)
Figure 5-17. MTTF as a function of the GFP PDU size and number of framers (bitsynchronous Hnk)
Generic Framing Procedure (GFP)
^^ s^ ^^ ^^^ ^O ^"^ /
181
Z' /
^v^\.^^'/
/
GFP PDU Size (Octets) Figure 5-18. MTTF as a function of the GFP PDU size and BER for M=2 (octet-synchronous links)
Figure 5-18 shows the impact of the links' BER on the MTTF for an octet-synchronous link. Our calculation shows that for Option 2 the MTTF is sensitive to the BER above 10""^ and becomes virtually insensitive to the BER below that range. For core networks, the expected BER is much below this threshold. For this scenario, one can assume insensitivity to the BER and use the resulting MTTF in an initial determination of good values for M and A^ >2. Option 1 again shows the impact of the lost frame acquisition opportunities when the working frame sizes are relatively small compared with the maximum GFP PDU size. Although not shown, the MTTF is still an increasing function of the GFP PDU size in octets. Similar observations apply to bit-synchronous links. For common packet sizes in today's Internet, two parallel framers, using links with a BER of 10"^ or better, provide an MTTF of about 1.5 GFP PDUs. Each framer (checker) needs to check at least two consecutive HEC field matches before declaring proper GFP PDU delineation. This is also the optimal time, since initial synchronization is typically initiated halfway into one GFP PDU, and the subsequent GFP PDU must also be checked and judged correctly framed. The MTTF analysis suggests that even for very large packets (64 Kbytes), M=2 or M=3 seems adequate. Since such large packets are not common in current data networks and since the introduction of real-time services in the IP networks make it more unlikely that such large packets will become more prevalent in future IP networks, M=2 should suffice for most practical scenarios. Note also that, although not shown, the synchronization performance can be further improved both by taking into account the actual contents of the
182
Chapter 5
assumed PLI fields, particularly when the working MTU is less than the maximum possible GFP PDU, and by reusing previously released framers shown to have pointed to a false GFP PDU.
5.5.5
Scrambler Resynchronization Delay
For the independent scrambler, both the transmitter scrambler and the receiver descrambler need to be synchronized for the receiver to be able to send valid PDUs to higher layers. If the scrambler state is transmitted every k GFP PDUs, then it will take an additional k/2 frames for the scrambler states to be synchronized. Of course, if the loss of synchronization is communicated to the transmitter, then the transmitter can suspend the transmission of all user traffic and continuously send only the scrambler state messages. These messages are short (12 bytes long). In this case, the resynchronization will be achieved within the transmission time of 1.5 (short) frames after the transmitter is informed of the loss of synchronization. When random errors occur, the header CRC corrects single bit errors in the GFP header. If the header CRC detects errors in the GFP Header that cannot be corrected, it will send the receiver to the Hunt state. Burst errors will almost always cause the GFP receiver to enter the Hunt state. Generally, burst errors in fiber systems appear to last between 20 and 40 ms. Once the error burst is over, an additional time interval of either 2 frames (for selfsynchronous scramblers) or 2 + k/2 frames (for the independent scrambler) will elapse before link resynchronization is achieved. This interval is insignificant at OC-3c transmission speeds and above.
5.5.6
Link Efficiency
Figure 5-19 compares the datalink transport efficiency between GFP and PPP over HDLC-like framing as defined in RFC-1662 [10]. From the viewpoint of the data link layer, one can readily identify two sources of transmission inefficiencies: framing overhead and payload encoding overhead. Framing overhead is typically associated with protocol control information. Payload encoding deals with transparency issues encountered during data transport, such as avoiding the receiver misinterpreting user data as control information, guarding against malicious attacks, or maintaining bit transparency. GFP introduces a fixed amount of framing overhead — either 8 bytes without an FCS field or 12 bytes with an FCS field. In comparison, PPP in HDLC-like framing requires a minimum of 79 bytes of framing overhead, depending on the size of the FCS field. The HDLC encoding overhead is
183
Generic Framing Procedure (GFP)
variable and loosely bounded. In the best-case scenario, there are no occurrences of the Flag and Escape Control bytes in the data packet and, no encoding overhead is added at all. In the worst case, the packet consists exclusively of Flag and Control Escape bytes, and the encoding overhead is 100%. For purely random data, it is easy to show that the average encoding overhead is about 0.78%. Yet it is not uncommon to find video and compressed voice data traces in which the flag or escape patterns account for 30% to 40% of the bytes. Such large payload size variations can strongly interfere with most QoS management mechanisms, demand looser engineering rules, and hence, decrease overall link efficiency for HDLCencapsulated PDUs.
100
90
^
80
1^ W
70
-•-GFP HP PPP Best -^K- PPP Random "»- PPP Worst
60
50
40
1
40
1
64
1
128
1
256
1
512
1
1024
1
4048
1
8096
16192
GFPPDUSize (Octets)
Figure 5-19, GFP and PPP/HDLC bandwidth efficiency as a function of the PDU size
184
Chapter 5
Other encoding algorithm may be used in place of HDLC byte stuffing. For instance, the Consistent Overhead Byte Stuffing (COBS) algorithm [19] encodes input data into variable-length code blocks by eliminating a target byte from the data stream. Each code block begins with a code byte indicating the length of the code block followed by up to 254 bytes of data. For purely random data, COBS efficiency is about 0.23%. Yet, COBS overhead is also variable, with a best-case overhead of 1 byte for packets without the Flag byte and a worst-case overhead of 1 additional byte for each 254 data bytes.
5.6,
APPLICATIONS
Most of this traffic originates in corporate LANs, which are today over 90% based on Ethernet technology. This situation makes a compelling case for Ethernet as the service interface towards the end users. Public networks, however, are larger SONET/SDH based. This situation makes a compelling case for SONET/SDH as the interface to the public transport network. Below we describe common GFP applications for Ethernet-oriented services. Further applications can also be found elsewhere [20, 21].
5.6.1
Ethernet Private Lines
The simplest GFP application is as a flow mapper/demapper into a TDM channel, such as a SONET/SDH path or an Ethernet segment, for an Ethernet Private Line (EPL) service. Service interfaces to the end-users are standard Ethernet PHYs, while the transport over the TDM network is GFP based. There, GFP provides a mechanism to extend the native datalink protocol (such as PPP or the IEEE 802.3 MAC [22]) over an existing transport infrastructure. This scenario is depicted in Figure 5-20. Note that this approach only requires enhancements to the edge of the transport network as opposed to a brand new data transport infrastructure.
Generic Framing Procedure (GFP) Hybrid Network Element KVlOOMbps 1 GbE
185 Dedicated VCG Per Client
Client A Client BClient C Ethernet MAC Frames
Figure 5-20. TDM-based Private Line Services via GFP & Virtual Concatenation
GFP in combination with SONET/SDH Virtual Concatenation and the Link Capacity Adjustment Scheme (LCAS) [23] provides a simple mechanism not only to adapt the user traffic very tightly to the actual bandwidth demand (e.g., a 1 Gig Ethernet client can be mapped into 21 STSIs or 7 STS-3cs as part of a virtual concatenation group -[VCG], as opposed to burning a single OC-48c -48 STS-ls- lines). VCGs provide a simple mechanism for service providers to offer subrate Ethernet transport services in increments of STS-ls. LCAS provides a flexible mechanism for service bandwidth modifications and failure management in a hitless manner. Note that the same traffic adaptation model can be used to create a wide variety of similar private line services for SANs and broadcast video applications.
5,6,2
Virtual Leased Lines
An application for the GFP Linear Extension Header format allows even finer granular subrate transport services, as illustrated in Figure 5-21. In that scenario, GFP itself is used to create multiple subchannels within the GFP payload rather than relying on the SONET/SDH layer. Each GFP channel can be allocated an arbitrary fraction of the transport bandwidth. Each channel behaves as a Virtual Ethernet Private Line. This approach is particularly attractive for newer multichannel component interconnect interfaces such as SPI-4/SP5. It does require a packet-level scheduler at the GFP layer.
186
Chapter 5 Hybrid Network Element lQ/100Mbps 1GbE
^0 P orts
Client A -I
^
VCG ABC VC-n-xv
Client B — f Client
C^ Ethernet MAC Frames
Packet Fabric
GFP-F Mapper
^
Shared VCG among Clients
TDM Fabric
Figure 5-21. Packet-based Virtual Private Line Services via GFP Linear Headers or VLAN tags
5.6,3
Packet Rings
A third application uses the GFP Linear Extension Header in conjunction with Ethernet Bridging functions to create a packet-based logical ring over a TDM infrastructure, as illustrated in Figure 5-22. In this scenario, there is a point-to-point path between the neighboring network elements. On each link, the entire SONET/SDH link is dedicated to the packet ring. The GFP payload can be shared in any arbitrary fashion among the clients using the ring transport services of GFP. A similar capability could be constructed via the proposed Ring Extension Header. Although the ring procedures are currently under study, it is also possible to reuse alternative ring procedures such as the one currently being developed under the IEEE 802.17 [24] working group.
10/100 Mbps lGt)E
I/O Perls Ethernet MAC Frames
W\
^k-k JU t l A ^ =1^9
\>acket
Fabric_
13
GFP-F f Mapper
Hybrid TDM/Ethernet Network Element
TLS: 1-to-l mapping of access ports to VLANs
10/100
i
IEEE 802. ID/W
I
or IEEE 802.17 Over ~ SONET/SDH Path Patti
\
IP/MPLS to VLAN mapping
VCGs/a/ed
^"^^''e'
/
S---H"--'ir m
;°£g^
rZ
III
Edge Router/:
• • •
ServirP?; .qwit(
Figure 5-22. Packet-based Ring Services via GFP Linear Headers or VLAN tags
Generic Framing Procedure (GFP)
5.7,
187
FUTURE DIRECTIONS
Most of the work on developing GFP so far has been focused on defining client-specific adaptation procedures for a variety of constant bit-rate (Layer 1) or packet-oriented (Layer 2+) client signals. In this regard, it is expected that the direct mappings for other client signals will be enhanced to include the most commonly used Layer 2+ protocols, including IPv4/IPv6, MPLS, and the OSI protocols, among others. Work is also under way to complete the work on a subrate mode for Fibre Channel signals. There has also been much discussion about the need for a new native packet transport mode specially optimized for SONET/SDH and OTN transport networks and the potential use of GFP for this purpose. GFP already provides the means to propagate label-switching information for statistical multiplexing of any number of client signal (via the Extension Headers) as well as the means to support separate in-band or out-of-band management and control channels for client- and server-layer resource management via the GFP Type field. These tools afford the means of defining a lightweight packet transport protocol as an alternative to more established approaches such as ATM or MPLS.
5.8. [1] [2] [3] [4]
[5]
[5] [6] [7] [8]
REFERENCES ITU-T Recommendation G.7041/Y.1303, The Generic Framing Procedure (GFP), 2003. American National Standard for Telecommunications, Synchronous Optical Network (SONET) PayloadMappings, ANSI Tl. 105.02, 2002. ITU-T Recommendation 1.432, B-ISDN UserNetwork Interface — Physical Layer Specification, 1993. ISO/EIC 3309:1991(E), Information Technology — Telecommunications and information exchange between systems — High-level Data Link Control (HDLC) Procedures — Frame Structure, 4th Edition. International Organization for Standardization., 1991. ISO/EIC 4435:1991(E), Information Technology — Telecommunications and Information Exchange Between Systems — High-level Data Link Control (HDLC) Procedures — Elements of Procedures, 4^^ Edition. International Organization for Standardization, 1991. American National Standard For Telecommunications, Synchronous Optical Network (SONET): Physical Interfaces Specifications, ANSI Tl. 105.06, 2000. ITU-T Recommendation G.707, Network Node Interface for the Synchronous Digital Hierarchy (SDH), 1996. ITU-T Recommendation G.709, Interfaces for the Optical Transport Network (OTN), 2001. J. Carlson, P. Langner, J. Manchester, and E. Hernandez-Valencia, "The Simple Data Link (SDL) Protocol," RFC 2823, May 2000.
188 [9]
[10] [11] [12] [13] [15]
[16]
[17] [18]
[19] [20] [21] [22]
[23] [24]
Chapters American National Standard For Telecommunications, Integrated Services Digital Network — Core Aspects of Frame Protocol for Use with Frame Relay Bearer Service, ANSI Tl.618-1991, June 1991. W. Simpson (Ed.), PPP in HDLC-like Framing, RFC 1662, July 1994. A. Malis and W. Simpson, PPP over SONET/SDH, RFC 2615, June 1999. ITU-T Recommendation X.85, IP over SDH using LAPS, 20Q\. ITU-T Recommendation X.86, Ethernet over LAPS, 200\, J. Baldwin, B. Bharucha, B Doshi, S. Dravida, and S. Nanda, "AAL2 — A new ATM adaptation layer for small packet encapsulation and multiplexing," Bell Labs Technical Journal, AprilJune 1997. B. Doshi, S. Dravida, P. Magill, C. Siller, and K. Sriram, "A broadband multiple access protocol for STM, ATM, and variable length data services on hybrid fiber-coax networks," Bell Labs Technical JourndX, JulySeptember 1996. D. Fiorini, M. Chiani, V. Tralli, and C. Salati. "Can we trust in HDLC?" ACM Computer Communication Review, pp. 61-80. 1994. I.Fair, V. Bhagava and Q. Wang, "On the power spectral density of self-synchronous scrambled sequences," IEEE Transactions on Information Theory, Vol. 44, No. 4, pp. 16871692, July 1998. S. Cheshire and M. Baker, "Consistent overhead byte stuffing," Proceedings of SIGCOM'97, September 1997. E. Hernandez-Valencia, "Hybrid Transport Solutions for TDM/DATA Networking Services," IEEE Comm, Magazine, Vol. 40, No. 5, pp. 104112, May 2002. M. Scholten, Z. Zhu, and E. Hernandez-Valencia, "Data Transport Applications Using GFP," IEEE Comm. Magazine, Vol. 40, No. 5, pp. 96103. May 2002. IEEE 802. ID, (ISO/IEC 15802-3:1998), IEEE Standard for Information TechnologyTelecommunications and Information Exchange Between Systems—IEEE Standard for Local and Metropolitan Area Networks—Common Specifications—Media Access Control (MAC) Bridges, 2002 Edition. ITU-T Recommendation G.7042/Y.1304, The Link Capacity Adjustment Scheme (LCAS ),200\. IEEE P802.17, Resilient Packet Rings (RPR), Draft version 2.2, April 2003.
Chapter 6 SYNCHRONIZATION OF OPTICAL NETWORKS An overview of network-level synchronization ^Geoffrey M. Garner, ** Gert H. Manhoudt "^Consultant, ** AimSys BV
6.1.
THE FIELD OF NETWORK SYNCHRONIZATION ENGINEERING
6.1.1
Introduction
The branch of network engineering that studies the distribution and quality of clock signals that are used in the public telecommunications network calls itself synchronization network engineering. In today's telecommunications networks, the clocks in transmission and switching equipment are often required to operate at equal or almost equal frequencies in order to transport signals between them that carry digital information and to do so without introducing single bit errors or bursts of errors. Synchronous operation of equipment that is spread out over a large geographic area requires a distribution network for synchronization information. The effects that cause degradation of this synchronization information can be divided in two categories. First, there is continuous degradation of these signals due to the accumulation of phase noise, caused by imperfect components and designs. This causes jitter and wander on the digital signals that are transported over the network. Too high levels of jitter and wander can cause bit errors, loss of frame, or controlled slips. Second, there may occasionally be a complete failure of a synchronization link, leaving network elements or entire network parts without synchronization information. The design of the synchronization network tries to minimize the effect of both the continuous phase noise accumulation and the effect of incidental
190
Chapter 6
loss of synchronization. Sections 6.2 through 6.4 of this chapter describe the theory and practice of phase noise, while Section 6.5 concentrates on the protection against loss of synchronization reference due to link failures. 6.1.1.1
Short History
The history of the specification of network synchronization, jitter, and wander is closely coupled to the history of digital transmission. Only when transmission became digital did the timing of the symbols on the line become important. The earliest problems to tackle involved jitter. Initially, digital transmission systems were used for point-to-point transmission to replace the Frequency Domain Multiplexed (FDM) systems in use at the time. The advances in integrated electronics made it possible to build the more complex digital Time Domain Multiplexed (TDM) systems. These allowed, in principle, the building of transmission paths that had no degradation, irrespective of the length of the path, as long as the "bits" on the line were recovered in each regenerator without error. To allow TDM systems to operate error free, the bits had to be sent on the line at very regular, equidistant, points in time. Deviation from this ideal was called jitter, and limits were set on its magnitude so as to control the number of bit errors made in the receiver. The next step in the evolution of digital systems was the concatenation of multiple digital systems, by directly patching through the 64 kbit/s DSO digital signals themselves on channel banks. This practice required the clocks of these systems to be equal, because the applied (primary) TDM multiplex method required the timing of the tributary signals to be coupled to the aggregate clock. This system of interconnected TDM systems became a nationwide network when digital switching was introduced. This required all switches and all 64 kbit/s signals to be synchronous. To make sure that this synchronicity was, under all conditions, guaranteed, a specific clock distribution network was deployed. It required the specification clock accuracies, reference switching, and holdover behavior to limit impairment due to wander and frame slip. This was the situation when SDH and SONET were introduced. The SDH/SONET multiplexing system required that the clock distribution network of PDH would no longer be carried over El or DSl trunk signals, but would be shifted to the OC-M/STM-N carriers of SONET/SDH. Moreover, the SDH/SONET network itself needed to be synchronized to avoid accumulation of large amount of wander in its payload signals.
Synchronization of Optical Networks
191
The last step was the introduction of OTN, This network is basically again an analog (FDM) network, but instead of multiplexing RF frequencies, optical wavelengths are multiplexed. But similar to the PDH network, the OTN network itself can operate asynchronously and still transport a synchronous STM-N/OC-M payload.
6,2.
BACKGROUND ON TIMING, SYNCHRONIZATION, AND JITTER
This section provides background on timing, synchronization, and jitter, and their importance in digital networks, A more detailed overview of this subject, with an emphasis on SONET/SDH networks, is given in [1].
6.2.1
Basics of Digital Transmission, Timing Jitter, and Alignment Jitter
At a fundamental level, a network transports client information from an ingress to an egress point over a number of links and nodes. In the case of a digital network, the information is transported as successive bits at a rate that may be constant or variable. Whatever the rate is at the ingress, each bit has associated with it an instant of time at which the bit is transmitted. If the client signal is constant bit rate (CBR), which is the case for Plesiochronous Digital Hierarchy (PDH), Synchronous Optical Network (SONET), Synchronous Digital Hierarchy (SDH), and Optical Transport Network (OTN) signals, and if the client itself has no timing impairments, then the client bit times at the ingress are equally spaced with period T equal to the reciprocal of the bit rate. This is illustrated in Figure 6-1. At a low layer, a bit may be transported over a single link as an analog pulse, with different pulses representing 1 and 0. The specific forms of the pulses are determined by the line coding. For example, a non-return-to-zero (NRZ) line coding represents 1 by a particular voltage or optical power level (depending on whether the link is electrical or optical, respectively) and 0 by a different voltage or optical power level. For simplicity in the examples given here, the levels may be taken as 1 Y or 0 dBm for binary 1 and 0 V or -10 dBm for binary 0.^ Then a stream of NRZ-encoded bits with constant rate and no impairments is represented mathematically as a train of such pulses [2]
^(0- Y.''n8(t-nT)
(6.1)
192
Chapter 6
where ^(0 = rectangular pulse of height 1, extending from t = 0 to t = T a, =n'^ hit (an e {0,1}) T= size of unit interval in units of time (seconds) = l/(bit rate in bits/s) xit) = amplitude function for pulse train at transmitter. An NRZ-encoded bit stream is illustrated in Figure 6-2. In this example, the bit time (sometimes referred to as the significant instant) is associated with the leading edge of the pulse/ The train of pulses, Eq. (6.1), may be represented by an actual signal (baseband signal) or may be used to modulate a higher-frequency carrier. The latter is normally the case in an optical transmission system, where the signal modulates an optical carrier of a given wavelength. At the receiver end of the link, the pulses will be distorted due to impairments in the transmission process. There also are, in practice, timing impairments, which shift the pulses in time relative to the equally spaced ideal time instants. Finally, noise is present due to various sources in the link. The resulting amplitude function for the pulse train at the receiver is [2] y(0= f^aMt-nT-e[nT]-T,)-h?j(t)
(6.2)
where (square brackets denote a discrete-time function) h(t) = distorted pulse due to transmission impairments e[nT] = shift in timing of pulse due to timing impairments 7 = size of unit interval in units of time (seconds) = l/(bit rate in bits/s) y(t) = amplitude function for pulse train at receiver T](t) = noise due to all sources TQ = average propagation delay between transmitter and receiver. An NRZ-encoded bit stream with distortion and timing impairments, and with the average propagation delay TQ removed, is illustrated in Figures 6-3 and 6-4. Figure 6-3 shows the signal with only timing impairments, while Figure 6-4 shows it with both timing impairments and distortion. The function e[nT] is the phase-error function (also referred to as the phase).
Synchronization of Optical Networks
T
2T
193
3T
4T
5T Time
Figure 6-7. Bit times for CBR signal with no timing impairments
Pulse Height 1
1
1
0
1
0
1
(vj
... ol
1r
3T
2T
4T
5T Time
Figure 6-2. Example NRZ-encoded bit stream, with no distortion (amplitude) or timing impairments, and no noise
e[3T] ^
e[2T]—> Pulse Height
1
1
e[5T]—> 1
0
0
1
(V)
• •• 0
T
2T
3T
4T
5T
Time
Figure 6-3. Example NRZ-encoded bit stream, with timing impairments but no distortion
194
Chapter 6
0
T
2T
3T
4T
5T
Time
Figure 6-4. Example NRZ-encoded bit stream, with timing impairments and distortion
recovered data
received data Decision Circuit i i
C*lock Recovery
Figure 6-5. Schematic of sampling and decision process for data recovery
e(s)
eXs)
Figure 6-6. Functional block for clock recovery low-pass filter
To determine whether a received pulse is a 1 or a 0, it must be sampled as close to the shifted pulse time nT + e[nT] as possible.^ A decision circuit compares the sampled value with a threshold. The sampling and decision process is illustrated in Figure 6-5. The mathematical theory of this process is described in detail in a number of references (see, for example, [2]-[5]) and will not be repeated here. However, we note that, in general, a larger magnitude of any impairment or noise source results in higher probability of a bit error. In particular, a larger offset between the ideal sampling time nT + e[nT] and the recovered sampling time nT + er[nT] results in higher
Synchronization of Optical Networks
195
probability of a bit error. The offset between the ideal sampling time and the recovered sampling time, CainT] = e[nT] - eXnT], is referred to as the alignment jitter. The determination of the recovered sampling instant is done with a clock recovery circuit. These circuits are often implemented using phase-locked loops (PLLs). Clock recovery circuits are described in more detail in [2], [6], and [7]. Functionally, the clock recovery circuit acts as a low-pass filter on the actual phase error e[nT\ to produce a recovered phase error eXnT] that is as close to e[nT\ as possible. This process is illustrated in Figure 6-6, where the actual and recovered phase error are related by e^{s) = H{s)e{s) with H{s) the transfer function of a suitable low-pass filter. alignment jitter is related to the actual phase error by e, is) = [1 - H{s)]e{s) s H^ {s)e{s)
(6.3) Then, the
(6.4)
Since H{s) is a low-pass filter, H^is) - 1 - H{s) is a high-pass filter. Therefore, the alignment jitter is equal to the phase-error filtered by a suitable high-pass filter; alternatively, it is equal to the short-term variations in the phase error. Eq. (6.4) also illustrates that, since He{s) is a high-pass filter, sufficiently slow variations in the phase error e{s) will result in very small alignment jitter, i.e., the variations will be tracked. However, fast variations in e{s) above the bandwidth, or corner frequency, of the clock recovery circuit will not be tracked, and will result in larger alignment jitter. Higher-frequency variation in the phase error is referred to as timing jitter. If Figure 6-5 represents the clock and data recovery process at a receiver, the data may be retransmitted on the next link (unless this is the final link), with possibly some buffering at the receiver node. There are two possibilities for the timing of the retransmitted data. First, the data may be transmitted with a clock that is independent of the timing of the received data (and therefore independent of the clock timing the data at the upstream node). In this case, the data buffer in the receiver node will eventually overflow or underflow, resulting in a slip."^ This mechanism is used in Public Switched Telephone Network (PSTN) switches; however, it is not used in the SONET, SDH, and OTN physical layers. Second, the transmit clock may be derived from (or be the same as) the recovered timing from the received data. This mechanism is used in OTN 3R regenerators and in SDH/SONET and PDH regenerators. Because OTN 3R^ and SDH regenerators also must process some overhead, the data must be buffered. In this case, it is advantageous (and, in the case of OTN, required) to filter the
196
Chapter 6
recovered clock in Figure 6-5 with a second, narrower-bandwidth phaselocked loop to further reduce any jitter. This second PLL can help control jitter accumulation over multiple 3R regenerators. However, the data buffer must be sufficiently large to accommodate momentary timing differences between the input data (clock) to the receiver and the filtered clock output of this second PLL. In general, the buffer must be made larger as the bandwidth of the second PLL is made smaller. The second, narrowerbandwidth PLL is sometimes referred to as a dejitterizer,
6,2.2
Jitter Tolerance, Transfer, Generation, and Network Limit
The ability of equipment to accommodate input jitter is referred to as jitter tolerance. As mentioned above, higher-frequency input phase error, or timing jitter, results in alignment jitter, or a difference between the ideal and actual sampling instants; larger alignment jitter means larger bit error probability or bit error ratio (BER). More precisely, a larger alignment jitter means that the actual sampling point is further from the center of the pulse; since real pulses are not exactly square, the actual sampling point is more likely to be on a portion of the pulse that is rising or falling and therefore closer to the decision threshold.^ There is then a greater probability of noise causing the sample to fall on the wrong side of the decision threshold, resulting in a bit error. The effect of jitter on BER can be mitigated by increasing the average signal power relative to the noise level. This approach is used in defining and specifying jitter tolerance. Jitter tolerance for an optical receiver is defined as the minimum level (i.e., minimum peakto-peak amplitude) of sinusoidal jitter of a given frequency that must be accommodated, which results in a 1 dB power penalty.^ Specifically, the average signal power is increased by 1 dB relative to the noise, which results in a drop in BER. Jitter is then added, and its level is increased until the BER is equal to its level prior to increasing the power. This is the level of jitter that the equipment can tolerate, and it must not be smaller than the specified jitter tolerance. Sinusoidal jitter, i.e., sinusoidally varying input phase error, is used in specifying jitter tolerance. This is conservative, because sinusoidal jitter is closer to its positive and negative peak values for a greater fraction of time compared with more realistic jitter distributions (e.g., Gaussian). Since input jitter above the clock recovery PLL bandwidth is not tracked, the jitter tolerance is approximately the same for frequencies above this corner frequency. For lower frequencies, the fact that the alignment jitter is equal to the input jitter filtered by a low-pass filter means that, for a given level of alignment jitter, the input jitter that results in this outcome as a
Synchronization of Optical Networks
197
function of frequency has a -20 dB/decade slope (assuming the PLL has a 20 dB/decade roll-off). To show this, assume the clock recovery PLL has a first-order^ closed-loop response with transfer function H(s) = - ^ ^
(6.5)
where CO] = 27if\ is the PLL corner frequency. Then H^{s)^\-H{s) = -^—
(6.6)
S + CO^
\HX27gf)\ is illustrated schematically in Figure 6-7 (this is an asymptotic approximation; the actual log-magnitude transfer function is 3 dB lower at the corner frequency/i). Let the input signal have sinusoidal phase error with amplitude Ao(f) (the amplitude can depend on frequency). Since the magnitude of the phase error transfer function relates the amplitude of the input phase error to the amplitude of the output phase error (specifically, it is the ratio of the latter to the former), the amplitude of the alignment jitter, A«(/), may be written \(f)
= \H^ {27gf)\\ if) = - 1 ^ =
Then I f2 _^ f2
(6.8)
198
Chapter 6
(log scale)
/, = PLL corner frequency
frequency (log scale)
Figure 6-7. Schematic of log-magnitude transfer function of clock recovery PLL
Now set Aaif) equal to the alignment jitter amplitude that corresponds to a 1 dB power penalty, i.e., the value of alignment jitter amplitude that would result in unchanged BER if the power were increased 1 dB. This value of Aaif) is independent of frequency, and therefore A^if) does depend on frequency. Ao(/) is the jitter tolerance and is illustrated schematically in Figure 6-8. Ao(/) has a slope of -20 dB/decade for frequencies that are small compared with the PLL corner frequency, / j , and is equal to A« for frequencies that are large compared with f\. In practice, jitter is often specified in terms of peak-to-peak values rather than amplitude (i.e., zero-topeak values); this corresponds to multiplying both sides of Eq. (6.9) by 2. The full jitter tolerance mask is illustrated in Figure 6-9 (adapted from Figure L2 of [8] and Figure L2 of [9]), where the frequencies f\ and fi represent the band widths of the dejitterizer PLL and clock recovery circuit, respectively. Similar arguments apply to the dejitterizer PLL, except here the jitter tolerance is determined by the data buffer size. A more detailed discussion of this is given in Appendix I of [8] and Appendix I of [9]. A jitter tolerance mask of the form in Figure 6-9 represents the minimum level of jitter that equipment must tolerate. Therefore, it may be used to define the network limit, i.e., the maximum level of jitter allowed in the network. Since the actual jitter in a network is generally not sinusoidal, the network limit is defined using high-pass filters. The wide-band jitter is defined as the peak-to-peak value of the phase error signal filtered by a highpass filter with corner frequency/i. The high-band jiiitY is defined as the peak-to-peak value of the phase error signal filtered by a high-pass filter with corner frequency fi. The high-band and wide-band jitter must not exceed Ai and A2 (given in Figure 6-9), respectively. The PDH, SDH/SONET, and OTN specifications also define low-pass filters to be
Synchronization of Optical Networks
199
applied when measuring the jitter network limits; these have bandwidths that are more than an order of magnitude greater than the high-band jitter highpass measurement filter, and are intended to represent an upper practical limit to the spectral content of the jitter that can arise in the network.^
(log scale)
20 dB/decade
frequency (log scale)
Figure 6-8. Schematic of jitter tolerance of clock recovery PLL
peak-to-peak amplitude (log scale)
20 dB/decade
A, = dejitterizer PLL jitter tolerance
A2 = clock recovery PLL jitter tolerance
/, = dejitterizer PLL bandwidth
f, = clock recovery PLL bandwidth
frequency (1»8 ^^^1^)
Figure 6-9. Relation between jitter tolerance mask and dejitterizer and clock recovery PLL bandwidths (adapted from Figure 1.2 of Reference [8] and Figure 1.2 of Reference [9])
Clock recovery circuits and dejitterizer filters, in practice, produce some amount of jitter, and this must be limited to prevent excessive jitter accumulation. The jitter generation of a piece of equipment is defined as the peak-to-peak jitter of the output when the input is jitter-free. Jitter
200
Chapter 6
measurement filters must be specified. For OTN [8] and Option 1 SDH [10], high-band jitter generation and wide-band jitter generation are defined, with measurement filters consistent with the jitter tolerance masks and network limits. For Option 2 SDH (SONET) [10], [11], the jitter generation measurement filters for OC-48/STM-16 and lower rates are not consistent with the network limits and jitter tolerance masks; however, the jitter generation and measurement filters are consistent with the network limits and jitter tolerance for OC-192/STM-64. The lack of consistency for SONET OC-48 and lower rates is mainly historical. Finally, timing jitter can accumulate over a chain of regenerators. Early studies of jitter accumulation, based on first-order filter regenerator models, are given in [12]. Later studies using second-order filter models that accounted for gain-peaking are given in [13]. Both types of results are described in [2]. The results show that limiting the regenerator bandwidth limits the jitter accumulation. In addition, the second-order filter results show that gain peaking can cause jitter accumulation to increase sharply after some number of regenerators. More recently, jitter accumulation studies were performed for OTN 3R regenerators using a model that explicitly accounts for noise in the PLL phase detector (PD), voltagecontrolled oscillator (VCO), and input. The model studies are documented in Appendix IV of [8], and the results were used to derive the 3R regenerator jitter requirements in [8]. Some of these results are used in examples in Section 6.4.2.
6.2.3
Mapping and Multiplexing
Often, a client signal is mapped into a server layer where the timing of the server is independent of the timing of the client. This is necessarily true when a number of client signals, each with timing independent of the others, are multiplexed into a higher-rate server layer; in this case, the server timing cannot be simultaneously traceable to all the client timings because the latter are all different. However, there are cases where a single client signal is mapped into a server layer (of slightly higher rate to account for server layer overhead), with the client and server timing independent of each other. Note that even if the multiple client signal clocks are specified to have the same nominal rate, the actual signal rates will not be the same if they are provided by different clocks because the frequency of a clock is never specified exactly; rather, a nominal rate plus a tolerance is specified (frequency tolerance and frequency offset are described in Section 6.2.5). A mapping of a client signal into a server layer with the client and server timings independent of each other is referred to as an asynchronous mapping. A mapping of a client signal into a server layer with the server timing traceable
Synchronization of Optical Networks
201
to the client timing (and no other relation between client and server byte or frame boundaries) is referred to as a bit-synchronous mapping. In PDH, asynchronous multiplexing of lower-rate signals into higher-rate signals is defined. In SONET and SDH, asynchronous, bit-synchronous, and bytesynchronous mappings of PDH clients are defined; however, in the vast majority of cases only the asynchronous mappings are used. In OTN, asynchronous and bit-synchronous mappings of CBR clients are defined. In bit-synchronous mapping, the server timing can be obtained from the client timing using a PLL that multiplies the client clock by the ratio of server to client nominal rate. The effect on jitter is similar to that of a regenerator. However, if the client signal is lost, the server timing will also be lost unless there is an alternative source of timing. One way of providing this alternative is for the PLL to be part of a clock that can enter free-run condition when its input is lost; another way is to have a separate clock whose timing is used when the client input is lost. In either case, requirements would be necessary to limit the phase and frequency transient when the client is lost and server timing switches to another source. In asynchronous mapping, the server timing is independent of the client timing. The client rate must be adapted to the server rate. The most common schemes for rate adaptation use a form of stuffing at the mapper coupled with destuffing and filtering at the demapper (the mapper is sometimes referred to as a synchronizer, the filter plus destuffer are collectively referred to as a desynchronizer). The client signal is buffered at the mapper; the client bits enter the buffer at the client rate and leave the buffer at the average rate of the server payload area (this rate is less than the server rate due to the presence of server layer overhead). In general, these two rates differ; therefore, if this process continues indefinitely, the buffer will eventually overflow or underflow. To prevent this, the buffer fill is monitored and, based on an algorithm whose input is the buffer fill, either extra or less payload information may be transmitted. The extra payload information is transmitted in designated server overhead known as negative stuff or negative justification. If less payload information is transmitted, designated bits or bytes in the payload area of the server frame are filled with dummy information rather than client information; this is known as positive stuff or positive justification. Server layer overhead known as stuff control or justification control indicates in each server layer frame whether a stuff has been done and, if so, whether it is positive or negative; this information is needed by the demapper. The delivery of client bits to the demapper is not uniform. Most of the time the bits are delivered at the actual server rate. However, there are regular gaps due to server layer overhead. In addition, there are either gaps due to negative stuff or extra bits due to positive stuff. The timing
202
Chapter 6
information embedded in this irregular bit stream is referred to as a gapped clock. This gapped clock is filtered by a desynchronizer (usually a PLL) to produce a clock with uniform rate (a regular clock). However, the resulting regular clock does contain some jitter and phase error due to the stuffs and overhead. Typically, requirements are placed on the desynchronizer to ensure that its jitter and wander (i.e., low-frequency phase error) generation (output jitter/wander in the absence of any jitter/wander on the client input at the mapper, and no additional sources of jitter or phase error in the server layer) are acceptable. In most cases, the regular gaps due to fixed overhead are easy to filter because they are of sufficiently high frequency; alternatively, if it is possible to buffer some data, a regular clock that runs at the rate with the fixed overhead removed can be derived from a clock that runs at the rate with the fixed overhead present using a PLL. However, the phase error waveform for the gapped clock that contains stuffs will tend to have a low-frequency envelope that is more difficult to filter. The jitter due to this envelope is referred to as waiting time jitter, and may be more difficult to filter if its frequency is sufficiently low relative to the desynchronizer bandwidth. In almost all cases of interest, the stuff unit is either 1 bit, 1 byte (8 bits), or an integral number of bytes. The number of stuff units per server layer frame is related to the nominal server and client rates and range of client and server frequency offset that must be accommodated. For example, if the nominal server payload area average rate is higher than the nominal client rate (this is the case for multiplexing PDH signals into higher-rate PDH signals and for mapping PDH signals into SDH Virtual Containers (VCs)), there will be a constant nominal positive stuff rate if the client and server frequencies are exactly nominal. The maximum server and minimum client rates (based on their frequency tolerances) determine the required maximum stuff opportunity rate. The minimum server and maximum client rates determine the required minimum stuff opportunity rate (which may be negative indicating negative stuffing). The relation among rate of stuffing, nominal client and server rates, and client and server frequency offsets is quantified for OTN in Appendix I of ITU-T Recommendation G.709 [14]. This is done for both mapping of CBR (e.g., SDH) clients into ODUk and multiplexing of ODUk into ODUm (m > k). The OTN CBR client mappings allow positive and negative byte justification (this is referred to as positive/negative/zero or +7/0/-7 byte justification). In addition, the multiplexing of ODUk into ODUm (m > k) allow positive justification of 2 bytes or 1 byte, and negative justification of 1 byte (this is referred to as +2/+//(9/-7 justification). The mappings used for PDH multiplexing allow positive bit justification (the rates and tolerances are such that negative bit justification would never be needed).
Synchronization of Optical Networks
203
The most straightforward algorithm for determining when to do a stuff is to monitor the mapper buffer and do a positive stuff if the buffer empties below a negative threshold and a negative stuff if the buffer fills above a positive threshold. This scheme is used for PDH multiplexing, with the simplification that only positive (bit) stuffing is needed because the minimum server payload area rate is greater than the maximum client rate; therefore, only one threshold must be considered. When the stuff ratio (the fraction of stuff opportunities for which a stuff is done) is close to a number that is the ratio of small integers, the phase waveform tends to have the low frequency envelope referred to above, and the jitter tends to be larger. The straightforward algorithm is used for OTN CBR client mappings defined in [14]. The CBR client demapper (desynchronizer) requirements of [8] provide for acceptable jitter and wander performance for the CBR clients. The OTN CBR client desynchronizer bandwidth is sufficiently narrow compared to the jitter measurement filters for the STM-N clients. It is possible to consider other more advanced justification algorithms where, for example, client phase information is explicitly carried in the server overhead or where threshold modulation is employed. Such schemes can result in reduced jitter or wander. However, they are not discussed here because the conventional scheme is found to produce acceptable jitter and wander accumulation for OTN. In the case of SDH, the asynchronous mapping of some PDH clients, e.g., DS3 and E4, use purely positive bit stuffing with the conventional algorithm. However, some mappings, e.g., DSl, El, and E3, use +1/0/-1 bit stuffing; i.e., there are two stuff bits per frame and, if the VC~11, VC-12, or VC-3 and DSl, El, or E3 frequencies (respectively) were exactly nominal, one stuff bit would contain data and the other would contain stuff.
6,2.4
Pointer Adjustments
In addition to mapping a client signal into a server layer, it is also often desired to cross-connect client signals at an intermediate switching node. Specifically, the client signals multiplexed within incoming server layer signals may not all have the same destination; it may be necessary to switch them to different outgoing server layer signals. In addition, it may be desired to drop one of the multiplexed client signals at a node and replace it with another client signal of the same type that originates at this node (i.e., perform add/drop). In principle, the cross-connect and add/drop functions could be accomplished by completely demultiplexing and demapping the client signals from each incoming server layer signal at a node, switching the client signals to the appropriate output ports, and remultiplexing them into new server layer signals. In PDH networks and OTN, this is the procedure
204
Chapter 6
that is used. However, in SONET and SDH networks, the cross-connect and add/drop functions are performed with the aid of the pointer mechanism. The SONET and SDH frame formats and multiplexing hierarchies are specified in [15] and [16]. Schematics of the SONET and SDH multiplexing structures are given in Chapter 4 of the present book. SONET and SDH clients are mapped into containers, shown at the bottom of Figures 4-1 and 4-2 of Chapter 4, For example, DSl, El, E3, DS3, and E4 clients can be mapped into C-11, C-12, C-3, C-3, and C-4 containers, respectively. Overhead (the nature of which is unimportant for the discussion here) is added to create a VT1.5 Synchronous Payload Envelope (SPE) (SONET) or VC-11 (SDH), VT-2 SPE (SONET) or VC-12 (SDH), STS-1 SPE (SONET) or VC-3 (SDH), or STS-3c SPE (SONET) or VC-4 (SDH), respectively. The SONET VT and STS SPEs and SDH VCs float in their respective frames. For example, the SONET STS-1 SPE may start anywhere in columns 4 through 90 of the OC-1 frame, and the starting point is indicated by a pointer (specifically, by the HI and H2 bytes in row 4, columns 1 and 2, respectively). The timing for the outgoing SONET or SDH signals (OC-N or STM-N) from a Network Element (NE) where cross-connect or add/drop functions are performed (i.e., where the SONET Line or SDH Multiplex Section (MS) is terminated) is, in general, independent of the timing of the incoming signals. The bytes of each incoming STS-N SPE/VC-N are buffered and then placed in the respective outgoing OC-N/STM-N, with the starting position indicated by the pointer. Since the outgoing timing is independent of the incoming client timing, the buffer fill for each STS-N SPE/VC-N will change over time. In addition, since the long-term average frequencies of the incoming and outgoing signals may be different, the buffer will eventually overflow or underflow and result in data loss if nothing is done. To prevent this, the buffer fill is constantly compared with upper and lower thresholds. If the upper threshold is exceeded, a negative pointer adjustment is performed. Specifically, in the next frame, extra data bytes (e.g., 1 byte for STS-1 and 3 bytes for VC-4) are written to the H3 overhead byte(s). In addition, the starting point of the STS-N SPEA^C-N shifts towards the beginning of the OC-N/STM-N by the number of extra data bytes written. This process is referred to as a negative pointer adjustment. Conversely, if the lower threshold is exceeded, a positive pointer adjustment is performed. Specifically, in the next frame, data bytes adjacent to the H3 byte(s) (e.g., 1 byte for STS-1 and 3 bytes for VC-4) are filled with stuff. In addition, the starting point of the STS-N SPE/VC-N shifts towards the end of the OCN/STM-N by the number of extra data bytes written. This process is referred to as a positive pointer adjustment.
Synchronization of Optical Networks
205
From the standpoint of timing, a pointer adjustment is equivalent to a justification of the same magnitude and sign. A positive or negative pointer adjustment of M bytes results in M fewer or extra client layer bytes being transmitted; this is equivalent to +M/0/-M byte justification (note that here, i.e., in SONET or SDH, the client layer is the VT/STS SPE or VC). The result of a pointer adjustment is a phase step of +M or -M bytes. One main difference between the pointer and justification mechanisms is that the pointer indicates where in the OC-N/STM-N the client (VT/STS SPE or VC) begins. In a justification mechanism, the client framing must be determined separately. However, this has no impact on timing, jitter, and wander. If a client signal traverses a number of SONET/SDH NEs, pointer processor buffer fill variations and pointer adjustments may occur at each NE. The phase difference over a time interval between the client signal at the network egress (i.e., where the client is demapped) and network ingress (i.e., where the client is mapped) is equal to the net change in total fill of all the buffers over that time interval. During normal operation, SONET and SDH networks are synchronized, i.e., the timing for the NEs is specified to be within a certain frequency tolerance of nominal, and phase variation (jitter and wander) is constrained (timing signal imperfections and the characterization of timing performance are described in Sections 6.2.5 and 6.2.6). Under these conditions, the buffer fill variations tend to be relatively slow, and the phase variation is wander (i.e., its frequency content is below 10 Hz; see Section 6.2.5). In addition, a pointer adjustment at the egress node, where the client signal is demapped, results in a phase step equal to the size of the pointer adjustment. This phase step results in both jitter and wander. The jitter can be controlled by filtering the pointer adjustment with an appropriate desynchronizer (in the same manner that jitter due to justifications is controlled, as described in the previous subsection). The long-term wander for an isolated pointer adjustment (isolated means that the time between successive pointer adjustments is long compared with the desynchronizer time constant) is equal to the magnitude of the pointer adjustment; this cannot be reduced as long as the pointer adjustment is isolated. The short-term wander (e.g., the phase variation over various time intervals) can be controlled with an appropriate desynchronizer. Note that pointer adjustments at intermediate nodes impact jitter and wander only to the extent that they result in buffer fill variations (this impacts wander) and pointer adjustments at the final node where the signal is demapped (this impacts jitter and wander).
206
Chapter 6
6.2.5
Timing Signal Imperfections
All timing signals are imperfect, i.e., the phase or frequency as a function of time differs from the desired phase or frequency. First, the frequency of a timing signal may differ from the desired, or nominal, frequency, by a fixed amount. This is referred to as frequency offset. The fractional frequency offset, y, is defined as y = ' - ^
,6.,0)
where V = actual frequency Vo = desired, or nominal, frequency. In Eq. (6.10), y is a pure fraction; it is also often expressed in parts per million (ppm). The frequency tolerance of a clock or timing signal is the specified maximum allowable absolute value for y. Second, the frequency of a timing signal may change with time. The rate of change is the frequency drift rate, which is often expressed in ppm/s. Often, the maximum frequency drift rate is specified for a transient event. Third, a timing signal may contain random phase noise, i.e., the phase error as a function of time is a random process. This can be characterized by power spectral density (PSD, described below in Section 6.2.5.1); additional, more convenient measures are given in Section 6.2.6. Fourth, it was indicated in previous sections that jitter, or the result of passing the phase error process through a high-pass filter, is useful in specifying the performance of clock recovery circuits and 3R regenerators. The specified high-pass filter is typically first order; its corner frequency depends on the signal in question (i.e., it is part of the specification), but is always greater than 10 Hz. Finally, wander is phase variation whose frequency components are less than 10 Hz. It is of interest when considering slip performance and synchronization distribution. The total instantaneous phase error of a timing signal at time t may be written [17]
Synchronization of Optical Networks
207
where 0(0 = instantaneous phase error at time t (UI) D = frequency drift rate (s'^) Oo = initial phase error (UI) (p(t) = instantaneous phase noise random process (UI). The instantaneous phase error may be expressed in radians by multiplying Eq. (6,11) by 2;r, and in units of time (e.g., s) by dividing Eq. (6,11) by VQ. When the phase is expressed in units of time, often the symbol x is used, rather than <[>, i.e., x{t) = XQ + yt'^-Dt^+^
(6.12)
Note that x(t) is sometimes used to represent just the random component of phase error expressed in units of time (i.e., the final term in Eq. (6,12) [17], whereas in Eq. (6.12) it represents the total phase error. Both conventions are used in this chapter, with the definition clearly indicated for each use. 6.2.5.1
Phase Noise'^
The phase noise random process, gj(t) in Eq. (6.12), may be characterized by its Power Spectral Density. In the remainder of this subsection, the phase noise will be expressed in units of time and represented by .^(0- The twosided PSD, Sjc(co), is defined by
2^-^°°
(6.13)
where R^i f) is the autocorrelation function. /?^( f) is defined by R^(T) = E[x{t)x(t + T)]
(6.14)
where E[] denotes expectation (ensemble average). In characterizing clock noise, usually the one-sided PSD is used; this is related to the autocorrelation function by
208
Chapter 6 S^{f)
=
4rR^(T)cos27fTdT ^
(6.15)
R^{T)=
^S^{f)COS27tfTdT
Eqs. (6.13)-(6.15) assume that the random phase process x(t) is widesense stationary, i.e., that the autocorrelation function depends only on the difference between the times of the two samples (or that Eq. (6.14) is independent of 0-'^ However, it will be seen shortly that many of the stochastic models used to characterize phase noise have power law PSDs, and some of these models are nonstationary. This problem is addressed by realizing that, in real systems, high- and low-frequency cutoffs exist. For example, a theoretical high-frequency limit is provided by the fact that it makes no sense to sample faster than the bit rate of the signal; a tighter limit may be imposed by the measurement system. A low-frequency cutoff is provided by the fact that measurements are made over finite time intervals. These issues are described in detail in Appendix I of [18]. As is stated in [18], results of measurements and calculations are most meaningful if they are independent of the low-frequency cutoff as this frequency approaches zero. In any case, the following points hold when using nonstationary models: 1. When making measurements, one should use statistics that converge. This was one consideration that led to the TVAR and TDEV parameters described in the next subsection, as well as to the related parameters Allan Variance and Modified Allan Variance (see [19], [20], [18], and various references given in those documents for more details on these parameters). 2. When performing analytical calculations, one must ensure that mathematical operations (e.g., summations, integrals) converge. Measurement of phase noise in clocks and oscillators has shown that the form of the power spectral density is generally a linear combination of power law terms [19]:
fi=oJ
k
(6.16) h^
h.
" f f
f
K
r
Each term is considered to represent a different noise type. The first term in Eq. (6.16) represents familiar white noise (white phase modulation, or
Synchronization of Optical Networks
209
WPM) that has a flat power spectral density. The third term in Eq. (6.16) represents white frequency modulation (WFM). Since phase is the integral of frequency, this white frequency noise results in a random walk in phase. The second and fourth terms in Eq. (6.16) represent flicker phase modulation (FPM) and flicker frequency modulation (FFM), respectively. The fifth term represents random walk frequency modulation (RWFM). Mathematical models have been developed in which the flicker noise terms may be considered as half (i.e., fractional order) integrals of white noise terms [21], [22], [23]. The fundamental physical processes that give rise to flicker noise are not nearly as well understood as those that give rise to white noise (e.g., the latter can arise from thermal motion of atoms and molecules). However, this particular lack of knowledge has not prevented the characterization of noise in clocks and oscillators. Eq. (6.16) can be generalized, if desired, to include power law dependences involving fractional exponents. In this case, P would take on any value (real number) between 0 and 4. In addition, there could, in principle, be an arbitrary number of terms. It can be shown that Gaussian noise with power spectral density of the form h/^J^ is wide-sense stationary (i.e., the autocorrelation function, as given by Eq. (6.14), depends only on r and not on 0 if 0 < P < 1 and nonstationary if J3> \ (see [24] for details).
6,2.6
Characterization of Timing Performance
This section describes several useful measures of jitter and wander performance. These measures include peak-to-peak jitter, root-mean-square (RMS) jitter, maximum time interval error (MTIE), time variance (TVAR), and square-root of time variance, or time deviation (TDEV). These measures, along with other terminology for network timing, jitter, and synchronization, are defined in [17] and described in more detail there. 6.2.6.1
Peak-to-Peak and RMS Jitter
Let CalnT] be the jitter process, i.e., the result of passing the phase error process through an appropriate jitter measurement filter (7 is the bit period). The RMS jitter, JRMS^ is defined as J,^,=^E[el(nT)]
(6.17)
where £"[•] denotes expected value, Eq. (6.17) assumes the jitter process is stationary; this assumption is valid because of the high-pass filtering. In
210
Chapter 6
practice, the jitter process is ergodic and RMS jitter may be estimated by replacing the ensemble average with a time average. Peak-to-peak jitter over a specified time interval is the difference between the maximum jitter value over that time interval and the minimum jitter value over that time interval (in defining the minimum value here, the sign of the jitter must be accounted for; the minimum value is algebraically the smallest value). In a rigorous definition, the maximum and minimum values should be defined as specified quantiles of the jitter distribution. However, in practice, the maximum and minimum values of a single sample of the jitter process are used. The time interval must be long compared with the time constant of the jitter measurement filter. For SDH/SONET and OTN line jitter, a 60-second interval is used. 6.2.6.2
Maximum Time Interval Error (MTIE)
MTIE is the peak-to-peak phase error of a timing signal for a specified observation interval, expressed as a function of observation interval. For a given timing signal of duration r^ax and observation interval S < r^ax^ the peak-to-peak phase error is computed for all subintervals of the total duration that are of length S, MTIE for observation interval S is the largest of these peak-to-peak values. MTIE is defined in this manner for 0 < 5 < r^ax- As with peak-to-peak jitter, MTIE should be defined as a specified quantile of a random variable defined by the above operation. However, in practice only a single phase history sample is generally used, and MTIE is estimated as the value of that sample (for each observation interval). MTIE may be defined rigorously as follows [17]. Define the random variable X (which is a function of the observation interval S, and may therefore also be thought of as a random process indexed by the parameter
sy X(5)=
max
max [x{t)\-
min [x{t)\\
(6.18)
Then MTIE(5) is defined as a specified percentile, p, of X(5). For a sampled data set obtained from a single measurement, MTIE can be estimated as
MTIE(nTo)= max (max x(i)lk+n
min x(/)), k
'
n= 1, 2,..., A^-1 (6.19)
Synchronization of Optical Networks
211
where To is the sampling interval and A^ is the number of samples in the measurement interval. Eq. (6.19) gives a point estimate with no information on what quantile it actually represents. To obtain an estimate of a specified quantile of X(S), multiple measurements must be made, and the following methodology may be used [17]. Let Xi, X2,...XM be a set of independent measurement samples of MTIE, for an interval of length S, for M measurement periods each of length Tmax- Assume that the samples have been put in ascending order, i.e., Xi < X2 < ... < XM- Let x^ be the jS^ quantile of the random variable X, Then a confidence interval for x^, expressed as the probability that x^ falls between the samples Xr and X, (with r<s), is given by P{X^ <x.< X,}= y
—
j3'(l-j3f-',
(6.20)
where ?{•} denotes probability. Eq. (6.20) holds under very general conditions; the only assumption needed is that the successive samples (e.g., successive measurements) are independent and identically distributed. This assumption is valid if the external environment has not changed appreciably over the timescale of the measurement. 6.2,6.3
Time Variance (TVAR) and Time Deviation (TDEV)
TVAR is a second order statistic of the phase noise random process. As indicated in Section 6.2.5.1, the phase noise processes in clocks and oscillators have power spectral densities that vary as 1//^, where a is an exponent between 0 and 4. For a > 1, such a random process is nonstationary and has first and second moments that depend on time. In general, the traditional estimator for the classical variance will not converge for such a process. However, the estimators for the TVAR and TDEV statistics do converge. In addition, the slope of TDEV (or TVAR) on a loglog scale is easily related to a; this means that one may determine a (i.e., the noise type) from a TDEV plot, TVAR is defined as the expected value of the square of the second difference of the average phase error, where the average is over an interval x referred to as the integration time [17]: TVAR(r) = -E[{A^xf], 6
(6.21)
Here, x(t) is the phase error process and the bar denotes time average. TDEV is the square root of TVAR. TVAR and TDEV are most useful when x(t) represents only the random component of phase error (in particular, they
212
Chapter 6
are useful when the random process has a power law power spectral density as described above). Note that TDEV and TVAR for a constant phase offset and for a constant frequency offset are both zero due to the second difference operation (this may be clearly seen from Eq. (6,22) below). TVAR is estimated using (where x = nXo, NXQ is the total length of the data sample, and n = \,2,,.., integer part (N/3)) [17]
(6.22)
6.2.7
Wander Network Limits and Wander Performance
As indicated in Section 6.2.5, wander is defined as phase variation whose frequency components are less than 10 Hz. The control of wander is important because there are many cases in networks where a signal passes from one timing domain to another via a slip buffer. In this situation, the timing of the signal entering the slip buffer is independent of the timing of the signal leaving the slip buffer. If the phase difference between the two signals becomes large enough, the buffer will overflow or underflow. This outcome is referred to as a slip. If the occurrence of a slip is dealt with by throwing away the current contents of the buffer, recentering, and reacquiring framing for the signal, the slip is referred to as uncontrolled. However, it is possible to delete an entire frame on buffer overflow or repeat an entire frame on buffer underflow (providing that at least one frame of data is always saved). This process is referred to as a controlled slip, and is more desirable because it avoids the need to reacquire framing (and results in overall less data loss). A prominent example of the use of the controlled slip mechanism is with the buffering of data at 64 kbit/s switches in the PSTN. While the switches are ordinarily synchronized, the synchronization is not perfect. Each incoming DSl or El has a slip buffer that can accommodate at least a single frame plus an additional 18 |is worth of data (hysteresis), i.e., the buffer may be larger than this, but it cannot be smaller. This means that if the phase difference between the data entering the buffer and leaving the buffer varies sufficiently, the buffer will overflow or underflow. If this happens, an entire DSl/El frame is deleted or repeated. The magnitude of phase difference needed to cause a single slip depends on the initial conditions and the detailed phase variation. Since slip buffers can accommodate only a maximum phase variation without a slip occurring, a long-term frequency offset between the incoming
Synchronization of Optical Networks
213
and outgoing rates will eventually result in a slip. After a slip has occurred in this situation, the buffer fill will be one frame's worth of data from one edge. Continued operation will cause the buffer fill to move towards that edge. Therefore, the time between slips is equal to the time needed for the accumulated phase difference between the incoming and outgoing signals to equal one frame, or 125 |as for DSl or El. ITU-T Recommendation G.811 specifies a long-term frequency accuracy for a Primary Reference Clock (PRO of 10'^^ [25]. This means that, when PSTN switches are synchronized, their long-term frequencies differ from nominal by no more than 1 part in 10^\ In the worst case, where the incoming and outgoing slip buffer signals are off by 10'^^ in opposite directions, a slip will occur every (125 X 10-^ s)/(2 X 10-^' s/s) = 6.25 x lOS s 72 days. If the clock timing a switch enters holdover, the slip rate will be larger, and depends on the quality of the holdover. Controlled slip rate objectives for an international digital connection, and allocations of these objectives to international, national transit, and local portions of the connection, are given in ITU-T Recommendation G.822 [26]. The discussion above indicates that slip rate is controlled by limiting the long-term frequency differences between network clocks. However, phase noise also results in phase difference that can cause slips. Therefore, the long-term wander for traffic signals is also limited. The limit is specified in ITU-T Recommendations G.823 and G,824 (for the 2048 and 1544 kbit/s PDH hierarchies, respectively) as 18 |as over 24 hours. As indicated above, this limit is the minimum required slip buffer hysteresis. This is a long-term wander limit; it is MTIE for an observation interval of 24 hours (see Section 6.2.6.2). In addition to controlling long-term MTIE (i.e., over 24 hours), shortterm wander is specified in ITU-T Recommendations G.823 and G.824 for traffic and synchronization signals (i.e., interfaces) via MTIE and TDEV masks (the former for both traffic and synchronization interfaces, and the latter only for synchronization interfaces). The synchronization wander limits are referenced in ITU-T Recommendation G.825 for SDH signals; the reason the synchronization wander limits are referenced is that all SDH signals are required to be suitable for carrying synchronization. Wander limits are not specified for OTN (i,e,, OTUk or ODUk) signals, as these signals are not required to carry synchronization. However, it is required that synchronization be carried by SDH clients of OTN, and the OTN specifications ensure that SDH clients will meet the G.825 jitter and wander network limits (and, by reference, G.823 and G.824 synchronization interface wander limits).
214
63.
Chapter 6
ROADMAP OF CURRENT ITU-T RECOMMENDATIONS ON TIMING, AND JITTER, FOR OTN, SDH, AND PDH
Table 6-1 shows the applicable ITU-T recommendations pertaining to timing, synchronization, and jitter for OTN, PDH, and SDH networks. The applicable recommendations pertaining to rates, formats, and interfaces and to other (nontiming/jitter) equipment specifications are also shown, as well as recommendations pertaining to network performance and network architecture. Table 6-1 is not a complete list of all ITU-T recommendations for the respective networks; e.g., it does not show recommendations pertaining to network management. The jitter and wander equipment specifications include the jitter and wander generation, transfer, and tolerance requirements for the various network elements. Physically, these specifications end up applying to various phase-locked loops. In the case of SDH, the requirement that the network is normally synchronized gave rise to more stringent specifications for the clock in network elements where the SDH multiplex section (MS) is terminated, compared with SDH regenerators where only the regenerator section (RS) is terminated. This more stringent clock is referred to as the SDH Equipment Clock (SEC) (for Option 2, this is equivalent to the SONET Minimum Clock (SMC)), and its requirements are in a separate ITU-T recommendation, G.813. Other SDH equipment requirements for timing and jitter, e.g., SDH regenerator and desynchronizer requirements, are in G.783, which also contains SDH equipment requirements not pertaining to timing or jitter. For OTN, the equivalent of the equipment clock is the clock that creates the timing for the ODUk. This clock is referred to as the ODUk Clock (ODC). Two such ODC's are specified: ODCa for asynchronous client mappings and ODUk multiplexing, and ODCb for bit-synchronous client mappings. Since the ODCa and ODCb requirements are not nearly as stringent as the SDH Equipment Clock (SEC) requirements, it was decided to include these in the same recommendation as the other equipment requirements pertaining to OTN timing and jitter. Two other ODC's are also specified: ODCr for 3R regenerators and ODCp for the CBR client desynchronizer. The OTN interface and equipment timing/jitter specifications are all in Recommendation G.8251 [8]. The OTN equipment specifications not pertaining to timing and jitter are in G.798 (G.798 references G.8251 for timing and jitter specifications).
Synchronization of Optical Networks
215
Table 6-1. Summary of ITU-T Recommendations for PDH, SDH, and OTN covering timing, synchronization, and jitter (Recommendations for rates and formats, equipment specifications, architecture, and performance are shown for completeness) SDH OTN Hierarchy PDH (2048 PDH (1544 kbit/s kbit/s hierarchy) hierarchy) Jitter/wander network limits
G.823
G.824
G.825
G.8251
Jitter/wander equipment requirements
G.921
(Note 1)
G.783
G.8251
Equipment clock
Not Applicable
Not Applicable
G.813
G.8251
Node clocks
G.812
G.812
G.812
Not Applicable
Primary Reference Clock
0.811
G.811
G.811
Not Applicable
Synchronization Distribution
None
None
G.781
Not Applicable
Definitions and terminology for timing/synchronization
G.810
G.810
G.810
G.810
Rates, formats, and interfaces
G.703, G.704
G.703, G.704
G.707
G.709/Y.1331
Equipment specifications
G.921
(Note 1)
G.783
G.798
Network architecture
None
None
G.803
G.872
Network (service) performance
G.826
G.826
G.828, G.826
G.8201
Note 1: No applicable Recommendation.
The specifications for node clocks (a node clock is referred to as a Synchronization Supply Unit (SSU) in Option 1 networks and a Building Integrated Timing Supply (BITS) in Option 2 networks) are in Recommendation G.812. The specifications for the Primary Reference
216
Chapter 6
Clock (PRC) are in Recommendation G.811. These recommendations specify the clocks, and therefore are not strictly limited to any network type. However, these clocks are not required for OTN, and are therefore indicated as not applicable in Table 6-1, because the OTN is not synchronized. Recommendation G.810 defines the applicable terminology for timing, synchronization, and jitter, and is applicable to all network types.
6.4.
TIMING AND JITTER REQUIREMENTS FOR SONET/SDH AND OTN
This section uses the concepts introduced in Section 6.2 to describe the timing and jitter requirements for SONET/SDH networks and the Optical Transport Network (OTN). While most of the SONET and SDH standardization occurred during roughly the 1985-1995 timeframe and the technology is now mature, OTN standardization has been relatively complete only for several years, and OTN technology is not widely deployed. When work on OTN standardization began, OTN was intended as the next generation of transport network after SONET and SDH. It is a new transport networking layer [27]. It uses Dense Wavelength Division Multiplexing (DWDM), which results in Tbit/s capacity per fiber, and allows for wavelength level switching. It provides for enhanced 0AM and networking functionality (compared with SDH) for all services. It provides for service transparency for clients. The characteristics and features of the OTN are described in more detail in [27]. The OTN transport includes multiple optical channels, with nominally one per wavelength. Each optical channel transports a digital frame structure, which has payload and overhead areas. The paths transported by the optical channels can have rates of 2.5, 10, or 40 Gbit/s; i.e., the granularity of the paths is relatively large. The clients (payloads) are mapped into this frame either asynchronously or bit-synchronously. When SDH and SONET were developed, several considerations gave rise to timing/synchronization and jitter requirements. First, it was necessary to ensure that PDH clients transported over SDH or SONET networks would meet applicable timing and jitter requirements at network interfaces. This was needed to ensure interoperability with existing PDH networks. This also resulted in careful consideration being given to defining the client mappings. Second, the PDH networks and certain other services needed to be synchronized. Prior to the introduction of SDH and SONET, synchronization was transported on the PDH facilities (DSl in North America and El in Europe). With the introduction of SDH and SONET, it was not possible to transport synchronization via PDH signals that were
Synchronization of Optical Networks
217
SONET or SDH clients, because the VT1.5 (VC-11) or VC-12 pointer adjustments would result in the wander requirements being exceeded. Instead, it was necessary to transport synchronization directly via the SDH or SONET layer network. Third, jitter requirements for the SDH/SONET layer network were needed to ensure acceptable bit error performance for the SDH/SONET layers. The above gave rise to jitter, wander, and synchronization requirements for SDH and SONET. The requirements contain two key ingredients. First, the SDH and SONET networks are required to be synchronized. This means that the timing signal provided to each SONET or SDH network element is normally traceable to a Primary Reference Source (PRS) or Primary Reference Clock (PRC), respectively (see Section 6.5). Such a signal has a long-term frequency accuracy of 1 part in 10^\ relative to Coordinated Universal Time (UTC) frequency. The signal also normally meets several other requirements, e.g., on short-term stability. Second, the PDH network digital hierarchies, network synchronization architectures, and clock hierarchies in use in North America and Europe are different. This gave rise to a number of the jitter, wander, and synchronization requirements (both network interface requirements and equipment requirements) for SDH being different for networks deployed in North America and in Europe. The ITU-T recommendations covering SDH jitter and clock requirements refer to Option 1 and Option 2 networks. The former are SDH networks optimized for transporting PDH clients of the 2048 kbit/s hierarchy; the latter are SDH networks optimized for transporting PDH clients of the 1544 kbit/s hierarchy. The Option 2 SDH networks are, in effect, SONET networks^^ The specification of two options for SDH results in a certain lack of transparency. There is no guarantee that a PDH signal of the 1544 kbit/s hierarchy will meet its respective jitter and wander requirements if it is transported over an SDH network that meets Option 1 jitter and synchronization requirements. Likewise, there is no guarantee that a PDH signal of the 2048 kbit/s hierarchy will meet its respective jitter and wander requirements if it is transported over an SDH network that meets Option 2 jitter and synchronization requirements. In addition, the Option 1 and Option 2 synchronization requirements are different. In developing the OTN, one goal was to avoid the above two ingredients. First, it was desired that the OTN be transparent for the respective services that are transported, e.g., SONET/SDH, Asynchronous Transfer Mode (ATM), Internet Protocol (IP), 1 and 10 Gbit/s Ethernet (GbE), etc. This means there should be a single option for OTN, with a single set of requirements. There should not be separate OTN options for Option 1 and 2 SDH clients. Second, it was desired to avoid the need to synchronize the OTN layer. This would avoid the need for the OTN layer to concern itself
218
Chapter 6
with transport and 0AM for network synchronization. One result is that there is no need to specify synchronization status messages (SSMs) and their behavior at the OTN layer. It was recognized, however, that provision must still be made for the transport of synchronization, and it was decided that synchronization should be transported via the SDH or SONET clients. Therefore, it is required that a SONET or SDH client transported over multiple OTNs meet the respective interface jitter and wander requirements in ITU-T Recommendation G,825 on egress from the final OTN. Such signals would then be suitable for carrying both traffic and synchronization. The above gives rise to timing and jitter requirements for the OTN that are less complicated than the corresponding requirements for SDH. It also allows the specification of a single, global set of timing and jitter standards for OTN, rather than two options as was the case for SDH. The following subsections provide a brief overview of the SONET/SDH and OTN timing and jitter requirements. The subsections are roughly organized such that requirements that must be chosen consistently are discussed together (e.g., the requirements pertaining to server layer jitter, client jitter and wander, and clock requirements are discussed in separate subsections). The specific ITUT Recommendations that contain the SDH and OTN timing and jitter requirements are listed in Table 6-1 above.
6.4.1
SEC and ODC Frequency Accuracy, Clock Modes, Pull-in and PuU-out/Hold-in Ranges
The free-run accuracy, pull-in range, and pull-out range for the Option 1 SEC are specified as ± 4.6 ppm. The free-run accuracy, pull-in range, and hold-in range for the Option 2 SEC are specified as ± 20 ppm. Note that the Option 1 SEC specifies pull-out range to constrain the pull-out process, while the Option 2 SEC uses hold-in range. Both terms (as well as pull-in range) are defined in ITU-T Recommendation G.810 [17], and also are discussed in [28]. The pull-out range is the largest offset between the SEC reference and a specified nominal frequency, within which the SEC stays locked regardless of the rate of change of reference frequency. Essentially, the SEC will remain locked if the reference undergoes a frequency step provided that the magnitude of the step does not exceed the pull-out range. The pull-out range is a dynamic limit. The hold-in range is the largest offset between the SEC reference and a specified nominal frequency, within which the SEC stays locked if the reference frequency changes infinitely slowly. The hold-in range is a quasi-static limit (the term quasi-static means that the limit is applied to the case where the change in frequency occurs infinitely slowly). The Option 1 and Option 2 SECs support free-run, locked, and holdover modes. The free-run accuracy for ODCa, ODCb, ODCr, and
Synchronization of Optical Networks
219
ODCp (see Section 6.3 for a brief description of the ODUk clock (ODC)) is specified as ± 20 ppm. All four clocks support free-run mode. ODCb, ODCr, and ODCp support locked mode and have pull-in and pull-out ranges of ± 20 ppm. ODCa is always free-running and, therefore, does not support locked mode or have pull-in or pull-out ranges defined. None of the ODCs support holdover mode because the OTN layer is not synchronized.
6.4.2
STM-N and OTUk Jitter Network Limit and Tolerance, STM-N Regenerator and ODCr Jitter Generation and Transfer, and STM-N and OTUk Jitter Accumulation
As described in Section 6.2.2, jitter network limits must be specified such that the jitter tolerance of equipment (e.g., clock recovery and dejitterizer PLLs) will not be exceeded. Jitter generation and jitter transfer of equipment must be specified such that jitter accumulation over a reference chain of equipment (e.g., some number of 3R regenerators) will not exceed the network limits. Historically, the jitter specifications for SDH were developed well before those for OTN, and the specifications for STM-1, -4, and -16 were developed before those for STM-64. For STM-1, -4, and -16, both highband and wide-band network limits and tolerance were specified as 0.15 Unit Intervals peak-to-peak (UIpp) and 1.5 UIpp, respectively. The corner frequencies for the high-pass measurement filters, which are the same as the jitter tolerance corner frequencies, are different for the three rates. The specifications for SDH Options 1 and 2 are the same, and are contained in ITU-T Recommendation G.825 [9].^^ The jitter generation and transfer specifications for STM-1, -4, and -16 were developed around the same time as the jitter tolerance and network limit specifications. The jitter transfer specifications are the same for Options 1 and 2; however, the jitter generation specifications are not the same. For Option 1, both high-band and wide-band jitter generation limits are specified, of 0.1 UIpp and 0.3 UIpp, respectively. The measurement filters correspond to the measurement filters for the network limits. For Option 2, a jitter generation limit with a single high-pass measurement filter of 12 kHz is specified. The jitter generation limit is 0.1 UIpp and 0.01 Ulrms, i.e., both limits must be met simultaneously, but with the single measurement filter defined. The measurement filter corner frequency is between the network limit high-band and wide-band corner frequencies for STM-1, -4, and -16 (see ITU-T Recommendation G.783 [10]). Jitter accumulation studies over a chain of SONET/SDH regenerators are not documented in the SONET or SDH specifications for STM-1, -4, and -16
220
Chapter 6
(studies are documented for STM-64; these are discussed shortly)/^ However, jitter accumulation studies based on the Option 1 SDH regenerator specifications were performed during the development of the OTN specifications, and are documented in Appendix IV of ITU-T Recommendation G.8251 [8]. These studies use a frequency domain model for cascaded second-order, linear PLLs, each of which contains a phase detector, proportional plus integral loop filter, and voltage-controlled oscillator (VCO) (see [6], [7], and [28] for more details on PLL models). Models were developed for both systematic and random jitter accumulation, based on previous work in [2], [12], and [13]. However, only the random jitter accumulation results are used, because it is assumed that a narrow-band dejitterizer PLL is always present. Systematic jitter accumulation occurs due to pattern-dependent jitter in the clock recovery PLLs; the presence of a dejitterizer PLL between two successive clock recovery PLLs decorrelates the pattern-dependent jitter. In addition, the jitter accumulation models do not model the clock recovery and dejitterizer PLLs individually; rather, a regenerator is assumed to consist of a single PLL that met the jitter generation and transfer requirements. The PLL parameters are chosen to achieve the desired bandwidth and gain peaking. Using STM-16 as an example, these are 2 MHz and 0.1 dB, respectively. N,
+
+
4-
^^
1
^v
1
H3"0]
Y •
Figure 6-10. Schematic of linear PLL model, used to model regenerator (adapted from [8]). The noise sources represented by N] and N2 are WPM; the noise source represented by Ni, is a combination of WPM and WFM
A schematic of the linear PLL model used to model a regenerator is shown in Figure 6-10 (adapted from Figure IV.2-1 of [8]). This is a standard PLL linear model (see [6], [7], and [28]). The phase detector is modeled by the difference between the input plus noise signal U •{• N] and the feedback signal Y (i.e., the second summation from the left) followed by the gain Ka. The loop filter is assumed to be proportional plus integral (i.e., an active filter), and is modeled by the l-^b/s block. The VCO frequency is proportional to the filtered phase error; it is modeled by the Ko/s block, with the l/s integrating the frequency to produce phase. Additive noise
Synchronization of Optical Networks
221
generation is modeled at the PLL input (the input N\ just before the feedback path), at the loop filter (the input N2 between the forward gain and loop filter), and in the VCO (the input A^3 just after the VCO, before the PLL output). The first two noise sources enter into the model in the same way and can be combined; their combined effect is modeled as WPM. The effect of the PLL is to low-pass filter this noise. The VCO noise is assumed to have PSD that depends on 1// at low frequencies (WFM), and is constant at high frequencies (WPM). The Quality Factor, or g, of the VCO determines the relative magnitude of each component (or, equivalently, the breakpoint where the two regions join). Specifically, the VCO noise has a WFM characteristic for frequencies less than//, =/o/2g, and a WPM characteristic for frequencies greater than//,, where/o is the line rate (i.e., the VCO center frequency). The details of this noise model are given in [29]. The effect of the PLL is to high-pass filter the WPM and WFM generated in the VCO. Full details of the noise accumulation model are given in Appendix IV of [8]. The simulation results are shown in Figures 6-11 and 6-12, in the form of relative jitter accumulation (jitter after A^ regenerators divided by jitter for one regenerator). Figure 6-11 shows results for low-pass filtered WPM (i.e., WPM in noise sources A^i and A^2) and high-pass filtered WPM (i.e., WPM, but no WFM, in noise source N3', this corresponds to a VCO with infinite loaded Q). The frequency domain model actually produces RMS jitter; it is assumed that the ratio of RMS jitter after N regenerators to RMS jitter for one regenerator is the same as the corresponding ratios for peak-to-peak jitter; this means that peak-to-peak jitter is a constant factor times RMS jitter. The results in these figures, expressed in this dimensionless form, also apply if the rates and bandwidths are all multiplied by the same scale factor. The values of loaded Q shown in Figure 6-12 cover a range expected in practice. The ratio of high-band jitter network limit to jitter generation is 0.15/0.1 = 1.5. Figure 6-11 shows that if all the jitter generation is in the form of low-pass filtered WPM that just meets the high-band jitter generation specification, the network limit is exceeded after approximately 10 regenerators. Figure 6-12 shows that if all the jitter generation is in the form of high-pass filtered VCO noise that just meets the high-band jitter generation specification, the network limit is exceeded after approximately 10 and 15 regenerators for Q = 30 and 100, respectively (the jitter accumulation is much slower for Q = 535, and hardly accumulates at all for high-pass filtered WPM (Figure 6-11)). The ratio of wide-band jitter network limit to jitter generation is 1.5/0.3 = 5. This level of jitter accumulation appears to be reached after approximately 100 regenerators in the worst cases in Figures 6-11 and 6-12.
222
Chapter 6
Actually, the situation is much better than this, as this conclusion assumes that a regenerator just meets the wide-band jitter generator specification. However, the simulations also showed that a regenerator that just meets the high-band jitter generation specification is within the wide-band jitter generation specification by between a factor of 2 and 3, i.e., the high-band jitter generation specification is the more stringent of the two. The results indicate that, under the assumptions of the model, the STM16 jitter network limit is exceeded if more than approximately 10 regenerators are cascaded. However, actual networks likely perform much better than this, mainly because SDH regenerators tend to have bandwidths that are much narrower than 2 MHz. As will be seen shortly in the results for OTN, a modest reduction in the bandwidth results in a large reduction in the jitter accumulation.
Wide-band jitter, low-pass WPM High-band jitter, low-pass WPM Wide-band jitter, high-pass WPM High-band jitter, high-pass WPM
12
\
__ .—'
r^""
.__
• '
50
100
150
200
Number of 3R Regenerators
Figure 6-11. Simulation results for jitter accumulation over STM-16 regenerators. Assumptions: regenerator bandwidths meet SDH Option 1 (G.783) requirements, random jitter accumulation, no WFM in high-pass (VCO) noise cases
Synchronization of Optical Networks
223
Wide-band jitter, VCO noise with Q = 30 High-band jitter, VCO noise with Q = 30 Wide-band jitter, VCO noise with Q = 100 High-band jitter, VCO noise with Q = 100 Wide-band jitter, VCO noise with Q = 535 High-band jitter, VCO noise with Q = 535 12
1
1
1
1
1
1
1
1
c
10
/ / / /
CD O)
-
'/
-
:
. . . . . < • - • • • ' :
t0
. . ^
4
y^-^
^ ^ __— -^^-^^ =5
2
^^=^=^=9=r^9=rtr-=^ p_.:r$:=:"^- v--^
-^
——:
^^""^"""^'^ 50
100
150
200
Number of 3R Regenerators
Figure 6-12. Simulation results for jitter accumulation over STM-I6 regenerators. Assumptions: regenerator bandwidths meet SDH Option 1 (G.783) requirements, random jitter accumulation, high-pass (VCO) noise with WFM and WPM and indicated Q-factor
Chapter 6
224
Wide-band jitter, low-pass WPM High-band jitter, low-pass WPM Wide-band jitter, high-pass WPM High-band jitter, high-pass WPM
— — — I I
\c
c
0 D) 0
1
1
1
I
I
I
1
I
1
1
1
1
I
1
-
/••-
10 /
DC
,•
-T— L
0
8
03 k-
-
(1) ti
:;;;o c0
6
D) 0
"
•
.''
-
-
cr -z. 4 Vi^
0
-
^ L-
0
)
2
-
/' n
t , 1
1 , 1
.1
50
1
1
1
. 1
100
1 . -.1
1..
150
i
i
1
1
200
Number of 3R Regenerators
Figure 6-13. Simulation results for jitter accumulation over OTUl and 0TU2 3R regenerators. Assumptions: 3R regenerator bandwidths meet OTN (G.8251) requirements, random jitter accumulation, no WFM in high-pass (VCO) noise cases
Synchronization of Optical Networks
225
___ Wide-band jitter, VCO noise witli Q == 30 o
—
—^. —^_. o-.'.. 0
Higli-band jitter, VCO noise witii Q =:30 Wide-band jitter, VCO noise with Q := 100 Higii-band jitter, VCO noise with Q =: 100 Wide-band jitter, VCO noise with Q = 535 High-band jitter, VCO noise with Q =:535
12
0) O) 0)
10
0 4=: CO 0
c
0 D) 0
4:i CO
s
50
100
150
200
Number of 3R Regenerators
Figure 6-14. Simulation results for jitter accumulation over OTUl and OTU23R regenerators. Assumptions: 3R regenerator bandwidths meet OTN (G.8251) requirements, random jitter accumulation, high-pass (VCO) noise with WFM and WPM and indicated Q-factor
In developing the OTN specifications, the network limits and jitter tolerance for OTUl and 0TU2 were chosen to be the same as for STM-16 and STM-64, respectively. In addition, the Option 1 SDH jitter transfer and generation specifications were used as a starting point. Since the corresponding OTN and SDH rates are very similar, the jitter analyses for the two will be approximately the same if all the bandwidths are the same.'^ The specifications for STM-64 were not fully developed when OTN was being developed. However, the above analysis would hold for STM-64 if all
226
Chapter 6
the bandwidths were scaled by the same factor as the rate (i.e., by a factor of 4), and imply similar results for OTU2 for the same bandwidths. It was decided that OTN jitter accumulation should be acceptable over at least 50 3R regenerators. This Hypothetical Reference Model (HRM) is documented in Appendix II of G.8251. Figure 6-11 shows that, for low-pass filtered WPM, the high-band jitter grows by a factor of approximately 1.8 after 50 3R regenerators. Figure 6-12 shows that, for loaded Q of 30 and 100, the high-band jitter grows by a factor of approximately 1.6 after 50 3R regenerators. Since the ratio of the high-band jitter network limit to highband jitter generation is 0.15/0.1 = 1.5, the network limit will not be met. The discussion above for Option 1 SDH indicated that the growth factor of 1.5 would be reached after approximately 10 to 15 regenerators (with the exact number depending on the specific noise component and the loaded Q). The results in Figures 6-11 and 6-12 show that OTN high-band jitter network limit cannot be met with the above assumptions. The assumptions can be made consistent by (1) relaxing the network limit, (2) tightening the jitter generation, (3) using an HRM with fewer 3R regenerators, or (4) tightening the transfer bandwidth. Approaches (1) and (2) are more difficult to achieve technically than (4), while approach (3) would be too limiting in the number of 3Rs allowed. It was decided, therefore, to tighten the transfer bandwidth requirement, and was found that a tightening by a factor of 8, to 250 kHz and 1 MHz for OTUl and 0TU2, respectively, would provide for acceptable high-band jitter accumulation over 50 3R regenerators. Simulation results are given in Figures 6-13 and 6-14. The results show that the high-band jitter accumulation factor for 50 regenerators is very close to 1.0 for low-pass filtered WPM and for VCO noise with Q as small as 30. For OTU3, the requirements cannot be taken directly from STM-256 because the latter are not yet specified in G.825. The most straightforward approach would be to scale the measurement filters and transfer bandwidths by a factor of 4 and leave the jitter limits the same as for OTUl and 0TU2. This was done for the high-band jitter network limit and generation. However, for the wide-band jitter, it was decided that, from an implementation point of view, it would be preferable to keep the high-pass measurement filter the same as for OTU2 (i.e., 20 kHz) and instead increase the network limit to 6.0 UIpp and the generation to 1.2 UIpp. The 0TU3 noise transfer bandwidth is 4 MHz; additional simulations were done to verify that the wide-band jitter accumulation would be acceptable. The jitter transfer bandwidths of 250 kHz, 1 MHz, and 4 MHz for OTUl, 2, and 3, respectively, are narrower than the high-band jitter measurement filter corner frequencies (1 MHz, 4 MHz, and 16 MHz, respectively). The relation between jitter network limit and jitter tolerance generally means that the clock recovery circuit must track jitter of frequency less than the high-
Synchronization of Optical Networks
227
band jitter measurement filter corner frequency, and therefore cannot have a bandwidth less than this. This implies that the OTUl, 2, and 3 regenerators will, in general, use two PLLs, i.e., a wide-band PLL in the clock recovery followed by a narrower-band dejitterizer PLL. Jitter specifications for STM-64/OC-192 were developed subsequent to developing the OTUk jitter specifications. It was decided to copy the 0TU2 specifications for network limit, jitter tolerance, jitter transfer, and jitter generation. This decision also means that, for STM-64, the jitter generation specifications for Option 1 and Option 2 are the same. The OC-192 jitter specifications for SONET in Tl. 105.03-2003 [11] are consistent with those for STM-64 in G.783 [10]. Note that jitter specifications for STM-256 are not yet specified in G.783, and jitter specifications for OC-768 are not yet specified in T 1.105.03. Jitter tolerance requirements for OTUk were chosen to be consistent with the network limits. In contrast with SDH, jitter tolerance for OTUl, 2, and 3 are not specified below 500 Hz, 2 kHz, and 8 kHz, respectively. These frequencies correspond to a jitter level of 15 UIpp. Higher levels of lowfrequency jitter are not expected to occur in the network; extending the masks to higher levels at lower frequencies would increase testing complexity while providing little value.
6A3
Jitter and Wander Accumulation for PDH Clients of SDH Networks and SDH Clients of OTN
As indicated at the beginning of Section 6,4, a server layer must ensure that transported clients meet applicable jitter and wander requirements. Server layer requirements are determined by starting with client layer requirements and using a top-down approach that is used in many areas of engineering. The server layer requirements may be determined by first considering an appropriate HRM for each client application. The HRM is a description of the types of networks and equipment that the client signal traverses endto-end. The HRM will likely include existing network types, i.e., types of transport in use prior to the introduction of the server layer technology; this will ensure that the introduction of the server layer will allow interworking with existing networks. There may be more than one HRM for a single client, e.g., if there are multiple applications involving that client. After deciding on one or more HRMs, the client network interface jitter and wander requirements are budgeted to the respective network components or jitter/wander producing effects within the HRM. Concurrent with the budgeting, potential requirements are set for the server layer equipment. Analyses are then performed using the equipment requirements to determine
228
Chapter 6
if the jitter and wander budgets can be met. Usually the analyses are performed using simulation; however, they may be analytical when closedform solutions are available. The process is usually iterative; initial analyses often show that some jitter or wander components exceed their budget allocations. This issue is addressed by changing the budget allocations and/or changing the equipment requirements. HRMs are typically not part of the network requirements, i.e., there is no requirement that a network operator design a network in accordance with an HRM. Rather, the HRM is a documentation of the assumptions used in relating the client layer network-interface and server layer equipment requirements. HRMs are usually documented in informative annexes or informative appendices of Recommendations or standards. In addition to HRMs for client layer jitter/wander accumulation, one may consider HRMs for server layer jitter/wander accumulation and for accumulation of phase error, jitter, or wander in synchronization reference chains (i.e., clock chains). For example, it was indicated in Section 6.4.2 that the OTN 3R regenerator jitter generation and transfer specifications were set such that jitter accumulation would be acceptable over an HRM consisting of a chain of 50 regenerators. As a second example, an HRM for SDH network synchronization is given in ITU-T Recommendation G.803 [30]. That HRM allows 1 PRC and, for Option 1 networks, up to 10 SSUs and 60 SECs, with no more than 20 SECs between any pair of SSUs.'^ 6.4.3.1
PDH Clients of SONET/SDH Networks
HRMs and associated jitter and wander budgets for the transport of DSl and DS3 signals over SONET (Option 2 SDH) networks are given in Tl.105.03 [11]. Four HRMs are given (jitter and wander, for each of DSl and DS3). The two wander HRMs are also given in ITU-T Recommendation G.824 [31]. An HRM for the transport of an El signal over an Option 1 SDH network is given in Recommendation G.823 [32]. As an example, the SONET HRMs were developed based on considerations for North America when SONET was developed (i.e., in the early 1990s). Specifically, they assume that a DSl or DS3 path can traverse up to two Inter-Exchange Carrier (lEC) networks, two Local Exchange Carrier (LEC) networks, and two customer (i.e., private) networks. Each lEC network is assumed to consist of eight SONET islands, each LEC network is assumed to consist of six SONET islands, and each customer network is assumed to consist of two SONET islands. Each SONET island is assumed to consist of a mapper node (where the DSl or DS3 client is mapped into the VT1.5 or STS-1, respectively), 10 pointer processor nodes, and a demapping desynchronizer at the 10^ node. The pointer processors are
Synchronization of Optical Networks
229
STS-1 or VT1.5 for DS3 and DSl clients, respectively. The SONET islands are referred to as islands because the client transport is over SONET within an island but not between islands. There is no regeneration of the client between the islands; a demapped client is mapped directly into the next island with whatever jitter and wander has accumulated up to that point. The DSl and DS3 jitter accumulation HRMs, and DS3 wander accumulation HRM, consist of all 32 SONET islands, i.e., a DSl or DS3 is assumed to traverse two LEG networks, two lEC networks, and two customer networks between client terminal equipment. The DSl wander accumulation HRM consists of 8 island; this is because it is assumed that the applications where DSl wander is relevant are those where the DSl transport is between two PSTN (Public Switched Telephone Network) switches, and that such switches will always be within one carrier network. Therefore, the worst-case DSl HRM is the lEC HRM, which consists of eight islands. The DSl and DS3 wide-band jitter network limits are both 5 UIpp, with a 10 Hz high-pass jitter measurement filter [33]. The DSl and DS3 high-band jitter network limits are both 0.1 UIpp, with an 8 kHz and 30 kHz high-pass jitter measurement filters, respectively [33]. Since the wide-band limit exceeds the high-band limit by a factor of 50 but the high-band measurement filter bandwidth exceeds the wide-band bandwidth by factors of 800 and 3000, respectively, the wide-band limit is the more limiting of the two.'^ The DSl and DS3 jitter budgets include components for mapping into SONET islands and PDH mapping/multiplexing, positive pointer adjustments for synchronized islands, negative pointer adjustments for synchronized islands, and periodic pointer adjustments (both continuous and gapped periodic sequences are allowed; see [10] and [11]) due to loss of synchronization in one island. The loss of synchronization component is included because it was decided that jitter accumulation should be acceptable if one island is in holdover for up to 24 hours. The mapping/multiplexing components are controlled by limiting the desynchronizer bandwidth. The component due to periodic pointer adjustments is controlled by limiting the desynchronizer bandwidth and range of SONET island frequency offsets, due to loss of synchronization, for which client payload integrity is guaranteed. This range is ±4.6 ppm, which is the maximum frequency offset of a SONET Minimum Clock (SMC) that has been in holdover for 24 hours [34]. The components due to positive and negative pointer adjustments for synchronized islands are controlled by limiting the desynchronizer bandwidth and specifying the synchronization noise performance at the SONET NE synchronization interface. This latter limit is the synchronization network limit, and is specified by MTIE and TDEV masks (see below).
230
Chapter 6
The DSl and DS3 wander network limits are both 18 [is peak-to-peak over 24 hours (see [35] and [36]). The DS3 wander budget includes components for (PSTN) switch synchronization, mapping into SONET islands and PDH mapping/multiplexing, temperature effects on the fiber, periodic pointer adjustments due to loss of synchronization in one island, clock transients in the synchronization distribution network, and pointer processor buffer fill variations and pointer adjustments due to random synchronization phase noise (i.e., when synchronized). The DSl wander budget does not include the periodic pointer adjustment and clock transient components; the other components are analogous to those in the DS3 budget, though the values are different. The reason the DSl budget does not include the periodic pointer adjustment and clock transient components is that these are sufficiently large that the network limit could not be met if they were included (for example, a gapped periodic VT1.5 pointer adjustment sequence can have wander exceeding 10 [is). Instead, these are not considered part of normal operating conditions. Unlike the case of jitter, the desynchronizer bandwidth has negligible impact on the 24-hour wander. This is because the main source of wander is NE synchronization noise and resulting pointer adjustments and pointer processor buffer fill variations. The buffer fill variations are typically slow compared with the desynchronizer time constant, and the mean time between pointers is typically much greater than the desynchronizer time constant. The primary means of controlling wander in SONET/SDH networks was to specify a sufficiently good network synchronization interface limit (via MTIE and TDEV masks) that there would be sufficiently small pointer processor buffer fill variation and sufficiently few pointer adjustments. The SONET network synchronization interface MTIE and TDEV limits (see [37], [34], [38] and [31]) were developed after extensive DSl wander accumulation simulations were performed over several years. MTIE masks for DSl and OC-N reference interfaces (e.g., interfaces to Building Integrated Timing Supply, or BITS, clocks and to SONET NEs, respectively) were developed during the development of the SONET DS3 jitter specifications. In addition, a DSl Reference TDEV mask was developed at this time. The MTIE and TDEV masks are specified in ANSI Tl.lOl [37]. Along with these masks, SONET DS3 desynchronizer requirements were developed that enabled the DS3 jitter network limit to be met. Subsequently, it was found that level of phase noise allowed by the DSl reference TDEV mask was too high to allow the DSl and DS3 wander network limits (18 |is peak-to-peak over 24 hours) to be met. As a result, a new SONET synchronization interface TDEV mask was developed. This mask is one order of magnitude below (i.e., better than) the DSl reference TDEV mask, and is specified in ANSI Tl. 105.09 [34] (and also documented
Synchronization of Optical Networks
231
in ITU-T Recommendation G.813 [38]). This mask is not sufficient to allow the 18 |j.s peak-to-peak wander over 24 hours to be met for the DSl HRM. However, additional analyses showed that this mask does allow the controlled slip requirements of ITU-T Recommendation G.822 [26] to be met. It was decided that this outcome would provide for acceptable performance because (1) the underlying purpose of the DSl wander requirement is to ensure that the G.822 slip requirements are met, and (2) not all networks are expected to contain a large number of VT1.5 (VC-11) islands. The slip performance analyses are described in Appendix II/G.813 [38], As explained above, requirements are placed on the SONET DSl and DS3 desynchronizers to ensure that the respective jitter and short-term wander network limits are met. These requirements are given in the form of pointer test sequences and mapper/demapper tests. The tests are specified in ANSI Tl.105.03 [11] and ITU-T Recommendation G.783 [10]. The pointer test sequences represent the various network conditions that can arise (and the network conditions are represented in the jitter and wander budgets), e.g., single (random) pointer adjustments, randomly occurring pointer bursts, pointer bursts due to phase transients, and continuous and gapped periodic pointer sequences (with and without an added and a canceled pointer adjustment due to a single pointer adjustment caused by random phase noise). The various tests constrain peak-to-peak jitter and MTIE (the latter in the form of MTIE masks) when the respective test sequences are applied. Tests are also specified to constrain mapping peak-to-peak jitter and MTIE. 6.4.3.2
SDH Clients of OTN
The OTN physical layer is not used to transport synchronization; however, synchronization is transported via the SDH clients. Therefore, the effect of the OTN on SDH client jitter and wander accumulation for an appropriate synchronization reference chain must be considered. Appendix II of G.8251 describes the SDH client synchronization reference chain. The HRM was developed by first considering the pure SDH synchronization reference chain in ITU-T recommendation G.803. As indicated at the beginning of Section 6.4.3, that HRM allows 1 PRC and, for Option 1 networks, up to 10 SSUs and 60 SECs, with no more than 20 SECs between any pair of SSUs.^^ For OTN, it was considered that the SECs between successive SSUs would be replaced by OTN "islands", i.e., the STM-N would be mapped into an OPUk at one SSU location and demapped at the other. There are still assumed to be 10 SSUs, and 20 SECs at the end of the chain are still allowed.
232
Chapter 6
The OTN "island" between successive SSUs has an STM-N client mapping/demapping. It is also allowed to have up to nine other mappings/demappings, so as to account for ODUk multiplexing. This means that the total number of mappings/demappings in the reference chain is 100. The model was then generalized such that the mappings would not have to be evenly divided among the nine SSU pairs, i.e., some of the islands could be large and some small. Finally, for the case where there is no ODUk multiplexing, it was decided to still consider the effect of up to 100 mappings/demappings; for this case, these would all be STM-N to OPUk. This synchronization reference chain was then used to determine (1) the STM-N to OPUk asynchronous mapper buffer hysteresis (the mapping is +1/0/-1), and (2) the ODCp transfer bandwidth, gain peaking, and roll-off. The first requirement is chosen such that, in the worst case, the long-term (24-hour) wander is acceptable for SDH. The second requirement is chosen such that the peak-to-peak jitter and short-term wander are acceptable for SDH for the case of asynchronous mapping. The STM-N to OPUk asynchronous mapper buffer hysteresis is limited to 2 bytes for STM-16, 8 bytes for STM-64, and 32 bytes for STM-256. This is approximately equal to 6.4 ns per mapper. Since the mapping is +1/0/-1, the maximum long-term wander that can possibly occur is equal to this hysteresis. Therefore, for 100 mappings/demappings, the maximum possible long-term wander is 640 ns peak-to-peak. This is sufficiently below the long-term wander limits at the end of a synchronization reference chain of 5 |as peak-to-peak for Option 1 [32] and 1.86 |a,s peak-to-peak for Option 2. Note that the maximum possible long-term wander of 640 ns will not necessarily be reached; whether it is reached depends on the actual client and ODCa frequency offsets from nominal and the actual wander generation in these clocks. However, by taking the above approach of limiting the buffer hysteresis, the task of network engineering becomes trivial. The long-term wander requirement is met as long as the number of mapper (multiplex)/demapper (demultiplex) pairs does not exceed 100. In addition, with this approach it was possible to avoid the need to do long-term wander accumulation simulations (which would be extremely time consuming). The ODCp maximum bandwidth and gain peaking were chosen to be 300 Hz and 0.1 dB, respectively, and the roll-off was chosen to be 20 dB/decade. With these assumptions, simulations of wide-band and high-band jitter accumulation and of short-term wander accumulation were performed for two cases. In the first case, the STM-N client and ODCa clocks were assumed to be free-running, with actual frequency chosen randomly (and separately for each clock) within the frequency tolerance requirements of ± 20 ppm. In the second case, the STM-N and ODCa clocks were assumed to be very close to but not exactly equal to nominal frequency, but sufficiently
Synchronization of Optical Networks
233
close that the resulting rate of stuffs would be less than the desynchronizer bandwidth (specifically, the offsets were within ± 0.05 ppm, which corresponds to a maximum stuff rate of 31 Hz for STM-16 mapped into OPUl; this is approximately 0.1 times the desynchronizer bandwidth of 300 Hz). This latter case was considered to be the worst case, because here the stuffs are filtered individually by the desynchronizer. In fact, for wide-band jitter a rough estimate of the worst case, which occurs for STM-16 mapped into OPUl, is easily obtained by considering an 8 UI step filtered first by a 300 Hz low-pass filter, followed by a 5 kHz high-pass filter (the desynchronizer and wide-band jitter measurement bandwidths, respectively). The result is approximately 0.4 UI peak; the peak-to-peak is therefore twice this, or 0.8 UI. This is well within the 1.5 UIpp jitter limit. This conclusion was confirmed via time domain simulations of multiple OTN islands. Similarly, the high-band jitter is well within the 0.15 UIpp limit, because the jitter limit is 0.1 times as much while the high-band measurement filter corner frequency is 200 times as much (1000 kHz/5 kHz). The short-term wander requirements at a synchronization interface are stated in terms of TDEV masks (see Section 6.2.5 for a brief discussion of TDEV). The masks are given in Recommendation G,813 [38]; TDEV for Option 2 is limited to 10 ns for integration times between 0.05 s and 10 s, and to 12 ns for integration times between 0.1 s and 7 s (both TDEV masks then increase for longer integration times). Simulations were performed (see Appendix VIII of [8]) for STM-16 transported over 100 OTN islands, with the same two sets of assumptions on the clock offsets as for the jitter analyses. As with the jitter analyses, the case with small but non-zero clock offsets was the worse case. TDEV for this case was found to reach a maximum of approximately 2.5 ns for an integration time of approximately 0.5 s, and then decrease. Therefore, the network interface TDEV requirement is met. Similar analyses were performed for reference models that include ODUk multiplexing. These led to the same ODUp maximum bandwidth and gain peaking requirements, namely, 300 Hz and 0.1 dB, respectively. In addition, the HRMs led to maximum mapper buffer hysteresis of 2 bytes for ODUl multiplexed into 0DU2 or 0DU3, and 8 bytes for 0DU2 multiplexed intoODU3[39].
6,5.
RELIABLE DISTRIBUTION OF SYNCHRONIZATION
The previous sections were devoted to the description of slow and fast phase variations, i.e. phase noise, of the synchronization signals, and their
234
Chapter 6
effect on the OTN, SDH/SONET, and PDH transport planes. In this section, the focus is shifted towards the subject of reHable synchronization distribution. Even if synchronization signals have perfect quality, and therefore zero phase noise, the connections over which they are transported may fail, and the synchronization distribution will then be disrupted. Several measures are available to the synchronization network engineer to design reliable synchronization distribution over transport connections that are inherently not entirely reliable.
6.5.1
The Need for Synchronization
For network elements of the PSTN switching network, such as digital switches, remote switching modules, PDH cross-connect systems, primary multiplexers, and ISDN equipment, it is the objective to operate synchronously as much as possible. Any asynchronism in this layer of the telecommunications network results in a phenomenon called {controlled) octet slip. Octet slip means that at the level of 64 kbit/s bearer channels, octets are deleted or repeated in order to align an input signal with the local clock of the network element in question. Octet slips are considered an impairment to the service that runs over the bearer channel, and their occurrence needs to be minimized. ITU-T Recommendation G.822 [26] provides information on the number of octet slips that are considered acceptable. The effect of an octet slip on the service it carries depends on the nature of the service (see Table 6-2). Table 6-2. The effect of an octet deletion or repetition at the 64 kbit/s level Service Effect of octet slips Voice (64 kbit/s PCM) Sometimes an audible click HDLC Data One or two lost frames per slip: throughput loss depends on error detection and recovery scheme of application ADPCM Voice (32 kbit/s) More than twice as bad as 64 kbit/s PCM voice Compressed Video (DSl) Each slip can wipe out one or more lines Facsimile Depending on system used, each slip can wipe out up to eight lines
Synchronous operation in the PSTN switching layer relies on the transport network to carry synchronization information between the network elements of the PSTN. In the days that primarily PDH technology was used for the transport of digital signals, the El and DSl carrier frequencies (2048 kHz and 1544 kHz) were used in the 30-channel and 24-channel markets, respectively, to carry this synchronization information. With the introduction of SDH and SONET, this role was taken over by the STM-N carriers. This change means that in order to synchronize the PSTN network elements, the
Synchronization of Optical Networks
235
El, DSl, and STM-N signals need to be synchronized to some common clock. An important difference between PDH and SDH as transport technologies is that PDH network elements themselves do not need to be operated synchronously in order to transport a synchronous El or DSl signal without introducing levels of wander that are unacceptable for synchronization purposes, thanks to the specific PDH bit-stuffing mechanism. In contrast, SDH networks cannot transport El or DSl payloads with sufficient phase transparency for synchronization purposes, due to the nature of the SDH pointer adjustment mechanism. This is why in the case of SDH, the STM-N carrier signals have been introduced as carrier for synchronization information and, as a consequence, the SDH network elements need to be synchronized as well. Fortunately, the synchronization of the SDH network elements also has a beneficial effect on the wander accumulation of payload signals as explained in the previous sections, since it greatly reduces the probability of pointer adjustments. For the distribution of synchronization information, the presence of OTN networks hardly matters. The synchronization information is still transmitted via the STM-N carrier phase and frequency. Mapping an STM-N signal as a client in an ODUk signal and transporting and cross-connecting it through an OTN network does not affect the timing properties of the recovered STM-N signal at the OTN egress, even if multiple OTN networks are traversed. The mapping of STM-N signals in the ODUk is defined such that the timing properties of the STM-N signals are maintained, even if all nodes in the OTN network have a free-running local clock. This property of the OTN network is comparable to PDH networks, which also are capable of maintaining the frequency and phase properties of synchronous El and DSl signals, while the higher-order PDH signals are derived from free-running network element clocks.
6.5.2
Synchronization Areas
In the public network there is a clear need for a high-quality "master clock" that serves as the synchronization source for all network elements in a certain area, called a Synchronization Area. In such an area, all other network elements operate synchronous with this master clock. 6.5.2.1
Synchronization Reference Chains
In a synchronization area, synchronization information is distributed by interconnection of the network elements via STM-N links or, in case a network element has no STM-N interface, by a synchronous El or DSl
236
Chapter 6
interface. Using phase-locked loop techniques, the timing reference information is recovered from the incoming synchronization link and used to control the frequency of the internal oscillator(s) in the equipment, which in turn synchronizes the outgoing signals. In this way, many network elements can be cascaded, and all are eventually timed by the master clock of the area. The length of the resulting chain of clocks has to be restricted. The longer such a chain becomes, the more phase noise will be accumulated, which degrades the quality of the synchronization information. It is part of the network synchronization plan to design the synchronization paths in such a way that the number of clocks that form a chain, starting at the master clock of the area, remains limited. The amount of degradation or phase noise accumulation on a synchronization path depends on the number and type of clocks that are in the synchronization chain. Each clock will generate some phase noise due to component imperfections, but will also filter phase noise generated by clocks upstream in the chain, depending on the frequency of the noise. In addition, interruptions in the path of the clock chain cause additional phase excursions. For practical networks, the composition of synchronization chains varies considerably. To be able to get a handle on this, a "synchronization reference chain" has been composed for standardization purposes as a model for such chains. In ITU-T Recommendation G.803 [30], a model chain is defined as consisting of one PRC clock as a master and up to 10 SSU and up to 60 SEC clocks slaved to it, with the restriction that the number of SEC clocks in tandem is never more than 20. This model is used throughout Option 1 networks. For Option 2 networks, another model is assumed (in Telcordia GR-436-CORE [40]), where each chain consists of one PRS (Stratum-1 clock) at the start of the chain, followed by a number of Stratum-2 clocks with at most one SMC between two successive Stratum-2 clocks (collocated with the upstream Stratum-1 or Stratum-2 clock) and possibly up to 16 SMCs at the end. The various clock types are summarized in Tables 6-3 and 6-4,
Synchronization of Optical Networks
237
Table 6-3. Overview of standardized clock types in Option I networks (SDH markets) Clock type Applicable standard Quality indication Purpose IxlO'' lifetime PRC (Primary ITU-T Rec.G.811 Master clock, standaccuracy Reference Clock) alone equipment SSU-T (Synchronization Supply Unit — Transit node)
ITU-T Rec.G.812, type I (modern equipment) or type V (older equipment)
3x10'^ holdover stability (period of months)
Slave clock, standalone equipment or built-in in larger network elements (switch or crossconnect)
SSU-L (Synchronization Supply Unit — Local node)
ITU-T Rec.G.812, type VI (older equipment only)
1x10'^ holdover stability (period of days)
Older type slave clock, built-in in larger network elements (switch or cross-connect)
SEC (SDH Equipment Clock)
ITU-T Rec.G.813, Option 1
5x10"^ holdover stability (period of minutes)
Slave clock, built-in in SDH network elements
Table 6-4. Overview of standardized clock types in Option II networks (SONET markets) Clock type Applicable standard Quality indication Purpose 1x10"'^ lifetime Master clock, standPRS (Primary ITU-T Rec. alone equipment Reference Source) G.811/ANSIT1.101 accuracy Stratum-2
ITU-T Rec. G.812, type II
1.6x10'^ lifetime accuracy 1x10-'° holdover stability (period of 24 hours)
Slave clock, standalone equipment or built-in in larger network elements (switch or crossconnect)
Stratum-3E
ITU-T Rec. G.812, type III
4.6xlO-Mifetime accuracy 1.2x10'^ holdover stability (period of days)
Slave clock, built-in in larger network elements (switch or cross-connect)
Stratum-3
ITU-T Rec. G.812, type IV (older equipment only)
4.6xlO"Mifetime accuracy 3.9x10-^ holdover stability (period of 24 hours)
Older type slave clock, built-in in larger network elements (switch or cross-connect)
SMC (SONET Minimum Clock)
ITU-T Rec. G.813, Option 2
5x10"^ holdover stability (period of minutes)
Slave clock, built-in in (small) SDH network elements
238
Chapter 6
The standards have been constructed so that if a network element experiences phase noise as if it is located at the end of a synchronization reference chain, the overall jitter and wander requirements are met for signals transported through such networks. Given the composition of the reference chains, the real network clock chains will generally be much shorter, so the network level performance will be better, even if occasionally some chains are longer than the reference chain. 6.5.2.2
Synchronous and Plesiochronous Operation
A network is said to operate synchronously when all network elements in that network have their internal oscillators (i.e., their "local clock") locked to a single reference clock, the master clock of that network. In this situation, the long-term average frequencies of all local clocks are equal. When considered over a shorter period of time, there may be deviations due to jitter and wander effects, but these are essentially random phase effects that are bounded^^ in the long run. For a network to operate synchronously, it is necessary to distribute the reference signal from the master clock to all network elements of that network. The geographical area in which this situation is the normal state of affairs, i.e., in the absence of failures, is called a Synchronization Area, In practice, most operators design their network to be a single synchronization area, unless the area they serve is very big or consists of multiple disjoint areas, in which case there can be more than one Synchronization Area in a single operator's domain. Since digital signals may need to traverse different Synchronization Areas to reach their destination, each transition between such areas may cause an octet slip at the level of the 64 kbit/s bearer channel. ITU-T Recommendation G.822 prescribes that, under normal conditions, at most one single octet slip is allowed in a 70-day period between adjacent Synchronization Areas. One octet corresponds to 125 |is in the time domain, and the build-up of a time difference of 125 |is in 70 days corresponds to a relative frequency difference of 2xl0"^\ So as long as the master clocks of Synchronization Areas have a frequency that deviates less than 1x10'^^ from the absolute (UTC), the Recommendation is fulfilled. The situation in which the long-term average frequencies of local clocks of adjacent network elements deviate less than 1x10'^^ from the absolute is called Plesiochronous Operation. In case frequency differences are substantially larger than allowed by G.822, but each digital signal still meets its own frequency specification, the network operation is called asynchronous. Table 6-5 gives an overview of
Synchronization of Optical Networks
239
the frequency limits that apply for the various digital signals in asynchronous operations. Table 6-5. Maximum allowed relative frequency deviations for digital signals Signal type Maximum relative frequency offset 4.6x10-^ (20x10-^ for AIS) STM-N (OC-N) DSl 32x10-^ El 50x10-^ 30x10"^ DS2 E3 20x10-^ 20x10-^ DS3 15x10-^ E4 OTUk 20x10-^
Note that of the signals listed in Table 6-5, only STM/OC-N, El, and DSl signals normally operate plesiochronous or synchronous. For OTUk and higher-order PDH signals, asynchronous operation is the normal mode of operation, since such signals are not used to transport synchronization within the network. The PDH multiplexing scheme is designed such that synchronous El or DSl signals maintain their phase and frequency properties when they're carried over higher-order PDH links, even when these carriers themselves run off frequency but still within the bounds given in Table 6-5. The OTN is designed such that synchronization can be transported via SDH clients. 6.5.2,3
Traceability and Quality
From the description of Synchronous and Plesiochronous operation, it is clear that synchronization information has to be distributed from a very accurate master clock with less than 1x10"^^ fractional frequency error to all other network elements in the Synchronization Area. To achieve a long-term accuracy of this level, it is necessary to apply Cesium technology. This technology is too bulky and too expensive to apply in each network element— hence the need for synchronization distribution. Synchronization information can be distributed by using a chain of interconnected clocks. The first clock of the chain acts as the "Master Clock" of the chain, and the frequency and phase information of this clock is used as the "Reference Signal" by each subsequent element (slave clock) in the chain to control its output signal, using a phase locked loop technique. All clocks are thus slaved to the master clock of the chain. The concept of Traceability is based on the practice of chaining clocks. All clocks in the chain are said to be traceable to the first clock or master clock of the chain: their long-term average frequencies are equal.
240
Chapter 6
Since the master clock is the eventual source of the synchronization for all slave clocks in the connected synchronization chain(s), it is important that its quality is adequate. The "Quality" of a clock is determined by multiple factors, but for the discussion of reference chains, the long-term accuracy of its frequency is the most important. Tables 6-3 and 6-4 list the most important quality characteristics of standardized clocks. 6.5.2.4
PRC/PRS Autonomy
Another concept that is useful in the discussion of reliable distribution of synchronization signals is the notion of PRC/PRS autonomy. ITU-T Recommendation G.811 provides the specification of PRC (Option 1 networks) or PRS (Option 2 networks) equipment,^^ of which the most important is the long-term frequency accuracy requirement of lxlO'^\ Such a clock is suitable as the master clock of a Synchronization. However, the link between two clocks may fail and an important question in that case is how long the slave clock, without its accurate reference present, can maintain its output phase and frequency within specification. This period of time is called the PRC Autonomy or PRS Autonomy and plays a role in determining the reference protection strategy. It is illustrated in Figure 6-15.
Allowed Phase Drift - Actual Phase Drift
1 X 10-11
PRC/PRS Autonomy
Figure 6-15. PRC/PRS Autonomy
Time (log)
Synchronization of Optical Networks
241
The PRC/PRS Autonomy and the long-term frequency accuracy are the parameters that determine the quality of a clock. The master clock has the highest quality in a synchronization area. Slave clocks can have very different specifications with regard to PRC/PRS autonomy and frequency accuracy. In the ITU-T Recommendations G.812 and G.813, a number are defined, with frequency accuracies between 3.2x10'^ and 1x10"^ and PRC/PRS Autonomies ranging from seconds to days. The former require quick automatic reference mechanisms; the latter may just wait until the failed link is repaired.
6.53
Reference Duplication and Reference Selection
The time-tried method to enhance reliability is duplication. Simply put: a Network Element can accept multiple input signals to be a potential source of synchronization information, but only one of them is active at any given time. In case this reference fails or a better reference becomes available, the network element may perform a reference switch. The need for reference duplication increases for equipment with clocks that have shorter PRC Autonomy. On the other hand, for Stratum-2 clocks with their excellent hold-over stability, reference duplication is not strictly necessary, since there is sufficient time to repair a reference connection in case of failure, before the frequency of the clock drifts too far away. On the lower end of the clock quality spectrum, and especially for SEC and SMC clocks, operation with duplicated (or triplicated, etc.) references is almost mandatory. With the introduction of multiple synchronization references for a single clock, unavoidably the problem of reference selection is introduced. A mechanism needs to be in place to perform automatic reference switching. Two such mechanisms are in use, namely, selection based on priority and selection based on quality. 6.5.3.1
Priority-based switching
The priority-based reference selection mechanism is very simple. Each potential timing reference input is assigned a certain priority by the network operator. This priority is typically expressed as a number, starting at 1 for the highest priority and incrementing for each alternative reference. The system will then automatically select the acceptable reference with the highest priority as the active reference.
242 6.5.3.2
Chapter 6 Quality-based switching
A second reference selection criterion is the quality of the reference signals. Obviously, if there are multiple references available, it is desirable to select a reference that is traceable to a PRC rather than a reference that is only traceable to an SSU-T, for instance. Unfortunately, it is normally not possible for clocks to distinguish between these traceabilities, based on the frequency and phase properties of the signal. To establish this would require a highly sophisticated decision algorithm in combination with a very accurate internal clock. This setup is only feasible for high-quality standalone clock equipment. For other network elements, the traceability of a reference can only be established based on some form of messaging. In PDH networks, such messaging does not exist'^' but in SDH networks the traceability of a reference signal is conveyed in a 4-bit code in the SI byte of the STMN/OC-N (multiplex) section overhead. Of course, this process can only function properly if the proper message is inserted at the beginning of the chain and is propagated along the chain. The requirements regarding Synchronization Status Messages, as this traceability information is normally called (ITU-T Rec. G.781), are trying to ensure exactly this behavior. 6.5.3.3
Reference Acceptance
In the context of reference selection, switching a reference can be acceptable or unacceptable. The reference selection process operates on the set of acceptable references. Acceptable references are those that are derived from alarm-free input signals and that do not show phase or frequency excursions that are beyond the acceptance levels for the equipment in question. Detecting phase and frequency excursions in the potential references of a network element is not always possible. In principle, only phase and frequency differences can be observed. By comparing, pairwise, the phase and frequency differences between multiple input references and the information from the oscillator(s) in the system itself, it is possible to disqualify references (or internal oscillators) by a system of "majority voting." Depending on the complexity and importance of the equipment in the synchronization network, such mechanisms may be implemented to a certain extent.
Synchronization of Optical Networks
6.5,4
243
Synchronization Status Messages
The presence of Synchronization Status Messages (SSMs) containing reference traceabiUty information in the STM/OC-N overhead allows the synchronization network engineer more options to design adequate reliability, which is especially welcome if sufficient independent references are not available to use in a strict priority-based reference selection scheme. 6.5.4.1
Message Set
Given the differences in the standardization of clocks in Option 1 and Option 2 networks, the SSM code points are also different (see Table 6-6). In principle the rule is that the lower the number, the better the traceability of a reference, but there are a few exceptions to that rule, as included in Table 66. Table 6-6. Available code points in SI byte for SSM messages (after ITU-T Rec. G.781) SI code Option 1 interpretation Option 2 interpretation 0000 invalid STU - Traceability Unknown 0001 PRS - Primary Reference invalid Source (G.811) 0010 PRC - Primary Reference Clock invalid (G.811) 0100 SSU-A - Synchronization Supply TNC - Transit Node Clock UnitA(G.812,typeIorV) (G.812, type V) 0111 invalid ST2-Stratum 2 (G.812, type II) 1000 SSU-B - Synchronization Supply invalid Unit B(G.812, type VI) 1010 invalid ST3(G.812,typeIV) 1011 SEC - SDH Equipment Clock invalid (G.813, option 1) 1100 invalid SMC - SONET Minimum Clock (G.813, option 2) 1101 invalid ST3E - Stratum 3E (G.812, type III) 1110 invalid PROV - Provisionable by network operator nil DUS-Don't Use for DUS-Don't Use for Synchronization Synchronization
Synchronization can also be carried over signals other than STM/OC-Ns, most importantly El and DSl signals. Such signals normally don't carry SSMs, but an exception is made for dedicated El and DSl signals that interface between stand-alone clock equipment and other network equipment
244
Chapter 6
in an office. Such El and DSl signals do not carry payload, but can code SSM information in their overhead (in the TSO frame overhead in the case of El and in the frame Data Link channel in the case of DSl). 6.5.4.2
SSM Protocol Rules
In conjunction with the SSM message set, a set of protocol rules is defined for SDH and SONET network elements that govern the processing of the SSM messages between network elements. SSM message processing is defined for network elements that have a relatively low-level clock like an SEC or SMC in option 1 and option 2 networks, respectively. The protocol rules introduced in this section are illustrated in an example in the next section. 6.5.4.2.1 SSM Selection The selection of a reference based on SSM messages is very straightforward. The reference that has the best traceability is selected, provided that the traceability is better than or equal to the quality level of the internal clock of the network element. In case the quality of the internal clock is better than the traceability of any reference, the internal clock is run without reference in holdover mode. It becomes the "master clock" of the subnetwork that is ''downstream" in a synchronization sense. This situation is normally the case only during failures in the network so that the connection with the PRC or PRS of the network is lost. A final rule for reference selection is that references that carry DUS (Don't Use for Synchronization) can never be selected. In case multiple references have the same traceability, the selection is done according to a provisioned set of priorities that are assigned to each reference input. This means that the SSM-based selection scheme does not replace the priority-based scheme; it is merely added to it. 6.5.4.2.2 SSM Forv^^arding When the SSM of the selected reference is known, this value is forwarded on all outgoing STM-N links. In case the internal clock is in holdover mode, the outgoing SSM is determined by the quality of the internal clock. 6.5.4.2.3 SSM Looping To avoid a timing loop between two neighboring network elements each selecting the other as its reference, an additional rule for the processing of SSMs is that in case a reference is selected by a network element, the signal
Synchronization of Optical Networks
245
in the return path is given the special DUS (Don't Use for Synchronization) message. 6.5.4.2.4 Manual SSM Assignment Two additional options are available to the synchronization engineer to facilitate the use of SSMs, which are useful on the boundaries between a network that uses SSMs and one that doesn't. Each input reference can be assigned a certain traceability (overriding the SSM that is possibly present in the signal). On each output, the DUS message can be inserted (overriding the normally transmitted SSM as established by the protocol). 6.5.4.3
SSM based restoration applied in a ring network
To illustrate the operation of the SSMs and their protocol rules, the example of a simple ring network is worked out. In Figure 6-16, a simple SDH ring network of six network elements connected by STM-N links, labeled A through F, is shown. Each network element has an SEC-type internal clock. NE A receives (duplicated) synchronization signals from outside the ring with PRC traceability (the traceability is shown in the ellipses) and acts as the timing master for the network elements in the ring. The traceability of the reference of node A, PRC, is forwarded on both outgoing STM-N links on the ring. The ring nodes B through F take their reference from either the "east" or the "west" line input. The selection is based on the received SSM and, if those are equal, on a provisioned priority. In the normal situation, the ring is provisioned such that the active synchronization runs clockwise. This result is achieved by provisioning the input from the clockwise direction with a higher priority, i.e., "1, " than the counter-clockwise input, which has priority "2. " The priorities are in the small circles. The outgoing STM-N signal gets the same SSM as the selected reference, i.e. "PRC, " except the output in the direction of the active reference, which is assigned "DUS" to avoid a direct timing loop between adjacent nodes. The active synchronization topology is denoted by the solid lines, and this network forms, indeed, a "tree." When the link between nodes B and C fails, the active reference of node C will become unavailable, and this node will attempt a reference switch. However, the only alternative reference carries DUS and is not usable. Therefore, node C switches to "SEC," representing the internal clock in hold-over. Node D detects the changing of the traceability from PRC to SEC and, for lack of an alternative reference, accepts the degradation of the
246
Chapter 6
traceability, which is still not worse than its internal clock. Node D forwards the new SSM to E, which reacts exactly the same way.
(gC^
c
D
1 ||||SEC|||| i
i^Uus"^
^^^^
^ >
C5^/'
(SEC )
•B I ^^^^^^^^ fP Cj^^
Figure 6-16. SSMs applied in an SDH ring network: normal situation
Node F also sees the degradation of the traceability from PRC to SEC on its active reference, but it has a better alternative, i.e., the input from node A, which still is traceable to a PRC. Node F will make a reference switch to the other input and update the SSMs on its outputs accordingly: DUS back to node A and PRC to node E. Figure 6-17 shows exactly this situation. The restoration process of the ring synchronization completes when nodes E, D and C, in that order, switch reference to the other side because that input carries a better traceability (PRC) than the currently active one (SEC). In the final situation, the synchronization reference chain is split into two parts (Figure 6-18): one clockwise from node A to node B and the other counterclockwise from node A to node C. The process to reach this situation completes within a few seconds. This ensures that the phase transient caused by the short period in the holdover experienced by nodes C, D, and E remains well within 1 |is^^ Once the failed link is repaired, the synchronization situation will return to the situation in Figure 6-16, thanks to the assigned priorities, which cause the synchronization switching to show revertive behavior.
Synchronization of Optical Networks
247
an
0
(SEC)
<so '<SE>
Active synchronization Stand-by synchronization
<SD
c
D
C!E>^ Blliliiiiipj
^ CSD>
^0o:
Figure 6-17. SSMs applied in SDH ring network: restoration in progress
Figure 6-18, SSMs applied in SDH ring network: restoration completed
248
Chapter 6
6*5,5
Satellite Timing
6.5.5.1
GPS
The Global Positioning System (GPS: a general introduction is given in [42]) can very suitably be used as an alternative to synchronization distribution (see, e.g., [43] and [44]) via STM-N signals. The satellites in the GPS system continuously transmit signals for positioning purposes, from which a very accurate frequency can be derived. The satellites themselves contain a combination of duplicated Cesium and Rubidium clocks, which are kept synchronous by ground stations that operate exactly on UTC. A GPS receiver can construct an accurate reference from the signals of the satellites in view. Since the GPS signals have very good long-term stability, but relatively poor short-term stability, the GPS receiver needs to be integrated in a system that contains an SSU-T or Stratum-2 level clock to improve the short-term stability. The combination of a GPS receiver and a high-quality slave clock can easily meet the G.811 requirements for PRC/PRS (and Tl.lOl requirements for the latter) network master clocks, avoiding the costly Cesium technology. This method of implementing the master clock function is quickly gaining popularity. GPS is the best-known positioning system, but alternative satellite systems can also be used for the purpose of deriving accurate timing. 6.5.5.2
Effect of Satellite Timing on Network Synchronization
The introduction of satellite signals as timing reference signals for telecommunication networks can ease the distribution problem enormously. However, there are a number of technical issues that need to be taken into account. • A clock that uses GPS signals for reference purposes needs to have an internal oscillator that has very good short-term stability properties so as to be able to average the reference signals over a sufficiently long time before applying the average as a reference. The averaging process diminishes the effect of the short-term instability (phase wander in the mHz range) of the GPS signal. Rubidium and high-quality quartz oscillators can provide adequate stability. • The reception of GPS signals requires antennas and GPS receivers. This equipment needs to be duplicated for high availability. Long connections to a suitable roof location (with sufficient open view of the sky) are sometimes a problem.
Synchronization of Optical Networks
249
Network operators base their synchronization design on different levels of GPS-derived synchronization. Rather common is to use the GPS signals as a backup for stand-alone synchronization equipment (ST2 or SSU-T clocks). In this case, only one reference through the network is required, which eases the design and diminishes the risk for timing loops. Stand-alone synchronization equipment is present in many larger telecommunication offices, especially in North America. The presence of GPS allows an operator to create many small synchronization areas, since he or she may have many G.811 equivalent clocks. Traffic between such synchronization areas may be considered synchronous because all GPS locked clocks use the same reference, i.e., the GPS signal, although clocks of different vendors may use slightly different algorithms to synthesize a reference signal out of the GPS information. The phase differences between GPS locked clocks easily remain within one 64 kbit/s octet (125 |is), so from a G.822 perspective, controlled octet slip does not occur and the operation is synchronous.
6.5,6
Synchronization Network Engineering
6.5.6.1
Requirements for Synchronization Protection
In a Synchronization Area, the set of actively used references by all network elements in the area, at that point in time, forms the current synchronization network. This network has a so-called spanning tree topology, i.e. it forms an interconnected but loop-free graph [45]. When a link in this network fails, the network element at the receiving end selects an alternative path which changes the graph. However, the selection rules for reference switching should be such that, in case a synchronization reference link fails or multiple references fail at the same time, the outome is as follows: 1. Timing loops are not formed due to reference protection switching, i.e., the current synchronization network keeps a "tree" shape. Timing loops are cases in which a set of network elements uses active reference signals from one another in a circular fashion. Such loops are detrimental because the frequency of these elements may collectively drift away uncontrollably. 2. Clock hierarchy is not violated due to reference protection switching. The clock hierarchy is dictated by the quality of a clock and the traceability of its reference signals. The objective is that a clock of a certain quality should never select a reference that has a traceability that is worse than
250
Chapter 6
its own quality. In such a case, it would be better to switch to holdover operation. 3. The Synchronization Area is not split into disjoint islands (as far as possible with multiple failures), i.e., the current synchronization network remains a "spanning" tree for as long as possible. 6.5.6.2
Synchronization Plan
The design of the synchronization network in a Synchronization Area is often contained in a synchronization plan that provides maps of the network elements and transmission paths in the area and the provisioning data that pertain to the synchronization properties for all these network elements. This determines the topology of the synchronization network, both in the absence of failures and in the presence of one or more failures. Apart from the synchronization plan of the area itself, also the synchronization relationship with neighboring Synchronization areas needs to be taken into account, as well as the synchronization plan within each telecom office in the area.
6,6,
CONCLUSIONS AND CLOSING REMARKS
6.6.1
Conclusions
This chapter has provided an overview of timing, jitter, and synchronization concepts; timing and jitter requirements for SONET, SDH, and OTN; and network synchronization engineering. The chapter has emphasized the key differences between OTN and SDH/SONET timing, namely: • The OTN is not required to be synchronized, and its physical layer is not required to transport synchronization. Instead, synchronization is transported via the SDH clients. • There is a single set of OTN timing and jitter requirements that is applicable globally (unlike SDH, which has two Options). • SDH networks need to be synchronized to be able to use the STM-N and OC-N carrier signals as carriers of synchronization information as well and to minimize the wander accumulation on SDH payload signals. • With the aid of high-quality clock equipment located at strategic places in the network, it is possible to reliably distribute synchronization through the OTN, SDH, and PDH networks to the PSTN equipment.
Synchronization of Optical Networks
251
•
The synchronization network engineering of the SDH network is for these reasons more complex than for OTN or PDH networks, but it allows the use of relatively inexpensive equipment clocks in SDH equipment. These differences result in OTN timing being simpler from both the standpoints of hardware design and network management. For the former, the ODC requirements are less stringent than the SEC requirements in SDH, mainly in that none of the ODC's must meet the holdover and more stringent transient requirements of SDH. For the latter, there is no need to manage a synchronization network at any of the OTN layers.
6,6.2
Closing Remarks
The "building" of specifications listed in Table 6-1 of this chapter is, in essence, a long and detailed justification for the conclusions summarized in the bullet points above for the various transmission technologies: PDH, SDH/SONET and OTN. Most of this work has been carried out in the years 1988 through 2001, the years in which the SDH/SONET and OTN technologies were conceived and introduced in transport networks worldwide. The body of specifications is quite complete, and most standardization professionals consider the subject of network synchronization more or less finished, so participation in these particular standards activities is dropping. Two trends are discussed briefly that can potentially revive this interest. Mobile networks have not been specifically addressed in this chapter, but in general the synchronization strategy of such networks is not very different from the "wireline" optical networks that have been discussed extensively. The mobile part of mobile networks is actually only the few kilometers between the base station and the hand-set, the so-called air interface. From a synchronization perspective, the main problem is to ensure that the carrier frequencies of the base stations are sufficiently stable. Bringing a reference of SDH/SONET quality to the base station is sufficient for that purpose. Since the base station interfaces are often PDH interfaces (El, DSl), dropped from an SDH/SONET network, a technique called retiming is applied to force the frequency accuracy of the SONET/SDH network on the outgoing El or DSl drop signal. The technique of resynchronization is widely applied, but not standardized in detail. This is a relatively small potential area of standardization work. A more challenging problem for network synchronization specialists is likely to be the rise of Constant Bit Rate (CBR) services over packet networks, i.e., ATM, IP, or MPLS. Examples of such services are voice (VoIP) or video transport over IP networks and, lately, the work on the "Pseudo-Wire" specification in IETF. There is some experience with
252
Chapter 6
modeling Telephony over ATM, an area in which the mathematical techniques explained in this chapter have been applied to the Cell Delay Variation (CDV) of ATM-cells. This quantity is comparable to the phase noise found in TDM networks, although the numbers (and so the related problems) are one to several orders of magnitude larger. When shifting from ATM to IP or MPLS the analysis problem becomes harder, since the packet length in IP and MPLS networks is not constant as in ATM networks, and, in addition, the packet delay variation is still larger than in ATM. In addition, the Quality of Service (QoS) aspects of the transport is much less regulated and standardized than in ATM. There is still much work to do to relate QoS parameters to the real-time requirements for the arrival times of packets and the perceived responsiveness of the network by the end-users. So far, some commercial implementations of VoIP exist and success has been claimed. In practice, this success has always been achieved on relatively small networks where bandwidth and routing capacity were not limiting factors. In large-scale deployments, this may no longer be true, which calls for extensive modeling and testing, eventually leading to a next generation of standards documents.
6.7.
NOTES
1. In practice, other considerations come into play. For example, electrical levels may be chosen to be +V and -V, so that the average level is DC-free for 50% ones density. Optical power levels may be chosen consistent with requirements on average power; in addition, extinction ratio (ratio of optical power level corresponding to 1 to optical power level corresponding to 0) is finite. 2. In general, the bit time may be associated with any convenient property of the pulse. The property chosen depends on the modulation scheme. 3. Since, in the notation here, this time corresponds to the pulse leading edge, an additional fixed offset time should be added so that the pulse is sampled near its center. 4. The slip will be controlled or uncontrolled, depending on the mechanism implemented. With a controlled slip, an entire frame (based on a defined frame structure) is repeated or dropped. With an uncontrolled slip, the buffer is reinitialized and data is deleted or repeated as necessary. With a controlled slip, reframing is not necessary because frame boundaries are not lost. Note that it is assumed here that the server layer is the physical layer, i.e., the client signal is not being mapped or multiplexed into a higher-rate server layer that is itself a digital layer. In this latter case, justifications and/or pointer adjustments may be used to prevent data loss when the transmit (server) and receive (client) clocks are different. These mechanisms are described in Sections 6.2.3 and 6.2.4. 5. The term 3R denotes reamplification, reshaping, and retiming. The term is used in OTN because there a link can contain optical amplifiers that, in general, only perform reamplification (an optical amplifier not used in a regime where it distorts the pulses is a IR process). The term is not used in SDH/SONET because there all regenerators are 3R. Some PDH networks have used 2R (reamplification and reshaping) and 3R regenerators together; however, these were not standardized.
Synchronization of Optical Networks 6. 7.
8.
9.
10.
11. 12. 13.
14. 15.
16. 17.
18. 19.
253
Using the terminology of eye diagrams, there is more eye closure; see references [3] and [5]. This definition of jitter tolerance applies only to optical receivers. For electrical receivers, the transition between an error-free signal and loss of frame is abrupt, and a stable error rate cannot be easily created by attenuating the signal. In reality, the PLL will be at least second-order (with first-order loop filter); however, the argument remains the same as long as the roll-off is still -20 dB/decade and the gain peaking is small (both are true in practice). Actually, timing impairments can occur at frequencies up to the Nyquist rate, i.e., onehalf the bit rate. These high-frequency impairments are not measured using typical SONET/SDH jitter test sets. Such impairments are due to specific effects, and are instead considered separately in the power budget in SONET/SDH and OTN. A different specification and measurement approach is taken in defining jitter for 10 Gbit/s Ethernet [41]. The discussion in this subsection is taken from the discussion in Sections 5.3.1 and 5.3.2 of [20], with some condensation. Section 5 of [20] contains a summary of concepts used in describing time, frequency, and network synchronization. There is also an assumption that the mean of x{t) is zero; however, any nonzero mean can be included in the deterministic part of the phase error. The term SONET is used in the North American (Tl) standards; the ITU-T recommendations use the term Option 2 SDH. The historical situation is actually somewhat more complicated. The specifications in ANSI, for SONET, were not originally fully consistent with G.825. However, the jitter tolerance and network limits for OC-3, -12, and -48 were made consistent with G.825 in the 2003 revision of T 1.105.03 [11] (however, the Option 1 and Option 2 jitter generation and transfer specifications are not the same for these rates). In addition, both SONET and SDH have specified reduced jitter tolerance regenerators, referred to as Type B regenerators. In this chapter, only the normal jitter tolerance regenerators are discussed; these are referred to as Type A regenerators. In OTN, there is only a single set of specifications, and all these complications were avoided. The studies were contained in contributions to the respective standards bodies. For sufficiently close optical channel spacings, additional jitter may result from crossphase modulation, a nonlinear effect. This effect is not considered in G.8251 [8], Further work is needed to specify the jitter due to this effect, or any other nonlinear effects. For Option 2 networks, the corresponding numbers are under study but, in practice, are typically smaller. A rough justification for this conclusion may be seen from the fact that the zero-to-peak response of passing a unit step through a low-pass filter followed by a high-pass filter, where the low-pass filter bandwidth is much less than the high-pass filter bandwidth, is approximately the ratio of the low-pass bandwidth to the high-pass bandwidth. Therefore, if the high-pass measurement filter bandwidth is increased by a factor, zeroto-peak jitter is reduced by approximately the same factor. For Option 2 networks, the corresponding numbers are under study but, in practice, are typically smaller [30]. Whether or not a network of clocks, locked to some master, has mutual phase differences that remain bounded over time is the subject of some controversy. Two mechanisms can be mentioned that cause temporary loss of lock, which can over very long times accumulate unbounded. First, the phase detector in a slave clock has essentially a periodic design, so the internal noise of an oscillator in the PLL circuit or the phase noise
254
Chapter 6
on the reference signal may cause the clock to "skip" a cycle, which is never recovered. Second, temporary loss of reference is normally accommodated by a "fly-wheel" mechanism called "hold-over". If the loss of reference extends over a longer period there is a probability that some cycles are lost after regaining lock. A more precise statement is that in a synchronous network the phase differences are bounded on the very short timescales associated with a single cycle as long as the clocks maintain lock. 20. PRS clocks have to meet the short-term stability requirements of ANSI Tl.lOl [37], which are slighdy more stringent than those in ITU-T Rec. G.811. In this sense, a PRC and a PRS are not exactly equal. However, this difference has no consequence for the discussion. 21. An exception must be made for SDH-type signals operating at PDH (E3, DS3, and E4) bit rates as defined in ITU-T Recommendation G.832. These signals contain a one bit "Timing Marker" that can contain extremely basic Traceability information. However, G.832 signals find little or no application in real networks. 22. The value of 1 |is has been the objective for Option 1 networks that has been used in constructing the rules for the SSM controlled synchronization process for SECs, as defined in ITU-T Rec. G.781. This number has been taken from older G.812 clock specifications that allowed maximum output phase transients of this magnitude. To meet the 1 |as phase transient limit under worst-case conditions for a 20 node ring, it has to complete its restoration process within 15 s. A phase transient of 1 )LIS can cause a burst of 7 pointer adjustments at the AU4 and STSl levels, a well-known test sequence for SONET equipment (ANSI Tl. 105.03). For Option 2 networks, the reconfiguration process can take longer due to the narrower bandwidth of the Option 2 SEC (0.1 Hz or less) compared to the Option 1 SEC (1-10 Hz). As a result, the phase transient magnitude will be larger in Option 2 networks.
6.8.
REFERENCES
[1] John C. Bellamy, "Digital Network Synchronization," IEEE Communications Magazine, Vol. 33, No. 4, pp. 70-83, April, 1995. [2] Patrick R. Trischitta and Eve L. Varma, Jitter in Digital Transmission Systems, Artech House, Norwood, MA, 1989. [3] Sergio Benedetto, Ezio Biglieri, and Valentino Castellani, Digital Transmission Theory, Prentice-Hall, Englewood Cliffs, NJ, 1987. [4] Heinrich Meyr, Marc Moeneclaey, and Stefan A. Fechtel, Digital Communication Receivers, Synchronization, Channel Estimation, and Signal Processing, Wiley, New York, 1998. [5] Govind P. Agrawal, Fiber-Optic Communication Systems, Wiley, New York, 1997. [6] Heinrich Meyr and Gerd Ascheid, Synchronization in Digital Communications, Volume 1, Phase-, Frequency-Locked Loops, and Amplitude Control, Wiley, New York, 1990. [7] Dan H. Wolaver, Phase-Locked Loop Circuit Design, Prentice-Hall, Englewood Cliffs, NJ, 1991. [8] ITU-T Recommendation G.8251, The Control of Jitter and Wander Within the Optical Transport Network (OTN), ITU-T, Geneva, November 2001; Amendment 1, June 2002; Corrigendum 1 June, 2002. [9] ITU-T Recommendation G.825, The Control of Jitter and Wander within Digital Networks which are Based on the Synchronous Digital Hierarchy (SDH), ITU-T, Geneva, March 2000; Erratum 1, August 2001.
Synchronization of Optical Networks
255
[10] ITU-T Recommendation G.783, Characteristics of Synchronous Digital Hierarchy (SDH) Equipment Functional Blocks, ITU-T, Geneva, February 2004; Corrigendum 1, June 2004. [11] ANSI Standard T 1.105.03-2003, Synchronous Optical Network (SONET) — Jitter at Network Interfaces, ANSI, 2003. [12] C. J. Byrne, B. J. Karifin, and D. B. Robinson, Jr., "Systematic Jitter in a Chain of Digital Regenerators," Bell System Technical]ournal, pp. 2679-2714; November 1963. [13] E, L. Varma and J. Wu, "Analysis of Jitter Accumulation in a Chain of Digital Regenerators," Proceedings of IEEE Globecom, Vol. 2, pp. 653-657, 1982. [14] ITU-T Recommendation G.709Ar.l331, Interfaces for the Optical Transport Network (OTN), ITU-T, Geneva, March 2003; Amendment 1, December 2003. [15] ANSI American National Standard Tl. 105-2001, Synchronous Optical Network (SONET) — Basic Description Including Multiplex Structure, Rates, and Formats, 2001 (supplement Tl. 105a published in 2002). [16] ITU-T Recommendation G.707A'.1322, Network Node Interface for the Synchronous Digital Hierarchy (SDH), ITU-T, Geneva, December 2003; Amendment 1, August 2004; Corrigendum 1, June 2004. [17] ITU-T Recommendation G.810, Definitions and Terminology for Synchronization Networks, ITU-T, Geneva, August 1996; Corrigendum 1, November 2001. [18] J.A. Barnes, A.R. Chi, L.S. Cutler, D.J. Healey, D.B. Leeson, T.E. McGunigal, J.A. Mullen, Jr., W.L. Smith, R.L. Sydnor, R.F.C. Vessot, and G.M.R. Winkler, "Characterization of Frequency Stability," IEEE Transactions on Instrumentation and Measurement, Vol. IM-20, No. 2, pp. 105—120, (reprinted in NIST Technical Note 1337, March 1990), May, 1971. [19] D. B. Sullivan, D. W. Allan, D. A. Howe, and F. L. Walls, Characterization of Clocks and Oscillators, NIST Technical Note 1337, U.S. Dept. of Commerce, National Institute of Standards and Technology, March 1990 (see, for example, section IX of paper B.l). [20] Geoffrey Garner, Application of Time and Frequency Stability Parameters to the Characterization of ATM Cell Delay Variation and Traffic, Contribution to Tl Standards Project, TIX 1.3/96-086, TlAl.3/96-068, October, 1996 (available via http://www.atis,org) [21] J. A. Barnes and D. W. Allan, "A Statistical Model of Flicker Noise," Proceedings of the IEEE, Vol. 54, No. 2, pp. 176-178, February 1966. [22] Benoit B. Mandelbrot and John W. Van Ness, "Fractional Brownian Motions, Fractional Noises and Applications," SIAM Review, Vol. 10, No. 4, pp. 422-437, October 1968. [23] Jens Feder, Fractals, Plenum Press, New York, 1988 (Chapter 9). [24] Gennady Samorodnitsky and Murad S. Taqqu, Stable Non- Gaussian Random Processes, Chapman & Hall, New York, 1994. [25] ITU-T Recommendation G.811, Timing Characteristics of Primary Reference Clocks, ITU-T, Geneva, September 1997. [26] ITU-T Recommendation G.822, Controlled Slip Rate Objectives on an International Digital Connection, Extract from the Blue Book, ITU-T, Geneva, November, 1988. [27] Maarten Vissers, "Optical Transport Network & Optical Transport Module," Beyond SONET/SDH Conference, Paris, France, April 2001. [28] Roland E. Best, Phase-Locked Loops, Theory, Design, and Applications, 2"'' edition, McGraw-Hill, New York, 1993. [29] D. B. Leeson, "A Simple Model of Feedback Oscillator Noise Spectrum," Proceedings IEEE, pp. 329-330, February 1966.
256
Chapter 6
[30] ITU-T Recommendation G.803, Architecture of Transport Networks Based on the Synchronous Digital Hierarchy (SDH), ITU-T, Geneva, March 2000. [31] ITU-T Recommendation G.824, The Control of Jitter and Wander within Digital Networks which are Based on the 1544 kbit/s Hierarchy, ITU-T, Geneva, March, 2000. [32] ITU-T Recommendation G.823, The Control of Jitter and Wander within Digital Networks which are Based on the 2048 kbit/s Hierarchy, ITU-T, Geneva, March, 2000. [33] ANSI Standard Tl.102-1993 (R1999), Digital Hierarchy — Electrical Interfaces, ANSI, 1999. [34] ANSI Standard T l . 105.09-1996 (R2002), Synchronous Optical Network (SONET) — Network Element Timing and Synchronization, ANSI, 1999. [35] ANSI Standard Tl.403-1999, Network and Customer Installation Interfaces — DSl Electrical Interface, ANSI, 1999 (Supplements T1.403a-2001, T1.403b -2002), ANSI, 1999,2001,2002. [36] ANSI Standard Tl .404-2002, Network and Customer Installation Interfaces — DS3 and Metallic Interface Specification (Revision and Consolidation of TL404-I994 and TL404a-1996), ANSI, 2002. [37] ANSI Standard Tl.101-1999, Synchronization Interface Standard, ANSI, 1999. [38] ITU-T Recommendation G.813, Timing Characteristics of SDH Equipment Slave Clocks (SEC), ITU-T, Geneva, March 2003. [39] ITU-T Recommendation G.798, Characteristics of Optical Transport Network Hierarchy Equipment Functional Blocks, ITU-T, Geneva, October 2001 (to be published). [40] Telcordia Technologies Generic Requirements document GR-436-CORE, Digital Network Synchronization Plan — Issue 1 with Revision 1, Telcordia Technologies, June 1996. [41] IEEE 802.3ae-2002, IEEE Standard for Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications-Media Access Control (MAC) Parameters, Physical Layer and Management Parameters for 10 Gb/s Operation, 2002. [42] Elliott D. Kaplan (Ed.), Understanding GPS — Principles and Applications, Artech House, 1996. [43] Wayne Hanson, "Time and Frequency Transfers by Satellite," Chapter 12 of Proceedings of the NISTTime and Frequency Seminar, Boulder, CO, June 8-10, 1993. [44] W.J. Klepczynski, GPS and PTTI, Chapter 13 of Proceedings of the NIST Time and Frequency Seminar, Boulder, CO, June 8-10, 1993. [45] Kenneth H, Rosen, Discrete Mathematics and Its Applications, A^^ ed., McGraw-Hill, New York, 1999 (Chapter 8).
Chapter 7 SYNCHRONIZATION ARCHITECTURES FOR SONET/SDH SYSTEMS AND NETWORKS
P. Stephan Bedrosian Agere Systems
7.1.
SYNCHRONIZATION CONCEPTS
This chapter provides an overview of issues that must be addressed when designing synchronization functionality used in SONET/SDH systems and networks. In order to understand the place that timing plays in a SONET/SDH network or system, the following questions must be addressed first. Why is synchronization needed? Synchronization is a means of defining or decoding data. As such, timing imperfections can cause data errors. Therefore, an accurate clock is needed for accurate data recovery. • How is synchronization administered? Synchronization flows through a hierarchical structure. In North America, this hierarchy is divided into discrete categories of accuracy or quality called stratum levels. Clocks of a particular stratum level may provide timing signals to other clocks of an equal or lesser stratum. In this way, the quality of the timing distribution is controlled. • Why is source traceability important? The term traceability is commonly used in timing terminology to describe the synchronous nature of a timing signal. However, traceability alone is not sufficient to establish its suitability for a particular timing application. Rather, the origin or source traceability of the timing information is of great importance. Source traceability provides a means for determining the flow of a timing signal. Such knowledge
258
Chapter 7 is important for preventing timing loops, which essentially destroy the integrity of timing signals and lead to the impairment of data transport. Output Data (with Noise)
Input Bit Stream Data Latch
H^
Clock Recovery
Output Data (with Delay) - - 1 0
Bit Recovery "-^ High Bandwidth tracks signal and noise
External Timing Source BITSorSSU
n_nj
FIFO Buffer Read
Loop Timing Source
••\. Internal Timing Source
1 1
Filtered Clock
3 ruu
Bit Retiming Low Bandwidth Filters out HF noise
Figure 7-1. Symbolic representation of timing recovery
A high-level symbolic representation of synchronization and its role in accurate data recovery is presented in Figure 7-1. A bit-stream containing data with an embedded clock (or service clock) is typically the source signal. This signal may contain noise as added by artifacts of various transmission media. As a result, the first step is to accurately recover each bit of information; the second step is to perform a bit retiming. Bit recovery is done by first recovering the service clock along with the noise from the transmission medium. Since the majority of this noise is typically over 10 Hz (possibly in the MHz), a high bandwidth recovery function is used. There are numerous schemes for this recovery process; however, most involve detecting transitions on the incoming bit stream and processing the time between transitions to derive a series of alternating ones and zeros. The ability of any timing recovery algorithm to successfully generate a recovered clock will depend on the noise and data density (ones density) on the incoming bit stream. The end result of the timing recovery process is a clock that tracks the transitions on the incoming bit stream. Once a recovered clock is obtained, it is used to directly decode the data from the incoming bit stream. To provide the largest margin, the recovered clock should be shifted in time so that its edges occur in the middle of the bit or symbol interval of the bit stream. If the relationship between the recovered clock and the bit stream tend to move past a bit boundary, then it is possible to generate a bit error. It is for this reason that the clock recovery process tends to use a high bandwidth tracking mechanism to follow any
Synchronization Architectures for SONET/SDH Systems and Networks
259
phase/frequency modulation (sometimes referred to 2iS jitter) that can occur on the incoming bit stream. In order to control the frequency or phase variations on the recovered data, the data can be reclocked by a new clock domain. This process typically involves writing the data into a buffer with the recovered clock. The data is then read out of the buffer with a clock from the new clock domain. The source of timing for this new clock domain can be obtained from a number of sources, including loop/line, through, external, or internal sources. Figure 7-2 illustrates that there are numerous sources for the clock reference of the filtered clock process. These diagrams provide a listing of the various methods used to derive these sources from the external network. It should be noted that not all of these methods are suitable for every application. That is not to say that some methods are better than others; rather, various transmission applications will favor some of these methods over others. For example, if a repeater application is being considered, then the through timing mode will be most appropriate. If a large public SONET/SDH network requires timing, then either the external or line, or a combination of these two modes, may be used depending on the network architecture.
Loop Timing — A timing reference is selected from a single SONET/SDH interface Line Timing — A timing reference is selected from multiple SONET/SDH interfaces Througii Timing — Timing is independently recovered in the direction of transmission DS1/E1
External Timing — A timing reference is selected from a DS1/E1 reference Line/External Timing ~ The NE provides a derived DS1 or E1 to a co-located BITS/SSU that uses this reference to distribute timing back to the NE and other co-located NEs.
Figure 7-2. SONET/SDH Network Element (NE) timing modes
260
Chapter 7
Loop/line timing mode: This mode derives a clock from the received facihty. The derivation usually involves low pass filtering of the recovered clock to remove the unwanted jitter and short-term phase variations. Longer-term phase or frequency variations, however, will be tracked. Loop timing mode involves deriving timing from a single transmission facility, while line timing involves multiple transmission facilities. In order to prevent a timing loop, loop/line timing should not be used on both ends of the connection. Rather, an independent timing reference must be used to establish a clock domain. Therefore, an external or internal timing source must be the source of timing in the clock chain. Through timing mode: This mode derives timing associated with each direction of the transmission facility. So, for each direction of transmission, a separate clock is recovered and used for the outgoing transmit clock. This timing mode is typically used in a 3R (retime, reshape, and regenerate) repeater function, where transmit timing must equal the receive timing. External timing mode: This timing mode relies on using a clock that is typically derived and distributed by a building integrated timing supply (BITS) or synchronization supply unit (SSU). The input to the BITS/SSU is typically a stratum 1 traceable input that can be sourced from a GPS receiver or from a SONET/SDH facility. By using such high-quality timing inputs, a stratum 3E or better BITS is able to filter out timing impairments at a stratum 3E level or better. This process provides a much cleaner clock than could be derived from a line-timed NE alone. When sourced from a GPS receiver, both ends of the transmission facility can use external timing inputs without limitations. It should be noted, however, that the phase difference between the service clock and external clock will be controlled but unbounded. Since the BITS/SSU at each end is establishing a physical clock source based on a GPS input, the frequency and phase between two such clocks will not be identical. However, it is known that the frequency difference at any stratum 1 reference will be off by no more than 10'^^ Therefore, the maximum frequency difference between two GPS timed BITS (or, for that matter, any two stratum 1 timed BITS) will be 2xl0•^^ Line/External timing mode: This is a timing mode that allows the BITS/SSU to use a timing reference carried by the SONET/SDH facility. Since BITS/SSUs do not have any interfaces that can directly receive timing from a SONET/SDH facility, a co-located SONET/SDH LTE (line terminating element) provides the handoff of this timing signal. The co-
Synchronization Architectures for SONET/SDH Systems and Networks
261
located LTE extracts timing from the SONET/SDH frame and then uses this timing to create a special output timing signal called a derived DSl or EL The derived DSl has a nominal bit rate of 1.544 Mbit/s and is formatted as a framed DSl with all-ones data content. The framing can be optioned as either a superframe (SF) or extended superframe (ESF) format. In addition, to providing a derivation of the SONET/SDH frequency, the ESF formatted derived DSl is also able to provide some or all of the synchronization status messaging (SSM) contained in the SI byte of the line overhead. Details on how the derived DSl signals the BITS/SSU are presented in Section 7.7.5.3 of this chapter. The derived El format can be either a framed data signal at 2.048 Mbit/s or a frequency at the El data rate of 2.048 MHz [1]. If the 2.048 MHz format is used, only frequency information can be carried between the SDH LTE and SSU. If the framed 2.048 Mbit/s signal is used, it may contain a channel associated signaling (CAS) or common channel signaling (CCS) framing format with or without CRC-4. In these framed formats, the El can carry SSMs contained in the SI byte of the line overhead [2]. Internal timing mode: This timing mode creates a timing source from an independent frequency source (oscillator) embedded in the actual receiving equipment or NE. The initial state of this timing source is important when determining the quality of its timing. If the internal timing source was "trained" to an external reference within the last 24 hours (e.g., PRS timing source) then it is said to be in holdover mode. If not, the internal timing source is said to be in free-run mode. It should be noted that the accuracy of this mode is also dependent on environmental conditions (e.g., aging, temperature, vibration, voltage variation, etc.). More information on internal timing mode is presented in Section 7.7.6.
7.2.
TIMING TRACEABILITY
The term traceability deals with the synchronous nature between timing signals. Unfortunately, the term by itself does not adequately address the origin or timing properties of these signals. Is the phase difference between two traceable clocks bounded? The answer depends on the architectural arrangement of the two clocks. Basically, this boils down to the concept of timing flow from one or more physical clock sources. Once this answer has been given, then the phase characteristics between two such "traceable" clocks will be known.
262
Chapter 7
Therefore, two classes of traceability can be defined to address these concerns: source traceability and frequency traceability.
7.2.1
Source Traceability
Source Clock
fi^ .DCZ==> f\^ .D^=> f^ \ Source-Traceable Clocks
MTIE generation is bounded
Figure 7-3. Source -traceable frequency relationship
Source traceability: As shown in Figure 7-3, source traceabihty is a relationship where the frequency of all clocks in a system is referenced back to a single physical clock. Under normal operating conditions, all clocks will have the same average frequency in a source-traceable system. Thus, the phase error or Maximum Time Interval Error (MTIE) between all clocks in such a system is bounded. MTIE is a measurement metric that specifies the maximum excursion of a timing signal's phase (commonly called time-delay) compared with a stable reference (see Chapter 6, Section 6.2.6.2, for a precise definition of MTIE). A series of nested measurements are taken over time intervals of increasing length. Each of these measurement intervals or "bins" is then analyzed for the maximum value of time delay. As the measurement interval (commonly called observation time) increases, the maximum value of time delay will either stay the same or increase. The MTIE sampling process is usually limited to variations in the wander region (10 Hz and lower). MTIE is typically presented in graphical form and plotted against observation time. Due the statistical nature of the MTIE measurement, specialized measurement equipment is typically used to do this analysis. Another measurement metric useful in analyzing time delay is Time Deviation (TDEV). TDEV is a statistical calculation of the rms energy of a timing signal's phase noise (see Chapter 6, Section 6.2.6.3, for a precise definition of TDEV). In short, TDEV provides a measure of the instability of a timing signal in the wander region. Measured values of TIE are taken through a bandpass filter that is adjusted for increasing values of integration
Synchronization Architectures for SONET/SDH Systems and Networks
263
time. Due to the statistical nature of TDEV, a measurement period of at least 12 times the maximum integration time is needed. Therefore, a maximum integration interval of 1000 seconds requires at least 12,000 seconds of data. As with the MTIE measurement, specialized equipment is used to perform the TDEV analysis. Examples of source-traceable timing are typically seen at an equipment or NE level. In SONET/SDH LTEs, there is typically a single clock frequency selected on a system basis to be used throughout the system. A dedicated timing distribution system is used to provide source-traceable clocks at a circuit pack, shelf, and bay level. If redundancy is needed, then duplicated clock paths and reference switching components are also used.
Source Clock (A)
Source Clock (B) ^^
\ ""^
^*^
—•
Frequency-Traceable Clocks
MTIE generation is unbounded but controlled
Figure 7-4. Source -traceable frequency relationship
Frequency traceability: As shown in Figure 7-4, frequency traceability is a relationship where the frequency of all clocks in a system are referenced back to clocks of know frequency tolerance. Under normal operating conditions, all clocks will have frequencies bounded by a predetermined range. Thus, the phase error between all clocks in such a system is unbounded but controlled.^ The average rate of phase drift between frequency-traceable clocks, i.e., the frequency offset, can be computed by knowing the average frequency difference between any two clocks. For example, if a fixed frequency offset between two clocks is 1 part per million (PPM) they will drift at a linear rate of 1 |is per second. This relationship is shown in the following formula for time delay [3] related to frequency offset: X PPM offset = X ^is per second drift
(7.1)
264
Chapter 7
Though Eq. (7.1) assumes a constant frequency offset, it is useful in computing time interval error (TIE ) and MTIE. One of the important points that this relationship shows is that the phase drift, or time delay, will increase linearly over time for a constant frequency offset. Therefore, for a known frequency offset, the MTIE generation is controlled but unbounded. Examples of frequency traceability are typically seen in GPS timing distribution networks. In this case, the source clocks are the GPS receivers and co-located BITS. Therefore, each BITS/SSU will be synchronized to a worst-case accuracy of 10"^ ^ Locally at each BITS/SSU, timing is distributed via source-traceable means to ensure that all externally timed equipment receives a timing source that is frequency and phase synchronous.
Source Clock
^reierence
i
Source-traceable feedback path
MTIE generation is unbounded and uncontrolled
Figure 7-5. Source/frequency-traceable relationship of a timing loop
If the traceability characteristics in a synchronization network are not managed properly, then it is possible to create timing loops. Timing loops are caused when the source-traceable nature of the individual clocks is such that their input source is referenced to their output. This feedback arrangement, shown in Figure 7-5, has a destabilizing effect on the output frequency and will cause the phase error to be both unbounded and uncontrolled. Systems experiencing timing loops will typically have output frequencies that oscillate between the extreme frequency limits of the internal oscillators. The conditions leading to excessive phase movement can compromise data recovery and ultimately lead to data loss. It is for this reason that the integrity of data transport requires that source traceability be managed. Source traceability can be managed in several ways depending on the transport facilities that carry timing. If GPS receivers and co-located BITS/SSU are used, then, by definition, the source of the BITS/SSU will not
Synchronization Architectures for SONET/SDH Systems and Networks
265
be referenced to their outputs, and no timing loops will occur. If, however, any line-timing (or derivatives) mechanism is used, then it is possible to create timing loops under some conditions. For example, if both NEs on either side of a transmission facility are using loop-timing mode, a timing loop will be created. Therefore, certain precautions must be taken when designing line-timed networks. In SONET/SDH networks, synchronization status messages can be used to manage the flow of timing in either a linear or ring network. Typically, this messaging is used to allow timing to travel in only one direction in the network. The return path is often blocked by messaging and can be effectively used to control timing flows and avoid creating timing loops. Typically, timing loops form when NEs undergo some sort of dynamic event that causes an NE's timing source to change. Such events could be the result of protection switching due to a network or equipment fault, manual switching caused by a maintenance activity, or a combination of the two. Unfortunately, in these cases, either multiple faults or the removal of various protection mechanisms lead to the creation of timing loops. Even worse, timing loops can occur without alarm or warning that would indicate that the timing flow is at fault. Timing loops can cause transmission impairments that result in bursty transmission errors. Bursty transmission errors are typically created as receive buffers overflow or underflow due to timing variations between the equipment clock and service clock. Thus, the transmission equipment is first suspected as the source of the fault. Sometimes, the BITS/SSU may indicate an end-of-range of its oscillator during a timing loop event. Thus the timing card or oscillator of the BITS/SSU may be initially and incorrectly identified as the source of the timing problem. This will obviously cause delay and expense in rectifying the timing loop situation. Timing loops are typically localized at a single NE but can affect several pieces of equipment located in multiple remote locations in a network. Therefore, good records are needed to verify timing flows and to safeguard against creating a timing loop under fault conditions.
266
7.3.
Chapter 7
SYNCHRONIZATION DISTRIBUTION
Stratum 1
Stratum 3
Figure 7-6. Source -traceable clock distribution
The concept of synchronization distribution for communications systems has its origins in the Bell System. The Bell System used a frequencytraceable timing distribution system based on a single, physical stratum 1 clock located in Hillsboro, Missouri, USA. The frequency from this clock was distributed to various point-to-point nodes in the network with stratum 2 clocks. These stratum 2 clocks, in turn, distributed timing to other stratum 2 or 3 nodes. Finally, the stratum 3 clocks distributed timing to other stratum 3 or 4 nodes. By distributing timing to nodes of an equal or lesser stratum level, the source traceability relationship of the clocks between nodes was preserved. This source-traceable distribution scheme is shown in Figure 7-6.
Synchronization Architectures for SONET/SDH Systems and Networks
267
% %
^
Figure 7-7. SONET/SDH frequency-traceable clock distribution
In today's SONET/SDH networks, it is not always practical to distribute timing in a source-traceable manner. A source-traceable arrangement would require a complex network of point-to-point links to overlay the existing SONET network. Instead, a frequency-traceable means of timing distribution is employed, as illustrated in Figure 7-7. In this case, frequencytraceable timing is distributed by means of the global positioning system (GPS) [3] satellite network.^ Maintained by the U.S. Department of Defense (DOD), the GPS radionavigation system is capable of providing synchronization information to better than 1 part in 10^^ accuracy (i.e., an accuracy of 10'^^). The GPS satellites are placed in six circular orbits 20,200 km (10,900 nautical miles (NM)) above the earth at an inclination angle of 55 degrees with a 12-hour period. The satellites are spaced in orbit so that at any time a minimum of six satellites will be in view to users anywhere in the world. This constellation of GPS satellites transmits its atomic-clock-based timing to earth-based GPS receivers, which lock onto these signals and recover the timing. Due to a number of factors related to the recovery and distribution of these GPS clock signals, no two GPS receivers will have exactly the same frequency and phase. In fact, from a timing relationship, the only thing that is guaranteed is that the frequency accuracy will be within 1 part in 10^^ accuracy of UTC (Coordinated Universal Time). Therefore, over time, it is expected that the phase error between any two GPS receivers will be unbounded. However, knowing that the frequency error between any two GPS receivers will be no worse than 2 parts in 10^\ the maximum drift rate is controlled. This timing relationship is referred to as plesiochronous [4] timing.
Chapter 7
268
Once GPS timing is recovered, it is usually sent to a BITS or SSU, which then distributes this timing to other local equipment. Typically located in a central office, the BITS/SSU provides many levels of duplication and redundancy, including holdover, to preserve the continuity, accuracy, and availability of this timing signal. When timing is sent to a SONET/SDH LTE, it is used as a basis for timing the outgoing synchronous payload envelope (SPE). The timing from the SPE can be recovered by downstream SONET LTEs for use as a linetiming source. SONET/SDH topologies can encompass linear or ring, or a mixture of the two. In all cases, SONET/SDH line-timing can be used while maintaining both source and frequency traceability. A mechanism used to enforce timing flow is the use of synchronization status messaging. This allows timing flows to be directed within a SONET/SDH network.
7.4.
NETWORK ELEMENT (NE) ARCHITECTURE
'^m'y:zr^^^W'
Timing Processing
m ^
Timing Distribution
* ^
Timing Engines
Figure 7-8. Example SONET/SDH Network Element (NE) architecture
A basic SONET/SDH NE may be illustrated as a series of functional blocks that recover, process, and distribute intrasystem timing signals (Figure 7-8). In the design of a timing architecture, there are a number of design factors that will influence the type and placement of timing
Synchronization Architectures for SONET/SDH Systems and Networks
269
functionality. For the purposes of Figure 7-8, a redundant multicircuit card system has been chosen to illustrate timing flows. The two basic functional blocks of this system are timing engines (TEs) and timing distributors (TDs). The TEs perform the bulk of the timing processing on a system level, including clock recovery, filtering, and backup operating mode. The TDs provide a fan-out and synthesis of this timing to the end devices. The placement of these functional blocks is greatly dependent on the system architecture. They can be implemented as stand-alone circuit packs or modules located in a system or can be part of other common equipment (e.g., system controller card). Typically, the TEs are placed in the category of common equipment. In small to medium-sized equipment (single to multishelf systems), there is only a pair of TEs. The TEs are usually configured in a redundant relationship that provides source-traceable timing for the system. The TDs are typically placed on the cards or modules that require the distributed clock to function. In this case, the TDs can be used to distribute, fan-out, and synthesize the system clock to match the timing or functional needs of its circuit pack or module.
7.4.1
Timing Engine (TE) Functions
The timing engine (TE) operation can be broken down into four distinct functions, including reference selection, clock recovery, synchronization retiming, and synchronization distribution. These functions are used to select a system timing reference and perform an intermediate distribution of this reference to the system line-cards or modules. It should be noted that the actual operation of these functions is dependent on system architecture and implementation. For example, small or single card systems will not have the redundancy and cross-coupling needed by larger systems. The qualification of input references and the performance of the clock will be more sophisticated for better stratum-level clocks. Therefore, these functions should be viewed as guidelines that need to be adapted for various architectures. Reference Selection: When a timing signal is input to a NE, it can come from a line recovery process or external means. In either case, the NE system timing must be extracted from this timing signal. In addition, fault detection is typically done at this point to verify the suitability of the input signal as a timing reference. It may also be desirable to support redundancy by providing multiple inputs to a single clock engine. Therefore, the TE must also perform a source selection of all available timing inputs.
270
Chapter 7
Clock Recovery: Next, a distinct clock signal must be produced or extracted from the input reference. At this point, it may be desirable to perform a frequency synthesis of this clock signal to create a common frequency for the synchronization retiming function. It should also be noted that at this point, if a mate clock engine is being used, a cross-coupling of this clock signal can be made. Such an arrangement will ensure that both TEs are using the same exact clock signal. Synchronization Retiming: This process is often associated with a low bandwidth phase locked loop (PLL) process. In essence, the selected clock signal is frequency filtered, phase aligned, and possibly synthesized to create a more stable copy of the input clock. A synchronizing or framing pulse may also be generated here for system frame alignment. Clock mode selection is also done here. Holdover or free-run mode may be enabled in the event of a clock source failure or the need to operate on internal timing. The choice of holdover stability is dependent on the intended service application and will determine the minimum frequency stability during external synchronization reference failures. In addition, cross-coupling may also be done here as well to align the clock and possibly sync pulse outputs. Synchronization Distribution: This process is used to provide the correct voltage level and format to drive these timing signals across backplanes to other elements in the system. The key here is to produce exact copies of the clock signal without adding detrimental jitter or skew. In addition, since all outputs of a duplicated system may require that all clocks have tight phase deviation characteristics (zero delay), a cross-coupling of these output signals will ensure that this requirement is met.
7.4.2
Timing Distributor (TD) Functions
The TD operation can be broken down into two distinct functions: (1) synchronization selection and (2) synthesis and fan-out. These functions are used to transform the system timing reference to meet the actual synchronization requirements of a line-card or module. Again, it should be noted that the actual operation of these functions is dependent on system architecture and implementation. For example, if all line-cards in a system need to have a common and synchronous frame synchronization signal, the TD needs to be able to generate this in a deterministic way. That is to say, the operation of all TDs in the system must be identical to produce framesynchronizing signals that are in frequency and phase. The timing needs of multiservice platforms may require that the TEs output common synchronizing signals to all line-cards. Therefore, it is the
Synchronization Architectures for SONET/SDH Systems and Networks
271
TDs in the line cards that will need to transform these system timing signals to the specific or multiple frequencies needed to support these services at the line-card level. Synchronization Selection: The synchronization selector terminates and recovers the timing signal of each input fed from the TEs. Since redundancy is preferred in multicircuit card systems, the selection between clocks from each TE is performed. In the case where clocks and frame sync signals are sent as a pair from each TE, the sync selector must preserve the phase relationship between these even, under protection switching cases. Synthesis and Fan-out: The synthesis function, typically located on each line card, is essential for creating the various frequencies needed by each line card. This also simplifies the timing distribution from the TE to the TD by only requiring a single set of timing signals to be input to the TD. Finally, these output clocks are fanned out to the specific devices that require input clock signals. Again, synthesized and fanned-out clocks may require tight phase variation characteristics (zero delay), so proper planning of the clock trees on individual modules or circuit packs will be needed. Timing Distributor (TD) Application Example A key element of any system timing application is the TD function. Typically found on most system circuit pack or modules, the TD supports the local timing needs of these subsystem components. With more functionality being placed in ASIC or ASSP devices, it really becomes an issue of supporting the timing needs of these devices. Multiservice platforms are currently the trend in communications equipment. This means that a single circuit-pack or module should be able to support a variety of services or functions. For example, a SONET/SDH mapping module may also need to support Ethernet mapping via GFP [5] (generic framing procedure) structured mode. Such a requirement will necessitate the need for not only SONET/SDH clock frequencies but also those of Ethernet. Figure 7-9 shows an example of how a multiservice timing distributor might be structured.
272
Chapter 7
Working Clock From System Timing Cards
Reference Selector
Protect . Clock Working Sync Protect Sync
From System
•*
^
_^:
Programmable Clock Generation
^ 3 2.048 MHz " ^ ^ 44.736 MHz
Low Speed SONET/SDH Clock Generation
Reference^""*!
3
^ g ;
Controller MM^MMi^^m
> 38.88 MHz
< Z j 77.76 MHz V f
High Speed SONET/SDH Clock Generation
Serial Interface
A
W
Sync Generation
g ^
155.52 MHz
^ ^
622.08 MHz
^ ' "^ 8 kHz Sync
w^ 's^ii^^^'mm^
Examples of possible Output Frequencies
J
^
Figure 7-9, System block architecture of the TSWCO1622
Figure 7-9 shows a high-level block diagram of the TSWCO 1622, a multiservice timing distributor currently available from Agere Systems. This timing distributor is an integrated device capable of operating in redundant system architecture while providing a wide number and variety of timing signals for use in SONET/SDH, ATM, and PDH applications. This device contains all functions necessary for hitless reference switching, frequency synthesis, clock and sync signal generation and fan-out, and communications. The input reference switching function of the TSWCO 1622 is capable of supporting several input clock frequencies, including 51.84, 38.88, 19.44, 8.192, 6.48, 2.048 and 1.544 MHz, and 8 kHz. The choice of input frequency needs to be done on a system level. Typically, if the system timing requires that both a clock and common frame synchronizing signal be distributed, then an 8 kHz signal will be distributed by the TE to all the TDs. When the 8 kHz reference clock frequency is used, the TSWCO 1622 will generate its output 8 kHz sync signal aligned to this input. In this way, all the frame-synchronizing pulses generated by the TDs in a system will be synchronous. If, however, a frame synchronizing pulse is not required to be synchronous throughout the system, then the choice of input frequency becomes the one that most effectively complements the service clocks. It should also be noted that, in order to meet network MTIE and TDEV requirements, the input clock reference to the TSWCO 1622 must be compliant with all requirements for SONET Minimum Clock (SMC) as defined in ANSI T l . 105.09 or for an ITU SDH Equipment Clock (SEC) as defined in ITU-T Recommendation G.813. Operating in a redundant system configuration, TSWCO 1622 can accept two timing inputs of the same frequency. These clock inputs will typically come from the two system TEs. Integral to the input clock reference section
Synchronization Architectures for SONET/SDH Systems and Networks
273
is the clock protection switch and control circuit. It is up to this section to validate input references based on the presence of clock edge transitions. The TSWC01622 can also be optioned to switch from a failed input autonomously and switch back to a restored input via a revertive switching capability. Should both of the two timing references fail, the TSWC01622 can also use a third input reference to provide a free-run-type capability on a line card or module basis in order to maintain a minimum level of service. Should input clock switches occur, the TSWC01622 can perform these hitlessly. A hitless switch is one where various timing relationships are preserved before, during, and after the switch. These ensure that the output clock and sync signals will not impair the service quality or functionality of their associated line card or module. One such relationship is the number of clock cycles between frame-synchronizing pulses. Another relationship is the preservation of clock phase at the output ports. Since timing signals from the TEs can follow different backplane routes, they will commonly have slightly different phases. The TSWC01622 recognizes the clock skew between these input timing references and is able to perform a phase buildout whereby the skew is equalized. Therefore, the resulting output clocks are capable of meeting the MTIE and TDEV switching requirements previously mentioned. The TSWC01622 is able to output a wide number of standard output frequencies and types to support the needs of multiservice platforms, including SONET/SDH/ATM, Packet over SONET, and PDH services. These frequencies are 622.08, 155.52, 77.76, 51.84, 44.736, 38.88, 34.368, 32.768, 24.704, 19.44, 16.384, 8.192, 4.096, 2.43, 2.048, and 1.544 MHz. Since these output frequencies are programmable, the device configuration can be customized for a variety of applications. The programmable outputs use fractional synthesis to provide almost any frequency within the output buffer's range (-100 kHz to 66 MHz). The device also provides dedicated output ports that support CMOS, LVPECL, or LVDS output levels. An example of using the TSWC01622 in a multiservice platform can be seen in Figure 7-10, which shows how a timing distributor can be used to provide the timing needs. The TSWC01622 is the timing distributor for a line card or module that also contains an Agere Systems Ultramapper"^'^ (TMXF8422) device. The Ultramapper is a multiservice device that supports the following services and features: •
SONET/SDH Line Interface with integrated clock/data recovery — terminates 622 Mbit/s STS-12/STM-4 or 155 Mbit/s STS3/STM-l — built-in CDR (clock and data recovery)
Chapter 7
274 Ultramapper
TSWC01622 CK155P0 CK155N0
THSCP THSCN
24.704 or 49.408 MHz CKPDH3
DS1XCLK* 32.768 or 65.536 MHz
CKPDH2
CKPDH1 CK51
P« 44.736 or r i 34.368 MHz
E1XCLK* DS3XCLK
Mux and Fanout Selection
8 kHz (if required)
i 34.368 MHz p 44.735 or 11 34.368 or K 51.84 MHz .
E3XCLK
DS3DATAOUTCLKf3:11 SYNC8K
^^^^S * Full ITU G.783 combined jitter compliance can be acliieved when clock rates fortheDSIXCLKand E1XCLK are set to 49.408 and 65.536 MHz, respectively
Figure 7-10. Timing connections Between Agere Systems TSWC01622 and Ultramapper^^
Overhead processor (TMUX) — for all transport and path overhead bytes — insertion and extraction of overhead bytes — software controlled linear 1+1 protection STS-12/STM-4 Pointer Processor and STS-1 XC — STS/AU pointer processing to align receive frame to system frame — Full intermediate path monitoring for all nonterminated STS1/AU3/TUG3 — Muxes STS-1 into STS-3 or STS-12 via pointer processing SPE/AU-3 Mapper — maps up to six DS3 or E3 clear channel (framed or unframed) signals into STS-1 or TUG3 — supports all valid Tl/El/Jl multiplexing structures into STS1 and STS-3/STM-1 M l 3 Multiplexer / Demultiplexer — three sets of x28 DSl or x21 El or x7 DS2 to/from DS3 or E3 VTA^C Mapper — monitors/terminates VT path overhead for 84 VT1.5/TU11 or 63 VT2/TU12 — Asynch and Byte synch VT mapping Digital Jitter Attenuator (DJA)
Synchronization Architectures for SONET/SDH Systems and Networks
•
275
— PLL free operation using built in DJA in VTA^C or M l 3 mode T l / E l , DS2, DS3 Cross Connect (xC) x84/x63 Tl/El/Jl Framer Bank — supports T l , E l , and Jl framing modes — supports Tl and El unframed and transparent transmission formats — Facility data link supports SLC96, ESF, and CEPT — HDLC- 192 channels with a maximum rate of 64 kbits; 128byte FIFO per channel
In order for the Ultramapper to support the tributary mapping of all these various rates into a SONET/SDH payload, numerous clocks at precise frequencies must be generated from a common system source. This is where the flexibility and versatility of a TD designed to support such a multiservice device proves to be advantageous. Using a single TSWC01622, the eight clock frequencies and one frame sync are able to be generated from a single redundant pair of timing inputs. Even more useful is the ability of the TSWC01622 to provide hitless protection switching using autonomous or manual switching modes.
7.4.3
Network Element System Architecture
Figure 7-11 shows how a typical SONET/SDH NE could be structured. Multicard equipment is typically composed of various classes of functionality, requiring specific clock types. Such functional classes include common equipment cards that can provide power, synchronization, and control functions for the system and modules or line cards that provide access to transport signals. The choice of what is placed on each card is determined by the amount of circuitry required by each function, how much power it requires, or how much heat it dissipates. Timing engine functions typically use physically large oscillators and will require some accommodation for this size. Modules or line cards typically comprise the largest functional type on a system level. The synchronization needs of each line card are dependent on the type of interface and functions that must be performed. It is for this reason that a system needs to be designed to support the end-service timing requirements. This not only includes providing the correct frequency but also includes availability and quality of the timing signal, even in the presence of switching, fault, or maintenance events.
Chapter 7
276
8ITS/SSU rnmJng Source
Timing Distributor On every Module or Line Card
Figure 7-11. Example SONET/SDH NE clock routing
Another important aspect of the NE synchronization system is that it needs to be scalable. Many platforms tend to evolve or be repackaged to address various markets. Scalability covers the need for many changes or derivatives of a system, including the need for higher operating capacity, higher I/O capacity, and simplex or duplex operation. It will be shown that by using a few basic synchronization building blocks, a scalable architecture can be easily realized. Such an architecture will not only scale to provide appropriate service performance but also address various cost targets that are equally important.
7.4.4
Small Network Element Architecture
Figure 7-12 is an example of a scalable small NE architecture. The physical requirements of this system comprise a single board or chassis. Here, equipment redundancy is not important. So, in this case, the simplex architecture does not have the need for redundant timing circuitry. This contributes to the low cost and low capacity that makes this architecture desirable. Essentially, the timing needs for this architecture rely on the support of just a single TE. The input source to the timing engine can come from either an external source or from the line. In the case of a clock source failure, either a holdover or free-run mode would be sufficient for operation. If desired, a redundant timing input can be supported that will give the small NE a measure of fault tolerance without hardware redundancy.
Synchronization Architectures for SONET/SDH Systems and Networks
211
However, due to the low cost/performance nature of this design, fauh tolerance is not expected.
Figure 7-12. Scalability example of a small NE
7.4.5
Medium Network Element Architecture
Figure 7-13 is an example of a scalable medium NE architecture. The physical requirements of this system comprise a single shelf with multiple circuit cards or modules. In this case, due to multiple cards, redundancy and fault tolerance are part of the architectural requirements. Therefore, a duplex architecture comprised of a pair of timing engines and a timing distributor on each line card would be provided. The cross-coupled TEs are capable of external or line-timing options. Line timing inputs typically come from any of the active line cards. In this case, the SONET/SDH line-terminating processor would provide the SSM associated with each line-timing source as part of the selection process. Every line card has a TD that supports the efficient transport of timing information from the TEs. The TD will typically accept a single clock or frame sync signal from each TE. The TD then perform hitless source selection between each TE as well as controlling any phase shifts by using phase build-out where needed.
Chapter 7
278
in BITS/SSy ~ withGPS/
'"'-U' v,v"> i-r-i
^-v'^^i\'Q^' TE - System Timing Cards
TD - Line Card/IVIodule Timing
Figure 7-13. Scalability example of a medium NE
7.4.6
Large Network Element Architecture
Figure 7-14 is an example of a scalable large NE architecture. The physical requirements of this system comprise a multibay and or multishelf system with multiple circuit cards. Since multiple timing faults could potentially impact a large number of users, redundancy and fault management are integrated into this architecture. The NE system timing is controlled by a pair of cross-coupled TEs located in the common equipment shelves. Here, the TEs will select between external, line-timed, or internal timing options. Cross-coupling will ensure that signals between the pair of TEs are coordinated. As before, the line-timed inputs and associated SSMs will come from active line-cards. Due to the extensive nature of the architecture, it may be advisable to have an intermediate stage of line source selection on a per-shelf or per-bay basis. This intermediate TD function will allow a limited number of line timing sources to be switched to each TE. In this way, all possible line timed sources can be made available to each TE with a minimum amount of cabling. Likewise, the timing signals (clock and frame sync signals) from the TEs are then sent to an intermediate pair of TDs located on each shelf. These
Synchronization Architectures for SONET/SDH Systems and Networks
279
TDs provide a fan-out to all the line cards on the shelf — a process that reduces the number of output ports that a TE would need. Finally, the intermediate TDs output the set of clock and frame sync signals to the TDs located on each line card. As before, each TD will be able to perform hitless source selection between each set of TE inputs as well as controlling any phase shifts by using phase build-out where needed. When using external timing mode, there are a number of strategies that can be used to provide this clock to the SONET NE. Each of these methods provides a different set of advantages that favor various local synchronization distribution designs.
i<&T^§^^'^'-^^'^-^''^p'^^yyi?z^^^
^>„;.v TE - System Timing Cards
TD - Shelf/Bay Timing
TD - Line Card/Module Timing
Figure 7-14. Scalability example of a large NE
7.5.
EXTERNAL TIMING CONFIGURATIONS
External timing configurations involve providing SONET/SDH equipment with external timing references from a BITS or SSU. There are a number of techniques that can be used, each with a set of advantages or
Chapter 7
280
applications that makes it suitable in some but not all circumstances. The suitability of any one method is dependent on the SONET/SDH equipment architecture, office cabling distribution, and placement of equipment in an office.
7.5.1
Direct-Source Timing Method
The direct-source timing method involves cabling a single BITS/SSU clock output directly to a single timing input port (Figure 7-15). This oneto-one mapping requires that, for every NE clock input, a unique BITS/SSU output will be used. For NEs that support multiple BITS/SSU terminations, this method affords the highest redundancy, since the fault of any one output will not affect any of the other outputs. The direct-source arrangement of BITS/SSU clock taps not only contributes to a higher BITS/SSU output density but also require the highest number of cable runs between the BITS and NE. If DSl signals are used, then there is typically a 655-foot cabling limitation from the BITS to the NE. This limitation is based on the maximum amplitude of the DSl at the DSX-1 and a nominal DSl termination impedance (at the NE) of 100 ohms. If DSO signals are used, they have a 1,500 foot [6] cabling limitation from the BITS to the NE. This distance limitation is based on the phase variation from NE to NE when supporting native DSO services using composite clock (CC) external synchronization.
NE1
BITS/SSU with GPS
Timing DS1 Outputs
NE2
Figure 7-15. Direct-source timing method
Synchronization Architectures for SONET/SDH Systems and Networks
7.5.2
281
Bridged-Source Timing Method
Bridged-sourced timing method essentially allows a single BITS output to be shared between two input ports of an NE. Typically used to feed redundant NE timing inputs, bridging provides an exact frequency and phase copy of the BITS clock at an amplitude of 20 dB [7] down from the DSX-1 level.
432 Q Bridging Resistors (IVIay be part of timing pacl<)
Bridging Amplifiers
100 Ohm Termination (IVIay be part of NE baclcplane)
BIIS/SSU wJthOt»S
Timing DS1 Output '
Figure 7-16. The Bridged -source timing method
An example of bridging is illustrated in Figure 7-16. Here, a DSl source is terminated at the NE with a 100 ohm resistive load. This characteristic impedance ensures that the DSl waveshape will be compliant at the end of the cable run. From here, high impedance TE inputs may be placed across this single load. In the case of a 1:2 split, resistive bridging can be used. Figure 7-16 shows how a bridging resistor arrangement would be used to split the input DSl timing signal between two external inputs of a single NE. A pair of 432 ohm resistors is placed in series with the tip and ring sides of the 100 ohm resistor and connected to each external input. The input impedance of each external input is assumed to be 100 ohms resistive. Approximately 20 dB amplification is used to boost the input DSl signal to the level of an unbridged and terminated DSl. A resistive bridging circuit consisting of a pair of 432 ohm resistors and the transformer termination (assumed to be 100 ohms resistive) presents a 964-ohm load to the parallel 100 ohm termination. This yields a total load of
282
Chapter 7
91 ohms at the end of the cable from the BITS/SSU. Two such bridging inputs with a common parallel 100 ohm termination (shown in Figure 7-16) will yield a reduction in the termination impedance to 83 ohms. Though this dual-bridged termination will result in decreased amplitude at each NE input, the bridging amplifiers can be designed to account for this and boost the bridged DSl signal to the proper level. The advantage of using a resistive bridging circuit is that it is backward compatible with the direct-source timing mode inputs. In other words, a system designed for direct-source timing mode can be modified/upgraded to use bridged-source timing mode. This is done by adding external bridging resistors and modifying the TE circuit to include bridging amplifiers. Another advantage of using the bridging circuit on both DSl inputs is that it allows one of the timing packs to be removed without disturbing the input timing signal to the mate pack.
7.5.3
Line/External Timing Method
Line/external timing essentially allows one or more NEs to provide synchronization information that is carried over a SONET/SDH facility to a co-located BITS. Since BITS/SSUs are unable to directly extract timing information from a SONET/SDH facility, a line-terminating SONET/SDH NE is needed [8] In this way, timing may be recovered directly from the SONET/SDH frame and used to generate a derived DSl signal that is used to externally time the BITS/SSU. The BITS/SSU then uses this timing reference to generate outputs that are fed back to the sourcing NE and other co-located NEs. Lastly, the SONET/SDH LTEs use this timing to process and transmit outgoing SONET/SDH frames. The derived DSl signal is created by the SONET/SDH LTE expressly for the purpose of transporting synchronization information. This signal is formatted as an all ones with either a super-frame (SF) or extended superframe (ESF) format. The use of DSl framing is also used to allow signaling between the SONET/SDH NE and BITS/SSU. In the case where SONET/SDH synchronization status messaging is used, it is necessary to provide this information to the BITS/SSU to influence its ability to either switch between external input references or enter a backup mode of operation (holdover). In essence, this allows the SI byte message to be transported to the BITS/SSU without the need for a direct SONET/SDH connection. Figure 7-17 illustrates the configuration used for the Line/External timing method.
Synchronization Architectures for SONET/SDH Systems and Networks
283
mm BITS/SSU
Timing DS1
-©
z.
5r NE
Figure 7-17. Line/external timing method
When the derived DSl uses the SF format, the SONET/SDH NE communicates with the BITS/SSU by changing the format of the derived DSl to a blue signal format (all ones without framing). When the BITS/SSU receives a blue signal on an external timing reference, it will stop using it as a reference (if currently active) and switch to a valid reference. The use of this method of signaling is commonly called threshold AIS generation [9]. A SONET/SDH NE will generate a blue signal on an SF-formatted derived DSl under the following conditions: •
Input SSM "Do not use for synchronization (DUS)" — whenever an incoming SONET/SDH SI byte is set to DUS, it is not used to synchronize the processing or transmission functions of the NE. Failure to comply will commonly result in a timing loop that will lead to the impairment of data transmission. Therefore, under no circumstances should a timing source bearing the DUS message be used for NE or network timing.
•
The input SSM is equal to or worse than a preset threshold value as set on the SONET/SDH NE — In this case, the SONET/SDH NE may have the ability to compare the incoming SSM with a preprovisioned value or threshold. When the incoming SSM is equal to or better than the threshold value, the SONET/SDH NE will
Chapter 7
284
generate a SF-formatted derived DSl. When the incoming SSM is worse than the threshold value, the SONET/SDH NE will generate a blue signal. In this way, the SONET/SDH NE will ensure that the co-located BITS only receive a timing source of equal or better stratum level. Table 7-L Translation between SI and ESF SSMs 1 1 Quality 1 Level
1
1 ^
First-Generation SSMs Description DSl ESF Data Link Code Word Stratum 1
00000100
Traceable
11111111
Synchronized
00001000
Traceability
11111111
SI Bits (bSb8)
Quality Level
0001
1
0000
2
Unknown
1 ^ 1 ^ 1 ^
Second-Generation SSMs Description DSl ESF Data Link Code Word Stratum 1
00000100
Traceable
11111111
Synchronized
00001000
Traceability
11111111
|
SI 1 Bits (b5b8) 0001
0000 1
Unknown
Stratum 2
00001100
Traceable
11111111
Stratum 3
00010000
Traceable
11111111
SONET
00100010
Minimum Clock
11111111
0111
3 4
1010
Stratum 2
00001100
Traceable
11111111
TNC Traceable
01111100
0111 1 0100 1
11111111 1100
5
Stratum 3E
01111000
Traceable
11111111
1101 1
Traceable
1 ^ 1 ^
Stratum 4
00101000
Traceable
11111111
Don't Use for
00110000
Synchronization
11111111
Provisionable
01000000
N/A
j
1111
6
7
Stratum 3
00010000
Traceable
11111111
SONET
00100010
Minimum Clock
11111111
1010 1 1100
Traceable 1
User
1 Assign1
1110
8
11111111
Stratum 4
00101000
Traceable
11111111
Don't Use for
00110000
Synchronization
11111111
Provisionable
01000000
N/A
able 9 User Assign1
1111 1 1110 1
11111111
able
When the derived DSl uses an ESF format, the SONET/SDH NE communicates with the BITS/SSU by using a set of selected code words in
Synchronization Architectures for SONET/SDH Systems and Networks
285
the ESF data link. These code words carry the SI byte information from the SONET/SDH hne overhead, in essence providing a translation of the SSMs between the SONET/SDH and PDH formats. Table 7-1 shows the translation between the SI and ESF SSMs for each generation.
7.5.4
Mult Timing Metiiod
This method allows multiple NEs to be externally timed from a pair of BITS/SSU timing references. This inter-NE timing is performed by daisychaining the external BITS/SSU source between the NEs. As shown in Figure 7-18, each DSl/El timing signal is daisy-chained independently between the TEs. An advantage to using the mult timing method is that it relieves the number of timing outputs needed at the BITS/SSU. It also greatly reduces the amount of cabling from the BITS, thereby contributing to an overall cost savings.
External Timing DS1/E1
Figure 7-18. Mult timing method
As with any daisy-chained configuration, there are susceptibilities that must be addressed. If the daisy-chain is broken, it may affect other NEs in the same chain. It is for this reason that the maximum distance between daisy-chained equipment must be kept relatively short, typically spanning a single bay or adjacent bays. The other concern is the ability to maintain
286
Chapter 7
signal continuity of an intermediate NE that is undergoing maintenance or is being taken out-of-service. It is for this reason that a passive means of daisy-chaining is preferred over an active implementation. For example, will signal continuity be preserved if both the TE packs in an intermediate NE are physically removed? If the answer is no, then all subsequent NEs in the chain will lose their external input and enter a backup mode of timing. There are several ways to maintain signal continuity in the event of a pack pull. Two common methods that provide continuity of the daisy-chain to downstream equipment use either backplane shorting connectors or backplane mounted relays. In addition to providing signal continuity, there is also the transient effect of inserting or pulling a pack. Therefore, additional procedures may be needed to address this. For example, if it is necessary to remove an active TE in a daisy-chained NE, all downstream NEs need to select timing from the standby TE. This selection may be done by forcing all NEs to use the same BITS/SSU reference as the standby TE. Since this represents a manual reference switch, the transient performance of each NE's system clock should be deterministic and controlled. Once all the standby TEs are stable, all NEs will be forced to use the standby reference (now active). Then the intended TE may be removed with no impact on the NE or other NEs in the daisychain.
7.6.
CLOCK BACKUP MODES AND IMPLICATIONS
Clock backup modes of operation are typically used to maintain a system timing function in the presence of an external timing distribution fault. Such faults can cause source traceability to an external reference to be interrupted for a few seconds, several hours, or days. Since timing distribution is all about maintaining source traceability to an external reference, its loss can compromise the integrity of the underlying service. Clock backup modes of operation fall into two basic categories: holdover and free-run. Holdover Mode: This mode is typically used to maintain accurate timing for external timing distribution faults of relatively short duration up to 24 hours. The limitation of 24 hours is not based on the functional aspects of the holdover circuit; rather, the technical standards that define holdover only specify its performance for the first 24 hours after a fault. Depending on implementation and environmental conditions, it is possible for a holdover circuit to maintain proper holdover performance for much longer than 24 hours.
Synchronization Architectures for SONET/SDH Systems and Networks
287
In holdover mode, the TE essentially stores the frequency and phase value of the last good external clock reference prior to the external timing fault. These values are then used as a basis to steer the timing reference located in the TE for the duration of the holdover event. In theory this sounds easy, but in practice there are a number of rather significant issues associated with correctly computing and storing the frequency and phase values. The significance of this will be evident when we look at the four phases of holdover mode: 1. Holdover mode initiation: Initiation involves a system command to the TE that holdover mode is required. The initiation command can be as simple as a manual command from a management port, or as part of a manual timing reference switch. More server causes of holdover initiation may be due to detected system faults. Some of these faults can be as simple as a loss of signal (LOS) or frame (LOF) of the incoming timing reference. If SONET/SDH line timing is used, a change in the SSM can trigger a holdover event if the threshold is crossed or a DUS message is received. Fault events can also be caused by an upstream reference walking off in-frequency. An off-frequency input reference condition is more complicated to detect due to the need for a precision frequency standard. Yet there are implementation-specific methods that can be used to accomplish this comparison without the need for such a standard. If crosscoupling of input references is used, then multiple standby references (or an ensemble of references) can be monitored and compared with the active reference. A majority voting scheme can be used to determine if a significant frequency difference exists between the active and the ensemble references, which can then trigger the appropriate fault detection and corrective action. 2. Entry into holdover mode: Once a fault condition of the timing reference has been detected, it must be categorized and the appropriate action must be taken. Though holdover entry is used in some systems to guard against a multifault condition, some systems employ short-term holdover as part of the regular switching scheme. The reasoning behind a short-term holdover event is to allow sufficient time to qualify the new input reference while not relying on the previous external reference. In either case, entry into holdover requires the successful execution of a number of simultaneous processes.
288
Chapter 7 a. Calculation of holdover value: The calculation of the holdover value is based on an average of the previous input frequency/phase information relative to the external reference. The averaging time can range from a few seconds to several hours. The averaging time and its benefit to overall system performance are left as an implementation detail. b. Phase offset computation: The phase offset value is the difference between the actual TE clock and the holdover value. Since the holdover value represents an average value, it will most likely be different from the actual frequency and phase of the TE clock. If an instantaneous switch between the external reference and the TE clock occurs, a phase discontinuity at the TE clock may result. Such discontinuities may violate the system timing specifications (MTIE, TDEV, frequency stability). Therefore, the computation of a static phase offset may be necessary to allow a stepless phase change during the actual holdover switch. The other alternative is to perform a gradual phase change from the current TE clock phase to that of the computed holdover value. c. Holdover mode switch: This switch involves transferring frequency and phase control from an external frequency process to internal means. Again, due to the need to avoid phase discontinuities per industry standards, an instantaneous switch cannot be performed. Rather, the switch must allow for either a static phase build-out, or a gradual change in frequency and phase to the desired holdover values. Should a phase transient occur during entry into a stratum 3/3E holdover, it must occur within the first 64 seconds of holdover [10]. The magnitude of this transient at a DSl interface is dependent on observation time, as shown in Table 7-2. For stratum 2 holdover entry, such a transient must occur within the first 5000 seconds of holdover [10]. The magnitude of this transient at a DSl interface is dependent on observation time, as shown in Table 7-3. Again, the performance of this switch is implementation specific.
Synchronization Architectures for SONET/SDH Systems and Networks
289
Table 7-2. Stratum 3/3E holdover transient vs. S at DSl interfaces Observation Time, S (Seconds) S< 0.001326 0.001326<S<0.0164 0.0164<S<564
1
64 <S
MTIE (nanoseconds) No specification 61000 X S 1000 + 50 X S No specification
Table 7-3. Stratum 2 holdover transient vs. S at DSl interfaces
1
Observation Time, S (Seconds) S< 0.014 0.014 <S<0.16
MTIE (nanoseconds)
0.16<S<5000
150 + .lxS
No specification 7.6 + 885 X S
1
3. Holdover mode maintenance: Once in holdover mode, all frequency and phase correction must be done by the TE. Depending on the stability of the holdover clocks (e.g., stratum level requirement per ANSI or ITU-T), various means may be used to regulate frequency drift and MTIE generation. Shortterm frequency stability of oscillators is generally influenced by a combination of various factors, including voltage regulation, oscillator temperature stability, shock, and vibration. Table 7-4 [10] summarizes the performance of stratum 3/3E holdover, including the initial frequency offset, frequency drift rate, and fractional frequency offset due to temperature variations. Table 7-4. Summary of Stratum 3/3E holdover performance Initial Frequency Offset Frequency Drift Rate Fractional Frequency Offset Due to Temp. 1 Variations
Stratum 3 0.05 ppm 4.63 X lO'^ppm/sec 0.3 ppm
Stratum 3E 1 0.001 ppm 1.16 X 10"^ppm/sec 0.01 ppm
290
Chapter 7 As can be seen in Table 7-4, oscillator temperature variations are typically the most important parameter to control. Influenced by ambient temperature variations, high-stability oscillators are able to compensate for these effects on their temperature stability by various methods. a. Ovenization: The basic premise here is to put the temperature-sensitive components in a miniature, insulated chamber that is heated to a constant temperature. In this way, ambient temperature influence on the oscillator is greatly minimized. Oscillators using ovens typically have an "O" as a prefix designation (OCXO — oven-controlled crystal oscillator; OCVCXO — oven-controlled voltage-controlled crystal oscillator). There are various implementations of this using either single or dual oven chambers. Ovenized oscillators are typically able to achieve stratum 3E or better performance. However, it should be noted that the power requirements and physical size of an ovenized oscillator will be greater than a nonovenized version. b. Temperature Compensation: The basic operation of temperature compensation schemes is to characterize the operation of the oscillator over a temperature range, and then employ a scheme that compensates the oscillator's performance when subjected to that range. Oscillators using this scheme typically have a "T" as a prefix designation (e.g., TCXO — temperature-compensated crystal oscillator; TCVCXO — temperaturecompensated voltage-controlled crystal oscillator). This method typically involves characterizing the temperature/frequency curve of each oscillator and storing this data as a lookup table in each oscillator. The actual temperature of the oscillator is then measured and the output frequency of the oscillator adjusted in real time to nullify the effects of temperature or temperature change. The performance of this scheme is dependent on the accuracy of every intermediate function: lookup table, temperature measurement, and frequency compensation. As such, temperature compensation is typically found on stratum 3 and lower stability oscillators.
Synchronization Architectures for SONET/SDH Systems and Networks
291
4. Exit from holdover mode: This approach involves a system command to the TE to end holdover mode and use an external timing reference. The initiation of this command can be as simple as a manual command from a management port, or as part of a manual timing reference switch. Typically, exiting holdover mode should only be done if it is certain that the cause for holdover has been resolved. Since holdover mode requires a known good reference to establish frequency stability, it is imperative that the external timing source be good before switching from this mode. There are several steps that are typically taken before leaving holdover mode. These steps are implementation specific and are presented as a guide to what might logically be needed. a. Qualification of input signal: As previously stated, the state and stability of the external reference must be established before exiting from holdover mode. Requalification of a failed reference can be done for two cases: loss of timing input, and off-frequency events. For the case of a lost timing input, requalification times of the input are on the order of 10 to 30 seconds [11]. However, for off-frequency events, significantly more time may be required. Qualification times in the order of 100 seconds for stratum 3e to 4, and up to 600 seconds for stratum 2, are allowed. The reason for the extended qualification times is to allow sufficient duration for frequency measurement techniques to verify the stability and/or accuracy of the external reference. b. Calculation of phase offset: Due to the strict adherence to MTIE requirements, all switching events need to be kept to less than 150 ns for stratum 2 and less than 1000 ns for stratum 3e to 4e. In order to accomplish this, phase build-out capability may be needed. More than likely, the phase between the newly qualified input reference and that of the TE holdover clock will not be in phase. Therefore, a measurement of the phase difference needs to be done and the offset calculated. Generally, it is not required that stratum 3 to 4 clocks perform a phase build-out during reference switching [12].
292
7.7.
Chapter 7 c.
Input reference switch: The switching involves transferring the frequency and phase control from the internal control of holdover to an external process. As before, due to the need to avoid phase discontinuities per industry standards, an instantaneous switch cannot be performed. Rather, the switch must either allow for a static phase build-out (using the previously calculated value) or rely on a gradual change in frequency and phase to reflect the external timing reference.
d.
Lock-on time stabilization: This is the time it takes the TE synchronization retiming process to fully reflect the external reference. If a PLL is used by the TE as a retiming mechanism, a transient overshoot of the output frequency may occur due to the loop acquiring lock. The duration and amplitude of this overshoot is influenced by a number of system control parameters. However, a fixed set of control-loop parameters may not provide optimal performance for both quickly acquiring lock and providing a sufficient level of filtering during locked mode. Therefore, a possible implementation strategy is to have one set of control-loop parameters to optimize lock-on time and then another to optimize the normal "locked" mode. Clock fast mode [13] is the term used to describe the condition where the lock-on time parameters are optimized.
SYNCHRONIZATION GUIDELINES
Many of the concepts presented in this chapter are intended to provide guidance when using synchronization in SONET/SDH equipment or managing synchronization in networks. The point to stress is that there is no one best way to accomplish this; rather, there are a series of choices that both the designer and end-user must make in order to achieve a certain level of performance or a minimum level of availability versus the cost of the implementation or management. Thus it becomes a matter of balance between these variables to arrive at a suitable solution. Source/frequency traceability: This is fundamentally the most important synchronization concept. At a minimum, the origin of synchronization signals needs to be either known or managed such that
Synchronization Architectures for SONET/SDH Systems and Networks
293
timing loops cannot occur. Typical strategies involve locating a GPS/BITS or SSU with the network equipment and then using external timing to synchronize the local equipment. The use of synchronization status messaging can provide limited network management of a preferred direction of timing. In this way, each SONET/SDH LTE knows the quality level of the timing source being used by its neighbor. This knowledge is sufficient to allow a quick determination of the suitability of an incoming line to be used as a timing reference. It should also be noted that the majority of issues involving source traceability typically happen as a result of either a faulted condition or the actions taken to correct faulted conditions. Therefore, the administration and planning of synchronization networks must include all possible failure scenarios. In some cases, if the network topology is not always known, it may be better to enter a back-up mode of operation vs. trying to switch to another available line reference. Synchronization redundancy and availability: This is a key concept that is usually made on the basis of service type and implementation cost. Functionality that supports these concepts is usually tied to architecture. Typically, redundancy and availability are of great importance for large systems or systems that have a large traffic throughput. In these cases, the failure or interruption of network timing will have a significant network impact and ultimately lead to service disruption. Therefore, the use of redundant elements in network equipment will lessen the probability that a single point of failure will cause a traffic-impacting event. In addition, the use of a cross-coupled architecture will enhance the ability to seamlessly switch between redundant synchronization functions in a network element. Such seamless switching may be useful not only during an equipment or network fault but also in verifying that the standby or redundant equipment sides are functional and will be available when needed.
7.8. NOTES 1.
2.
In addition, two clocks that have exactly the same frequency can have unbounded phase difference if a random walk phase noise (White Frequency Modulation (WFM)) process is added to the phase of one of the clocks. Note that a stratum 1 clock need not be implemented using GPS, e.g., an implementation can be based on a Cesium clock without using GPS. However, GPS-based solutions are practical and cost effective.
294
Chapter 7
7.9. REFERENCES [I] [2] [3] [4]
[5] [6] [7] [8]
[9]
[10] [II] [12] [13]
ITU-T Recommendation G.812, Timing requirements of slave clocks suitable for use as node clocks in synchronization networks, Section 12, June 1998. ITU-T Recommendation G.704, Synchronous frame structures used at 1544, 6312, 2048, 8448 and 44 736 kbit/s hierarchical levels. Section 2.3.4, October 1998. ANSI Tl.101-1999, Synchronization Interface Standard —Section 4 Definitions, 1990. M. A. Lombardi, L. Nelson, A. N. Novick, V. Zhang, (TIME AND FREQUENCY DIVISION - 847), "Time and Frequency Measurements Using the Global Positioning System (GPS)," Measurement Science Conference, January 2001, Anaheim, CA January 01, 2001. ITU-T Recommendation G.7041A^.1303, Generic framing procedure (GFP), December 2001. Telcordia GR-378-CORE, Generic Requirements for Timing Signal Generators, Section 2.1 Synchronization, Issue 2, February 1999. Telcordia GR-378-CORE, Generic Requirements for Timing Signal Generators, Section 4 — Input Criteria, Issue 2 February, 1999. Telcordia GR-253-CORE, Synchronous Optical Network (SONET) Transport Systems: Common Generic Criteria, Section 5.4.5.1 — Timing Distribution on Derived DSl Signals, Issue 3, September 2000. Telcordia GR-253-CORE, Synchronous Optical Network (SONET) Transport Systems: Common Generic Criteria, Section 5.4.5.2.2 — Message Translation, Issue 3, September 2000. ANSI Tl .101-1999, Synchronization Interface Standard, Section 8.2.2 — Stratum 3/3E clock, 1999. Telcordia GR-1244-CORE, Clocks for the Synchronized Network: Common Generic Criteria, Section 3.7 — Reference Validation Times, Issue 2, December 2000. Telcordia GR-1244-CORE - Clocks for the Synchronized Network: Common Generic Criteria, Section 5.7 — Phase Build-out, Issue 2, December 2000. ANSI Tl.101-1999, Synchronization Interface Standard - Section 4 Definitions, 1999.
Chapter 8 NETWORK SURVIVABILITY
To the memory of my wife, Lynda Ghani Abbas Marconi Communications Ltd.
8.1.
INTRODUCTION
Network Survivability is a term that refers to the ability of the network to maintain an acceptable level of service during a network or equipment failure or traffic signal degradation. There are several survivability mechanisms covering a wide range of network architectures, technologies, and allocation of network resources. In the last ten years, the ITU-T has developed a number of recommendations for generic and technology-specific protection schemes. This chapter will review these recommendations and their applications in the network using the ITU-T terminology.
8.2.
NETWORK SURVIVABILITY TECHNIQUES
Generally, there are two types of network survivability techniques: a. Network Protection: This is the replacement of a failed or degraded working resource with a preassigned standby resource. Protection mechanisms tend to be deterministic in nature. When a failure occurs, what will happen is fairly easy to predict and cater for. Generally, the protection action is completed in tens of milliseconds. Protection mechanisms are autonomous and operated independently
296
Chapter 8 from the control or management planes. There are several protection schemes of various architectures and for various technologies. Most of these schemes are defined in the following ITU-T recommendations: • Rec. G.808.1 [1], Generic Protection Switching—linear trail and subnetwork protection, • Rec. G.873.1 [2], Optical Transport Networks (OTN)—Linear protection, • Rec. G.841 [3], Types and characteristics of SDH network protection architectures, • Rec. 1.630 [4], ATM protection switching, and • Rec. Y.1720 [5], Protection switching for MPLS networks.
b. Network Restoration: This is the replacement of a failed or degraded working resource by rerouting using dynamically allocated spare resources. Restoration is nondeterministic, and the network behavior in failure conditions is less predictable. The restoration action may take seconds to complete depending on the amount of traffic being restored. Restoration requires control plane actions. There are a number of ITU recommendations defining control plane architectures, signaling, and routing. These are as follows: • • • • •
8.3.
Rec. G.807 [6], Requirements for automatic switched transport networks (ASTN) Rec. G.8080 [7], Automatic switched optical networks (ASON) Rec. G.7713 [8] series. Distributed call and connection management Rec. G.7714 [9] series. Automatic discovery techniques Rec. G.7715 [10] series. Routing in ASON
SURVIVABILITY OFFERED BY PROTECTION
ITU Recommendation G.808.1 provides an overview of generic aspects of linear protection switching and covers protection schemes that are applicable to the Synchronous Digital Hierarchy (SDH), Optical Transport Networks (OTN), and ATM. In this section, various aspects of protection switching will be addressed.
Network Survivability
8.3.1
297
Network Objectives
Generally, the following network objectives are considered important and should be met for any protection switching schemes: a. Switching time. This is the time taken to initiate and complete the protection switching event. It excludes fault detection time and any hold-off time. A value of 50 ms has been standardized. However, this value may be difficult to meet under all network failure scenarios. For example, for certain applications, such as ring protection where the ring circumference is large, it may not be met. b. Transmission delay. The physical length of the fiber and the processing time of the protection protocol along the path determine the transmission delay. Generally, the transmission delay on 1000 km is approximately 5 ms, and processing delay is in the range of hundreds of microseconds. However, in order to meet the protection switching time, it is necessary to impose a certain limit on the transmission delay. c. Hold-off time. This is a useful parameter for limiting the protection switching actions in a nested protection in multilayer multitechnology transport networks. In nested protection switching schemes, it is necessary to impose a hold-off time on the outer protection so that the inner protection switching can restore the traffic before the outer protection is activated. A standardized holdoff timer, provisionable from 0-10 seconds in steps of 100 ms for SDH and OTN and 500 ms for ATM, is mostly used. d. Switch Initiation. Transmission and connectivity defects are used as criteria for initiation protection switching. These defects may be high error rate, loss of signal, or path trace mismatch. Generally, a threshold is set for transmission defects.. For example, a threshold of an error rate, say lE-8, may be set on the working path, and once this level is crossed, a protection switch is triggered for selection of the protection path. Similarly, for a connectivity defect —a path trace mismatch defect, for example— after a persistency check, the destination port triggers the protection switch for the selection of the protection path.
8.3.2
Protection Switching Architectures
Basically, there are five types of architectures: 1+1, l:n, m:n, (1:1)" and rings. The first four types are for linear protection. Each type has its own advantages and disadvantages as detailed below.
298
Chapter 8
8.3.2.1
1+1 Protection Architecture
The 1 + 1 architecture is the least complex type of protection, where the protection channel is dedicated as a backup facility to the working channel. The normal traffic signal is bridged onto the protection channel at the source end-point of the protection domain, as shown in Figure 8-1. The normal traffic signal is transmitted simultaneously on both the working and protection channels to the sink endpoint of the protection domain. A selection between the working and protection signals at the sink end is made on the basis of a predetermined criterion, such as loss of signal or signal degrade. Permane Bridge
Normal Operation
Working Selector
X ^
j
\ / \ J
j ^
J
\ j
•
Normal Operation
Selector
Protection Figure 8-1. 1+1 protection architecture
The main disadvantage of this type of protection is the 100% extra capacity needed in the network. 8.3.2.2
1 :n Protection Architecture
The 1 :n architecture is a shared type of protection where one dedicated protection channel is shared as a backup facility by n working channels, as shown in Figure 8-2. When the normal traffic signal on the working channel is impaired due to transmission or connectivity defects, the normal traffic signal is transferred to the protection channel at both the source and sink endpoints of the protected domain. When more than one working channel is impaired, only one normal traffic signal can be protected at any one time.
Network Survivability
299
The main advantage of this architecture is that the protection channel can be used to transport extra traffic when the protection channel is not used. However, this architecture tends to be complex, and it does not support dual node interconnect for protected subnetworks.
Normal (1)
Selectors
Bridges
Bridges
AIS - Alarm Indication Signal FDI - Forward Defect Indication Figure 8-2. l:n protection architecture
Selectors
300 8.3.2.3
Chapter 8 m:n Protection Architecture
The m:n architecture is another type of shared protection where m dedicated protection channels are shared as backup faciUties by n working channels, as shown in Figure 8-3. The value of m is typically < n. The bandwidth of each of the protection channels should be adequate to protect any of the n working channels. Generally, when the normal traffic signal on a working channel is impaired, first the normal traffic signal is assigned to an available protection channel and then the traffic signal is transferred from the working to the protection at both the source and sink endpoints of the protected domain. Obviously, only m working channels can be protected at any one time. The main advantage of this architecture is that the protection channels can be used to transport extra traffic when they are not used. However, this architecture tends to be complex, and it does not support dual node interconnect for protected subnetworks. 8.3.2.4
(1:1)" Protection Architecture
The (1:1)" protection architecture is an n parallel 1:1 protection architecture and behaves in a manner similar to 1 :n. There are n protection channels sharing same the bandwidth that are the backup facilities for n working channels, as shown in Figure 8-4. The protection channel bandwidth is capable of protecting any of the n working channels. When the normal traffic signal over the working channel is impaired, the normal traffic is first assigned to the associated available protection channel. This is followed by transferring the normal traffic signal on the working channel to the assigned protection channel at both the source and sink endpoints of the protected domain. This architecture allows for only one working channel to be protected at any one time. The main advantage of this architecture is that it allows routing the working traffic over different equipment and resources and thus avoiding a common point of failure. This protection architecture is applicable to cell and packet networks such as ATM and MPLS.
Network
301
Survivability Bridge
Working (1)
Bridge T"
nr I Normal (1)
' ^ ^'. Working (2)1
U
I •-
-n Working (n)
u - /I
•-
I I I I
—•-
:
^
—
-# I Normal (2) I I
I
^^
-n
I Normal (n) I I
Extra traffic (n+1) Protection (0) -^1
I
ir
A
I I I
I I Extra traffic
I
Protection (m-1) •—•
(n+m)
•^1
I 4—
>s'
• ^ AIS/FDI
AIS/FDI I I Selectors
I Selectors Figure 8-3. m : n protection architecture
Chapter 8
302
Normal (1)
"
Working (1)
Normal (1)
V^ I
Protection (1) I I —1-
Normal (2)
I I
Working (2)
Normal (2)
•T !•I I
Normal (n) M
Protection (2)
^
I I I I ^-1
Working (n)
A*-
1v Protection (n) |
I
V
I I
Extra Traffic (n+1)
^
I
Protection (0)
-I-
Normal (n)
T I I I I I I I ^ !-••
Extra Traffic (n+1) M—
l#-
Bridge
Selector
I Bridge
AIS/FDI
Selector
AIS/FDI
Figure 8-4, (1:1)" protection architecture
8.3.2.5
Ring Protection Architecture
Currently, the ITU is developing a new recommendation, G.808.2, which will provide an overview of generic aspects of ring protection. One simple application of ring protection is the self-healing ring, where a failure of one of the links between the ring nodes results in the ring reconfiguring itself and looping the traffic at the nodes nearest to the failure away from it. Another
Network Survivability
303
form of ring protection is shown in Figure 8-5, which is described in ITU Recommendation G.841 and is referred to as SDH Multiplex Section Shared Protection Rings (MS-SPRING). G.841 provides SDH equipment-level specifications to implement this type of protection.
8.3.3
Protection Switching Parameters
8.3.3.1
Switching Types
There are two types of protection switching, namely, unidirectional and bidirectional. a. Unidirectional switching : Switching, in this case, is completed when the normal traffic signal is selected from the protection channel at the end detecting the fault. For 1+1 architecture, the sink-end selector is activated only, and no communication with the source end is required. For other protection switching architectures, the sink-end selector and the source-end bridge are activated. This requires a communication channel between the two ends of the protected domain, which is referred to as the Automatic Protection Switching (APS) channel. The APS channel is terminated at the connection functions at both ends of the protected domain. The unidirectional protection switching is a simple scheme to implement, and under multiple failure conditions there is a greater chance of restoring traffic faster than with bidirectional switching.
STM-M Working Path
Zl. STM-M/ Au-4 Source—
[M
%^ ^
•
3^:7.
STM-N Protection Path
STM-M/ Au-4 Destination
Figure 8-5. SDH - MS - SPRING
304
Chapter 8
b. Bidirectional switching : In this type of switching, the normal traffic signal is switched from the working channel to the protection channel at both ends of the protection span. For 1+1 architecture, the selectors at the sink and source ends are activated. For other protection switching architectures, selectors and bridges at both sink and source ends are operated. The two ends of the protection span communicate via the APS channel to initiate the transfer of the normal traffic signal. Priority, request and switching commands are contained within the APS protocol. The bidirectional protection switching results in a minimum number of Severely Errored Seconds (SES) during repair and maintenance of the working path. This is due to fewer switching events and thus fewer disruptions to normal traffic. 8.3.3.2
Operation types
There are two types of protection switching operations, namely, revertive and nonrevertive. a. Revertive operation: When the defect or the external commands are cleared on the working channel, the normal traffic signal returns to the working channel that was in use prior to these events. Revertive operation is likely to be favored for the following reasons: • In some applications, a protection capacity is needed for frequent network capacity rearrangement. • The protection channel may be of much lower performance such as large delay. • Network management needs to be simplified. To prevent frequent operation of the protection switch due to intermittent defects such as bit error rate fluctuation, a fixed period of time is allowed to elapse before a normal traffic signal can use a recovered working channel, where the Signal Fail (SF) and Signal Degrade (SD) have been cleared. This period is called a wait-to-restore period. I It is generally configurable, and a settable value from 5-12 minutes is standardized. b. Nonrevertive: The normal traffic signal stays on the protection channel and does not return to the working channel after the defect or external commands are cleared. The advantage of nonrevertive operation is that, in general, it will have less impact on traffic performance. 8.3.3.3
Protocol Types
All protection switching types require the APS protocol to coordinate the bridge and selector actions at both ends of the protection domain. The only exception is the 1+1 unidirectional protection switching scheme. Different
Network Survivability
305
APS protocols are required depending on the type of protection, selector, and bridge used. The near-end node (A) and the far-end node (B) of a protection domain communicate with each other via the APS channel. The number of times the APS protocol transverses the protection span between nodes A and B determines the protocol type. To reduce the protection switching time it is necessary to minimize the number of communication cycles between nodes A and B. The APS protocols associated with these communication cycles are referred to as 1-phase (B to A), 2-phase (B to A and A to B), or 3-phase (B to A, A to B, and B to A). The 1-phase protocol is employed in (1:1)" architecture, the 2-phase protocol in 1+1 bidirectional architecture and the 3phase protocol in all architectures. The 3-phase protocol is most commonly used, since it is applicable to all architectures and prevents misconnection occurrence in all conditions. However, it does suffer from increased switching times due to triple exchange of APS messages between the two ends of the protected domain. 8.3.3.4
Protection Switching Temporal Model
Figure 8-6 illustrates the protection switching model whose parameterare defined below. Detection Time Tl: This is the time interval from the instant of the network impairment occurrence and its detection Hold-off time T2: This is the time taken to confirm that the defect conditions require protection switching Switching operation time T3: This is the time taken to complete the processing and transmission of the APS messages to invoke protection switching. Switching transfer time T4: This is the extra time needed to complete the protection switching operation Recovery time T5: This is the time needed after completion of the protection switching operation to fully restore the protected traffic. The protection switching time (T3 + T4) is standardized to be less than 50 ms. Furthermore, restoration is the total time take to restore protected traffic and is the summation of all the time intervals above (i.e., Tl + T2 + T3 + T4 + T5).
Chapter 8
306
8.3.4
Protection Switching Classes
83.4.1
Trail Protection
Trail protection is a dedicated end-to-end protection across an entire network or multiple networks. It protects against faults in the server layer as well as connectivity faults and performance degradation defects in the client layer. It can be deployed in various network architectures such as mesh or rings, and there is no limitation on the number of network elements (NEs) within the protected trail. Trail protection is quite a versatile protection class, and it can be deployed in all combination of protection switching architecture, types, and operation. Figure 8-7 illustrates the generic concept of trail protection. It is worth noting that trail protection at the path level requires an additional termination function on the cross-connect equipment, which results in additional cost relative to other classes. Trail protection can be applied on individual trails or group of trails.
Onset of network impairment
Activation of SFor SD signals
Start of protection switching operation
The last control/ command received
The protection switching operation is completed
Protected traffic fully restored
T3
'
T4
Protection Switching time
Figure 8-6. protection switching temporal model
Network Survivability
307 Working Channel
Traffic Signal
Traffic Signal
Protection Channel
Protection Domain TT - Trail Termination
PS - Protection Switching
Figure 8-7. Trail protection
8.3.4.2
Subnetwork Connection Protection (SNC-P)
SNC-P is used to protect a segment of a trail within a network or multiple networks. The subnetwork can be between two Connection Points (CPs), as shown in Figure 8-8, or between a CP and Termination Connection Point (TCP), or between two TCPs as the entire end-to-end network connection. SNC-P can operate in various protection architectures and switching and operation types. It can be used in various network structures, such as mesh or rings or mixed, and can be applied in any layer in a layered network. SNC-P is further characterized by the way defects are monitored within its domain. a. SNC/I — Inherent: In this variant, SNC operates on the server layer defects conditions only. The server layer termination and adaptation functions determine the signal fail and signal degrade (SF/SD) conditions. b. SNC/N -Nonintrusive: This approach requires the use of nonintrusive monitoring functions to determine the SF/SD conditions. It can be applied using end-to-end detection, where the server layer defects, continuity/connectivity defects, error degradation, and end-to-end OAM overhead are monitored. Alternatively, it can be applied using a sublayer detection, which is similar to end-to-end detection except that the sublayer OAM overhead is monitored. This is typically applied in a segment sublayer (referred to as the tandem connection (TC) layer) and referred to as SNC/S. TC monitoring functions are also deployed to
Chapter 8
308
determine the SF/SD conditions. Protection can be applied on individual SNCs or a group of SNCs.
Traffic r Signal PS
k
J
Trafllc Signal PS
S N C Protection Domain
Figure 8-8. SNC/I protection
Working Channel
Traffic r Signal
Traflfic Signal PS
Protection Channel
SNC Protection Domain E —Endlo-End Trail Termination M —Nonintrusivc Monitor
PS —Protection Switching
Figure 8-9. 1 + 1 SNC/N protection
Network Survivability
309
Traffic Signal
Traffic Signal
E —End-to-End Trail Termination S —Protection Sublayer Trail Termination
PS —Protection Switching
Figure 8-10. SNC/S protection
8.3.4.3
Unidirectional Path Switch Ring (UPSR)
S0NET[11] defines the above protection scheme, where the working channels for both directions of bidirectional traffic may follow different physical paths in a protected ring. In general, it can be used for both SNC/I and SNC/N but not SNC/S or trail protection. This scheme may result in a longer transfer delay, and it is different for the two directions of traffic as compared with schemes where the working channels for both directions of traffic follow the same physical path.
8.3.5
Hold-off Timer
In a nested protection mechanism, a single failure may trigger multiple recovery mechanisms, which may interact with each other, resulting in undesirable and unknown network states. To prevent this state, a hold-off timer is used. This allows the innermost protection switching to operate first and to restore traffic before the outermost protection switching is activated. Hold-off timers provide the means to limit unnecessary protection switching events. In some applications a hold-off timer is used in each protection selector. For example, in 1+1 SNC/N and SNC/I protection switching types, hold-off timers are used to prevent early switching due to the differential delay difference between the long and short route. A hold-off timer is activated when one or more defect conditions such as SD or SF are detected in the protected group, and it runs for a provisionable period from 0 to 10 seconds. Recommendation G.841 defines provisionable steps of 100 ms for SDH, and a similar value is defined for OTN.
310
Chapters
When the time interval of the hold-off timer expires, the SF/SD status of all the traffic signals is passed to the protection switching process to act upon them. The SF/SD conditions are not required to be present during the entire duration of the hold-off period, since only the states at the end of the holdoff period are required.
8.3.6
Protection Switching Trigger Criteria
Recommendation G.808.1 defines the Signal Fail (SF) and Signal Degrade (SD) conditions and the combination rules associated with the layer network. It also presents tables that illustrate defects that contribute to SF and SD conditions in various network transport technologies. For example, among SDH conditions that contribute to SF are Loss of Signal (LOS), Alarm Indication Signal (AIS), path mismatch (TIM), Loss of Frame (LOF), etc. The SD condition is caused by digital degradation due to increased error rate.
8.3.7
Null Signal
The null signal is inserted by the connection function and is ignored at the sink end of the protection. The null signal is transported by the protection channel when the channel is not transporting extra traffic or a normal traffic signal. Typical examples of null signals are an Unequipped signal in SDH, Optical Data Unit — Open Connection Indication (ODUk-OCI) in OTN, no signal in ATM/MPLS, and test signals.
8.3.8
Automatic Protection Switching (APS) Protocol
8.3.8.1
The APS signal
An APS signal is required to coordinate the protection switching actions at both ends (A and B) of the protected domain. It is transported via the APS channel and is generally allocated to one or more protection channels. The APS signal contains the following messages: •
Request/state type: to identify the highest-priority fault conditions, external commands, or the state of the protection process
Network Survivability • •
8.3.8.2
311
Requested and bridged signals: to identify the null, normal traffic and extra traffic signals Protection configuration: to identify the use of an APS channel, the protection architecture, switching, and the operation type External Commands
External commands are issued externally to initiate the protection process. Generally, only one external command is issued per protection group. External commands can be preempted or denied by higher-priority failure conditions, requests, or states. These commands are useful to do the following: • to change equipment configuration for maintenance purposes • to disable access to protection • to test the protection process • to freeze the current state of the protection process to prevent further action • to clear previous external switch commands 8.3.8.3
Protection Switching Process States
Three states of the protection switching process are identified below. •
•
•
No Request (NR): This means that in the event of a failure, the normal traffic signal is selected from the working channel. The protection channel may be carrying the null signal, extra traffic, or bridged traffic in the 1+1 protection architecture. Do Not Revert (DNR): This is used in a nonrevertive operation to maintain a normal traffic signal selected from the protection channel Wait-to-Restore (WtR): This is used in a revertive operation to prevent frequent selector operation due to intermittent failures. After clearing the SF/SD conditions on the working channel, the normal traffic signal will remain selected from the protection channel until the Wait to Restore time has expired. At the end of the WtR period and in the absence of any other event or command, the state will change to NR .
312 8.3.8.4
Chapters Priority
In the APS protocol, various priorities are assigned to fault conditions, external commands, or protection states. These are generally applied locally at each endpoint or between endpoints in the protection domain.
8.3.9
Examples
The ITU has developed a number of recommendations that cover technology-specific protection switching schemes. The modeling methodology used in developing these recommendations is based on ITU Recommendation G.805 [9] a.
Recommendation G.841: This Recommendation defines the implementation of different protection schemes and architectures for SDH networks. It covers protected entities such as SDH linear multiplex section protection, a segment of an SDH end-to-end path (i.e., SNC protection), or an entire end-to-end path (i.e., higher-order or lower-order linear VC trail protection). It also specifies the requirements for Multiplex Section Shared Protected Ring (MSSPRING).
b.
Recommendation G.873.1: This Recommendation defines various linear protection schemes for Optical Transport Networks (OTNs) at the Optical Channel Data Unit (ODU) level. It also defines the objectives and applications for these schemes. The protection schemes covered by this recommendation are ODU SNC/I, ODU SNC/N, and ODU SNC/S. OTN ring protection is currently being developed in the ITU draft Recommendation G.873.2.
c.
Recommendation Y.1720: This Recommendation defines the requirements and mechanisms for 1+1, 1:1, shared mesh, and packet 1+1 protection switching functionality for MPLS networks. The defined mechanisms are specified to support end-to-end point-topoint Label Switched Paths (LSPs).
d. Recommendation 1.630: This Recommendation defines architectures and mechanisms for ATM VP/VC protection switching and ATM VP Group protection switching. The Recommendation also describes individual and group protection, arrangement of protected domains, and resource allocation policies. The protection mechanism defined
Network Survivability
313
in this Recommendation includes protection switching trigger, holdoff mechanisms and protection switching control protocol.
8.3.10
Optical Transport Networks (OTN) Survivability
The recent advances in Dense Wavelength Division Multiplexing(DWDM) technology resulted in its deployment in Optical Transport Networks (OTNs) worldwide. DWDM allows the transport of various bit rates on a multitude of optical wavelengths, which are closely spaced, on a single optical fiber. The ITU has developed a number of recommendations defining frame format and OTN interfaces (Recommendation G.709) and the physical parameters (Recommendation G.959.1). In terms of survivability, OTN protection schemes are similar to those defined for SDH and SONET networks and are based on the protection schemes defined in ITU Recommendation G.808.1. ITU Recommendation G.873.1 defines linear protection schemes for OTN. Figure 8-11 shows an example of 1+1 Optical Subnetwork Connection (OSNC) protection. It shows that the digital client signal is applied to two optical transmitters and transported over two optical wavelengths, one as a working channel and the other as the protection channel. At the receiver, a defect on the optical path results in selecting the digital client signal from the protection optical wavelength. For shared ring protection, the work in the ITU is at an early stage, but it is based on SDH shared ring protection. Using such a protection scheme, each connection, which is protected, is provided with a preassigned 1:1 protection route and capacity. The protection connection itself does not carry a copy of the working connection under nonfailure conditions; therefore, the capacity is not occupied and can be used for the extra traffic with a lower priority. The extra traffic itself is not protected. This protection capacity can be shared by other protection connections on a link-by-link basis. To restore the OTN ring network after a failure, the affected working connections are switched to counterdirectional routes on an end-to-end basis with preassigned wavelengths. Obviously, an APS protocol is required for such a protection scheme.
Chapter 8
314 Optical Receiver* Demux+ Monitor
Optical jrransmitter+ Multiplexer Digital
iDigital
Client Signals
I^Sel.
Client Optical Receive r+ Demux+
Optical {Transmitter Multiplexe
Signals
Optical Fiber
Figure 8-11. An example of 1+1 Optical Subnetwork Connection (OSNC) protection
8.4.
SURVIVABILITY OFFERED BY RESTORATION
In the last few years, there has been considerable interest in the use of control plane technology such as Automatic Switched Optical Networks (ASON) and Generalized Multiprotocol Label Switching (GMPLS) in transport networks for restoration. Restoration differs from protection, as it does not use a dedicated spare capacity for the protection path. Instead, alternative routes are found from the available spare capacity in the network. Therefore, there is a degree of sharing of the available spare capacity between a number of working routes. The restoration mechanism offers the following advantages: • fast recovery of unprotected circuits and thus an increase in their availability • improved time-to-repair for unprotected services • under major outage, a reduction of the number of circuits that need to be manually rerouted Additionally, restoration offers capacity savings compared with network protection schemes. However, restoration should not be considered as a complete replacement for protection. For the restoration to work efficiently, a high degree of meshing in the network is needed. Generally, this approach is better suited to core network, which is highly meshed and operating at VC-4 or optical wavelength level. The outer core network, with its low meshing and the need for dual parenting protection, will dictate the use of protected architectures such as ring protection. In the last four years, the ITU-T approved a number of recommendations defining control plane architecture, routing requirements, signaling, and discovery [5-9]. ITU recommendation G.8080 specifies the architecture and requirements for the automatic switched transport network as applicable to SDH transport networks, as defined in G.803, and Optical Transport Networks, as defined in Recommendation G.872. It describes the set of control plane components
Network Survivability
315
that are used to manipulate transport network resources in order to provide the functionality of setting up, maintaining, and releasing connections.
8.4.1
Network Restoration Techniques
Basically, restoration may be implemented in the following manner: a. Preplanned routing with centralized route calculations: In this type of restoration, the route calculations are not made in real time, and thus optimal routes can be achieved prior to the occurrence of the network failure. These preplanned routes can be enhanced or amended and downloaded to the network elements. b. On-the-fly restoration with centralized route calculations: In this type, routes are calculated using the available spare capacity in the network after the failure has occurred. The limitation of this scheme is that restoration can take longer. The reason for this delay is the post-failure route calculations, which are only efficient when topological convergence of the routing protocol has occurred and the management system has obtained all the relevant alarm information. c. Distributed techniques: This type uses distributed rather than centralized route calculations and can use either on-the-fly or preplanned route calculations. All the above schemes are sometimes augmented with crankback should the route calculations fail to converge.
8.4.2
Restoration time
Generally, restoration is nondeterministic, and thus the way the network performs under failure conditions is much less predictable and depends on many factors such as failure location, type of failed circuit, the topological distribution of the spare capacity, and traffic distribution. The speed of operation of restoration is likely to be slower than protection schemes, since on failure a new path has to be computed and setup across the network. It is inevitable that restoration will take longer than dedicated protection schemes. While the time for restoration of a single circuit may look impressive, in reality significant failures such as fiber cut will result in restoration of many circuits over many individual links depending on the size of the network and the failure type. Restoration is a serial process, and therefore, restoration times tend to be cumulative. The larger the scale of the failure, the slower
316
Chapters
the restoration time. In some applications, priority is assigned to traffic to restore important services first. The restoration process involves the computation of new routes using an existing prefailure network topology map. This process may resuh in an attempt to use bandwidth that is no longer available; in addition, the topology map may have been modified, or other nodes may have used the available paths for restoration. These issues obviously prolong restoration time. During restoration, it is likely that many nodes will try to restore traffic at the same time. Since all nodes use the same network topology map, it is possible that two or more nodes will attempt to use the same capacity as part of their paths. Since only one node can do so (first come, first served), the others will have to roll back and try again. This process will also contribute to prolonged restoration time. Restoration time is also influenced by the traffic granularity used in the restoration process. Core networks with a high degree of meshing operating at the VC-4 level, and above such optical wavelengths, are better suited for implementing restoration. The reason is the lower number of circuits, which is more manageable in terms of topology and route calculations than a large number of lower bit rate circuits, as in VC-12. Overall restoration time is therefore affected by many factors, all of which contribute to its total value.
8.4.3
Interoperability
Interoperability is one of the key requirements of restoration-based networks as they become widely deployed. Most of the current restoration deployment in today's networks utilizes proprietary implementation. Interworking between protection domains can be achieved at two basic interfaces, namely, a User-Network Interface (UNI) or a Network-Network Interface (NNI). The UNI approach is based on a client-server relationship where the client requests bandwidth from the server over the interface without any need for client and server to know each other's network topology or signaling/routing protocols. The NNI approach tends to be more complex, since it requires topology interchange and hence compatibility of the signaling/routing protocols between the different protection domains. Therefore, one domain knows the topology of the other and thus can determine a complete route through the network. Figure 8-12 illustrates these features.
Network Survivability
317
Client A
Client B
Transport Network
Figure 8-12. An example of topology on UNI and E-NNI control and data planes
In terms of standards, the ITU and the Optical Internetworking Forum (OIF) has achieved considerable progress over the last three years. Interworking has recently been demonstrated between a number of operators and vendors by the OIF at the SuperComm 2004.
8.5.
LINK CAPACITY ADJUSTMENT SCHEME (LCAS)
ITU-T Recommendation G.7042 describes a technique, referred to as Link Capacity Adjustment Scheme (LCAS) and covered in Chapter 4 in more detail, for autonomously increasing or decreasing the bandwidth on a path in SDH or OTN transport networks. LCAS can also provide another dedicated end-to-end scheme for network survivability. It can offer survivability across single or multiple networks and can operate in various network structures such as mesh and rings. Generally, LCAS protects against faults in the server layer and connectivity faults and signal degradation in the client layer. It can operate in all combinations of protection architecture, switching, and operation types. Figure 8-13 shows the generic concept of LCAS where network survivability is offered. The scheme operates by removing the fractional payload transported on any member of the Virtual Concatenated Group (VCG) that experiences fauh conditions. The result is a reduced payload size.
318
Chapter 8 Transport entities^ over Route 1
Traffic Signal
Traffic Signal
Sublayer Protection Domain E —Endto-End Trail Termination LPS —LCAS Protection Switching Function Figure 8-13. Survivability using LCAS
8.6.
MULTILAYER SURVIVABILITY
Current networks a support variety of technologies such as SDH/SONET, OTN, ATM, Ethernet, IP, etc. This situation has resulted in multilayer transport networks with a variety of layer nesting depending on technology deployment and network evolution. Each technology layer may support a variety of protection and restoration schemes. Therefore, multilayer
Network Survivability
319
survivability uses two or more nested protection mechanisms, while singlelayer survivability employs a single end-to-end or cascaded protection schemes. A good survivability strategy in a multilayer network is to select the survivability schemes associated with these technologies that offer optimal performance to deliver the desired QoS more cost-effectively than can be achieved in a single layer network. Multilayer survivability can effectively combine the merits of the protection mechanisms of each of its constituent layers, such as the transport and service layer protection schemes. However, it is necessary to ensure that a single failure cannot trigger multiple protection mechanisms, which may interact with each other, resulting in undesirable and unknown network states. A good approach to the design of a multilayer network survivability strategy is to identify the usefulness of nesting of protection mechanisms and to identify cases where interworking should be avoided. ANSI Tl-TR-68-2001 [10] report describes various network scenarios for multilayer survivability and recommends a strategy for its implementation.
8.7. REFERENCES [I]
ITU Recommendation G.808.1, Generic Protection Switching—linear trail and subnetwork protection [2] ITU Recommendation G.873.1' Optical Transport Networks (OTN)—Linear protection [3] ITU Recommendation G.841, Types and characteristics of SDH network protection architectures [4] ITU Recommendation 1.630' A TMprotection switching [5] ITU Recommendation Y. 1720' Protection switching for MPLS networks. [5] ITU Recommendation G.807, Requirements for automatic switched transport networks (ASTN) [6] ITU Recommendation G.8080, Automatic switched optical networks (ASON) [7] ITU Recommendation G.7713 series, Distributed call and connection management [7] ITU Recommendation G.7714 series, Automatic discovery techniques [8] ITU Recommendation G.7715 series, Architecture and Requirements for Routing in Automatic Switched Optical Networks [9] ITU Recommendation G.805, Generic Functional Architecture of Transport Networks [10] ANSI Tl report Tl-TR-68-2001, Enhanced Network Survivability Performance, 2001. [II] ANSI Tl.105.01-2000, Synchronous Optical Network (SONET) — Automatic Protection, 2000.
This page intentionally blank
PART 2 Services Offered Over Transport Networks
This page intentionally blank
Chapter 9 METRO ETHERNET OVERVIEW AND ARCHITECTURE
Nan Chen Strix Systems; Metro Ethernet Forum President
9.1.
METRO ETHERNET DEMAND AND REQUIREMENTS
Ethernet is rapidly evolving into an end-to-end solution extending from the LAN to the metro core, the metro edge, and even the "last mile." The key drivers behind this evolution are the compelling cost-effectiveness of Ethernet, the ubiquity of Ethernet, and broadband services. In order for Ethernet to truly address the requirements of MANs, it must scale performance levels, deliver carrier-class protection mechanisms, guarantee services, and support TDM traffic. The future of Ethernet as the foundation for metro carrier networks lies in its ability to support these carrier-class features, which in turn will enable service providers to offer a broad set of services.
9.1.1
Network Resiliency
To deliver the network resiliency/protection and rapid fail-over rates required by carriers and service providers, developers of metro Ethernet gear must overcome the limitations of Ethernet's incumbent spanning tree protocol (STP), an Ethernet technology that delivers 45 -second recovery rates from link failures. The rapid spanning tree protocol (RSTP), a recent new standard from IEEE, has reduced the recovery time to a few seconds.
324
Chapter 9
While this time frame represents a significant improvement over the existing STP standard, certain carrier appHcations such as TDM-based voice and financial data transactions need the greater protection mechanisms to provide recovery in under 50 milliseconds. Increasingly, metro Ethernet solutions are emerging with 50 ms protection for ring and mesh-based topologies, while preserving the cost advantages of Ethernet and its MAC. The key advantages of using these solutions for protection switching are follows: • Facilitates alternative switching paths • Enables 50-msec protection by using labeled switch-path tunnels • Elimates the need for protection among different transport protocols isn't needed MPLS is the only protocol used • Provides visibility and enables manipulation at the packet flow level. • Allows providers to optimize their bandwidth for protected traffic and to allocate less bandwidth for unprotected traffic • Enables carriers to use alternative routes for switched traffic according to bandwidth availability • Eliminates constriction to certain topologies: rings or mesh
9.1.2
Traffic and Performance Management
As metro Ethernet becomes an alternative transport infrastmcture, the best-effort Ethernet will not be adequate, since new universal transport will need to provide guaranteed services with regard to bandwidth, delay, jitters, etc. The fundamental building block for guaranteed services is a connectionupon which Service Level Specifications (SLS) are assigned and enforced. In addition, a connection is always uniquely dedicated to a customer and/or a service, thereby ensuring data security. With connectionbased flows, carriers can enforce service-level specification on a connection flow basis, across the network. For example, committed information rates and excess information rates can be configured and assigned as an attribute of a connection flow. MPLS offers the ability to take advantage of packet information, enabling a flexible service provisioning architecture that allows carriers to assign SLAs based on customer, IP address, and application type.
9.1.3
Circuit Emulation Services
Another key carrier-class feature that will propel the use of metro Ethernet is support of TDM services via circuit emulation services. At the customer premises, metro Ethernet edge switches can packetize TDM traffic via interfaces such as Tl/El, T3/E3, OC-3/STM-1, etc.. This packetized
Metro Ethernet Overview and Architecture
325
traffic is assigned the highest available classification tag for transport across the network. Clocking is supported for accurate and reliable TDM voice transmission. The connection tunnel itself provides guaranteed, secure service levels across the metro network, with interfacing to the publicswitched telephone network occurring at a central point. The allure of such carrier-class capabilities is clear: Carriers can deliver high-bandwidth data services and at the same time continue to support their traditional voice services on a single network. Ethernet offers a costeffective, scalable transport solution across the metro core with support for existing SONET/SDH-based services at the metro edge. The ability to map Tl/El, T3/E3, and 0C-3/STM-1 interfaces over an Ethernet network makes this option compelling for carriers who wish to migrate their legacy services in a phased approach and to integrate new Ethernet transport with existing infrastructure. SONET/SDH investments are preserved, while a new, more profitable solution is realized for metro transport.
9.2.
METRO ETHERNET FORUM CHARTER
The Metro Ethernet Forum (MEF) [1] was founded in June 2001 with a mission to accelerate the adoption of metro Ethernet as the technology of choice in metro networks worldwide. The primary priorities of the MEF are to define: • Ethernet Services for metro transport networks. Such services will be delivered over native Ethernet-based metro networks and could also be supported by other transport technologies. • Carrier-class Ethernet-based metro transport technologies by specifying architecture, protocols, and management for Ethernet-based metro transport networks The secondary priorities of the MEF are (when deemed necessary) to define: • Work to be done by other organizations on other transport technologies • Non-Ethernet interfaces, if not defined by other organizations Figure 9-1 shows the technical areas in which MEF is developing recommendations.
Chapter 9
326 Technical Areas 3E
3C
Architecture
Services
Transport
Management
Protection I— Requirement & Framework
_ Services ' Model
^ Reference ' Model
, Traffic Management
, User Network • Interface (UNI)
Test
, EMS-NMS Info Model .EMS I— Requirement
• Protection lA
L_Testlng Methods
Figure 9-1. Categorical topical coverage within MEF
9.3.
METRO ETHERNET NETWORK (MEN) ARCHITECTURE
9.3.1
MEN Reference Model
The basic network reference model of a MEN [2] is depicted in Figure 92. Two major functional components are involved: • the subscribe/customer edge equipment, and • the public MEN transport infrastmcture
Subscriber
Subscriber
T
Metro Ethernet Network (MEN)
^
Ethernet Virtual Connection
j UNI
11 I
i s-
i
» j
1
UNI 1 Private 1 , i c n H . ! 1 1- i1 Customer H-i J 1 \ 1 Network j T l "ser J L
.--.tS
i
w <
'^
1
End-to-End Ethernet flow
1
^
w
Figure 9-2. Basic Network Reference Model
Reference point T, also referred to as the UNI (User Network Interface) reference point, demarcates the boundaries between the public MEN and a private customer network. Reference point S is the conceptual point that demarcates the boundaries between the private customer network equipment, when present, and the end-user terminal equipment generating an Ethernet frame flow. If no other private network infrastructure exists between the subscriber terminal equipment and the public MEN equipment, the S and T
Metro Ethernet Overview and Architecture
327
reference points coincide. Unless otherwise stated, the term subscriber/customer equipment is used to refer to all private customer equipment outside the MEN. An Ethernet flow represents a particular (potentially noncontiguous, e.g., consecutive Ethernet frames may belong to different flows) unidirectional stream of Ethernet frames that share a common treatment for the purpose of transfer steering across the MEN. In particular, an end-to-end Ethernet flow refers to the flow of Ethernet frames between the communicating terminal equipment (TE) that creates and terminates the Ethernet frames. The Ethernet Virtual Connection (EVC) is the architecture construct that supports the association of UNI reference points for the purpose of delivering an Ethernet flow between subscriber sites across the MEN. There may be one or more subscriber flows mapped to a particular EVC (e.g., there may be more subscriber flows identified by the flow classification rules at the ingress point to network than EVCs). The mapping of Ethernet flows to EVCs is service specific and specified in the MEF Ethernet Service Model specification and also in Chapter 10, of "Ethernet Services over Metro Ethernet Networks."
9.3.2
MEN Layer Network Model
The MEN layer network model specified in this architecture framework defines the MEN in terms of three layer network components: the Ethernet Services layer supporting basic Layer 2 (L2) Ethernet data communication services; a set of one or more supporting Transport Services layer(s); and an optional Application Services ayer supporting applications carried on the basic L2 Ethernet services. In addition, each of these layer networks may be further decomposed into its data, control, and management plane components. This layer network view of a MEN is illustrated in Figure 9-3.
Chapter 9
328
.Mm
0
c m
.E
m c S.
"5 sL
•c-
m
o O
Application Services Layer (e.g., IP, MPLS, PDH. etc.)
Ethernet Services Layer (Ethernet Service PDU)
CL
£L
u> m
m S'.
D Transport Services Layer (e.g., IEEE 802.1, SONET/SDH, MPLS)
Figure 9-3. MEN layer network model
9.3.2.1
Ethernet Services Layer (ETH Layer)
The Ethernet Services layer, also referred to as the ETH layer, is responsible for the instantiation of Ethernet MAC-oriented connectivity services and the delivery of Ethernet service frames presented across welldefined internal and external interfaces and associated reference points. The ETH layer is also responsible for all service-aware aspects associated with Ethernet MAC flows, including operations, administration, management and provisioning capabilities required to support such Ethernet connectivity services. The service frame presented by the ETH layer external interfaces is Ethernet unicast, multicast, and broadcast frames conforming to the IEEE 802.3-2002 frame format. The Ethernet Services layer is a single layer network. The detailed architecture framework for the Ethernet Services Layer is provided in a separate Ethernet Layer Framework Architecture document of the MEF. 9.3.2.2
Transport Services Layer (TRAN Layer)
The Transport layer, also referred to as the TRAN layer, supports connectivity among ETH layer functional elements in a service-independent
Metro Ethernet Overview and Architecture
329
manner. Various layer network technologies and interconnect approaches may be used to support the transport requirements for the Ethernet services layer. Sample transport layer networks include IEEE 802.3 PHY, IEEE 802.1 bridged networks, SDH VC-n/VC-n-Xc, SONET STS-n/STS-4n-.Xc, ATM VC, OTN ODUk, PDH DSl/El, MPLS LSP, etc. Those transport layers are supported by their respective server layers; e.g., SDH STM-N Multiplex Section, ATM VP, OTN OTUk, PDH DS3/E3, MPLS LSP, IP, fiber, etc. This model may be applied recursively downwards into the transport layer network stack until the physical transmission medium (fiber, copper, coax, wireless) is reached. 93.2.3
Application Services Layer (APP Layer)
The Application Services layer, also referred to as the APP layer, supports applications carried on the basic Ethernet services across the MEN. Various application services may be supported over the basic Ethernet services supported by the Ethernet services layer. Sample services include the use of ETH layer as a TRAN layer for other layer networks such as IP, MPLS, PDH DSl/El, etc. The APP layer may also include add-on functions to complement ETH layer services. Each APP Layer may support other APP Layers. This model may be applied recursively upwards into the application layer network stack.
9.3.3
MEN Reference Points
A MEN reference point identifies a set of layer network reference points used for demarcating administrative boundaries where a link traverses open interfaces specified by the MEF. Figure 9-4 shows the relationship among external architectural components to a MEN, their associated interfaces, and their reference points. External components to a MEN elements include (1) the subscribers to the MEN services, (2) other MEN networks, and (3) other (non-Ethernet) transport and service networks. Subscribers connect to a MEN at a User-Network Interface reference point. Internal network elements (NE) are interconnected via Internal Network-to-Network Interfaces, or INNIs (not shown). Two autonomous MENs may interconnect at an External NNI (E-NNI) reference point. A MEN may interconnect with other transport and service networks at a Network Interworking NNI (NI-NNI) or a Service Interworking NNI (SI-NNI) reference point. An Ethernet Wide Area Network (E-WAN) refers to any MEF-defined ETH services-aware network that provides connectivity between two or more MENs via E-NNIs. A MEN may use non-Ethemet-based transport and service elements as internal architectural components. In some cases, these architectural
Chapter 9
330
components may be integrated in a single network element (NE). In the latter case, an integrated and hybrid MEN may be deployed such that the NI-NNI and/or SI-NNI are logical reference points within a particular network equipment in the MEN. Note that any of the transport/services networks identified in Figure 9-4 may indeed belong to the same service operator (e.g., they may represent different business units within the same company).
other L2/L2+ Services Networks (e.g.. ATM. FR, IP)
Subscriber
Service Interworking NNI
Metro Ethernet Network (MEN) Service Provider X
Subscriber
• NNI
J1
Network Interworking NNI . ^ ^
External NNI
Wide Area Network (E-WAN) Service Provj<JerY
1 1
illiSliiMiiiiiliiWi
iiiiWiii^^Wiii
Subscriber
Network Interworking NNI
MEN Service Provider X
Figure 9-4: MEN External Interfaces and associated reference points
9.3.3.1
User-Network Interface (UNI)
The User-Network Interface (UNI) is the interface used to interconnect a MEN subscriber to its MEN service provider(s). The UNI also provides a reference point for demarcation between the MEN operator's equipment that enables access to the MEN services and the subscriber access equipment. Therefore, the demarcation point indicates the location where the responsibility of the service provider ends and the responsibility of the subscriber begins. The specific location of the UNI reference point (T) is specified in the MEF UNI document.
Metro Ethernet Overview and Architecture
331
Subscriber Site A
\ \ \
Subscriber Site B Metro
I UNI UNI Ciiemt
11 \ 1 \
^ ^
UNI 1 Metwork}
Ni
Ethernet
j
Network
! Metwork 1
UNI 1.
^ -1
T
(MEN)
^
Ethernet Virtual Connection
1
j
1
1 Cllerit 1
UNI 1
T
W
^ End-to-End Ethernet flow
w
Figure 9-5. The UNI and the MEN Reference Model
Functionally, the UNI is an asymmetric, compound functional element that consists of a client side, referred to as the UNI-C, and a network side, referred to as the UNI-N, as illustrated in Figure 9-5. Thus, the term UNI is used to refer to these two functional elements, and generically, to the data, management, and control plane functions associated with them. 9.3.3.1.1 UNI Client (UNI-C) The UNI-C (UNI Client) is a compound architectural component of a MEN that represents all the functions required to connect a subscriber to a MEN. Individual functions in a UNI-C are entirely in the subscriber domain and may or may not be managed by the service provider/network operator. From the perspective of the MEN, the UNI-C supports the set of functions required to exchange data, control, and management plane information with the MEN subscriber. As such, the UNI-C includes functions associated with the Ethernet services infrastructure, the transport network infrastructure, and (if present) application-specific components. 9.3.3.1.2 UNI Network (UNI-N) The UNI-N (UNI Network) is a compound architectural component of a MEN that represents all the functions required to connect a MEN to a MEN subscriber. The individual functions in a UNI-N are entirely in the service provider/network operator domain. From the perspective of the subscriber, the UNI-N supports the set of functions required to exchange data, control, and management plane information with the MEN. As such, the UNI-N includes functions associated with the Ethernet services infrastructure, the transport network infrastructure, and (if present) application-specific components.
332 9.3.3.2
Chapter 9 External Network-to-Network Interface (E-NNI)
The External Network-to-Network Interface (E-NNI) is an open interface used to interconnect two MEN service providers. The E-NNI provides a reference point for network equipment and Ethernet service demarcation between the two directly attached MENs. The E-NNI also provides a reference point for NEs and Ethernet service demarcation between a MEN and an Ethernet service-aware Wide Area Network (E-WAN). Transport interfaces and Network Interworking capabilities other than those associated with native Ethernet physical interfaces (e.g., see Section 9.9.3.3.4) may be supported across this interface. In addition, the term E-NNI is used to refer generically to the protocol exchange that exists at the E-NNI reference point between the architectural elements in each of the MENs that support the ENNI delineation functions. The specific location of the E-NNI reference point is specified in the MEF E-NNI framework document. 9.3.3.3
Internal Network-to-Network Interface (I-NNI)
The Internal Network-to-Network Interface (I-NNI) is an open interface used to interconnect NEs from a given MEN service provider. The I-NNI provides a reference point for Ethernet service demarcation between the two directly attached NEs. Transport interfaces and Network Interworking capabilities other than those associated with native Ethernet physical interfaces may be supported across this interface. In addition, the term I-NNI is used to refer generically to the protocol exchange that exists at the I-NNI reference point between the architectural elements in each of the MENs that support the I-NNI delineation functions. 9.3.3.4
Network Interworking Network-to-Network Interface (NINNI)
The Network Interworking NNI (NI-NNI) is an open interface that supports the extension of transport facilities used to support Ethernet services, and associated EVCs, over an external transport network(s) not directly involved in the end-to-end Ethernet service. The NI-NNI is intended to preserve the characteristic information of a subscriber's flow. The NI-NNI also provides a reference point for demarcation between the two MEN service provider interfaces attached via public transport networks. Examples of other public transport networks include OTN, SDH/SONET, ATM, Frame Relay RPR, etc. The term NI-NNI is also used in this chapter to refer to the protocol exchange that exists at the NI-NNI reference point and the
Metro Ethernet Overview and Architecture
333
architectural element in each of the MENs responsible for the support of the NI-NNI delineation functions (NI-NNIIWF). 9.3.3.5
Service Interworking Network-to-Network Interface (SI-NNI)
The Service Interworking NNI (SI-NNI) is an interface that supports the interworking of an MEF service with services provided via other service enabling technologies (e.g., Frame Relay, ATM, IP, etc.). The SI-NNI provides a reference point for demarcation between a MEN and another public service network. Examples of other public services networks include ATM, Frame Relay, and IP. The term SI-NNI is also used in this chapter to refer to the protocol exchange that exists at the SI-NNI reference point and the architectural element in each of the MENs responsible for the support of the SI-NNI delineation functions (SI-NNI IWF). 9.3.3.6
Other MEN Access Arrangements
The basic MEN reference model illustrated in Figure 9-2 presumes a oneto-one relationship between the MEN subscriber port and the MEN service provider port. Indirect means of access into the service provider equipment supporting the UNI functions, e.g., via a so-called feeder or Access Networks, may be required in deployment scenarios where Ethernet services are introduced over preexisting access technologies (e.g., PDH, SONET/SDH, or Hybrid Fiber/Coaxial networks) or over alternative Ethernet transport facilities. 9.3.3.6.1 Service Node Interface (SNI) The Service Node Interface (SNI) is an interface that supports the extension of the MEF UNI capabilities across an intermediate access network not directly involved in the end-to-end Ethernet service. As illustrated in Figure 9-6, the SNI provides a reference point for demarcation between the network location where Ethernet Service attributes are enforced (MEF ESM) and a packet-aware Access Network that aggregates subscriber flows at a packet level (layer 2) into a common transport channel. In this scenario, the UNI and SNI reference points are equivalent to T (TB) and V (VB) in the ISDN (B-ISDN) terminology. The Access Network strictly provides a packet-based transport function for the access portion of the connection between the subscriber and the MEN. For this reason, the SNI reference point is also informally referred to as the Virtual UNI reference point. As such, the SNI is intended to preserve in a transparent manner the characteristic information of a subscriber's flow. The specific location of the SNI reference point is specified in the MEF TMF specification.
Chapter 9
334
: Sufc^cribir i^ \ SNi
Metro Ethernet Network
\ Syb$ci1l:^rS
Figure 9-6. Reference points for other access arrangements into a MEN
9.3.4
MEN Architectural Components
A functional modeling approach is used in this framework to represent the various architecture components of all the layer network of a MEN. The functional model is based on the architectural constructs created to describe connection and connection-less oriented transport networks in ITU-T Recommendations G.805 [3] and G.809 [4]. This chapter describes the main architectural concepts defined by these two Recommendations as related to MEF architecture framework. Three types of architectural components are identified: (1) Topological Components, (2) Transport Components, and (3) Processing Components. Topological and transport components are used to represent abstract connectivity constructs. Processing components are used to represent abstract system components that affect information transfer. These concepts are further discussed in this section. Formal definition of topological components, transport entities, and reference points can be found in the ITU-T Reconmiendations G.805 and G.809. 9.3.4.1
Topological Components
A topological component provides the highest level of abstraction for the description of an architectural component of a transport network. It defines inclusion/exclusion relationships between sets of like functions and associated reference points. There are four topological components of interest: • Layer Network: A complete set of logical or physical ports (see also access group) of the same type that may be associated for the purpose of transferring information. The transferred information is in terms of a well-defined traffic unit of the particular layer network and is termed its Characteristic Information (CI).
Metro Ethernet Overview and Architecture
335
•
Subnetwork: A partition of a layer network used to affect the steering of specific user data within a portion of a layer network. In the ITU-T terminology, the term subnetwork is reserved for connection-oriented networks. The term Flow Domain is used in the context of connectionless layer networks, such as Ethernet. • Link: A (fixed) connectivity relationship between a subnetwork or access group and another subnetwork or access group. The terms Flow Point Link and Flow Point Poll Link are used in the context of connectionless layer networks, such as Ethernet. • Access Group: A group of co-located logical or physical ports, with associated processing functions that are connected to the same subnetwork or link. Basically, an access group represents the logical access ports into a given subnetwork or flow domain. CI is defined in ITU-T Recommendation G.805. It is used to specify units of information, including a specific unit format, which is transferred on connections within the given layer network. The CI format is always defined in a technology-specific manner for each network layer in its associated architecture framework document. The Ethernet MAC layer, the IP network layer, the SONET/SDH High Order/Low Order layers, or even a fiber infrastructure are example of layer networks. 9.3.4.2
Transport Components
A transport component, or transport entity, provides the means to affect the transfer of information between reference points. Two types of transport entities, and their associated reference points, are defined: • Connection: A transport entity that represents an aggregation of one or more connection-oriented traffic units with an element of common routing. Referred to as Flow in the context of the connectionless layer network. • Connection Point: A reference point that represents a location of transfer of connection-oriented traffic units between topological components. Referred to as a Flow Point/Flow Point Pool in the context of the connectionless layer network. • Trail: A transport entity that represents the transfer of monitored and adapted characteristic information of client layer network between two access points. Typically used to represent the association between source and destination(s) on a per-traffic-unit basis. Referred to as a Connectionless Trail in the context of the connectionless layer network. • Trail Termination Point: A reference point that represents a location of insertion/extraction of monitored and adapted information characteristic
336
•
Chapter 9
to a given layer network (as opposed to the information presented by the chent of the layer network). Referred to as a Flow Termination Point in the context of the connectionless layer network. Access Point: A reference point where the output (input) of a trail termination is bound to the input (output) of an adaptation or the output of an adaptation function.
93.4.3
Processing Components (the MEN Functional Elements)
A processing component provides the actual means to affect the transfer of information at a given reference point. The concept of a Functional Element (FE) is used in this architecture framework to represent the specific set of processes, or functions, within the MEN services or transport network that act on a particular collection of input data to produce a specific collection of output data. Functional elements may also be used to represent compound functions, i.e., a collection of other predefined functional elements It is the intent of the MEF architecture framework to adopt existing functional elements, and associated functional models, from accredited standards organizations and industry forums. In particular, functional elements for Ethernet LANs are derived from IEEE 802.3-2002 and IEEE 802.1Q-1998 specifications. Functional models for connection-oriented functional elements are derived from ITU-T Recommendation G.805. Functional models for connectionless functional elements are derived from the ITU-T Recommendations G.809. Detailed functional models for functional elements in the MEN architecture framework are outside the scope of this chapter. 9.3.4.3.1 Generic MEN Functional Elements Two generic functional elements are defined in ITU-T Recommendations G.805 and G.809 to distinguish the processes required: (1) adapt a client signal for transport across a server layer network, and (2) generate a traceable flow/connection across the layer network. Both source and sink versions of the processes are defined: • Adaptation Function: A transport processing function that converts the server layer network trail information into the characteristic information of the client layer network (and vice versa). • Termination Function: A transport processing function that accepts adapted characteristic information from a client layer network at its input, adds information to allow the associated trail to be monitored (if supported), and presents the characteristic information of the layer network at its output(s) (and vice versa).
Metro Ethernet Overview and Architecture
337
Note that the Adaptation Function is an interlayer function that contains processing aspects of both the chent and the server layer networks. In addition, ITU Recommendation G.806 [5] defines a generic functional element to stir flows within a network: • Connection Function: A transport processing function that transfers information (potentially transparently) from a given input to one or more outputs. Note that a Connection Function is the smallest subnetwork / flow domain (also referred to as Flow Domain Function).
9.3.5
MEN Layer Relationship to the Architecture Model Components
This section discusses the relationships between the ETH, TRAN, and APP layers, the operational planes, and the generic topological components. 9.3.5.1
Operational Planes and the MEN Layer Networks
Three operational planes are distinguished: Data Plane Control Plane Management Plane The Data Plane, also referred to as the user/transport/forwarding plane, provides the functional elements required to steer the subscriber flow, and supports the transport of subscriber traffic units among MEN NEs. The Control Plane provides the functional elements that support distributed flow management functions among NEs participating in the MEN data plane. The control plane also provides the signaling mechanisms necessary to support distributed setup, supervision, and connection release operations, among other flow-control functions. The Management Plane provides the functional elements that support Fault, Configuration (including flow and/or connection configuration). Account, Performance, and Security (FCAPS) fiinctions, as well as any related Operations Administration and Maintenance (0AM) tools. Subscribers and external networks connected to a MEN are likely to include similar layers and planes. The information exchange between the subscriber and MEN Management and Control Planes across defined reference points is restricted (and may be absent) according to the implementation agreements for the UNI, NNI, and other external IWF. • • •
Chapter 9
338
9.3.5.2
MEN Network Reference Model and the Topological Components
A MEN itself consists of physical components (e.g., network elements, ports, etc.) and logical components (e.g., meters, policers, shapers, virtual switches, links, etc.). A MEN architecture is further described by defining the associations between points in the network and the interconnected topological and/or functional components. The partition of a MEN into layer networks places a bound on the scope of the various MEN topological and functional components. Access groups connect customers from the client layer network to the services supported by the server layer network in a client/server relationship. Formally, a reference point describes any binding between the input and output of processing functions or transport entities (see ITU-T Recommendations G.805/G.809). The relationship between the MEN reference model, its layer networks, and its topological components is illustrated in Figure 9-7. Note that within a particular service provider network, multiple Ethernet layer network domains (e.g., ETH Subnetwork A and ETH Subnetwork B) and different kinds of transport technologies (e.g., TRAN Subnetwork A, TRAN Subnetwork B, and TRAN Subnetwork C) may be used to instantiate a particular MEN.
APP Subnetwork A
APP Subnetwork B Application Services Layer (Optional)
Transport Services Layer
C
I TRAN-FE I
TRAN Subnetwork A
J} "'^Link
^
I TRAN-FE | ^
^
. " ^
TRAN Subnetwork B
\^ Link
\ TRAN-FE | _^
•
TRAN Subnetwork 0 \
Figure 9-7. Sample relationship between MEN Reference Model and the Architectural Components
Metro Ethernet Overview and Architecture
339
Note also that a given layer network (e.g., IP, MPLS, SONET/SDH, PDH) may play a dual role with the Ethernet services layer: • as a transport layer providing transport services to the Ethernet service layer, and • as an application service layer using the service provided by the Ethernet service layer. For instance, the Transport and Application Services Layers may be further partitioned into additional layer networks and associated protocols. This generic layer network modeling principle is illustrated in Figure 9-8. For a compound functional element XYZ that expands multiple layer networks, the terms APP XYZ, ETHXYZ, and TRANXYZ are used to refer to the specific set of functional elements within the compound functional element that form part of the APP, ETH, and TRAN layers, respectively (if present).
1 VoIP
1 ^^^
TCP/UDP
1 TCP/UDP Ethernet 802.3
1Ethernet '^
MPLS Ethernet
Ethernet
Others Protocol Suites
PDH
Other Apps
t»«/
t^^^i^ Ethernet Services Layer
Ethernet P802.1ad
m
MPLS VC LSP
ATMVC
MPLS Tunnel LSP
ATMVP
ODUk LOVC
802.3 PHY
I
OTUk
HOVC
OCh
STM-N
OTM-n
=1=
=1=
Medium (Fiber, Copper, Coaxial, Wireless, etc.)
Others
Figure 9-8. Sample decomposition of a MEN into layer networks and protocol stacks
340 9.3.5.3
Chapter 9 MEN Reference Link Model
The term link is used in this architecture framework to refer to a topological component that describes a fixed connectivity relationship, and available transport capacity, between a pair of subnetworks (flow domains), or a subnetwork (flow domain) and an access group, or a pair of access groups. From the point of view of the MEN architecture framework, links are further classified according to the MEN layer network they support and their relationship to internal/external reference point. Links may be used in a variety of different arrangements to produce more complex link models. For instance, a link at layer network N may instantiated by one or more links at layer network N-1 to create a nested link. Multiple-layer network N links may be aggregated into a single-layer N link to create a compound link. In this arrangement, each aggregated link is also referred to as a component link. With respect to the MEN reference points (Figure 9-4), this framework document classifies links into two classes: • Access Link: A link that provides connectivity across an UNI reference point • Trunk Link: A link that provides connectivity across an NNI reference point With respect to the MEN layer network(s), this framework document classifies links into three classes: • APP Link: A link in (one of) the APP layer(s) • ETH Link: A link in the ETH layer • TRAN Link: A link in (one of) the TRAN layer(s) Multiple links may exist between any given subnetwork (flow domains) and access group or pair of subnetworks (flow domains) or access groups. While links are established and maintained at the time scale of the server layer network (e.g., EMS/NMS provisioned links vs. control plane provisioned links), they are not limited to being provided by a server layer network and can also be provided by client layer network connections. Figure 9-9 illustrates the high-level relationship between link types and the MEN UNI and E-NNI reference points. Separate specifications will provide detailed relationships between processing functions and reference points for any MEF-specified interface.
Metro Ethernet Overview and Architecture
341 Logical Connectivity Physical Connectivity
^
CD
T^
APP Link
^mmmmmmm - ^
c^ '^— ^^m _• PiB i ^ - ^
CZ
.^I^H
ETH Access Link
p^
""""l
^^"P ^ ^
^ >TRAN Access Link ^
A
ETHfrtfeui
^^^^^t^i TRAN Trunk Link ^ ^
^
•i ^
1
• P
TRAN Access Link
MEN 2
Figure 9-9. MEN Reference Link Model (example)
ACKNOWLEDGMENTS Metro Ethernet Forum Enrique Hernandez-Valencia
9.4. [1] [2] [3] [4] [5]
REFERENCES
Metro Ethernet Forum, www. metroethemetforum.org. Technical Specification, MEF 4, Metro Ethernet Network Architecture: Framework, Part 1: Generic Framework. ITU-T Recommendation G.805, Generic functional architecture of transport networks, March 2000. ITU-T Recommendation G.809, Functional Architecture of Connectionless Layer Networks, March 2003. ITU-T Recommendation G.806, Characteristics of Transport Equipment — Description Methodology and Generic Functionality, February 2004.
This page intentionally blank
Chapter 10 ETHERNET SERVICES OVER METRO ETHERNET NETWORKS
Bob Klessig Cisco Systems,, Metro Ethernet Forum Member of the Board and Co-Chair of the Technical Committee
10.1.
INTRODUCTION
One of the highest priorities for the Metro Ethernet Forum has been the estabhshment of standards for Ethernet services. These standards will allow Subscribers to successfully plan and integrate Ethernet services into their overall networks and also to be able to do such integration with services from more than one service provider. These standards will also allow equipment vendors to implement capabilities in both service provider equipment and subscriber equipment such that the Ethernet services can be efficiently provided by the service providers and accessed by the subscribers. To accomplish these goals, Ethernet services must be described in precise technical details. The descriptions of the fundamental constructs for Ethernet services are presented in Section 10.2. Service features are described in Section 10.3. The material in this chapter is based on [1].
10.2.
SERVICES MODEL
The services model for Ethernet services is portrayed in Figure 10-1 and is further described in the following subsections.
Chapter 10
344
Customer Edge (e.g., router) (CE)
User Network
Customer
Service Attributes Figure 10-1. Ethernet services model
10.2.1 Customer Edge View The MEF Ethernet services are described from the point of view of the subscriber equipment, referred to as the Customer Edge (CE). Thus the services are defined only in terms that are observable to the CE. The types of technology and the architecture inside the Metro Ethernet Network (MEN) are invisible. By defining services in this manner, the MEN can evolve independently of the evolution of the subscriber's network without disruption of the service that the subscriber is given.
10.2.2 User Network Interface The User Network Interface (UNI) is the physical demarcation point between the responsibility of the service provider and the responsibility of a single subscriber. A UNI must be dedicated to a single subscriber. A UNI is frequently an RJ-45 socket on a service provider-owned Ethernet switch that is placed on the subscriber's premises. Another typical example is an RJ-45 socket on a service provider-owned patch panel. As implied in Figure 10-1, another way of viewing the UNI is that it is located where the CE begins and the MEN ends when looking outward from the MEN. It follows that the MEF services are described as the UNI-to-UNI behavior. For example, frame loss performance is defined in terms of frames received by the MEN at a UNI and frames delivered by the MEN to one or more other UNIs. A key goal in the development of the MEF Ethernet services is that existing, Standard Ethernet devices should be able to attach to an MEN (at a UNI) and successfully work with the service. This makes MEF Ethernet
Ethernet Services Over Metro Ethernet Networks
345
services the first wide-area services, with over 100 miUion existing devices capable of using the services. It follows that the protocols operating across the UNI between the CE and the MEN must be Standard Ethernet [3]. (See Appendix A for background material regarding Standard Ethernet.)
10.2.3 Service Frame There are several physical layers defined in [3], and it is that expected that the different UNIs will have different physical layers. For example, an enterprise might have a Gigabit Ethernet physical layer UNI at headquarters and Fast Ethernet physical layer UNIs at branch offices. However, all UNIs have the same Layer 2 protocol, as defined in [3]. Service frames are Ethernet frames that are exchanged between the MEN and the CE across the UNI. A service frame sent from the MEN to the CE at a UNI is called an egress service frame. A service frame sent from the CE to the MEN at a UNI is called an ingress service frame. 10.2.3.1
Format
The service frame is just a regular Ethernet frame beginning with the first bit of the Destination address through the last bit of the Frame Check Sequence (FCS). As per [3], a service frame that contains an IEEE 802.IQ tag can be up to 1522 bytes in length, and a service frame that does not contain an IEEE 802. IQ tag can be up to 1518 bytes.^ 10.2.3.2
Delivery Transparency
When an ingress service frame is delivered by the MEN to one or more UNIs, the fields of the egress service frame are identical to those of the ingress service except as follows: • The egress service frame may have an IEEE 802.IQ tag when the corresponding ingress service frame does not. • The egress service frame may not have an IEEE 802.IQ tag when the corresponding ingress service frame does have such a tag. • The egress service frame may have an IEEE 802.IQ tag that has a different value from the IEEE 802.IQ tag in the corresponding ingress service frame. In all three cases, the FCS will be recalculated and thus will probably be changed from ingress to egress.
346 10.2.3.3
Chapter 10 Error Detection
If an ingress service frame has an invalid FCS, it will be discarded by the MEN. This is a consequence of the fact that with MEF Ethernet services, the disposition of an ingress service frame is based on the content of the frame header. If a frame header contains bit errors, then the service frame could be delivered to the wrong destination. Discarding service frames with errors avoids this undesirable result.
10.2.4 Ethernet Virtual Connection In theory, an MEN could behave just like an Ethernet shared medium segment where each ingress service frame is replicated and delivered to all other UNIs. This LAN-like behavior can be acceptable within a single organization but is clearly not acceptable for a public network. In an MEN there must be some way to limit communication between UNIs. In MEF Ethernet services, this mechanism is called an Ethernet Virtual Connection (EVC). Each instance of an MEF Ethernet service is based on an EVC. 10.2.4.1
Definition of EVC
An EVC is defined as an association of two or more UNIs. Such UNIs are said to be in the EVC. An ingress service frame sent into an EVC can be delivered to one or more of the other UNIs in the EVC. It cannot be delivered to a UNI that is not in the EVC or back to the originating UNI. Service frames cannot leak into or out of an EVC. An EVC can be thought of as a form of Layer 2 VPN. The UNIs in the EVC seem to be in their own Layer 2 network. However, it is possible for a given UNI to be in multiple EVCs, which means that an EVC is more than a simple Layer 2 VPN. The basic EVC concept is illustrated in Figure 10-2. Two service instances are shown in the figure. The first service instance, based on EVC 1, allows headquarters to communicate with the backup data center. The second service instance, based on EVC 2, allows headquarters and the branches to communicate among themselves. The UNI at headquarters is in both of the EVCs. The router at headquarters would typically be configured with two sub-interfaces on the port attached to the UNI. Communications between the branches and the backup data center must go via the router at headquarters that would route between the two sub-interfaces.
Ethernet Services Over Metro Ethernet NetM^orks
347
Backup Data Center
Branch 2
Headquarters
Branch 1
Figure 10-2. Example of two service instances
There are two types of EVCs, as detailed in the following two subsections. 10.2.4.2
Point-to-Point EVC
A point-to-point EVC associates exactly two UNIs. EVC 1 in Figure 10-2 is an example of a point-to-point EVC. For a point-to-point EVC, all ingress service frames at one UNI, with the possible exception of Layer 2 control protocol messages (see Section 10.3.8.1), will typically be delivered to the other UNI. 10.2.4.3
Multipoint-to-Multipoint EVC
A multipoint-to-multipoint EVC associates two or more UNIs. EVC 2 in Figure 10-2 is an example of a multipoint-to-multipoint EVC. At first glance, it might appear that a multipoint-to-multipoint EVC with two UNIs is the same as a point-to-point EVC. However they are different because additional UNIs can be added to the multipoint-to-multipoint EVC. There are more options for delivery of ingress service frames (that are not Layer 2 control protocol messages) for a multipoint-to-multipoint EVC than for a point-to-point EVC. Broadcast service frames (all ones Destination MAC address) and multicast service frames (multicast Destination MAC address) are replicated and delivered to all other UNIs in the EVC. Unicast service frames (unicast Destination MAC address) can be handled in one of two ways:
348
Chapter 10
•
They can be replicated and delivered to all other UNIs in the EVC. This makes the EVC behave like a shared media Ethernet. • The MEN can learn which MAC addresses are "behind" which UNIs by observing the Source MAC addresses in service frames and deliver a service frame to only the appropriate UNI when the Destination MAC address has been learned. When the Destination MAC address has not been learned, the service frame is replicated and delivered to all other UNIs in the EVC. This makes the MEN behave like a learning bridge. It is important to know which technique is being used on a multipoint-tomultipoint EVC, since the behavior can impact the bandwidth consumed at each UNI and the compatibility with higher layer protocols.
10.2.5
Identifying an EVC at a UNI
In general, there can be more than one service instance, i.e., EVC, at a given UNI, as shown at the UNI at Headquarters in Figure 10-2. This means that for each service frame at the UNI, there must be a way to determine the EVC with which it is associated. This function is accomplished with the Customer Edge VLAN Identifier (CE-VLAN ID) and the CE-VLAN ID/EVC map, as described in the following two subsections. 10.2.5.1
CE-VLAN ID
There are 4095 CE-VLAN IDs, numbered 1, 2, ..., 4095. The CE-VLAN ID is derived from the content of the service frame as follows: • For a service frame that has an IEEE 802.IQ Tag, and for which the 12bit VLAN ID in the tag is not zero, the CE-VLAN ID is equal to the VLAN ID in the Tag. • Untagged and priority tagged^ service frames have the same CE-VLAN ID, and the CE-VLAN ID value is configurable to any value in the range 1, ...,4094 at each UNI. The special treatment of untagged and priority-tagged service frames is consistent with IEEE 802.1Q [4]. An IEEE 802.1Q-compliant Ethernet switch will treat untagged and priority tagged frames as belonging to a default VLAN, and the default VLAN is configurable on each port of the switch. 10.2.5.2
CE-VLAN ID/EVC Map
The CE-VLAN ID/EVC map associates each EVC at the UNI with one or more CE-VLAN IDs.
Ethernet Services Over Metro Ethernet Networks Service Frame Format
CE-VLAN IP
349 EVC
Untagged Priority Tagged Tagged, VID = 1 Tagged, VID = 2 Tagged, VID = 3
Tagged, VID = 4094 Tagged, VID = 4095
> 4094"^ > 4095
^ Blue
CE-VLAN ID/EVC Map Figure 10-3. Example of a CE-VLAN ID/EVC map
Figure 10-3 shows an example of CE-VLAN ID/EVC map. In this case, CE-VLAN ID 1 is mapped to the blue EVC, CE-VLAN ID 2 is mapped to the red EVC, and CE-VLAN ID 4094 is mapped to the green EVC. A tagged ingress service frame with VLAN ID 2 will be delivered by the MEN according to the properties of the red EVC. An untagged ingress service frame will also be delivered by the MEN according to the properties of the red EVC. A tagged ingress service frame with VLAN ID 1 will be delivered by the MEN according to the properties of the blue EVC. In this example, an ingress service frame with VLAN ID not equal to 1,2, or 4094 will be discarded because the associated CE-VLAN ID is not mapped to an EVC. An egress service frame from the blue EVC will be tagged with VLAN ID 1. An egress service frame from the red EVC could apparently have three formats, namely, tagged with VLAN ID 2, untagged, or priority tagged. If CE-VLAN ID Preservation (see Section 10.3.1) is not in force, the egress service frame is untagged. Section 10.3.1 describes the format when CEVLAN ID preservation is in force.
10.3.
SERVICE FEATURES
The basic constructs of the service model described in Section 10.2 provide a flexible foundation for the definition of a number of services. The details of these various services are determined by the service features that are described in this section.
350
10.3.1
Chapter 10
CE-VLAN ID Preservation
If a subscriber attaches IEEE 802.IQ bridges as CEs to the UNIs in an EVC, it is possible that the subscriber will want to maintain his or her VLAN structure among the sites at the UNIs. In this case, the CE will generate tagged service frames with VLAN IDs that correspond to the subscriber's VLANs. It would be very inconvenient if the MEN changed those VLAN IDs, since the subscriber would be forced to use different VLAN ID values for the same VLAN and coordinate with the service provider in setting those values. To address these issues, the service provider could offer CE-VLAN ID preservation. When an EVC has this feature, the relationships between ingress and egress service frames shown in Table 10-1 are maintained by the MEN. Table 10-L CE-VLAN ID preservation Ingress Service Frame Untagged J
Egress Service Frame Untagged Tagged with VLAN ID equal to that of the Ingress Service Frame
Clearly, CE-VLAN ID preservation is useful in situations like that above. However, as we shall see in Section 10.3.3, there are other scenarios where it is valuable to not have this feature.
10.3.2
AU-to-One Bundling Map
For many subscribers, all that is required is a single service per UNI needing minimal configuration of the CE. This has been the typical configuration for the Transparent LAN services that have been offered by service providers since the early 1990s. The All-to-One Bundling Map is the way to achieve this scenario. With the All-to-One Bundling Map, the CE-VLAN ID/EVC map is configured such that all CE-VLAN IDs map to a single EVC at the UNI. In addition, the EVC must have the CE-VLAN ID preservation feature enabled. Since all CE-VLAN IDs are mapped to a single EVC, it follows that there can only be one EVC at the UNI. Figure 10-4 shows an example of an All-to-One Bundling Map. As can be seen in the figure, any ingress service, no matter what the format, is mapped to the red EVC. In addition, CE-VLAN ID preservation means that the format and VLAN ID, if tagged, will not be changed by the MEN. Note that there is no need for the subscriber and service provider to coordinate the use of the VLAN ID values in tagged service frames.^
Ethernet Services Over Metro Ethernet Networks
Untagged Priority Tagged Tagged, VID = 1 Tagged, VJD = 2
Tagged, VID = 4094 Tagged, VID = 4095
351
CE-VLAN IP
-4095 CE-VLAN ID/EVC Map
Figure 10-4. Example of All-to-One Bundling Map
The high degree of transparency with the All-to-One Bundling Map makes the resulting service much like a dedicated private Ethernet. When the EVC is Point-to-Point, the result is easily substituted for an existing private line such as a DS3. When the EVC is multipoint-to-multipoint, the result is much like the classical Transparent LAN service. Figure 10-5 illustrates an example of how these two configurations could be used by an enterprise to connect headquarters to the branches and to connect headquarters to a disaster recovery service provider. The multipointto-multipoint EVC service can be thought of as a backbone LAN connecting the sites of an enterprise and is frequently called LAN Extension. Disaster Recovery Service Provider
Branch
Branch Bridge or Router
Private Line Replacement
Figure J 0-5. Examples of the use of All-to-One Bundling
LAN Extension
352
Chapter 10
103.3 Service Multiplexing In the example of Figure 10-5, a separate port on the CE is required for each EVC at the headquarters. In many cases, this setup is acceptable and even desirable. But when many EVCs need to be accessed at a given location, consuming a port on the CE for each can both be costly in equipment and yield complex physical arrangements. The way to avoid these disadvantages is to enable multiple EVCs on a UNI. Service multiplexing is the term used to describe the case where multiple EVCs can be made available at a UNI. The two approaches to service Multiplexing are described in the following subsections. 10.3.3.1
One-to-One Map
When the CE-VLAN ID/EVC map is such that at most one CE-VLAN ID is mapped to an EVC, it is called a One-to-One Map."^ Untagged Priority Tagged Tagged, VID = 1 Tagged, VID = 2 Tagged, VID = 3
Tagged, VID = 4094 Tagged, VID = 4095
• 4095 CE-VLAN ID/EVC Map
Figure 10-6. Example of a One-to-One Map
Figure 10-6 shows an example of a One-to-One Map. At this UNI, only ingress tagged service frames with VLAN ID 3 and 4094 will be delivered to the destination UNI(s). Egress service frames at this UNI will be tagged with only VLAN ID values 3 and 4094. When the EVCs are point-to-point, the resulting service is analogous to the Frame Relay. Figure 10-7 shows an example of the use of point-to-point maps that illustrates this analogy. The ISP router is able to use a single port, e.g., Gigabit Ethernet, to reach several ISP customers with a point-to-point EVC to each. The CE-VLAN ID/EVC map for each UNI is shown in the figure, and it can be seen that the CE-VLAN ID is different at each UNI in an EVC. In particular, each ISP customer uses CE-VLAN ID 2000 to identify the EVC to the ISP, while the CE-VLAN IDs used at the ISP UNI are 178, 179, and 180. In this case, not using CE-VLAN ID preservation is
Ethernet Services Over Metro Ethernet Networks
353
valuable. It makes it simple for the ISP customers to configure their routers by using a well-known VLAN ID value (2000) and still makes it possible for the ISP router to identify the EVC for each ISP customer. This setup is analogous to the Frame Relay where the DLCI that identifies the Permanent Virtual Connection (PVC) can be different at each UNI. Internet Service Provider 178 ^ E V C ^ 179^EVC2 180<->EVC 2000 ^ EVC^ ISP Customer 3 2000 <^ EVC ISP Customer 1 Router
2000 o EVC2 ISP Customer 2 Frame Relay PVC Replacement
Figure 10-7. Example of the use of a One-to-One Map Internet Service Provider Routers
ISP Customer 1 Router
ISP Customer 2 Redundant Service Access
Figure 10-8. Example of Redundant service Access
354
Chapter 10
The One-to-One Map can be used with any mix of point-to-point and multipoint-to-multipoint EVCs. Figure 10-8 shows an example of redundant access to an ISP. Service multiplexing is shown at each UNI to an ISP router. Routing protocols running among the ISP and ISP customer routers will control which ISP router is used for each customer and will reroute the packets to the redundant router should the first ISP router fail or otherwise become unreachable. 10.3.3,2 Bundling Map It is also possible to have service multiplexing and multiple CE-VLAN IDs map to an EVC. This configuration is called a Bundling Map, Figure 109 shows an example. In this example, CE-VLAN IDs 1 and 3 are mapped to the red EVC. Untagged Priority Tagged Tagged, VID = 1 Tagged, VID = 2 Tagged, VID = 3
CE-VLAN ID
EVC
Tagged, VID ~40M Tagged, ViD=s 4095 CE-VLAN ID/EVC Map
Figure 10-9. Example of a Bundling Map
The use that is envisioned for the Bundling Map is an enhancement of LAN Extension. Consider the LAN Extension shown in Figure 10-5 and suppose that IEEE 802. IQ bridges are used as CE. The service in Figure 105 is not aware of the VLANs being used by the enterprise. So, for example, a broadcast frame sent on a given VLAN will be delivered to all the enterprise sites only to be discarded by the CE if that VLAN is not present at a site. With a Bundling Map approach, traffic on a given enterprise VLAN can be directed to an EVC going to only the enterprise sites where the VLAN is present. This yields more efficient use of the WAN bandwidth at the cost of the subscriber and service provider having to coordinate the content of the CE-VLAN ID/EVC map. When there is a Bundling Map, an EVC that has more than one CEVLAN ID mapped to it must have the CE-VLAN ID preservation feature.
Ethernet Services Over Metro Ethernet Networks
355
This requirement is motivated by the envisioned use of a Bundling Map described in the previous paragraph.
103.4 Feature Constraints The CE-VLAN ID/EVC map provides a high degree of flexibiHty and is part of the power of Ethernet services. The price for this flexibiUty is complexity in the sense that not all configurations of the map can coexist. These constraints are described in the following subsections. 10.3.4.1 Maps at UNIs in an EVC If a UNI in a given EVC has an All-to-One Bundling Map, then all UNIs in the EVC must have an All-to-One Bundling Map. To see why this is necessary, consider a point-to-point EVC called X, with UNIs A and B and the CE-VLAN ID/EVC maps shown in Figure 10-10. If the CE at UNI A sends a service frame with CE-VLAN ID = 4002, the CE-VLAN ID preservation feature mandates that the resulting egress service frame at UNI B have CE-VLAN ID = 4002. But the CE at UNI B would not be able to determine whether this service frame came from EVC X or EVC Y. And any attempt to respond with a service frame with CE-VLAN ID = 4002 would result in the response being carried on EVC Y, not EVC X. The only way to avoid this mishandling of service frames is to have all UNIs in EVC X have an All-to-One Bundling Map. UNI A
CE-VLAN ID 1-4095
UNIB
EVC X
CE-VLAN ID 1 - 4000 4001-4095
EVC X Y
Figure 10-10. Example of broken CE-VLAN/EVC maps
10.3.4.2
Maps at a UNI
Some CE-VLAN ID/EVC map properties are not compatible with each other. Each column in Table 10-2 shows map properties that are compatible with each other. Note that an All-to-One Bundling Map means that only one EVC can be present at the UNI.
356
Chapter 10
Table J 0-2. Compatible map properties One-to-One Map Al!-to-One Map Bundling Map Multiple EVCs
Combination 1 v^
Combination 2 ^
Combination 3
Combination 4
Combination 5
V
^ ^
>/ V'
10.3.5 E-Line and E-LAN Service The MEF in [2] has defined two general service types, namely, Ethernet Line service (E-Line) and Ethernet LAN service (E-LAN). An E-Line service is any service based on a point-to-point EVC, while an E-LAN service is any service based on a multipoint-to-multipoint EVC. Table 10-3 shows how ELine and E-LAN relate to the service features described so far and how the various combinations can be used. Table 10-3. E-Line and E-LAN service types CE-VLANID/EVC Map EVC Type Characteristic Point-to-Point Mii!tipoint4o-MiiMpoint All-to-One Private Line Replacement LAN Extension One-to-One Frame Relay Replacement Redtindant access to semces Bundling VLAN Extension VLAM Extension E-Line E-LAN
10.3.6 Class of Service In the last few years, data networks based on IP have become a platform for both classical data applications such as email and also real-time applications such as Voice over IP. The integration of real-time applications has generated a requirement that data networks be able to differentiate packets from different applications and provide differentiated performance according to the needs of each application. In MEF Ethernet services, this differentiation is referred to as Class of Service. The following subsections describe the method of identifying different instances of Class of Service and the different performance for each class. 10.3.6.1
Identifying Class of Service
A service provider is likely to offer several Classes of Service. For example, three classes might be offered: • One intended for non-time-critical data such as electronic mail, • One intended for time-critical data such as nuclear reactor sensor data, and
Ethernet Services Over Metro Ethernet Networks
357
•
One intended for real-time applications such as Voice over IP. However, the number of instances of Classes of Service for a given subscriber could be much larger. For example, pricing considerations might cause a service provider to charge differently for a given Class of Service between different end points. New York to Boston may well be priced less than New York to London. To accommodate this flexibility, MEF services allow identification of a Class of Service instance for a service frame by either the EVC or by the combination of the EVC and the userjpriority field in tagged service frames. In the latter case, a set of user__priority field values can identify the Class of Service instance. The EVC or combination of EVC and set of user_priority field values is called a Class of Service Identifier. As an example, consider the CE-VLAN ID/EVC map shown in Figure 10-6 and suppose that the red EVC has just the Standard Class of Service while the blue EVC has both the Standard and the Premium Classes of Service. Then the Class of Service for tagged service frames could be as shown in Table 10-4. Table 10-4. Example of Identifying Class of Service CE VLAN ID userpriority 3 0,1,...,7 4094 0,1,2,3 4094 4, 5, 6, 7
10.3.6.2
Class of Service Standard Standard Premium
Performance Parameters
The point of having different Classes of Service is to provide different performance to service frames according to each frame's Class of Service. The MEF [1] specifies three types of performance parameter: • Frame Delay: The length of time it takes a service frame to go from the ingress UNI to the egress UNI. • Frame Delay Variation: The variation in frame delay for service frames with the same Class of Service instance. • Frame Loss: The success or failure of delivery of a service frame to the egress UNI. The precise definitions of these parameters for a point-to-point EVC are contained in [1]. Performance for a multipoint-to-multipoint EVC will be addressed by the MEF in the future. The MEF is not planning on specifying values for these parameters that service providers should meet. Rather, by standardizing the definition of the parameters, they can be used as a basis for service-level specifications that will allow a subscriber to compare service offerings and negotiate a meaningful service level agreement with a service provider.
Chapter 10
358
10.3.6.2.1 Frame Delay Performance Objective for a Point-to-Point EVC The frame delay for a service frame is defined as the time elapsed from reception at the ingress UNI of the first bit of the ingress service frame until the transmission of the last bit of the service frame at the egress UNI. Figure 10-11 illustrates this definition. Of course, this definition is only meaningful for a service frame that is delivered.
Service Frame Direction
Time
First bit in
Last bit out
Figure 10-11. Frame delay for a service frame The frame delay performance Objective is defined by the three parameters in Table 10-5. A point-to-point EVC meets the frame delay performance Objective for the interval T if at least P percent of the service frames that arrive at an ingress UNI during the interval that have a green bandwidth profile compliance level (see section 10.3.7) and that are delivered, have delay less than or equal to d. Table 10-5. Parameter T P _d_
Parameters used to define frame delay performance objective Description Time interval during which service frames arrive at an ingress UNI (time units) Percentage Delay objective (time units)
Ethernet Services Over Metro Ethernet Networks
359
10.3.6.2.2 Frame Delay Variation Performance Objective for a Point-toPoint EVC Frame delay variation is defined for pairs of delivered service frames as the delay of the first service frame minus the delay of the second service frame. The Frame delay variation performance objective is defined by the four parameters in Table 10-6. A point-to-point EVC meets the frame delay variation performance objective for the interval T if at least P percent of the pairs of service frames that arrive at an ingress UNI during the interval, that have a green bandwidth profile compliance level (see 10.3.7), that arrive / time units apart, and that are delivered, have a frame delay variation of less than or equal to v. Table 10-6. Parameter T P / _v;
Parameters used to define frame delay variation performance objective Description Time interval during which service frames arrive at an ingress UNI (time units) Percentage Time between arrival of two service frames at an ingress UNI (time units) Delay variation objective (time units)
10.3.6.2.3 Frame Loss Performance Objective for a Point-to-Point EVC A service frame is defined as lost if it should have been delivered but was not.^ The frame loss performance objective is defined by the two parameters in Table 10-7. A point-to-point EVC meets the frame loss performance objective for the interval T if no more than L percent of the service frames that arrive at an ingress UNI during the interval and that have a green bandwidth profile compliance level (see 10.3.7) are lost. Table 10-7. Parameter T J^
Parameters used to define frame loss performance objective Description Time interval during which Servicefi-amesarrive at an ingress UNI (time units) Loss objective (percentage)
10.3.7 Bandwidth Profiles Most UNIs will have a line rate of at least 100 Mbps. Yet many subscribers will not need and will not want to pay for that much bandwidth. The vehicle for letting subscribers pay for the bandwidth they need is the bandwidth profile.
Chapter 10
360
The bandwidth profile is a characterization of the lengths and arrival times for ingress service frames at a UNI. When a bandwidth profile is applied to a sequence of service frames, each service frame is classified according to its level of compliance with the bandwidth profile. The bandwidth profile also includes how the MEN should treat a service frame depending on its level of compliance. 10.3.7.1 Parameters and Algorithm The bandwidth profile is the Ethernet version of the Frame Relay committed information rate or the ATM sustained cell rate. It defines longterm average bandwidths as well as limits on the amount of data in back-toback service frames. Reference [1] defines the service frame size and arrival time characterization with six parameters: 1. Committed Information Rate (CIR): a nonnegative number expressed as bits per second. 2. Committed Burst Size (CBS): a nonnegative number expressed as bytes. When CIR > 0, CBS must be at least as large as the maximum length of a service frame. 3. Excess Information Rate (EIR): a nonnegative number expressed as bits per second. 4. Excess Burst Size (EBS): a nonnegative number expressed as bytes. When EIR > 0, EBS must be at least as large as the maximum length of a service frame. 5. Coupling Flag (CF): a binary variable with value either 0 or 1. 6. Color Mode (CM): a binary variable with value either "color blind" or "color aware." The description of the level of compliance determination is via two token bucket algorithms, as shown in Figure 10-12. Committed Information Rate Tokens Overflow Commrttecl Burst Size
Excess ^ Information ^ Rate Tokens Overflow
Excess Burst Size C-Bucket
E-Bucket
Figure 10-12. Graphical depiction of bandwidth profile
Ethernet Services Over Metro Ethernet Networks
361
Each bucket holds up to CBS and EBS tokens, respectively, with one token representing one byte. Tokens are added to each bucket at the rate of CIR/8 and EIR/8 respectively. When the bucket becomes full, additional tokens overflow and are lost. Since each token is a sort of permission to the CE to send one byte of data, this loss of a token is a "use it or lose it" mechanism for bandwidth use. When an ingress service frame is classified, its length is compared with the tokens in the C-bucket. If the number of tokens is at least equal to the length of the service frame, the frame is declared "green," and tokens equal to the frame length are removed from the bucket. If there are not sufficient tokens in the C-bucket, the service frame length is compared with the tokens in the E-bucket and the process is repeated. If there are sufficient tokens in the E-bucket, the frame is declared "yellow"; otherwise, it is declared "red." The precise algorithm is shown in Figure 10-13. Here the Service frames have lengths {/,}, and arrival times {/,} fory = 0, 1, .... The number of tokens in each token bucket is denoted by Bc(tj) and Be(tj), respectively. Finally, Bc(to) = CBS and Be(to) = EBS. When CF = 1, it is possible that unused tokens from the C-bucket can be put into the E-bucket. This could happen if EIR is smaller than CIR and a long burst of service frames is followed by a period of no traffic. At the end of the burst, both token buckets would be essentially empty. During the quiet period, the C-bucket could completely fill before the E-bucket, resulting in the "overflow" tokens (represented by 0{tj) in Figure 10-13) being placed in the E-bucket. When CF = 0, the two token buckets operate independently. Service Frame of length Ij arrives at tj
B/tj) = min{CBS, B/tj_j)+CIRx(tf-tj_j)/8} 0(tj) = max[0, B/tj_j)+CIRx(tj-tfJ/8-CBS\ B/tj) = mm{EBS, B/tjJ+EIRxftj-tjJ/S+CFxOdj)} JilL.
Ij^B/tj)
I
Ij^B/tj) Yes
Yes Declare frame green B/tj)=^B/tj)^lj
No Declare frame red
Declare frame yellow B/tj)-B/tj)^lj
Figure 10-13. Bandwidth profile algorithm
362
Chapter 10
The algorithm defined in Figure 10-13 is referred to as color blind because it does not take into account a color that might be associated with each service frame as it crosses the UNI. Reference [1] also specifies a color aware version of the algorithm, and this is where the Color Mode parameter is used. This is done in anticipation of the specification of a standard way to indicate the color in a service frame at the UNI. Such a standard does not yet exist. 10.3.7.2
Disposition of Service Frames
The classification of each service frame governs how the MEN will handle the frame. Reference [1] specifies the following MEN actions: 1. Green: Deliver the service frame according to the service attributes of the service instance and the service level specification performance objectives (see Section 10.3.6.2). 2. Yellow: Deliver the service frame according to the service attributes of the service instance, but service level specification performance objectives do not apply. 3. Red: Discard the service frame. An example will help to understand the implications of the above points. Consider an enterprise connecting two locations together with a point-topoint EVC using an All-to-One Map at both UNIs, and further suppose that both UNIs are 100 Mbps. The enterprise has determined that the average bandwidth needed is 20 Mbps and that less than 0.1% service frame loss is needed for adequate performance. In this case, the parameter values shown in Table 10-8 could be used along with an SLA that guarantees less than 0.1% service frame loss. CIR is set equal to 20 Mbps and CBS is large enough to handle more than seven maximum-sized service frames. Given the bandwidth needs of the enterprise, most of the Service frames will be declared green and should have less than .1% service frame loss. Table 10-8. Example bandwidth profile parameter values Parameter Value CIR 20 Megabits/sec CBS 15,000 bytes EIR 20 Megabits/sec EBS 6,000 bytes CF 0
Ethernet Services Over Metro Ethernet Networks 10.3.7.3
363
Application of Bandwidth Profiles
There are three ways that a bandwidth profile can be appUed to ingress service frames. These are described in the following subsections. 10.3.7.3.1 Per Ingress UNI In this case, a single bandwidth profile applies to all ingress service frames at the UNI. Figure 10-14 illustrates this approach. A single bandwidth profile is applied to all service frames irrespective of the EVC or Class of Service identifier.
n> EVCi
UNi
/( CE-VLAN Cos 0,1,2.3 (
CE-VLANCOS4.5
\{
CE'VLAN Cos 6 J
Bandwidth Profile
EVCo
Figure 10-14, Example of bandwidth profile per ingress UNI
10.3.7.3.2 Per EVC In this case, a single bandwidth profile applies to all ingress service frames on a given EVC. Figure 10-15 shows two bandwidth profiles, one for each of the two EVCs.
364
Chapter 10 f'\'^'M'-^P^/
D EVC,
um 1I ,1 EVC^
IJ
%$itB-^
1
E-VLAN Cos 0.1^.3 CE«VLAN Cos 4,5 CE-VUANC0S6J
\
3) 1
! Bandwidth Profile 1
i
1 Banciwicith Profile 2
[]
i
Figure 10-15. Example of bandwidth profile per EVC
10.3.7.3.3 Per Class of Service Identifier The most granular application of bandwidth profiles is per Class of Service Identifier. This apphcation is illustrated in Figure 10-16.
CE^VLAN Cos 0.1.211) CE-VLAN Cos 4.5 (
CE-VLANCOS6.7
T Bandwidth Profile 1 f Bandwidth Profile 2 " y Bandwidth Profile 3
Bandwidth Profile 4
Figure 10-16. Example of bandwidth profile per Class of Service ID
10.3.7.3.4 Constraints on Bandwidth Profiles As we will see in Section 10.3.7.4, when a bandwidth profile is used, it is important to configure the CE to smooth the traffic in a way that complements the bandwidth profile. Most routers have the capability to do this for a single bandwidth profile. However, few routers can smooth to complement the case when multiple bandwidth profiles are applied. Therefore, to be consistent with existing CEs and to simplify implementations, the application of bandwidth profiles is constrained. At most one bandwidth profile can be applied to an ingress service frame.
Ethernet Services Over Metro Ethernet Networks
365
This constraint on the application of bandwidth profiles leads to the following rules at a UNI: 1. If there is a per-UNI bandwidth profile, then there cannot be any other bandwidth profiles. 2. If there is a per EVC bandwidth profile on an EVC, then there cannot be any per CoS bandwidth profiles for instances of Class of Service on the EVC. Figure 10-14, Figure 10-15, and Figure 10-16 show examples of bandwidth profiles that are applied according to these rules. 103.7.4
Configuring CE
It has been well known since the rise in popularity of Frame Relay that the use of a bandwidth profile type of bandwidth constraint can lead to excessive frame loss and poor performance for an application using the service. For example, consider a host in an enterprise network well removed from the UNI that is carrying out a file transfer. The source host can send a TCP window of packets back-to-back. When the corresponding frames reach the UNI, if they cause the token bucket to exhaust, service frames will be discarded. The resulting lost packets will be detected by TCP in the host and retransmitted. However, TCP also slows down when loss is detected and the net result is throughput degradation. There are two ways to suppress this behavior. The first is to make CBS large. The second is to implement smoothing in the CE at the UNI. Smoothing buffers a service frame that would be declared yellow or red until tokens have accumulated to allow the service frame to be declared green. Such smoothing has been common in routers since the early days of Frame Relay. It is important to properly configure the smoothing in the CE to align with the bandwidth profile parameters so as to allow the most effective use of an Ethernet service.
10.3.8 Layer 2 Control Protocols A significant number of protocols have been standardized for use by bridges within an enterprise network. These are referred to as Layer 2 control protocols. When a CE is a bridge, it is likely to generate some of these protocols. It is important to know how the Ethernet service handles a service frame that is carrying a Layer 2 control protocol. As an example, consider the configuration of Figure 10-17.
366
Chapter 10
Bridge |
Bridge^
MBridge
Figure 10-17. Example of the need for Spanning Tree Protocol tunneling
In Figure 10-17, a loop is formed by the three EVCs, and there is a danger that a broadcast frame sent into an EVC on one of the UNIs would circulate forever, with dire consequences for the subscriber. To prevent this behavior, the subscriber would typically configure the bridges to run the Spanning Tree Protocol. This protocol would block some of the ports on the bridges to break the loop. But what happens in the case of Figure 10-17? If the service frames carrying the Spanning Tree Protocol are discarded by the MEN, each bridge will conclude that the ports attached to the MEN are not attached to other bridges and all ports would be in forwarding state creating a loop. On the other hand, if the Service frames carrying Spanning Tree Protocol are delivered from one UNI to the other, the bridges would break the loop by blocking the appropriate ports. 10.3.8.1
Handling Layer 2 Control Protocols
There are three ways that an MEN can deal with a Layer 2 control protocol: 1. Discard services frames carrying the protocol, 2. Participate as a peer of the CE, or 3. Deliver unchanged (called tunneling) to appropriate UNIs. When there is more than one EVC at a UNI, tunneling becomes complicated by the question of which EVC to use to carry the tunneled service frames. A recommendation on how to handle various Layer 2 control protocols can be found in [2].
Ethernet Services Over Metro Ethernet Networks 10.3.8.2
367
Bridges versus Routers as CE
When bridges are used as the CEs, it is important to tunnel most of the Layer 2 control protocols. In view of the complexity of tunneling when multiple EVCs are at a UNI, it is the author's recommendation that AU-toOne Bundling be used at all UNIs when bridges are used as CEs. In this case, protocols such as Spanning Tree should be tunneled When it is desirable to have multiple EVCs at a UNI, it is the author's recommendation that routers be used as CEs and Layer 2 control protocols be discarded by the MEN. There are many configurations that will work well and are not consistent with the above recommendations. However, for most uses of Ethernet services, the above recommendations provide easy-to-understand guidelines and will enable the subscriber to realize the full value of Ethernet services.
10.4.
CONCLUSION AND FUTURE WORK
The services specified in [1] establish a solid foundation upon which valuable Ethernet services can be deployed by service providers and used by subscribers. Such services are currently being offered and planned in all parts of the world. However, the MEF realizes that this is just the starting point (which explains the use of "Phase 1" in the title of [1]). Consequently the MEF has embarked on "Phase 2" services. As of this writing, the scope of Phase 2 has not been finalized, but it is likely to include: • Definition of performance parameters for multipoint-to-multipoint EVCs, • Specification of a point-to-multipoint EVC, which can be useful for supporting Internet access for a large number of small customers, • Additional ways to identify CoS, such as by a set of CE-VLAN IDs, and the IP DSCP field, • Egress bandwidth profiles, and • Alignment with the emerging IEEE 802.lad, Provider Bridges standard.
10.5. APPENDIX A: ETHERNET BASICS This Appendix provides a quick tutorial on the basics of Ethernet. Ethernet emerged as a way for multiple computers to connect to a single cable segment (called a shared medium) and to exchange information amongst themselves at high speeds, e.g., 10 Mbps. The protocol that each computer had to execute is referred to as CSMA/CD and can be thought of as
Chapter 10
368
being modeled on a meeting of polite, extroverted, fast talkers. Any computer can send but refrains from doing so if energy is detected on the cable (Carrier Sense Multiple Access). If two computers happen to start sending at the same time, each detects the collision and stops sending for a random period of time (Collision Detection). Each computer receives all frames transmitted and discards them if the destination MAC address in the frame does not match the computer's MAC address and is not a broadcast/multicast address. Of course, Ethernet networking has evolved greatly since these early days. Today, the vast majority of Ethernet networks are composed of computers and routers attached directly to Ethernet bridges (also known as switches). The links are usually full duplex so both the devices on the link can transmit and receive simultaneously. The bridges forward the Ethernet frames to the proper destination(s). From the point of view of Ethernet services, then, the CE connects to the UNI via a point-to-point cable and exchanges Ethernet frames with the service provider network. In the following, we dig a bit deeper into Ethernet as seen by the CE.
10.5.1 Ethernet Physical Layers There are a large number of Ethernet physical layers defined in [3]. Several have fallen into disuse in the market. Others are unlikely to be supported at the UNI by service providers. Table 10-9 summarizes the physical layers that are most likely to be supported at the UNI.^ Table 10-9. Physical layers likely to be used at the UNI Name Media Type Speed lOBASE-T 2 pair Category 3 UTP 10 Mbps lOBASE-FL 62.5 ^m MMF 10 Mbps 100BASE-TX 2 pair Category 5 UTP 100 Mbps lOOOBASE-T 4 pair Category 5 UTP 1000 Mbps lOOOBASE-SX 50 and 62.5 |iim MMF 1000 Mbps lOOOBASE-LX 50 and 62.5 |Lim MMF 1000 Mbps lOOOBASE-LX lO^mSMF 1000 Mbps
Maximum Length 100 meters 2km 100 meters 25 meters 220 - 550 meters 550 meters 5km
UTP — Unshielded Twisted Pair; MMF — Multimode Fiber; SMF — Single-Mode Fiber
10.5.2 Ethernet Media Access Control Layer The frame format for Ethernet is part of the media access control layer because it contains the information that is needed for the operation of CSMA/CD. Figure 10-18 shows the format.
Ethernet Services Over Metro Ethernet Networks
b^
b^
7 octets 1 octet 6 octets 6 octets 2 octets 46-1500 octets 4 octets
b^
b^ b^ b^ Preamble Start Frame Delimiter Destination MAC Address Source MAC Address Length/Type
369
b^
Data and Pad Frame Check Sequence
Figure 10-18. Ethernet media access control frame format
The bits of each octet are transmitted left to right (least significant to most significant) and the octets are transmitted top to bottom. Each field is briefly described below. 10.5.2.1
Preamble Field
This field allows a receiver to synchronize a receive clock. This synchronization is necessary in a shared media network where there can be periods with no transmissions and multiple transmit clocks, one for each other device on the LAN. 10.5.2.2
Start Frame Delimiter Field
This special bit pattern allows the receiver to find the start of the frame. 10.5.2.3
Destination Address Field
This field specifies the destination device or devices. The first bit indicates whether the address is an individual address or a group (multicast) address. The second bit indicates whether the address is globally or locally administered. Globally administered addresses are allocated to manufacturers and are usually "burned into" devices during manufacturing. The all ones address indicates a broadcast to all devices on the LAN. 10.5.2.4
Source Address Field
This field identifies the source of the frame.
370 10.5.2.5
Chapter 10 Length/Type Field
Depending on the value, this field can either indicate the number of data octets or the type of protocol running above the MAC layer. Today, most implementations use this field for protocol identification. 10.5.2.6
Data and PAD Field
This field contains the data that is being carried. If there are less than 46 octets of data, PAD bits are added to extend the field to be 46 octets. 10.5.2.7
Frame Check Sequence Field
This field is a CRC calculation that covers from the beginning of the Destination Address field through the end of the Data and PAD field. It is used to detect transmission errors.
10.5.3 Ethernet VLANs IEEE 802. IQ [4] added two new concepts to Ethernet, namely the virtual LAN and priority. The virtual VLAN concept allows an Ethernet to be logically divided into multiple LANs. For example, a large LAN or complex of LANs bridged together can be divided up into multiple (smaller) VLANs and a separate IP subnet assigned to each. In this case, from the point of view of a router, each VLAN operates just like a separate physical LAN. A broadcast frame sent on a given VLAN will only be propagated to and received by the devices assigned the VLAN. In addition, [4] defines up to eight levels of priority. Bridges compliant with IEEE 802. IQ can buffer and forward frames according to the different levels of priority. Figure 10-19 shows the Ethernet MAC frame format with the IEEE 802.IQ Tag Control Information. Four octets are inserted between the Source Address field and the Data field. The 802.IQ TagType field indicates that the Tag Control Information field follows, as opposed to the Data field.
Ethernet Services Over Metro Ethernet Networks
b^
b^ b^ b^ Preamble Start Frame Delimiter Destination MAC Address Source MAC Address 802.1QTagType Tag Control Information Length/Type
7 octets 1 octet 6 octets 6 octets 2 octets 2 octets 2 octets
46-1500
371
b^
Data and Pad
octets 4 octets
Frame Check Sequence
Figure 10-19. Ethernet media access control frame with tag control information
Figure 10-20 shows the details of the 802.IQ TagType field and Tag Control Information field. The 802. IQ TagType field always has the value 8100 Hex. The VLAN Identifier has 12 bits. The value 0 has a special meaning. It does not identify a VLAN but is used as a way to assign a priority to the frame. When the value is 0, the frame is called a priority tagged frame. The value FFF Hex (all ones) is reserved. Thus 4094 VLANs can be identified. The user_priority field contains the priority of the frame and has eight possible values. The Canonical Format Indicator (CFI) field is intended for use with Token Ring networks.
2 octets
1 0
2 octets
b^ 0 0 0 0 VLAN
b' 0 0
0 0 0 11 0 0 0 01 1 CFI 1 user priority Identifier (12 bits)
Figure 10-20. Tag type and tag control information
10.6.
NOTES
1. The reader may have heard about "double tagged" Ethernet frames in Metro Ethernet Networks. Such double tagging is frequently used internally in MENs. But double tagging is not standard and thus is not to be done at a UNI. 2. An Ethernet frame with an IEEE 802.IQ tag that has zero as the VLAN ID is called priority tagged.
372
Chapter 10
3. [4] specifies that the VLAN ID value 4095 is reserved, and thus an IEEE 802. IQcompHant CE should not generate a tagged service frame with this VLAN ID value. However, there seems to be no advantage in the service provider enforcing this constraint, and thus 4095 is allowed. Note, however, that the only mandate for the service provider to deliver such service frames is when CE-VLAN ID preservation is in force. 5. The One-to-One terminology is not used in [1], but it is a typical and important case for the CE-VLAN ID/EVC map. 6. Not all ingress service frames should be delivered. For example, ingress Pause frames should probably not be delivered. See Section 10.3.8. 7. 10 Gbps Ethemet standards exist, but we don't expect them to be supported at the UNI in the near term.
10.7.
REFERENCES
[1] Metro Ethemet Forum, Technical Specification MEF 10, Ethernet Services Attributes, Phase 1, 10 November 2003, http://www.metroethemetforum.org. [2] Metro Ethemet Forum, Technical Specification MEF 6, Ethernet Services Definitions Phase I, June 2004, http://www.metroethemetforum.org. [3] IEEE Project 802.3 — 2002, Information technology — Telecommunications and information exchange between systems —Local and metropolitan area networh Specific requirements — Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, 8 March 2002. [4] IEEE Project 802. IQ — 1998, IEEE Standards for Local and Metropolitan Area Networks: Virtual Bridged Local Area Networks, 8 December 1998.
Chapter 11 ETHERNET SERVICES OVER PUBLIC WAN
*Steven S. Gorshe, Ph.D and **Nevin R. Jones, Ph.D * PMC-Sierra, Inc., **Agere Systems, Inc.
11.1.
INTRODUCTION
11.1.1 Why Ethernet over the public WAN? A convergence of technology and applications has created an increased desire for Ethernet WAN connectivity. On the enterprise side, Ethernet is the dominant technology. Enterprise network administrators' familiarity and comfort with Ethernet creates a natural desire that their WAN technology also be based on Ethernet. A number of applications, including voice and video, are already being encapsulated into Ethernet and rim over their LANs. Another trend is for companies to have a number of remote offices that need to exchange data with the headquarters office or with each other. There can be significant advantages to placing data on common servers that can be accessed by all sites. The increased importance of the Internet for business applications has created an increasing desire for incrementally higher WAN interface rates. It is also common to want a single Internet firewall at one site that is accessed on the enterprise network by all sites. Most enterprise WAN data access today uses Frame Relay connections at rates of fractional DSl, full-rate DSl, fractional DS3, and full-rate DS3. In some cases, the data services use ATM instead of Frame Relay either for the customer connection or as a method of transporting the data through the core network. The Frame Relay-based services suffer from a lack of scalability in
374
Chapter 11
terms of both high-speed connections and ubiquitous multipoint networks. ATM suffers from being somewhat provisioning-intensive in large networks and bandwidth inefficiency. At the other end of the spectrum, some customers use WDM or dark fiber for their WAN or MAN connections with native Gigabit Ethernet, 10 Gbit Ethernet, or Fibre Channel. However, WDM equipment is expensive and not ubiquitously deployed, and dark fiber is not universally available, which limits the applications for these approaches. Another drawback of WDM and dark fiber applications is that unless they use the new G.709 [1] Optical Transport Network standard, there is no embedded overhead for the carrier to monitor the quality of the connection or to provide protection switching in order to guarantee their service level agreements (SLAs). Some larger customers use high-speed SONET OC-N / SDH STM-N connections that have typically terminated the Ethernet frames and re-encapsulated the data into PPP and use Packet over SONET/SDH (PoS) for transmission through the SONET/SDH pipe. On the carrier (network operator) side, the situation is somewhat complicated. In the wake of the bursting of the telecom "bubble" in the year 2000, the carriers face two clear drivers. First, they must deploy any new services on their existing SONET/SDH networks. Throughout the 1990s, carriers invested heavily in building SONET/SDH-based fiber optic networks, including developing the OAM&P systems to run them. The enormous capital investment required to build an overlay network for data transport, be it native Ethernet or DWDM based, makes overlay networks unrealistic. Fortunately, SONET/SDH has proven flexible enough to handle the challenge. As discussed in earlier chapters, recent developments that extend the useful life of SONET/SDH include virtual concatenation (VCAT) [2, 3] to create more flexible data channel sizes, Link Capacity Adjustment Scheme (LCAS) [4] to increase the flexibility and functionality of VCAT, Generic Framing Procedure (GFP) [5, 6] to encapsulate data frames for robust and bandwidth-deterministic transport, and the Resilient Packet Ring (RPR) that provides a very flexible technology for data access onto SONET/SDH access and metro rings. The second driver for the carriers is the desire to generate new, revenuebearing services. With data traffic continuing to grow faster than voice, offering WAN connectivity with higher bandwidth and enhanced service capabilities appears to be a natural direction for new services. Equipment and device manufacturers, who are also interested in building new products, have also seen WAN connectivity as a natural direction. Ethernet is in many ways an obvious technology choice for WAN connectivity. The job of enterprise network administration is simplified if all their sites can be treated as part of the same Ethernet network. Also, Ethernet interfaces are relatively inexpensive and are ubiquitous on
Ethernet Services Over Public WAN
31S
enterprise equipment. The development of VCAT and GFP allow efficient transport of Ethernet frames through SONET/SDH networks. Also, Ethernet bridge and router technology requires less provisioning and administration than technologies such as ATM. As some carriers begin a migration to using packet switching technology such as MPLS in their core networks rather than traditional circuit switching, Ethernet again looks like an excellent fit as an access technology. The relatively low cost of Ethernet enterprise equipment, however, has created a customer expectation that Ethernet WAN interfaces should be less expensive than DS1/DS3 Frame Relay interfaces. For the carriers, even if the Ethernet connectivity is provided through their SONET/SDH backbone network, they still have to develop new OAM&P procedures and tools to provide the service. In fact, although it is being actively addressed by multiple standards bodies, Ethernet currently lacks some of the OAM capabilities that will be required to monitor and guarantee the SLAs. So, even though customers and carriers agree that Ethernet (over SONET/SDH for the carriers) is the most attractive technology for highspeed WAN connectivity, carriers will still have to invest in equipment and management system upgrades while the customers demand and expect to pay less for the Ethernet-based services.
11.1.2 Organization of the chapter The information in this chapter primarily focuses on the output of the Ethernet transport work in ITU-T Study Group 15 (SGI5) and its series of new standards. SGI5 is the ITU-T body responsible, among other things, for standards related to public transport networks. Ethernet services in metro networks are the topic of Chapter 10. The status of standards that are currently approved or are in progress is summarized in Section 11.1.3. The chapter begins with a discussion and description of the Ethernet services that are being considered over public WANs. This description includes an examination of the different service characteristics and attributes that must be defined in order to provide the services. The next section explores different transport network models that can be used to support Ethernet services. Next comes a description of the Ethernet-based user network interfaces (UNIs) followed by a discussion of the network-tonetwork interface (NNI) that will be required for transport network equipment carrying Ethernet services. OAM is the topic of the following section. As already noted, some amount of OAM capability comes inherently from deploying Ethernet services over SONET/SDH (server) networks. Additional OAM capabilities will be required at the Ethernet client layer. The final section summarizes some of the protection and
376
Chapter 11
restoration technologies that can be used to guarantee carrier-grade service rehabihty for Ethernet WAN services. Table 11-1. Summar>f of related standards activities
Organization IEEE
Activities
Status
802.3ae
10 Gbit Ethernet, which included a WAN PHY interface to simplify interfacing to a SONET/SDH or G.709 OTN network Resilient Packet Rings: Working on a ring-based network for access and metro applications Ethernet in the First Mile, where work includes 0AM aspects for Ethernet Links, especially access links Provider Bridge specification - This is the Q-in-Q standard.
Approved
802.17 802.3ah (EFM) 802.1 ad
802. lag 802. lae ITU-T SG15 G.8011.1 (Q12) G.8012 (Qll) G.8010 (Q12)
1
Connectivity Fault Management, or Ethernet Service OAM MAC Security (MacSec), including authentication, authorization, and encryption (With input from ANSI TlXl) Ethernet Private line service
Approved Approved In progress
In progress In progress | Approved
Ethernet UNI and Ethernet Transport NNI
Approved Approved
G.8021[7] (Q9)
Ethernet Layer Network Architecture, which is largely to translate the IEEE 802 network material into ITU-T transport network terminology and models Ethernet over Transport — Ethemet Service Characteristics Ethemet over Transport—Ethemet Service Multiplexing, which will cover the multiplexing protocol(s) required to implement EVPL and EVPLAN Characteristics of Ethemet transport network equipment functional blocks
Q2
Studying Ethemet OAM aspects relating to access
G.8011 (012) G.esm (Q12)
Approved In progress
Approved 2004 (focus onEPL portion) In progress
1 ITU-T SG13 Y.1730 Y.ethoam (Q3) Y.ethps |(Q3)
Requirements for OAM functions in Ethemet-based networks and Ethemet services End-to-end and edge-to-edge aspects of Ethemet OAM including PM Ethemet protection switching
Approved In progress In progress
Ethernet Services Over Public WAN Table 11-1 - continued Metro Ethernet Forum (MEF) MEF is studying various aspects of Ethernet MANs, including Ethernet architecture, service model, service definition, traffic management, UNI and NNI definition, and 0AM. MEF work is covering all possible OAM flows, such as end-to-end, edge-toedge, access, interprovider, intraprovider, etc. MEFl Ethernet Services Model, Phase 1 MEF2 Requirements and Framework for Ethernet Service Protection in Metro Ethernet Networks MEF3 Circuit Emulation Service Definitions, Framework and Requirements in Metro Ethernet Networks MEF4 Metro Ethernet Network Architecture Framework — Part 1: Generic Framework MEF5 ^ • Traffic Management Specification: Phase I UNI Type 1 Specification of UNI, data-plane aspects UNI Type 2 Specification of UNI, control-plane aspects (ELMI) EMS-NMS MIBS for Ethernet and network management MEF Specifies functional elements of Ethernet trail, such Architecture, part as adaptation, conditioning, etc. 2 CESPDH Implementation agreement of PDH CES over Ethemet. Includes both AALl and Raw method Internet Engineering Task Force (IETF) Working on defining an Ethemet transport over PWE3 WG IP/MPLS using Martini drafts. This is mainly EVPL service using UDP, L2TP, or MPLS as multiplexing layer Requirements for Virtual Private LAN Services PPVPNWG (VPLS) Working on framework and service requirements of L2VPNWG Ethernet-based VPN, and defining EVPLAN service using IP/MPLS.
111
Approved Approved Approved Approved Approved In progress In progress In progress In progress
In progress
In progress
In progress In progress
11.1.3 Related standards activity The current amount of standards activity is a good indication of how many companies and organizations see Ethemet WAN as the next key step both for Ethemet and for the pubhc transport network providers (i.e., carriers). The major standards activities are summarized in Table 11-1. Each standards organization has its own areas of expertise. The majority of the standards that will be required for the public transport network are being developed in the Q12 and Ql 1 groups of ITU-T SGI5. This work was partitioned not only logically by topic but also in a manner that allowed for
378
Chapter 11
the earliest possible approval of useful standards/recommendations. The initial set of standards was approved in mid-2004 (see Table 11-1). Those recommendations that will require more study and debate prior to consensus are targeted near the time of the publication of this volume. ITU-T SGI5 has established liaison contact with the other standards organizations and forums where their input is required or desired. For example, the G.ethsrv work is expected to use a considerable amount of input from the MEF regarding the definition of services. Multiple organizations are working on operation, administration, and maintenance (0AM) aspects of Ethernet MANs/WANs. 0AM is critical once Ethernet is extended beyond the customer premises, especially when multiple transport service providers carry the traffic. In a multiple carrier environment, for example, the 0AM is crucial for determining the locations of problems and degradations when they occur. From a transport network provider standpoint, this OAM requirement is an area where SONET/SDH really shines. The OAM capabilities inherent in the SONET/SDH backbone allow full monitoring and protection of the transmission facilities and transport path through the SONET/SDH network.
11.1.4 DeHnition of some technical terms in this chapter The sources of these definitions are the appropriate ITU-T Recommendations, as indicted at the end of the definition. 1. Access group: A group of co-locatedyZow termination functions that are attached to the same^/ow domain ox flow point pool link. (G.809) [8] 2. Characteristic Information (CI): A signal with a specific format, which is transferred onflows. The specific formats are defined in technologyspecific standards. (G.809) 3. Flow: An aggregation of one or more traffic units with an element of common routing. (G.809) 4. Flow domain: A topological component used to effect forwarding of specific characteristic information. (G.809) 5. Flow point: A reference point that represents a point of transfer for traffic units between topological components. (G.809) 6. Flow point pool: A group of co-located flow points that have a common routing. (G.809) 7. Flow point pool link: A topological component that describes a fixed relationship between a flow domain or access group and another flow domain or access group. (G.809) 8. Flow termination: A transport processing function. There are two types of flow termination, namely, a flow termination sink and a flow termination source. (G.809)
Ethernet Services Over Public WAN
379
9. Link flow: A transport entity that transfers information between ports across a flow point pool link. (G.809) 10. Topological component: An architectural component, used to describe the transport network in terms of the topological relationships between sets of points within the same layer network. (G.809)
11.2.
SERVICE TYPES AND CHARACTERISTICS
The description of Ethernet transport services varies depending on one's vantage point. This chapter approaches Ethernet transport from the perspective of the transport network or service provider, while the customer view of Ethernet transport services is presented in Chapter 9. This chapter follows the approach of ITU-T Rec. G.8011 [9] in its discussion of the Ethernet service types and characteristics of Ethernet from the transport network / service provider viewpoint. The G.8011.X series covers specific services within the framework of G.8011. One can think of the goal of the network provider as making the Ethernet service look like an extension of the customer's Ethernet LAN. In order to provide this transparency, the network provider view must take into account a number of items that are not necessarily directly visible to the customer. These items are presented in this section in terms of Ethernet connection attributes and their associated parameters. As described in Chapter 10, an Ethernet Virtual Connection (EVC) provides the connection between two or more customer UNIs such that Ethernet frames (service frames) associated with that EVC can be transferred between these UNIs and not to any UNIs that are not associated with that EVC. Consistent with ITU-T G.8011, this chapter uses the more generic term EC (Ethernet Connection) rather than EVC. Figure 11-1 illustrates an Ethernet connection with its different reference points from the standpoint of both the Ethernet MAC layer network (ETH) and the Ethernet physical layer network (ETY). Figure 11-1 also illustrates the different Ethernet service areas in a multicarrier Ethernet connection. These three service areas are the access (UNI-C to UNI-N), end-to-end/customer-to-customer (UNI-C to UNI-C), and edge-to-edge/intracarrier (UNI-N to UNI-N).
Chapter 11
380
ETH
ETY
FD = Flow domain Figure 11-1. Illustration of Ethernet service areas (from ITU-T Rec. G.8011)
From a customer's perspective, the EC connectivity can be one of two types: • •
Line connectivity (point-to-point) LAN connectivity (point-to-multipoint or multipoint-to-multipoint)
From a transport network viewpoint, these line and LAN connections can be provided either through dedicated transport channels (including router bandwidth) or through a shared medium. The former is referred to ^s private service, and the latter as virtual private service. The difference between a private line connection and a virtual private line connection is illustrated in Figure 11-2. The service type variations are summarized in Table 11-2.
Ethernet Services Over Public WAN
381
Customer A Equipment
Customer B Equipment
Carrier Equipment
Carrier Equipment
Customer B Equipment
a) EPL for two customers, each with his or her own TDM channel
Ethernet
Ethernet
Customer A Equipment
Customer B Equipment
Carrier Equipment
Carrier Equipment
Customer B Equipment
b) EVPL for two customers where they share a TD!\/I channel for increased efficiency Figure 11-2. Illustration of private and virtual private connections
Table 11-2. Summary of the types of Ethernet services Resource sharing Connectivity Service type Point-to-point Dedicated EPL (Ethernet Private Line) Shared EVPL (Ethernet Virtual Private Line) Multipoint Dedicated EPLAN (Ethernet Private LAN) Shared EVPLAN (Ethernet Virtual Private LAN) Note : The MEF (see Chapter 9) refers to EPL and EVPL as E-Line services and EPLAN and EVPLAN as E-LAN services.
11.2.1 Ethernet connection (EC) attributes The Ethernet connection service attributes that must be taken into account in a transport or service provider network are summarized in Table 11-3 and described in more detail in the following text.
Chapter 11
382
Table 11-3. Ethernet connection service attributes (derived from ITU-T Rec. G.8011)
Service attribute parameters and values
EC service attribute Network connectivity Transfer characteristics
Separation Link type Connectivity monitoring Bandwidth profile UNI Hst Preservation Survivability
Point-to-point, point-to-multipoint, multipoint-to-multipoint Address (deliver conditionally or unconditionally) Drop Precedence (drop randomly, drop conditionally, or not applicable) Class of Service Customer Spatial or logical Service instance Dedicated or shared Sublayer monitoring: On demand, proactive, none Inherent monitoring: Proactive Specified An arbitrary text string to uniquely identify the UNIs associated with the EC VLAN ID (yes or no) Class of Service (yes or no) None, or server-specific
11.2.1.1 Network connectivity As noted, one way in which Ethernet services can be characterized is by the type of desired customer connectivity. The types of connectivity are • • •
Point-to-point Point-to-multipoint Multipoint-to-multipoint
Figure 11-3 shows examples of these different connectivity types. Figure 11-3(a) shows a basic point-to-point connection between customers through a transport network. Figure 11-3(b) shows a popular point-to-multipoint topology known as hub-and-spoke. Figures ll-3(c)-(e) show examples of various multipoint-to-multipoint topologies. Care must be taken to avoid confusing the customer connectivity (logical topology) with the physical topology of the underlying network providing that connectivity. For example, Figures 11-3(b) and 11-3(d) use the same physical topology. The difference between a hub-and-spoke and a star network is that a star network provides arbitrary multipoint-to-multipoint connectivity among all the customer nodes while a hub-and-spoke network connects the hub customer node to each of the spoke customer nodes (pointto-multipoint). Any connectivity between spoke nodes would have to be provided by a router at the customer's hub node. A logical hub-and-spoke
Ethernet Services Over Public WAN
383
network could be provided over the physical topology of any of the networks in Figures ll-3(b)-(e). Figures ll-3(c)-(e) illustrate common transport network topologies. In reality, a transport network will often consist of a combination of star, ring, and more arbitrary mesh subnetworks.
a) Point-to-point
b) Hub and spoke
c) Ring
d) Star CE
CE
CE = Customer Edge e) Aribtrary Figure 11-3. Network connectivity examples
Section 11.3 discusses transport network models.
Chapter 11
384 ETH UNI
PQR network entity
XYZ network entity'
THFD ETHFD
7 ETH UNI
ETH UNI t Customer M
ABC network entity
Customer N
ABC, PQR, XYZ are server layer networks (can all be the same or different); they may be CO-CS, CO-PS, CLPS Figure 11-4. Network portion of the multipoint-to-multipoint topology (from ITU-T Rec. G.8011)
When discussing multipoint-to-multipoint connectivity, it is common to refer to the network topological component that provides the desired forwarding of information between source and destination UNIs as a flow domain} In the example of Figure 11-4, if customers M and N are exchanging data, each is connected to a flow domain (FD), with the flow domains being connected through an Ethernet link flow (LF) over the ABC network entity. A point-to-point connection can be characterized as either not having a flow domain or as having a flow domain with only two flow points (i.e., the two endpoints of the network connection). The point-topoint connection is typically described as not having a flow domain, since a flow domain implies a layer network with inherent switching/routing and other layer network capabilities. A flow domain with only two points is really the degenerate case of a multipoint network and leaves open the potential of adding additional flow points (UNIs here) to the network.
Ethernet Services Over Public WAN
385
11.2.1.2 Transfer characteristics The transfer characteristics of a network relate to what frames are to be dehvered to the destination unconditionally, are to be delivered conditionally, or may be dropped. In the case of Ethernet, the three parameters that determine the disposition of a frame are address. Drop Precedence (DP), and Class of Service (CoS). For the address, a frame can either be delivered unconditionally, regardless of its destination address, or be delivered for only some destination addresses. The DP indicates the relative priority of the frame if it encounters a congestion situation in which frame dropping must occur. If dropping is based on the DP, frames are said to be dropped conditionally. Another option would be dropping randomly (i.e., drop the overflow frames from a full queue). For some services, frames cannot be dropped and hence DP is not applicable. The CoS parameter, which is based on the DP and indicates the frame's class queuing, is not fully defined at this time. 11.2.1.3 Separation (customer and service instance) Separation refers to how the traffic of each customer or service instance is kept separate from the traffic of others. In the case of customer separation, it is the traffic from different customers that is separated. In the case of service instance separation, it is the different service instances that are separated, even for the same customer. A spatial separation implies a circuit switched network (e.g., a TDM network in which each customer has its own TDM channel or facility, or a virtual circuit such as in an ATM network). Logical separation implies that customer or service instance traffic is separated at the packet level (i.e., based on per-packet information such as address or tag values). 11.2.1.4 Link type A link can either be dedicated to a single customer service instance or shared among multiple service instances. For a dedicated link, a single customer service instance has a one-to-one mapping to a set of one or more Ethernet links and the associated server layer trail (i.e., a spatial separation from other service instances). As such, the service instance does not compete for bandwidth/resources (transport and switch fabric bandwidth) with other service instances, and does not allow multiplexing on the access link. (See Figure 11-2(a).) On the other hand, a shared link allows more than one service instance to share that link, (i.e., logical separation) which
386
Chapter 11
means that the service instances can compete for the hnk resources. (See Figure 11-2(b).) 11.2.1.5 Connectivity monitoring Connectivity monitoring is the mechanism by which network nodes determine their ongoing connectivity to their neighbor nodes in that layer network. See Section 11.6 for a discussion of this and other 0AM topics. 11.2.1.6 Bandwidth profile A bandwidth profile specifies the parameters that a traffic flow must meet at a UNI or NNI. Policing of the bandwidth profile is typically done at the edge of the transport network. See Chapter 10 for a more detailed discussion of bandwidth profiles. 11.2.1.7 UNI list For the purposes of management and control, a service provider assigns an arbitrary string to uniquely identify each UNI. 11.2.1.8 Preservation Preservation refers to whether a customer's Ethernet frame VLAN ID and/or CoS are preserved through the transport network. If the value is preserved, it will have the same value at the egress UNI that it had at the ingress UNI of the transport network. In some circumstances, however, it may be necessary or desirable to change these values within the transport network. For example, the service provider may perform a translation between the VLAN ID values that a customer uses on the customer side of the network to a different set of VLAN IDs that are used within the service provider network. Another example is that if a frame fails to meet the specified bandwidth profile, the ingress node may choose to let it pass into the transport network but will set its DP value to a higher value so that it is prioritized for dropping if it encounters congestion. 11.2.1.9 Survivability Survivability pertains to the ability of the network to continue to provide service under situations in which a fault(s) exists in the network. See section 11-7 for more details.
Ethernet Services Over Public WAN
387
11.2.2 Ethernet Private Line (EPL) service EPL, as illustrated in Figures 11-5 and 11-2, consists of point-to-point Ethernet connections using reserved, dedicated bandwidth. With EPL, the transport network effectively looks like a "piece of wire" from the Ethernet client perspective. From the transport network provider standpoint, however, the transport network (server layer) provides the performance monitoring and protection capabilities required for guaranteeing the service level agreement (SLA) with the customer.
Customer
Equipment
^
^-\
Carrier Network /
Carrier x ^ Equipment
^xX^
^
^
Carrier Equipment
Customer
Equipment
Figure 11-5. EPL service illustration
The primary advantages of EPL are the simplicity of setting up a dedicated circuit, and the security for the traffic that is inherent when it is isolated in its own TDM channel.^ Sharing bandwidth can lead to greater bandwidth efficiency in the transport network due to statistical multiplexing gains. However, it is more difficult to administer, since it requires additional effort (e.g., traffic engineering and monitoring) in order to guarantee the customer SLA. EPL service is described in ITU-T G.8011.1 [10]. The EPL connection characteristics are summarized in Table 11-4. ITU-T G.8011.1 defines two types of EPL services. For Type 1 EPL, the CI transferred between the UNIs is the Ethernet MAC frames. As described in Section 11.3, the Ethernet preamble, start of frame delimiters, and interframe characters are discarded, and the Ethernet MAC frames are then encapsulated (e.g., into GFP-F) and mapped into the transport channel. Type 2 EPL, which is only defined for IGbit/s Ethernet, treats the 8B/10B line code information as the CI to be transferred between the UNIs. As discussed in Chapter 5, the data and control code information from the Ethernet signal's 8B/10B line code characters are translated into a more bandwidthefficient 64B/65B block code, and multiple 64B/65B codes are then mapped into a GFP frame (GFP-T). The primary advantages of Type 2 EPL are the preservation of control codes (primitive sequences of special line code
Chapter 11
388
characters) and lower mapping latency. While is possible to also define Type 2 EPL for 4B/5B-encoded 100 Mbit/s Ethernet, there has been no formal request for this service so far. Table 11-4. EPL connection characteristics (derived from ITU-T Rec. G.8011.1)
EC service attribute Network connectivity Transfer characteristics
Separation Link type Connectivity monitoring Bandwidth profile
UNI list Preservation Survivability
Service attribute parameters and values Point-to-point Address — deliver unconditionally Drop Precedence — not applicable Class of Service Customer Spatial or logical (always connection oriented) Service instance Dedicated None, on-demand, or proactive Committed information rate (CIR) and committee burst size (CBS) An arbitrary text string to uniquely identify the UNIs associated with the EC VLAN ID is preserved Class of Service is preserved None, or server-specific
11.2.3 Ethernet virtual private line service (EVPL) EVPL is also a line service; however, the line can be derived from a flow domain and allows sharing network resources among multiple customers or service instances in order to achieve more efficient use of those resources. EVPL is illustrated in Figure 11-2(b). In addition to allowing more efficient use of transport network resources, another potential advantage of EVPL is to reduce the number of UNIs required at the customer edge. This is illustrated in Figure 11-16 in Section 11.4.1. For the customer edge node on the left to connect to four other nodes would require four different UNIs and their associated ports. Service multiplexing is the packet multiplexing of multiple ECs onto a single UNI. While EVPL is still under study in ITU-T SGI5, its expected connection characteristics are summarized in Table 11-5. For virtual connections, the separation is logical (i.e., at the packet level). Due to the sharing of network resources, it is possible that frames may be dropped due to congestion. Also, as discussed above, a service provider may wish to perform VLAN ID translation at the boundaries of the transport network.
Ethernet Services Over Public WAN
389
Table 11-5. Expected EVPL connection characteristics
EC service attribute Network connectivity Transfer characteristics
Separation Link type Connectivity monitoring Bandwidth profile UNI Hst Preservation Survivability
Service attribute parameters and values Point-to-point Address (deliver conditionally or unconditionally) Drop Precedence (drop randomly, drop conditionally, or not applicable) Class of Service Customer Logical Service instance Shared None, on-demand, or proactive Specified An arbitrary text string to uniquely identify the UNIs associated with the EC VLAN ID (yes or no) Class of Service (yes or no) None, or server-specific
11.2.4 Ethernet private LAN (EPLAN) service An EPLAN provides LAN-type connectivity between multiple customer sites through dedicated channels. Figure 11-6 illustrates some of the different basic transport network topologies that can support this service. From the customer viewpoint, these topologies are equivalent (i.e., the carrier network architecture is transparent to the customer). In Options 1 and 3, the carrier does the switching at the edge of the network. Option 3 does the switching at one end of the network rather than at each end. In Option 2, the traffic is brought to a centralized switch (or a number of centralized switch points) in a star connection. Since the switching is performed at Layer 2 in these examples, an MSPP can be used to implement Options 1 and 3. Open issues to be resolved for an EPLAN standard include the following: • How do the customer and carrier specify the bandwidth requirements? For example, if the traffic was evenly distributed among the different customer nodes, the bandwidth between nodes could be specified on the basis of CIR. The more realistic scenario, however, is that multiple customer nodes will want to simultaneously communicate with a single node (e.g., remote sites communicating with a headquarters office). A safe policy would be to reserve enough bandwidth for each node to simultaneously receive data at full rate from each other node; however, this would be too inefficient to be practical. • Closely related to the above issue, how much buffering must the carrier provide to handle congestion, and what will the discard policy be?
390 •
Chapter 11
Is protection handled at Layer 1 (e.g., SONET APS) or Layer 2?
Ethernet PHY Ethernet PHY Customer Equipment Customer Equipment
a) Mesh-type connectivity
Customer Equipment
b) Traffic hauled to a centralized switch point(s)
Ethernet PHY
Customer Equipment Customer Equipment
c) Edge node serves as a bridge or router Figure 11-6. EPLAN illustrations
Ethernet Services Over Public WAN
391
While EPLAN is still under study, its expected connection characteristics are summarized in Table 11-6. Table J1-6. EPLAN expected connection characteristics
EC service attribute Network connectivity Transfer characteristics
Separation Link type Connectivity monitoring Bandwidth profile
UNI list Preservation Survivability
Service attribute parameters and values Multipoint-to-multipoint (and probably point-to-multipoint) Address — deliver unconditionally Drop Precedence — (for further study) Class of Service Customer Spatial or logical (always connection oriented) Service instance Dedicated None, on-demand, or proactive Committed information rate (CIR) and committed burst size (CBS) An arbitrary text string to uniquely identify the UNIs associated with the EC VLAN ID is preserved Class of Service is preserved None, or server-specific
11.2.5 Ethernet virtual private LAN service EVPLAN is a combination of EVPL and EPLAN. The transport channel bandwidth is shared among different customers, as are the routers in the carrier network. Ultimately, the sharing of bandwidth in the transmission channels and switch fabrics give EVPLAN the potential for very costeffective carrier network resource utilization. Clearly, however, EVPLAN is the most complicated network architecture to administer. The open issues regarding EVPLAN transport architectures include all of those already discussed for EVPL and EPLAN; however, the magnitude of some of these issues is greatly increased for EVPLAN, which in turn restricts some of the potential solution space. For example, the tagging mechanism to differentiate the data from different customers, and the different data flows within each customer data stream, must have an adequately large address space. (E.g., the 4K address space of VLAN tags makes them impractical for large EVPLANs. Also, their applicability to only Ethernet frames further lessens their appeal for a generic data network.) While EPLAN is still under study, its expected connection characteristics are summarized in Table 11-7.
Chapter 11
392 Table 11-7. EVPLAN expected connection characteristics
EC service attribute Network connectivity Transfer characteristics
Separation Link type Connectivity monitoring Bandwidth profile UNI Hst Preservation Survivability
11.3.
Service attribute parameters and values Multipoint-to-multipoint (and probably point-to-multipoint) Address (deliver conditionally or unconditionally) Drop Precedence (drop randomly, drop conditionally, or not applicable) Class of Service Customer Logical Service instance Shared None, on-demand, or proactive Specified An arbitrary text string to uniquely identify the UNIs associated with the EC VLAN ID (yes or no) Class of Service (yes or no) None, or server-specific
TRANSPORT NETWORK MODELS IN SUPPORT OF ETHERNET CONNECTIVITY SERVICES
The preceding sections provided a discussion about the service types and characteristics associated with Ethernet WAN connectivity services. This section will provide a discussion of the issue of transport models and architectures as well as a functional description of their underlying client signal adaptation and termination processes. As the demand for Ethernet managed connectivity services increases, transport and service providers are finding that over the near- to mediumterm time frames, economic optimality is best secured and customer demand best served by implementing a connection-oriented packet-switched (COPS) transport infrastructure by leveraging the existing connection-oriented circuit switched (CO-CS) TDM networks. Within this framework, and of vital importance to this emerging CO-PS transport model, there is a need for a multiplexing scheme that enables efficient, payload-transparent transport of multiservice packets and Ethernet MAC frames over optical networks. To better understand the variety of applications being pursued and addressed by this emerging transport model, we provide a set of representative service scenarios in Figures 11-7 through 11-10. In Figure 11-7, we depict the aggregation of a single customer's multiple flows over the access portion of the network, where those multiple flows may be differentiated based on class of service (high-priority guaranteed
Ethernet Services Over Public WAN
393
bandwidth such as voice, video, etc. vs. best-effort services such as e-mail, Internet access, etc.), quahty of service (error performance, latency, delay variation), or type of traffic (voice, video, Ethernet data access. Fibre Channel Storage Area Networking, etc.). Enterprise HQ
® 1
Multiple flows aggregation <(ni+n2+n3)xVTi.5
t 1^ &
7 f
A^CMSS
1
^
1 riJvT1£ x ^ ^ | ^ 3 x V T - t 3 ^
^
1 ^ ^ Enterprise Branch Site
. . ^ » 1
^P^
1
1 mATijA 1
1
^
ouppiitir o n e
»
1 ^^^^ Business Partner Site
Figure 11-7. Single-customer flow aggregation
Figure 11-8 provides an example of the delivery of data services to multiple customers co-located in one or more MxUs over the Metro Area. MxUs can be apartments or multidwelling units (MDUs), office buildings or multitenant units (MTUs), and hotels or multihospitality units (MHUs). Multiple flows aggregation VC3
ISPB
ISPC
Figure 11-8. Multiple-customer flow aggregation
Chapter 11
394 CO
Multiple flows aggregation
VC3
EO
Figure 11-9. DSLAM uplink aggregation
Depicted in Figure 11-9 is an application example of aggregating traffic from multiple Ethernet-based DSLAM uplinks. Figure 11-10 provides an application example where EoS/PoS/DoS (Ethemet/Packet/Data over SONET/SDH) traffic from multiple, partially populated SONET/SDH pipes are groomed, at the packet level. In the remainder of this section, we will provide a description of some of the feature requirements that are being discussed in the industry to provide support for the generalized service multiplexing mechanism that is vital for the deployment of EPL and EVPL connectivity services over the infrastructure of the emerging CO-PS network.^ In many service multiplexing applications, large numbers of packet flows need to be multiplexed. For example, an access interface to a service provider edge router should serve 10-20 remote aggregators, each aggregating 10-50 flows. This results in 100-1000 flows per access interface. Maximum efficiency of the transport network, as well as the router access interfaces, is achieved if all the flows are multiplexed in the packet domain rather than TDM domain, before feeding the router. Minimum cost (capital and operational expense) is achieved if we can keep these aggregators "dumb," i.e., transport layer aggregators rather than higher layer aggregators (e.g., routers). Hence the transport network should enable multiplexing of thousands of multiprotocol, bursty, data flows into a single SONET/SDH interface. In the direction from the edge router or Ethernet
Ethernet Services Over Public WAN
395
switch to the access network, individual packets must be labeled such that the access network would be able to identify the physical destination port of the packet.
Multiple flows aggregation
ISPC
Figure 11-10. EoS/PoS/DoS aggregation
Video on-demand distribution to residential customers is another example where thousands of flows should be multiplexed asynchronously over the access ("feeder") network. Any generalized multiplexing procedure or mechanism should be capable of tagging a service packet with a unique identifier and carrying the tagged packet over the optical transport link. The tagging of flows on ingress with unique (per flow) tags enables the multiplexed flows to be uniquely separated and identified as to ownership and destination at the farend of the multiplexed link. This process would, in a very fundamental sense, effectively enable a "PVC" (Permanent Virtual Circuit) mode multiplexing model, whereby tag-based multiplexed flows could be connected to the appropriate endpoints or destinations across the SDH network. In order to maximize the utilization of the transport network, transport service providers may offer oversubscribed services. In that case, packet multiplexing can be performed in an arbitrary manner, or in a more controlled fashion, with support for SLAs. Accordingly, client-agnostic priority information and drop precedence information must be carried with the packets inside the transport network.
396
Chapter 11
The multiplexing capability should be payload (service) agnostic and should permit the transport network to evolve independently of the services networks, while supporting the addition of new client services as they evolve. The tagging mechanism should allow for SLA function support. In general, if the emerging CO-PS-based transport network is to be capable of addressing the preceding discussed service requirements, then it must be outfitted with a mechanism that not only provides for service multiplexing support but more importantly must enable the suite of connection-oriented attributes that are vitally necessary for the realization of a CO-PS network. In this regard, the following functional characteristics are of first-order importance to the emerging CO-PS network: • Tag the packets with appropriate flow/service ID that guarantees uniqueness over the service domain • Provide for unconstrained allocation of multiplexing identifier tags • Support an extensible set of packet service types • Support a service-type identifier so that traffic can be directed to the appropriate adaptation function • Support packet multiplexing using traffic management capabilities that are suitable for controlled aggregation of best-effort traffic, but that can be extended to support contracted SLAs • Provide client agnostic drop precedence (SLA-non-conforming) indication to aid in meeting QoS characteristics • Provide client agnostic priority/CoS indication to support CoS characteristics • Provide protection for the header/tag, with the following protection characteristics: • —Insertion or removal of tags must not compromise end-to-end performance monitoring • —Tag stacking in support of multiple domains or domain extension must not compromise end-to-end or domain-specific performance monitoring • —^The tag must be protected against transmission channel errors in order to prevent misrouting of data packets • —^A simple means of administering tag IDs must be provided that is compatible with existing carrier practice. As the CO-PS network emerges with the above-described functional attributes, at a very fundamental transport level it should still behave as a traditional SONET/SDH network (depicted in Figure 11-11). The key difference would be the existence of a more sophisticated connectionoriented and QoS/CoS paradigm.
397
Ethernet Services Over Public WAN
Service Provider GbE/FX
FE/FX
Interoffice Wm
Feeder
Distribution
0
Figure 11-11. SONET/SDH physical layer transport for EoS
The Figure 11-11 physical layer would be expected to provide the traditional transport infrastructure functions such as the following: • • •
Uses SDH/Optics (use of Fiber/Copper Ethernet at edge) Provides physical connectivity between nodes - STM-« Rings etc. Handles protection at physical level and transmission performance management When the CO-PS aspects are taken into account, the Figure 11-11 network layer becomes modified as in Figure 11-12. There we observe support for a so-called Packet Switching sub-layer that: Implements packet labeling Provides Logical connectivity between ports Performs statistical multiplexing of customer traffic Management & CoS Handles protection at Packet level
Ensures Traffic
Figure 11-12 can be modified to provide a layered network view that takes into account the architectural attributes necessary for implementing CO-PS-based EPL services. This setup is depicted in Figure 11-13.
398
Chapter J]
Service Provider^ _
Packet switching sublayer *' ~- ^ ..^
(7bt.. ^^
VC4r!y^- Packet^Ring,,,-.»'
••*llli«|>^
'•
„.^2=^J
LilP****4 IL—*•
"
.
.MMUUj-^
llj 1 [/
'
''
FE/FX
/ " T L
I r^TI Mill! !.ir?M 5 ^ - - ^ -n j I K ^ 1 K «.^ svMi
,^.
Q
'='^P
FE/FX
m i
FE/FX
Interoffice f ^ Packet Ring
/I
Feeder
f ^
Distribution
Packet
A
Adaptation
Figure 11-12. Packet switching sublayer overlay for EoS
The diagrammatic representations and discussion in this section have addressed examples of the layered architectures for CO-PS-based support for EoS. Several paragraphs ago, we also outlined the functional attributes that would be necessary for the effective transformation of the current SONET/SDH transport infrastructure into one that provides a CO-PS basis for EoS. With this background in hand, we shall now begin a discussion of the details of the packet mechanism that would be necessary for the implementation of this new network paradigm. A CO-PS network can realistically be realized on the basis of leveraging two existing technologies (MPLS and GFP). To effectively support the COPS infrastructure, both technologies would require (varying levels of) enhancement and modification. In the remaining paragraphs of this section, we shall summarize the basic frame formats and identify, where possible, the necessary modifications to properly support CO-PS.
Ethernet Services Over Public WAN
399
p-t-p Connectivity Services SLS»BE SLS=15Mb/$
Broadbond Access Service
'•;,Vc4nv;','-'V-'V
.''..'ii*<W'»»'
'/', ;-'- -- ; - ' - <^»M'^»'»Mi>''^^V'«'**'*'f!r''/':
ISP
i Interoffice Packet Ring
Packet , Aggregation
4
Adaptot ton
gjjjj^ Service Port
Figure 11-13. Layered architecture for EPL services
We begin with a brief examination of MPLS as a CO-PS candidate. MPLS is a complex series of protocols encompassing traffic engineering (TE), packet forwarding (i.e., data plane), and label switched path (LSP) setup. For our purposes at hand, the aspect of MPLS that is most relevant for CO-PS is the data plane (MDP). Figure 11-14 provides the essence of MDP. One unfortunate aspect of MDP is that it does not appear to unambiguously provide for CoS and DP (drop precedence) in the shim header.'* In fact, there are two different approaches to support CoS and DP. One way is via L-LSP (Label-inferred LSP), and the other is by way of E-LSP (EXP-inferred LSP).^ In addition, L-LSP maps the CoS information into the label value. The drop probability is mapped into the EXP bits in the MPLS shim header (e.g., like the CLP bit in ATM). E-LSP maps both the CoS and the DP information into the EXP bits in the MPLS shim header. An alternative approach has been suggested^ in which the CoS and DP information is encoded in an extended GFP header (eGFP). There is a substantial similarity between eGFP and the L-LSP approach taken by RFC 3270. The approach that was, however, proposed for adoption by eGFP has the advantage of being able to leverage the eHEC mechanism that natively
Chapter 11
400
protects the extension header of the GFP frame. Figure 11-15 gives a visual depiction of the eGFP frame format.
L-LSP
Forwarding + CoS
Label (20 bits)
EXP (3b) S(1b)
Forwarding
E-LSP
DP
TTL(8b)
CoS+DP
Figure 11-14. MPLS Data Plane (MDP)
Both MDP and eGFP would require some modification in order to effectively support (in the short to medium term) the emerging CO-PS network. The modifications required in both cases are depicted in Table 11-8 and Table 11-9.^ In summary, the emerging CO-PS network can be made realizable in the short to medium term by leveraging the existing SONET/SDH network infrastructure and enhancing it with a generalized packet multiplexing capabihty. This approach would allow the service and transport provider to efficiently deploy EPL and EVPL services (with the necessary customer traffic isolation and protection) and benefit from optimizing their network resources through statistical gain with guaranteed performance (delay, jitter, loss). An architectural approach that is based upon either an MDP or a eGFP (or a combination of these technologies) could be the basis for a scalable, efficient, and simple CO-PS path layer network. Moreover, given its hybrid TDM/Packet nature, it would be competitively capable of cost-effectively supporting legacy transport services relative to full packet switching infrastructures.
TBD
Label (16 bits)
COS (3b)
DP (3b) S(1b)
Reserved (9b)
Figure 11-15. Extended GFP (eGFP) fFrame format
Ethernet Services Over Public WAN
401
Table 11-8. MDP frame format-required modification for CO-PS Tributary Slot ID 1 20-bit label CoS/DP 3-bit EXP Hierarchy | Infinite tunneling capability Tunnel Connection MPLS 0AM Monitoring Nested Connection MPLS 0AM Monitoring Pro-active FM/PM 1 MPLS 0AM (CV, FFD, FDI, BDI) On-demand fault location MPLS 0AM (ping) SNCP MPLS protection switching; further enhancements/extensions are required SPRing To be developed Mapping IP mapping is available; ATM, FR, ETH mappings are under development in IETF PWE3 Layer Network Interworking MPLS/ATM, MPLS/FR are under development in Q.5/13
Table 11-9. Extended GFP (eGFP) frame format-required modifications for CO-PS network Tributary Slot 1 Proposed 16/20-bit Flow ID. CoS/DP Proposed 3-bit Priority and DP fields Hierarchy Proposed accommodation for stacking in the extension header Tunnel Connection To be developed Monitoring Nested Connection To be developed Monitoring Pro-active FM/PM To be developed On-demand fault location To be developed SNCP To be developed SPRing To be developed Mapping To be developed Layer Network IW To be developed Switching 1 Development of connection-oriented GFP switching
11.4.
ETHERNET CLIENT INTERFACES
The client interface (i.e., the UNI) for Ethernet services will typically be an Ethernet physical interface. In the past, the UNI for WAN data
Chapter 11
402
connections was typically based on telecom signals (e.g., DSl, DS3, fractional-DSl, etc.), which required either special customer equipment (e.g., a Tl multiplexer) or special WAN port cards on customer routers. Being able to use an Ethernet interface is a significant advantage for enterprise customers. Ethernet interfaces are typically less expensive than telecom WAN interfaces, especially for higher bit rates, and are supported by relatively inexpensive Ethernet LAN routers. Using an Ethernet interface also allows the enterprise network administrators to keep their OAM in the Ethernet domain and allows the carriers to handle the telecom signal domain. The Ethernet UNI service attributes that are common for all services are shown in Table 11-10, and the attributes that are service dependent are shown in Table 11-11. Table J J-I O.Vm serviceattributes common to all services
Layer ETH ETY
UNI Service Attribute
Service Attribute Parameters and Values
MAC service UNI ID UNI EC ID PHY speed PHY medium
IEEE 802.3 frame format Arbitrary text string to identify each UNI instance Arbitrary text string to identify each EC instance 10 Mbit/s, 100 Mbit/s, 1 Gbit/s, or 10 Gbit/s IEEE 802.3 physical interface
Table IJ-I I. Service-dependent UNI service attributes
Layer
ETH
ETY
UNI Service Attribute
Service Attribute Parameters and Values
Multiplexed access VLAN mapping Bundling Bandwidth profile Layer 2 control protocol processing PHY mode
Yes, no Specify Yes, no, all-to-one For further study for most services Block, process, or pass per protocol on ingress Generate, or none per protocol on egress Full duplex, half duplex, or auto-negotiation
The UNI service attributes common to all services are reasonably selfexplanatory. The service-dependent attributes are explained as follows.
11.4.1 Multiplexed access The multiplexed access aspect of the UNI relates to whether a customer edge node has an individual UNI associated with each of the far-end UNIs to which it is connected, or whether a UNI is shared for the connections to multiple other customer UNIs. These cases are illustrated in Figure 11-16.
Ethernet Services Over Public WAN
4xUNI
a) Multiple point-to-point EPL (one EVC per UNI)
b) Service multiplexed UNI (e.g., for multiple EVPL) Figure 11-16. Service multiplexing example
403
404
Chapter 11
11.4.2 VLAN mapping Ethernet (per IEEE 802.IQ) allows the insertion of tags into Ethernet MAC frames in order to create virtual LANs (VLANs). When a customer desires to preserve this VLAN segregation of traffic through the WAN, the carrier may simply preserve the entire MAC frame, including the VLAN tag. It is possible that both the customer and service provider desire to use VLAN technology. (More will be said about this topic in Section 11.5.) If the service provider also wishes to use VLAN technology, then it must either insert a second ("stacked") VLAN tag into the MAC frame or perform a translation of the customer VLAN tag at the ingress in order to conform to the service provider's VLAN tag assignments, and then restore the customer VLAN value at egress. See Chapter 10 for a more detailed discussion of this topic.
11.4.3 Bundling Bundling refers to whether multiple customer VLAN IDs can be mapped into a single EC at a UNI. The case where all VLAN IDs are mapped to a single EC is called all-to-one bundling. See Chapter 9 for an extended discussion on bundling. It should be noted that all-to-one bundling and multiplexed access are mutually exclusive. This is because multiplexed access is the multiplexing of multiple ECs into a UNI, and mapping all VLAN IDs into a single EC means that there is a single EC at that UNI rather than multiple ECs.
11.4.4 Bandwidth profile The bandwidth profile refers to the characteristics of the traffic that a customer presents to the network at the UNI. The parameters include such things as guaranteed committed information rate (CIR), peak information rate (PIR), burst sizes, and what is done with excessive traffic. See Chapter 10 for an extended discussion on bandwidth profiles and their policing algorithms.
11.4.5 Layer 2 Control Protocol processing Layer 2 Control Protocols are used by enterprise network bridges for a variety of functions. Depending on the protocol application and the WAN service, these protocols can either be passed transparently through the WAN, processed (with the network provider equipment acting as a peer to the
Ethernet Services Over Public WAN
405
customer equipment), or discarded at the UNI. See Chapter 10 for a detailed discussion of carrying these protocols through metro networks, which also applies to transport networks.
11.4.6 Summary of UNI Service Attributes for Different Services The UNI service attributes are currently only defined for EPL service (ITU-T G.8011.1). These attributes are summarized in Table 11-12. EVPL and EVPLAN (especially EVPL AN) will typically make use of bundling and multiplexed access and have more complex bandwidth profiles. Table 11-12. Summary of UNI service attributes for different services UNI Service Attribute Service Attribute Parameters and Values Layer
ETH
ETY
11.5.
Multiplexed access VLAN mapping Bundling Bandwidth profile Layer 2 control protocol processing PHY mode
EPL No No All-to-one CIR, Committed Burst Size (CBS) All are passed except PAUSE frames, which are discarded. (The network equipment providing the UNI-N may generate PAUSE frames.) Full duplex
ETHERNET TRANSPORT NETWOIOC TO NETWORK INTERFACE (NNI)
We see from the depiction in Figure 11-24 in Section 11.6 that there are typically different client server functions involved in a communications network relationship. Graphically depicted in that figure are the separate functions of customer, service provider, and transport provider. These different functions imply not only a difference in the client server interface relationship models but also a difference in the information entities that are actually passed between each client server pair. In a typical communications network based upon client server architecture, customers would usually connect to the service provider network for service over a user network interface (UNI).^ In a similar manner, the service provider would connect for transport service to the network of a transport provider over a network-to-network interface (NNI). This interface is deployable in a couple of different configurations. One way is to interconnect the separate administrative network domains of the same
406
Chapter 11
service or transport provider. This is often described technically as an Intra Domain Interface (laDI). Another method is to interconnect two or more network domains of different service or transport providers. This is technically described as an Inter Domain Interface (IrDI). We will dedicate the remainder of this section to discussing the specifics of the different NNI approaches that are likely to be adopted for the support of Ethernet managed connectivity services. In Figure 11-17, we provide a graphical depiction of an laDI for a generic Ethernet over Transport (EoT) service.^ In this example, we see that there are two separate administrative domains for the same service provider. ^^ Note also the fact that the same NNI is used both internally to the administrative domain and externally for the same provider network.
Service Provider A Administrative Domain A
Service Provider A Administrative Domain B
Figure 11-17. laDI NNI for Ethernet over Transport Service (adapted from ITU-T Draft Rec. G.8012[ll])
Turning to the IrDI, we note that (as depicted in Figure 11-18) this case is architecturally indistinguishable from the laDI case except for the fact that the network infrastructure is independently owned. The NNI inside the individual administrative domain is exactly the same in terms of features and functionalities as the NNI connecting the two networks.
Ethernet Services Over Public WAN
407
31
Ety-UN»
t
j
•'fl"'"''''-"''"'flim I |Ety4JNr
J Service Provider B Administrative Domain B
Service Provider A Administrative Domain A
Figure 11-18. IrDI NNI for Ethernet over transport service (adapted from ITU-T Draft Rec. G.8012)
As was mentioned above, there are several transport technologies (represented by different layer network models) that are capable of supporting Ethernet connectivity services. Figure 11-19 provides a graphic depiction of the different layer networks and their NNI relationships that are available to the service and transport operators for supporting Ethernet connectivity services. In Figure 11-19, we note that there is an NNI defined at the MAC level for connecting two laDI or IrDI network elements (NEs). Note also that no specific identity has been given to the server transport technologies. What is of importance in this case is the fact that the client Ethernet physical interface (ETY) could be just as easily transported over server transport technology A, B, or C as long as the appropriate laDI or IrDI NNIs are in place in the different NEs.
ETH
ETH UNI 11 II
ETY
ETY UNI II II
ETH NNI II II
ETH
SrvA NNI ETY
SrvA
II
ETH SrvA NNI II SrvA SrvA II
ETY
ETH UNI II II
ETH
ETY UNI II II
ETY
SrvB NNI SrvB NNI SrvB NNI II II II SrvB SrvB SrvB SrvB II II 11 SrvC
SrvC NNI SrvC NNI SrvC NNI II II II SrvC SrvC SrvC II II II
Figure 11-19. EoT server layer networks (from ITU-T Draft Rec. G.8012)
Chapter 11
408
The discussion above has estabhshed that NNIs are based on either laDI or IrDI relationships. These latter entities are also typically defined in NEs situated at the boundary points of the network. To proceed with developing our understanding of the role of the NNI in supporting Ethernet connectivity services, we will now explore some of the details of the functional elements of the NNI. In this regard, Figure 11-20 will be useful in helping us with our understanding of these details. From Figure 11-20, we see that the typical NNI functionality provides support for the structured transfer of information associated with the following functions: 1. Management plane "^e.g., plane management and resource management functions 2. Control plane ^ call control, connection control, and signaling functions 3. User/Data plane -^ user information flow transfer and the associated inband flow and error management controls
UNI/ NNI
TNE
Figure 11-20. Three planes of Ethernet UNI and NNI (from ITU-T Draft Rec. G.8012)
In a manner similar to the structure in Figure 11-20, the NNI function is further decomposed into plane-specific functions corresponding to the management, control, and user/data planes (i.e., NNIM, NNIC, and NNID, respectively). In the remainder of this section, we will provide greater details
Ethernet Services Over Public WAN
409
on the N N I D for EoT. Figure 11-21'' provides an illustration of the basic signal structure for the NNID. In Figure 11-21, several transport server technologies for the transport of the Ethernet client signal are identified. For the CO-PS-based case that we have been discussing, SONET/SDH and PDH would be the most directly relevant transport technologies. Although not entirely visible in Figure 11-21 (but nevertheless there), the N N I D function associated with the aforementioned technologies would provide for such functions as client signal adaptation and mapping, error management, link state signaling, flow control, and connection monitoring. From the graphics in Figure 11-21, we can see that the methodology for mapping and adaptation of the Ethernet MAC client signal is dependent upon the underlying transport server technology. Since GFP is the preferred method for such mappings for PDH, SDH/SONET and OTH transport server technologies, we will limit our discussion to GFP.'^ To repeat in summary some of the information from Chapter 5, the GFP ingress source mapping process accepts the Ethernet MAC client signal and performs an octet-aligned insertion into the payload area of the GFP-F frame. It then follows this by attaching a core header, which includes a payload length indicator (PLI) and core header error check (cHEC) fields. Added also is a payload header with support for discriminators for payload type (PTI), presence of GFP payload field FCS (PFI), and extension header (EXI), as well as an indicator of user payload (UPI) and an error check over the type field.'^ A graphic of the preceding described process is depicted in Figure 11-22. The intent of Figure 11-22 is to provide the reader with a graphic understanding of the encapsulation process and how the Ethernet client signal format is mapped into the GFP-F frame format for subsequent transport across the WAN. Figure 11-23 continues this analysis from a more functional architectural perspective. Our purpose here is to provide a more comprehensive architectural modeling view of the manner in which the Ethernet WAN transport service via the N N I D is actually implemented.
Chapter 11
410
Clients (e.g., IP, MPLS, PDH) _SNAR-
_ J
LLC
\
ETHP
Era.
ETHS -GFPflLAPB ETYn
GFPflikpb-
64B/Md-
PDH Pqe, Pq^ Pqe-X Pqs-Kvl
SDH VC VC-n Xv VC-4| •Xc
SDH VC-44d4c
OTH ODwt ODUk-Xv
ATM VC
MPLS
EoP
EoS
EoS
EoO
EoA
EoM
Native Etliernet
RPR
_E^
Etiiernet-over-Transport
Figure 11-21. ETH interface signal structure (from ITU-T Draft Rec. G.8012)
2< CO
Link Header
M_SDU
<
ETH
Link Trailer
Payload
^/
,
/
Payload
Link Frame
|
\ MAC PCS
/ core payload Header Header PLI cHEC
GFP Payload Information Field
GFP-F
PTI=000 PFI=0 EXI=0000 UPI=0x01 tHEC
Figure 11-22. GFP-F encapsulation of the Ethernet MAC frame (adapted from ITU-T Draft Rec. G.8012)
In Figure 11-23, the Ethernet connectivity service is defined betw^een a source Ethernet process at CPE_A and a sink Ethernet process at CPE_B.^'*
Ethernet Services Over Public WAN
411
We are able to also observe a fragment of the client signal source adaptation process at the CLE in CPE_A that presents the source client signal (formatted per UNI requirements) to the ETY sink termination process in the source NE of the service provider. From there the client signal physical interface is adapted and provided to the source GFP-F adaptation process for mapping onto the SDH/SONET transport layer. This mapping by the GFP process as well as the signal conditioning and structuring by the Sn-X function is done strictly according to the requirements of the NNID-
ETC_CI
\ljpM^?.^^
\ETY/ETH/^:/^-x/GFP->;
^qFp-F/ETH/:;;g
^^^^^'/,
ETY CFClient Link
x l
ETY Cl/ClientLink SDH/SONET Transport Network
Figure 11-23. Functional model of Ethernet over SDH/SONET (EoS)
11.6.
OAM
The material introduced thus far in this chapter has taken the reader through the motivation for Ethernet WAN services, the various service types and characteristics, service models, and transport architecture models. In this section, we shall introduce a discussion about the operational and administrative and management aspects of the service, the so-called OAM dimension. From a general perspective, the issue of OAM is almost exclusively a concern of WAN as opposed to LAN network operators. LANbased systems typically reside within the domain of an enterprise and, for the
412
Chapter 11
most part, commit only to providing best-effort service delivery. On the other hand, most operators of public transport communication networks pay very special attention to the issue of OAM as the mechanism designed to assure operational quality and efficiency of the network assets. More particularly, it is commonly recognized that OAM functionality is vitally important in public transport networks to guarantee operational ease, resource efficiency, and performance predictability. This all leads to a streamlining of the operational cost of running the network and ensures that its behavior is in conformance with agreed-upon metrics of SLAs established with customers (ITU Recommendation Y.1730, Clause 7]) If there is to be a broad-based deployment of managed Ethernet connectivity services, the networks offering such services must be equipped with OAM capabilities in a manner that is comparable with existing transport technologies like SONET/SDH, PDH and ATM. This requirement is made even more necessary when one takes into account the fact that service and transport operator networks are typically hierarchically layered, with various levels of OAM requirements and responsibilities at every layer. Complicating the preceding issue is the problem of delivering the Ethernet connectivity service within the context of a multioperator framework, with the ensuing difficulty of detecting and diagnosing connectivity and other network- and service-related issues. At a more specific level, for the public network operator engaged in providing an Ethernet managed connectivity service, there are a number of very significant reasons why it is critically necessary for OAM to be supported. In what follows, we provide a discussion of some of the primary ones. An important consideration for OAM support in the network relates to the need for the network operator to be able to proactively anticipate any basic availability and performance characteristics of the network that would negatively affect the quality of the transported service contracted for by its customers. This ability allows the network operator to take timely remedial action to minimize the extent and duration of any service-affecting malady. Furthermore, the information gleaned from such surveillance enables the operator to make the appropriate billing adjustments to the customer's account in the event of any degraded or service outage episodes. Streamlined OAM capabilities within the operator's network would tend to allow for the timely and structured detection, diagnosis, and correlation of defects, errors, and faults during the provision of the transport service. An immediate positive outcome would be a lowering in the levels of support staff that would be required to be deployed (both in office and in the field) to troubleshoot and solve customer network connectivity services issues. This
Ethernet Services Over Public WAN
413
reduction would contribute importantly toward the perennial objective of minimizing operational expenditure and costs (OPEX). Public network operators, as a rule, assume that their customers' data is confidential and very important to them. Consequently, such service attributes as guaranteed confidentiality, security, and maintained integrity of the customer's traffic is only supportable on the basis of a deployed and operational 0AM system. This internal network capability would generate appropriate remedial actions to minimize any negative effects of defects that resulted in the customer's traffic being misrouted to an unauthorized end point. A final reason (and benefit) for OAM support in public Ethernet transport networks relates to the fact that such a mechanism removes the customer from being a critical part of the defect detection and reporting system. Thus, a streamlined OAM system would tend to maximize the number of defects that are automatically detected, thus obviating the need for the customer to report a detected problem to the transport provider. It leads also to the ability to identify and sectionalize defects to the appropriate errant source (i.e., to the appropriate layer, whether it is within the client Ethernet layered network structure or within the network operator's transport domain). This ability not only leads to the transport operator being able to provide a more flexible, maintainable, and reliable transport service but also more importantly, leads to an increase in customer satisfaction and trust in the reliability and integrity of the operator's transport service. At this point, we shall provide a high-level introduction to the idea of the OAM domain. This discussion is intended to provide a conceptual starting point in the mind of the reader prior to introducing the more complex layered view. Figure 11-24 introduces the three domain areas of customer, service provider, and transport operator and indicates not only the extent of the "service" but also the extent of the respective responsibilities and areas of administrative control of the three domains. The preceding discussion focused essentially on motivating OAM and highlighting some of the more important benefits it affords both the network operator and his or her customers engaged in a (SLA) managed Ethernet connectivity service. At this point, we will approach this issue from a more formal and detailed perspective. To aid in the discussion, we will employ the reference architectures models depicted in Figures 11-25 and 11-26 to illustrate the details of OAM for the case of point-to-point topologies. These reference architecture examples are most relevant for the cases of EPL and EVPL service types.
Chapter 11
414 Iranspon rt-OYider
^p^^?*=»)Pwider ^^'---^''''"''''^^
Customer
! i ^
!
\%
Access !
!
Transport Core
-
\
1 Customer Domain
:
i t 1
1 ^ !^ 1 ^
1 1 ^
1
^
«^ W^% Operator Domain
Provider Domain Operator Domain
^P^^?9^
1
Customer
Access \
1 1
!
1 ^! 1 ^1
|
!
1
1 i
^i Ps
i 1
f
i
i
P*^
Fj 1 Operator Domain
Figure 11-24. OAM domain service vs. network
In Figure 11-25, we depict an example of a layered reference network architecture for a point-to-point Ethernet transport flow. The perspective is based upon the ITU-T functional modelling methodology defined in Recommendation G.805 [12] and G.809.^^ Several points are worthy of mention. First, from a demarcation perspective, the equipment or so-called network elements (NEs) associated with functional reference points A and D are (typically) customer-located equipment (CLE). Second, these NEs indicate several functional relationships, including an association with the termination flow points (TFPs) for the Ethernet service as well as the ingress and egress of the transport network traffic flow. In addition, the NEs associated with reference points B and C are connected with the network of the operator. As such, they are typically situated at the edges of the transport network domain and form the ingress and egress to the operator's network. In this particular example, the client traffic flow (i.e., the ETH link flow) is described as being transported over a generic single server trail (S), which could be a CO-CS or a CO-PS network. ^^
Ethernet Services Over Public WAN
415
NNI ETH LF
UNI link flow
ETHFP
-x®
ETHLF
transit link flow
ETHFP
>^ UNI link flow
UNI-UNI flow network flow
Figure 11-25. Example of point-to-point Ethernet flow reference model (1) (from ITU-T Rec. Y.1730)
If we were to now allow for the case in which the Ethernet cHent traffic to be transported is sourced from a bridge device as opposed to a single termination flow point, we would then have Figure 11-26 (where the connection into the adaptation function is now a flow point (FP) as opposed to a TFP). Note that one important implication of this architecture is that the UNI-to-UNI relationship is no longer the same. In the nonbridging TFP case, the UNI-UNI flow is coterminous with the network flow. In contrast, for the bridging case, the UNI-UNI flow is now not the same as that of the network flow. The importance of the above functional representations lies in the fact that details of the functional processes and relationships are made visible to the implementer. Consequently, it is very easy to visualize and understand the processing flow of the client signal as it traverses the various functional process blocks (which may or may not reside in specific network or client located equipment). Visible too are the areas of administrative demarcation (for data and 0AM transport) both from the customer and the network operator perspectives. With this framework, it is now quite easy to understand the administrative span of the OAM/ME pertaining to the access
Chapter 11
416
link versus edge-to-edge versus UNI_C-to-UNI_C versus lJNI_N-toUNI N. We will now turn to a more detailed discussion.
NNI ETHFP UNI link flow
>^
ETHLF transit link flow
ETHFP
>^ UNI link flow
UNI-UNI flow network flow
Figure 11-26. Example of point-to-point Ethernet flow reference model (2) (from ITU-T Draft Rec.Y. 1730)
The reference network architectures discussed above represent only a few of the possible architectural models for the point-to-point case. For instance, these reference architectures could be modified to include multiple network operator domains or even multiple server layer technologies involved in the transport process. Note also that such architectural relationships could be multipoint-tomultipoint (i.e., LAN-based relationships). It should also be noted that although these represent a more complex set of architectural relationships, the fundamental 0AM interactions pertain. Since our mission is to develop a fundamental understanding of the OAM function in a network environment, the basic architectures depicted above for the point-to-point case are adequate for our purposes. At this point, let us introduce the notion of a maintenance entity (ME), An ME represents the OAM flows across the FP and TFP reference points and is useful for the demarcation of the maintenance relationships between
Ethernet Services Over Public WAN
417
the various layers and sub-layers in the client and network operator administrative domains. If we look at either Figure 11-25 or Figure 11-26, it is very easy to see where the OAM ME relationships would be situated. Thus, for example, there would be the ETH layer network relationship of the customer UNI to customer UNI. There would also be a similar relationship at the ETH layer between the UNIs for NEs B and C. In addition, there are the segment OAM ME relationships, which would include the following areas: 1. Between flow points on the boundary of a provider's network 2. Between flow points on the boundaries of two adjacent provider networks 3. Between any flow points as required 4. ETY link connection OAM (as defined in Clause 57 of IEEE 802.3ah) The direct implication of points 1 and 2 in the preceding list is that, depending on the particular OAM flow, a typical network operator would seek to restrict its operational range to within its administrative boundary. Thus, for example, it is to be expected that segment OAM flows between flow points on the perimeter of a provider network may be restricted from interacting with the networks belonging to a given customer or other network provider. In a similar vein, segment OAM flows between flow points on the boundaries of two adjacent transport provider networks may not be allowed to interact with the networks belonging to a given customer network or another transport network provider. At this juncture, it would be useful to summarize the relationship between the maintenance entities described above (and defined in Section 9 of ITU Recommendations Y.1730 and G.8010 [13]) and the OAM flows. In this regard. Table 11-13 provides a summary mapping of these relationships. Several key relationships immediately emerge from this tabular presentation. First, we observe that the ME-level UNI-UNI (customer) relationship is really a UNI_C-to-UNI_C ME interaction, as viewed more practically from the Recommendation G.8010 perspective. Likewise, the ME-level UNI-UNI (transport provider) relationship is more importantly a UNI_N-to-UNI_N ME relationship, as viewed from the Recommendation G.8010 vantage point. Other important ME interactions that come into sharper relief include the intradomain versus interdomain OAM flows as well as those that are link OAM flows restricted either to the customer-totransport provider access link or to the transport provider-to-transport provider NNI.
Chapter 11
418
Table 11-13. Mapping maintenance entities to OAM flows (adapted from ITU-T draft Rec. Y.ethoam)
G.8010
Y.1730 ME
OAM flows
UNI-UNI (customer)
UNI-UNI Flow
UNI-UNI (provider) Segment (PE-PE) intra-provider
Transit Flow Transit Flow
Segment (PE-PE) inter-provider Segment (any to any)
Transit Flow Transit Link Flow Transit Flow Transit Link Flow UNI Link Flow Transit Link Flow
ETY Link OAM -UNI ETY Link OAM -NNI
ME UNI_C to UNI-C ME UNI_N to UNI_N ME Intra Domain ME Inter Domain ME
Access Link ME Inter Domain ME
With OAM ME relationships and reference point locations now understood, we will consider the specific functions that are necessary for the practical operation of an OAM ME capability. In this regard, the following functions have been identified as being vital for such a capability: • Continuous connectivity check (CC) • Alarm suppression function • Intrusive Loopback and nonintnisive Loopback • Path trace • Discovery • Performance monitoring • Survivability function (e.g., protection switching, restoration, etc.) The above are broadly representative of the OAM functions that have been identified as important by standards-making organizations and forums like the ITU-T, IEEE, and the Metro Ethernet Forum (MEF). Although these groups are at various stages in their independent definition of the mechanism to convey the OAM information across the different transport networks, they are uniform in their decision to adopt a frame-based approach. From its perspective, the ITU-T has proposed a single generic format (Figure 11-27) for all Ethernet OAM messages.^^ It is differentiated from the standard Ethernet data frame by its Ethertype (which has not yet been allocated by the IEEE and which will likely have different values for
Ethernet Services Over Public WAN
419
customers versus transport provider networks) and by its direct support for a field differentiating the OAM type, which identifies such events as loopbacks, performance monitoring, and defect and anomaly signaling. Also included in the frame format is its optional support for VLAN tags, which would indicate the service instance corresponding to the OAM message (e.g., only bridge nodes participating in that service instance will process/forward the OAM message).
OAM MAC DA
1
OAM MAC SA
1 Ether type (VLAN) 1
VLAN Tag
1 Ether type (OAM) 1 VER Type
1
OAM Type
Length Service-ID
OAM Data
1
^^^
1
Figure 11-27. Generic OAM message format
11.7.
PROTECTION AND RESTORATION
For WAN transport, the service protection will typically be provided by the underlying transport network. Service restoration can also be provided, however, at the Ethernet layer using Ethernet protocols. Both approaches are summarized briefly here.
420
Chapter 11
11.7.1 Service Protection or Restoration Provided by the Transport Network Transport networks have traditionally been engineered for ultra-high service availability in order to ensure reliable communications of emergency voice traffic. The objective is for a fault to be detected and the service restored through a protection switching action within 60 ms (10 ms for the detection and 50 ms to complete the switch). A protection switch is the routing of the traffic from the failed facility or equipment to backup facilities or equipment that has been reserved in advance.*^ In most cases, the protection channels are permanently reserved, with their bandwidth either being unused during fault-free conditions or carrying low-priority traffic that can be dropped if the channel is required for protection. In the case of TDM transport systems such as SONET/SDH, the protection switch is between TDM channels or fibers [14]. In the case of ATM, the protection switch is between virtual circuits [15]. The Link Capacity Adjustment Scheme (LCAS) discussed in Chapter 4 provides an alternative approach for restoring data services. For constant bit rate services such as voice, it is necessary to have protection bandwidth that is at least equal to the service bandwidth. In the case of packetized data, however, it is possible in many cases to maintain the service connection at a lower data rate in the event of a network problem. LCAS provides this capability through its control of virtually concatenated SONET/SDH, asynchronous/PDH, or OTH channels. As discussed in Chapter 4, virtual concatenation is the construction of a larger channel through the combining of multiple smaller member channels. The data is then interleaved onto the constituent member channels in a byte-by-byte round-robin basis. The individual members can take different routes through the transport network with the NE that terminates the virtually concatenated channel providing the buffering to compensate for the differences in delay that result from the different routes. In the event that a subset of the members fail, LCAS provides the signaling mechanism between the virtually concatenated channel's source and sink to allow the source to place the traffic onto only those members that have not failed. This allows for a very powerful new paradigm for network operators, since, for services that can tolerate the reduced data rate during faulted conditions, the operator can now provide very fast service restoration without having to reserve dedicated protection channels.
Ethernet Services Over Public WAN
421
11.7.2 Service Restoration at Layer 2 When constructing a complex Ethernet network, it is important that only a single switching path exist between any pair of nodes. Otherwise, loops would exist that cause instability in forwarding tables and would cause broadcast data to be unnecessarily multipUed (i.e., broadcast storms). The Ethernet solution to this issue is the spanning tree protocol (STP). The STP sends Layer 2 Control Protocol messages between the Ethernet nodes in order to establish an overlaid logical tree structure on the network. Messages are periodically sent between adjacent nodes to confirm that no changes have occurred to the network connectivity. When a fault occurs, the node detecting the failure (i.e., the change in network connectivity) will initiate a new spanning tree construction. The network is then unavailable for the 30-60 seconds that it can take for the STP to resolve. A new rapid STP (RSTP) protocol has been developed, but it still takes seconds to resolve. While this is a very powerful and flexible service restoration tool, there are several issues when it is applied to Ethernet WAN services. Of course, the larger the network in terms of number of nodes and geographical reach, the slower the STP resolution. Another potential issue occurs if the network operator uses Ethernet routing technology within its network. Here, conflicts between customer and network operator STPs must be avoided. Another issue is the interaction of the 50 ms protection switching (e.g., SONET/SDH) with the STP. If the Ethernet node and the transport network node both detect the fault, it would cause unnecessary extended network unavailability if the STP were initiated for a fault that will be quickly protected by the SONET protection switch. It is desirable, then, for the Ethernet node to wait at least 50 ms to see whether the problem clears before it initiates the STP.
11.8.
CONCLUSION
Due to a convergence of business requirements for higher data connectivity rates and more flexible services, along with the availability of new data transport technology, wide-area Ethernet data transport is becoming increasingly important. This chapter has provided an introduction to some of the new technology and standards that have been developed (and are being developed) to enable carriers to provide Ethernet transport services. A key point is that this is an evolutionary approach that builds on the customers' capital investment in Ethernet technology and the transport providers' SONET/SDH backbone networks and OAM procedures. All the pieces are in place for offering EPL services. The tools and standards to
422
Chapter 11
provide the more complicated ELAN, EVPL, and EVPLAN services are currently being developed in the various standards bodies. While some proprietary solutions exist, carriers require standards-based solutions for any significant deployment of new services. The use of standards helps guarantee that the solution will be available from multiple vendors, will be stable and supported for many years, and ideally should also be less expensive due to economies of scale. While each carrier will probably want to offer variations on the basic service types discussed here in order to differentiate them from their competitors, these services provide the framework from which they will build.
11.9.
NOTES
1. A topological component describes a portion of a layer network in terms of the topological relationships between the (flow) points. See Chapter 2 for a discussion of layer networks. 2. It is also possible to provide EPL and EPLAN using constant-rate ATM or MPLS virtual circuits instead of TDM circuits. 3. Much appreciation is extended to Dr. Gilad Goren of Native Networks (Israel) for his valuable contribution to the development of the ideas and material discussed in this section of the chapter. 4. See IETF RFC 3032. 5. See IETF RFC 3270. 6. Cisco, Agere Systems, PMC-Sierra, and others submitted a technical contribution to the May 2003 ITU-T SG15AVP3 meeting proposing that CoS and DP information be encoded in a GFP (Recommendation G.7041) extended header. See WD34, "A GFP based approach for the proposed Generalized Multiplexing Procedure." This proposal is currently documented in the G.7041 Living List for future consideration. 7. Alcatel SEL presented a technical contribution to an ITU-T SG15AVP3 interim meeting (January 2004) discussing the modifications outlined above. See "Future CO-PS layer network." 8. For more details on the UNI, see the discussion in Section 11.4. 9. Note that EoT (i.e., SONET/SDH, PDH, OTN, and MPLS) is not the only technology approach to interconnecting NNIs. Ethernet physical interfaces themselves (ETy) could be used as the NNI. See Clause 6, ITU-T Recommendation G.8012. 10. Administrative domains within the same provider network typically demarcate either a technology or a geographical or some other distinguishing boundary. For different providers, the separation boundary is usually based upon infrastructure ownership. 11. Standardization efforts are most advanced for the case of the NNIp and continue to progress for the NNIM and NNIc- To simplify the discussion, we will focus only on the NNID.
Ethernet Services Over Public WAN
423
12. See ITU-T Rec. G.8012, Clause 6.1 ff, for further details on LAPS and the other mapping methodologies. Note that LAPS has several known deficiencies versus GFP. These are discussed in detail in the Lucent Technologies white paper, authored by HernandezValencia, E., "Generic Framing Procedure: A Next Generation Adaptation Protocol for Data Transport over SONET/SDH and Optical Transport Networks" (2001). As an additional note, the ITU-T has designated that GFP be used at international network boundaries or at the boundaries of different network operators unless an alternative is mutually agreed to. 13. See ITU-T Rec. G.7041 and ITU-T Rec. G.806 for a more detailed description of the GFP client signal mapping process. 14. Note that sink and source Ethernet processes depend on the direction of the link connection and information transfer. 15. See the discussion by Varma and Daloia in Chapter 2 of this book. 16. See Section 11.3 for a more detailed discussion on the CO-CS and CO-PS networks. 17. The specific 0AM frame formats adopted by the IEEE 802.3ah and the MEF are discussed respectively in IEEE Draft P802.3ah (Clause 57) and in MEF draft, Service OAM Requirements and Framework.
18. In mesh networks that are constructed with digital crossconnect systems (DCS), it is possible to have the DCSs restore the traffic by communicating among each other to determine the protection path through the network in response to a fault. While this approach typically allows the network operator to reserve much less overall network bandwidth for protection, it can be difficult to meet the 50 ms switch time.
11.10. REFERENCES [1] ITU-T Recommendation G.709), Interfaces for the optical transport network (OTN), 2001. [2] ITU-T Recommendation G.707, Network node interface for the Synchronous Digital Hierarchy (SDH), 2003. [3] ITU-T Recommendation G.7043/Y.1343, Virtual concatenation of Plesiochronous Digital Hierarchy (PDH) signals, 2004. [4] ITU-T Recommendation G.7042, Link Capacity Adjustment Scheme (LCAS) for Virtual Concatenated signals, 2004. [5] ITU-T Recommendation G.7041 A'. 1303, The Generic Framing Procedure (GFP), 2004. [6] ITU-T Recommendation G.8040, GFP frame mapping into Plesiochronous Digital Hierarchy (PDH), 2004. [7] ITU-T Recommendation G.8021A^.1341, Characteristics of Ethernet transport network equipment functional blocks, 2004. [8] ITU-T Recommendation G.809, Functional architecture of connectionless layer networks, 2003. [9] ITU-T Recommendation G.8011 A". 1307, Ethernet Services Framework, 2004.
424
Chapter 11
[10] ITU-T Recommendation Gm\\.\fY.n()l.\, Ethernet Private Line Service, 2004. [11] ITU-T Recommendation G.8012A^.1308 (2004), Ethernet UNI and Ethernet NNI [12] ITU-T Recommendation G.805, Generic functional architecture of transport networks, 2001. [\2>]\1\}~T RQQommQnd2ii\or\ Gm\()IY A2>()6, Ethernet Layer Network Architecture,2()QA. [14] ITU-T Recommendation G.808.1, Generic Protection Switching —Linear Trail and Subnetwork Protection, 2003. [15] ITU-T Recommendation 1.630, ATM Protection Switching, 2000.
Chapter 12 ETHERNET SERVICES OVER MPLS NETWORKS
Iftekhar Hussain Cisco Systems, Inc.
12.1.
VIRTUAL PRIVATE NETWORKS
For corporations and enterprises with geographically distributed sites, network connectivity between different sites is essential to meet increasing demands for voice, video, and data communication. Initially, corporate networks were interconnected using dedicated transport facilities such as DSl/El and DS3/E3. Service Providers (SPs) leased transport facilities as a service to their customers. A network in which sites are interconnected using dedicated transport facilities is called a private network. Using this type of network connectivity, the cost of offering a private network services was very high for the SPs and their customers. Additionally, the provisioning of new services was a slow and laborious task.
12.1.1 Traditional Layer 2 Virtual Private Networks A network in which sites are interconnected using circuits over a shared network infrastructure is called Virtual Private Network (VPN). When all sites in a VPN belong to the same organization, a VPN can be viewed as providing intranet connectivity. On the other hand, when various sites in a VPN belong to different organizations, the VPN can be thought of as providing the extranet connectivity. The fact that multiple VPN customers share the network infrastructure is what sets a VPN apart from a private network. The shared network infrastructure is known as the VPN backbone.
426
Chapter 12
The sharing of the VPN backbone allows SPs to offer VPN services to their customers at lower costs. A VPN that interconnects a set of customer sites over a shared network infrastructure and allows them to communicate based on Layer 2 frames is known as Layer 2 VPN (L2VPN). In contrast, a VPN that interconnects a set of customer sites over a shared network infrastructure and allows them to communicate based on Layer 3 addresses (e.g., IP addresses) is known as Layer 3 VPN (L3VPN), Thus the distinguishing characteristic of an L2VPN, in comparison to an L3VPN, is that in L2VPNs packet forwarding is carried out at L2 such as ATM, FR, and Ethernet. Figure 12-1 shows an example of a L2VPN using ATM Virtual Connections (VCs).
Customer Hcigc (F'F.) Dc\ ice
Provider Edge (PE) Device
Shared VPN Backbone
Figure 12-1. Traditional Layer 2 Virtual Private Networks
12.1.2
Classification of VPNs
Generally, VPN services may be provisioned and managed by SPs or customers. A VPN for which the SP participates in management and provisioning of the VPN service is termed a Provider Provisioned VPN (PPVPN). There are many ways in which an SP can participate in the management and provisioning of a VPN service. Correspondingly, there is a wide spectrum of VPN types. The following attributes are useful for classifying VPNs: • Service Layer (e.g., L2 versus L3): the layer at which VPN service is offered by the SP • VPN Edge Device (e.g., CE-based versus PE-based): the device where VPN-specific functions are performed • Service Connectivity (e.g., point-to-point versus point-to-multipoint): the type of connectivity the VPN service offers
Ethernet Services Over MPLS Networks
All
One such taxonomy of PPVPN technologies is depicted in Figure 12-2. PPVPN
Point-to-Multipoint
Point-to-Point
CE-based
RFC2547 Style
Multicast
Virtual Router Style
Unicast
Figure 12-2. Taxonomy of PPVPN technologies [1]
12.1.3
Multiservice Converged Packet Switched Backbone
Although L2VPNS based on ATM VCs and FR Data Link Connection Identifiers (DLCIs) were easier to provision and had a lower cost than dedicated leased lines, they still had some drawbacks. For example, this type of L2VPN approach restricted the SP's backbone to a single transport technology such as ATM/FR links, which made it burdensome to share the same physical transport facilities for Internet and VPN traffic. Even when Internet and VPN infrastructures could share the backbone transport facilities, they needed separate administration and maintenance. Although provisioning of ATM VCs and FR DLCIs was easier and relatively less cumbersome than dedicated lines, it was still tedious. For example, adding a new customer site to an existing VPN required provisioning an ATM VC to every other site in the VPN. Traditional L2VPNs work well from the customer's point of view; however, the costs of maintaining separate network infrastructures and the administrative burden of provisioning these VPNs have led SPs to migrate their legacy L2 and emerging L3 services onto a common multiservice IP/MPLS packet switched network (PSN). The following discussion assumes some basic familiarity with the MPLS (refer to Appendix A for a quick overview of MPLS technology).
428
n.l.
Chapter 12
L2VPNS OVER MPLS BACKBONE
There are two main types of L2VPN services that an SP can offer to its customers, namely, Virtual Private Wire Service (VPWS) and Virtual Private LAN Service (VPLS) [2]. A VPWS is a point-to-point L2VPN service. In contrast, a VPLS is a point-to-multipoint L2VPN. A VPLS emulates Local Area Network (LAN) service over the Wide Area Network (WAN), which allows interconnecting LAN segments on geographically dispersed customer sites, as if they were connected to the same LAN. In both VPWS and VPLS, L2 frames are transported across the IP/MPLS backbone. In both types of L2VPN, a CE (e.g., CEl) transmits a frame to a PE (e.g., PEl); PEl encapsulates the frame in one or more additional headers and transports the frame to another PE (e.g., PE2). The PE2 in turn removes the encapsulation header and sends the frame to another CE (e.g., CE2). With the exception of some specific services such as point-to-point versus point-to-multipoint, both VPWS and VPLS employ a number of functional components such as header encapsulations. Therefore, to avoid repetition of common functions, it is more efficient to decompose L2VPN functional components into service independent (common) and servicespecific components.
12.2.1 L2VPNs Architecture Generic Components This section describes generic components that are common in all L2VPNs. The service-specific components and architectures are described later in the pertinent sections. 12.2.1.1 Attachment Circuit (AC) In all types of L2VPNs, a CE device (a router or a switch) attaches to a PE device via a physical connection (e.g., Ethernet port) or logical connection (e.g., VLAN port) termed an Attachment Circuit (AC). An AC may be an Ethernet port, a VLAN port, a FR DLCI, or an ATM VPI/VCI, and so forth. An AC carries L2 frames between a pair of CE and PE devices. 12.2.1.2 Pseudowire (PW) In all types of L2VPNs, a L2 frame between two PE devices is carried over another logical connection termed a Pseudowire (PW). Thus any given L2 frame first travels on an AC from a CE to a PE, then on a PW from a PE to another PE, and finally on another AC from a PE to a CE.
Ethernet Services Over MPLS Networks
429
A PW is a mechanism that emulates the essential attributes of a telecommunications service such as FR, ATM, Ethernet, Time Division Multiplexing (TDM), and SONET/SDH over IP and/or MPLS PSN [3]. A protocol data unit (PDU) that contains all the data and control information necessary to emulate the desired service is known as a PW-PDU. Figure 123 shows the logical protocol layering of a generic PW for different types of services. It is worth mentioning that each type of service is emulated using a separate PW. For a given emulated service, to a CE device, the PW appears as an unshared dedicated circuit. The PW protocol layers are discussed in the following sections. ^l n.lriulurrtP Klhrriic(, KK, Ccll(AIM) Ar.MAAl.5) V. J K. )
DSI, DSV Kl,
Packet (r.R., SONKI/SDII SI'K, \'r, N\l)SO)J
('rll(AT.M) ATMAAL5)
l)SI,l)SV Kl, K3) ^
SONK.I SDH
Service PayloaJ (such a.s packet, cell, ilMrcain. and stnicliircd bilslrc:ini)
PayioLid liming nw y been rr using Rc:»l TimeProtocol (R'lT)
Pavload Convcrccncc TiminK SetiuciK'ing PWlXmulliplcxer PSN Convergence PSN n:U:i Ijnk V ^J
Pliy.Mcal
SucilasPackcI(>^cI SONI: ! "snilurl-lhcmcWAcrlilKT
Pnckot Switched Network (PSN)
Figure 12-3. Protocol Stack Model for PW Emulation Edge to Edge (PWE3)
12.2.1.2.1
Encapsulation Layer
The PW encapsulation layer contains three sublayers, namely, Payload Convergence, Sequencing, and Timing. 12.2.1.2.1.1 Payload Convergence The primary function of the Payload Convergence Layer is to encapsulate the incoming (CE to PE direction) payload in PW-PDUs. In the outgoing direction (PE to CE direction), the Convergence Layer replays the native data units on the physical interface attached to the destination CE.
430 Xl.lA.lA.l
Chapter 12 Timing
Delivery of native services such as structured and unstructured bistreams requires availability of an accurate timing recovery mechanism depending upon the characteristics of the transport network. For example, the clocks used to synchronize SONET equipment are stratum 3 (or better accuracy) clocks that are normally traceable to a primary reference source (a clock source that provides a timing signal with long-term accuracy at 10"'' or better with verification to the Universal Time Coordinated (UTC)). Therefore, the emulated service must also duplicate the timing characteristics as closely as possible to that expected of a native service. The Timing Sublayer provides two synchronization-related functions, namely, clock recovery (the extraction of output transmission bit timing information from the delivered packet stream) and timed payload delivery (the playing out of noncontiguous PW PDUs to the PW output interface with a constant phase relative to the input interface). Generally, timing signal can be distributed through external mechanisms such as Building Integrated Timing System (BITS), Stand Alone Synchronization Equipment (SASE), and Global Positioning System (GPS) or extracted from the bitstream using an adaptive clock recovery mechanism. To facilitate extraction of timing information from the packet stream at the receiver, the timing information from the sender can be carried using the Timestamp field of the Real Time Protocol (RTP) [4]. 12.2.1.2.1.3 Sequencing Functions In an IP/MPLS network, the packets carrying PW-PDUs may arrive out of order, may arrive duplicated, or may never arrive at the destination PE. The Sequencing Layer provides services that enable in-order and unduplicated frame delivery to the CE and additionally enable detection of frame loss. 12.2.1.2.2
Demultiplexer Layer
A PSN tunnel is a logical link that provides a data path across the backbone. The main function of the PW Demultiplexer Layer is to provide a demultiplexer field to allow multiple PWs to be carried in a PSN tunnel. The demultiplexer field allows the receiving PE device to distinguish one PW from others. In general, depending on the tunnel protocols, the demultiplexer field may have a different format. For example, when PWs are being carried in an MPLS tunnel, the PW demultiplexer field contains an MPLS label. To summarize, the PW Demultiplexer Layer provides the
Ethernet Services Over MPLS Networks
431
ability to carry multiple PWs within a PSN tunnel transparently across the backbone. Other than PE devices that must perform functions related to PWPDUs (such as encapsulation, decapsulation, and sequencing), PWs are invisible to the other backbone devices (e.g., P routers). 12.2.1.2.2.1 Fragmentation and Reassembly After accounting for the PW and the packet switched network headers (such as IP or MPLS headers) if the combined size of the payload and the associated network headers exceeds the path Maximum Transmission Unit (MTU) of the network, fragmentation and reassembly at the PE devices is required in order for the packet to be delivered across the PSN. In theory, CEs should be able to adapt the packet size according to the path MTU to avoid fragmentation in the network. In practice, there may be situations when a CE lacks this capability. If the CE cannot adhere to an acceptable MTU size, the PE should be able to perform PW fragmentation and reassembly. 12.2.1.2.3 Service-Specific PW Preprocessing In general, at the PE some form of service-specific preprocessing (such as Ethernet bridging, ATM VPIA^CI header translation, SONET Virtual Tributary (VT) cross-connection) on the native data units received from the CE is needed before PW PDUs can be transmitted on the PW. The PW preprocessing can be divided into two components, namely. Native Service Processor and Forwarder.
12.2.1.2.3.1 Native Service Processing (NSP) The purpose of the NSP is to perform service-specific operations based on the semantics of the payload. In the case of Ethernet service, for example, the NSP function includes frame processing and may include additional functions such as stripping, overwriting, or adding VLAN tags, physical port multiplexing and demultiplexing, PW-PW bridging, L2 encapsulation, and so forth. 12.2.1.2.3.2 Forwarders A forwarder is a logical module in the PE that that selects the PW to use to transmit a payload received on an AC. The selection of a particular PW may be based on incoming AC, the contents of the payload (e.g., packet
432
Chapter 12
header), or some statically/dynamically configured forwarding information. Based on the type of service (e.g., point-to-point or point-to-multipoint), the forwarder may selectively forward payload from one AC to exactly one PW or one or more ACs to multiple PWs (Figure 12-4 and Figure 12-5 depict a point-to-and point-to-multipoint forwarder, respectively).
AC
Forwarder
PW
Forwarder
MPLS Network Figure 12-4. Forwarder for a point-to-point service.
MPLS Network Figure 12-5. Forwarder for a point-to-multipoint service.
AC
Ethernet Services Over MPLS Networks
433
12.2.1.3 VPN Tunnels In all type of L2VPNs, data traffic between PE devices is transported over VPN tunnels. A VPN tunnel is a logical link between two PE (or CE) entities that is used to carry VPN traffic across the backbone. In general, a tunnel is implemented by encapsulating packets within another header that are transmitted between those two entities. For example, in PE-based VPNs, a PE-PE tunnel provides connectivity between two PE devices. 12.2.1.3.1 Motivations for Tunnels One of the main motivations for the use of tunneling in VPN applications is to be able to transport customer packets with nonunique addressing information between the VPN edge devices. For example, customer networks often use private or nonunique IP addresses. However, in many VPN applications (as, for example, in L3VPNs [5]), a single VPN edge device such as a PE router can provide VPN service to multiple customers even if those customer networks have overlapping addresses. The fact that the customer addresses are not globally unique means that IP packets from a customer cannot be transmitted to the correct destinations over the shared VPN backbone in their native form. In other words, some form of additional header encapsulation (tunneling) must be utilized to forward packets to their correct destinations. Thus, a tunneling protocol attaches an extra encapsulating header (which, in the case of MPLS, corresponds to one or more labels) to a VPN packet, and this additional header information is then used for forwarding the packet between the VPN edge devices. There are other important reasons for using tunnels in VPN applications, such as the need to isolate traffic from different customers and to provide different quality of service (QoS) and security characteristics. For example, QoS and security requirements of different VPN customers may differ and can be satisfied by using different tunnels with the appropriate characteristics. 12.2.1.3.2 Hierarchical Tunnels If you were to form VPN tunnels across the backbone for each instance of VPN, devices in the backbone such as P routers would need to be VPNaware and to maintain state for each VPN tunnel, which from a network scalability point of view is highly undesirable. A better solution is to establish one tunnel between each pair of VPN edge devices and then multiplex multiple VPN-specific tunnels through the single outer tunnel. With this approach, the amount of state depends only on the number of VPN
434
Chapter 12
edge devices, not on the number of VPNs. A tunnel that encapsulates one tunnel within another is known as a hierarchical tunnel, 12.2.1.3.3 Tunneling Protocols Several protocols can be used to establish and maintain VPN tunnels, including Generic Routing Encapsulation (GRE) [6,7], IP-in-IP [8,9], IPsec [10-13], and MPLS [14,15]. Each tunneling protocol can be characterized in terms of a common set of characteristics such as the format of the encapsulation header and the overhead introduced by the encapsulation, how the VPN-related information is inferred from the packet's encapsulation header, whether an explicit signaling protocol is required for setting up the tunnel, whether the tunneling protocol allows hierarchical tunneling, and whether the tunneling protocol supports mechanisms to detect tunnel failures and to respond appropriately to restore service. This chapter focuses mainly on the use of MPLS based tunnels. 12.2.1.3.4 MPLS Tunnels This section briefly describes distinguishing characteristics of MPLS tunneling protocols for VPN applications. A detailed survey of other tunneling techniques can be found in [16]. In MPLS networks, routers are known as label switching routers (LSRs) that forward packets based on labels. A label is a short, fixed-length, physically contiguous identifier, which is used to forward a packet instead of an IP address and usually has a local significance. The sequence of LSRs that a labeled packet traverses, starting at ingress LSR and terminating at the egress LSR, is called label switched path (LSP) (see Appendix A for an overview of MPLS). In the PPVPN terminology, an LSP corresponds to a VPN tunnel. In general, MPLS tunnels have following characteristics: • Tunnel encapsulation is based on label stack, and the label value is used as multiplexing field • Tunnels can be multiplexed within other tunnels. For example, in the MPLS backbone, P routers only need to maintain state for the topmost label in the label stack. This means that the VPN-specific state of the nested tunnels is not visible to the P routers. • Tunnels are set up and maintained using signaling protocols such as Label Distribution Protocol (LDP) or Resource Reservation Protocol (RSVP).
Ethernet Services Over MPLS Networks
435
12.2.1.3.5 Carrying PWs over MPLS Tunnels The protocol layering to carry PW emulated services (e.g., Ethernet) over MPLS networks is depicted in Figure 12-6. Each packet header carries two labels, namely, a bottom label and a top label. The bottom label represents a particular PW within the tunnel and provides the PW demultiplexing function to distinguish one PW from another. The top label represents a route (an LSP) across the backbone to the egress PE. The bottom label is invisible to the devices in the backbone until the packet arrives at the PE. In other words, the bottom label is meaningful only to the PE devices. In the remainder of this chapter, the term tunnel label refers to the top label and the term PW label refers to the bottom label.
Slrip-ofTprcaFTihic an J frame check sum (FCS)
Attach the daia link iivcr hcadci —^
Attach the physical !a\cr hcadt
Paytoad (LthcrnciFraiix)
'C:^
Control Pa>!oaJ (Ethernet Framed WurJ
FW Ubfl
Tunnel j [.abd
Tunnel Label
PIT Header
SON FT Header
Payload (r.lhL-nicl Frank)
Conlrol Word
Uhtl
Tunnel Label
Header
Attach data link (PPP) and _ ph>$icall3>ei{S0NRl) headers Packet is lahcl.su itched _ inthcMPl.^neiuork
Figure 12-6. Protocol layering for carrying Ethernet PWs over MPLS tunnels
|
Header
436
12.3.
Chapter 12
METRO ETHERNET SERVICES
This section briefly describes Ethernet services with the goal of motivating ensuing discussions relating to the delivery of Ethernet services over MPLS using VPWS and VPLS L2VPNs. For complete details on various Ethernet service features and attributes, refer to Chapter 10, "Metro Ethernet Services".
12.3.1 Ethernet Virtual Connection (EVC) To visualize and characterize Ethernet connections like other L2 circuits such as FR DLCI and ATM VCs, the notion of Ethernet Virtual Connection (EVC) is very useful. An EVC is an association between one or more User Network Interfaces (UNIs) for exchanging an Ethernet frame among the associated entities. An EVC can refer to point-to-point or multipoint-tomultipoint connectivity. A point-to-point EVC is associated with exactly two UNIs. A multipoint-to-multipoint EVC is associated with two or more UNIs. A point-to-multipoint EVC is a special case of the multipoint-to-multipoint EVC in which one UNI is designated as the root node and the remaining UNIs as the leaf node such that a frame received at the root UNI is transmitted to all leaf nodes. Based on the notion of EVC, there are two types of Ethernet services that can be defined, namely, Ethernet Line (ELine) Service and Ethernet LAN (E-LAN) Service.
12.3.2 E-Line Service E-Line Service uses point-to-point EVC and thus can be used to offer point-to-point Ethernet Virtual Leased Lines between two CE devices (refer to Figure 12-7). From the previous discussion, recall that E-Line Service is analogous to traditional L2VPN services based on ATM or Frame Relay virtual circuits. Later on, we will discuss how E-Line Service can be offered over an MPLS backbone using VPWS.
12.3.3 E-LAN Service E-LAN Service is based on multipoint-to-multipoint EVC and thus can be used to connect geographically dispersed customer sites in a Metro area and make them appear as if they were on the same LAN (refer to Figure 128). In the following section, we will discuss how E-Line Service can be offered over an MPLS backbone (WAN) using VPLS.
437
Ethernet Services Over MPLS Networks
T^^^"SP•fff ^ '^ '^ ^n\ In t^^'
Metro Ethernet Network (MEN)
^=3
CH
Figure 12-7. An example of Ethernet Line (E-Line) service
UT'i-r^
Metro Ethernet Network (MEN)
\^ ^^• • • • S T n In Cli
Figure 12-8. An example of Ethernet LAN (E-LAN) service
12.4.
METRO ETHERNET SERVICES OVER MPLS
This section describes the VPWS and VPLS architecture framework, which can be used to provide E-LINE and E-LAN services over an MPLS backbone.
438
Chapter 12
12.4.1 Emulation of E-Line Services using VPWS As discussed previously, a VPWS is an L2VPN that provides L2 pointto-point services. In VPSW, PE devices behave as MPLS-capable L2 switches and provide a logical interconnect of L2 circuits such that pair of CE devices appear to be connected by an L2 virtual circuit. When a CE device transmits a frame on such an L2 virtual circuit, the CE device at the other endpoint of the virtual circuit receives the frame. In the VPWS case, the forwarding of frames from one CE device to another is completely determined by the virtual circuit on which the frame is transmitted. In other words, the forwarding of frames is not affected by the contents of the frame header, and the PE acts as a virtual circuit switch.
12.4.1.1 VPWS Reference Model An Ethernet PW allows Ethemet/802.3 PDUs to be transported across an IP or an MPLS network. A point-to-point Ethernet PW emulates a single Ethernet link (an EVC) between exactly two endpoints. Figure 12-9 shows the VPWS reference model for emulation of point-to-point Ethernet services over MPLS. The protocol stack for VPSW over SONET-based MPLS transport is also depicted in Figure 12-9. Most of the entities in this reference model such as AC, PW, CE, and PE have already been described in the previous section. In general, VPSW can be used to offer a variety of L2 point-to-point services such as Ethernet, ATM, FR, TDM, and so forth (for example, refer to Figure 12-3). This is accomplished by using service-specific encapsulation methods to carry L2 frames across IP or MPLS networks. For this purpose, different encapsulation methods have been specified [17-21]. This section specifically describes the use of the encapsulation method defined in [17] to transport Ethernet frames over MPLS networks (to see protocol layering for Ethernet over SONET-based MPLS network, refer to Figure 12-6) and the use of the LDP extension defined in [22] for setting up Ethernet PWs.
Ethernet Services Over MPLS Networks
439
Ijmilalcd SciAJce
AUachniciit Circuit • r* AlU
PR-PI' Pscuclowirc
-V AUachinciit Circuit-
— lM-;-PI- Tunnel
I-thcrnc( I'ramcs Cl-2
MPLS Network
MPLS tunnel is setuj"'. usin-LDPorRSVP-T!-:
Figure 12-9. VPWS Reference Model for emulation of E-Line service over MPLS network
12.4.1.2 Ethernet Modes (Raw versus Tag) There are two modes in which an Ethernet can operate, namely, raw mode or tag mode. When operating in the tag mode, each frame carries an 802. IQ VLAN tag and the Native Service Processing (NSP) component at the two PW endpoints processes the tag. By contrast, in the raw mode, the NSPs do not process the tag (i.e., the VLAN tag passes transparently through the NSP). Depending on whether the tag is service delimiting or nonservice delimiting, the NSF may need to handle it differently in the tag mode and the raw mode. 12.4.1.3 VLAN Tag Processing The VLAN tag is said to be service delimiting if the tag is placed in the frame by the service provider equipment and the tag is meaningful to that equipment only. For example, consider a deployment scenario where the service provider has employed a LAN switch to aggregate traffic from multiple customers. In this case, the LAN switch may apply VLAN tags to distinguish streams of customer traffic from one another, and then forward the frames to the PE. The tag is said to be nonservice delimiting if the CE placed it in the frame and the tag is meaningless to the PE. Whether a tag is service or nonservice delimiting has an important implication for the processing of frames at the Ethernet PW endpoints. For example, in the raw mode, the service-delimiting tag is not transmitted over the PW. In this case, if the service delimiting tag is present in the frame
440
Chapter 12
received from the CE, the PE strips it off before transmitting on the PW. In the tag mode, the service-dehmiting tag must be transmitted. In this case, if the service-dehmiting tag is not present in the frame received from the CE, the PE must prepend the frame with a dummy VLAN tag before sending the frame on the PW. 12.4.1.4 Establishing Ethernet PWs via LDP As discussed previously, to transport L2 PDUs from ingress PEl to egress PE2 across an MPLS network over a tunnel, PEl pushes two labels (a tunnel label as the top label and a PW label as the bottom label). The distribution of tunnel labels is accomplished using LDP [23] or RSVPTE[24]. This section describes how LDP is used to signal PW labels. A bidirectional PW consists of two LSPs between PEl and PE2 (refer to Figure 12-9). The first step toward setting up a PW involves establishment of LDP session between PEl and PE2. To establish an LDP session between them, the PEs must know each other's address. A PE may learn a remote PE address through configuration or autodiscovery procedures (for example, using BGP [25]). PEl and PE2 then establish an LDP session in downstream-unsolicited mode. Before a PE can begin signaling PW labels to a remote PE, the PE must know the Target Attachment Identifier (TAI) of the remote PE. In VPWS, a PW can be thought of as connecting exactly two forwarders. To identify a particular forwarder, through configuration or based on some algorithm, each forwarder is associated with an Attachment Identifier (AI) that is unique in the context of the PE router in which the Forwarder resides. The combination of can identify a globally unique forwarder. To identify a set of forwarders that is part of the same group, an AI is defined to contain an Attachment Group Identifier (AGI) plus an Attachment Individual Identifier (All). We may think of an AGI as a VPN-id or a VLAN identifier. For example, if a bidirectional PW connects PEl with PE2, the LSP in the PEl to PE2 direction can be identified as « P E 1 , , PE2, , PEl,
Ethernet Services Over MPLS Networks
441
as CE-facing interface MTU size). For PWs setup, LDP has defined the following two new FEC TLVs: • PWid FEC (FEC type 128) • Generalized ID FEC (FEC type 129, as shown in Figure 12-10)
rrr
1 1
1 1 I
FEC Type(=129)
AG I
1 1 1~n
1 II 1
c
1
|15|!6|
1
1
1
1
1
123 124 1
11 J T3in
PW info length
PW Type
Length
II
Value
Value (continued)
SAII
Length
Value
1
\ 1
Value (continued)
TAII
Length
Value
>
i
< t
Value (continued)
Figure 12-10. Generalized ID FEC format [22]
The PWid FEC element is intended for the case in which both PW endpoints have been provisioned with the same AI. On the other hand, the Generalized ID FEC element is used when the PW endpoints can be uniquely identified and when it is desired to auto-discover the identity of the remote endpoints instead of configuring them statically.
12.4.2 E-Line Service Emulation Walk-Through Example This section provides a walk-through example of E-Line emulation over MPLS using the network diagram shown in Figure 12-11. The following discussion assumes that a bidirectional MPLS tunnel between PEl and PE2 has already been established using LDP or RSVP-TE. Hence, the label exchange for the tunnel setup is not shown. The MPLS network is assumed to be based on packet over SONET transport. For the sake of simplicity, the data link (i.e., PPP) and physical layer headers (i.e., SONET) are not shown. The sequence of steps described here may not necessarily be followed by a specific implementation. 1. PEl and PE2 learn each others addresses and establish a LDP session between them. Before PEl and PE2 can begin signaling PW labels, they
442
2.
3.
4.
5.
6. 7.
8.
Chapter 12
need to know TAI. PEl and PE2 can learn addresses and TAI information via configuration or some auto-discovery procedure (such as BGP auto discovery [25]). To setup the PW in the PEl-to-PE2 direction, PE2 allocates a label L2 and advertises this label to PEl using an LDP Label Mapping message with FEC type 129 containing SAII, AGI, and TAIL On receipt of the LDP Mapping message, PEl checks whether the message contains a valid TAI for its forwarders. If PEl cannot use the received TAI to identify one of its forwarders, then PEl sends a Label Release message to PE2, with a Status Code meaning invalid TAI. Assume that the received TAI is valid, thus PE2 accepts this label binding information. Before PE2 can program its MPLS forwarding plane to attach this label as the PW label (bottom label) for PDUs in the PE2 to PEl direction, PE2 needs to ensure that the PW is set up in the opposite (PE2 to PEl) direction. To set up the PW in the PE2 to PEl direction, using similar procedures as described for PE2 earlier, PEl allocates a label LI and advertises this label to PE2 using an LDP Label Mapping message with FEC type 129 containing SAII, AGI, and TAIL At this stage, the bidirectional PW setup is complete (for example, PEl should be using label L2 as the PW label for sending PDU to PE2, and PE2 should be using label 1 for sending PDUs to PEl). Now let us consider the CEl-to-CE2 data flow. In the incoming direction (CEl to PEl), PEl strips off the preamble and frame checksum (FCS) from the received Ethernet frame, prepends the optional control word to the frame, selects the appropriate PW, prepends the PW label L2 to the resulting packet, prepends the appropriate tunnel label to the resulting packet, and then transmits the labeled packet along the tunnel (refer to Figure 12-6 for protocol stack layering). When carrying Ethernet over an IP or MPLS network, packets containing Ethernet PW PDUs may arrive out of order at the destination PE. The control word (a four byte header) provides a sequence field to support in-order frame delivery. The labeled packet is forwarded based on the tunnel label (top label) across the MPLS network. When the packet arrives at PE2, PE performs the reverse set of operations in the outgoing (PE2-to-CE2) direction. For example, PE2 strips off the tunnel and PW encapsulation after processing the frame as required by the control word, strips off the control word, and hands the resulting frame to the Forwarder/NSP, which regenerates the FCS and replays the frame to the attached CE2 interface. A similar set of operations is performed in the other direction of traffic (CE2 to CEl). For example, PE2 strips off the preamble and FCS fields of the Ethernet frame received from CE2, prepends the optional control
Ethernet Services Over MPLS Networks
443
word to the frame, selects the appropriate PW, prepends the PW label LI to the resulting packet, prepends the appropriate tunnel label to the resulting packet, and then transmits the labeled packet along the tunnel (refer to Figure 12-6 for protocol stack layering). Finally, when the packet arrives at PEl, PEl strips off the tunnel and PW encapsulation headers and hands over the resulting frame to the Forwarder/NSP, which regenerates the FCS and replays the frame to the attached CEl interface. 0 ©
nslabUsh PE1-Pi:2 l.DP session for PW setup
Pi:2 signals label 1.2 toi'hl
©
l>i;i siiiiKils label I.I u>l>i;2
"^
I:lhcnicl I'lai
^ Q •:;:
^ CE2
MPLS Network (2) I'W setup is complete
© pi;i F.tlurnct Fr:imc
-^
iMhcrncl
attach cs two la be Is 1.2
Tunnel L:il)cl
©PI:: frnmc
rcnun cs the la K-IS L2
Tunnel
-^
Kthc Fr;i
I1C(
(J) l-orwarcl packet using tunnel label
(^
Similar proccssinu in the rc\ crsc direction o{ How
Figure 12-11. E-Line emulation over MPLS using VPWS service
12.4.3 Emulation of E-LAN Services using VPLS A VPLS is a Layer 2 service that emulates LAN service over the packet switched network and makes it possible to interconnect LAN segments on geographically dispersed customer sites as if they were connected to the same LAN. In a VPLS, the attachment circuit between CE and PE can only carry Ethernet frames. 12.4.3.1 VPLS Reference Model Figure 12-12 shows the VPLS reference model and the internal logical structure of a VPLS PE. Like VPWS, in VPLS, CE and PE devices are connected through ACs. The functionality of a VPLS PE device is quite
444
Chapter 12
Figure 12-12. VPLS Reference Model for emulation of E-LAN service over MPLS network
similar to a VPWS PE. For example, both VPWS and VPLS PE devices can be thought of mapping ACs to PWs. The VPLS PE and VPWS PE devices mainly differ in the forwarder functionality. A VPWS PE forwarder performs one-to-one mapping between AC and PWs, whereas a VPLS PE forwarder performs many-to-many mapping between ACs and PWs. In a VPWS PE, the forwarding decision of a frame is not affected by the content of the frame header but is completely determined by the virtual circuit on which the frame is to be transmitted. That is, a VPWS PE device behaves as a virtual circuit switch. In contrast, in a VPLS device, the forwarding decision of a frame is based on L2 header information (e.g., MAC address). That is, a VPWS PE device behaves as a bridge. A VPLS PE device may be thought of containing a forwarder module and a bridge module(s). The bridge module performs functions such as learning and aging out (removing) Ethernet MAC addresses. The forwarder maps frames from ACs to the appropriate set of PWs. Figure 12-13 shows that CE devices connect to the bridge module of a VPLS PE through ACs. The bridge module attaches to the forwarder through the emulated LAN interface. An emulated LAN consists of a set of VPLS forwarder modules connected by PWs, where PWs are carried over PE-PE tunnels across the backbone. A set of forwarders (no more than one in each VPLS PE) connected by PWs is known as a VPLS instance. Thus, the bridge module of each PE in the VPLS instance attaches (via an emulated interface) to the Emulated LAN. Hence, to a CE device, the VPLS backbone appears as a set of PE bridges attached to an Emulated LAN.
Ethernet Services Over MPLS Networks
445
Figure 12-13. Logical structure of an emulated LAN using VPLS
A physical LAN has built in broadcast and multicast capabilities. For example, when a device sends a frame to a particular device (unicast), all other devices on the LAN receive the frame. So how does a VPLS emulate the broadcast and multicast capabilities of a physical LAN? When a forwarder receives a unicast frame over its emulated LAN interface, unlike real LAN, the forwarder does not send the frame to all other forwarders but only to one forwarder that can deliver the frame to the correct destination address. How does the PE learn about the reachability of a MAC address? Conceptually, a PE maintains a separate L2 forwarding table for the bridge module and the forwarder module. What is the difference between the functionality of two L2 forwarding tables? The L2 forwarding table of a bridge module maps a MAC destination address (DA) to an AC and/or an emulated LAN interface. In contrast, the L2 forwarding table of a forwarder maps a MAC DA to a PW only. As a result, a bridge module performs MAC address learning on ACs as well as emulated interfaces, whereas a bridge module performs MAC address learning on PWs only. This means that when a VPLS forwarder module receives a frame from a PW whose MAC source address (SA) is previously unknown, the forwarder associates this MAC address with the corresponding outgoing PW (refer to [27,28] for further information on MAC address learning procedures). Similarly, a bridge module learns the MAC address on ACs and an emulated LAN interface. In short, the PE bridge function treats the emulated LAN like a normal LAN. Specifically, the PE bridge makes its forwarding decisions as it would on any normal
446
Chapter 12
LAN, based on MAC SA learning. Although conceptually the bridge module's forwarding table and the forwarder's forwarding table are treated as distinct entities, a particular implementation may support these functions within a single forwarding table. 12.4.3.2 Avoiding VPLS Forwarding Loops Connecting a set of forwarders via PWs forms a VPLS instance, and the set of PWs thus forms an overlay topology. Depending on the form of interconnectivity between forwarders, several types of overlay topologies are possible. For example, if every forwarder in a given VPLS is connected by exactly one point-to-point PW to every other forwarder in the same VPLS, a full mesh overlay topology is formed. In a tree-structured overlay topology, every forwarder in a particular VPLS is assigned a particular position (level) in the tree such that a given forwarder is connected via at most one PW to a higher-level forwarder (the root of the tree is at the highest level). By allowing more than one forwarder at the highest level (provided that the forwarders at the highest level are fully meshed), one can form a variant of the tree-structured overlay topology. Each VPLS forwarder performs MAC learning to populate its L2 forwarding table. To do this, each forwarder treats each PW as a bridge port and employs standard MAC learning procedures over the point-to-point PWs. To achieve loop-free forwarding over the overlay topology, forwarding decisions of VPLS forwarders require some coordination. In general, loopfree L2 forwarding requires the use of the Spanning Tree Protocol (STP) between bridges. While STP works well in the enterprise LAN environment, its poor resiliency characteristics may not be suitable for use in the highavailability service provider backbone. VPLS obviates the use of STP in the backbone through a combination of full mesh topology and split-horizon forwarding techniques [26]. For example, in a full mesh overlay topology to ensure loop-free forwarding, when a forwarder receives a frame over a PW it does not forward that frame over any other PW. Thus, when a forwarder receives a unicast frame with a known DA over its emulated LAN interface, the forwarder sends that frame on exactly one point-to-point PW. In contrast, when a forwarder receives a multicast or unicast frame with unknown DA, it sends a copy of the frame over each point-to-point PW in the same emulated LAN. Similarly, in a tree-structured overlay topology, to ensure loop-free forwarding, when a frame is received over a PW from a higher level it is not transmitted over a PW that leads to a higher level.
Ethernet Services Over MPLS Networks
447
12.4.3.3 VPLS PW Encapsulation Like VPWS, in VPLS the payload on the ACs consists of Ethernet frames with or without VLAN headers, and thus VPLS also uses the same Ethernet encapsulation format as described in the previous section. For VPLS, handling of a service-delimiting tag can be summarized as follows. If the packet, as it arrives at the PE, has an encapsulation (e.g., VLAN tag) that is nonservice delimiting, then that encapsulation is preserved as the packet is sent into the VPLS. If the packet has an encapsulation that does contain the required VLAN tag, an appropriate tag is attached [17]. Similar to VPWS, in VPLS, the PW encapsulations are in turn encapsulated within an MPLS tunnel encapsulation. 12.4.3.4 VPLS PW Setup Each VPLS requires a globally unique identifier (a VPN-id). Before a PE can set up PWs to other PEs in the VPLS, the ACs attaching the CEs to the PEs must be provisioned on both the PEs and the CEs in that VPLS, a forwarder for that VPLS must be provisioned on each PE, the local ACs of that VPLS must be associated with the forwarder, and the forwarder must be provisioned with the identifier (a VPN-id) of the VPLS to which it belongs. The mapping of a VPLS identifier into a set of remote PEs that belong to the same VPLS can be provisioned or accomplished through an auto-discovery procedure such as BGP auto-discovery. Once the set of remote PEs is determined, the VPLS PEs can use LDP to set up a fiill mesh of PWs among the forwarders in a particular VPLS. As described earlier, LDP has defined two new FEC TLVs for establishing PWs, namely, PWid FEC (type 128) and Generalized ID FEC (type 129). For the VPLS PWs setup, the Generalized ID FEC with following assignment of fields is used: • Control bit (c) is set if the control word is needed on a PW. • PW type value indicates the method of encapsulation, namely, Ethernet (i.e., raw mode or Ethernet VLAN (i.e., tag mode). • The AGI, Length, and Value fields contain the unique identifier of the VPLS (VPLS-id). • TAII, and SAII fields are null, since the VPLS PWs terminate on MAC learning tables (many to many forwarders) rather than on individual ACs. • Interface Parameters include parameters such as MTU (which must be the same for all PWs in a given VPLS) and Requested VLAN ID (for a PW of type Ethernet VLAN, this parameter may be used to signal the insertion of the appropriate VLAN ID by the ingress PE before sending the packet on a PW).
448
Chapter 12
12.4.4 E-LAN Service Emulation Walk-Through Example This section provides a walk-through example of E-LAN emulation over MPLS using the network diagram shown in Figure 12-14. The goal is to set up a VPLS by establishing a full mesh of Ethernet PWs between PEl, PE2,
0
l-:st;iblisli 1.DP session f) Setup PWs
MPl.S B:ukl)()iu-
Q) P!-l :ill;iclics two l.ih I 1 iliciiK-t I I 1 iliciiKi I , , I Tunnel j L Li-'!]]h:_l ~*" I 1 KwiK- I I L:il>vl I 0
0
0
VV.l removes llie labels •liiiim.-I 1
I MlK-niol I
I-otAvai(i packet usiiij'. Iiinnel label
Similar pioccssiiu', in liie reverse clireclion of How
Figure 12-14. Emulation of E-LAN service over MPLS
and PE3. The following discussion assumes that bidirectional MPLS tunnels between PEl, PE2, and PES have been already established using LDP or RSVP-TE. The label exchange for tunnel setup is not shown. The MPLS network is assumed to be based on packet over SONET transport. For the sake of simplicity, the data link (i.e., PPP) and physical layer headers (i.e., SONET) are not shown. The sequence of steps described here may not necessarily be followed in this order by a specific implementation. L PEl, PE2, and PE3 learn each other's addresses and estabhsh an LDP session between them. Before PEl and PE2 can begin signaling PW labels, they need to know the VPLS-id. Each PE is provisioned with a unique VPLS-id. PEs can learn addresses and Remote Forwarder Selector (i.e., VPLS-id) through an auto-discovery procedure. 2. To set up a full mesh of bidirectional PWs using an LDP Label mapping message with FEC type 129, PEl advertises a label 12 to PE2 and a label
Ethernet Services Over MPLS Networks
3.
4. 5.
6.
449
13 to PE3. Similarly, PE2 advertises a label 21 to PEl and a label 23 to PE3, and PE3 advertises a label 31 to PEl and a label 32 to PE2. Assume that in the LDP signaling message the control (C) bit was set (which means the ingress PE must attach the control word) and the PW type was set to Ethernet VLAN (tag mode). Now let us consider flow of a packet from CEl to CE2. Suppose when the packet leaves CEl, it has a MAC SA of CI and a destination MAC DA of C2. Assume that PEl has learned MAC DA C2 from PE2 and associated with a PW label 21 advertised by PE2. If PEl does not know the MAC DA C2, it would multicast this packet to both PEl and PE2. In the incoming direction, PEl strips off the preamble and frame checksum (PCS) from the received Ethernet frame, prepends the optional control word to the frame, selects the appropriate PW, prepends the PW label 12 to the resulting packet, prepends the appropriate tunnel label to the resulting packet, and then transmits the labeled packet along the tunnel. The labeled packet is forwarded based on tunnel label (top label) across the MPLS network. When the packet arrives at the PE2, it performs the reverse set of operations in the outgoing (PE2 to CE2) direction. For example, PE2 strips off the tunnel and PW encapsulation after processing the frame as required by the control word, and strips off the control word, and hands the resulting frame to the Forwarder/NSP, which regenerates the PCS and replays the frame to the attached CE2 interface. A similar set of operations is performed in the other direction of traffic (CE2 to CEl).
12.5.
IMPORTANCE OF VPLS FOR METRO ETHERNET SERVICES
To provide multipoint-to-multipoint Ethernet services between multiple enterprise sites in geographically dispersed locations, Service Providers must be able to extend native Ethernet service from one metro area to another in a reliable and scalable fashion. The number of unique VLAN IDs and the poor convergence characteristics of the STP limit the scalability and availability of Ethernet networks, respectively. As VPLS PWs use MPLS labels, VPLS can support a large number of unique identifiers (e.g., over a million labels). As VPLS PWs are carried within MPLS tunnels, which can be quickly (in tens of milliseconds) rerouted around link/node failures using mechanisms such as MPLS FastReRoute [29], VPLS can support highavailability services. Thus by enabling two key service attributes, namely.
Chapter 12
450
scalability and high availability, VPLS is expected to play a pivotal role in the large-scale deployment of Metro Ethernet services (see Figure 12-15).
y, Metro Ar ia A1
Mcli-Q Area C
I VPLS P t l
Metro Area B iVIBN MPLS Backbone
Ml-N Figure 12-15. Inter-Metro Ethernet extension via VPLS
12.6.
SUMMARY
This chapter described architectures of the Layer 2 Virtual Private Network (L2VPN) services such as Virtual Private Wire Service (VPSW) and Virtual Private LAN Service (VPLS) that are being standardized in the Internet Engineering Task Force (IETF). VPWS and VPLS can be used to support E-Line and E-LAN services being standardized by the Metro Ethernet Forum (MEF) across a Wide Area Network (WAN). Specifically, MPLS-based L2VPN services are the main focus of this chapter. The material in this chapter should allow network engineers, administrators, and planners to understand how they could offer E-Line and E-LAN services over the converged packet switched network infrastructure. The content of this chapter is based on IETF standards and does not necessarily reflect any particular networking vendor's L2VPN implementation.
Ethernet Services Over MPLS Networks
12.7.
451
APPENDIX A: MPLS BASICS
This Appendix provides a brief tutorial on the basics of the Muhiprotocol Label Switching (MPLS) technology.
12.7.1 Forwarding Equivalence Class In conventional IP forwarding, when a router needs to forward an IP packet, it performs a longest prefix match for the packet's destination address in the forwarding information base (FIB) to obtain the next hop and the outgoing link information. The aforementioned IP forwarding procedure is then carrier out in a sequence of routers as the packet traverses the network toward its destination. Since each router has a fixed (finite) number of outgoing links, packets with different destination IP addresses are forwarded along the same outgoing link. A group of packets that is forwarded over the same path is said to form a Forwarding Equivalence Class (FEC). As a concrete example, a router typically considers two packets to be in the same FEC if there is some address prefix P in that router's forwarding tables (i.e., FIB) is such that P is the longest match for each packet's destination address [14].
12.7.2 Labels Unlike IP, which bases its forwarding on a packet's destination IP address, MPLS forwards packets based on other information in the header that is referred to as a label. A label is a short, fixed-length value that identifies a particular FEC. Figure 12-16 shows the format of an MPLS label. A common type of FEC is known is known as the Address Prefix FEC, which is based on packet's IP destination address. In general, however, the FEC can be based on information other than an IP destination address, such as a PW identifier (as discussed in the main body of this chapter). The association between an FEC and a label is known as label-to-FEC binding. The FEC-to-label binding and its distribution is accomplished through a set of procedures and messages commonly referred to as MPLS signaling protocols (MPLS signaling protocols are further discussed later on in this appendix) The label of an incoming packet is known as the incoming label, and the label of an outgoing packet is referred to as the ""outgoing label. In general, a packet may contain one or more labels that are collectively known as a label stack. The labels in the stack are organized as a last-in-first-out (LIFO) stack (Figure 12-17 shows a label stack). An unlabeled packet can be viewed
Chapter 12
452
as having a label stack of depth 0. As described in the main body of this chapter, transport of Ethernet frames over an MPLS network involves using a label stack where the bottom label is the pseudowire (PW) label and the top label is the tunnel label. 23 24
15 16 Label (20 bits)
Etpcrimentiti (3 bib)
Time-Tfl-Livt
S
(8 bit«)
When set, this field indicates bottom ofthe label stack Figure 12-16. MPLS label format
12.7.3 Label Encoding The term Multiprotocol in MPLS refers to the fact that MPLS is capable of switching packets or frames from multiple Layer 3 and Layer 2 protocols. MPLS can operate over multiple data link layer technologies such as Ethernet, Point-to-Point protocol (PPP), and ATM. An example of MPLS label encoding for the case of Ethernet and PPP is depicted in Figure 12-18. Top of the stack
Label Stack Entry (level d) Stack Depth
Bottom of stack (the S bit in this entry is set to one)
Label Stack Entry (level 2) Label Stack Entry (level 1)
-Each label stack entry is 32 bits -
Figure 12-17. MPLS Label Stack
Ethernet Services Over MPLS Networks
45 3
MPLS belong to layer 2 5 in the OSImodd v
MPLS over Bhemet over Fiber/Copper
•
| Fiber or Copper | Ethernet Header ^
M a s over PFP over SOIET/SDH
•
|
SOJET/SDH
|
PFPHeader
^
^
^
Layer 3 Header |
Paykad
{ l ^ S t a d c | Layer 3 Header |
Payload
Figure 12-J 8. MPLS label stack encoding
12.7.4 Label Switched Router (LSR) A router that is capable of supporting MPLS is referred to as a label switched router (LSR). An interconnection of LSRs forms an MPLS networks. An LSR at the edge of the MPLS network is known as an edge LSR. In general, any LSR whose neighbor does not support MPLS can be considered as an edge LSR. In contrast, a transit LSR's neighbors are expected to support MPLS. Depending on its location in the MPLS network, an LSR can perform different set of label operations.
12.7.5 Label Stack Operations — imposition, disposition, swapping In the incoming direction, an edge LSR attaches one or more labels (i.e., label stack) to unlabeled packets. The action of adding one or more labels is commonly known as label imposition. In the outgoing direction, an edge LSR removes labels and thus turns labeled packets into unlabeled packets. The action of removing the label stack from a labeled packet is commonly referred to as label disposition. In contrast with an edge LSR which either attaches or removes the label stack, a transit LSR only replaces the incoming label with an outgoing label. The action of replacing an incoming label with an outgoing label is commonly referred to as label swapping.
12.7.6 MPLS Control Plane The phrase MPLS control plane refers to the set of tasks performed by MPLS signaling protocols. The main function of the MPLS signaling
454
Chapter 12
protocols is to establish, maintain, and release label switched paths. The MPLS control plane consists of signaling protocols such as Label Distribution Protocol (LDP) [23] and Resource ReSerVation Protocol (RSVP) [24].
12.7.7 IVLPLS Forwarding Plane The label information provided by the control plane is used to populate forwarding tables BGP. The information provided by the MPLS control plane protocols is used to populate state in the forwarding tables. The forwarding state is in turn used for switching packets from input to output ports based on label information. The MPLS forwarding tables make up the forwarding plane of an LSR.
12.7.8 Label Switched Path (LSP) The sequence of LSRs that a labeled packet traverses, starting at the ingress LSR and terminating at the egress LSR, is known as a label switched path (LSP), Figure 12-19 shows an example of a LSP. As MPLS LSPs are unidirectional, a separate LSP in each direction is required for bidirectional traffic flows. The path of an LSP can be selected either hop-by-hop or specified explicitly by the source node. When the path of an LSP is selected hop-byhop (for example, along the path computed by the interior gateway protocol such as Open Shortest Path First (OSPF)), such an LSP is termed a hop-byhop routed LSP. In contrast, an LSP whose path is explicitly specified at the source node is called an explicitly routed (or a traffic engineered) LSP. For instance, LDP establishes hop-by-hop routed LSPs, whereas RSVP-TE establishes explicitly routed LSPs.
Ethernet Services Over MPLS Networks
455
LSPs 1 J Edge LSR
Edge LSR
Unlabeled IP Packets
\
Unlabeled IP Packets
Figure 12-19. MPLS label switched paths.
12.7.9 Benefits of MPLS Technology The key benefits of MPLS above and beyond those of IP are the apphcations it enables, such as Layer 3 VPNs (L3VPNs), Layer 2 VPNs (L2VPNs), traffic engineering (TE), and fast rerouting (FRR) of traffic around hnk/node failures. These applications are possible because MPLS decouples forwarding from the control plane, which means that the same forwarding component can be used without any special considerations for a particular application.
12.8. REFERENCES [1]
L. Andersson and T. Madsen, "PPVPN terminology," IETF Work in Progress, July 2004. [2] L. Andersson and E. Rosen, "Framework for Layer 2 Virtual Private Networks (L2VPNs);' IETF Work in Progress, June 2004. [3] S. Bryant and P. Pate, "PWE3 Architecture," IETF Work in Progress, March 2004. [4] H. Schulzrinne et al., "RTP: A Transport Protocol for Real-Time Applications", RFC 1889, January 1996. [5] E. Rosen and Y. Rekhter, "BGP/MPLS IP VPNs," IETF Work in Progress, March 2004. [6] D. Farinacci et al., "Generic Routing Encapsulation (ORE)," RFC 2784, March 2000. [7] G. Dommety, "Key and Sequence Number Extensions to GRE," RFC 2890, September 2000. [8] C. Perkins, "IP Encapsulation within IP," RFC 2003, October 1996. [9] A. Conta et al., "Generic Packet Tunnehng in IPv6 Specification," RFC 2473, December 1998. [10] S. Kent et al., "Security Architecture for the Internet Protocol," RFC 2401, November 1998.
456
Chapter 12
[11] S. Kent et al., "IP Authentication Header," RFC 2402, November 1998. [12] S. Kent et al, "IP Encapsulating Security Payload (ESP)," RFC 2406, November 1998. [13] D. Harkins et al, "The Internet Key Exchange (IKE)," RFC 2409, November 1998. [14] E. Rosen et al, "Multiprotocol Label Switching Architecture," RFC 3031, January 2001. [15] E. Rosen et al, "MPLS Label Stack Encoding," RFC 3032, January 2001. [16] R. Callon and M. Suzuki, "A Framework for Layer 3 Provider Provisioned Virtual Private Networks," IETF Work in Progress, October 2003. [17] L. Martini et al, "Encapsulation Methods for Transport of Ethernet Frames Over IP/MPLS Networks," IETF Work in Progress, July 2004. [18] L. Martini et.al, "Encapsulation Methods for Transport of ATM Over IP/MPLS Networks," IETF Work in Progress, July 2004. [19] A. Malis et al, "SONET/SDH Circuit Emulation over Packet (CEP)," IETF Work in progress, June 2004. [20] C. Kawa et al, "Frame Relay over Pseudo-Wires,"/£'rF Work in progress, February 2004. [21] L. Martini et al, "Encapsulation Methods for Transport of PPP/HDLC Over IP and MPLS Networks," IETF Work in Progress, April 2004. [22] L. Martini et al, "Pseudowire Setup and Maintenance using LDP," IETF Work in Progress, July 2004. [23] L. Andersson et. al, "LDP Specification," RFC 3036, January 2001. [24] D. Awduche et. al, "RSVP-TE: Extensions to RSVP for LSP Tunnels," RFC 3209, December 2001. [25] Ould-Brahim et al, "Using BGP as an Auto-Discovery Mechanism for Layer-3 and Layer-2 VPNs" IETF Work in progress, May 2004. [26] M. Lasserre et al., "Virtual Private LAN Services over MPLS,", IETF Work in Progress, August 2004. [27] IEEE 802.1D-2003, IEEE Standard for Local and Metropolitan Area Networks: Media Access Control (MAC) Bridges, 2003. [28] IEEE 802.1Q-1998, IEEE Standards for Local and Metropolitan Area Networks: Virtual Bridged Local A rea Networks, 1998. [29] P. Pan et al, "Fast Reroute Extensions to RSVP-TE for LSP Tunnels," IETF Work in Progress, November 2004.
Chapter 13 METRO ETHERNET CIRCUIT EMULATION SERVICES
Nan Chen Strix System; Metro Ethernet Forum President
13.1.
METRO ETHERNET CIRCUIT EMULATION SERVICES
13.1.1 Circuit Emulation Service Definition Ethernet CES provides emulation of TDM services, such as N x 64 kbit/s, Tl, El, T3, E3, OC-3, and OC-12, across a Metropolitan Ethernet Network (MEN). The objective is to allow MEN service providers to offer TDM services to customers. Hence it allows MEN service providers to extend their reach and addressable customer base. For example, the use of CES enables metro Ethernet transport networks to connect to PBXs on customer premises and to deliver TDM voice traffic alongside data traffic on metro Ethernet. The CES is based on a point-to-point connection between two InterWorking Functions (IWF). Essentially, CES uses the MEN as an intermediate network (or virtual wire) between two TDM networks. This setup is handled as an application of the Ethernet service, using the interworking function to interface the applications layer onto the Ethernet services layer.
Chapter 13
458 13.1.1.1
TDM Line Service (T-Line)
The TDM Line (T-Line) service provides TDM interfaces to customers (N X 64 kbit/s, Tl, El, T3, E3, OC-3, OC-12, etc.), but transfers the data across the MEN instead of a traditional circuit switched TDM network. From the customer perspective, this TDM service is the same as any other TDM service, and the service definition is given by the relevant ITU-T and ANSI standards pertaining to that service. From the provider's perspective, two CES interworking functions are provided to interface the TDM service to the Ethernet network. The CES interworking functions are connected via the Metro Ethernet Network (MEN) using point-to-point Ethernet Virtual Connections (EVCs), as illustrated in Figure 13-1. The TDM Service Processor (TSP) block shown in Figure 13-1 consists of any TDM grooming function that may be required to convert the TDM service offered to the customer into a form that the CES IWF can accept. For example, the TSP may be a Framer device, converting a fractional DSl service offered to the customer into an N x 64 kbit/s service for transport over the MEN. The operation of the TSP is outside the scope of this chapter. The TSP and the CES IWF may physically reside in the Provider Edge (PE) unit at the provider's nearest point-of-presence, or in a service provider-owned box in a customer location (e.g., a multitenant unit). From the architecture perspective, there is no difference between these alternatives.
TDM Subscriber Ethernet UNI Demarcation
Ethernet UNI TDM Subscriber Demarcation
Figure 13-1: TDM line service over Metro Ethernet Networks
Metro Ethernet Circuit Emulation Services
459
13.1.1.1.1 Operational Modes of a T-Line Service The basic T-Line service is a point-to-point, constant bit-rate service, similar to the traditional leased-line type of TDM service. However, service multiplexing may occur ahead of the CES interworking functions, (e.g., aggregation of multiple emulated Tl lines into a single T3 or OC-3 link), creating a multipoint-to-point or even a multipoint to multipoint configuration (see Figure 13-2). This service multiplexing is carried out using standard TDM multiplexing techniques, and is considered as part of the TSP block rather than the CES interworking function. The TDM interface at the input of the CES interworking function is the same as that output from the CES IWF at the opposite end of the emulated link. It is the TSP that may be used to multiplex (or demultiplex) that TDM service into the actual TDM service provided to the customer. This setup allows a TDM service to a customer to be provided as a collection of emulated services at lower rates.
Figure 13-2: Possible TDM virtual private line configurations
Hence there are three possible modes of operation of a T-Line service: 1. Unstructured Emulation mode (also known as structure-agnostic emulation) 2. Structured Emulation Mode (also known as structure-aware emulation) 3. Multiplexing mode Modes (1) and (2) are point-to-point connections. Mode (3) permits multipoint-to-point and multipoint-to-multipoint configurations, although all these modes are operated over simple point-to-point EVCs in the MEN.
460
Chapter 13
13.1.1.1.1.1 Unstructured Emulation Mode In Unstructured Emulation Mode, a service is provided between two service endpoints that use the same interface type. Traffic entering the MEN on one endpoint leaves the network at the other endpoint and vice versa. The MEN must maintain the bit integrity, timing, and other client payload-specific characteristics of the transported traffic without causing any degradation that would exceed the requirements for the given service as defined later in this chapter. All the management, monitoring, and other functions related to that specific flow must be performed without changing or altering the service payload information or capacity. Examples where unstructured emulation mode could be implemented are leased line services or any other transfer-delay-sensitive (real-time) applications. The specific transport rates and interface characteristics of this service are defined later in this chapter. 13.1.1.1.1.2 Structured Emulation Mode In Structured Emulation Mode, a service is provided between two service endpoints that use the same interface type. Traffic entering the MEN on one endpoint is handled as overhead and payload. The overhead is terminated at the near endpoint, while the payload traffic is transported transparently to the other end. At the far endpoint, the payload is mapped into a new overhead of the same type as that at the near endpoint. The MEN must maintain the bit integrity, timing information, and other client payload-specific characteristics of the transported traffic without causing any degradation that would exceed the requirements for the given service as defined later in this chapter. All the management, monitoring, and other functions related to that specific flow must be performed without changing or altering the service payload information or capacity. An example of such a service is the transport of OC-3 when the SOH is terminated at both ends and the STS-1 pay loads are transported transparently over the MEN. A second example is a fractional DSl service, where the framing bit and unused channels are stripped and the used channels transported across the MEN as an N x 64 kbit/s service. The specific transport rates and interface characteristics are defined later in this chapter. 13.1.1.1.1.3 Multiplexing Mode In the multiplexing mode, multiple lower-rate transparent services are multiplexed at a specific service endpoint of the MEN into a higher digital hierarchy. Similarly, a higher-rate service may be decomposed into several lower-rate services. For example, a customer may have several sites — a head office with a full DSl connection, and several satellites with fractional DSl connections, as shown in Figure 13-3. The same architecture can be
Metro Ethernet Circuit Emulation Services
461
used for the multiplexing of other rate services, e.g., several full DSl services onto a single DS3 or multiplexing of VT-1.5s into an STS-1. The specific transport rates and interface characteristics are defined later in this chapter. Service multiplexing is typically performed in the TDM domain as part of a TSP, not in the Ethernet domain. A fractional TDM link going into the MEN comes out as a fractional TDM link, or at least as a payload containing solely that fractional link. Multiplexing and demultiplexing is performed outside the CES IWF as part of any native TDM signal processing, as shown in Figure 13-3. Therefore the customer service is multiplexed, but the emulated service (i.e., the service handled by the IWF) is structured. Fractional DSl payload to be multiplexed
Fractional DS1 link
TDM multiplexing function (TSP) Head Office
Full DSl link
Point-to-point EVCs Fractional DSl links
Satellite sites
Figure 13-3: Example multipoint-to-point T-Line multiplexed service
13.1.1.1.2 Bandwidth Provisioning for a T-Line Service The Metro Ethernet service provider will need to allocate sufficient bandwidth within the network to carry the T-Line service. This may require very fine granularity of bandwidth provision to allow efficient allocation. For example, an N x 64 kbit/s service may have very low bandwidth requirements where N is small. This could result in very inefficient provisioning if the smallest unit increment of bandwidth provision is 1 Mbit/s. The following sections outline three possible schemes to provision the bandwidth efficiently for low-data-rate CES.
462
Chapter 13
13.1.1.1.2.1 Bandwidth allocation at 100 kbit/s granularity The bandwidth required to offer emulation of N x 64 kbit/s services increases with N in steps of 64 kbit/s, plus an overhead for encapsulation headers. Therefore, in order to be able to allocate MEN bandwidth efficiently for such services, the bandwidth needs to be provisioned in similar step sizes. It is recommended that to achieve reasonable efficiency, the granularity of service provisioning should be 100 kbit/s or smaller. However, most existing equipment only allows for bandwidth provision at multiples of 1 Mbit/s (or 1.5 Mbit/s for much SONET-based Ethernet equipment), so this fine level of granularity is not guaranteed to be available. Therefore, alternative means of allocating bandwidth may need to be used to provide a reasonable level of efficiency. 13.1.1.1.2.2 TDM multiplexing Multiple TDM services between the same two IWFs may be multiplexed together in the TDM domain before being carried across the MEN. Demultiplexing at the far end is also accomplished in the TDM domain. For example, several fractional DSl services could be multiplexed onto a single full DSl link before transmission across the MEN. Bandwidth can then be provisioned for the full link, rather than individually for each fractional customer service. Similarly, the same architecture can be used for multiplexing of other rate service, e.g., several full DSl services onto a single DS3 or SONET link. 13.1.1.1.2.3 Ethernet multiplexing Another alternative is to multiplex several circuits across a single Ethernet Virtual Connection, as shown in Figure 13-4. All TDM circuit emulation between any two given IWFs is handled across a single point-to-point EVC. Individual circuit emulated links are carried across that EVC using Ethernet multiplexing. Customer separation is maintained using this multiplexing. The total bandwidth of circuit emulation traffic between two points is known in the case of constant bit rate, always on service. Therefore, the amount of traffic across that EVC is known and can be provisioned accordingly. This allows efficient bandwidth provisioning. For example, five fractional TDM services at 384 kbit/s could share a single 2 Mbit/s EVC, and don't have to be allocated 1 Mbit/s each.
Metro Ethernet Circuit Emulation Services
463
TDM links (may be full or fractional)
Customer sites (may or may not belong to the same custor^er)
Circuit emulated links
Point-to-point EVCs
Figure 13-4: Multiplexing across a single Ethernet Virtual Connection
13,1.1.2
TDM Access Line Service (TALS)
The second type of TDM service that can be offered over MetropoUtan Ethernet Networks is where the MEN service provider hands off one (or both) ends of the network to a second network (e.g., the PSTN). For example, Figure 13-5 shows the case where the customer interface is TDM, and the handoff to an external network (in this case the PSTN) is via some form of TDM or SONET trunk line (e.g., OC-3). The prime use of this type of service is where the MEN is used as an access network onto the second network. With TALS service, the customer-facing IWF can be located on the customer's site but is still under the control of the MEN operator. Multiple customer services may be multiplexed up onto a single trunk to be handed off to the second network (Figure 13-5). As with T-Line service, this multiplexing is implemented using conventional TDM techniques in a native signal processing block. In some instances, both ends of the emulated TDM service may be within the service provider network, e.g., backhaul of ATM or SONET services. As with T-Line service, the customer-facing TSP and CES IWF may physically reside in the Provider Edge (PE) unit at the provider's nearest point-of-presence, or in a service provider-owned box in a customer location (e.g., a multitenant unit). From the architecture perspective, there is no difference between these alternatives.
Chapter 13
464
TDM Subscriber Ethernet UNI Demarcation
Figure 13-5: Handoff of a multiplexed trunk to an external network
13.1.1.2.1 Operational Modes of a TALS Service The TALS service is essentially very similar to the multiplexed, multipoint-to-point T-Line service. The two services use the MEN in the same manner (see Figure 13-6). The only difference is that the final multiplexed service (e.g., OC-3) is handed off to another network rather than to an end customer. As such, it may have some performance requirements deriving from the requirements of the second network that are not present in the T-Line service. The MEN must maintain the bit integrity, timing, and other clientpayload-specific characteristics of the transported traffic without causing any degradation that would exceed the requirements for the given service as defined later in this chapter. All the management, monitoring, and other functions related to that specific flow must be performed without changing or altering the service payload information or capacity.
I
I'uinl (o puini V\
III \mM\
cnsrwF
TSP! CES IWK
!ir
, ^ ^
" TDM
\
TSP: CES IWF
Figure 13-6: Possible TDM handoff configurations
Metro Ethernet Circuit Emulation Services 13.1.1.3
465
Customer-Operated CES
It is also possible for customers to operate the Circuit Emulation Service themselves across an E-Line service (see Figure 13-7). However, in order to operate CES at a reasonable level of quality, the service provider will need to offer ?i premium SLA, with tighter definitions of parameters such as packet delay, variation in packet delay, and packet loss. The requirements section of this document covers the parameters that need to be controlled, as well as the appropriate range of values over which CES can be operated with acceptable quality. Some customers, for cost reasons, may choose to operate CES across a standard E-Line service with no special SLA. In this case, the level of quality experienced is entirely the customer's own responsibility. Since from the service provider's perspective this is purely an Ethernet service, the definition of this service is considered to be outside the scope of this chapter, other than to document possible parameters of a service-level specification for CES-capable E-Line service.
Service Provider Network TDM Link
CES IWF
TSP (optional)!
>i^^W^^
CUET Ethernet UNI Customer Demarcation
1^1
\ 1 TDM Link
CUET Ethernet UNI Customer Demarcation
Figure 13-7: Customer-Operated CES over a Metropolitan Ethernet Service (e.g., E-Line)
13.1.1.4
Mixed-Mode CES Operation
A further scenario is a mixed mode service, where the customer interface is Ethernet at one side and TDM at the other. Therefore the customer is providing its own interworking function at one end of the service, and this function must interoperate with the service provider's interworking function at the far end (Figure 13-8).
466
Chapter 13
This service may be operated using the same methods as the other services described in this chapter. However, it may create significant issues as far as troubleshooting hnks between the service provider and the customer. For example, it may be difficult to determine whether a fault is in the customer's own equipment or in the service provider's equipment. Service providers offering mixed-mode services should be aware of the potential issues involved. Resolution of these types of issues, as well as any operations and management issues involved with running between IWFs in different administrative domains, are for further study. The MEF does not plan any further work on the definition of such a service at present.
H
TSP (qjtional)
CES IWF
1 TDM Link
CUET Ethernet UNI Customer Demarcation
Ethernet UNI TDM Subscriber Demarcation
Figure 13-8: Mixed-mode service
13.1.2 Circuit Emulation Service Framework This section is intended to provide a framework within which the actual requirements for the Circuit Emulation Service can be understood. 13.1.2.1
General Principles
In most service provider networks, SONET/SDH has been used as the technology for transporting customers' PDH or SONET/SDH traffic. The purpose of the CES is to allow the MEN to serve not only as a transport of Ethernet and data services but also as a transport of customer's TDM traffic. The CES solution in the MEN should therefore make the MEN behave as a standard SONET/SDH and/or PDH network, as seen from the customer's perspective. The intention is that the CES customer should be able to use the
Metro Ethernet Circuit Emulation Services
467
same (legacy) equipment, regardless of whether the traffic is carried by a standard SONET/SDH or PDH network, or by a MEN using CES. 13.1.2.2
Service Interface Types
There are two basic service interfaces in the TDM domain. These are shown in Figure 13-9, and are defined as follows: 1. TDM Service Interface: The TDM service that is handed off to the customer or TDM network operator. TDM services fall into two categories namely, unstructured and structured. 2. Circuit Emulation Service Interface (CES TDM Interface): The actual circuit service that is emulated between interworking functions through the MEN. For unstructured TDM service, the CES Interface carries all information provided by the TDM Service Interface transparently. The service is emulated in its entirety by the IWF, including any framing structure or overhead present. For structured TDM service, the TDM service interface is operated on by the TSP (TDM Service Processor) to produce the service that is to be emulated across the MEN. A single structured TDM service may be decomposed into one or many CES flows, or two or more structured TDM services may also be combined to create a single CES flow. TDM Service Interface
Customer TDM — 7 ^ Service
CES TDM Interface TSP (optional) (e.g. framing, mux/demux)
CES IWF Type
Ethernet Interface
CES Payload
CES IWF TDM Packetization
Adds DA, SA, and CRC fields
Figure 13-9: Functional Elements and Interface Types
13.1.2.2.1 Examples of TDM Service Interfaces Table 13-1 shows examples of possible TDM service interfaces. These service interfaces are defined as standardized PDH, SONET, or SDH interfaces. Unstructured TDM services are either emulated as is by the CES IWF or mapped by the TSP onto the CES interface without modifying their content. These services preserve the framing structure as well as SONET/SDH overhead as input to the TDM service interface. These services provide point-to-point connectivity across a MEN.
Chapter 13
468
Structured services may be either demultiplexed into any of their defined service granularities or multiplexed with other structured TDM services. For example, an OC-3 structured service can be demultiplexed into three separate STS-1 signals. Each of these signals can then be circuit emulated by different IWF across the MEN to one or more service endpoints. At these IWFs, these STS-ls can be reassembled with other STS-ls (from different originating services) to form an entirely new OC-3 signal (provided each originating site uses a PRS level clock). Structured service can be used to provide point-to-point, point-to-multipoint, and multipoint-to-multipoint TDM services. Table 13-1. TDM service interfaces TDM Service Unstructured TDM Interface Service DSl Yes DS3 Yes El Yes E3 Yes
Structured TDM Service Yes Yes Yes Yes
OC-1
Yes
Yes
OC-3
Yes
Yes
OC-3c STM-1
Yes Yes
Yes Yes
STM-lc
Yes
Yes
OC-12
Yes
Yes
OC-12c STM-4
Yes Yes
Yes Yes
STM-4C
Yes
Yes
Structured TDM Service Granularity N X 64 kbit/s DSl,Nx64kbit/s N X 64 kbit/s El, Nx 64 kbit/s, DSO STS-1, VT-1.5, VT-2 STS-1, VT-1.5, VT-2 STS-3C VC-11(DS1),VC12(El),VC-3 {DS3, E3, other) VC-4, VC-3, VC11,VC-12 VT-1.5(DS1),VT2 (El), STS-1 (DS3, E3, other), STS-3C STS-12c VC-11(DS1),VC12 (El), VC-3 (DS3, E3, other), VC-4 VC-4-4C
13.1.2.2.2 TDM Service Processor (TSP) The TDM Service Processor is dejfined as a function, operating in the TDM domain, that 1. modifies the bit rate or contents of the service to be emulated (e.g., overhead termination, removal of unused channels in a fractional service) 2. multiplexes two or more TDM services into a single service to be emulated, and demultiplexes the composite service into its component
Metro Ethernet Circuit Emulation Services
469
services at the remote end as required (e.g., multiplexing of several N x 64 kbit/s services onto a larger N x 64 kbit/s, or multiplexing of several DSl services onto a DS3) 3. transparently maps the TDM service onto a different service to be emulated (e.g., asynchronous mapping of a DSl onto a VT-1.5) By this definition, therefore, a Tl/El Framer would be considered a TSP (terminates the framing bits, breaks the service down into an N x 64 kbit/s service). Another example of a TSP would be a DSl to VT-1.5 mapper. 13.1.2.2.3 Circuit Emulation Interworking Function (CES IWF) Circuit Emulation Service is defined as an application service in terms of layered network model defined in the MEN Architecture framework document. It uses the MEN as an intermediate network (or virtual wire) between two TDM networks. The Circuit Emulation Interworking Function is therefore defined as the adaptation function that interfaces the CES application to the Ethernet layer. The CES IWF is responsible for all the functions required for an emulated service to function, e.g., data packetization and depacketization (including any padding required to meet the minimum Ethernet frame size), sequencing, synchronization, TDM signaling, alarms, and performance monitoring. Listed in Table 13-2 are the defined interface types required to support circuit emulation over the MEN. These services are divided by payload type, CES Interface type, CES rate, and IWF type. 13.1.2.2.3.1 PDH Circuit Emulation Service PDH emulation services provide transport of PDH services through the MEN. Managed by the IWF, as defined in Table 13-2, these services can be used to support structured or unstructured TDM service interfaces. As structured TDM services, the input TDM service can be demultiplexed into its service granularity as defined in Table 13-1. This service granularity includes DSO, N x 64 kbit/s, DSl, El, DS3 or E3. Likewise, multiple structured TDM services can be multiplexed into a single service and circuit emulated across the MEN as this new service. For example, a multiple N x 64 kbit/s services can be multiplexed into a DSl that is circuit emulated across the MEN and output as a DSl structured service. These interface rates and IWFs are presented in Table 13-2. 13.1.2.2.3.2 SONET/SDH Circuit Emulation Service SONET/SDH emulation services provide transport of SONET/SDH services through the MEN. Managed by the IWF, as defined in Table 13-2, these services can be used to support structured or unstructured TDM service interfaces. As structured TDM services, the input TDM service can be
Chapter 13
470
demultiplexed into its service granularity as defined in Table 13-1. This service granularity includes STS-1, STM-1, or the appropriate virtual container. Likewise, multiple structured TDM services can be multiplexed into a single service and circuit emulated across the MEN as this new service. These interface rates and IWFs are presented in Table 13-2. Table 13-2. CES TDM interface definition CES TDM Interface Payload Type Type PDH NDSO DSl El E3 DS3 SONET/SDH STS-1 STS-3 STS-3C STM-1 STS-12 STS-12c STM-4 SONET/SDH VT-1.5 Tributary VT-2 VC-11 VC-12 VC-3 STS-IP STS-3P VC-4 STS-3cP STS-12P STS-12cP VC-4-4 VC-4-4cP
CES Rate
IWF Type
N X 64 kbit/s 1.544 Mbit/s 2048 kbit/s 34368 kbit/s 44.736 Mbit/s 51.84 Mbit/s 155 Mbit/s 155 Mbit/s 155 Mbit/s 622.08 Mbit/s 622.08 Mbit/s 622.08 Mbit/s 1.728 Mbit/s
NDSO-CE DSl-CE El-CE E3-CE DS3-CE STS-1-CE STS-3-CE STS-3C-CE STM-1-CE STS-12-CE STS-12C-CE STM-4-CE VT-1.5-CE
2.176 Mbit/s 1.728 Mbit/s 2176 kbit/s 48384 kbit/s 50.112 Mbit/s 150.336 Mbit/s 150.336 Mbit/s 150.336 Mbit/s 601.344 Mbit/s 601.344 Mbit/s 601.344 Mbit/s 601.344 Mbit/s
VT-2-CE VC-ll-CE VC-12-CE VC-3-CE STS-1 P-CE STS-3P-CE VC-4-CE STS-3cP-CE STS-12P-CE STS-12cP-CE VC-4-4P-CE VC-4-4cP-CE
SONET/SDH emulation service provides emulation of the following services: 1. Higher-order Virtual Container (HOVC): VC-3/STS-1, VC-4/STS-3c, VC-4-4c/STS-12c 2. Lower-order Virtual Container (LOVC): VC-llA^T-1.5, VC-12/VT-2, none/VT-3, VC-2/VT-6, VC-3/none SONET terminology refers to HOVC as Synchronous Payload Envelope (SPE) and LOVC as Virtual Tributary (VT). Figure 13-10 shows an example of the functional blocks used in the SONET/SDH structured emulation service. For the SONET/SDH interface,
Metro Ethernet Circuit Emulation Services
All
a SONET/SDH framer TSP is used to terminate and handle the section and Hne overheads. The SONET/SDH virtual containers (SPEs) are extracted and passed to the SONET/SDH IWF(s). Each virtual container (SPE) can be sent to a different destination IWF. The ECDX and EFT are used to add the required multiplexing and Ethernet headers and trailers. For PDH interfaces, a SONET/SDH mapper TSP is used to map the PDH signal (T1/E1/T3/E3) into SONET/SDH virtual containers (SPEA^T). Each virtual container can be sent to a different destination IWF and be either groomed to a SONET/SDH interface or mapped back to a PDH interface.
TSP Mapper
IWF TSP
U ECDX
H EFT
Framer
Figure 13-10: Functional elements of SONET/SDH emulation service
13.1.2.2.4 Emulated Circuit Demultiplexing Function (ECDX) The Emulated Circuit Demultiplexer (ECDX) is a function, operating in the packet domain, that 1. selects one (or more) IWF as the final destination of each Ethernet frame received from the MEN based on an Emulated Circuit Identifier (ECID) attribute within the frame header 2. prepends an ECID attribute for each Ethernet frame sent to the MEN to allow circuit demultiplexing on the MEN egress 3. assigns the length/type field to each Ethernet frame sent to the MEN By this definition, equipment supporting 8 DSl TDM service interfaces would be able to multiplex 8 DSl interface onto a single EVC. Each emulated circuit would be identified by a unique ECID attribute. The ECDX function would examine the ECID attribute of received Ethernet frames and compare it to its local ECID to IWF association. According to the look-up result, the ECDX would pass the circuit emulation payload to the relevant IWF for processing.
472
Chapter 13
13.1.2.2.5 Ethernet Flow Termination Function (EFT) A termination function is defined as follows: A transport processing function which accepts adapted characteristic information from a client layer network at its input, adds information to allow the associated trail to be monitored (if supported) and presents the characteristic information of the layer network at its output(s) (and vice versa) In the context used here, an Ethernet Flow Termination function takes an adapted payload from the ECDX (the MAC client information field), along with a Length/Type attribute describing it as a CES payload. It adds the MAC Destination and Source addresses, and finally the frame check sequence. In the CE-bound direction, the EFT takes in an Ethernet frame from the MEN. It determines whether this contains a CES payload from the Length/Type field, and forwards it to its associated ECDX function for passing to the appropriate CES IWF. 13.1.2.2.6 Direction terminology For each direction of an emulated circuit, there is a pair of CES interworking functions. The MEN-bound IWF handles the packetization of the TDM data, encapsulation into Ethemet frames, and forwarding into the Ethernet network. The corresponding CE-bound IWF extracts the TDM data from the Ethemet frames and recreates the TDM service. Each direction of the service is handled separately by a second pair of IWFs. Hence, for a given MEN-bound IWF, the corresponding CE-bound IWF is at the other side of the MEN. 13.1.2.3
Synchronization
Figure 13-11 shows the synchronization domains needed to support CES.
473
Metro Ethernet Circuit Emulation Services
TSP
CESIWF
CES IWF
TSP
© SDcEi
© SDcE2
Figure 13-11: Timing distribution architecture showing clock domains The notations used in Figure 13-11 are described in Table 13-3. Table J3-3. Timing distribution definitions Attribute Description CES IWF Circuit emulation service interworking function CEn Customer edge devices terminating/ originating TDM circuits TSP The TDM Service Processor performs all TDM functions necessary to support structured or unstructured TDM emulation service. These functions are performed in the TDM domain using standardized techniques. MEn Metro Ethernet devices providing Circuit Emulation Service transport in the MEN PHY Physical interface terminating Ethernet traffic •^ The TDM-end service circuits =• The CES IWF providing edge-to-edge-emulation for the TDM circuit SDedn The synchronization domain used by Ethernet network elements (NEs) in the MEN cloud. This timing information is used to transport packets between the Ethernet NEs providing the CES. Since each of these devices operates independently, the synchronization mode of each device must be provisioned separately. Typical timing modes include line, external, and free-run. SDiwFn The synchronization domain used by CES IWF. This timing is used to transport the circuit emulated service across the MEN. This synchronization domain is specific to the CES interface types as shown in Table 13-2 (CES Interface Definition). Details of this synchronization domain are contained in the lA reference as shown in Table 13-2 (CES Interface Definition). SDxDMn The synchronization domain used by the TSP. This timing is used to transport TDM signals between the TSP and CE. This synchronization domain supports through timing mode. SDcen The synchronization domain used by the CE devices to establish the TDM service clock. This timing is used to transport TDM signals between the CE and the TSP. This synchronization domain supports external, line, or free-run timing modes.
One of the objectives of edge-to-edge-emulation of a TDM circuit is to preserve the service clock with a performance level as specified in the
474
Chapter 13
relevant Implementation Agreements. For example, the performance objective could be to meet the G.823 Traffic Interface specification. The service clock can be either generated at a CE via external timing mode or recovered from the TDM bit stream via line/loop timing mode. It should be noted that loop timing mode is a recovery process where timing is extracted from only one available input bit stream. Line timing mode is a recovery process where timing is extracted from one of several input bit streams. Since the number of available timing inputs is based on network design and system architecture, this chapter will use the term line timing when describing timing modes where either line or loop timing are used. Typical timing modes for a point-to-point CES connection are shown in Table 13-4. Table 13-4. CE synchronization modes and expected timing performance Timing CEl CE2 Timing Comments Option Timing Mode Mode External to External to 1 CEl and CE2 will operate in a Plesiochronous PRS PRS mode. If connected to other TDM circuits (in traceable traceable other synchronization domains), network slipsource source rate objectives will be met. External to Line timing CE2 will have the same average frequency as PRS mode CEl. If connected to other TDM circuits (in other traceable synchronization domains), network slip-rate source objectives will be met. Line timing External to CEl will have the same average frequency as mode PRS CE2. If connected to other TDM circuits (in other traceable synchronization domains), network slip-rate source objectives will be met. Free-run Line timing CE2 will have the same average frequency as mode mode CEl. If connected to other TDM circuits (in other synchronization domains), undesirable slip performance may result. Line timing Free-run CEl will have the same average frequency as mode mode CE2. If connected to other TDM circuits (in other synchronization domains), undesirable slip performance may result. Free-run Free-run CEl and CE2 synchronization domains will operate independently. TDM data will mode mode experience slips between CE 1 and CE2 and TDM circuits in other synchronization domains.
When the TDM circuit is transported via CES, this continuous signal is broken into packets at the MEN-bound IWF of the CES connection and reassembled into a continuous signal at the CE-bound IWF of the CES connection. In essence, the continuous frequency of the TDM service clock is disrupted v^hen the signal is mapped into packets.
475
Metro Ethernet Circuit Emulation Services
In order to recover the service clock frequency at the egress of the CES connection, the interworking function must employ a process that is specific to the CES interface type. The description and requirements of the IWF service clock recovery are contained in the Implementation Agreement for that service. 13.1.2.3.1 CES Interworking Function— Synchronization Description The synchronization block diagram for the CES Interworking function is shown in Figure 13-12.
External Timing Reference
Data
IWF Processor O-K Data
<—
©
Figure 13-12: CES IWF Synchronization Reference Model The notations used in Figure 13-12 are described in Table 13-5. Table 13-5. CES IWF synchronization definitions Attribute Description Performs all data, addressing, and timing functions necessary to support circuit IWF emulation over the MEN. Processor A line timing option may be used to provide the IWF with a timing reference. TDM TDM Line timing will extract timing from the TDM service SDJDMLine Ethemet Line timing will extract timing from either the MEN SDED or from the Ethernet arriving Ethemet frames. Line An External timing option may be used to provide the IWF with a timing EXT reference. An internal timing reference may be used to provide the IWF with a timing FR reference. This internal timing reference may either be a free-running oscillator or an oscillator in holdover mode. The synchronization domain for the IWF. The definition and requirements for SDT, the SDiwF may be found in the lA for the specific IWF per Table 13-2 (CES Interface Definition). Physical interface terminating Ethemet traffic ETF
476
Chapter 13
The main goal of the CES IWF is to preserve the service clock of the TDM service through the MEN. The CES IWF must preserve the accuracy of the service clock despite the levels of MEN jitter. The operation and definition of the interworking functions are specific to the CES interface type, as shown in Table 13-2 (CES Interface Definition Table). Specific information about operation and requirements for specific interworking functions is contained in the implementation agreements as indicated in Table 13-2 (CES Interface Definition Table). The IWF can use a variety of timing inputs to use as a reference for service clock recovery. A list of the five available timing inputs used by the CES IWF are shown below: 1. Line timing (from the CE): This mode is used to recover timing from a CE. In order for this timing mode to be PRS traceable, the CE must be provisioned to recover and transmit PRS-traceable timing. Line timing should be used if synchronization messaging is available (e.g., SONET or SDH line timing sources). 2. Line timing (from the MEN): This mode is used to recover timing from the MEN. In order for this timing mode to be PRS traceable, the adjacent Ethernet NE (in the MEN) must be provisioned to recover and transmit PRS traceable timing. Line timing should be used if synchronization messaging is available (e.g., SONET or SDH line timing sources). Line timing can also include deriving a clock from incoming Ethernet packets (e.g. use of adaptive or differential clock recovery methods). 3. External Timing: This mode is used to recover PRS traceable timing from a co-located building integrated timing supply (BITS). Timing is typically sent to the TSP/IWF as an all-ones DSl or El with framing. Synchronization status messaging (SSMs) may also be transmitted over the DSl link using a "blue signal" (all ones without framing) or via SSMs on the ESF data link as defined by ANSI Tl.lOl. 4. Free-Run: This mode is considered a stand-alone mode. It should only be used when a suitable line or external timing reference is not available. The frequency and stability of this timing mode is determined by the TSP/IWF's internal oscillator. 5. Holdover: This mode is usually considered a backup or protection timing mode. Holdover mode may be initiated when the external or line timing references have been lost due to a failure. This failure is indicated to the CES IWF via a loss of frame (LOF), loss of signal (LOS), or SSM indication. Unlike free-run, this timing mode relies on the TSP/IWF's internal oscillator, which has been trained by an external timing reference (e.g., a PRS-traceable timing source). See ANSI Tl.lOl for performance specifications of this and other timing modes. Holdover may also be used when there is a cessation of activity on the
Metro Ethernet Circuit Emulation Services
All
MEN-bound TDM interface (e.g., due to the normal operation of a variable bit rate IWF). 13.1.2.3.2 Synchronous IWF and Associated Tributaries Synchronous IWFs require that the IWF synchronization domain (SDIWF) at both the MEN-bound IWF and the CE-bound IWF use common clock synchronization (e.g., timing that is traceable to the same physical clock). Unless specific mechanisms have been put in place to ensure that a common clock is distributed between both IWFs, it should be assumed that the IWFs are not synchronous. It is not reasonable to assume the IWFs have suitable clocking information unless it has been specifically provided for. Tributaries to the IWFs consist of those TDM services that originate at the CE and are terminated at the IWF. The service clock of each tributary may be either synchronous (PRS traceable) or asynchronous (not PRS traceable). 13.1.2.3.2.1 Synchronous IWF and Tributaries Synchronous IWF: In this scenario, the synchronization domains at the MEN-bound and CE-bound IWFs (SDJWF) use common clock synchronization (e.g., timing that is traceable to the same physical clock). Synchronous Tributary: The synchronization domains of the CE devices (SDcen) require that at least one of the CE devices use external timing that is PRS traceable. The other CE, in the CES connection, may be either line timed to the originating CE or externally timed to a PRS-traceable source. 13.1.2.3.2.2 Synchronous IWF and Asynchronous Tributaries Synchronous IWF: In this scenario, the synchronization domains at the MEN-bound and CE-bound IWFs (SDJWF) use common clock synchronization (e.g., timing that is traceable to the same physical clock). Asynchronous Tributary: The synchronization domains of the CE devices (SDcen) require that at least one of the CE devices be externally timed to a non-PRS-traceable source or use its internal free-running oscillator. The other CE can be either line-timed or externally timed, or may operate on its internal free-running oscillator. Of these options, only the line-timed option will provide slip-free performance under nonfault conditions (see Table 13-4). 13.1.2.3.3 Asynchronous IWF and Associated Tributaries In the Asynchronous network, the synchronization domains of all IWFs (SDIWF) do not use a common source of timing. For example, the IWFs may receive plesiochronous timing. Tributaries to the IWF consist of TDM
478
Chapter 13
services that originate at the CE and are terminated at the IWF. At the TDM service processor, the service clock of each TDM tributary (SDcen) is terminated. This service clock will be used to support any functions necessary to create stmctured or unstructured TDM emulation services. After these services are created, the service clock and the TDM data are then input to the CES IWF. 13.1.2.3.3.1 Asynchronous IWF, Asynchronous Tributaries Asynchronous IWF: In this scenario, not all the synchronization domains of the IWF devices (SDIWF) use common clock synchronization (e.g., timing from the same physical clock). Timing distribution for the IWF may be configured with each either line-timed to the MEN, line-timed to a dedicated CE, or externally timed to a PRS traceable source. Asynchronous Tributary: The synchronization domains of the CE devices (SDcen) require that at least one of the CE devices be externally timed to a non-PRS traceable-source or use its internal free-mnning oscillator. The other CE can be either line-timed, can be externally timed, or may operate on its internal free-running oscillator. Of these options, only the line-timed option will provide slip-free performance under nonfault conditions (see Table 13-4). 13.1.2.3.3.2 Asynchronous IWF, Synchronous Tributaries Asynchronous IWF: In this scenario, not all the synchronization domains of the IWFs (SDJWF) use common clock synchronization (e.g., timing from the same physical clock). Timing distribution for the IWFs may be configured with each either line-timed to the MEN, line-timed to a dedicated CE, or externally timed to a PRS traceable source. Synchronous Tributary: The synchronization domains of the CE devices (SDcen) require that at least one of the CE devices use external timing that is PRS traceable. The other CE, in the CES connection, may be either line-timed to the originating CE or externally timed to a PRS traceable source. 13.1.2.3.4 Synchronization Administration Proper synchronization administration is crucial to support the transport of TDM emulation and circuit emulation services. Each synchronization domain in the CES needs to be considered and provisioned independently. Table 13-6 provides a summary of typically available options per synchronization domain.
Metro Ethernet Circuit Emulation Services Table 13-6. Timing options per synchronization domain CE IWF TSP SDcE SDiwF SDTDM External PerIA External Line Line - CE Free-Run Line-MEN Free-Run
479
MEN SDED
PRS Traceable Non-PRS Traceable
Proper synchronization administration begins with the following concepts: • Separate and Diverse: Synchronization equipment and paths should follow separate and diverse paths. This reduces the chance that a single point of failure will cause service disruption. This philosophy extends to equipment placement, cable routing, powering sources, and network element configuration. • Synchronization Trail: All synchronization flows have a defined source and end. The context of this flow is based on a logical flow rather than a topology (which could be linear, ring, mesh, or a combination of these). The flow can even span multiple transport technologies including Ethernet, SONET, and SDH. The source of a synchronization trail should be either from an independent reference that is PRS traceable or from a free-running oscillator. The synchronization trail ends on a specific device that does not further propagate the source timing information. Such a terminating device is necessary to prevent timing loops, which will cause transport errors in the physical layer. Intermediate equipment in the synchronization trail can use timing information derived from the source in a daisy-chained configuration. It should be noted that upstream timing events (protection switching, holdover events, clock failures, etc.,) will be propagated to downstream equipment in this configuration. For these cases, downstream equipment will need to have predetermined fault modes to deal with these circumstances when they arise. Such fault modes may include protection switching to use a different line-timed reference or entry into holdover. • Service Clock Preservation :The service clock is defined at the timing source used to generate the client signal. In a CES transport network, the service clock frequency must be preserved in order to have error-free transport of the physical layer. That is to say, the average bit rate of the transmitting and receiving Customer Edge devices must be the same. Otherwise, the transport capability of the CES will be compromised. • Synchronization Traceability: In synchronization network planning, it is important to know where a timing source originates, how it flows in the network, and its quality. The concept of traceability provides the
480
Chapter 13
answers to these quantities. Synchronization flows can be described in terms of source traceability. That is to say, that timing originates at a physical location and is used by equipment in a synchronization trail either by external or line-timing options. Common clock distribution is an example of a source-traceable clock scheme. In the case offrequency traceability, the quality of a timing signal can be specified. For example, the frequency of synchronization sources may be specified as being PRS traceable or accurate to within 1 e-11 (.00001 parts per million). Plesiochronous clock distribution is an example of a frequency-traceable clock scheme. Three synchronization administration models are presented in the following sections. These models have been chosen to illustrate how they may be used to accommodate the CES definitions. These models are as follows: • Single Service Provider-Owned Network: A single service provider controls the entire transport synchronization trail. The service synchronization trail is separate. • Multi Service Provider-Owned Network: Multiple service providers control the transport synchronization trail. The service synchronization trail is separate. • Private (customer-owned) Network: A customer controls the entire transport and service synchronization trails. Table 13-7 summarizes these synchronization administration models and shows how they map to the CES service definitions. Table 13-7. CES services and supporting synchronization administration models CES Service Sync. Admin. Model Notes Can also be used with Private TDM Virtual Private Line Single Service Provider Network model. service model Handoff to PSTN does not TDM Access Line service Single Service Provider retime the pay load (DSl or model DS3). Therefore, there is no new sync trail. Customer-operated CES IWF at CES egress may be Multi service Provider over Metro Ethernet owned by a different service model provider.
13.1.2.3.4.1 Single Service -Provider-Owned Network The synchronization administration for a single service -provider-owned network is illustrated in Figure 13-13.
Metro Ethernet Circuit Emulation Services
SDci
!
SD,wF
I
SDTDM
Transport Timing
481 gD,
OI^IWF
1
SDTDM
Service sync trail
IWF T^
i
'l
\ \^
SDcE
CE2
\
Transport " sync trail Service Timing
Figure 13-13: Synchronization Administration for a Single Service Provider Network
A summary of available timing options for the Single Service Providerowned-network is presented in Table 13-8. Table J3-8. Timing Options for Single Service Provider- Owned Network Device Timing Domain Timing Options Notes CE SDcE External Customer Option: Requires a collocated BITS or SSU to supply a timing input to the CE. May be used as a source for the synchronization trail. Customer Option: May be used at the end Line of a synchronization trail. Customer Option: May be used as the Free-Run source for the synchronization trail. IWF SDn SeelA Service Provider Option: Timing recovery for CES IWF must match the IWF type. Details can be found in the IA as specified in Table 13-2 (CES Interface Definition). TSP SDT Service Provider Option: Requires a External collocated BITS or SSU to supply a timing input to the PE. Line-CE Not used, since CE and TSP are on different synchronization trails. Line-MEN Service Provider Option: Used if the IWF can provide timing. Consistent with the service needs of the TSP. Options and capabilities for a specific IWF are specified per the appropriate I A. Service Provider Option: Used only if there Free-Run is no suitable line or external timing source. PRS Traceable MEN SDED Service Provider Option: Requires that all elements use PRS traceable timing. Non-PRS Service Provider Option: Requires that all Traceable elements use PRS traceable timing.
Note that in the Single Service Provider configuration, the synchronization trails for service-timing and transport-timing are separate. Separation of these synchronization trails is typically done for administrative and liability reasons. An end customer will not be able to manage a service
482
Chapter 13
provider's timing equipment. Likewise, a service provider would typically not use timing from a CE due to the reliability and liability issues associated with a single point of failure. Separate synchronization trails also mean that the physical timing source used to derive the service clock is not the same as that used in the transport network (TSP/IWF and MEN). This is not to say that the source for the service and transport cannot be PRS traceable, simply that one is not the source for the other. 13.1.2.3.4.1.1 Service Timing— Single Service Provider Network The Customer Edge synchronization domain (SDCE) is owned and maintained by the customer. Timing for the CE (SDCE) requires that at least one CE be the source of timing. Referring to Table 13-4, any of the six options listed can be used. The choice of which option to use will depend on economic and service needs. A PRS-traceable source can be highly reliable and accurate but requires physical placement and substantial engineering support. It is for this reason that option 1 may be more expensive to administer than options 2 and 3. Options 4 and 5 rely on the CE's internal oscillator to provide a frequency source for the service clock. In this case, due to the relaxed need for additional timing equipment (BITS or SSUs) this option may be less expensive to administer than option 1 and require less engineering support. Option 6 may yield undesirable slip performance due to the frequency difference between the free-running clocks. This option is generally the least expensive but also has the lowest overall performance. 13.1.2.3.4.1.2 Transport Timing— Single Service Provider Network Timing for the single service provider network assumes that the service provider owns the TSP/IWF and all the Ethernet devices in the MEN. The equipment will encompass synchronization domains SDJWF, SDTDM , and SDED. For this case, it is up to the service provider to perform the synchronization administration of all transport and synchronization equipment. Regardless of synchronization options that the service provider may use, as listed in Table 13-5, it is the ability of the service provider's network to accurately transport the service clock that is most important. Therefore, the operation of the CES interworking function is key to transport timing. The CES interworking function (SDJWF ) must use timing that is consistent with the IWF type. The TSP synchronization domain (SDJDM ) provides a foundation for the CE IWF. PRS-traceable timing may be available via externally timing the TSP/IWF or line timing (from the MEN). It should be noted that the use of
Metro Ethernet Circuit Emulation Services
483
synchronization messaging (SSM) between the MEN and TSP/IWF is required to always ensure that the TSP/IWF is receiving PRS-traceable timing. SSMs facihtate fault or protection switching from upstream failures. If SSMs are not available, then external timing is the preferred timing mode for PRS-traceable timing. 13.1.2.3.4.2 Multi Service Provider-Owned Network Synchronization administration for a Multi Service -Provider-Owned Network is illustrated in Figure 13-14. ,^.„ Service sync trail ^ ^ Transport i Provider-A f t Provider-B | sync trail JTransport Timing; [Transport Timing;
Provider - C | Transport Timing;
Figure 13-14: Synchronization Administration for a Multi Service Provider Network
A summary of available timing options for a Multi Service -ProviderOwned Network is presented in Table 13-9. Note that in the Multi Service Provider configuration, the synchronization trails for service timing and transport timing are separate. Also notice that the transport timing can be TDM based (PSTN) or Ethernet based (MEN). Separation of these synchronization trails is typically done for administrative and liability reasons. An end customer will not be able to manage a service provider's timing equipment. Likewise, a service provider would tj^ically not use timing from a CE due to the reliability and liability issues associated with a single point of failure. Separate synchronization trails also means that the physical timing source used to derive the service clock is not the same as that used in the transport network (TSP/IWF and MEN). This is not to say that the source for the service and transport cannot be PRS traceable, simply that one is not the source for the other. 13.1.2.3.4.2.1 Service Timing —Multi Service Provider Networl^: The Customer Edge synchronization domain (SDCE) is owned and maintained by the customer. Timing for the CE (SDCE) requires that at least one CE be the source of timing. Referring to Table 13-6, any of the six
484
Chapter 13
Table 13-9. Timing options for multi service provider-owned network Device Timing Timing Notes Domain Options Customer Option: Requires a collocated BITS or SSU to CE SDcE External supply a timing input to the CE. May be used as a source for the synchronization trail. Line Customer Option: May be used at the end of a synchronization trail. Free-Run Customer Option: May be used as the source for the synchronization trail. PSTN SDTDM External Service Provider Option: Requires a collocated BITS or SSU to supply a timing input to the PE. Line-CE Customer Option: May be used to recover timing from the received client signal. This mode should only be used if synchronization messaging is available (e.g., SONET or SDH line-timing sources). Line-TSP Service Provider Option: Used only if the connecting TSP equipment is PRS traceable. This mode should only be used if synchronization messaging is available (e.g., SONET or SDH-line timing sources). Free-Run Service Provider Option: Used only if there is no suitable line or external timing source. IWF SDiwF See lA Service Provider Option: Timing recovery for CES IWF must match the IWF type. Details can be found in the lA as specified in Table 13-2 (CES Interface Definition). TSP SDTDM External Service Provider Option: Requires a collocated BITS or SSU to supply a timing input to the PE. Line — Customer Option: May be used to recover timing from CE the received client signal. This mode should only be used if synchronization messaging is available (e.g., SONET or SDH line-timing sources). Line — Service Provider Option: Used only if the connecting PSTN PSTN equipment is PRS traceable. This mode should only be used if synchronization messaging is available (e.g., SONET or SDH line-timing sources). Line— Service Provider Option: Used if the IWF can provide MEN timing. Consistent with the service needs of the TSP. Options and capabilities for a specific IWF are specified per the appropriate I A. Free-Run Service Provider Option: Used only if there is no suitable line or external timing source. PRS MEN SDED Service Provider Option: Requires that all elements use Traceable PRS-traceable timing Non-PRS Service Provider Option: Requires that one or more Traceable elements not use PRS-traceable timing.
options listed can be used. The choice of options to use will depend on economic and service needs. A PRS-traceable source can be highly reliable and accurate but requires physical placement and substantial engineering
Metro Ethernet Circuit Emulation Services
485
support. It is for this reason that option 1 may be more expensive to administer than options 2 and 3. Options 4 and 5 rely on the CE's internal oscillator to provide a frequency source for the service clock. In this case, due to the relaxed need for additional timing equipment (BITS or SSUs), this option may be less expensive to administer than option 1 and require less engineering support. Option 6 may yield undesirable slip performance due to the frequency difference between the free-running clocks. This option is generally the least expensive but also has the lowest overall performance. 13.1.2.3.4.2.2 Transport Timing —Multi Service Provider Network Timing for the Multi Service Provider Network assumes that no one service provider owns the PSTN, TSP/IWF, and all the Ethernet devices in the MEN. The equipment will encompass synchronization domains SDIWF? SDTDM, and SDED. For this case, it is up to each service provider to perform the synchronization administration of all transport and synchronization equipment in its domain. Regardless of synchronization options that the service provider may use, as listed in Table 13-10, it is the ability of the service provider's network to accurately transport the service clock that is most important. Therefore, the operation of the CES interworking function is key to transport timing. The PSTN synchronization domain (SDTDM) preserves the service clock through the PSTN. PRS-traceable timing may be provided to all the PSTN network elements (NEs) externally by BITS or via line timing from adjacent NEs. It should be noted that the use of synchronization messaging (SSM) between NEs may be required to ensure that each NE is receiving PRStraceable timing. SSMs facilitate fault or protection switching from upstream failures. If SSMs are not available, then external timing is the preferred timing mode for PRS-traceable timing. The TSP synchronization domain (SDTDM ) provides a foundation for the CE IWF. PRS-traceable timing may be available via externally timing the TSP/IWF or line timing (from the MEN). It should be noted that the use of synchronization messaging (SSM) between the MEN and TSP/IWF is required to always ensure that the TSP/IWF is receiving PRS-traceable timing. SSMs facilitate fault or protection switching from upstream failures. If SSMs are not available, then external timing is the preferred timing mode for PRS-traceable timing. Private (Customer-Owned) Network Synchronization administration for a private network is illustrated in Figure 13-15.
Chapter 13
486 SD,wF
Transport Timing
gp
ouFiwF SDTSP
SDcE CE2
:
Service Timing
Figure 13-15. Synchronization Administration for a Private A summary of available timing options for a private network is presented in Table 13-10. Table 13-10. Timing options for a private network Device CE
Timing Domain SDcE
Timing Options External Line Free-Run
IWF
SDiwF
See lA
TSP
SDTDM
External
Line—CE
Line— MEN
Free-Run
MEN
SDED
PRS Traceable Non-PRS Traceable
Notes Customer Option: Requires a collocated BITS or SSU to supply a timing input to the CE. The most accurate means of timing distribution. Customer Option: May be used at the end of a synchronization trail. Customer Option: May be used as the source for the synchronization trail. Service Provider Option: Timing recovery for CES IWF must match the IWF type. Details can be found in the IA as specified in Table 13-2 ( CES Interface Definition). Service Provider Option: Requires a collocated BITS or SSU to supply a timing input to the TSP. The most accurate means of timing distribution. Customer Option: May be used to recover timing from the received client signal. This mode should only be used if synchronization messaging is available (e.g., SONET or SDH line-timing sources). Service Provider Option: Used if the IWF can provide timing. Consistent with the service needs of the TSP. Options and capabihties for a specific IWF are specified per the appropriate lA. Service Provider Opfion: Used if the IWF can provide timing. Options and capabilities for a specific IWF are specified per the appropriate lA. Service Provider Option: Requires that all elements use PRS-traceable timing Service Provider Option: Requires that one or more elements not use PRS-traceable timing.
Note that in this configuration, the synchronization trails for service timing and transport timing overlap. This is allowed because the customer owns and operates the equipment that supports all the synchronization domains. In this case, the customer and service provider are the same entity,
Metro Ethernet Circuit Emulation Services
487
so combining synchronization trails should not create any problems. The rules for timing, as with the single and multi service provider cases, should still be followed. 13.1.2.3.4.2.3 Service Timing - Private Networlc The Customer Edge synchronization domain (SDCE) is owned and maintained by the customer, just as in the single service provider case. In fact, all timing aspects related to private networks are identical to those of the single service provider case. 13.1.2.3.4.2.4 Transport Timing —Private Networli: Timing for the private network assumes that the TSP/IWF and Ethernet devices in the MEN are all owned by the same customer. This equipment encompasses synchronization domains SDIWF, SDTDM, and SDED. The options for configuring this equipment are shown in Table 13-10. The timing options for the Private Network are the same as those of the single service provider case, with the exception of the line-CE mode for the TSP/IWF. In this mode, the TSP/IWF are allowed to use synchronization from client signal as sent by the CE. In order to ensure the best-quality timing, it is recommended that the CE sourcing this client signal use PRStraceable timing. It should be noted that the use of synchronization messaging (SSM) between the CE and TSP/IWF are required to always ensure that they are receiving PRS-traceable timing. SSMs facilitate fault or protection switching from upstream failures. If SSMs are not available, then external timing is the preferred timing mode for PRS-traceable timing. 13.1.2.4
Performance Monitoring and Alarms
The CES should allow the operation of the normal mechanisms for monitoring the performance of the TDM service, at the level of the TDM interface type. Performance monitoring is one-way, and occurs between two endpoints. 13.1.2.4.1 Facility Data Link The Circuit Emulation Service will carry any signal that meets the bit rate requirement. In the particular case of DSl circuits using ESF framing, a Facility Data Link (FDL) may be present in the signal. The DSl ESF Facility Data Link is used to carry once-per-second Performance Report Messages. These messages carry information on numbers of CRCs, framing errors, line code violations, and other impairments detected over the last second.
488
Chapter 13
The CES IWF is allowed to monitor the FDL, and to modify the relative position of the FDL with respect to the TDM payload, but must not change messages carried by the FDL or insert new FDL messages. For example, the Interworking Function may be required to monitor Performance Report Messages. 13.1.2.4.2 Alarms 13.1.2.4.2.1 Unstructured Service For unstructured services, all alarms received at the input of the Service Interface are carried through to the output Service Interface without modification, since they are embedded in the data on the wire. In addition to this, the IWF can detect a loss of signal (LOS) at the IWF Service Interface. Upon detection of LOS, the IWF is required to notify the IWF at the opposite end of the CES service. 13.1.2.4.2.2 Structured Service For structured N x 64 service, the alarm status is not necessarily propagated within the data. Several kinds of alarms can be detected at the point where the Service Interface is received by the IWF. Some alarm situations require that an alarm condition detected at the point where the TDM Service Interface is received by the MEN-bound IWF be propagated downstream to the CE-bound IWF responsible for reproducing the bit stream. 13.1.2.4.2.3 Buffer Underflow and Overflow The IWF at the egress of the MEN will require a buffer in which the reassembled data stream is stored before it is transmitted out the Service Interface. The size of this buffer will be implementation dependent, but it must be large enough to accommodate expected MEN frame jitter, while small enough to not introduce excessive delay in the emulated circuit. This buffer will be subject to overflow or underflow if slight clocking differences exist between the upstream and downstream IWF, or in the presence of unexpectedly large network jitter. In the case of an underflow, or data starvation condition, data will have to be inserted into the TDM stream until a new Ethernet frame has arrived. The data to be inserted is implementation dependent. Under some circumstances, such as a failure in the MEN network carrying the emulated service, the flow of Ethernet frames to the reassembly unit will stop for an extended period. This situation is effectively the same as an LOS condition on the TDM network. If this condition persists for a long period, itshould be signaled to the downstream TDM equipment using a Trunk Conditioning procedure. For most applications, implementers are
Metro Ethernet Circuit Emulation Services
489
advised to use a 2.5 ± 0.5 s integration period, in a manner analogous to that used to integrate Loss of Signal to declare a red alarm state. Although not required as part of this specification, implementers may wish to consult Bellcore GR-1113-CORE and ETSI ETS 300 353 Annex D for advice on the handling of various fault conditions. 13.1.2.4.3 End-to-End Delay End-to-end delay requirements are application specific. However, it should be noted that excess delay could have adverse effects on some traffic types, such as voice. 13.1.2.5 Service Impairment This section addresses impairments to the emulated TDM service caused by errors within the MEN. Principally, it addresses the relationship between the performance parameters of the underlying Metro Ethernet Network (MEN) transport to the defined service impairment metrics for the TDM service being emulated - errored seconds (ESs) and severely errored seconds (SESs). 13.1.2.5.1 Errors Within the MEN Causing TDM Service Impairment There are three performance parameters defined for an EVC that have an impact on the TDM service impairment metrics: Frame Loss, Frame Delay, and Frame Jitter. In addition, any bit errors induced in the data flow across the MEN will also have an impact on the TDM service, although there is no performance metric defined to measure this within the MEN. 13.1.2.5.1.1 Frame Loss Frame loss is defined as the percentage of in-profile frames ("green" frames that are within CIR/CBS) not reliably delivered over a measurement interval T. Frame Lossj = (1 - frames delivered to destination / total frames sent to destination) X 100 Frame loss will cause a burst of bit errors in the reconstructed TDM stream. 13.1.2.5.1.2 Frame Delay and Frame Jitter Frame Delay is defined as the maximum delay measured for a percentile (P) of successfully delivered frames over a measurement interval T. Frame Jitter can then be derived from the Frame Delay measurement using the maximum and minimum of Frame Delay samples over same measurement interval T and percentile (P). Frame Jitter can be calculated as follows:
490
Chapter 13
Frame Jitterj,? = Frame Delayx,? (Max. measured delay value - Min. measured delay value) Typically the Frame Jitter is used to size the reassembly buffer in the IWF at the egress of the MEN. This buffer is sometimes referred to as the jitter buffer. However, the Frame Jitter figure is calculated on a percentile of frame delay values, where the percentile is defined as part of the Service Level Specification. Therefore, a small percentage of frames will arrive outside the specified jitter level. Depending on the size of the jitter buffer, these frames may either arrive too late to be played out or too early to be accommodated within the buffer. Such frames will then be discarded, and must be considered lost for the purposes of reconstructing the TDM service. 13.1.2.5.1.3 Bit Errors Bit errors induced in the MEN will normally be detected by a CRC error in the frame check sequence. Therefore, the errors will cause the whole frame to be discarded. On some occasions a frame containing bit errors may still yield a correct CRC value, but this outcome is expected to be extremely rare. Therefore, as far as the IWF is concerned, any bit errors in the data stream will appear as lost frames. 13.1.2.5.1.4 Frame Error Ratio and IWF behavior The collective sum of all the above errors (frame loss, excess frame jitter and bit errors) can be aggregated into a single measure termed Frame Error Ratio (FER), defined as the total of all effects leading to the loss of or discarding of a frame. For the purposes of TDM emulation, an Ethernet frame is deemed to be errored if it 1. fails to arrive at the egress IWF 2. arrives too late to be played out 3. arrives too early to be accommodated in the jitter buffer 4. arrives with bit errors, causing the frame to be discarded In order to maintain timing integrity, the IWF must then insert an equivalent number of octets of data into the reconstructed stream. The data to be inserted is application and/or implementation dependent. If the frame should subsequently arrive, it should be discarded. 13.1.2.6
TDM Signaling
It is not normally required for the Circuit Emulation Service to intercept or process TDM signaling, e.g., Channel Associated Signaling (CAS) or Common Channel Signaling (CCS). Signaling is embedded in the TDM data stream, and hence it is carried end-to-end across the emulated circuit. The
Metro Ethernet Circuit Emulation Services
491
signaling can be extracted by the end equipment from the data that has been transported across the MEN. The exception is N x 64 kbit/s service where CAS signaling is supported. N X 64 kbit/s Service with CAS requires direct recognition and manipulation of the signahng bits by the CES IWF. This mode is necessary to support multiplexed N x 64 kbit/s applications requiring DSl Robbed Bit Signaling or El CAS support, where the association between the signaling bits and the channel may otherwise be lost. 13.1.2.7
Loopbacks
Loopbacks should be supported at the level of the TDM interface type. Loopback of internal multiplexed levels (e.g., a DSl level inside an OC-3 line) is outside the scope of this document. 13.1.2.7.1 Provider-Controlled Loopbacks It is suggested that provider-controlled loopbacks should be supported in the manner illustrated in Figure 13-16, where the direction of loopbacks is from PEl (near end) towards PE2 (remote end). Loopback for PE2 to PEl is the reverse of Figure 13-16. The aim of the loopback is to isolate the fault with respect to PEl. Therefore, four loopbacks at the system level are suggested: 1. a PEl-CEl facility loopback, which should be located at a point within the provider's network as close as practicable to the Network Demarcation, for example, at a final span line repeater or SmartJack; 2. a PEl terminal loopback of the TDM interface; 3. a PEl terminal loopback of the MEN interface; and 4. a PE1-PE2 MEN loopback. The loopback at the Network Demarcation must comply with the appropriate specification for the TDM interface type (e.g., ANSI for a DSl interface).
492
Chapter 13 Cli4 ^««
:i
2
€El\
PE2
MEN
PEl
4
I
^M>om^
"W^
;
%mmm,^
Network Demarcation
Network Demarcation
Figure J 3-16: Provider-cControlled loopback points
13.1.2.7.2 Customer-Controlled Loopbacks It is also possible for customers to use loopbacks to verify the operation either of their own equipment or of the service being provided. The aim of the loopback is to isolate the fault with respect to a particular item of customer equipment (e.g., CEl), while treating the service provider network as a "black box." Therefore, two loopbacks are suggested, as illustrated in Figure 13-17: 1. Local loopback at CEl 2. Remote loopback at CE2 The required loopbacks for CE2 are the reverse of those shown in Figure 13-17.
CEl
PEl
w
MEN U . «. ««. ^
^
PE2
€1^ ^
;
2
^ ^ ^ ^
Network Demarcation
Network Demarcation
Figure 13-17: Customer-cControlled loopback points
Metro Ethernet Circuit Emulation Services
493
Customer-controlled loopbacks are often initiated under manual control, although signaling may be provided to operate the remote loopback. The nature of such signaling is implementation specific. Customers are not normally able to control the provider loopbacks shown in Figure 13-16, except via a service call to the provider concerned. 13.1.2.8
Protection
TDM networks usually include protection mechanisms for service recovery in less than 50 ms. The CES solution should fit into such a protection scheme. This means the following: • A CES service running over a MEN should have capabihties for sub-50 ms protection within the MEN part. • A failure of the TDM interface connecting a TDM legacy network to the MEN should cause a switchover to an alternative TDM interface. This switch should interact both with the MEN and with the legacy network so that both sides of the interface start working with the alternative interface. The following scenarios may be relevant for CES protection. 13.1.2.8.1 Scenario 1 — Dual Unprotected Services In dual unprotected service, topologies a failure of one of the interfaces connecting the TDM and MEN, or inside the MEN, will cause the TDM network to work with the alternative CES connection. This setup may use 1+1 protection, 1:1 protection, or 1:N protection, but it is outside the scope of the CES solution.
O ^ uap^tected txmix^m. \
MEM CBS imp««tertM ^i&me^KM 1
Figure 13-18: Dual unprotected services
The CES service is a nonprotected service. Protection is established through the mechanisms of the TDM network connected on both sides of the MEN. For example, a SONET network connected on both sides of the CES connection could perform the protection switching based on either APS (Automatic Protection Switching) or BLSR (Bidirectional Line Switched
494
Chapter 13
Ring). Since the protection-related information is carried in the hne overhead (LOH), there are two possibiHties: • The CES service is not terminating the Hne layer. The protection information is carried across the MEN to the other side. • The CES service is terminating the line layer. Upon failure of the path in the MEN, the PE updates the relevant protection bytes in the LOH within the requirements of SONET/SDH, so that the SONET/SDH equipment can make the switching in time. This may be by asserting AIS/RDI bits for the line. 13.1.2.8.2 Scenario 2 —Dual Protected Services The dual protected service scenario is similar to the previous one; however, each CES connection is protected inside the MEN.
Figure 13-19: Dual protected services
In the case of a failure in the TDM interface, the behavior is similar to the previous scenario. In the case of a failure inside the MEN, both the MEN internal mechanisms may trigger bypass, and the TDM network may initiate a switch to the alternate TDM interfaces as well. This setup requires careful design in order to avoid races between the TDM and the MEN protection mechanisms. 13.1.2.8.3 Scenario 3 —Single Protected Service In The single protected service scenario, a single TDM interface is used on each side of the MEN. In this case, the CES connection in the MEN needs to be protected using MEN protection mechanisms. A failure inside the MEN would be bypassed in less than 50 ms so that service is maintained.
Metro Ethernet Circuit Emulation Services
495
Figure 13-20: Single protected service
This scenario does not cover the case of TDM interface failure. The endto-end protection mechanisms require detecting the Uvehness of an end-toend path. This is done generically through an OAM mechanism. Note that in the case of CES, traffic is constantly flowing through the active path (unlike Ethernet connections, where there may be silent periods on the connection). Receiving traffic at the end of the path can therefore be used for detecting path liveliness, and can trigger switchover in case of failure. 13.1,2.8.4 Scenario 4 —Single-to-jPual Interface Service
^»-<>Z,^
'^^^
Figure 13-21: Single-to-dual interface connection
In the single-to-dual interface service topology, there is a single TDM interface on one side of the MEN and two on the other side. This scenario is typical of a network where the right side is a server/hub— for example, multiple users connected with DSl lines to a central office with OC-3. The subscribers may get a single DSl interface, while on the other side there are 2 OC-3 uplinks so that a failure of the OC-3 uplink (or even the TDM equipment attached to it) will still enable using the other OC-3 link. In this case, traffic coming in from CEl, CE2, and CE3 is sent on both internal connections to CE A and CE B. It is multiplexed into a single higher-rate TDM link going towards CE A & CE B. On the other direction, traffic coming from CE A & CE B is demultiplexed, and each lower rate is sent towards its destination CE (1/2/3). The links between the end-subscribers in CE 1/2/3 and the MEN are not protected. However, each such link affects only one subscriber. On the central office side, each link affects multiple subscribers, and is therefore protected.
496
Chapter 13
Each CES on the left side can select the source for (red or blue of) the lower rate it sends towards CE1/2/3. In the other direction, each CES connected to CE 1/2/3 sends two copies of the traffic, one towards CE A and one towards CE B. Both CE A and CE B receive traffic on their TDM links and can select between them. 13.1.2.9 Service Quality In general, CES should aim to perfectly emulate the TDM services, thus allowing service providers a migration path to MENs while not sacrificing service quality. Noncompliance of an emulated service to the TDM service requirements as specified in the relevant ANSI and ITU documents shall be explicitly stated in the implementation agreement, as well as the conditions under which this noncompliance occurs. 13.1.2.10
Efficiency
The CES solution should try to minimize both network delay and the network bandwidth needed for transporting the TDM traffic. During setup of an emulated circuit, the CES solution should enable the MEN provider to favor efficiency over network delay and vice versa, by being able to define the size of the TDM payload transmitted in each Ethernet frame. For example, a small payload results in low latency, since the CES IWF does not have to wait long for the Ethernet frame to fill. However, the ratio of header to payload is large, making small Ethernet frames inefficient in terms of bandwidth consumption in the MEN. Therefore, the payload should be made as large as possible, while still keeping within the latency budget for the emulated circuit. It is not required to be able to dynamically adjust the size of the payload while an emulated circuit is in operation.
ACKNOWLEDGMENTS Metro Ethernet Forum Tim Frost
13.2.
REFERENCES
Metro Ethernet Forum, Technical Specification MEF 3, Circuit Emulation Service Definitions, Framework and Requirements in Metro Ethernet Networks, April, 2004.
Chapter 14 METRO ETHERNET NETWORK RESILIENCY AND TRAFFIC MANAGEMENT Nan Chen Sirix Systems; Metro Ethernet Forum President
14.1.
METRO ETHERNET NETWORK RESILIENCY
14.1.1 Introduction Protection in Metro Ethernet Networks (MEN) [1] can encompass many ideas. Protection is a self-healing property of the network that allows it to continue to function with minimal or no impact to the network users upon disruption, outages, or degradation of facilities or equipment in the MEN. Naturally, there is a limit to how much the network can be disrupted while maintaining services, but the emphasis is not on this limit but rather on the ability to protect against moderate failures. Network protection can be viewed in two ways: • From the viewpoint of the user of the MEN services, the actual methods and mechanisms are of minor concern; it is the availability and quality of the services that are of interest. These can be described in a Service Level Specification (SLS), a technical description of the service provided, which is part of the Service Level Agreement (SLA) between customer and provider. • The other viewpoint is that of the network provider. The provider is tasked with translating the SLSs of all the customers (and future customers) into requirements for the network design and function. We do not study this translation here; it is an area of differentiation and specialization for the provider and depends on the policies that the
498
Chapter 14
provider will use for protection. What we do study is the mechanisms that can be used to provide protection. Any protection scheme has three clear components: • Detection refers to the ability to determine network impairments. • Policy defines what should be done when impairment is detected. • Restoration is the component that acts to "fix" the impairment; it may not be a total restoration of all services and depends on the nature of the impairment and the policy. We focus on the detection and restoration mechanisms and leave the choice of policy to the providers. However, the policy itself cannot be ignored and is based on the services supported. Detection and restoration can be done in many different ways in the MEN. The techniques available depend on the nature of the equipment in the network. The requirements have their basis in the interpretation of Service Level Specifications for Ethernet services (such as availability, mean time to restore, mean time between failure, etc.) in terms of network protection requirements (such as connectivity restoration time, SLS restoration time, protection resource allocation, etc.). In other words, the protection offered by the network is directly related to the services supplied to the user and the requirements derived from the need to protect the services provided to the user. In most cases, an Ethernet Virtual Connection (EVC) implementing an Ethernet service traverses different transport networks (also known as transports), and therefore the end-to-end protection may involve different mechanisms. For example, many transport networks may be involved: Ethernet, Ethernet over DSL, Ethernet over SONET/SDH, MPLS, and data link layer switching as Ethernet. In the case of Ethernet protection, technologies such as Rapid Spanning Tree Protocol (RSTP) or Link Aggregation may be used to provide protection at the ETH layer. An Ethernet Line service EVC is built of a single ETH-trail, while an Ethernet LAN service EVC is built of a number of ETH-trails. The protection requirements have two distinct goals, one set of which is specified for Service Level Specifications (such as protection switching time) and can be measurable parameters, and the other set of which is specified for providers of the service (such as Protection Control Requirements specifying protection configuration) and is not directly reflected in a Service Level Specification but is required from the provider. Examples for such requirements are those that relate to control, manageability, and scalability of a protection scheme. The following topics are examples of those discussed in the requirements section:
Metro Ethernet Network Resiliency and Traffic Management
499
• • • • • • • •
Protection switching times; Failure detection requirements; Protection resource allocation requirements; Topology requirements; Failure notification requirements; Restoration and revertiveness requirements; Transparency for end-user; and Security requirements: e.g., separation between LAN & MAN protection mechanisms. Observe that if all EVCs passing through a specific connected part of the network are known to have similar protection requirements, it is sufficient for this part of the network to comply with the specific requirements that are needed by the EVCs of services passing through it. An example is the lastmile: protection requirements are directly related to the customers needs. The framework section deals with models and mechanisms specific to the Metro Ethernet. We can make use of any existing mechanisms for protection of transport, and upper-layer protection mechanisms can sit on top of lower-layer protection mechanisms to provide a unified protection approach. The model allows protection mechanisms to be enabled as part of each layer (ETH layer or TRAN layer) in the network.
14.1.2 Protection Terminology This section defines the precise MEF protection terminology. 14.1.2.1
Protection Types
A network can offer protection by providing alternative resources to be used when the working resource fails. There is specific terminology for the number and arrangement of such resources. 14.1.2.1.1 1+1 The Protection Type 1+1 uses the protection resources at all times for sending a replica of the traffic. The protection merge point, where both copies are expected to arrive, decides which of the two copies to select for forwarding. The decision can be to switch from one resource to the other due to an event like resource up/down, etc., or can be on a per-frame/cell basis. The selection decision is performed according to parameters defined below (e.g., revertive, nonrevertive, manual, etc.).
500
Chapter 14
UA2A2 mm The m:n Protection Type provides protection for n working resources using m protection resources. The protection resources are only used at the time of the failure. The protection resources are not dedicated for the protection of the working resources, meaning that when a protection resource is not used for forwarding traffic instead of a failed working resource, it may be used for forwarding other traffic. The following subsections define the important special cases of m:n protection. There are two variants of the m:n protection type, one in which a protection resource can be used concurrently for forwarding the traffic of a number of working resources, in case a few of them fail at the same time. The other variant is when the protection resource is able to forward the traffic of a single working resource at a time. 14.1.2.1.2.1 1:1 The 1:1 Protection Type provides a protection resource for a single working resource. 14.1.2.1.2.2 n:l The n:l Protection Type provides protection for one working resource using n protection resources. 14.1.2.1.2.3 l:n The l:n Protection Type provides protection for n working resources using one protection resource. In this protection type, the protection resource is shared for protection purposes by the n working resources. 14.1.2.2
Failure Types
Failures may occur in network nodes or on the links between nodes. 14.1.2.2.1 Fail condition (Hard Link Failure) Fail condition is the status of a resource in which it is unable to transfer traffic (e.g.. Loss of Signal, etc.). 14.1.2.2.2 Degrade condition (Soft Link Failure) Degrade Condition is the status of a resource in which traffic transfer might be continuing, but certain measured errors (e.g.. Bit Error Rate, etc.) have reached a predetermined threshold.
Metro Ethernet Network Resiliency and Traffic Management
501
14.1.2.2.3 Node Failure A Node Failure is an event that occurs when a node is unable to transfer traffic between the links that terminate at it. 14.1.2.3
Resource Selection
14.1.2.3.1 Revertive Mode The protection is in revertive mode if, after a resource failure and its subsequent repair, the network automatically reverts to using this initial resource. The protection is in nonrevertive mode otherwise. Automatic reversion may include a reversion timer (i.e., the Wait To Restore), which delays the time of reversion after the repair. 14.1.2.3.2 Manual Switch A Manual Switch is when the network operator switches the network to use the protection resources instead of the working resources, or viceversa. By definition, a Manual Switch will not progress to failed resources. A manual switch may occur at any time according to the intention of the network operator, unless the target resource is in failure condition. 14.1.2.3.3 Forced Switch A Forced Switch is when the network operator forces the network to use the protection resources instead of the working resources, or viceversa, regardless of the state of the resources. 14.1.2.3.4 Lockout A lockout command on a resource makes the resource not available for the protection of other resources. 14.1.2.4
Event Timing
The terminology distinguishes events, which occur at particular instants (points in time), and times, which are the time durations between events. \4A2AA Impairment Instant The Impairment Instant is the point in time that the failure event occurs. 14.1.2.4.2 Fault Detection Instant The Fault Detection Instant is the point in time at which the failure is detected and declared. The Fault Detection Instant may be different for different network elements.
502
Chapter 14
14.1.2.4.3 Hold-off Instant It may be desirable to delay taking any action after the Fault Detection Instant. The Hold-off Instant is the instant at the end of this delay period, if there is one. Otherwise, it is the same as the Fault Detection Instant. The hold-off instant is useful when two or more layers of the same network provide protection. In such a case, the hold-off instant in an upper layer gives an opportunity to the protection in a lower layer to take place before the upper layer protection acts. 14.1.2.4.4 Connectivity Restoration Instant The Connectivity Restoration Instant is the first point in time after impairment that user traffic can begin to be transferred end-to-end. At the connectivity restoration instant, services affected by the failure already deliver user traffic end-to-end but may not do so with the performance required by the SLS. In case the impairment caused only degradation in performance to the point of losing SLS compliance, the connectivity restoration instant is defined to be identical to the impairment instant. 14.1.2.4.5 SLS Restoration Instant The SLS Restoration Instant is the first point in time after impairment that user traffic can begin to be transferred end-to-end with the original performance guarantees. 14.1.2.4.6 Reversion Instant In revertive mode, the Reversion Instant is the point in time at which the original resources are again used. This point in time may be the same as the SLS Restoration Instant. UA2AJ Detection Time The Detection Time is the difference between the Fault Detection Instant and the Impairment Instant. 14.1.2.4.8 Hold-off Time The Hold-off Time is the difference between the Hold-off Instant and the Fault Detection Instant. The Hold-off Time may be zero. 14.1.2.4.9 Connectivity Restoration Time The Connectivity Restoration Time is the difference between the Connectivity Restoration Instant and the Impairment Instant.
Metro Ethernet Network Resiliency and Traffic Management
503
14.1.2.4.10 SLS Restoration Time The SLS Restoration Time is the difference between the SLS Restoration Instant and the Impairment Instant. 14.1.2.4.11 Reversion (wait to restore (WTR)) Time In revertive mode, the Reversion Time is the difference between the repair instant of the original resource and the Reversion Instant. 14.1.2.4.12 Timing Relationships Figure 14-1, below, shows event instants and times, as defined above, on a timeline. Some times shown may actually be identical. Other times besides those defined above may be of interest.
Impairment
Fault Detection
Instant
Instant
HoldOff Instant
Connectivity Restoration Instant
SLS Restoration Instant -•
Time
-•^ Detection Time
Hold Off Time
Connectivity Restoration Time <
• SLS Restoration Time Figure 14-L Illustration of event timing
14.1.2.5
Other Terms
14.1.2.5.1 Shared-Risk Link Group (SRLG) A Shared Risk Link Group (SRLG) is a group of links that share a resource whose failure affects all the links in the group. For example, a bundle of fibers connecting two sites is an SRLG.
504
14.1.3 14.1.3.1
Chapter 14
Discussion of Terminology Timing Issues
Different applications and different users require different restoration times. In some provider network designs, a faster restoration time may require more network resources. For example, one technique of providing fast restoration is by creating one or more protection paths per working path that needs to be protected, including the provisioning of bandwidth for the protection path. For this reason, it is beneficial to define a number of different restoration times that can be provided for different resources. 14.1.3.2
SLS Commitments
Protection paths can either maintain or reduce bandwidth, and can alter other SLS characteristics. For example, a path that is assigned with a certain amount of CIR can be assigned with PIR instead when going through the protection path. In this way, the protection path requires fewer resources to be provisioned. In [2] and [3], two different types of protection times are mentioned, namely, recovery time and full restoration time. These correspond to the Connectivity Restoration Time and the SLS Restoration Time defined above. The differentiation in the terms is based on the SLS provided on the protection path, as well as the time it takes to provide the SLS provided by the original working path. In terms of providing various SLS levels on the protection path, it can be beneficial to provide a two-stage protection mechanism where the first-stage protection switching occurs (rapidly) onto the limited protection path, which is a protection path with reduced SLS commitment. The equivalent protection path, which is a protection path with full SLS commitment, can then be installed and the traffic switched from the first-stage protection path. Note that such full restoration time (SLS restoration time) may or may not be different from the recovery time (connectivity restoration time), depending on whether limited or equivalent protection path is used as the first-stage protection switching. The protection type 1+1 uses the resources in the protection path at all times for sending a replica of the traffic. The protection merge point decides which of the two copies to forward. On the other hand, 1:1, l:n, n:l, and m:n protection use only one path at a time, and therefore have the advantage that the protection-provisioned bandwidth can be used for other purposes when there is no failure.
Metro Ethernet Network Resiliency and Traffic Management
505
14.1.4 Protection Reference Model To deliver protection to Ethernet services implemented over Metro Ethernet Networks (MEN), a reference model has been created to allow description of a unified protection structure. The Protection Reference Model (PRM) allows a consistent description of protection capabilities applied on these services across various transmission technologies, network topologies and policies, thereby enabling the description of protection of services in the ETH layer (Ethernet Line Service, Ethernet LAN service, etc.). The following PRM section highlights the main functional elements of protection in a MEN. The elements are described in following sections through a "bottom-up" approach. The model shows a single layer in the architecture, which can be a transport layer or the ETH layer. The TRANS layer could consist of multiple layers, so there can actually be layering of transport networks, and protection can be provided at each of these layers (for example, Ethernet over MPLS over SONET). This is shown in Figure 14-2, where each layer may contain protection capabilities and may run above a lower layer that might contain protection capabilities as well. The entire protection scheme is controlled according to the application protection constraint policy.
New Protection Reference Model ETH layer
JEEPP
i iALHP
]\WnM>^
ilLAG
' I Protedion i ^ Pratedion llechanisiii " 1
1
]\ ' ;
Topology
1 TRANS layer : EEPP 1
:
'
i : ALNP
] \MPip
; i LAG
! i
* 1 rimiisciion MEF Piutedion Mechanisiii
r Topology
Figure 14-2. The PRM model (two layers are shown, from a stack of two or more)
506 14.1.4.1
Chapter 14 Transport
The purpose of the Transport Layer is to provide transfer of data between MEN elements. Many transports provide error-checking mechanism and data flow controls. The Transport Layer leverages any native mechanisms that the transport technology provides to gain protection from faults. The type of protection available may be local (e.g., a given node or link) or end-to-end (e.g., a set of nodes and links traversed by a "virtual" link) depending upon the technology used. The scenarios of the MEF protection architecture can be divided into two categories: 1. The service is carried natively over the transport layer and protection is achieved at the transport layer. An example is carrying Ethernet traffic in Ethernet over SONET (EoS), where the protection is done at the SONET layer. 2. The protection is done above a transport layer. Here there are two subtypes: • A transport layer is not capable of providing protection, or its protection capability is ignored by the protection mechanism of the upper layer. An example is Ethernet transport with the protection performed at the ETH layer or in an MPLS layer above it. • A transport layer and the protection mechanism of the upper layer are working in conjunction to bring the required protection SLS. An example is where the ETH layer containing a protection mechanism is implemented over an interconnected succession of SONET transport networks with 1+1 capability. The SONET 1+1 capability repairs local failures on certain links, while ETH layer protection is used where SONET protection is not applicable or as an additional end-toend protection method. The ability of protection mechanisms to be independent of the transport technologies allows metro networks to be deployed utilizing various transmission technologies, interconnected to create a heterogeneous network infrastructure. Protection mechanisms can span various transmission technologies (transports) regardless of whether each of these transports can deliver native protection capabilities. As each individual subnetwork of transport is utilized in a MEN, protection mechanisms could be requested from these transports to match an end-to-end protection SLS. If a transport does not have the ability to offer such services, then protection capabilities are performed at a higher or a lower layer, to ensure end-to-end protection SLS.
Metro Ethernet Network Resiliency and Traffic Management 14.1.4.2
507
Topology
Protection requires the topology to be such that it does not hinder an endto-end protection SLS. Depending on the specific technology, topology discovery may also be important to ensure that nodes (or a management utility) understand how to support the required protection. There can be many ways of delivering topology discovery. The topology is different at each layer of the MEN, since the internal topology of the lower layer is not visible to the upper layer and vice versa. The topology may look different when one looks at different layers of the network. At the ETH-layer, the network is built of Ethernet Connect Funtions (ECFs), interconnected by ETH-links. Different example topologies at the ETH layer are • ECFs on the edges only • ECFs at edges and in the core (e.g., grooming of Ethernet service frames for efficiency improvement, where EVCs are supported using multiple transport layers) Each TRAN-layer subnetwork over which the ETH layer is implemented has its own topology, built of Transport Connection Functions (TCFs) interconnected by TRAN-links. Protection can be provided in a specific layer if the topology at that layer contains enough redundancy. A service can be protected even if the topology at a specific layer does not provide enough redundancy, as long as the protection at other layers creates an end-to-end protection for the service at the ETH-layer. We discuss how the mechanisms described possibly apply to a few layers and technologies. For this reason, we use the terms links, nodes and Network Elements, where: • Network Element (NE, node) refers to a device containing an ECF or a TCF, depending on the layer. • Link refers to an ETH-link or a TRAN-link, depending on the layer. 14.1.4.3
MEF Protection Mechanism
The following styles of network protection mechanisms are currently under consideration: 1. Aggregated Line and Node Protection (ALNP) service 2. End-to-End Path Protection (EEPP) service 3. MP2MP protection service 4. Link Protection based on Link Aggregation
Chapter 14
508
The protection services can be layered one on top of the other in any combination. For example, the ALNP can protect the network facilities while EEPP provides an additional protection at the path level. EEPP supports 1+1, 1:1, and l:n protection mechanisms, and ALNP supports 1:1 as well as l:n facility protection. 14.1.4.3.1 Aggregated Line and Node Protection (ALNP) ALNP provides protection against local link and nodal failure by using local path detour mechanisms. In this case, local "backup" or "detour" paths are created along the primary path that bypass the immediate downstream network element NE or the logical link and immediately merge back on to the primary path. The detour path may provide l:n protection or 1:1 protection of the primary paths in the network. The backup paths are either explicitly provisioned, as described as an option in [1] or are implicit in the technology, as in SONET/SDH ULSR/BLSR.
LcKallletau'
isnE
"
^ ^ NetWMfc Element
Piinuy
ME
NE
•~n
(NE) W.r%«»1 Yk»tf««i«.
P^Ai2
«.
NE
Figure 14-3. ALNP
Protection with short restoration times is possible in many cases with ALNP because many failure events can be instantaneously detected without long feedback loops end-to-end. The restoration time actually depends on the
Metro Ethernet Network Resiliency and Traffic Management
509
local failure detection time, which may differ in different scenarios. As each failure is detected (at each link or node), ALNP protects many end-to-end paths (with similar end-to-end protection SLSs) in a single restoration invocation. If a lower layer transport subnetwork has the ability to deliver services that are similar to those that ALNP provides at an upper layer, then the native protection mechanism of the transport subnetwork can be used and ALNP can be bypassed. If a transport subnetwork in a layer below the layer at which ALNP operates does not support native protection capabilities to support a specified SLS, then it is the responsibility of the Aggregated Line and Node Protection (ALNP) mechanism to deliver the protection required according to the specified SLS. Bidirectional Line Switching Redundancy (BLSR) capabilities of SONET and SDH, and MPLS local repair are examples of ALNP derivatives in specific transports. ALNP may deliver a 1 :n protection capability with a sub-50-ms restoration time and other default parameters. (Other restoration times could also be supported and invoked, depending on the protection SLS specified and on failure detection capabilities.) The protection ALNP should deliver will be dependent on the protection desired for the service or services it protects. ALNP provides the ability to aggregate many end-to-end paths in a hop-by-hop and node-by-node manner. At any time, both the ALNP and other protection mechanisms in transport layers below the layer at which ALNP executes could offer similar protection capabilities. Interoperability is achieved in this case by configuration of the hold-off time of the ALNP mechanism such that the lower layer protection mechanism converges before the ALNP mechanism at the upper layer decides whether to take action. To protect each link and node using ALNP, the mechanism for generating ALNP protection paths is preferably done using an automated scheme or is implicit in the transport technology. A possible mechanism for automatic creation of protection paths is to allow the specification of the protection parameters desired as part of the trail creation. Upon detection of the first request (for a given protection SLS for a specific trail), or earlier (e.g., at network setup), protection paths with certain protection parameters are created for each given transport subnetwork. 14.1.4.3.2 End-to-End Path Protection (EEPP) End-to-end path protection (EEPP) is the ability to provide a redundant end-to-end path for the primary path. This mechanism can be used to augment ALNP. A variation of this method can be used to protect partial segments of the end-to-end path within the same layer if such capability is supported by the protection mechanism at the specific layer.
Chapter 14
510
lAMa] D e t o v
1NE
Netwoik Elenoeiit (NE) \
Piuiuy
^
NE
^ ^ ^ • " ^
NE
•^ r 4
I.ni-«l IWinMVi ^^***'*''*»*ta
P^A2
\
ME
1S[E
-
/
^
\
/ . . . . ^
5S^
Mi^
Figure 14-4. E]EPP
Figure 14-4 illustrates the use of a secondary path for EEPP as well as detour paths for ALNP. In an EEPP scenario, a path is created from a source node to a destination node. Alternative or secondary paths are then created with different routing segments that protect the primary path. The number of redundant paths needed is policy defined and has implementation limits (each of these redundant paths could consume network resources such as bandwidth, CPU cycles, and memory). The intelligence for computing redundant paths (that are not part of the primary path resources) can be done with an online constraint mechanism (e.g., CSPF) or offline traffic engineering tools. Each of these redundant paths is end-to-end in nature and therefore provides redundancy in a different manner than ALNP. The EEPP handles protection on segments of the global path, and in some cases could be provisioned endto-end, and can provide redundancy when a transport segment along the path cannot provide protection of any kind (this includes ALNP or native transport protection). The EEPP could also be used when ALNP protection is available at each transport subnetwork but further redundancy is desired for path diversification. The restoration time of EEPP can be much longer
Metro Ethernet Network Resiliency and Traffic Management
511
than for ALNP and is dependent on the protection type that is used. There are a few types of protection that can be used: • l-fl— This type of configuration requires that the redundant paths are fully established and active. All data sent on the primary path is also copied to the redundant path(s). The receiver (which is the node at which the two paths merge) coordinates and decides which of all the available paths (primary and secondary) is used at each point in time. This decision can be performed on a per-path basis according to OA&M, for example, or on a per-packet basis, where each packet is taken from a path from which it was received, in which case a sequence-number field can be added to the packets so that the receiver can correlate between the two packet streams. This type of redundancy can achieve very fast restoration times (milliseconds), since the receiver decides if the primary path has failed based on alarm indication or performance information derived from the primary path. However, this type of redundancy will consume double the bandwidth and hardware resources (CPU, fabric, memory, etc.) since it is always active and passing data. • 1:1 Cold Standby — This type of configuration requires that the redundant paths have their routing information calculated ahead of time, but the redundant paths are not established until failure of the primary path; the source node establishes the redundant path only when failure has occurred on the primary path, resulting in long restoration times. • 1:1 Hot Standby — This type of configuration requires that the redundant paths have their routing information calculated ahead of time and established during the service activation time of the primary path: the redundant path(s) are kept active waiting for the primary path to fail. The chief determination of the time to repair a failure is the detection time, since the switchover to any redundant path(s) can occur very quickly. The drawback to this type of redundancy is that the redundant paths consume network resources even though they are not passing data. Based on protection policy, however, the setup of the redundant paths may be made with fewer resources in order to give fast restoration for part of the traffic immediately. Cold standby can be invoked later onto restore full traffic BW. • Shared Redundancy — Since a single failure in the network may only affect a subset of the primary paths, there is an opportunity to share the same protection resource among multiple primary paths. There are many schemes that achieve sharing of the protection resources by exploiting this fact: 1 :n, ring, and shared mesh protection are some of the wellknown sharing mechanisms.
Chapter 14
512
14.1.4.3.3 MP2MP protection The E-LAN service is a multipoint-to-multipoint service that requires connectivity between all its UNIs. Depending on the implementation of the E-LAN service, the protection schemes above may not be sufficient for protecting it. The reason is that the implementation of an E-LAN may be involved with one or more ECFs, which are interconnected by a number of ETH-trails. A failure of such an ECF is not covered by EEPP and ALNP as described above. The implementation of an E-LAN service may include implementation of multipoint-to-multipoint connectivity at the TRANS-layer as well. Three methods are typically used for multipoint-to-multipoint protection of Ethernet service or transport: • Split horizon bridging with full mesh connectivity • Spanning Tree or Rapid Spanning Tree • Link redundancy With Split-Horizon bridging, a full mesh topology is created between the TTF (Trail Termination Function) entities (each is an ECF or an TCF, depending on the layer under discussion), creating the protected domain. Each trail in the full mesh of trails is a point-to-point trail and may contain nodes (ECFs or TCFs) in the same layer or in a lower layer.
Access Inlcs
Access
Figure 14-5. Split-horizon bridging with full mesh connectivity
Metro Ethernet Network Resiliency and Traffic Management
513
Split-Horizon bridging is performed as follows: Each TTF maintains a bridging database controlling its bridging function. Each frame received by one of the TTF entities from an access-link is forwarded according to the bridging database of that TTF, to one, all, or some of the other TTF entities. Each copy is transmitted to one of the remote TTF entities through the direct trail leading to that remote TTF. Frames received by a TTF from one of the trails of the full mesh are forwarded by the TTF only to access links. With split-horizon bridging, the protection techniques discussed above are sufficient for protection of the MP2MP service, as long as each of the trails connecting the TTF entities is protected. A split-horizon-bridging subnet can serve as a subset of a larger bridged network by connecting it to other bridging components. In this case, its bridging elements may not be TTF entities, but ordinary ECF/TCF entities with split-horizon bridging capabilities. The Spanning Tree Protocol is defined in IEEE 802.ID; the Rapid STP is defined in IEEE 802.Iw. These protocols provide protection in a network in which the TTF entities are connected in a partial mesh, and each of the TTF entities performs 802.ID compliant bridging between the links and trails connected to it (access links as well as trails of its own layer). ECF/TCF entities through which the trails between the TTF entities pass may also perform 802.ID bridging. Observe that 802.ID bridging requires all links between the bridging entities to be bidirectional; therefore, this scheme requires all trails between ECF/TCF entities that perform bridging to be bidirectional. IEEE 802.ID requires the bridging to be performed over a subset of the network that forms a spanning tree of that network, and here is where STP and RSTP come to help, creating a spanning tree of trails that participate in the bridged network, which spans all TTF and ECF/TCF entities implementing the service. STP requires fast aging or reset of the bridging databases in case of a change in the topology of the created spanning tree. As described in MEF services, STP-BPDUs may be • Processed at the UNI, in which case the subscriber network becomes part of the network for which a single STP is calculated • Tunneled by the service, in which case, the service is perceived by the subscriber network as a single segment. In this case, a subscriber STP can be created between its sites • Dropped at the UNI, in which case the subscriber should manually ensure that his or her network does not contain loops going through the service. Note that tunneling and discarding also mean that an internal (MEN) STP can be created that is separated from the subscriber STP. In case tunneling is performed, the subscriber STP is then transparently tunneled through the MEN.
Chapter 14
514
TCF
ECF/ TCF
' "'<,;,''' f
TTF
Figure 14-6. Link redundancy
With the Hnk-redundancy scheme, a single TTF attaches a bridged network of ECF or TCF entities (depending on the layer), using two pointto-point trails, which do not necessarily end at the same ECF/TCF on their other side. The TTF under discussion chooses at any time a single operational trail to work with. The TTF uses one of the mechanisms available in the technology of the specific layer (e.g., OA&M) for monitoring the operational status of the two trails. The TTF forwards frames received from its access link to the chosen trail, and processes or forwards frames received from this trail to its access links. Frames received from the other trail are dropped. When the TTF decides to change the trail, which is used for forwarding, it should inform the bridged network to which it is attached that a topology change has happened so that the bridging entities in it can initiate fast aging or flush their database. 14.1.4.4
Link Protection Based on Link Aggregation
For an Ethernet transport, Link Aggregation allows one or more Ethernet links connecting the same two nodes to be aggregated into a Link Aggregation Group (LAG). A LAG logically behaves as a single link. The
Metro Ethernet Network Resiliency and Traffic Management
515
frames that each of the two nodes transmits through the LAG are distributed between the parallel links according to the decision of that node. The LAG distribution function should be such that it maintains the order of frames within each session using it. A LAG may be made of N parallel instances of full duplex point-to-point links operating at the same data rate. The Link Aggregation Control Protocol (LACP) is defined in IEEE 802.3, and is used by neighboring devices to agree on adding links to a Link Aggregation Group. Figure 14-7 is an example of a LAG topology:
Ettiemet Links
ECF/TCF
ECF/TCF
Figure 14-7. Link aggregation
One of the features of a LAG setup is protection against failures of each of the links composing the LAG. When one of these links fails, the nodes at the two sides of the LAG simply change their LAG distribution function to spread the traffic between the remaining links. When a link recovers, the LAG distribution function changes again to include it. An optional LAmarker protocol ensures that frame order is preserved during these changes. The LAG scheme can be applied at the ETH-layer as well, by creating a number of point-to-point ETH-trails between two ECFs. The LACP can serve in this case as the OA&M procedure detecting the failure and recovery of the ETH-trails so that the LAG distribution function can be adapted accordingly.
516 14.1.4.5
Chapter 14 Application Protection Constraint Policy (APCP)
Functionally, APCP mediates between the subscriber and the MEN. It facilitates all the functions that are necessary to interpret service requests from MEN subscribers and to trigger services provisioning actions in the MEF transport network. Thus, a service request through either UNI or a management interface first comes to the APCP, where the type of service and its parameters get interpreted. Then, on behalf of the subscriber request, the APCP will in turn request (if validated) the MEN to provide the requested transport with desired capabilities/characteristics.
14.1.5 Requirements for Ethernet Services protection mechanisms The following are the requirements for the Metro Ethernet protection schemes. Every protection mechanism in the MEF protection framework is to be evaluated based on these requirements. The requirements are generic in the sense that they apply to all services supported by Metro Ethernet Networks (E-Line, E-LAN, CES). 14.1.5.1
Service-Related Requirements
14.1.5.1.1 Service Configuration Requirements R-1. It MUST he possible for a subscriber to request different protection parameters for Ethernet services. The requested parameters SHOULD include the connectivity restoration time and SLS restoration time. R-2. An EVC of an Ethernet service with SLS that requires protection MUST he protected along all ETH-trails from which it is composed. R-3. Protection parameters MUST be defined on the level of per-service or a group of services. End-to-end service protection MAY be implemented utilizing multiple mechanisms along the flow path. 14.1.5.1.2 Restoration Time Categories Various applications require different connectivity and/or SLS restoration time. The restoration time (connectivity and/or SLS) that is required for a specific service is dependent on the needs of the application that the user plans to execute over that service. R-4. It SHOULD be possible to request a connectivity and/or SLS restoration time of the network for each service. The following SHOULD be the restoration-time categories that the services can choose from for
Metro Ethernet Network Resiliency and Traffic Management
• • • •
517
identifying the required connectivity restoration time and SLS restoration time: Sub-50 ms restoration time Sub-200 ms restoration time Sub-2 seconds restoration time Sub-5 seconds restoration time. Typical examples may be as follows: 1. CES applications require sub-50 ms SLS restoration time, 2. an SLS restoration time of sub-200 ms is sufficient for certain CES applications, 3. some real-time or semi-real-time application may also require also sub-200 ms SLS restoration time, 4. sub-2 seconds connectivity restoration time ensures that a LAG implementation over the service using LACP in fast mode does not reconfigure due to the failure, 5. TCP-based applications usually settle for sub-5 seconds connectivity restoration time, and 6. a connectivity restoration time of sub-5 seconds ensures that STP and RSTP do not start reconfiguring the network. These examples do not serve to categorize applications according to the restoration time, but only serve as examples for providing motivation.
14.1.5.2
Network-Related Requirements
14.1.5.2.1 Protected failures R-5. The protection mechanisms SHOULD be able to protect from any one of the failure types, for example: 1. Fail condition (Hard-link failure, e.g. LOS, LOF, etc.) 2. Degrade condition (Soft-link Failure, e.g., BER, CRC errors greater than a threshold) 3. Node failure R-6. The protection mechanisms M4 7 protect from misconfigurations. Protection switching MUST not cause misconnections as a side effect of its own operation. 14.1.5.2.2 Degrade Condition Threshold Degrade condition is a status of a resource in which traffic transfer might be continuing, but certain measured errors (e.g., packet loss, bit error rate, etc.) have reached a predetermined threshold. R-7. With a protection scheme in which the SLS is preserved during failures, the predetermined threshold MUST be less than the maximum amount of packet loss allowed according to the SLS of the services flowing
518
Chapter 14
through the resource. The provider M^Fset a less restrictive threshold on the resources if the SLS of all services flowing through the resource allows that. One way for the network provider to meet the above requirement is to ensure that the packet-loss commitment in SLS of the services is limited according to the expected amount of packet loss allowed in the resources of the network. It should be noted that, depending on the protection scheme and layer, other measurements than packet loss may be used and a relation to packet loss might not be possible. After the protection trigger (threshold crossing), it will take time (protection switching time) before the traffic is restored. During this time, additional packet loss will occur. 14.1.5.2.3 Transport Layer Protection Mechanisms Interaction The MEN layering model allows layering within the TRANS layer. Therefore when one layer executes above another layer, both may belong to TRANS, or the upper one may be the ETH-layer and the lower one may be a TRANS-layer. R-8. An upper layer protection mechanism SHOULD be designed to work in conjunction with lower layer transport protection mechanisms, such as SONET 1+1/1:1, RPR, etc., as available. Each protection mechanism that is allowed to execute in a network including these lower layers MUST support configuration of the hold-off time such that the lower layer protection mechanism converges before the protection mechanism at the upper layer decides whether to take action. Note that the protection at the lowest layer doesn't need to support a hold-off timer. In some cases, it MAY be required that the various layers will act independently. This depends on the protection policy for the network. 14.1.5.2.4 Protection Control Requirements The following is a list of parameters and controls relevant for protection schemes. 1. Hold-Off Time 2. Revertive/nonrevertive mode 3. Reversion (Wait To Restore) Time 4. Manual switch 5. Forced switch 6. Lockout Items (1), (2), and (3) above are configurations of protection parameters, while (4), (5), and (6) are actually control mechanisms that control the current protection operation mode. The requirement for support of a hold off timer has been covered already in the above subsection.
Metro Ethernet Network Resiliency and Traffic Management
519
R-9. The protection at each (ETH/TRANS) layer MAY enable configuration of the revertive/nonrevertive mode and reversion (Wait To Restore) time. These parameters and controls MAY he applied according to the protection poHcy, Protection MAY be applied in different layers of the MEN, and revertive/nonrevertive mode and reversion (Wait To Restore) time SHOULD be applied to each mechanism separately. R-10. The protection at each (ETH/TRANS) layer SHOULD enable configuration of at least one of the following controls: • Manual switch • Forced switch • Lockout Protection MA Y be applied in different layers of the MEN, and the above three controls SHOULD be applied at each layer separately. 14.1.5.2.5 Bidirectional Switching There are two variants of the m:n protection type. In the first, a protection resource can be used concurrently for protecting a number of working resources, in case a few of them fail at the same time. In he second, a protection resource is able to pass the traffic of a single working resource at a time. R-11. In the case of an m:n protection scheme, in which the protection resource is able to pass the traffic of a single working resource at a time, and if the working and protection resources pass traffic in two directions, the bidirectional switching mechanism MUST be used for controlling the use of the protection resource. The motivation is that the bidirectional switching mechanism ensures that both directions of the same working resource switch concurrently to the protection resource. This outcome ensures that the network does not get to a situation in which one direction of one working resource and another direction of another working resource are protected by the protection resource, so no single working resource is completely protected. An example of a mechanism that achieves this result is the APS mechanism of SONET/SDH. 14.1.5.2.6 Robustness R-12. Each protection mechanism MUST monitor protection standby resources for failures. 14.1.5.2.7 Backward Compatibility Requirements R-13. When upgrading nodes in the network to support new protection mechanisms, these nodes MUST interoperate with nodes on the same network that are not yet upgraded to include these capabilities or schemes.
520
Chapter 14
14.1.5.2.8 Network Topology R-14. A Metro Ethernet Network may consist of different subnetwork topologies. The protection framework SHOULD support these different topologies, although a specific scheme may be limited to few topologies. R-15. The protection scheme SHOULD provide resource diversity such that the working and protection paths do not share a common resource in the network. The protection scheme SHOULD allow the required level of diversity to be according to the operator policy. The policy A//( 7 require link diversity, node diversity, station diversity, fiber diversity, cable diversity, duct diversity, and geographical diversity. The operator SHOULD be able to control the required diversity controlling either by the network topology or the protection scheme and provisioning parameters. R-16. To provide protection for a specific EVC, a protection scheme MUST protect each of the network resources within the network topology through which traffic of that EVC flows. 14.1.5.2.9 QoS In many cases, a trade-off exists between efficient use of network resources and the extent of QoS preservation. The following requirement is helpful for operators for controlling this trade-off. R-17. Each protection method SHOULD enable the operator to define to what extent the original QoS is kept, up to full equivalent behavior. 14.1.5.2.10 Effect on User Traffic R-18. The protection mechanism SHOULD maintain the SLS requirements for loss of traffic and disordering during switching to protection and restoration events. 14.1.5.2.11 Management Requirements for Protection Schemes R-19. A facility implementing a protection scheme SHOULD support a management interface that will expose to management applications the following parameters: • All control parameters defined in previous sections • Status of the working and protection paths • Signaling of events that represent changes of status of the working and protection resources • A failure event, which causes protection switch, SHALL cause an alarm to be sent by the involved Network Element(s) to the Element Management System.
Metro Ethernet Network Resiliency and Traffic Management
14.1.6
Framework for Protection in the Metro Ethernet
14.1.6.1
Introduction
521
The key for the framework is the choice of protection mechanisms. Other aspects are important, but are handled elsewhere: • The protection framework supports arbitrary transports. • Protection solutions at each layer SHOULD be independent of the internal topology of the underlying layer. • The APCP allows a subscriber to specify the parameters of the protection desired. This can be done through management-based configuration or the UNI. Thus, our framework discussion focuses on the protection mechanisms and the failure detection mechanisms described above. The model that has been developed has that property that it is open to new approaches and innovation within the layers. The description here reflects current discussion and can be expanded in the future. 14.1.6.2 MEF Protection Schemes In this section, we outline some implementation approaches to providing MEF protection. Many protection mechanisms may be operating in the network at the same time. One reason is the existence of a variety of services requiring different parameters of protection. Some of these protection mechanisms may offer protection at different layers of the network, whereas others may offer different protection parameters such as restoration time, failure coverage, etc., at the same layer. However, operating many protection mechanisms at the same time in the network requires a coherent interworking strategy so that a triggering of multiple protection mechanisms for protecting the same traffic can be treated appropriately. Note that the use of multiple protection mechanisms for the same traffic may be desired because, for example, different protection mechanisms in the network may provide different failure coverage and restoration time. To ensure coherent operation, the protection requirement section states that each protection mechanism that is allowed to execute in a network including these lower layers MUST support configuration of the hold-off time such that the lower layer protection mechanism converges before the protection mechanism at the upper layer decides whether to take action.
522
Chapter 14
14.1.6.2.1 OAi&M-Based End-to-End Path Protection (EEPP) This end-to-end path protection mechanism requires at least two paths to be created from the source node to the destination node for providing the same service. One of these paths is regarded as the primary path, and the others are redundant paths. The redundant paths are provisioned such that they are disjoint in nodes, hnks, and shared-risk Hnks. In this way, a faikire of a single link or node will disconnect only one of the redundant paths at the most, so another can still be used and the service is maintained. This method can also be used to protect segments of the end-to-end path. Possible modes of EEPP protection are the 1:1 mode, in which two redundant paths are provisioned, but only one is used at a time; the n:l mode, in which n+1 paths are provisioned, but only one is used at a time; the 1 :n mode, in which a single protection path is used for protecting n disjoined working paths; and the 1+1 protection mode, in which two paths are used concurrently —each data packet is sent along both paths and the sink node decides which to use. An end-to-end OA&M protocol is used for sensing the availability of a path. With 1+1 protection, it is sufficient to have a one-way OA&M protocol. With 1:1, l:n, or m:n protection types, the source node is notified of the failure of the source-to-destination path. This means that the OA&M protocol SHOULD be a two-way protocol. An example for an OA&M protocol for MPLS can be found in ITU SGI3. Specific transports, like SONET/SDH, ATM, etc. have their own OA&M mechanisms that can be utilized for EEPP at the specific layer. Other OA&M mechanisms may also be used for the same purpose. Another variation of this scheme is when the end-to-end connectivity information is provided by out-of-band connections to a management station. 14.1.6.2.2 Aggregated Line and Node Protection (ALNP) The ALNP scheme uses a set of local-protection tunnels for protecting each link and each node in the network. In this way, an end-to-end SLS of fast (e.g., sub-50 ms) protection is accomplished. The ALNP scheme can be applied to networks of any topology, provided that the topology provides a redundant path for each of the network resources. ALNP is built to interoperate with any kind of link between the nodes implementing it. This link can be a point-to-point physical link, a protected transport (in which case protection-tunnels are required only for border nodes), a fast protected subnet (again, only border nodes require ALNP protection), and virtual links (in which case an OA&M procedure is required for indicating the state of the virtual link and of its end nodes).
Metro Ethernet Network Resiliency and Traffic Management
523
14.1.6.2.3 Packet 1+1 End-to-End Path Protection Packet 1+1 provides high-reliability, hitless end-to-end path protection. Packets from and application flow receiving packet 1+1 service are dual-fed at the source side onto two disjoint paths. In the simplest case, these paths can be node and link disjoint but in general may involve more complicated notions such as shared risk link groups. On the destination side, one of the copies of the packet is selected from the two possible received copies. Note that the incoming packet is selected from any of the two disjoint paths, irrespective of the path from which the last packet was selected. Thus packet 1+1 treats both paths as working. This is different from the traditional transport 1+1 scheme, where each path is designated as working or protection, and the packets are selected from the working until a detection of failure on the working causes a switching to the protection path. Thus, compared with traditional transport 1+1, packet 1+1 does not require explicit failure detection and protection switching. It also does not require any signaling. This setup allows the packet 1+1 service to recover from any failure that only affects one of the two disjoint paths providing the service instantaneously and transparently. Note that these failures are protected irrespective of the layer, including physical, link, TRANS, and ETH layers, at which the failure occurred, as long as the failure is in the layer at which the mechanism is provided or in a layer below it. In other words, when the mechanism is provided at a specific layer, failures at that layer and in the layers below it are covered. Preserving sequentiality requires defining a new protocol to include sequence numbers. Packet 1+1 requires that the source side assign the same but distinct identification to each dual-fed packet. This can be easily achieved by, e.g., assigning sequence numbers to packets. Each pair of duplicated packets will get the same sequence number but one distinct from the other pairs of duplicate packets. Based on the carried sequence numbers, the destination node is able to identify duplicate packets and select one of them. Note also that if a resequencing is not also provided, unerrored packets may be discarded to preserve sequentially. 14.1.6.2.4 Shared Mesh Protection End-to-end shared protection scheme is targeted to provide guaranteed restoration while using a minimal amount of protection bandwidth in a general mesh topology. Other end-to-end protection schemes either require a dedicated protection path for each primary path, such as 1+1 and 1:1, or provide a limited sharing of protection bandwidth, such as 1 :N. Compared with them, shared protection scheme provides very flexible sharing. Instead of dedicating protection resources for every primary path in the network, a pool of protection resources can be set aside that can then be used for
524
Chapter 14
restoring the primary paths that get affected by the failure. Protection resources are allocated to allow restoration of all the protected traffic from any single possible failure in the network. For each protected primary path, the protection resources are allocated at the time of activation of the primary path. Arrival of a request to establish shared mesh restoration service between two nodes prompts computation of a pair of disjoint paths between them with two necessary constraints. First, sufficient bandwidth is allocated along the route of the primary path to accommodate the requesting traffic. Second, either already reserved protection bandwidth along the protection path is sufficient to guarantee restoration from any single failure along the primary route, or the available bandwidth along the protection path is allocated to be enough to accommodate the additional bandwidth needed for protecting the new primary path. Note that sharing is achieved by always first trying to accommodate a new request with already allocated protection resources. This can be achieved by keeping track, for each link in the network, of the amount of resources that will be required to protect all the primary paths that will be switched to it after any single failure in the network. This information can be either maintained in a centralized fashion at a server or distributed to the nodes in the network. In the case where the information is distributed on the nodes, each node only needs to keep track of the amount of resources required on each of its incident links. Note that since it is well accepted and verified that the probability of multiple concurrent failures in most networks is small, the scheme has been described to protect from any single failure in the network. Protection from multiple failures can be achieved through a straightforward extension.
14.2.
METRO ETHERNET TRAFFIC AND PERFORMANCE MANAGEMENT
14.2.1 Ethernet Traffic Management Overview Ethernet traffic management offers the essential components with which a service provider can offer a customer a range of services that are differentiated based on some performance metrics, e.g., frame loss and frame delay. Customers request a specific performance level as stated in the SLS. Service providers achieve levels of performance assurances by employing the necessary traffic management mechanisms both at the edge and inside the metro Ethernet network (MEN). Figure 14-8 shows the interaction between the customer network and the provider network across the User-Network Interface (UNI) between the CE
Metro Ethernet Network Resiliency and Traffic Management
525
(Customer Edge) and the PE (Provider Edge). Inside the customer network, a customer may implement IEEE 802. IQ VLANs and traffic management mechanisms. Those mechanisms are beyond the scope of this specification. Across the UNI, CE-VLAN ID (Customer Edge VLAN ID) is mapped to a single EVC. The EVC or a combination of an EVC and CE-VLAN-CoS is used by the PE determine the Class of Service (CoS) instance using the CoS Identifier.
MEN
Customer
CE-VLAN D
Figure 14-8. Ethernet traffic management overview
The CoS Identifier is used to identify the CoS instance applicable to a flow of Service Frames. A maximum of 32K (4095*8) CoS Identifiers MAY be supported at a UNI. A much smaller number of Classes of Services MAY be supported. The CoS Identifier can be viewed as an index that points to two fundamental building blocks that together define the service level received by a Service Frame. Those building blocks are the Bandwidth Profile and the forwarding class. The Bandwidth Profile, if present, is a set of traffic parameters that governs the expected arrival pattern of customer traffic per CoS instance. It provides a deterministic upper bound to the expected volume of traffic per CoS instance. The Bandwidth Profile parameters are inputs to a metering
526
Chapter 14
algorithm that verifies the conformance of incoming Service Frames per CoS instance. This allows network providers to engineer network resources to satisfy service performance requirements. Multiple Bandwidth Profiles may exist at the same UNI. The Bandwidth Profile, if present, SHOULD be enforced by the provider's network, since it is part of the SLS and is agreed upon between the customer and the service provider. QoS policy rules, by way of rate enforcement algorithms, are typically used to identify the specific actions taken by a service provider when Service Frames violate a Bandwidth Profile. The actions may be to drop or recolor (or reprioritize) the service frame to indicate high discard precedence within the MEN. The outcome of any such rate enforcement within the service provider network is a set of Service Frames that are labeled Green, Yellow, or Red based on their level of conformance to Bandwidth Profiles. The forwarding class defines the treatment inside the provider network received by a Service Frame belonging to a particular CoS instance. For example. Service Frames may be forwarded at the highest priority available at the nodal queues to assure some level of delay requirement. Forwarding class is outside the scope of this specification. For the details of traffic and performance management, please refer to Chapter 10 "Ethernet Services over Metro Ethernet Networks".
ACKNOWLEDGMENTS Metro Ethernet Forum Lior Shabtay
14.3. REFERENCES [1] Metro Ethernet Forum, Technical Specification MEF 2, Requirements and Framework of Ethernet Service Protection in Metro Ethernet Networks, February, 2004. [2] Sharma, V. and Hellstrand, F., Framework for MPLS-based Recovery, RFC 3469. [3] ITU-T Rec. G.841, Types and characteristics of SDH network protection architectures, October. 1998.
Chapter 15 SONET SERVICES FOR STORAGE AREA NETWORKS
Richard Orgias Nortel
15.1.
DATA GROWTH
There are few trends as dramatic or universal as the growth of data. The rate of creation of digital information is unprecedented with estimates of its annual growth pegged at double digit rates. Every year brings the addition of more digital information to the already considerable store that constitutes the sum total of human knowledge. One study from the University of California at Berkeley [1] estimates that by 2005, worldwide production of digital data will have grown to almost 100 exabytes (10^^ bytes). As more and more organizations adapt to the reality of 24/7 operation, the creation of data accelerates and the need to manage and make it useful grows in importance. Making sense of the vast store of information requires that it be organized. Organization requires that it be cataloged and stored. In order for data to have any meaningful value, the ability to retrieve it as required is also important. In the context of an organization, a big part of data management is simply to provide mechanisms for storing, duplicating, and providing ready-access to information. Of course, the type of information varies with the organization. In the case of a financial institution, it could be records associated with financial transactions. For a healthcare organization, it could be an x-ray or some other type of patient record. For an entertainment company, it might be a film clip or a music file. The common challenge faced by these disparate organizations is the requirement to support storage needs estimated to be
528
Chapter 15
doubling every 18 to 24 months according to research conducted by Gartner, an industry research and consulting firm. The data storage needs of an enterprise can be met in a number of ways. Among the media types used for storage of digital information are magnetic tape, optical disks, hard drives, and more recently solid-state disk drives. Once the data has been written to a medium, it can be duplicated and distributed so that a copy is available for use by multiple parties or used to replace data that is rendered inaccessible or unusable. Making copies of data is one means of ensuring that data loss does not expose an organization to financial loss or preclude it from meeting a requirement for data availability. The emergence of storage arrays, a group of disks that work as a unit, has facilitated the mass storage, protection, and distribution of data by making it possible to store and create copies of digital information on disk that can be accessed and modified in near real time.
15.2.
STORAGE NETWORKING
The drive to maximize computing resource utilization has resulted in a model in which processing power is separated from storage. This setup is typified by the server-based architecture present in most enterprise environments where applications are run from servers with data storage handled on separate dedicated storage devices. This trend continues to be reflected in the move away from direct attached storage (DAS) and toward a networked storage paradigm where storage resources are shared among multiple users. Networking can minimize the suboptimal use of storage resources by making it possible to share storage among multiple servers or end-user devices. Networking also enhances performance and eliminates bottlenecks by • Simplifying management, thereby eliminating the need for backup and maintenance of individual volumes • Enhancing availability by eliminating the need for an attached CPU device to support tasks such as backup and defragmentation • Simplifying recovery procedures by eliminating the need to restore both applications and data in the event of a failure A simple example of the benefits of a network can be illustrated by considering the case where the members of a small workgroup utilize the same application. In the scenario without a network (i.e., direct attached storage or DAS), the application and all associated files would be maintained on the individual computers of each member of the work group. Sharing of files as well as backup and maintenance would have to be handled at an
SONET Services for Storage A rea Networks
529
individual level. The result is the potential for variation in the version of the application being used, as well as variability in the frequency of backups. The consistency of backups will be determined by how diligent each individual is at maintaining his or her workstation. If a network is put in place that allows sharing of the application and associated files with access to both via a server, the ability to share information is enhanced and management of the application and associated files can be centralized. With a networked architecture, a single individual may act as the administrator of the network, with responsibility for keeping everything up to date. The administrator can ensure that applications are updated, backups are performed, and that unwanted our outdated files are purged, all by managing the common server and associated storage. This approach allows for more efficient management of data files and ensures consistency in the information being used by all members of the workgroup. Networking can also address the loss of processing efficiency that results from having to use CPU cycles to support backup and defragmentation on multiple devices. If data is stored on a shared medium, with access enabled by a network infrastructure, additional storage can be more efficiently added to the central pool, and maintenance activities can be performed on a single device instead of on multiple devices. In short, networking storage delivers quantifiable operational and financial benefits to an organization by reducing the complexity of managing a pool of storage assets. The difference between direct attached storage and network attached storage is illustrated in Figure 15-1. There are two distinct storage networking approaches. Network Attached Storage or NAS usually consists of a hard disk or multidisk RAID (redundant array of independent disks) systems connected to users and servers via a Local Area Network (LAN) infrastructure. An NAS allows sharing of applications and files by multiple users who are able to access a storage device via an IP address on a LAN. By contrast, the other storage networking option, a Storage Area Network or SAN, is a special-purpose network that supports the interconnection of data storage devices and servers and uses specialized protocols to optimize block-level data transmission. Some of the protocols used by storage networks include ESCON, Fibre Channel, FICON, SCSI, iSCSI, and Gigabit Ethernet. • ESCON [2] or Enterprise Systems Connection is a protocol developed by IBM in the early 1990s to interconnect its mainframe computers to each other and to attached storage. ESCON operates over fiber optic cable.
530
Chapter 15
Clients
Q
Clients
0 Storage
Direct-Attached Storage (DAS)
/Servers
Network Attached Storage Appliance
Shared Storage
Network-Attached Storage (NAS)
Figure 15-1. Direct Attached and Network Attached Storage
•
Fibre Channel [3] is an ANSI-specified protocol that allows the connection of servers and storage devices as well as the interconnection of storage controllers and drives. Fibre Channel operates over fiber optic and coaxial cable as well as ordinary twisted pair cables • FICON [4] or Fiber Connectivity is an IBM-developed high-speed input/output (I/O) interface for mainframe computer connections to storage devices. FICON offers higher physical link rates than ESCON, the previous standard that it replaces. • SCSI [5] or small computer systems interface is a set of ANSI-specified interfaces originally developed by Apple computer that allow personal computers to communicate with peripheral hardware such as disk drives, tape drives, CD-ROM drives, printers, and scanners. • iSCSI [6] or Internet Small Computer System Interface is an IETF developed protocol that allows SCSI commands to be carried over IP networks and can be used to link data storage facilities and to transmit data over local area and wide area networks (LANs and WANs). • Gigabit Ethernet [6] is an IEEE standard that supports transport of Ethernet frames over an optical medium at data rates of one billion bits per second. Gigabit Ethernet is typically used to build out LAN backbones and is compatible with lower-rate Ethernet transport. Table 15-1 highlights some of the key applications, data rates, and the possible reach with the protocols described above.
SONET Services for Storage Area Networks
531
Table 15-1. Storage Protocols and Applications
§m^^^MMM, ESCON Fibre Channel (FC)
200 Mbps 1 Gbps, 2 Gbps, 4Gbps, 10 Gbps
IGbps
FICON
SCSI
Up to 160 Mbytes/s
iSCSi
Varies
Gigabit Ethernet
IGbps, 10 Gbps
IBM Mainframe to peripheral connectivity Server-to-storage, server-to-server connectivity IBM mainframe to peripheral connectivity, evolution of ESCON, Interworks with FC Computer device interconnection, RAID interfacing IP-based SCSI variant for server-to-server connectivity, RAID interfacing Native packet transport
60 Km 50 Km
50 Km
Data center Data center 50 Km
In most instances, the nature of the applications will determine v^hether a SAN or a NAS solution is implemented by an enterprise. Many enterprises have both solutions.
15.3.
STORAGE AREA NETWORKS
SANs currently dominate as the netw^orking choice for mission-critical storage applications, since they are switched-fabric networks characterized by high-level connectivity coupled with fast and reliable access. SANs tend to be deployed in support of database-intensive applications such as transaction processing, ERP (Enterprise Resource Planning), and CRM (Customer Relationship Management) applications. ERP and CRM applications are critical elements of the IT infrastructure of many enterprises, with ERP facilitating the internal management of a business (maintaining inventories, tracking orders, purchasing, etc.) while CRM supports management of customer relationships. Both types of applications are usually built around databases that utilize block data as opposed to files, employ shared access, and must maintain current data for maximum effectiveness. NAS, which is typically cheaper to implement than
Chapter 15
532
a SAN and fits into the standard Ethernet environment, is better suited to file-based applications such as web-hosting, content delivery, and messaging. A SAN consists of two basic elements, storage media namely, tape drives and arrays, and networking infi-astructure, which includes switches, hubs, host bus adaptors (HBAs), and the cabling that interconnects the storage devices. A simplified SAN is shown in Figure 15-2.
Clients
10/100Base'
Application Serverp with Host Bus Adapters nfrarne Fibre Channel (F|C) SAN Switch
ESCON/F!CO^ AM"-^
ESCON/FICON
storage Devices Figure 15-2. Storage Area Network (SAN)
15.3.1 Factors Driving SAN Extension Due to the tendency of deploying infrastructure to meet local needs, many multisite organizations have developed storage architectures characterized by SAN islands — pockets of noninterconnected SAN
SONET Services for Storage A rea Networks
533
infrastructure. As is the case with direct attached storage architectures, SAN islands may suffer from sub-optimal resource utilization, since storage may be available but not at the site of greatest need. In addition, cost-effective scaling to accommodate growth may be a challenge, since storage capacity would have to be added at every site. A technology solution that enables interconnection or networking between SAN islands can increase asset utilization and lower total cost of ownership (TCO). Asset utilization would be increased by sharing the storage pool among a larger number of users, and TCO would be lowered by amortizing the cost of the storage infrastructure over the larger group. Interconnecting SANs may also improve access to information by enabling data consolidation. This consolidation may be physical or virtual. In the case of physical consolidation, high-speed network links would be used to provide multiple sites with access to a physically consolidated storage pool. With virtual consolidation, software would be used to manage physically separated but optically networked storage so that it appears monolithic. This enables users at one site to access available storage at a remote site. The interconnection of SANs is also being driven by a changing regulatory environment that is placing more stringent requirements on organizations regarding accessibility, security, and retention of information. By connecting SANs, an organization can increase the level of control it has over SAN traffic. Some of the specific regulations governing records management are the Health Insurance Portability and Accountability Act (HIPAA), which governs the healthcare industry, and the Sarbanes Oxley Act, which applies to all publicly traded corporations. In addition, the U.S. Food and Drug Administration regulates record keeping in the drug, medical device, and biotech industries, and the US Securities and Exchange Commission (SEC) has also imposed guidelines on financial institutions regarding minimum standards for data protection. Similar regulations exist or are being implemented in other parts of the world. One of the most common methods for complying with the regulatory regimes and for ensuring business continuity is data replication, a process in which data from a primary volume is copied or replicated on a secondary volume. In the past, this process has occurred with primary and secondary volumes located in close proximity to one another, but recent unexpected events, both natural and manmade, have demonstrated the need for increasing the geographic separation between primary and secondary sites. This drives the need for technologies that support extension of the distances between SANs from meters or hundreds of meters to tens or hundreds of kilometers.
534
Chapter 15
Each of the examples cited above— ehmination of SAN islands, data consolidation, regulatory pressure, and business continuity— is forcing organizations to consider SAN extension as a means of increasing the utility and value of their SAN infrastmcture. SAN extension allows the benefits of SANs to be enabled across a multisite network.
15.3.2 Fibre Channel: The Storage Protocol of Choice No discussion of SAN extension can exclude a reference to Fibre Channel, the serial data transfer architecture and protocol running on the majority of today's SANs. Fibre Channel is currently, and for the foreseeable future, the protocol solution of choice for organizations that need reliable, cost-effective information storage and delivery at Gigabit per second (Gbps) speeds and beyond. Developed in 1988 and ratified as an ANSI standard in 1994, Fibre Channel is the mature, safe solution for storage device communications. It was specifically designed to address the stringent requirements of network servers and their storage applications. Fibre Channel provides serial link connection between multiple nodes and is able to support multiple protocols including SCSI. Fibre Channel facilitates the high-speed transfer of large amounts of information at varying line rates over both electrical and optical media. Fibre Channel supports point-to-point or switched topologies with each node port connected to another node port or a fabric port. The configuration options are varied and can be selected based on the performance required. The topology choices are illustrated in Figure 15-3.
Point to Point
~ ' Loop
Figure J 5-3. Fibre Channel topologies
The Fibre Channel specification [3] defines a hierarchy consisting of five levels; FC-0, FC-1, FC-2, FC-3, and FC-4. The levels FC-0, 1, and 2 define a physical, signaling, and transmission protocol, respectively, for Fibre Channel. FC-3 defines common services for advanced features, and FC-4
SONET Services for Storage Area Networks
535
defines application interfaces. The structure of the Fibre Channel hierarchy is shown in Figure 15-4.
FC4 Protocols e.g SCSCI, IP, ATM, IPI, HIPPI
FC-3 Services FC-2 Framing/Flow Control FC-1 Encoding/Decoding FC-O 133Mbps to 4Gbps Figure 15-4. Fibre Channel Protocol Stack
Fibre Channel is able to meet the storage challenges of organizations large and small with the following advantages: • Solutions Leadership; Fibre Channel provides versatile connectivity with scalable performance. • Reliability: Fibre Channel, a most reliable form of communications, sustains an enterprise with assured information delivery. • Gigabit Bandwidth; 1 and 2 Gigabit solutions are available today, with 4 and 10 Gbps solutions under development. • Multiple Topologies; Dedicated point-to-point, shared loops, and scaled switched topologies are available to meet storage application requirements. • Scalable; From single point-to-point gigabit links to integrated enterprises with hundreds of servers. Fibre Channel delivers unmatched performance. • Congestion Free; Fibre Channel's credit-based flow control delivers data as fast as the destination buffer is able to receive it. • High Efficiency; Fibre Channel has very little transmission overhead, especially for moving large blocks of storage data. Fibre Channel is a well-established and proven protocol that has found application in SANs, computer clusters, and other data-intensive computing configurations. Any consideration of a distance extension technology for SANs needs to account for the requirements for Fibre Channel operation in order to be viable in the marketplace.
536
15.4.
Chapter 15
DISTANCE EXTENSION REQUIREMENTS
Storage applications often have very demanding performance requirements. This is not totally unexpected, given the business value that storage applications bring to an organization. Business continuity, one of the key drivers of storage infrastructure deployment, is about protecting mission-critical data. Since the stakes are so high, the performance of a storage network must be predictable and reliable. A distance extension technology introduced into the transport portion of a Fibre Channel SAN must meet some basic requirements, which include: • In-order delivery of block data • Low latency • Lossless data transport • High throughput • Network flexibility and resilience • Carrier grade • Scalability and performance Let's consider each of these requirements in turn. 15.4.1.1
In-order Delivery
The Fibre Channel protocol transmits data in blocks. In order to preserve the overlay application being run (usually a database-centered application like ERP or CRM, for example), these blocks must be received by the target device in the order that they were sent, since no reordering of blocks is accommodated by the protocol. 15.4.1.2
Low Latency
Latency is the term used to describe the amount of delay in a network. Data sent between two points in a network will incur latency from a number of sources, including adaptation (moving from a native FC format to an alternate scheme for transport, for example), equipment (since there is a certain amount of delay incurred as data transits a router, an add-drop multiplexer or gateway, for example), and the amount of distance that must be covered. Data that must travel a long distance will incur more latency than data that travels short distance. Latency impacts the performance of a SAN because overlying applications usually have some type of expected response time built into their operation. Increase latency too much and an application may not function as expected.
SONET Services for Storage A rea Networks 15.4.1.3
537
Lossless Delivery
Fibre Channel networks do not tolerate discarding of data. Every single piece of data sent must be received at the target in order to complete the transaction. If data is lost in transit, the entire block must be resent. This requirement increases the amount of time required to complete the action and introduces additional latency or delay, which in turn can degrade the performance of the overall application. 15.4.1.4
Throughput
Throughput is an important consideration when connecting SANs, since the replication or backup of data must usually occur within some predetermined window. The amount of data to be transferred and the amount of time available for the backup or transfer will determine how much bandwidth is required to meet an organization's storage networking need. Sustained and deterministic levels of throughput allows IT managers to have confidence that their storage applications will function in a predictable fashion. 15.4.1.5
Network Flexibility and Resilience
In addition to meeting the specific requirements of storage applications such as latency and throughput, there are also overall design considerations that must be accommodated to guarantee an infrastructure capable of supporting storage distance extension. In order to be truly competitive, a storage infrastructure must support standard data center protocols such as Fibre Channel, FICON, ESCON, and Gigabit Ethernet. These protocols are well established and broadly deployed. As such, the acceptance of a storage networking technology will be enhanced by its support for legacy protocols, particularly if it provides a means of supporting multiple protocol streams simultaneously. 15.4.1.6
Carrier Grade
The infrastructure choice for distance extension also needs to be carrier grade and provide end-to-end system availability of 99.999 percent (5 minutes of downtime annually) as well as provide support for peering with multiple carriers with no degradation in service. The ability to provide the sub-50 ms rerouting that has become standard in today's telecommunication network also enhances the storage application. The combination of carrier grade capability and fault tolerance will guarantee a robust network suitable
538
Chapter 15
for deployment by a carrier and well suited to supporting a mission-critical storage application like business continuity. 15.4.1.7
Scalability and Performance
Scalability and performance are two additional elements of a storage network that can be impacted by the technology selected for intersite network connectivity. In order to provide the level of support for the mirroring applications, essential to business continuity and disaster recovery, data recovery needs to measured in minutes, not hours, and the amount of data loss as a result of an interruption should approach zero. This type of performance needs to be supported on scalable storage infrastructure. With growth being a common characteristic of all storage networks, the infrastructure supporting storage connectivity needs to have the capacity to scale as the demand for bandwidth increases. Scalability includes flexible bandwidth choices and easy provisioning.
15.5.
DISTANCE EXTENSION ALTERNATIVES
There are a number of options available for storage distance extension. The right option for any situation will depend on the ability of the networking technology to cost-effectively deliver the performance demanded by the underlying applications. Two important factors that may determine the suitability of a distance extension technology in a storage application are the recovery-point objective (RPO) and the recovery-time objective (RTO) of the organization that owns the data. RPO is a measure of the period of time between data backups. A short RPO would imply backups occurring with greater frequency than a long RPO. Simply put, the RPO places a measure on the amount of information an organization can afford to lose. RTO is a measure of the amount of time that it will take an organization to recover its data after a failure. A short RTO implies a shorter recovery period than a long RTO. Together, RPO and RTO help define the type of distance extension solution that an enterprise can implement while still meeting its business objectives for storage. If an organization has a need for supporting mirroring of large amounts of mission-critical data, for example, it would probably need a distance extension solution able to sustain very short RPOs and RTOs. Such solutions would tend to support significant bandwidth and minimal latency. At the other extreme, support for an application that is
SONET Services for Storage A rea Networks
539
non-mission critical and that is characterized by a long RPO or RTO could be handled by distance extension options that offer lower bandwidth and throughput. Four of the current options available to organizations seeking distance extension solutions are described below. Among these options, SONETbased distance extension offers significant promise as a solution offering a very favorable trade-off between performance and cost and will be discussed in greater detail later in this chapter. The current options include: 1. Legacy access solutions, which provide a low-cost option for connecting storage systems with low bandwidth requirements. 2. WDM storage connectivity (lambda) solutions, which are significantly greater in cost to deploy but provide the highest bandwidth and reliability for the most demanding storage applications. 3. Storage/Fiber Channel (FC) over IP, which is becoming much more common with the advent of the Internet and with core IP backbones becoming available. These are low-cost solutions that take advantage of the prevalence of IP. The drawback is that IP QoS is still emerging at this time, which reduces the overall reliability and predictability of IP storage traffic. 4. Storage/FC over SONET/SDH, which is becoming a popular choice for sub-Gig-data-rate storage requirements. For the service providers this approach leverages the invested SONET infrastructure in place today, enabling them to provide an economical storage service to their customers.
15.5.1 Legacy Private Line The so-called legacy category of distance extension solutions utilizes traditional low-speed private line services like DS-l's and DS-3's for distance extension. This option carries storage traffic across the installed telecommunications infrastructure using ATM over SONET transport. Legacy solutions suffer from a lack of scalability, particularly disadvantageous in the face of a rapidly growing data pool.
15.5.2 WDM Wave division multiplexing (WDM), a technology that uses multiple wavelengths (or frequencies) of light to transport concurrent streams of digital information over fibre optic cable, is an ideal technology for storage extension because of its ability to meet the most rigorous performance requirements of storage networking.
540
Chapter 15
The value that WDM brings to storage is the fact that it is protocol agnostic. This means it can be used to transport any type of traffic, including storage traffic in its native format. This versatility allows a WDM network to transport native Ethernet (at any speed), native Fibre Channel, and ESCON, FICON, and any other storage protocol the user may choose to use. Furthermore, the ability of a WDM system to carry multiple types of traffic over different wavelengths on the same fiber only adds to its utility in data center applications. A WDM network can be used to connect multiple geographically separated sites and can scale to support data transfer rates measured in the hundreds of Gigabits per second. Among the advantages it offers is support for business-continuity applications that require high availability systems that are not susceptible to WAN outages. WDM systems offer massive scalability but usually require dark fiber and may not be cost-effective for all organizations.
15.5.3 Storage over IP 15.5.3.1
iSCSI
iSCSI is one of the three technologies being touted as an option for taking advantage of IP networking for storage applications. As Figure 15-5 shows, iSCSI operates on top of TCP (transmission control protocol), moving block data cells (or iSCSI packets) over an IP Ethernet network. This transport protocol consolidates SANs into a single IP network for both data and storage traffic, using Ethernet for the LAN and IP-over-optics for the WAN.
^^l
Mmi
Figure 15-5. iSCSCI hierarchy for storage
Because of the routing capabilities of an IP network, iSCSI is multipoint and provides any-to-any connectivity. iSCSI requires equipping both server and storage system with iSCSI components. This means incremental capital costs and higher server overhead associated with TCP processing. On the other hand, iSCSI scales well to 1 Gb and 10 Gb Ethernet and can result in administrative cost reductions in a pure TCP/IP network. Applications appropriate for iSCSI are fairly limited today, namely, local storage access where performance is not critical.
SONET Services for Storage Area Networks 15.5.3.2
541
FCIP
FCIP [8] is another technology being proposed as an option for storage networking over an IP network. Figure 15-6 shows the hierarchy in an FCIP network.
J ^ 'ir'.:
Ij-iihi
Figure 15-6. FCIP hierarchy for storage
This transport protocol introduces a new network device between the Fibre Channel SAN and the IP network, sometimes called an edge device because it resides at the edge of the Fibre Channel SAN. FCIP uses tunneling technologies by encapsulating Fibre Channel frames, whether FCP or SCSI, into IP packets and maps Fibre Channel fabric domains to IP addresses. FCIP supports existing Fibre Channel SAN hardware and software investments while allowing all SAN-connected data to be accessed over the IP network backbone without altering the Fibre Channel fabric, servers, storage devices, or software in any way. For this reason, many industry analysts agree that FCIP is the transport protocol that most seamlessly integrates IP into existing fiber channel storage area networks. Applications appropriate for FCIP include storage-to-storage applications such as disk mirroring, tape backup/restore, and FC SAN interconnection. 15.5.3.3
iFCP
iFCP is a gateway-to-gateway protocol that provides Fibre Channel fabric services over a TCP/IP network to FCP-based Fibre Channel devices. This protocol uses TCP to provide congestion control, error detection, and error recovery. This approach currently has very limited vendor support
15.5,4 SONET/SDH Storage over SONET and its ETSI equivalent, storage over SDH, use the widely deployed SONET [9] and SDH [10] infrastructure built to support voice transport to carry native storage traffic over considerable distances. With a few modifications to optimize the transport of block storage over a network built around the transport of 64 kbit voice, storage over SONET/SDH holds promise as a vehicle for providing affordable SAN extension capability to a very wide market segment including small-.
Chapter 15
542
medium-, and large-sized businesses. Table 15-2 provides a comparison of the alternative distance extension options. Table 15-2. Distance extension technology comparison L^ac)? Access
"BWDM
FCIF
++-h
++
++4-
++
+++
+
+4-+
++
++
+++
+
+++
4H-
++
+++
4-
+++
+++
+++
+++ i
+++
++4-
++
++
++
+4-
+4-4-
+++
+
+4-4-
+++
++
+++
++
4-+
-f++
+++
++
+++
+4-4-
+++
+
++
•' iSCSI ' SCM^:
ATM,FR)
r
i
Mrott^^mt
''''••y::'^'^i%mwA%>':'-'
+ Good
15.6.
++Better
+++ +++ !
+++ Best
SONET—AN IDEAL DISTANCE EXTENSION PROTOCOL
SONET is the undisputed king of the voice communication infrastructure. Every major telecommunications provider has networks built on SONET or, in the case of the ETSI world, on SDH. The reason is simple. SONET provides a standardized, high-reliability infrastructure for voice communications. With guaranteed interoperability between disparate networks and the ability to hand off traffic seamlessly at peering points, SONET and SDH have been linchpins in the development of a global telecommunications infrastructure. Given the significant investment in SONET/SDH networks around the world, it makes good economic sense for service providers and users alike to
SONET Services for Storage Area Networks
543
get the maximum benefit out of the network by using it wherever possible to carry different types of traffic. Storage traffic is emerging as a dominant class of traffic, and this has provided the impetus for the development of techniques to facilitate its transport over the SONET/SDH network. SONET-based connectivity offers an attractive option for networking storage area networks because it supports distance extension, which is increasingly important as enterprises implement business continuity solutions. The networking of storage with SONET connectivity presents both a challenge and an opportunity for service providers and enterprises. The opportunity is to create a new revenue stream based on the transport of native protocol storage between any two points where the provider has existing network connectivity. The challenge is to meet the availability, reliability, and performance requirements needed for the secure and effective support of storage applications such as mirroring or providing server cluster connectivity. Figure 15-7 shows how SONET fits in a SAN extension application.
SAN Connected with SONET Clients
Clients
Servers
Servers
Storage Data Center 1
Figure 15-7. Inter-Data-Center connectivity based on SONET/SDH
544
Chapter 15
15.6.1 Making SONET Fit— The Role of Standards While SONET in its current form offers an attractive connectivity option for networking storage, there are enhancements under way that are aimed at enhancing its efficiency as a storage networking protocol, particularly in distance applications. Three areas of interest are the following: • Generic Framing Procedure (GFP) [11] • Virtual Concatenation (VCAT) [ 10] • Flow Control 15.6.1.1
Generic Framing Procedure
Generic Framing Procedure (GFP) is an ITU standard that describes a flexible mapping technique for transparent transport of multiple protocols in a SONET or SDH network. GFP provides a low-overhead procedure for transporting both packet services and storage services including FICON, Fibre Channel, ESCON, Ethernet, and OC-n signals over SONET. GFP offers unambiguous implementation across multivendor networks, error correction schemes that enable the extremely low bit error rates critical for storage connectivity, and very efficient mapping into SONET without the addition of protocol baggage. Protocol baggage is the protocol overhead that occurs when data is mapped to other protocols from its native protocol. For example, prior to GFP, the transport of native Fibre Channel frame required it to be mapped onto TCP/IP, which was then encapsulated in an Ethernet frame and transported across the local area network. Subsequently, for its transport over the wide area network, further protocol conversion of Packet over SONET (POS) needed to be performed, where the Ethernet headers were stripped off and PPP/HDLC headers were appended to the IP packet and subsequently mapped onto SONET. This protocol conversion adds significant processing steps and protocol overhead (baggage) at each layer, as can be seen in Chapter 5. With GFP, the Fibre Channel frame is directly mapped into SONET with little protocol overhead. 15.6.1.2
Virtual Concatenation
Virtual concatenation, as discussed in Chapter 4, is an ITU standard that enhances link utilization and transport capacity efficiency by enabling complete flexibility in the allocation of bandwidth to client signals. SONET is able to transport storage traffic at different rates on standard interfaces; however, in many instances, the allocation of bandwidth to a specific storage signal has traditionally been inefficient. Many times a full OC-3, OC-12, or
SONET Services for Storage Area Networks
545
OC-48 has had to be assigned to transport this storage data, even though only 100 Mbps may have been required. 15.6.1.3
Flow Control
Flow control is a scheme for adjusting the flow of data sent from one device to another in a way that ensures that the receiving device can handle all the incoming data. This important mechanism helps to guarantee delivery between devices in a storage network. Since storage networks cannot tolerate discard, every bit of information sent must be received, and receipt must be acknowledged. Flow control is typically implemented via a credit system that ensures that the rate of data sent does not exceed the ability of the receiving device to receive, and hence no information is lost. The system uses credits to allow the sending device to keep track of how much capacity the receiving device has in its input buffer. Once the receiver's input buffer is full, the sending device will have no more credits and cannot resume sending information until it receives an acknowledgment in the form of buffer credits back. Buffer credits are critical to the transmission of storage traffic over distance because they maintain performance by allowing continuous use of the link and ensure zero data loss. The implementation of buffer credit-based flow control schemes on SONET access devices is a key element for ensuring the efficiency of a storage link that extends across the WAN. A WAN flow control system helps to eliminate a phenomenon known as droop, in which the performance of the extended SAN degrades because the distance between the initiator and target devices precludes a fast enough receipt of acknowledgment messages by the initiator to maintain its store of credits. If the initiator does not receive acknowledgment of delivery by the target and runs out of credits, it will stop putting new data on the link. This situation can arise on long links when the latency introduced by travel time on the link is sufficiently large to impact receipt of the acknowledgment message by the initiator. The end result is a situation in which, even though there is available bandwidth on the link, there is no data being placed on the link by the initiator device. Flow control standards are currently being considered under an initiative within ANSI Technical Committee T i l [12], which has a mandate to standardize additional configurations and protocols that support the extension of Fibre Channel networks over distance. This standards effort, which is being worked on under the title Fibre Channel Backbone - 3 or FC-BB-3 [13], will address standards for flow control as well as performance, timers, and management functions for configurations that support extended distance extension.
546 15.6.1.4
Chapter 15 Additional Benefits of SONET for Distance Extension
The addition of GFP, VCAT, and flow control capabilities to SONET are specifically aimed at optimizing its ability to transport block encoded protocols, including Fibre Channel. However, SONET offers a host of other attractive capabilities that make it ideally suited to distance extension. First and foremost is its ubiquity. SONET/SDH services are available in all geographies globally. This level of availability means that any organization with standard telecommunications connectivity has the potential to use SONET/SDH in a SAN distance extension application. Furthermore, the cost of a SONET/SDH service is within the reach of the average organization. The continued erosion of bandwidth prices will only enhance that attractiveness of SONET in distance extension applications. Another aspect of SONET that adds to its appeal as a distance extension technology is its reliability. SONET/SDH networks are robust. The industry term carrier grade is often used to describe the elements on which a SONET/SDH infrastructure is built. In practical terms, this is reflected in features such as equipment redundancy as well as fast-failover in the event of equipment or link failures. This level of reliability is perfectly suited to storage applications, which are by definition designed to address the problem of equipment failure. SONET networks also offer a level of security and deterministic behavior not found in IP networks, for example. SONET circuits are usually point-topoint. As such, they are less susceptible to snooping, and because their behavior can be accurately characterized, adjustments for factors such as latency can be accommodated when implementing applications. As previously noted, given the impact of latency on storage applications, these features are very useful when an organization is considering a distance extension technology. The ability to characterize the behavior of a SONET link makes it possible for a service provider to offer a customer a servicelevel agreement (SLA) with defined quality of service (QOS) guarantees. An SLA for a storage service would have to take into account the essential elements of service required to support a data replication service, for example. The SLA for the service would define metrics for the service, including availability, throughput, network transit delay, mean time to repair (MTTR), mean time to respond, data delivery ratio, and reporting requirements. A sample SLA is shown Table 15-3. SLAs are an essential element in the mass-marketing of telecommunication services and can be expected to facilitate SONET/SDH adoption in distance extension applications.
SONET Services for Storage A rea Networks
547
Table 15-3. Sample SLA of Storage Network extension
1 ' - 'SetViae I^vel'Guaraiitee/''
\
Availability Throughput Network transit delay Mean time to respond Mean time to repair Data delivery ratio Notification of problems
Domestic S^^Tice. 100%
1 Gbps 125 ms round trip 2 hours 4 hours 99.99% 30 minutes
•'• ,
'> M
1
Finally, the interoperability of SONET networks creates the potential for borderless distance extension. Since the storage traffic being carried across a SONET/SDH network is carried transparently, it can transit add-drop multiplexers, cross-connects, switches, and other elements of the SONET network with no need for special treatment by any of these devices. It can also be handed off from one carrier to another with no need for special consideration. The entire network, metro and long-haul, is available to a customer seeking to move data from one geographic location to another. This capability allows the owner of data to support storage applications ranging from backups to replication to mirroring. Of course, as previously described, the desired performance will drive the required bandwidth and the distance between sites, but the flexibility exists to meet and address a wide range of scenarios.
15.7.
SUMMARY
Data growth is one of the most pressing issues faced by all organizations. In the current environment, data is an important commodity that must be managed to meet business needs and protected to ensure regulatory compliance. Storage networking has emerged as a very important vehicle for simultaneously addressing the business need and fiduciary responsibilities of an organization. Storage Area Networking has proven itself to be the best storage networking option for management of missioncritical block-oriented data. As recent unexpected events have shown, geographic separation is one means of ensuring business continuity in an uncertain world. With the ongoing reductions in the cost of bandwidth and developments in technology, distance extension is proving to be a viable option for organizations looking to increase the separation between their SANs. Of the solutions available for distance extension, SONET/SDH connectivity offers significant promise for distance extension based on its
548
Chapter 15
attractive combination of cost and performance. With enhancements in its abihty to carry native storage traffic using GFP and to carry traffic more efficiently using VCAT, SONET/SDH offers a practical means of transporting storage traffic across multicarrier networks. SONET/SDH comes standard with carrier grade reliability, predictability, scalability, and protection of that bandwidth from end-to-end. The combination of GFP, VCAT, and flow control, which bring intra-data-center performance to interdata-center connections, allow SONET/SDH networks to meet the stringent demands for extending storage area networks.
15.8. REFERENCES [I] [2] [3]
[4] [5] [6] [7] [8] [9] [10] [II] [ 12] [13]
"How Much Information? 2003", hUp://vvwvv.simsJ>erkeley.edii/Yesearcli/projects/howmuch-inib-20Q3/. ESCON, http://\vwvv.redbooksJbmx-oiTi/abslracis/sg244662.html . Fibre Channel Framing and Signaling (FC-FS), ANSI INCITS.373:2003, October 27, 2003. Note: Published Til standards are available from the INCITS online store at http://www.incits.org or the ANSI online store at http://www.ansi,org. FICON, http:/7wvvvv.redbook$.ibm.coni/abstracts/sg246266.htmr?Qpen. SCSI-ANSI X3T10 Internet Small Computer Systems Interface, iSCSI, Internet Engineering Task Force, RFC 3720, April 2004. http://www.ieee802.Org/3/. Fiber Channel over TCP/IP, FCIP, Internet Engineering Task Force, RFC 3821, July 2004. ANSI American National Standard T1.105, Synchronous optical network (SONET) — Basic Description including Multiplex Structure, Rates and Formats. ITU-T Recommendation G.707/Y1322 Network Node Interface for the Synchronous Digital Hierarchy SDH, 2003. ITU-T Recommendation G.7041/Y.1303, The Generic Framing Procedure (GFP), 2003. http://wWW,t 11 .org. Fibre Channel Back Bone -3 (FC-BB-3), ANSI INCITS, work in progress, http://www.tlLorg.
PART 3 Control and Management of Transport Networks
This page intentionally blank
Chapter 16 ARCHITECTING THE AUTOMATICALLY SWITCHED TRANSPORT NETWORK ITU-T Control Plane Recommendation Framework
Alan McGuire , George Newsome , Lyndon Ong^, Jonathan Sadler , Stephen Shew , Eve Varma^ 'British Telecommunication Pic, ^Consulting Engineer, ^Ciena, "^Tellabs, ^Nortel, ^Lucent Technologies
16.1.
INTRODUCTION
Transport networks have traditionally been associated with manual provisioning of circuits for long-duration services based upon a centralized management system for configuration and provisioning. Originally, transport networks were completely manually operated, involving circuit orders on paper along with staff located in equipment stations to both execute the circuit orders (make connections) and locate and repair equipment faults. Each equipment generation has added more to automated network operation. PDH generation networks introduced remote operations, but provided little with regard to integrated management. SDH generation networks provided standards for maintenance features and equipment control, and some network operators even developed automated operational support systems capable of creating hundreds of circuits a day with connection setup taking minutes per connection [1]. This development has been adequate for automating provisioning within carrier-specific operations systems, but did not allow for easy operation between carriers. In fact, after ten years of wide-scale SDH deployment, there is still no existing platform for provisioning connections across multiple operators. Historically, switched services have been considered as connections that are set up and torn down using signaling protocols, while the setup and
552
Chapter 16
teardown of leased line services was performed via network management protocols. This distinction has been an artifact of the traditional demarcation between transmission and switching. The distinction between switchedservices-based and leased line services has begun to blur, partly due to the shortening length of contracts for leased lines, and many network operators and suppliers are developing control plane (see Section 16.2.1 for a discussion of planes) technology for application in transport networks. The goal has been to allow faster service provisioning, particularly between network operators, as well as the creation of new network services [1]. The advent of control plane technology, and associated orientation towards switched connection services, enables fine-grained control of a few specific services versus control of equipment in general. This service orientation facilitates technology independence, and the fine-grain aspect makes it more likely that interworking between carriers will be possible. This is because it is no longer necessary to interconnect all aspects of each carrier's operations system, but only to connect a single connection service. Utilization of control plane technology, however, does not remove the need for fault localization, performance management, or trouble ticketing. The control plane additionally offers opportunities for increased automation, which has traditionally always led to reduced operating costs. The application of control plane technology to transport networks has rapidly gained industry momentum, with various standards bodies and industry fora engaged in tackling various facets of this problem space. The ultimate vision is multivendor and multicarrier interoperable networking that supports end-to-end switched connection services on a global scale. To reach this goal, open standards for a distributed control plane must be established. Standardization activities have been under way within the International Telecommunication Union-Telecommunications Standardization Sector (ITU-T), the Internet Engineering Task Force (IETF), and the industry fora Optical Internetworking Forum (OIF) and ATM Forum. The ITU-T started "top down" with the development of the networking requirements for the generic automatic switched transport network (ASTN), working down into detailed protocol requirements. The IETF started "bottom up" in developing the Generalized Multi-Protocol Label Switching (GMPLS) umbrella of specifications based upon modifications and extensions of existing IP-based signaling and routing protocols [2]. The OIF has focused upon developing control plane implementation agreements based upon, wherever possible, available global standards and provides associated interoperability demonstrations with the intent of offering an early testing vehicle for the industry. The ATM Forum has primarily provided feedback and input regarding proposed extensions of protocols within their scope of expertise (i.e., PNNI). Iterative
Architecting the Automatically Switched Transport Network
553
communications among common participants, and liaisons among them, have been leading towards convergence of requirements and protocols so as to enable industry usage of a common/generic set of base protocols, with protocol extensions for transport domain application. Within this chapter, we focus upon ITU-T control plane standardization efforts, including requirements, architecture, and network models, and the relationship among requirements, architecture, and protocol solutions. The ITU-T Recommendations we will be covering deal with Requirements (G.807 [3]), Architecture (G.8080 [4]), Signaling (G.7713 [5], G.7713.1-3 [6-8]), Discovery (G.7714 [9], G.7714.1 [10]), Routing (G.7715 [11] and G.7715.1 [12]), Data Communications Network/Signaling Communications Network (G.7712 [13]), and Control Plane Management (G.7718 [14]). The draft ITU-T Recommendations on control plane management information modeling, G.7718.1, is touched upon but not detailed, as it was under development at the time of the writing of this chapter. The relationships among these Recommendations are illustrated in Figure 16-1.
omiticalty Switched Optical Network (ASON)
I
G.7713.1 (PNNI-based) n ! G.77132 :(GMPLS-RSVP-TE){
,__„ T I G.7714.1 I (Discovery ; Messages)
••
Y
G.7713.3 : (GMPLS-CR-LDP) I
ls-^s Based | ! Based |
Based
Figure 16-1. ITU-T control plane Recommendation structure
The depth of treatment will enable a reader to understand the reasoning used in the design of the various parts, though a detailed treatment of individual protocol specifications is beyond the scope of this chapter.
16.2.
NETWORK REQUIREMENTS (G.807)
Recommendation G.807, completed in July 2001, provides network-level requirements for the Automatic Switched Transport Network (ASTN) control plane, whose primary functions are related to the setup and release of
554
Chapter 16
connections across a transport network. It specifies the fundamental connection control functions, in single and dual homed applications, and other functions to provide for diversity in routing connections to support high availability services. We note that the dual homed requirements apply equally whether the customer has two connections to the same provider (protecting against physical plant failure) or whether the customer has connections to two different providers (protecting against failures within a single provider network). It is important to recognize this limited scope, as it was intended that management platforms continue to support other aspects of network operation. In particular, there is no mention of equipment provisioning or fault management functions being subsumed by the control plane. Because the time frame of the ITU-T SG 15 specification of the Optical Transport Network (OTN) was roughly concurrent with that for specification of the control plane, subtending SG 15 Recommendations (and many other documents) refer to the Automatically Switched Optical Network (ASON). However, there is nothing in the ASON Recommendations limiting their applicability to other transport technologies, and the terms ASON and ASTN have essentially become synonyms. Recommendation G.807 requirements are client and technology independent, and provide the foundation for the architectural specifications of switched transport networks and the technical specifications required to implement these networks for particular transport technologies. In the years following the 2001 issue of G.807, additional requirements have been added to ASON Recommendations as increased understanding has clarified some of the earlier requirements. It is expected that G.807 will be refined and enhanced in its next revision to reflect these insights.
16.2.1 Architectural Context While G.807 and G.8080 attempt to be explicit in defining the terminology used, there are a few terms that fall through the cracks (quite often this is because these terms are so commonly used by the engineers who draft these recommendations that those engineers no longer realize that these terms may not be familiar to others). A particular case is the sudden appearance of the term planes. The transport network, originally, was manually managed and all the equipment was simply transport. Later, automatic management via remote operations systems was introduced, and the terms transport plane and management plane were introduced. The transport plane refers to the components and resources being managed, while the management plane refers to the components and systems doing the managing. In terms of architectural models, the transport plane refers to
Architecting the Automatically Switched Transport Network
555
everything described by the G.805 [15] functional architecture (Chapter 2), and the management plane refers to the managed objects and the systems that operate on them. Since the management systems were remote from the managed equipment, a communications network was provided for the management applications to use. As this was a general-purpose communications network, the message communications protocol was designed to be independent of the transport equipment infrastructure being managed (even though many of the communications channels were provided by reserved capacity on these facilities). With the introduction of ASON, a new aspect of control was introduced, and the notion of control plane is a natural extension. The ASON control plane is the set of all components that provide the autonomous (from the point of view of a management system) operation of ASON services. It was thus implicitly understood that ASON would need a communications network, and it was implicitly assumed that such a network would have very similar properties to the management communications network that most transport network engineers are familiar with. This long-standing practice of separation between planes and having a communications network independent of the facilities is not generally found in IP networks. As we will later discuss, this has some important implications when signaling and routing protocols from the IP space are applied to the ASON architecture.
16.2.2 Call and Connection Control Connection control, essential to the operation of a transport network, may be provisioned, be signaled, or be some combination of the two (hybrid). Connections that are provisioned, by means of a management system or manual methods, involve configuration of every network element along the path with the information required to establish an end-to-end connection (i.e., G.805 network connection). Such connections are called permanent connections (PC) [3]. A switched connection (SC), illustrated in Figure 162, is dynamically setup via automated routing control (with or without a distributed route computation approach) and signaling functions on an endto-end basis.
Chapter 16
556
Figure 16-2. Example of switched connection service
A hybrid connection, known as a soft permanent connection (SPC) and illustrated in Figure 16-3, is set up by provisioning permanent connections at the ingress and egress ports of the network, with the control plane setting up the switched connection in between. An SPC has the properties of a leased line (private circuit) but is setup using a signaling protocol. All three of these approaches may occur within carrier networks, and may form the foundation for different service models and applications [2].
C
Management Plana
j^'
\ 1
/
;;^'NEU
ni
J
V'•
Control Plane
•>
Transport Plane '.
-NE;'-^- - NE
NE
ME - ^ . ' t
Switched Connections
'
X
/
I
Soft Permanent Connection Service
Figure J 6-3. Example of soft permanent connection service
Regardless of the approach, a critical requirement is that failures in the control or management plane should not affect the connection (we note that the same applies to software upgrades). In other words, in the event that signaling connectivity is lost, the transport network must maintain existing connections. Furthermore, when the control plane recovers, it should recover connection state information without affecting the live connections. This requirement is typically referred to as the separation of control and transport
Architecting the Automatically Switched Transport Network
557
planes and reflects the criticality of the transport plane maintaining an extremely high level of reliability. We note that the persistence of a connection itself is actually a function of the type of service-level specifications supported within a transport network. The majority of connections will have a persistency requirement (e.g., protected connections, or connections with some guaranteed level of availability). However, operators might decide to offer a "best effort" service-level specification for connections that are automatically released when a defect occurs on the connection. A call can be considered as a service provided to user endpoints, where multiple calls may exist between any two endpoints and each call may have multiple connections. The call concept provides an abstract relationship between users, where this relationship describes (or verifies) the extent to which the users are willing to offer (or accept) service to (from) each other. A call does not provide the actual connectivity for transmitting user traffic but only builds a relationship by which future connections may be made. Call control is therefore defined as a signaling association between one or more user applications and the network to control the setup, release, modifications, and maintenance of sets of connections. The concept of call control grew out of the vision of the intelligent switching network (IN), which was enabled by the concept that service control could be logically separated from connection control. This meant that new services could be offered, unconstrained by assumptions about the resources of the underlying network. This allowed the introduction of new value-added services in addition to the basic PSTN (Public Switched Telephony Network). Thus, a service request could still be made from a phone terminal, but the request would be delivered to a service manager control function rather than being directly delivered to the connection control fimction. The service manager is now a client of the original connection service and can employ that connection service as needed to carry out the service request. The service interface can now address resources that are completely unknown to the original connection service, enabling a wide range of new services to be created without affecting the underlying transport network. We can also consider call control itself to be a service that can be logically separated from connection control, allowing call control to be processed (and if necessary located) separately from connection control in the underlying transport network. The notion of considering calls and call control to be a kind of service is a reflection of the idea that the ability to set up connections in the network may be offered to others [16]. While call control functions were not explicitly discussed within G.807, there was an implicit assumption of their existence (e.g., discussion of call arrival rates and the need for associated congestion control mechanisms).
Chapter 16
558
In the PSTN world, examples of call control capabilities include features such as ringback, call divert, etc. In the transport world, a fundamental call property is its ability to be supported by multiple connections, where each connection may be of a different type and where each connection may exist independently of other connections within the call. The concept of the call allows for a better flexibility in how users set up connections and how the network offers services to users. In essence, a call allows: • Verification and authentication of a call prior to connection, which may result in less wasted resources • Support for virtual concatenation where each connection can travel on different diverse paths (e.g., taking an ESCON payload of 160 Mbit/s, adapting it using GFP [17], and mapping into a virtual concatenation group composed of 4 VC-3s, each at 48.384 Mbit/s) • General treatment of multiple connections that may be associated for the purpose of recovery; for example, a pair of primary and backup connections may belong to the same call • Use of a public and private addressing space (hosts using a public space, with the network using only an internal private addressing space) • Better upgrading strategy for service provider control plane operation, where call control (service provisioning) may be separated from switches and connections (where connection control may reside) An example of a single call with multiple connections for recovery purposes is illustrated in Figure 16-4. In this example, a single call exists between two users, and the network provides a highly available service. It accomplishes this by instantiating two connections between the users' points of attachment to the network, where user traffic is replicated on both connections. Should one of the connections fail in the network, the network can bridge traffic (at the egress) from the remaining connection such that the users do not perceive any change.
User 1 Q | -— • • ^
Call
^
3 ^ —4 ^
Connections
V ^ ***»» ^x^
**^
User 2
\
w
Figure 16-4. Example of call with two connections for availability purposes
Architecting the Automatically Switched Transport Network
559
Connections associated with a call may also vary over time. A call may exist with one connection and then at, a later time, another connection may be added. Another example is when a call with a single connection experiences loss of data transfer on that connection due to a failure. The call can still remain alive while a restoration action is initiated to create a second connection that, once active, will allow data transfer to continue. Another concept originating from the PSTN world is that of supplementary services in addition to mandatory services. Supplementary services provide more information regarding an existing service that may not be needed at all times but may be signaled when desired. The service access point for supplementary services (i.e., the service interface on the component that is offering the service) does not necessarily coincide with that for other mandatory services. This implies that different protocols may be used to support the service. Possible supplementary services that might be considered for control plane application include: • Customer management of Virtual Private Networks (Closed User Groups) • Route-Query • Route fragment caching operation (Route-Recording) • Directory services/client reachability Implicit in the discussion of requirements related to call and connection control are business and operational aspects, described in Section 16.2.3 below.
16.2.3 Business and Operational Aspects Control plane deployment will ultimately be set within the context of an operator's business model, and must support the business aspects of commercial operation. Deployment will also take place in the context of the heterogeneity of transport networks, which must also be accommodated in any control plane solution. In this section, we examine control plane requirements and implications arising from support for existing business models and transport network infrastructure options. Currently existing business models, one or more of which might be used by various organizations of the same network operator [18], include: 1. An Internet Service Provider (ISP) that owns all of its own infrastructure (i.e., including fiber and duct to the customer premises) and only delivers IP-based services 2. An ISP that leases some of its fiber or transport capability from a third party, and only delivers IP-based services on that infrastructure
560
Chapter 16
3. A service provider owning, for example, an SDH infrastructure, who offers SDH, ATM, and IP services, and who sells services to customers who may themselves resell to others 4. A Bandwidth Broker (or carrier's carrier) providing optical networking services (a subtle difference between this case and the previous case is that the bandwidth broker may not own any transport infrastructure(s) that support those services, and the connection is actually carried over third party networks) The above business models introduce requirements involving operational aspects that must be supportable by the control plane. Two of the primary areas relate to trust/security boundaries and billing considerations. We note that billing also means that there has to be a commercial agreement before purchasing bandwidth. This is true for a wholesaler even when selling in the same organization. Hence, a client platform requires financial authority to buy lower layer capacity, and service management needs to be aware of it. It is clear that policy and security needs vary among service providers, and that internal network service provider choices must be respected. For business model 1, since the infrastructure is fully owned by the ISP (an unusual scenario), there are no tmst issues as the infrastructure and service providers are one and the same. This is not the case for business model 2, where some of the infrastmcture is leased, and there is a trust boundary between the infrastructure provider and the ISP. A similar situation exists for business model 3, where there is a trust boundary between the service provider business and the client businesses (e.g., regarding the degree of visibility of internal service provider network routable addresses). For business model 4, there are clearly trust boundaries among all of the involved networks [18]. In addition to trust aspects, the different billing models employed among these business models must be accommodated. For example, ISP billing tends to be flat rate for a given access bandwidth. ISP billing mechanisms are not fine-grained, are not sensitive to distance, and do not track the number of different destinations {called parties). However, for business models 3 and 4, service providers would typically be interested in assuring not only that they can bill for any value added services they provide but also that they have the ability to consider various factors/parameters in their billing policies. Additional service provider requirements that impact control plane requirements include aspects that appear to be fairly obvious from a transport network infrastructure perspective, though not necessarily so from a data-oriented perspective. Examples of these include assuring that a carrier has the ability to control usage of its own network resources, i.e., that a carrier controls what network resources are available to individual services
Architecting the Automatically Switched Transport Network
561
or users. This ability allows the carrier to ensure that critical and priority services get capacity in the event of capacity shortages [19]. It also allows carriers to deploy their resources to support Service Level Agreements (SLAs) as they best see fit, supporting carrier-differentiated SLA realization. Another example is that the control plane solution should avoid inherent assumptions regarding optimizations for, or dependencies on, particular supported client services. This is evident from consideration of the aforementioned business models, where the first two only require support of IP services and the latter two require support for a range of client services. Aside from commercial business aspects, carrier choices related to the underlying transport network infrastructure also have implications for control plane requirements. The transport infrastructure has steadily evolved to include a wide range of bearer technologies, infrastructure granularity options, flexible capacity adjustment schemes, and survivability mechanisms. Network operators and service providers have deployed a range of bearer technologies, have chosen differing infrastructure evolution strategies, and have to cope with various considerations and constraints related to their operational support system (OSS) environments. It should similarly be expected that heterogeneity would occur in the deployment of the optical control plane, involving differing control plane signaling and routing protocol options and versioning, as well as management/control plane evolution scenarios. On top of this, it will be necessary to handle multivendor and multicarrier scenarios. Thus, from a pragmatic perspective, the requirements for control plane solutions must be developed with heterogeneous environments in mind so that they are able to coexist with the existing network. In other words, there should be no assumption that such solutions will be deployed in a green field and/or homogenous environment. In summary, examination of business and operational aspects results in an understanding that • Commercial "businesses" require a strong abstraction barrier to protect their operating practices and the resources of the business from external scrutiny or control • Provided value added services must be "verifiable" and "billable" in a value-preserving way • The transport network is segmented into portions belonging to different businesses • Transport networks are inherently heterogeneous (including the means by which they are controlled and managed) • Even within a specific business, further segmentation of the network can take place due to policy considerations (e.g., choice of survivability mechanism)
562
Chapter 16
There are a number of implications arising from these statements. For example, a control plane solution that satisfies commercial business requirements would allow a carrier to provide services without exposing the internal details of its network to their customers (as in the traditional PSTN). This leads to the concept of service demarcation points. The fundamental understanding that transport networks are inherently segmented into portions (or domains, as will be more fully articulated in the next section) drives the realization that the scope of connection control would not generally be on an end-to-end network connection basis. In the context of business and operational requirements, the goal of the control plane may be stated as supporting services through the automatic provisioning of network connections across one or more managerial/administrative domains. This involves both a service and connection perspective: • The service (call) perspective is to support the provisioning of end-to-end services while preserving the independent nature of the various businesses involved • The connection perspective is to automatically provision network connections (in support of a service) that span one or more managerial/administrative domains Indeed, the desire to provide and obtain services across an interface probably colors the ASON requirements more strongly than any other aspect of switched service, and leads directly to the recognition that a network of any size automatically has several business interests taking part.
16.2.4 Reference Points and Domains Recommendation G.807 identified three substantially different interfaces between • Service requester and service provider control plane entities (UNI), • Control plane entities belonging to different domains (E-NNI), and • Control plane entities belonging to one or more domains having a trusted relationship (I-NNI) During the development of subsequent Recommendations, it became apparent that the term interface was a misnomer, as interfaces tend to suggest physical connections. In particular, while working on G.8080, it was recognized that the UNI, E-NNI, and I-NNI interfaces from G.807 are in fact reference points, in that they are logical and are described by the information flows across the points. The flows themselves were characterized in terms of network details that were exposed or hidden from an information user. Each information flow supports a service (transport, signaling, routing, discovery).
Architecting the Automatically Switched Transport Network
563
and each service may have different endpoints. (The components responsible for routing are different from those involved in signaling, and are not necessarily co-located.) However, the reference point comprises all these services. Thus, it is essential to distinguish logical reference points from the physical interfaces supported by signaling and routing protocols that are carried over a communications network. The concept of domain, which was used in a conversational sense within G.807, has evolved over time as ASON requirements and architecture specifications have matured. Initially, it was considered useful from a requirements standpoint that all the equipment operated by a single carrier had its own group name, and the term domain was typically used. As work progressed on ASON architecture and routing requirements, the concept of a control domain was introduced, and more precisely described as an architectural construct that provides for encapsulation and information hiding. The characteristics of the control domain are the same as those of its constituent set of distributed architectural components. It was understood that control domains are generally derived from architectural component types that serve a particular purpose, e.g., signaling control domains, routing control domains, etc. The nature of the information exchanged between control domains across the E-NNI reference point, for example, captures the common semantics of the information exchanged among its constituent components, while allowing for different representations inside each control domain. Continued discussion of call and connection control led to further insights regarding the relationship between operator policy and establishment of domains. In summary, the domain notion embodied in the G.805 definition of administrative and management domains, and the Internet administrative regions (e.g., Autonomous Systems), has been generalized in the control plane architecture to express differing administrative and/or managerial responsibilities, trust relationships, addressing schemes, infrastructure capabilities, survivability techniques, distributions of control functionality, etc. Thus, a domain represents, and is characterized by, a collection of entities that are grouped for a particular purpose; hence, there are different types of domains. Domains are established by operator policies and have a range of membership criteria [4]. As domains are established via operator policies, it was further recognized that interdomain reference points (i.e., UNI and E-NNI) are actually service demarcation points, i.e., points where call control is provided. With this understanding, we can speak of reference points between a user and a provider domain (UNI), between domains (E-NNI), and within a domain (I-NNI), where the • UNI is a user-provider service demarcation point,
564
Chapter 16
•
E-NNI is a service demarcation point supporting multidomain connection establishment, and • I-NNI is a connection point supporting intradomain connection establishment. The fact that domains are created by policy, and have a range of membership criteria, should not be surprising. For example, when we introduced the concept of a subnetwork in Chapter 2, we indicated that subnetworks might be delimited according to a wide range of criteria, including such factors as administrative and/or management responsibility. Just as subnetworks are characterized by the points at their edge, with little regard to the equipment inside, so domains are characterized by the policies applied to the physical interfaces at their boundaries. The rationale for creating domains and subnetworks has everything to do with considerations that make sense to network operators, without the need for standardized rules for constructing them. It cannot be overly stressed, however, that reference points and domain boundaries are essential when the supported services are instantiated on some physical interface.
16.2.5 Architecture Principles Summarizing the discussions of the preceding sections, and referring to Figure 16-5 below, the ASON architecture framework reflects the policy boundaries that exist in transport networks [20]. • Calls are end-to-end service associations. While call state is maintained at network access points, it may also be required at key network transit points where it is necessary or desirable to apply policy. • When a call spans multiple domains, and hence E-NNIs, it is composed of call segments. Each call segment runs between a pair of call-state aware points, and the concatenation of call segments creates the call. • One or more connections are established in support of individual call segments. In general, the scope of connection control is limited to a single call segment, i.e., it does not typically span multiple call segments. The collection and concatenation of subnetwork connections and link connections provides end-to-end connectivity (i.e., the network connection). Some examples of multidomain scenarios that require the scope of connection control be limited to a single call segment include the following: [20] • The service is realized in different ways within each domain (e.g., technology, QoS). • Separate address spaces are used within each domain, especially when separately administered.
565
Architecting the Automatically Switched Transport Network • •
There is independence of survivability (protection/restoration) for each domain. There is a trust boundary.
Domain 3
Domain 1
User1
User 2
End-to-end call Call segments Connections LC
SNG
-•HLC%
SNCs
LC
SNCs
LC
Figure 16-5. Example of call with multiple call segments and connections
It is clear that a generic connection setup service exists that can be implemented with either management or signaling protocols. Further, the underlying protocols need not be from the telecommunications environment, since all that is needed is verification that they can support established telecommunications needs [16]. When we consider solutions from the Internet environment, it should be recognized that these bring along underlying principles and architectural aspects. Thus, it is important to understand Internet architectural principles and implications. The influences of the Internet "end-to-end" principle, and how it compares with the use of the call/connection model found in ASON and other connection-oriented services [20], is particularly interesting. The Internet "end-to-end" principle was described in a 1984 paper coauthored by J. H. Saltzer, D. P. Reed, and D. D. Clark [21]. Arguing that certain required end-to-end functions can only be performed correctly by the end-systems themselves, in RFC 3724 it "was articulated as a question of where best to put functions in a communication system" [22]. Again referring to RFC 3724 [22], "As the Internet developed, the end-to-end principle gradually widened to concerns about where best to put the state associated with applications in the Internet: in the network or at end nodes." As described in RFC 1958 [23] and in its update, RFC 3439 [24], "An endto-end protocol design should not rely on the maintenance of state (i.e., information about the state of the end-to-end communication) inside the network. Such state should be maintained only in the endpoint, in such a way that the state can only be destroyed when the endpoint itself breaks." In
566
Chapter 16
particular, "Hard state, state upon which the proper functioning of the apphcation depends, is only maintained in the end nodes" [22]. Thus, the preferred architectural principle of the Internet has been that connection control should be end-to-end and that no service state should be maintained at transit points within the network. It should be observed that the desire to avoid holding some call state information at transit nodes differs from the fundamental ASON requirement to facilitate service handoff between domains. The desire to make the scope of connection control end-to-end also differs from the ASON architecture framework, where we have seen that for most cases it is necessary that connection control be scoped to a single call segment. It is important to understand these classical Internet architectural principles when considering requirements and candidate protocol solutions. However, it is equally important to understand that the Internet architecture is itself evolving. The increasing commercialization of the Internet has stimulated some rethinking of its underlying architecture [25-28]. Quoting from [25], "An architecture of tomorrow must take into account the needs and concerns of commercial providers if it is to be accepted and thus to be able to influence overall direction". Several cited examples included the need for a framework for policy controls on interprovider routing, as well support for a variety of payment models for network usage. In moving from the purely technical considerations, which are of paramount importance for a community of users with shared goals and mutual trust, to a world in which the various stakeholders that are part of the Internet may have differing and sometimes conflicting objectives, a "tussle" [27] emerges. This "tussle" requires accommodation in the evolution of the Internet architecture and results in additional design principles that are organized around such concepts as separation of concerns, enabling choice, supporting the ability to bill for value-added services, and trust issues. For example, quoting from [22], "...prior to designing the protocol, the trust relationships between the network elements involved in the protocol must be defined, and boundaries must be drawn between those network elements that share a trust relationship." In particular, some key principles that have been introduced include the following: • "Modularize the design along tussle boundaries, even if there is no compelling technical reason to do so" [27]. • "Global communication with local trust"; "transparency modulated by trust" [28] • "The [New Internet] architecture should include a general notion of regions, to express differing interconnection policies, trust relationships, multiplexing mechanisms, etc. Regions may also support distinct
Architecting the Automatically Switched Transport Network
567
addressing regimes, performing any necessary address mapping as data crosses region boundaries" [28]. • "A future Internet should be designed without the requirement of a global address space" [28]. • "The Internet design should incorporate mechanisms that make it easy for a host to change addresses and to have and use multiple addresses. Addresses should reflect connectivity, not identity, to modularize tussle" [27]. The overall conclusions about the Internet end-to-end principle have been that "the end-to-end arguments are still valid and powerful, but need a more complex articulation in today's world". An illustrative example is where "the partitioning of functioning is done so that services provided in the network operate with the explicit knowledge and involvement of endpoints, when such knowledge and involvement is necessary for the proper functioning of the service. The result becomes a distributed application, in which the end-to-end principle applies to each connection involved in implementing the application" [22]. Restating, "a distributed approach in which the end-to-end principle applies to interactions between the individual pieces of the application, while the unbundled consequences, protection of innovation, reliability, and robustness, apply to the entire application." [22] Examination of the principles articulated in next generation architecture studies, and the ASON architecture, shows a striking level of consistency.
16.2.6 Supporting Functions and Requirements In this section we provide a summary of functions and requirements described in G.807. 16.2.6.1 Connection Management In general, connections in a transport network are expected to be bidirectional and symmetric. This differs from packet networking where the path in one direction is often independent of the path in the other, and must be set up separately. However, the capability to handle unidirectional or asymmetric connections should be supportable, if desired. Thus, a fundamental control plane requirement is to support the following connection capability types (for either SC or SPC): • Uni-directional point-to-point connection; • Bidirectional point-to-point connection; and • Uni-directional point-to-multipoint connection. It is also required that the control plane provide support for multihoming, which involves support for more than one link between the end users and the
568
Chapter 16
network. This can be subdivided into multihoming to a single network operator (e.g., for the purpose of resihence or load balancing) and multihoming to multiple network operators. Control plane signaling and routing capabilities must then also permit a user to request diversely routed connections from a carrier that supports this functionality. 16.2.6.2 Routing and Signaling To support connection management, G.807 identifies a routing function, which enables paths to be selected for the establishment of a connection through one or more operator networks. The routing function operates by ensuring that each element needing to select a path through the network has sufficient knowledge of network topology and resource utilization. In general, this is done by disseminating routing information throughout the network, creating several important trade-offs, which will be dealt with in later sections of this chapter. Routing brings requirements related to structuring the network for scalability, usually performed by hierarchical schemes that reduce the volume of data needing to be transferred or stored by removing detail as one moves up the routing hierarchy. This aspect is closely coupled to addressing schemes, which should allow for address summarization and which frequently achieve this by using the routing hierarchy to create an addressing scheme. Recommendation G.807 also introduced the primary connection management processes for signaling, including basic features for the UNI and NNI reference points at source and destination, as well as introducing additional procedures that may also be supported. These processes, features, and procedures provided an outline for specific behaviors and abstract messages in G.7713. 16.2.6.3 Naming and Addressing Coupled with the previous discussion on addressing, and considering our earlier observation of the need for a strong abstraction barrier between user and provider, G.807 also states that service provider addresses should not be communicated to users. It must be observed that, in general, an address is a concatenation of names, and a name only needs to be unique within a prescribed context. In many cases these distinctions are unimportant, and the term identifier is used. An implicit ASON requirement is that identifiers be unique within a single plane, so transport, management, and control planes are not constrained to use different identifier values. Indeed, it is also an implicit
Architecting the Automatically Switched Transport Network
569
requirement that a single plane may have several disjoint identifier spaces. This comes from the G.805 modeling techniques, which describe each network layer in isolation, leading to the usefulness of separate identifiers for each network layer. Further descriptions of identifier spaces are contained within G.8080 and G.7715.1 and are provided in Section 16.3.7. Some examples of drivers for separate user and provider identifier spaces include: • Enabling customers to take their identifier with them should they relocate (number portability) and/or avoiding a need to change their identifier if an operator altered their internal network structure • Avoiding client caching of provider addresses, (security/privacy considerations aside), potentially hindering network ability to evolve to larger/different address spaces or reallocate internal addresses • Ensuring a true multiclient server network (e.g., IP/MPLS, Ethernet, ATM, TDM). This does not imply that the same type of addressing could not be used in both user control planes and provider transport control planes as long as the semantics of the two identifier spaces are separate and distinct. (This would also be in conformance to RFC 1958 [23], which mandates exactly the same principle by requiring that "the internet level protocol must be independent of the hardware medium and hardware addressing" and that "this approach allows the internet to decouple its addressing mechanisms from the hardware".) • Assuring that identifier schemes are flexible enough to allow operators to use their existing routable addresses, when they so desire 16.2.6.4 Transport Resource Management Before an ASON control plane can be used to set up connections, it is clear that transport resources must be available for it to do so, which is a planning and provisioning exercise. This is especially true when ASON is being added to an existing network, as those resources are already under the control of management systems. An important provisioning requirement is to allow for variable resource partitioning between control and management, and to assure that operations that move resources between management or control responsibility do not affect the state of the resource. This requirement allows changes to be made to the network composition or responsibilities while the network is operating. 16.2.6.5 Admission Control Both calls and connections require support functions to decide whether the network should admit a particular call/connection. Call admission control
570
Chapter 16
is performed before any connections are requested and may include checking the customers' privilege to make the call as well as checking the called parties' willingness to receive the call. The latter can be done at various times depending on later design decisions. Connection admission control in general has to do with the availability of network resources to provide the connection. As mentioned earlier, G.807 does not explicitly discuss call control functions.
16.2.7 Signaling Communications Network Requirements Any distributed control system requires communications among the elements implementing the system, and the ASON control plane is no exception. The Signaling Communications Network (SCN) is an integral, yet independent, part of the overall control plane. To maintain the integrity of control message delivery, the SCN must meet several important requirements, including the need for control message reliability to be guaranteed in almost all situations, even during what might be considered catastrophic failure scenarios. There are significant differences between packet-switched and circuitswitched networks impacting the SCN. For example, in MPLS, the control plane messages and the data plane packets share the same transmission medium and hence the same reliability (i.e., the topologies are congruent). A failure affecting data packet forwarding also affects control packet forwarding and vice versa [29]. In contrast, within transport networks it cannot be assumed that the topology of the controlled network is identical to the topology of the network supporting control plane communications. Perhaps the most obvious example for this is to consider the ASON control plane controlling a WDM transport network in which control plane signaling is carried on an Optical Supervisory Channel (OSC). As laser and receiver failures on different wavelengths are generally independent, if the OSC fails, it cannot be assumed that the traffic has failed. Furthermore, in an OSC failure scenario, it is necessary to find another route for the signaling traffic in order to get to the next node via some disjoint path. This separation allows the SCN to be independently designed, and allows for optimum use of resources to achieve the desired control plane performance. The communications network designer is able, but is no longer forced, to use the embedded channels provided by some, but not all, transport technologies. In particular, if adequately secured, a LAN or WAN could also be used. This flexibility is of particular interest when designing communications to support Switched Connections (SCs).
Architecting the Automatically Switched Transport Network
571
16.2.8 Support for Transport Network Survivability Recommendation G.807 specifies that ASON must be able to support transport network survivability. Transport network survivability may be handled by classical protection approaches (i.e., via autonomous actions within the transport plane) or by ASON control plane actions. ASON can offer new mechanisms for survivability by using signaling mechanisms to reroute a failed connection. The discussion in G.807 distinguishes between protection and restoration based upon the usage of dedicated or shared resources. In later ASON work, the distinguishing characteristic for survivability is whether or not the control plane is involved. I.e., protection is described as a mechanism for enhancing availability of a connection through the use of additional, assigned capacity, whereas restoration is described as involving the replacement of a failed connection by rerouting using spare capacity [4]. (We note that an implicit requirement is that the control plane itself be survivable.) Perhaps most important, G.807 states that user requests for explicit survivability mechanisms in a carrier network are not supported because users should not have visibility into the internal details of the carrier network. However, the user is permitted to request diverse connections—that is, a group of connections with limited common routing.
16.3.
ARCHITECTURE (G.8080)
In this section we consider the architecture of the control plane as described in G.8080. Before doing so, however, it is useful to take a short diversion to understand the transport networking and management landscape before the development of G.8080 and how this factors into new developments in control plane technology. In essence, it is the need to interwork with legacy environments while providing a framework for future developments that dictated the nature of the tools and methods employed in the G.8080 architecture. The centralized management systems developed in the last ten years or so normally contain a database of the network (for the technology of concern) that can be used for a variety of purposes, including route calculation, circuit visualization, "plan and build" processes, inventory management, and capacity management. This database can also be related to other systems that provide features such as service management, fault management (within and between technologies), trouble ticket management, fiber and cable records, and so forth. It should be noted that network management capabilities vary considerably not only between operators but also within an operator's
572
Chapter 16
network among technologies and platforms. To some extent, this is a reflection on the size of the network, decisions regarding what to automate, and the maturity of the technology. In such management systems, the centralized route calculation drives the connection setup process by means of management protocols that communicate with management agents (in the form of element managers) in network elements. The communication with the network elements is generally by means of a data communications network (DCN), which is a router network dedicated to network management functions. This architecture can also be made hierarchical. A criticism often leveled at such an architecture is that there is a single point of failure. In reality, large network management centers have duplicated systems at a fallback center and have more than one connection to a DCN. The DCN also provides resiliency so that there is generally more than one way to reach a network element to allow communication via network management protocols. For many applications, such systems are perfectly capable of managing large networks. In many ways, these systems already provide all the functionality that is often associated with a control plane, including route calculation and signaling. Their major limitations come in two distinct forms: • The connection control is too slow for switched circuits (rather than soft permanent connections). Management protocols, which are multipurpose vehicles, are simply too slow for this purpose. It is much better suited to a specialized signaling protocol. • Centralized control does not necessarily provide optimal response times for the purposes of restoration survivability mechanisms. At the other extreme, sometimes depicted (mistakenly) as the control plane, are control systems located on every network element and communicating with one another via a communications network. The centralized network management system plays no part in real-time activities such as route calculation processes, though network element static configuration parameters are still under network management control. The control systems carry out a routing information exchange process operating asynchronously in the background, and a connection setup process acts in real time between cooperating nodes. Here, each network element contains a routing table providing a set of alternative routes from itself to every other network element that shares routing information. These routing tables have the same information as the route search in the centralized architecture. However, the tables are computed using information obtained from a routing protocol whereby each network element exchanges the routing tables of its neighbors. As such, a network element can update its own routing table to provide more
Architecting the Automatically Switched Transport Network
573
current information regarding network topology. This aids the connection setup process and speeds up restoration that does not use pre-calculated routes. In the development of a control plane architecture, it could be assumed that the fully distributed model is all that is required. However, this model does not reflect the way in which control plane technology will be used in many networks or the fact that control plane functionality can also be provided by network management protocols. This situation can be easily understood by consideration of the following scenario. A network operator with a large installed base of outer core network elements that are controlled using network management protocols wants to introduce control plane technology on a new generation of network elements deployed into the inner core. In such a case, the network operator is not going to "rip out" all the existing outer core network elements and replace them and it may not be possible (or cost-effective) to upgrade them, to provide control plane functionality. In such circumstances, the problem is how to provide the end-to-end configuration of circuits in such an environment. One way of solving this problem is illustrated in Figure 16-6. The end-toend configuration and route calculation are directed by the network management system. For those parts of the connection that can be set up using management protocols, the management system calculates the route and sets up the connection. For the control plane controlled portions of the network connection, the management system delegates functionality to the control plane. In effect, the control plane network elements are collectively seen as a virtual network element by the management system. The management system provides the input and output points for the connection(s) that traverses the control plane-enabled network elements and then leaves the actual route calculation and setup to the control plane. In this scenario, it is clear that we have both central route calculation and distributed route calculation working in partnership and two different sets of connection control protocols (one network management based, one signaling based) that interact with network elements in different ways. During the initial development of the control plane architecture in the ITU-T, it became evident that what was required was an architecture that allowed the control plane functionality to be distributed in any allowable fashion, e.g., to every network element, shared by a group of network elements, or centralized. Furthermore, depending on the set of functions that are required, some functions may be centralized and others distributed in a single instance of the architecture. As an example, signaling can be combined with centralized or distributed routing. This situation led to an architecture that has as its main tool the concept of a component, borrowed
Chapter 16
574
and slightly modified, from object-oriented analysis and programming. The use of components also allows all the power of the Unified Modeling Language [30] (UML), and the software tools associated with it, to be applied.
OSS Connect messages from OSS
^'^
^•,.
Subnetwork connections created by a management protocol in this part of network Routing under control of OSS Subnetwork connections created by control plane in this part of network End points defined by the OSS, routing between these points uses control plane Network element
Figure 16-6. End-to-end configuration using a combination of network management and control plane functionality
Recommendation G.8080 is intended to provide a comprehensive model that takes into account various commercial relationships, organizational structures, and operational practices. The goal of the G.8080 architecture is to identify the external interfaces that must have protocols defined, while maintaining the ability to verify the architecture against operations scenarios and functional distributions that are required by the various requirements documents. Historically, the Telecommunication Management Network (TMN) Recommendations approached the problem from the point of view of objects viewed through an interface. This led to an equipment centric, rather coarse interface, which did not allow an easy distribution of the necessary functionality to the most appropriate network element. An attempt was made to improve upon this approach by using the techniques of Reference Model for Open Distributed Processing (RM-ODP), which constitutes a framework of abstractions for the specification of open distributed systems. By enabling a separation of concerns, it provided a means for separating the logical specification of required behaviors from the specifications of physical architectures implemented to realize them [16].
Architecting the Automatically Switched Transport Network
575
The application of RM-ODP to telecommunications has been specified in ITU-T Recommendations G.851-01 [31], G.852-01 [32], G.853-01 [33], and G.854-01 [34]. These techniques take the approach of viewing the desired system from the point of view of the Enterprise, Information, Computation, and Engineering decisions that have to be made. The end result is a more fine-grained collection of interfaces, each providing a simple service that can be assigned to network elements to provide a wide range of solutions, each meeting different needs. However, this end result did not lend itself very well to constructing scenarios to verify that the interfaces specified were necessary and sufficient, since a tremendous amount of system behavior actually occurs within a network element, and internal implementations are not subject to standardization. In order to avoid the verification problems of the earlier work, G.8080 created a component architecture to facilitate the construction of reasonable scenarios. In UML, a component is defined as "a physical and replaceable part of a system that conforms to and provides the realisation of a set of interfaces" [30]. A component in this sense typically represents the physical packaging of otherwise logical elements, including classes and interfaces. In the context of G.8080, a component is defined as "an element that is a replaceable part of a system that conforms to and provides the realization of a set of interfaces". The subtle difference is that a component in G.8080 represents an abstract entity rather than a piece of implementation code. Thus, in this context, components represent logical functions rather than physical implementations. With this in mind, UML can be used in describing the G.8080 architecture. The means of deciding upon interfaces used the same analysis and design techniques from the RM-ODP application to telecommunications. Components were created by considering the lifetime of the objects in the system and the span of control of the resulting component. The result is a small set of components that support a wide range of implementation choices and allow scenarios to be constructed to validate the architecture against requirements. It is important to realize that the G.8080 architecture specifies components and interfaces on a per G.805 layer network basis. In what follows, we first consider how the control plane views network resources and then consider the components that make up the control plane. We note that the architecture described in G.8080 not only applies to connection-oriented networks but also could be employed, with some modifications, in connectionless networks. This outcome could be achieved by describing the transport network using G.809, which is a flow-based version of G.805. To give a hint as to how this might be accomplished, consider a flow in the limit which consists of a single packet. This packet
576
Chapter 16
can be considered as a self-describing short-lived connection that simply uses and releases transport resources as it moves through the network. Alternatively, consider the connection setup and release process operating at a faster and faster rate. Aside from some minor changes to accommodate terminology differences between G.805 and G.809, the only major change to the G.8080 architecture would be the removal of the concept of a call, which simply becomes a null function in the architecture. With this in mind, G.8080, with appropriate modifications, can be used to describe existing and future connection-oriented and connectionless control planes. We can therefore conclude that G.8080 can be used as the basis of a control plane architecture for any transport technology.
16.3.1 The Control Plane View of the Transport Network The description of G.805 transport network functions makes no reference to the control and management of these functions. Depending upon the desired control or management view (e.g., connection, fault, performance management), not all aspects of transport network functionality are of relevance. Thus, it is necessary to abstract the particular aspects of transport network functions that contain information relevant to the specific view. From the perspective of control, the relevant view is concerned with the management of connections. Some key abstractions that are relevant to the control plane are illustrated in Figure 16-7 and enumerated below: • The subnetwork points (SNPs) that need to be associated to form a connection (these are simply an abstraction of the connection points (CPs) in G.805) • The subnetwork connection (SNC) that represents the dynamic relationship between SNPs on a subnetwork • The link connection (LC) that represents a static relationship between SNPs in different subnetworks • A set of SNPs that can be grouped for the purpose of routing, thereby forming a subnetwork point pool (SNPP) • An SNPP link, which is a link associated with SNPPs in different subnetworks. The link contains LCs formed by the association of SNPs.
Architecting the Automatically Switched Transport Network
577
y Subnetwork ,
-§MC . r ^ .
SNP Link Connection / ^
SNC
Figure 16-7. The relationship between entities in the transport plane, the management plane, and the control plane [4]
Another key abstraction that is required for the purposes of routing is the routing area (RA). An RA is defined as being composed of a set of subnetworks, their interconnecting SNPP, links and the SNPPs that represent the ends of SNPP hnks exiting the RA (ilhistrated in Figure 16-8). This setup allows links to be addressable within the RA, hence allowing for step-bystep routing. In contrast, for a subnetwork, only the ends of the link connections are visible from within the subnetwork. We note that the critical distinction of link end visibility is only important to an observer inside the routing area. From the outside, subnetworks and RAs are identical, and this causes the terms subnetwork and RA to be used almost synonymously. The distinction between the two is usually obvious from the context. In the context of routing discussions in G.7715 and G.7715.1, the term node was adopted to denote either a subnetwork or an RA. This decision was based upon the earlier definition of RA within G.8080, where the lowest limit of recursion of an RA was two subnetworks interconnected by a link. In fact, with the updated definition provided in G.8080 Amendment 2 (i.e., the lowest limit of recursion of an RA is a subnetwork), a node and an RA are considered synonymous. As a result, RAs also have the property of recursive containment similar to subnetworks. This property enables support for hierarchical routing schemes. Recommendation G.7715 defines how successive sets of contained RAs form a routing hierarchy. Routing areas are thus the key concept that matches the organization of the control plane to the organization of the transport plane. We note that the scope of the management abstractions for the CTP and TTP objects is different from those in the control plane, reflecting their
Chapter 16
578
different roles and areas of responsibility. This distinction is also immediately apparent by describing the resources that the control plane manipulates in terms of G.805 architecture constructs. Partitioning
Routing area Subnetwork SNPP SNP
•
CTP(M.310O)
9
SNPP Link
SNP Link Connedion
^
N: c>
Unk Connection .
rt
Figure 16-8. Routing Areas, subnetworks, SNPs and SNPPs [4]
Now that we have introduced the abstractions that the control plane needs in order to manipulate resources in the transport plane, we can turn our attention to the manner in which this outcome is achieved.
16.3.2 Identifying Components The RM-ODP-based methods used to construct the G.8080 architecture focus on a single aspect (called a viewpoint) of the problem and solution space at a time. The ODP viewpoints are as follows: 1. Enterprise viewpoint, which is concerned with the purpose, scope, and policies that govern the activities of the specified system 2. Information viewpoint, which is concerned with the types of information handled by the system, together with applicable constraints 5. Computational viewpoint, which is concerned with the functional decomposition of the system solution into a set of objects that interact at interfaces, thus enabling system distribution 4. Engineering and Technology viewpoints, which are concerned with the infrastructure and choice of technology used to support the system As the intent of standards is to allow interworking, rather than to proscribe implementations, G.8080 makes the most use of the Information
Architecting the Automatically Switched Transport Network
579
and Computational viewpoints and limits itself to component interfaces that are essential to distribution. The Computational viewpoint is concerned with objects (in this context, G.8080 components) and interfaces. One may wonder how the objects/components in G.8080 were identified, as every Object-Oriented design method has its own description of how to find objects. Recommendation G.8080 built on existing work from the space of distributed transport management applications, which provided a basic set of objects that support network topological aspects, and then some more general principles were applied to identify additional components. This work is further described below. Work prior to G.8080, involving centralized control via management interfaces, assigned a subnetwork performer computational object [16] to manage the subnetwork and the link. The subnetwork performer embodies all the information there is to know about the subnetwork, with the most important information being the internal structure of the subnetwork (its internal subnetworks and links), which is essential to be able to route a connection across the subnetwork. Similarly, one can consider a link performer computational object that embodies all that can be known about the link—in particular, its composition in terms of sublinks and individual link connections. This performer would also be responsible for preventing the same link connection from being allocated to more than one connection request. Recommendation G.8080 takes many of the ideas associated with these performers, rather than the performers themselves, as axiomatic. In a distributed system, there is no single platform that can support these performers as single objects. Further, it was recognized that not all aspects of a performer need to be available at the same location or at the same time. This leads to a different distribution of routing information from signaling control (and hence a different collection of components and interfaces). Recommendation G.8080 components (i.e., RC and CC, and LRM, respectively, as described in Section 3.4) reflect the distribution of performer operations, factoring in the above considerations. The services these performers offer are realised by the collaborative interactions among their associated components. A final critical consideration involves component lifetime and degree of coupling with other components (and coupling with the transport infrastructure). This consideration has led to only one component being directly involved in any aspect of the underlying transport hardware (i.e., TAP, as described in Section 16.3.4).
580
Chapter 16
163.3 General Component Properties and Special Components The component and its interfaces are illustrated in Figure 16-9. The interfaces are defined based on the operations that the component is intended to carry out and are, therefore, component specific. Recommendation G.8080 also defines some component properties, expressed as special interfaces that every component can be assumed to have, though these are not mandatory. These special interfaces allow monitoring of the component operation and dynamically set internal policies and behavior. In addition, a special class of component {Port Controller) is provided to deal with external policies such as security. For example, one role of the Port Controller component is to validate that an incoming user connection is sending traffic according to the parameters that have been agreed upon in the service-level agreement.
1.
Messages
•
Interfaces
1
< name>
Figure 16-9. The component and its interfaces
The general component model describes protocol-neutral interfaces, which exchange primitives between components. There is one exception to this in the form of the Protocol Controller component class. This class combines several primitive streams into external protocols, which enable various distributions of the components among physical platforms.
16.3.4 Component Overview The control plane architecture can be described by means of a library of components that are illustrated in Figure 16-10.
Architecting the Automatically Switched Transport Network RC
CC
PC
(a) Routing Controller
(b)Connection Controller
(c) Protocol ContioUer
"^CaSkg ^PCC
Calkd PCC
J Netwofk CaBC
(d) Calling Party Call Controller
581
(e) Called party Call Controller
(f) Network CaU Controller
^LRMA
(g) Link Resource Manager A end
(h) Link Resource Manager Z end
0A
TAP
(i) Discovery Agent
(j) Termination & Adaptation Performer
Figure 16-10. The control plane component library
The components are summarized below. 1. Routing Controller (RC) The Routing Controller component is derived by distributing some functionality of a more abstract object called the Subnetwork Performer, which has complete information about all the contained nodes and links within its RA. Routing Controllers belonging to the same RA cooperate to ensure that each RC has a complete view of the internal RA topology. This cooperation takes place via a Routing Protocol and the results are made available to other components via the routing table. 2. Connection Controller (CC) The Connection Controller component is also derived from the Subnetwork Performer. Connection Controller components cooperate to set up connections. This is done by consulting the path computation function in the RC, which then returns the set of nodes and links to be traversed in order to reach the specified endpoint. 3. Link Resource Manager (LRM) The LRM is derived from the Link Performer, which knows all there is to know about a link, by distributing functions to both link ends (LRMA and LRMZ). LRMs are responsible for managing the resources available to the link and allocating a specific link connection when requested. LRMs also cooperate to avoid two connections being allocated to the
582
Chapter 16
same link connection when the connections are being set up from each end of the link. 4. Calling/Called Party Call and Network Call Controllers (CCC, NCC) Call Controller components cooperate to control the setup, release, and modification of calls. They are relevant to service demarcation points (i.e., UNI, E-NNI). As discussed earlier, these service demarcation points are established via inter- and intraoperator policies. Such policies can be applied in several different ways; i.e., either centrally or on each switch or at signaling aggregation points. (The first and the last imply a different distribution of function from either Connection Controllers or Link Resource Managers; this differing distribution dictates that call controllers are different components from either CCs or LRMs.) Network Call Controllers (NCCs) are relevant at the E-NNI and UNI (on the network side) service demarcation points. It is the NCC that makes the choice of technology to support the service by translating service requests into technology choices. This setup meets the domain boundary opacity requirements and allows the network to be most flexible. The NCC also handles other aspects of calls, such as restoration. The restoration architecture, to be described in Section 16.3.9, supports restoration between domain edges. The need for restoration, or lack thereof, is a call property, and the activation of restoration is within the lifetime of a call. Thus, there is no need for an additional component to support restoration. The case of the end user of the network is special, and because of this two additional call controllers have been defined. The Calling and Called Party Call Controllers (CCC) are relevant to the user-provider service demarcation points and are the components that access the network on behalf of the end user of the service. 5. Protocol Controller (PC) ASON components are defined on a per-layer network (G.805) basis and, where appropriate, at a single level in the routing hierachy. They communicate over abstract interfaces using primitives, so called to distinguish logical communications between component interfaces and the communication via an implemented protocol over physical interfaces. Protocol Controllers shield the components from any protocol details that are irrelevant to the component. (An example would be reliable message transport; the component assumes it, and a protocol controller provides it. Another excellent example is related to providing a layered security architecture. Security and authentication are provided by Protocol Controllers, not by the components themselves.) Protocol Controllers also allow primitives from several components to be merged into a single
Architecting the Automatically Switched Transport Network
583
message stream, thereby allowing implementations that handle as many layers and levels as is useful. 6. Discovery Agent (DA) The Discovery Agent deals with network resources that have not yet been assigned to a layer network. (An example could be cross-connects that can switch a wide range of signal types). The DA is derived by distribution of an abstract object that knows about all the uncommitted resources and learns, or verifies, how they are interconnected. After this learning/verification, the resources can be assigned to the desired layer network or link. 7. Termination and Adaptation Performer (TAP) All networks are ultimately supported by physical equipment, which needs to be controlled at some point in its lifetime. However, it is not necessary for ASON components to know anything about the hardware supporting the network, as ASON operations are hardware independent. The TAP is the only ASON component that understands hardware and must therefore be collocated with that hardware. The role of the TAP is to hide the details of physical equipment operation from ASON. Examples of hidden operations include the adjustment of the adaptation function when hardware capable of supporting several signal types is used, and the suppression of alarms when a link connection is left intentionally unused. We note that there is also a Traffic Policing component (TP), which is a subclass of the Port Controller component described in Section 16.3.3. When an incoming user connection sends traffic that violates the agreed parameters, the TP may instigate measures to correct the situation. Traffic policing is important when dealing with packet switched networks, and is included in ASON for completeness. However, it has no function in conventional circuit switched networks.
16.3.5
Interlayer Modeling
As discussed earlier, the G.8080 architecture specifies components and interfaces on a per-G.805 layer network basis. For a transport network that supports multiple adaptations, an ASON instantiation could logically contain multiple ASON control planes, one for each layer network. A client with a UNI interface could request different layer services from the same UNI implementation. In such a scenario, there is no dependence between calls requesting connection services at different layers. On the other hand, transport services exist where the client layer has no resources in the network except at the edges. An important example of this scenario is when
584
Chapter 16
Ethernet traffic is carried across a SONET/SDH network, in which there are no Ethernet switches or Ethernet Hnks, on behalf of a cHent that has requested Ethernet service. In addition to the Ethernet/SDH chent/server example, the interlayer model also applies to the relationship between a layer network that supports virtual concatenation and its server layers. Thus, it is important to be able to model the associated interlayer interactions that must be supported in such scenarios. The above scenarios have been addressed in recent G.8080 developments related to the extension to the Network Call Controller (NCC) component to include an interlayer interface that enables it to have a relationship with that server layer call. This relationship is recursive so that a set of related adaptations are formed. In other words, the NCCs display a recursive G.805 client/server layering relationship. This characteristic is analogous to the stack of adaptations represented by the TMF 814 PTP (Physical Topological Link Termination Point) construct [35]. This setup may be viewed as creating a "stack" of NCCs at different G.805 layers. Wherever an adaptation occurs in a stack of adaptations, an NCC at that layer is created. The decision to use an interlayer NCC interface is driven by policy, as there may be choice regarding which server layers to use. Figure 16-11 illustrates a two-layer example. In this example. Layer A does not have a connection between its NCCs because that transport layer is not present between them. Instead, an adaptation to a server layer (layer B) exists. Associations (labeled 1 and 2) between NCCs at the two layers exist to correlate the service at layer B being used by the client layer A. The model can be generalized to supporting multiple client NCCs with a single server NCC.
Layer B call segment ^ - ^ connection Figure 16-11. Layered Network Call Controllers
The interlayer interface to the NCC enables a client NCC to initiate the relationship with a server NCC or vice versa. When a server NCC initiates
Architecting the Automatically Switched Transport Network
585
the relationship, it presents a pair of SNPs (ends of a network connection) that can be used by the cUent layer for transferring client CI. The connection presented is able to transfer client CI, and no call action at the server layer is initiated. This process is used for an operation where a server layer has already established a call and this connection is presented to the client layer at a later point in time. The client layer may accept or reject the use of the offered SNP pair (connection). This model accommodates the business scenario where the adaptations occur in a single administrative domain as well as in multidomain scenarios (e.g., a scenario in which each layer network is operated by a different carrier). In the latter case, the NCCs may be on different platforms, and the interface between them may need to be exposed. In both cases, the instantiation of the NCCs is still independent on a per-layer basis. For example, a server layer may have a centralized NCC, whereas the client layer may have distributed NCCs. Other components do not require interlayer interactions because once an NCC determines that resources from its layer are to be used to support the call, subsequent actions are taken only within that layer (especially connection control and routing control). This setup confines interlayer knowledge and actions to the NCC. Note that this process differs from sending information about multiple layers in a protocol controller that serves multiple layers (e.g., routing) because the interlayer call model maintains a client/server layer relationship, whereas sending multiple layer information over an interface does not imply that the information between layers is correlated in any way.
16.3.6 Distribution models The G.8080 component architecture identifies components in such a way that the most commonly used distributions of functionality are supported. However, before discussing actual distributions, it is useful to discuss architectural principles that result in some components being fixed. These components are called anchored components (or anchored objects in other contexts). As G.8080 provides the foundation for specification of external interfaces needing standardization, an important principle is that of reducing the number of different interfaces, as well as simplifying those interfaces. For example, we see that the CC provides the same interfaces regardless of the size of subnetwork being controlled. Thus, rather than creating a completely new interface to a switching element, we simply note that the lowest level CC is anchored to the switch, i.e., it is fixed in the switching equipment. All other connection controllers can be freely distributed anywhere in the network. Similar arguments apply to the DA and TAP
586
Chapter 16
components, which are similarly anchored to the equipment they are responsible for. These decisions reduce interface variation and complexity. The architecture is designed to support the independent distribution of routing, switch operation, link control, and call control. A wide range of system designs, ranging from almost all functions being centralized to almost all functions being fully distributed, is possible using the same architectural components and standard interfaces.
16.3.7 An Example of Components in Action So far we have limited our discussion to the stmctural modeling of the control plane by identifying the types of component that are of interest and their interfaces. Describing the associations between components can identify further structure. This modeling of the static aspects of a system allows us to describe and specify the things that make up the control plane. UML provides tools for achieving this by means of class diagrams and object diagrams. However, what is really of interest in the G.8080 architecture is the interaction of control plane components with one another. This interaction occurs as a result of messages being exchanged between a group of components to achieve some defined purpose. An interaction and messages can be formally defined as follows [30]: "An interaction is a behavior that comprises a set of messages exchanged among a set of objects within a context to accomplish a purpose. A message is a specification of a communication between objects that conveys information with the expectation that activity will ensue". UML allows the dynamic aspects of the system to be described using interaction diagrams. This form of diagram allows several forms of action to be modelled as indicated in Table 16-1. Message Type Call Return Send Create Destroy
Table 16-1. Messages and actions Action Invoking an operation on a component. A component may send a message in the form of a call to itself. Returns a value to the requestor Sends a signal to a component. A signal is a named object that is sent asynchronously, e.g., an exception Creates a component Removes a component
(Note that the meaning of call in UML is in the context of UML as a modeling language and not as described in the network context.)
Architecting the Automatically Switched Transport Network
587
Interactions may simply be between two components. Alternatively, a message transmitted from a component to a second component may result in the second component generating a message that interacts with a third component and so forth. In this case, it is often useful to include information regarding the sequence of the messages. Such interaction diagrams in UML are referred to as sequence diagrams. An interaction may be between components of the same type or a set of components that contains different types of components. An example of the former in the control plane is the exchange of routing information between Routing Controllers. This is illustrated in Figure 16-12. Messages are exchanged via the Network Topology interface on each RC, and this information is used to configure routing tables with network topology information and network topology update information. An example of different components interacting is in the setting up of a connection. A simplified example is illustrated in Figure 16-13. The reason why we simplified the discussion is that connection setup behavior actually depends on the means of routing, e.g., hierarchical routing, step-by-step or source-based routing. G.8080 describes these interactions in detail. In our simplified example, the role of the RC is different from that of the previous example, and the interaction with the component is by means of a different interface.
r~ r~n ^^
\^
P>
RC
'v ' — T — '
>k
>k^ Broadcast local route table to nearest neighbors
>f
^VK >f "v ^—T—'
i n
Hr
^^
^ ^
RC
Figure 16-12. Exchange of routing information
The sequence of events is as follows: 1. A connection request arrives at connection controller a (CCa). 2. The CCa component queries the routing component (RC) by means of a Route Query request. The RC returns the outgoing link to be used.
Chapter 16
588
3. The CCa component then interacts with the Link Resource Manager to allocate SNP link connections. The LRM responds with acceptance or rejection of the request. 4. Once the CCa component receives confirmation from an LRM that the connection request has been accepted, a subnetwork connection can be established across the subnetwork controlled by the connection controller. The remaining parts of the sequence show the flow of confirmations that the connection has been set up. This continues until the confirmation is returned to the original user. The above is, as stated above, a much simplified version of an interaction provided for illustrative purposes. Recommendation G.8080 provides more detailed interaction diagrams for connection setup and call control. However, G.8080 does not describe all possible interactions between components. This is intentional, as more detailed interactions are protocol specific and can be described in recommendations derived from G.8080.
RC
2. Route Table Query]
. Connection Request 4. Connection Request N+1. Connection Confirmation
3a. SNP Link Connection Request
N. Connection Confirmation
CCb
3b. SNP Link Connection Request
LRM
Figure 16-13. Setting up a connection
16.3.8 Identifier Spaces When working with a distributed control plane, a consistent set of identifiers needs to be developed for use in the configuration and operation
Architecting the Automatically Switched Transport Network
589
of the signaling and routing protocols. Recommendation G.8080 has recognized four categories of identifier spaces that are used in the ASON architecture, specifically: • Identifiers for transport resources used by the control plane • Identifiers for control plane components • Identifiers locating control plane components in the DCN • Identifiers for transport resources used by the management plane Before discussing these identifiers and their use, it is necessary to define the two types of identifiers that exist, specifically names and addresses. Names are identifiers used to reference an instance. It should be understood that more than one name may exist for an instance, and a particular name may be only valid within a specific scope. Moreover, names for an instance are allowed to change. However, since names do not imply any specific structure of a set of instances, they are not required to be summarizeable. Addresses are identifiers that locate an instance in a topology. While addresses may be composed of names, whenever a name used as a component in an address changes, the location in the topology does not. Since addresses are defined in terms of locations in a topology, they are inherently summarizeable. This means that there is a common prefix for all instances located within the common part of the topology. Two examples of a name are the number for a mobile phone and freephone (or "1-800") numbers. In each case, a mapping is required from the name to an address. This can be accomplished with a directory function. Addresses and names have a scope associated with them, and the larger the scope, the more elements are needed in the identifier itself. At the highest scope (i.e., global), the identifier is complete in the sense that no further information is needed. Within a smaller scope, the identifier is relative to that scope and does not have the same meaning outside of that scope. For example, global telephony numbers include country codes whereas within a city, shorter numbers may be used but are relative only to that city. We observe that the terms name and address were not cleanly distinguished in standards documents through 2004, and consequently there is inconsistent usage of these terms. In the text below, we will use the term identifier unless the distinction is critical to the understanding of the discussion. It should also be emphasized that the syntax of an identifier format does not imply that the identifier is a name or address. For example, usage of an IPv4 syntax for a particular identifier does not imply that it represents an IP address. For example, when such syntax is used to identify transport resources, these identifiers are clearly not IP addresses. The various identifier spaces, and their relationships, are illustrated in Figure 16-14. Here, the transport plane resources are multiply identified by
590
Chapter 16
name spaces in other planes. The 0AM identifiers in the management plane include equipment names and CTP names. Within the control plane, both UNI/E-NNI Transport Resource and SNPP identifiers are also applied to transport resources. DCN (Data Commimications Network)
Views of transport plane resources
Transport Plane Resources
Figure 16-14. Identifier spaces and relationships [14]
16.3.8.1 Transport Resource Identifiers Transport resource identifiers are used by ASON control components to refer to G.805 transport plane resources. Two such identifiers are used [4, 36, 37]: UNI/E-NNI Transport Resource and SNPP. UNI Transport Resource identifiers are used to identify transport resources at a UNI reference point if they exist. Similarly, E-NNI Transport Resource identifiers are applied to transport resources at an E-NNI reference point. They represent the resources between the client and network (or between networks), not the transport network endpoints. These identifiers were referred to as UNI Transport Resource addresses in the 2001 version of G.8080, and as Transport Network Assigned (TNA) addresses in OIF UNI 1.0 [36]. However, the context of their usage indicates that they are, in fact, names that the calling party call controller and network call controller use to specify destinations in making a call (see Section 16.8). This was recognized and reflected within OIF E-NNI 1.0 [37] and G.8080 Amendment 2, i.e., the term UNI Transport Resource name is used in this context.
Architecting the Automatically Switched Transport Network
591
SNPP identifiers provide a link context for SNPs and are used by the control plane to identify transport plane resources. It is important to note that control plane component identifiers cannot be used as SNPP identifiers because they are from the wrong space. The G.8080 architecture allows for multiple SNPP identifier spaces to exist for the same transport resources. An SNPP address must be unique within the RA terminating the SNPP hnk. In general, an SNPP address is the concatenation of the names of any enclosing RAs, the lowest subnetwork, and any additional link context. This scheme allows SNPs to be located at any routing level. The SNP address is derived from the SNPP address concatenated with a locally significant SNP index. Depending on the scope of an SNPP, not all elements of the address are needed. For example, an SNPP address within the scope of a matrix (i.e., the smallest subnetwork) may have just the matrix identifier and a link identifier. An SNPP address at the top of the routing hierarchy may have just an RA identifier, and an SNPP address in the middle of the routing hierarchy may have a sequence of enclosing RAs plus a link identifier. UNI/E-NNI Transport Resource names are distinct from SNPP addresses because of the G.807 constraint that users should not be given internal network addresses. UNI Transport Resource names must be bound to SNPP addresses in order to enable routing across "the network" between transport resource names at the A-end and Z-end of a call. This binding may be changed without changing service aspects. In order for connection management to establish a connection to a destination UNI Transport resource name, an address resolution function is needed to map it to a corresponding far-end SNPP address (or addresses). The relationship between UNI/E-NNI Transport Resource names and SNPP addresses may be any of the following: • 1 :n — One UNI/E-NNI Transport Resource name may map to multiple SNNP addresses at the same reference point. • n:l — Many UNI/E-NNI Transport Resource names may refer to one SNPP address at the same reference point. • m:n — Many UNI/E-NNI Transport Resource names may refer to multiple SNPP addresses at the same reference point. The first two cases are illustrated in Figure 16-15.
Chapter 16
592 UN I Transport
AGC
SNPP1 Resouice 1
m
UNI-C
MatTK 1 (NeVrork Element)
SNPP2
a) Multiple SNPPs to one UNI Transport Resource
Anr.
UNI Transport Resources 1, 2
UNI-C
*
Matrki (Network Element)
SNPP1 b) Multiple UNI Transport Resources to one SNPP
Figure 16-15. SNPP and UNI Transport Resource relationships
16.3.8.2 Control Plane Component Identifiers Control plane components also require separate identifier spaces, since they may be instantiated differently from each other for a given ASON network. For example, one can have a centralized NCC with distributed CCs. Thus, separate identifiers are needed for RCs, NCCs, and CCs. Additionally, the PCs that are used for protocol-specific communication also require a separate identifier space. For example, the identifiers for the Signaling PCs must be unique in order to unambiguously specify a particular signaling channel [37]. 16.3.8.3 Data Communications Network Identifiers To enable control plane components to communicate with each other, a Data Communications Network (DCN) is used (described in Section 16.4). DCN addresses identify the points of attachment for the signaling and routing PCs that instantiate control plane communication functions (generating and processing messages in protocol specific formats). We note that several PCs may share a DCN point of attachment, and any given network element may have multiple points of attachment. For example, the signaling PC DCN address refers to the point where the signaling PC attaches to the DCN. Thus, the signaling PC DCN address is based on the topology of the DCN carrying signaling messages, rather than the topology of the transport plane or control plane components.
Architecting the Automatically Switched Transport Network
593
16.3.8.4 Management Plane Identifiers These identifiers are used by management entities that are located in Element Management Systems (EMSs) and Network Management Systems (NMSs). Identifiers used for OAM purposes include those defined for the TTP (Trail Termination Point) and CTP (Connection Termination Point) [38], illustrated in Figure 16-7. TTPs represent the signal state as a signal leaves a layer network and are associated with the G.805 termination function, while CTPs represent the signal state as it enters a layer network and are associated with the G.805 adaptation function. Existing operations, administration, and maintenance (OAM) address spaces generally describe a physical locality that supports maintenance and fault correlation activities.
16.3.9 Restoration Architecture There are several techniques available to enhance connection availability, which refers to the ability of the connection to provide service even though there is a fault in the network. Recommendation G.805 describes transport network availability enhancement techniques. The terms protection (replacement of a failed resource with a preassigned standby) and restoration (replacement of a failed resource by rerouting using dynamically allocated spare capacity) are used to classify these techniques. In general, protection actions complete in the tens of milliseconds range, while restoration actions normally complete in times ranging from hundreds of milliseconds to up to a few seconds. For G.8080, however, the mechanisms supporting the technique by which the connection is restored are far less interesting than whether the control plane is engaged in restoring it. Recommendation G.8080 therefore extends classical protection and restoration definitions to classify protection as any mechanism that is autonomous and requires no control plane intervention (no rerouting). Similarly, restoration is classified as any mechanism that is operated by control plane actions since these operations always involve rerouting the connection. In principle, rerouting can occur over any portion of the network, and it intuitively feels best to only replace the failed component (link or node). However, it is not always easy to determine which component has failed in a timely manner; neither is it easy to determine which points should switch in response to the failure. It is also advantageous to subdivide a large network into a number of independent recovery domains. Different mechanisms, which are appropriate to the topology and capabilities of the equipment deployed, may then be used in each domain. In this manner, a clean separation is provided between the networks of different operators or
Chapter 16
594
between work forces within a single network, and the availabihty is improved when the size of a restoration domain is Hmited. G.8080 has adopted the ATM Forum approach [39] towards domainbased rerouting. The G.8080 rerouting domain model is illustrated in Figure 16-16.
Rerouting Domain
Destination Node
Rerouting Domain or /<
E-NNI
I
Routing Area
Routing Area
Figure 16-16. G.8080 Rerouting Domain Model
The restoration architecture is static and defined by the points at which restoration action occurs. These points are the ingress and egress edges of the rerouting domain. The egress edge detects the failure and coordinates rerouting with the ingress edge. A rerouting domain supports a single recovery mechanism, and each recovery domain is responsible for maintaining the integrity of the portion of the call that transits that domain. Recovery domains may be nested; thus if the innermost recovery domain cannot maintain the integrity of the call segment, an enclosing recovery domain (with larger scope) may attempt to recover the call. A policy application is used to map the Class of Service for the call onto the types of recovery domains that may be used when routing connections. A general network offering restoration is thus a concatenation and/or nesting of rerouting domains, and the innermost domain is responsible for clearing the fault. While the rerouting domain has a specific purpose, it is clear that it must be possible to route across a rerouting domain. In practice, this means that a rerouting domain boundary must be coincident with, or contained within, an RA, and rerouting takes place entirely within the domain. This model allows for a very flexible network design, in which the scope of the restoration problem is determined by the design of the network. This architecture supports two forms of restoration. The hard rerouting service offers a service survivability mechanism for calls and is always in
Architecting the Automatically Switched Transport Network
595
response to a failure event. The soft rerouting service provides a means to reroute a connection for administrative purposes (e.g., path optimization, network maintenance, or planned engineering works). The latter operation is generally triggered via a request from the management plane and sent to the egress rerouting components. The rerouting components establish a rerouting connection to the ingress components. Once the rerouting connection is created, it is taken into service and the initial connection is deleted. This provides make-before-break switchover and ensures that the service interruption is limited to the switchover time of the tail-end switch. Of particular interest is the failure of the link between two rerouting domains. This type of failure may be handled by providing the interdomain link with an autonomous protection mechanism, by enclosing the interdomain link and its end switches in its own rerouting domain, or by enclosing both domains and the link in an enclosing scope rerouting domain. Which solution is chosen depends on many issues that are outside the scope of this chapter.
16.4.
SIGNALING COMMUNICATIONS NETWORK ARCHITECTURE (G7712)
Transport network elements already support data communication functionality for enabling transport of management messages between network elements and their managers (e.g., EMSs, NMSs). One consequence of distributed connection management is the need to support distributed signaling communications, i.e., the need for a mechanism to transport the signaling messages between communicating ASON nodes. Since new control plane capable network elements will need data communication functionality for both management and signaling applications, we consider each application as running over separate logical networks, i.e., the logical Management Communications Network (MCN) and SCN, respectively. The term DCN will be used to represent the physical communication network supporting the logical MCN and SCN. The logical MCN and SCN may be supported via physically separate DCN networks, or a single DCN may support both the logical MCN and SCN as two applications sharing the same communications network. Recommendation G.7712 provides requirements and architecture specifications for a Data Communications Network that supports the Internet Protocol (IP), including support for ASON signaling applications. Version 1 of G.7712 provides requirements for a DCN providing connectionless services. In developing G.7712, it was understood that the current Embedded-DCN is OSI based, and therefore interworking requirements
596
Chapter 16
between IP and OSI must also be specified in the Recommendation. Version 2 of G.7712 provides requirements for connection-oriented services as needed for certain applications (e.g., ASON signaling).
16.4.1 Signaling Methods There are two primary approaches to the transport of signaling messages, which are referred to as in-band and out-of-band. Each approach can be used exclusively of the other. Alternatively a network design may include both approaches e.g. in-band methods in the access network and out-of-band in the core network. The choice of approach is a major determinant in the design of an SCN. 16.4.1.1 In-Band approaches In-band signaling is defined as meaning that signaling messages are carried on the same channel as the user information. Examples of in-band signaling are TCP/IP and signaling on analogue access links in telephone networks. In-band signaling implies some degree of "intertwining" between user and signaling information. This allows the signaling channel to be used as a proxy for the health of the user traffic. With in-band signaling, the signaling messages follow the user traffic. A disadvantage of this approach is that it is not inherently secure. 16.4.1.2 Out-of-Band approaches In contrast to in-band signaling, out-of-band signaling is defined as meaning that signaling messages are carried in a separate signaling channel. An example of out-of-band signaling is SS7 in telephony networks. Out-of-band signaling can be further subdivided into two categories; channel-associated signaling and common channel signaling. In the former, there is a direct association between the signaling channel and the user information channel, while in the latter the signaling channel is shared between user information channels on a demand basis. Once signaling is taken out-of-band it can no longer act as a proxy for the health of the user information. The reason is that the signaling and information channels are now subject to independent failure mechanisms — a fault in one channel does not imply a fault in the other. A consequence of out-of-band control is that operations administration and maintenance (OAM) has to be built into the traffic units that transfer user information to validate the integrity of information transfer.
Architecting the Automatically Switched Transport Net^A^ork
597
Many signaling systems utilize out-of-band common channel signaling because of the following features: • Efficient use of resources as signaling capacity is utilized on a demand basis by the user information channels. • The signaling protocol can be developed independently of the user channel. • When setting up a call, signaling messages may be transmitted at the same time as the establishment of the user information channel, allowing for shorter setup times. This is not possible with channel associated signaling. • The signaling network is a separate network. This fact allows the signaling network to use the same topology as the transport network or a separate network topology can be used. Both forms of topology can be combined. Signaling and management communications can be logically out-of-band although sharing the same facilities as the data. Examples include: - Embedded communications channels in SDH. Although they are in the same frame as customer traffic, they are part of section overhead rather than payload. An ECC can be misconfigured or can fail separately from the customer traffic. It is not possible for the customer traffic to modify control traffic in the ECC. Traffic that uses the ECC may also use a separate DCN where appropriate - Optical supervisory channels (OSCs) where signaling traffic is carried on a separate wavelength from data traffic. There is no reason for the protocol structure of this wavelength to represent that of the other wavelengths
16.4.2 Delivery of Control Plane Messages The Data Communications Network (DCN) may be composed of various facilities to support the exchange of control plane messages, as illustrated in Figure 16-17. These include Embedded Communications Channels (ECCs) as well as separated dedicated facilities. • The physical transport facility may have an Embedded Communications Channel (ECC), such as the SONET/SDH network's DCC, to carry these messages, removing the need for additional facilities to be deployed besides the physical transport facility. • A separate facility may be deployed between the service provider and the customer, such as a leased facility (e.g., Tl or Ethernet), to carry the control plane messages. The DCN interconnects the various facilities via data communications equipment, allowing network elements connected via a transport link to
Chapter 16
598
communicate over a multihop path that can be different from the transport Hnk.
a)
/11 m t|___.
lii iirf \m ffil vietwork EElement
1
b)
/
"
~~'
/l i i f ^ i 1
^i i i f g i
»Jetwork EElement
Control Plane Message Facility Transport Facility (e.g. SONET/SDH)
Network Element
^ . -'"'SCN
m mi/
1
m mi
Network EElement
/Hi'*"I^ mi
ECC (e. g. SONET/SDH DCC) Transport Facility (e.g. SONET/SDH)
ill • \m\1
m m\i
c)
1
PI 'W{1 ffi ml Network Element
^
^
/illill^'^131 ^
'1 1
—
—
SCN Access Facility — Transport Facility (e.g. SONET/SDH)
m »|
sletwork EElement
Figure 16-17. Example of control plane message facilities
Whether the DCN for the control plane uses an ECC or a dedicated facility, or some combination thereof, is dependent on • The type of physical transport network elements • The locations of these network elements • The level of separation that is desired between the physical transport network and the DCN For example, some transport network elements, such as photonic crossconnects, may not have access to ECCs and therefore must utilize either LAN or WAN interfaces (in the form of an optical supervisory channel, a wavelength dedicated to carrying control and management information) to carry ASON signaling messages. Other transport network elements, because of their location in the network, may not have direct access to a LAN or WAN network and therefore will utilize ECCs to carry ASON signaling messages. Additionally, if it were desired that a DCN be physically separate from the physical transport network (possibly to prevent a single failure impacting both the physical transport network and the ASON signaling network), then only LAN or WAN interfaces would be utilized to carry ASON signaling messages.
Architecting the Automatically Switched Transport Network
599
In order to use a DCN, ASON-capable network elements must support data communication functionality to enable transport of the signaling messages. Such communications functionality consists of • Learning the ASON DCN topology so that an element can calculate the shortest path between itself and the ASON signaling destinations • Creating a forwarding table based on the shortest path calculations • Forwarding packets between ASON DCN interfaces based on the forwarding table entries
16.4.3 DCN Topologies When designing a DCN, a number of different topologies may be considered. This section reviews various DCN topology approaches, including pros and cons. 16.4.3.1 Full Mesh A full mesh, as shown in Figure 16-18, is the simplest topology. However, as the network increases in size, the number of links required to create the mesh grows exponentially. Each link adds capacity to the DCN that will likely be underutilized, since signaling messages are communicated between the Signaling PCs at DCN link ends. Furthermore, since routing topology updates are flooded among RCs in an RA, a mesh topology actually amplifies the number of routing messages needed.
Figure 16-18. Full mesh topology
Consequently, a full mesh topology is not recommended for a DCN.
Chapter 16
600 16.4.3.2 Congruent Topology
A congruent topology, as shown in Figure 16-19, maintains a DCN link in parallel to each transport link; i.e., the overall DCN topology is congruent with that of the transport network. In this topology, messages sent from a signaling PC at one DCN link end may flow to the signaling PC at the other link end using any available path, including signaling channels on other DCN links in the network.
—• Transport Link - - Routing Adjacency 0 = Transport Node 0 = Routing Controller Figure 16-19. Congruent topology [11]
In addition to making it hard to predict the path that will be used for signaling and routing messages, this topology also suffers from underutilized links, since each facility adds a new signaling channel. 16.4.3.3 Focused topologies (i.e., hub/spoke) A focused topology, such as the hub and spoke topology shown in Figure 16-20, maintains one DCN link from each node to a centralized DCN message switch. Since all signaling messages sent by a node are focused onto a single link to the centralized message switch, it is easy to predict the amount of messaging that a DCN link will see. Consequently, this network topology allows the best prediction of what sort of loads a network can sustain without becoming congested, even in the face of DCN link failure.
Architecting the Automatically Switched Transport Network
\\£il.
601
7/ ^ ^ ^-^--^ r^^^^-"""'^ _ ^ ^ ^ ^ ^ ^ ^ ^ J Central Message Switch — Routing Adjacency 0 = Transport Node 0 = Routing Controller
Figure 16-20. Hub/spoke topology [11]
This topology is generally not used, however, since the failure of a DCN link will cause a node to become completely isolated from the DCN. Instead, a modified form with two hubs, as shown in Figure 16-21, is typically used. This provides a secondary path to each node in case the primary fails.
I Central Message Switch — Transport Link ~ " Routing Adjacency ©= Transport Node 0 = Routing Controller
Figure 16-21. Dual hub/spoke topology
In order to guarantee that nodes will be able to send messages to each other even when one of the links to the central message switch is down, a DCN link is included between the central message switches. 16.4.3.4 Hybrid (Multitier) Topologies In large geographically dispersed networks, it may not be reasonable to maintain only one pair of central message switches. In this case, regions may have their own pair of switches, with a backbone of connections between the message switches. Figure 16-22 shows such a network
602
Chapter 16
topology. Within each region, a dual hub/spoke topology is maintained. However, between regions, a mesh network is maintained.
Figure 16-22. Hybrid (two-tier) hub/spoke—full mesh network
16.4.4 DCN Reliability Considerations Whether the DCN supporting the SCN provides connectionless or connection-oriented services has an effect on the level of reliability that can be supported. Failures affecting the DCN will impact: • New connection requests, since the signaling network may not be available to carry the messages related to the new connection request • The ability to tear down existing calls, since the signaling network may not be available to carry the messages related to tearing down an existing connection • The ability to restore existing connections, since the signaling network may not be available to carry messages related to restoration. (We note that this only applies when a failure exists on the signaling network as well as the on physical transport network) To allow ASON signaling messages to be delivered to their destination even in the event of DCN failure conditions, certain design requirements are imposed on the DCN. At a minimum, the DCN should be designed to provide diverse paths between any two communicating control plane network elements. Assuming such a design, even a DCN providing connectionless services will be able to deliver ASON signaling messages to their destination in the event of a failure once the routing tables have been updated to allow packets to route around the failure. Such a design may be sufficient for a DCN, carrying messages related to new connection requests and tearing down existing connections. However, whether this type of
Architecting the Automatically Switched Transport Network
603
design is sufficient to handle messages related to restoration of existing connections depends on the overall restoration time requirements. It is possible that the time required for the DCN to update its routing tables so as to route packets around the failure may negatively impact the ability to meet certain restoration time requirements. If such is the case, it may be necessary for the DCN to provide connection-oriented services that allow the DCN to more quickly route packets around the failure.
16.4.5 DCN Security Considerations There are certain security measures that must be taken when providing a control plane message facility to a network element that is outside of a service provider's administrative domain. The service provider must make sure that the facility does not provide access to private/secure data carried within the service provider's DCN (e.g., management data). If an ECC is used to provide transport of control plane messages, the transport network element terminating the ECC must guarantee that only the control plane messages are allowed across the ECC. We note that the transport network element may also be providing DCN communications in support of management applications and, therefore, the transport network element must be able to separate the management communications from the control plane. If a DCN is used, a firewall is necessary at the edge of the service provider's domain. The firewall is provisioned to only let control plane messages in and out of the service provider domain.
16.5.
SERVICE ACTIVATION PROCESS ELEMENTS
This section outlines the elements involved in customer call activation in a multidomain network, providing a framework for the sections that follow. These involve customer contracting, network planning and installation, and service validation and activation [2]. Note that while an SC example is utilized, these elements have applicability to an SPC as well. Customer contacting with a service provider for a set of transport services may encompass such aspects as contract duration, billing methods, service capabilities, routing and authentication policies, customer premises equipment requirements, registration and connectivity needs, and security considerations [2]. These aspects are not addressed by ASON Recommendations. Network planning and installation encompasses allocating sufficient transport resources to satisfy the terms of the contract, including the installation of new equipment as needed, and provisioning and configuration
604
Chapter 16
of the equipment (including identifier assignments). We note that automated discovery (see Section 16.6) is considered part of the network planning and installation phase and is performed after the processes related to partitioning of network resources for control plane actions has taken place [2]. Service validation assures, for example, that a UNI-initiated call request is authorized based on the contract and confirmation of the successful completion of the request. Request validation activities may include authentication of the user request, message-integrity verification, SLA verification, etc. Once this occurs, the user is authorized to request activation and release {call requests and call disconnects) of network services [2]. Service activation for completion of the user service request includes the detailed processes required to activate service between the customer locations via an automatic end-to-end connection setup. Routing information is exchanged over I-NNI interfaces, and reachability (or summarized routes, if available) is typically exchanged over E-NNI interfaces. Service activation also generally involves ensuring that adequate resources are available to support the requested service, and that the constraints placed by the request, based on SLA parameters (e.g., a certain level of availability), can be met when determining a route to reach the destination end-user across the various domains. The information exchanged between the call and connection controllers ultimately results in approval of the request and activation of the service [2]. Upon successful establishment of service, the call controller also starts the billing and associated business processes. When the user wants to terminate the service request, a disconnect/hang-up request is placed at the UNI, and the call controller stops the billing and transfers control to the connection controllers for tearing down the connection and freeing up the allocated network resources [2].
16.6.
DISCOVERY (G.7714)
The essence of automated discovery, as applied to transport networks, is for each pair of connected network elements to find out the identity of their neighbor element, to determine how their respective ports are mapped to each other, and to negotiate the services (transport entity capabilities) that will be supported across the transport entities interconnecting them. We note that for bidirectional links (which implies that the transmit and receive endpoints of the paired unidirectional links are identified by a single address) the discovery procedure is performed separately for each direction of the link. This separation is necessary because it is possible that within the fiber
Architecting the Automatically Switched Transport Network
605
pair that should make up this bidirectional link, one fiber could be connected properly while the other is not. The connectivity information derived from discovery is crucial for accurately building the network topology database used for computing the path for a connection. Additionally, the discovery procedure is an essential first step in establishing logical connectivity between control entities for exchanging signaling and routing messages. Discovery may also seed the process of establishing the control associations between routing and signaling functions managing the neighboring network elements. Without these associations, the ASON control plane will not be able to use the discovered links. The output of the automatic discovery process may be used by both traditional management systems and by the control plane. While it is possible to manually provision topology information, use of a manual approach is labor intensive, time consuming, and notoriously prone to human error. Additionally, manual provisioning makes it difficult to synchronize changes to network resources with management system databases. Without an automated mechanism such as discovery, these systems can easily lose synchronization with each other, resulting in the inefficient use of network resources or, worse, the systems not being able to compute paths for a connection. The G.8080 foundation for discovery architecture was not mature at the time of the first release of G.7714, which focused upon describing categories of discovery and methods, but has been incorporated into the 2005 version.
16.6.1 Discovery and Connectivity Verification Although they have different connotations, the terms discovery and connectivity verification are often used synonymously in the industry. Connectivity verification is generally associated with the carrier's operation systems and is, in fact, a separate process. Connectivity verification includes the important aspect of verifying the carrier's connectivity plan. Discovery, on the other hand, only determines the actual connectivity (and not what it should have been), and so does not directly verify the carrier's connectivity plan. There is thus a need for external intelligence (e.g., a management function) to take the results of the discovery process and check them against a connectivity plan. The latter is provided by connectivity verification. It should be noted that the basic discovery messages can only be carried in-band. This is unlike all the other control plane messages (e.g., signaling and routing), which can be carried either in-band (i.e., as a part of the data traffic) or out-of-band. In contrast, connectivity verification, when automated, can be carried either in-band or out-of-band.
Chapter 16
606
16.6.2 Discovery Architecture The G.8080-based discovery architecture involves the TAP, DA, LRM, and the protocol controller, as illustrated in Figure 16-23. Discovery is about finding the CP-CP connections across an entire network. In order to reduce the search space, hints may be provided based on previously discovered or configured information. For example, a trail may provide a hint about all its LCs, reducing the search to just the ends of the trail.
^ T A P V - ^ 1 Unit hints •PC
[
ConfigVLRMy
NIMHintK.,^^^ / \ CTP/LC hintr~~~VrAP >
(DA)W
(LKUT
rrApV-"^
Figure 16-23. Components of discovery architecture
The involved components and their roles are as follows: Discovery Agent (DA). The DA provides the necessary coordination for the discovery process. This includes collection of hints from the necessary components and coordinating with the DA on equipment matrices that may be controlled during the discovery process. Note that fully transparent switches cannot be bound to a particular layer network until discovery is complete. (This is because the characteristic information is determined by the trail supporting the discovered link connection). Because DAs may interact with other DAs in the network, it is important to recognize that DAs need identifiers having global scope. Termination/Adaptation Performer (TAP). The TAP provides a view of the status of the physical resources, e.g., link connections and trails (CTP and TTP). Since the TAP is associated with the G.805 termination/adaptation compound function (described in Chapter 2), it can be used to provide hints from the test-signals (the test set, CTP/TTP information) and from nonintrusive monitors (e.g., nonintrusive SNC monitors). We note that discovery operates on transport resources before control plane aliases (SNPs) have been allocated, so it uses CP and TCP names to discover CTP-CTP LCs. Link Resource Manager (LRM). The LRM provides the status of the link in terms of the allocated and available number of LCs. After the discovery
Architecting the Automatically Switched Transport Network
607
phase, SNPs are assigned to the discovered CPs and the LRM is configured with the SNP connections. This assignment is considered to be a management operation that may be delegated to the network element. This allows for "plug and play" operation as a specific management policy. The LRM may also provide hints related to link discovery (e.g., link name). Protocol Controller (PC). The PC provides for the protocolencapsulation of the primitives that constitute the DA-DA interaction. The protocol controller attaches to the signaling network point of attachment, which can be thought of as the DA signaling address.
16.6.3 Types of Discovery Recommendation G.7714, as first published in 2001, defines three general types of discovery functions that can be performed: (1) Layer Adjacency Discovery, (2) Control Entity Logical Adjacency Establishment, and (3) Service Capability Exchange discovery. In the years that followed, further architectural foundation was laid for auto-discovery within G.8080, which had implications on G.7714 concepts. At the time of going to press, G.7714 had just undergone revision and restructuring. Revised G.7714 (2005) provides clearer requirements for the discovery process, refines the terminology used for the transport entities being discovered, extends the use of discovery to the management plane, and provides more detail on the behavior of the Capability Exchange process (renamed Transport Entity Capability Exchange). The Capability Exchange processes in support of the control plane will be provided in a new Recommendation under development, G.7716, which addresses the initialization and restoration of the control plane. 16.6.3.1 Layer Adjacency Discovery Layer Adjacency Discovery (LAD) describes the process of discovering transport network connection and link connection endpoint relationships and verifying their connectivity. In the most basic terms, determining "who a network element's neighbor is at a given layer" is what layer adjacency discovery is about. Not all equipment will necessarily terminate/interpret/alter the characteristic information at all the layers. For example, in Figure 16-24 we show a simplified functional block view of an SDH/optical network comprising eleven network elements (NEs). Two of these network elements are optical amplifiers (NE #3 and #9), and two are WDM multiplexers/demultiplexers, all of which only understand and act upon the physical layer. Two of the elements are SDH HOVC O-E-O switches (NE #4
608
Chapter 16
and NE #8), which act upon the SDH physical, RS, MS, and HOVC layers. The end-user equipment, shown as routers in this example, is also assumed to act upon all the layers, i.e. physical, RS, MS and HOVC. 1^
^
NE#1
NE#3
Routei
X
OC-3 1
OC-3
AMP
X
oxc
OC-48 I X OC-48
oxc
NE#4
NE#6
NE#8
AMP NE#9
Rou^d NE#11
Router
L : : ' ••'• '• "•'.:J
NE#7
NE#5
NE#2
NE#10
Figure 16-24. Example SDH network scenario
Looking at Figure 16-24, for example, we can see that NE #1 has a physical layer adjacency with NE #3, as illustrated in Figure 16-25. PHY (physical) layer adjacency
MS
yw— NE#1
w
NE#3
./NE#4
MS
"^ 'V'
\oi^ V'
'^pY 'V'
NE#5
NE#6
NE#7
MS
""^
\PH7
v..
NE#9
V
NE#11
NE#as MS
NE#2
NE#10
Figure 16-25. Illustration of physical adjacency
Examining Figure 16-26, we can see NE #1*8 and NE #2's RS layer adjacencies with NE #4 (and NE #8's with NE #10 and NE #11).
w
w NE#1
V
NE #>••'''
NE^
NE#5
NE#6
w
MS
RS layer adjacency
NE#7
f: NE#8
/
V "'ME^ "••••••••., \
MS
NE#11 w MS
V w
w NE#2
NE#10
Figure 16-26. Illustration of RS layer adjacency
Architecting the Automatically Switched Transport Network
609
Similarly, Figure 16-27 illustrates NE # r s and NE #2's MS layer adjacencies with NE#4, and NE #8's MS layer adjacency with NE #10 and NE#11.
V
-M^^: •••••
NE#1
jSfE#3
'
V
MS layer adjacency
^MS
^
NE#4
^?W
W ^
^fW
NE#5
NE#6
NE#7
NE#8
'•"•MS""
NE#^
NE#n
•••MS-'
NE#2
NE#10
Figure 16-27. Illustration of MS layer adjacency
Finally, Figure 16-28 illustrates similar HOVC layer adjacency relationships. Different discovery message sets would have to be used to discover adjacencies at the various layers (e.g., RS, MS, or HOVC). The exact LAD methods and protocols are described in greater detail in Section 16.11.1. HOVC layer adjacency \iir»/
XHQ/
/W"
•'MS'
.•••*
NE#1
.•••NE#3
••-MS-''
• MS-
NE#4
^^^
^^^&
^5PW
N E #5
NE #6
N E #7
T NE#8
1^
NE #9 •••.,_ N E # 1 1
\M§--'
••••••MS--
T NE#10
NE#2
Figure 16-28. Illustration of HOVC layer adjacency
We note that G.7714 defines another level of adjacency, termed physical media adjacency (PMAD), which is conceptually no different from any other layer adjacency since the layer that is represented is the physical media layer, e.g., fiber layer. The mechanisms for providing PMAD for optical networks have not been standardized. This is in large part because such
610
Chapter 16
mechanisms would require the use of optical processing techniques at the wavelength and fiber level, areas that are not yet sufficiently mature. 16.6.3.2 Control Entity Logical Adjacency Establishment The 2001 version of G.7714 included discussion of Control Entity Logical Adjacency, or CELA. This was previously defined as the association that existed between two discovery processes to facilitate communication between a pair of control entities across the SCN. The term CELA was utilized prior to the development of the G.8080 discovery architecture, and prior to consideration that the management plane could benefit from the automatic discovery process. As mentioned earlier, the revised (2005) version of G.7714 allows the discovery process to be used by the management plane as well as the control plane, making the term CELA inappropriate. Since the appropriate G.8080 architectural construct is the DA (Discovery Agent), it was considered to replace CELA with the appropriate term, namely Discovery Agent adjacency. However, a DA adjacency does not need to be preestablished and may be created dynamically while other discovery subprocesses (e.g., LAD) are being executed. Furthermore, the communications that occurred across the adjacency were not in any way scoped by the adjacency, removing any functional distinction from the messaging services provided by the DCN. Thus, discussion of the Discovery Agent adjacency was not included in the revised Recommendation. 16.6.3.3 Service Capability Exchange The term Service Capability Exchange (SCE) was used in the first version of G.7714 as defining a process for capability information exchange. This process was used to allow information regarding the ends of the discovered facility to be exchanged, "bootstrapping" the control plane. However, since the term service is often used to describe end-user communication services, the term SCE introduced ambiguity. To avoid this ambiguity, in the 2005 version of G.7714 the term SCE was changed to Transport Capability Exchange (TCE), which more accurately expresses the intent of the process. Again, with the extension of the discovery process to the Management Plane, the scope of TCE has been limited to the exchange of Transport Plane Capability information, and the exchange of control plane or management plane specific information will be moved to other Recommendations. It has been recognized that the exchange of control plane or management plane information has the same requirements as the exchange of transport
Architecting the Automatically Switched Transport Network
611
plane capability information. As a result, the 2005 version of G.7714 includes the specification of a generic mechanism to perform capability exchange. It is expected that this mechanism will be reused by Recommendations that address the specific encodings for the exchange of control plane and management plane information.
16.6.4 Discovery Considerations across Administrative Boundaries Auto-discovery across user-provider interfaces is unique, largely driven by the fact that service provider proprietary information is not exchanged with the user. In that sense, user-provider discovery is a "single ended" discovery process. From the point of view of the provider, it is a mapping of user endpoint names to network addresses that are used for routing, and from the user's perspective, it is an acknowledgment of the availability of the user-to-network connections. Capability exchange is quite limited given that service level information is exchanged and agreed upon during the contract negotiation phase. Discovery over intercarrier interfaces, which are service demarcation boundaries, may also involve tmst boundaries (e.g., between different carrier networks). To this end, the discovery processes offer a provider a significant amount of control over the information that can be exchanged. Each capability that could be exchanged may be, by provider policy, excluded from the exchange process. Furthermore, the exchange process allows providers the ability to restrict the behavior of the entity on the other end of the link.
16.7.
ROUTING (G.7715 AND G.7715.1)
Recommendation G.7715 contains the architecture and requirements for routing in ASON, expanding on the routing function concept in G.807 and G.8080. Recommendation G.7715.1 contains more detailed requirements aimed specifically at link state routing instantiation of ASON routing. The basic principles are presented in this section.
16.7.1 Requirements The routing requirements described in G.7715 encompass architectural, protocol, and path computation aspects. Some key examples of architecture and protocol requirements are described within this section.
Chapter 16
612 16.7.1.1 Architectural
The routing architecture requirements specified in G.7715 reinforce and build upon G.807 and G.8080 requirements. One key requirement with fundamental implications is that the routing adjacency topology and transport network topology shall not be assumed to be congruent [11]. This separation between the transport topology and the routing control topology, and also between the latter and the DCN, means that their topologies may all be different from one another in any given ASON network. Figure 16-29 illustrates a routing area (RA) where the routing control topology forms a tree including all the nodes in the transport plane. (We stress that this tree, used by the RCs to forward routing messages, does not need to be congruent with the transport plane topology.) Separation allows, for example, a single RC to support multiple network elements, and to be addressed separately from the network elements.
— Transport Link - - Routing Adjacency ® - Transport Node 0 = Routing Controller
Figure 16-29. Example of directed topology message Flow [37] [11]
Route computation is achieved using information advertised by the routing protocol and is often subject to a set of optimization constraints. After the route is determined, connection management processes (i.e., signaling) are used to allocate network resources (i.e., subnet connections and link connections) before user traffic is carried on those paths. After signaling has been used to establish a connection, the routing functions are no longer needed. Hence routing protocols in transport networks are not involved in "data-plane forwarding" and therefore have no impact on established services, which is not the case in, for example, IP networks where the forwarding function is continually dependent on the availability and integrity of the routing function. This has important implications for the performance required from the routing control plane; for example, transport plane connections remain active and data can continue to be transported even when the routing control plane is unavailable. This setup is illustrated in Figure 16-30.
Architecting the Automatically Switched Transport Network OSPF )
^
613
IP router Peers
Routing Controllers
.'"Signeling"' I
Control Plant IP packet Data R an e
Cross Connect
I
|lPaddr[
IP prefix next hqj
SDH path ^ n p Forwarding^
A. Transport Routing a n d Forwarding
B. I P Routing a n d Forwarding
Figure J 6-30. Routing and forwarding examples
• •
•
Other G.7715 requirements include the following [11]: The routing information exchanged between routing control domains is independent of intradomain protocol choices. The routing information exchanged between routing control domains is independent of intradomain control distribution choices, e.g., centralized or fully distributed routing functions. The routing information shall support an abstracted view of individual domains, i.e., the topology advertised externally in the routing protocol may be an abstracted view of the actual internal domain topology. The level of abstraction is subject to operator policy.
Recommendation G.7715 also provides requirements addressing the need to provide for unique identification of RAs within a carrier network, as well as avoiding protocol dependencies between hierarchical routing levels. 16.7.1.2
Protocol
Along with requirements related to protocol robustness, scalability, and security aspects, key requirements for the routing protocol itself are defined below [11]: • The routing protocol shall be capable of supporting multiple hierarchical levels. • The routing protocol shall support hierarchical routing information dissemination including summarized routing information. • The routing protocol shall include support for multiple links between nodes and shall allow for link and node diversity.
Chapter 16
614 •
The routing protocol shall be capable of supporting architectural evolution in terms of the number of levels of hierarchies, and aggregation and segmentation of RAs.
Note that the term level is used specifically to refer to the use of hierarchy to support subdivision of transport network resources into RAs, which is analogous to partitioning of a transport layer network into subnetworks as described in Chapter 2. This should not be surprising, as the only distinction between a subnetwork and a RA is the visibility of the link ends. A simple example of hierarchical routing levels is illustrated in Figure 16-31.
Level n+1
Level n
dUlRA
Level n-1
Figure 16-31. Simple example of hierarchical routing levels
Using the drawing convention for partitioning from Chapter 2, Figure 1631 can be redrawn as Figure 16-32. We again stress the difference between transport network layering and partitioning, as described in Chapter 2.
Architecting the Automatically Switched Transport Network
615
O' Figure 16-32. Hierarchical routing levels illustrated using partitioning notation
16.7.2 Architecture The ASON routing architecture of G.7715 supports the various routing paradigms hsted in G.8080, i.e., hierarchical, step-by-step, and source based. The routing architecture apphes after the network has been subdivided into Routing Areas and the necessary network resources have been accordingly assigned [11]. In this section, we will build upon the discussion of Routing Areas (RAs) from Section 16.3.2.1. Associated with each RA is a Routing Performer (RP), which is an abstract entity that provides path computation services for the RA. Signaling to create connections in the transport plane uses path computation. Whatever path computation style an RA supports, the RP will have the necessary topology information to support it. An RP is realized in the form of RCs, as described in Section 16.3.3, which are distributed entities with partial or complete (via duplication and synchronization) routing information for that RA. This architectural arrangement is shown in Figure 16-33. Routing Controllers distribute topology information with each other, and when two RCs communicate, this is known as forming a routing adjacency. The set of routing adjacencies form a routing control topology. Routing adjacencies are communicated/instantiated over the SCN. The RCs communicate via Protocol Controllers (PCs) that support a particular routing protocol. Separation of the RC and PC components allows great flexibility; for example, a single PC (and the routing protocol it implements) may support RCs from different transport layer networks and multiple
616
Chapter 16
hierarchical routing levels. This capability is important for protocol efficiency since it enables carriage of information pertaining to multiple layer networks (and hierarchical routing levels) in one PDU, instead of having to run separate instances for each layer network.
m
••iiii Service offered by RP
\ Computational View
y
RPy
Realized by RCs
* Engineering View
\ RG
Figure 16-33. Relationship between RA, RP, RC
Creation of RAs is related to the scope of routing information flooding (scalability), which impacts both transport resource assignment to Level 0 RAs and routing hierarchy decisions.
Architecting the Automatically Switched Transport Network
617
Transport resources (physical transport network)
Figure 16-34. Possible assignment of resources to RAs — example 1
Some illustrative examples are provided in Figures 16-34 (above) and 1635 (below), respectively.
Possttile allocation of transport resources to Level 0 RAs
Scenarb (b)
Transport resources (physical transport network)
<^>—«^^"^«——^^^ Figure 16-35. Possible assignment of resources to RAs — example 2
As discussed earlier, the RA is the key concept that matches the organization of the control plane to the organization of the transport plane. We stress that the existence of E-NNIs (bounding control domains) should not be inferred
Chapter 16
618
to create RAs. Several examples of potential RA and control domain scenarios are shown below that illustrate this point. Figure 16-36 illustrates several alternative configurations of routing control domains, ranging from multiple RAs in a given routing control domain, as in scenario (a), to complete congruency, as in scenario (b). One example of the former scenario might correspond to the situation in which, for example, there is a single vendor homogeneous solution, but scalability considerations warrant more than one RA.
(a)
RA(Level n) Control Domain
(b)
Figure 16-36. Possible configurations of routing control domains
Correspondingly, there are also alternative configurations of routing control domains ranging from complete congruency to multiple control domains within an RA, the latter of which is illustrated in Figure 16-37.
[C3 RA(Le\eln) ') Cortrol Domain
Figure 16-37. Possible configuration of routing control domains
The configuration in Figure 16-37 might correspond, for example, to a scenario in which two vendors with heterogeneous control plane routing protocol implementations were deployed within the same RA. It should be noted that we would not expect routing control domains to be configured in a manner such that they would intersect RAs. Specifically,
Architecting the Automatically Switched Transport Network
619
while the raison d'etre for creation of these constructs derives from different factors, it would generally be expected that a network planner would consider decisions relating to these aspects in a coherent manner.
16.7.3 Hierarchy in Routing The routing architecture of G.8080 and G.7715 allows for different implementations of the routing functions. That is, the various routing functions can be instantiated in a variety of ways including using distributed, co-located, and centralized mechanisms. A G.7715.1 compliant link state routing protocol can be instantiated as a set of RCs, each one performing topology distribution for the RA it is associated with. Each RC has a replicated topology database representing the transport plane topology where database consistency is maintained by exchanging database synchronization messages between every pair of RCs. This information exchange occurs via the protocol controllers for each RC. Each protocol controller runs an instance of the link state protocol. Using the topology information advertised via the link state routing protocol, a source can calculate routes through the network along which connections may be established. The choice of source routing for path computation has some advantages for supporting connection management in transport networks. It is similar to the manner in which many transport network management systems select paths today. Also, it can be powerful when diverse path computation is needed or for implementing fast restoration, among other things. Figure 16-38 illustrates an example of a network with two levels of routing hierarchy, where the lowest level routing area is tied to the transport network's physical topology. Four routing areas (RAl, RA2, RA3 and RAIO) are defined, with the first three at Level 0 and the parent at Level 1. Internally, each of these RAs manages its own network topology and available resources; i.e., there exists some method for obtaining a path across the RA that can be established via signaling.
Chapter 16
620
Level 0 RAs
Figure 16-38. Simple network with two levels of routing hierarchy
As described earlier, each RA has one or more associated RCs. It should not be assumed that there is a one-to-one relationship between transport resources and RCs, as a single RC may support multiple network elements. An example of a possible RC distribution including this scenario is illustrated in Figure 16-39.
Figure 16-39. Possible distribution of RCs
As we have discussed earlier, an architectural requirement of Section 16.7.1.1 is that "routing information exchanged between routing control domains must be independent of intradomain protocol and distribution choices". From an RC perspective, this means the RC distribution within a routing control domain is not externally visible. Thus, an RC can "act on behalf of a routing control domain; we note that one or more RCs can assume this role. Each RC is described by an RC Identifier (RC ID) and is uniquely defined within its containing RA, which, in turn, is identified using a RA Identifier (RA ID). Again, as noted earlier, a single RC supporting multiple network elements may have its own RC ID. The grouping of RCs in RAs at different hierarchical levels is defined in a flexible manner. For example, there does not exist any association or set of rules between the RA ID and the RC ID or the identifier of any RC
Architecting the Automatically Switched Transport Network
621
belonging to that RA. Additionally, the term level only has a relative sense, since it is not fixed in value; thus, a new RA may be added as needed as the first hierarchical level, at the top of the current hierarchy, or even between two existing hierarchical levels. This flexibility yields great convenience in managing carrier's networks as they develop and grow. The routing architecture described in G.7715 is applicable to multilevel routing hierarchies and is very powerful for scaling networks while providing sufficient routing information to efficiently compute routes across multiple separately managed physical networks. Recommendation G.7715 does not specify the detailed routing protocol to be used but leaves it as an implementation decision. Recommendation G.7715.1 further defines requirements for link state routing protocols in a protocol-independent manner.
16.7.4 Routing Information Exchange The following section discusses the routing information that is available at the various routing levels. From the point of view of architecture, links are wholly contained within a routing area, so a link only exists in the lowest level RA that contains both link ends. Connection Controllers have identical scope. In this model, there is no notion of exchanging routing information up and down the levels of the routing hierarchy via RCs. However, implementations invariably handle all layers and levels within a single component, and in this case it is convenient to consider links to be attached to the physical switch, and to discuss the visibility of links in terms of information flow up and down the levels. While the rest of this section is in terms of information flows, it is important not to lose sight of the architectural separation that is still maintained. Routing information may be exchanged across different levels of the routing hierarchy (between an RC, its parent, and its child RCs), and the information flows between levels are not specific to any particular paradigm (e.g., centralized or link state). The transport plane topology is advertised as a set of nodes and links. Nodes may correspond to abstract or physical entities within the child RA, and no distinction is made between them in the advertisement. For example, referring to Figure 16-38, there is no distinction in the advertisement between the (abstract) nodes in the Level 10 RA, and the physical nodes in the Level 0 RAs. Recommendation G.7715.1 indicates that the type of information flowing upwards (i.e., level N to level N+1) and downwards (i.e., level N+1 to level N) both involve the exchange of reachability information and may include summarized topology information—in other words, the transformation of
622
Chapter 16
one RA topology as a virtual RA topology (in terms of nodes and links) for the purposes of summarizing routing information for advertisement in its parent RA. The transformation mechanism is not intended for standardization. Recent insight clarifies that this transformation of RA topology is concerned with calculating the cost of crossing a RA, and not with presenting a different view of the RA internal topology. The cost has been expressed in terms of a set of nodes and links because that is what today's routing protocols handle. The reader should be aware that the discussion is about cost; it is not about revealing RA internal details. It should be noted that G.8080 requires the topology exchanged to be specific to a layer of the transport hierarchy. Consequently, links will only be reported in a specific layer's topology if the link supports the signal type of that layer. As noted earlier, multiple layer topologies can be carried by one routing protocol as long as the link and node information exchanged is identified as being specific to a layer. Finally, to route a connection to a given customer, we must know through which RA the customer can be reached. The routing protocol must thus advertise client reachability information in the form of UNI~N SNPP addresses. A UNI-N SNPP address associated with a given RA is advertised by the RC(s) that represent that RA, so that other RCs will learn of the reachability of that UNI Transport Resource and pass this information onto their associated RAs. Note that the network must contain a directory that maps UNI-C Transport Resource names onto the local UNI-N SNPP addresses. Also note that the advertised SNPP address may be different from the internal SNPP address, with address resolution occurring at the RA boundary. There are two options for propagation of routing information at a given hierarchical level. The first is uploading the information to a centralized route server, such as a management station, where the information can be integrated and used for computing routes. This option is sometimes referred to as the path computation server approach. The second option involves propagating the information throughout the entire routing hierarchy, which requires that the information is disseminated among all the hierarchical routing levels. The routing information of an RA at the hierarchical level A^ (RAN) can be disseminated to the RA at the level N+1 (RAN+J). The information communicated among the cooperating RCs usually includes the set of reachable UNI Transport Resource identifiers, inter-RA links (from the perspective of RAN+I), and nodes. Nodes with internal detail (abstract nodes) may have a cost associated with them, which may be expressed in terms of a summarized/transformed topology of nodes and links. After the information is communicated to level N+1, the RCs in RAN+I will cooperate to advertise
Architecting the Automatically Switched Transport Network
623
the information so that the routing information associated with a small set of RAs will be learnt by others. Recommendation G.7715.1 describes two approaches by which the routing information of RAN+I can be provided to RAN. In the first approach, the RP in the containing RA at level N+1 provides the level N RP with the reachability and topology information visible at level N+1. The information visible at level N+l includes the information visible at consecutive upper levels. This information may then be used by the level N RP to compute a path that leaves the RA. In the second approach, recursive requests are made from the level N RP to the level N+1 RP upward towards the root of the routing hierarchy. The result of each request is analyzed by the requesting RP to determine the exit point utilized by the level N+1 RP. The RP will then update the path including the path computed through the level N RA, and return it to the requester. This approach is loop free as the routing hierarchy defined in G.8080 has strict containment (preventing a contained RA from containing a containing RA). 16.7.4.1
General Attributes
Recommendation G.7715.1 focuses in more detail on the attributes associated with usage of ASON compliant link state routing protocol to advertise transport plane topology between RAs. We divide the information disseminated via a routing protocol into node attributes (using the definition of node as described earlier) and link attributes, since these are the basic topological elements. Link attributes may be further classified as those that are layer independent, such as identification, and those that are layer specific, such as adaptation support. The way that these attributes are used by the routing protocol is normally dependent on the operator's policy (e.g., the operator may decide to advertise a restricted set of reachability information). We consider the different attributes below. 16.7.4.2 Node Attributes All nodes in a graph representation of a network belong to an RA; hence the RA ID is an attribute of all nodes. As discussed earlier, no distinction is made between abstract nodes and those that cannot be decomposed any further; the same attributes are used for their advertisement [12]. The following node attributes are defined [12]: • Node Identification (ID): The Node ID is the subnetwork ID that exists in an SNPP name. All node IDs advertised within an RA are allocated from a common name space for that RA.
Chapter 16
624 •
Reachability Information: Reachability information describes the set of endpoints that are reachable by the associated node. It may be advertised either as a set of UNI Transport Resource identifiers or a set of associated SNPP identifiers, the selection of which must be consistent within the applicable scope. For implementation purposes, it is important to identify when attributes are required to be supported in a protocol realization and must be present in advertisements, and when attributes are required to be supported but may not be present in an advertisement based on operator policy. Table 8-1, from G.7715.1 [12], provides this information, where capability refers to level of support required in the realization of a link state routing protocol, whereas usage refers to the degree of operational and implementation flexibility, i.e., the ability of the operator to define the use or non-use of the attribute by policy. Mandatory usage attributes are those that are needed as a minimum to support path computation.
Attribute Node ID Reachability
Table 16-2. Node attributes [12] Usage Capability Mandatory Mandatory Mandatory Optional
16.7.4.3 Link Attributes Recommendation G.7715.1 defines the following set of link attributes to be supported in link state routing protocols for ASON [12]: • Local SNPP Name: identifying the transport plane resource at the local SNPP link end • Remote SNPP Name: identifying the transport plane resource at the remote SNPP link end Table 16-3 provides implementation requirements for general link attributes.
Link Attribute Local SNPP Name Remote SNPP Name Layer specific characteristics
Table 16-3. Link attributes [12] Capability Mandatory Mandatory (refer to Section 16.7.4.4)
Usage Mandatory Mandatory
Note: When the remote end of a link is located outside of the RA, usage of the remote SNPP Name is optional.
Architecting the Automatically Switched Transport Network
625
16.7.4.4 Layer-Specific Characteristics Recommendation G.7715.1 defines the following set of layer-specific characteristics as attributes of a link [12]: •
Signal Type: This attribute identifies the characteristic information of the layer network. Since advertisements are layer specific, this information identifies the layer network being advertised. If advertisements for multiple layer networks are combined in a single protocol instance, this attribute allows advertised information to be forwarded to the RC for that layer network. • Link Weight: This attribute represents a vector of one or more metrics, each of which indicates the relative desirability of a particular link over another during path selection. • Resource Class: This attribute corresponds to a set of administrative groups assigned by the operator to this link. A link may belong to zero, one, or more administrative groups. • Local Connection Type: This attribute identifies whether the local SNP represents a TCP, or a CP or can be flexibly configured as either a TCP or a CP. Some links may, for example, support termination of connections but not transit of connections as a result, and should only be used if the connection terminates at the remote node. • Link Capacity: This attribute provides the sum of the available and potential link connections for a particular network transport layer. Other types of capacity information have not been precluded and are for further study in G.7715.1. Providing such information on a layer-specific basis allows more accurate connection routing, since it takes into account the potential for connections at one layer impacting the availability of a link for connections at another layer due to factors such as placement within the frame, which are not obvious from a simple measurement of capacity in total available bits per second. • Link Availability: This attribute represents a vector of one or more availability factors for the link or link end. Availability may be represented in different ways between domains and within domains. Within domains, it may be used to represent a survivability capability of the link or link end. In addition, the availability factor may be used to represent a node survivability characteristic. Link availability may be a constraint used in routing of paths supporting connections with higher class of service. • Diversity Support: This attribute represents diversity information with respect to links, nodes, and Shared Risk Groups (SRGs) that may be used
Chapter 16
626
during path computation. Such information can then be used in computation of paths for protection/restoration purposes. • Local Client Adaptations Supported: This attribute represents the set of cHent layer adaptations supported by the TCP associated with the Local SNPP. This is only applicable when the local SNP represents a TCP or can be flexibly configured as either a TCP or a CP. This type of information may be used when calculating paths requiring a specific adaptation when support may differ on a link-by-link basis. In Table 16-4, implementation requirements are specified for layerspecific link attributes. Table 16-4, Layer-specific characteristics [12] Capability Layer-Specific Characteristics Signal Type Mandatory Mandatory Link Weight Mandatory Resource Class Local Connection Type Mandatory Mandatory Link Capacity Link Availability Optional Diversity Support Optional Local Client Adaptations Supported Optional
16.8.
Usage Optional Optional Optional Optional Optional Optional Optional Optional
SIGNALING (G.7713)
Recommendations G.7713 and G.7713 Amendment 1 provide protocolneutral specifications for distributed call and connection management in ASON, which are thus applicable to multiple signaling protocols. In addition to the processes related to signaling communication, several other important issues are addressed, including: • Rainy-day scenarios that need to be covered to support the unlikely event of defects impacting the control plane. These may include defects of the signaling channel, or defects of the control plane itself. • The operation and communications between the call and connection control components in setting up and tearing down connections. These include specification of the messages and the information content of the messages, as well as the behaviors of the signaling mechanism. • Issues that need to be resolved to handle alarm suppression in the transport plane when connections are setup and removed. We note that ASON signaling components (especially the NCC) may be centrally instantiated. While G.7713 does not preclude this, it focuses upon protocol requirements for the case in which signaling components are
Architecting the Automatically Switched Transport Network
627
distributed. Information passed by the distributed signaling components and messages are defined in G.7713 in an abstract manner. If different signaling protocols are used in a common ASON network, they may interwork across various reference points by transferring equivalent messages and information elements between their protocol-specific encodings.
16.8.1 Call and Connection Management Operations This section describes call and connection management operations after the contract between the user and the provider has been established. Figure 16-40 (based upon Figure 6-Aml-l /G.77ISA". 1704 [5]) provides a simple high-level illustration of the interactions between end user (calling and called party), call controllers (CCC), and network call controllers (NCC) in a two-domain network. The Calling Party Call Controller (CCC-a) interacts with a Called Party Call Controller (CCC-z) by means of one or more intermediate network call controllers (NCCs) at service demarcation points (i.e., UNI, E-NNI). The Call Controllers perform the following actions: • The NCC correlates the SNCs to the call. • NCC-la and NCC-2z work with CCC-a and CCC-z, respectively, to correlate Link Connections(s), LC(s), to the call access segments. • NCC-1 works with its peer NCC-2 at domain boundaries to correlate LC(s) to the interdomain call segment. • The NCCs correlate the LCs and Subnetwork Connections (SNCs) that are associated with each call segment within their respective domains.
Domain 2
Domain 1
^ ^pia
ccc^
AGO
End-to-end call Call segments Connections
..^^»y...ErNNJ....tt NCC-1) r
A—
<
•
C-2
*^
CCC-z
NCC-2Z 1
^ ^ ^ »
AGO
!»•
-
^--> ^ ^^ LC ^
~— P SNC
^ W
• ^
LC
^
9
^
w. SNCs
^
^-~~-»» ^ w^ ^ LC
Figure 16-40. Interaction among Call Controllers in two domain example [5]
628
Chapter 16
Connection controllers (CCs) establish the connections associated with each call segment.
that
are
16.8.2 Basic Call and Connection Control Sequences Call and connection setup in transport networks uses a three-message sequence consisting of the SetupRequest message, which initiates the call and specifies the desired traffic characteristics for the connection; the Setuplndication message, which acknowledges establishment of the connection across the network, and the optional SetupConfirm message, which confirms end-to-end to the destination node that the connection has been made. 16.8.2.1
Call and Connection Setup
Referring to the example of call setup request processing in Figure 16-41 (based upon Figure 6-Aml-l IG.llUIYAlOA [5]), the Calling Party Call Controller, CCC-a, requests call setup, and the ingress NCC-la initiates processes to check the call request. These may include checking for authentication and integrity of the request, as well as constraints placed by policy decisions (described in Section 16.5). The request is also sent to NCC-2z and NCC-2 (at the service demarcation points). Processes included in the egress NCC-2z may include verifying that the call request is accepted end-to-end [5]. Upon successful checking, CCC-a continues the call setup request by initiating a connection setup request to its associated Connection Controller, CC-a. The connection request performs the coordination among respective CCs to set up and release connections. When a connection is required for the call, the call is not considered complete until the connection is in place [5].
Architecting the Automatically Switched Transport Network
ACC-1
TCC-1
629
ZCC-1
Connection setup CC-a: A-end Connection Controller CC-z Z-end Connection Controller ACC-n: A-end Connection Controller at Domain n ZCC-n: Z-end Connection Controller at Domain n TCC-n: Transit Connection Controller in Domain n
Figure 16-41. Example of call setup request processing [5]
Upon successful indication by the connection setup request process (across all call segments) the call setup request is successfully completed, and transfer of user characteristic information may begin. If the connection setup request process was unsuccessful, a call-denied notification is sent to the user [5]. 16.8.2.2 Call/Connection Release Call and connection release in transport networks, in its basic form, uses a two-message sequence consisting of the ReleaseRequest message, which initiates release of the connection and triggers release at the next hop of the connection, and the Releaselndication message, which is an acknowledgment of completion of the release of the local channel. Optionally, a Notify message may be sent to the remote end prior to the initiation of connection release, in order to prevent alarming due to in band performance monitoring that might be triggered by the connection release in the data plane. 16.8.2.3
Query
Finally, the ability to query the status of a connection or call is included through the QueryRequest message and the associated response through a Querylndication message so that it is possible to audit the state of the connection at a neighboring node and initiate a connection release or state resynchronization in case of a conflict.
630
Chapter 16
16.8.3 Signaling Attributes Distributed call and connection management attributes may be separated into attributes associated with the call and those associated with connections. In both cases, the scope of the attribute could be local, constrained to one reference point, versus global, which is carried across the network. Recommendation G.7713 Amendment 1 provides these attributes for UNI, E-NNI, and I-NNI signaling processing [5]. • UNI signaling processing includes call attributes as well as connection attributes for setting up LC(s) on user-to-network domain access links. Examples of call attributes include Calling and Called UNI Transport Resource Names, Call name, and policy attributes. It should be noted that call identity attributes have end-to-end scope. For example, the value of the UNI Transport Resource Name must be globally unique, and is assigned by the service provider. Examples of connection attributes include Initiating/Terminating Connection Controller Names and Connection Name. • E-NNI signaling processing includes call attributes as well as connection attributes for setting up LC(s) on interdomain access links. The call attributes are the same as for the UNI, though the Calling/Called UNI Transport Resource name may be carried transparently. Connection attributes include SNP and SNPP IDs, as well as Called/Calling Access Group Container(AGC) SNPP ID. An AGC is a single layer entity that can terminate multiple SNPP links and contains access groups, LRMs, and TAPs. - I-NNI signaling processing includes connection attributes. If call communications traverse I-NNIs, call parameters must be carried transparently. Abstract attributes at the E-NNI are provided in Table 16-5 (Table 7 3/G.7713A^.1704[5]):
Architecting the Automatically Switched Transport Network
631
Table 16-5. E-NNI Call and Connection attributes :5] Call vs. Connection Scope Attributes Call Calling UNI Transport End-to-end Resource name Call Called UNI Transport End-to-end Resource name Identity Connection Initiating CC/CallC Local attributes name Connection Terminating CC/CallC Local name Connection Connection name Local Call Call name End-to-end Connection SNPID Local SNPP ID Connection Local Service Connection Called AGC SNP ID End-to-end attributes Connection Called AGC SNPP ID End-to-end Directionality Call/connection Local Call Cos End-to-end GoS End-to-end Call ! Policy attributes Call/connection j Security Local Explicit resource list Connection Local Recovery Connection Local
16.8.4 Signaling Application Example An interesting signaling applications example is that for dual homing. Dual homing [40] refers to the scenario in which an individual user interacts with a provider network via more than one UNL Another form of dual homing is where a user interacts with two different provider networks via different UNIs. This configuration is commonly used to increase the reliability of a user's access to the network. If the transport link(s) to the network associated with one UNI fail, the other UNI can be used. This concept could also be applied over an E-NNI for multidomain connection reliability. Aside from increased access reliability, additional services can be supported for dual homed users: • Simple path diversity: In this scenario, illustrated in Figure 16-42, dual homed user 1 places two calls to a dual homed destination user 2, via each of its two UNIs over a provider network domain. The connections associated with each of the two calls could be established such that they
Chapter 16
632
do not share transport resources in the provider network. This is a feature being considered in the OIF for the UNI 2.0 Implementation Agreement.
umJ P ^ ^ <^ ^ '•'^ ^ -^%^NI User 1 r
Connections
UNIT
^^r""
-^
^ ^J
User 2
"* ^ '^y^uHi
Figure 16-42. Dual homing with simple path diversity
Multidomain path diversity: In this scenario, illustrated in Figure 8-43, one user is dual homed on domain 1 and the other is dual homed on domain 2. There are multiple E-NNIs between domains 1 and 2. Again, we consider the case where dual homed user 1 places two calls to destination user 2 via each of its two UNIs. The connections associated with each the two calls could be established such that they neither share transport resources in domain 1 or 2 nor traverse the same interdomain transport links.
Domain 1 ^ource
Domain 2 E-NNI
UNI
UNI^
Destination]
r User 2
Userl: UNI
^
E-NNI
UNI
Figure 16-43. Multidomain path diversity
•
Restoration from User Access Failure: In this scenario, again referring to Figure 16-42, consider a failure in one of the transport links associated with a particular UNI. It would be desirable to trigger a restoration action that would enable a connection to still be established between users 1 and 2 via the transport link associated with the alternate UNI. • Optimal Path: A dual homed user may prefer to utilize the transport link associated with one particular UNI versus another for a particular destination. A capability could be established to enable a user to do so. In order to support the above applications, we must consider the interactions between a call over one UNI and a call over the other UNI. This also introduces dependencies, or awareness, between calls within network domains. In other words, the Network Call Controllers for a dual homed user need to discover and then communicate with each other.
Architecting the Automatically Switched Transport Network
633
Further, for diverse path apphcations, a critical factor is whether a user knows that path diversity is required before the first path is estabhshed. This factor introduces the concept of dependent and independent calls. If the user knows diversity is required in advance, we consider these to be dependent calls. In this case, separate sequential call requests are made for each path. The service to the user is to be able to make a call request for another connection that should exclude elements of an existing path. In the case of independent calls, it is not known prior to any call establishment that diverse paths are needed. Here the requirement for a second diverse path was not known at the time the first connection was established. This situation differs from the previous case in that the source user does not know ahead of time that a second connection will be needed that is diverse from an existing one. This situation may arise when an existing transport user upgrades a service contract with a provider due to increased bandwidth needs. As with dependent calls, there are no requirements on when the two paths have to be set up in time relative to each other. In both cases, what is required in the network to support these dual homing applications is the ability for Network Call Controllers that share a common user to discover each other and then communicate regarding call context. NCCs can accomplish the discovery of other NCCs that share a common user by using the common CCC to inform the NCCs about each other. Further call contexts from one NCC can be transferred transparently to another dual homed NCC via the common CCC [40].
16.9. CONTROL PLANE MANAGEMENT While the goal of the ASON control plane is to provide a means of relieving management systems from the burden of connection management, it adds its own set of elements requiring management. The management of ASON is currently work in progress within ITU-T SG 15 and the TMF, and this section provides a brief description of the direction the work is taking in the several involved standards bodies. Specifications for network management are primarily being developed within the TMF and the ITU-T. The TMF is concentrating on the network aspects of the management problem, while the ITU-T is concentrating predominantly on the configuration of network elements. These efforts are complementary, and both are necessary for a complete management solution. The ITU-T initiated its work effort with G.7718, "Framework for ASON Management", which discusses the management requirements for the control plane. These requirements encompass the use of ASON constructs (e.g., RAs, SNPP links), management of calls and connections, and configuration.
634
Chapter 16
fault, performance, accounting, and security management for ASON. G.7718 also gathers the many management-related details from other ASON-related Recommendations into one place. Work is proceeding on G.7718.1, "Protocol-neutral management information model for the control plane view." By protocol neutral, we mean a design that can be implemented using several available management protocols; thus, the design is not restricted by the strengths (and weaknesses) of any particular protocol. While a full description of network management is beyond the scope of this chapter, it is useful to present some context regarding the principles of network management as practiced in the transport network. For many years, ITU-T network management standardization efforts have focused on managing the service instance and its relation to the network. Traditionally, the connection service instance is realized using transport resources across potentially nonhomogeneous network segments. This means that connections are first-class objects that are linked to the underlying hardware supporting the connection. Thus, the focus has been more on service rather than equipment, and it has been said, "Service is managed while equipment is repaired". The alternative approach is for the existence of a connection to only be inferable from properties of the hardware, which makes it difficult, if not impossible, to observe (and hence manage) any particular connection. The focus on service has given rise to a uniform method for modeling the transport resources and presenting management information that is independent of the underlying technology. The principle of managing the service instance has also been followed in ASON control plane management standardization, where the service instance is now the call supported by connections that are managed as before. The definition of a limited set of objects that are used as required on every network element also follows that model. This method is contrasted with that of only managing protocol instances, with little regard for uniform models or for the underlying network structure. The TMF has produced four documents, namely, TMF 513 Business Agreement [41], which details the business requirements; TMF 608 MultiTechnology Interface Agreement [42], which details the interface design in UML; TMF 814A Implementation Statement [43] and TMF 814 IDL Implementation [35], which define the interface in CORBA IDL. These documents provide a network view of the transport resources assigned to the control plane, as well as the calls and connections that are supported by that network. The assumption is that any necessary provisioning has already been done, and while the network manager can inspect the results of that provisioning, it does not itself do the provisioning. Network management interfaces are described in terms of managed objects, and these objects are derived from the real world entities that
Architecting the Automatically Switched Transport Network
635
describe the system. Transport-managed objects are derived from the atomic functions of the G.805 (and G.809) architectural models, as these models provide an implementation-independent view of the transport functions. The equivalent entities in ASON are the components, but ASON components themselves are merely used to describe interfaces that are externally observable. ASON components themselves, therefore, do not make good managed objects, since the components are merely a descriptive technique and not a functional specification. Our examination of the ASON architecture has shown that there are distinct classes of components that serve very different purposes. One set describes the topology of the network that ASON operates on, leading to objects for links and routing areas. These objects are sufficient to support the network view. The current view of the other ASON-managed objects is that the component interfaces and the Protocol Controllers that support them are a good basis for management. These interfaces exist on real equipment and have properties that need to be configured. The interfaces support real control plane services (signaling, routing, discovery), and a particular service may be enabled or disabled, or a particular protocol may need its timers to be set, depending on which physical interface is used. Service- and protocolrelated objects allow for policies to be set, and the users that are affected are determined entirely by how the managed objects relate to one another. Using the RC components. Figure 16-44 provides an illustration of how these concepts might apply to the Routing service. Other services are similarly handled. For completeness, the current view of ASON-managed objects for G.7718.1 is illustrated (in UML notation) in Figure 16-45; we note that these object classes may be different once the Recommendation is published. The essential element to understand with regard to these objects is the rationale behind their derivation; comprehension of the actual objects is straightforward.
636
Chapter 16
Figure 16-44. Routing service example
& d s f n g Transport ^
0 :1
SMPLmkCoimecSon
f
•.7"" Y ControtPlaneServtce
• < /
^
/ C^tContcrflerSer>ice
^
/ K \
\
:
-
i « * j
i i adaptaSon
i-^-H-
R-otoooiyMachanism
r H. •^%.. \ \ /
ConnedionControHerService
i
j
RoutingService
j
0 * i Connecfion
CarC^elaitRecord
\
ConnectonDetai IRecord
Figure 16-45. UML notation
;
Trait —
i
Architecting the Automatically Switched Transport Network
637
16.10. PROTOCOL ANALYSIS As was discussed in the introduction to this chapter, various standards bodies and industry fora have been involved in the development of control plane specifications, which are moving towards a convergence of requirements and protocols. This convergence has involved an iterative multiyear process of analysis, solution assessment, interoperability testing, and feedback resulting in the continuing evolution of supporting foundation protocols and ASON-specific protocol extensions. From a protocol perspective, this process spans areas ranging from "graceful restart" mechanisms to adding additional error codes/values, for example, for connection rejection. To recap, some key ASON requirements include support for • Noncongruent control plane, DCN, and transport plane topologies, with accompanying independent identifier spaces; • Separation of call (service) and connection, including zero or more connections associated with each call segment; • Call and connection operations arising only from explicit requests; • Control plane failures not causing connection failures; • Separation of location from identity (both of which are represented by the IP address in today's Internet [28]); • Modularized design around open interfaces at domain boundaries; • Layer-specific, in addition to layer-independent, information (e.g., per G.803 [44], G.872 [45], 1.326 [46], G.8010 [47], G.8110 [48]); • More than two levels of routing hierarchy; and • Rearranging/modifying address plans and Routing Areas. Considerable analysis of signaling protocols vis-a-vis requirements has been performed. Within this section, we cite several examples of requirements that were reflected in the foundation signaling protocols. Further examples, and elaboration thereof, are provided in more detailed references (e.g., [48]). Requirements that were reflected in ASON protocol extensions, as described in G.7713.1, G.7713.2, and G.7713.3, are described in Section 16.11. Similar analysis is currently taking place with regard to routing protocols. At the time of the development of this chapter, this work was not complete.
16.10.1 Analysis Approach As discussed in the introduction, the ITU-T started "top down" and the IETF started "bottom up" in developing standards to support the optical control plane. Within the IETF, protocol extensions for supporting dynamically provisionable optical cross-connects, initially termed MPAS
638
Chapter 16
{MPLambdaS), started in earnest during 2000. This work predated G.807, as described in Section 16.2, which provided a foundation of fundamental ASON requirements and architecture. As might be expected, iteration of the first GeneraUzed MPLS protocol proposal, introduced in November 2000 [49], was required to start addressing some of these requirements. Once G.8080 and G.7713 were approved, they were used to provide a further basis for determining the capabilities of the ASON network. During 2002, an intensive effort took place to methodically analyze the emergent GMPLS RSVP-TE signaling protocol vis-a-vis ASON requirements. This analysis was performed utilizing a traceability matrix showing, for each ASON requirement, what (if any) GMPLS feature fulfilled the requirement and where any gaps existed. For illustrative purposes, an excerpt from this traceability matrix is provided in Table 16-6. Comp. CC
CC CC CC CC CC CC CC CC CC CC CC
Table 16-6. Excerpt from Template of ASON Requirements Traceability Requirements REF# GMPLS RSVP-TE 1146 Connection setup — success (or Yes: Resv ResvConf) message failed: message Yes: ERROR SPEC Connection setup 1147 error 1148 Connection setup — failed: called party Yes:ERROR_SPEC24/ 5 busy Connection setup — failed: calling party No (requires extensions 1149 busy to error codes/values) 1150 Connection setup — failed: timeout Yes: ERROR SPEC24/5 Connection setup — failed: identity No (requires extensions 1151 error: invalid A-end user name to error codes/values) 1152 Connection setup — failed: identity No (requires extensions to error codes/values) error: invalid Z-end user name 153 Connection setup — failed: identity No (requires extensions to error codes/values) error: invalid connection name Connection setup - - failed: service error: Yes: 1154 invalid SNP ID ERROR SPEC24/6 1155 Connection setup — failed: service error: Yes: ERROR SPEC24/6 unavailable SNP ID Connection setup - failed: service error: No (requires extensions 1156 to error codes/values) invalid SNPP ID Connection setup - failed: service error: No (requires extensions 1157 to error codes/values) unavailable SNPP ID
The effort resulted in information on the degree to which the protocol satisfied ASON requirements, and stimulated work on refinements and extensions for identified gaps. In the evolution towards convergence of requirements and protocols to meet business needs, the industry has come a long way from the earliest
Architecting the Automatically Switched Transport Network
639
MPA.S/GMPLS protocol behaviors, where data and signahng were assumed congruent, the focus was on IP services, and the behavior was a mirror of that for data (e.g., connection tear-down without request).
16.10.2 Requirements Implications on Protocol Solutions This section provides some examples of protocol extensions resuhing from requirements considerations. The intent is not to provide an exhaustive enumeration but merely to provide a flavor for the nature of such extensions. For example, the requirement that congruence not be assumed among control plane, transport plane, and DCN topologies had some fundamental signaling protocol implications. "Classical" RSVP and RSVP-TE assume that the control signaling messages are sent in-band and a failure of a link or node results in both transport plane failure and control plane failure. In other words, it is assumed that the failure to receive a Path or Resv refresh indicates that the local state should be released. Similarly, in RSVP-TE the failure to receive Hello messages can be an indication that the link or adjacent node has failed [50]. However, this is not the case for out-of-band signaling where the control and transport planes can fail independently (and their failure detection should be performed independently). To ensure high reliability and robustness, and to prevent calls and connections from being released without an explicit request, all call and connection operations within the ASON network have to be explicitly requested. This means that signaling protocols should operate as a "hardstate" mechanism. PNNI and LDP are both hard-state mechanisms in that any operation must be explicitly requested. However, classical RSVP-TE is, by definition, a soft-state mechanism. To address this issue, extensions needed to be made to GMPLS RSVP-TE to allow it to behave in a "pseudo" hard-state manner. These included extending timers to handle an "infinite" timer for soft state, that is, adding a Restart Capabilities object (RESTART_CAP). This extension enables one to distinguish between restarting the control plane, and operating the transport plane without any control plane [50]. (It should be noted that other failures types may still occur which will result in a soft-state response from RSVP-TE; for example, a failure to receive consecutive refresh messages due to congestion, and not due to failure of node, may result in the failure of particular connections). Another example is related to the requirement that call and connection operations should only arise from specific requests, and thus, the act of establishing or tearing down a connection should not, of itself, cause alarms to be raised [1]. However, it was observed that a false alarm could occur during connection deletion if a source stopped sending a valid signal, or if a network node deleted its transport plane state, before all downstream nodes
640
Chapter 16
have stopped fault detection for the connection. To address this issue, an Administrative Status object was created to allow the ingress or egress of the connection to change the administrative status of the connection [50].
16-11. METHODS AND PROTOCOLS — DISCOVERY 16,11.1 Layer Adjacency Discovery Methods Consistent with the ASON control plane architecture, Layer Adjacency Discovery is performed on a per-transport-layer basis. However, the actual discovery protocol used can be the same at each layer. Only the mechanism to transport the discovery protocol messages may differ among layers. There are two methods that may be utilized for transporting discovery messages, described within G.7714 and G.7714.1: one using in-band overhead in the server layer, and the other using a test set in the client layer. • Type 1: In-band overhead in the server layer In this approach, the server layer trail overhead carries the discovery message, which is used to discover the peer TCPs (e.g., TCPss to TCP3R in Figure 16-46 [10]). The CP-to-CP relationships are inferred from the TCPto-TCP relationships using local knowledge of the configuration of the adaptation function and its relationship with the trail termination function. An important point to note about this method is that this overhead information persists through control entity failures. • Type 2: Test set in the client layer In this approach, a test signal carried in the client payload is used to discover the peer TCPs (e.g., TCPjs to TCPIR in Figure 16-46 [10]). The CP-to-CP relationship is inferred from the local knowledge of the matrix connection that was previously set up to connect the test signal to the desired CP.
Architecting the Automatically Switched Transport Network
^Jy22.2J;££^>E^^^Um \
Test Signal Carried in Client Paytoad| TCP,
^
641
V
i^TssCii^^i^Si^^s' Si?™''^*^''^^^™
A
aient Layer TCP,,
V irffer^ed
Test Signal Camed in V Trait Overhead \
T y#«
, t y p e 1 LAD prodess
Server
Figure 16-46. LAD discovery processes [10]
16.11.1.1 Type 1 — Overhead in Server Layer As discussed earlier, the mechanisms defined to support the layer adjacency discovery process apply on a per-layer basis. Within each of the layer networks that support the discovery process, different mechanisms are available. These may reuse the available Embedded Communications Channels (ECCs) for the particular layer. The following mechanisms are applicable to SDH layer networks: • RS layer: within the RS layer, the JO section trace and Section DCC may be used to support discovery of the RS TCP-to-TCP adjacency. • MS layer: within the MS layer, the Multiplex Section DCC may be used to support discovery of the MS TCP-to-TCP adjacency. • HOVC layer: within the HOVC layer, the higher-order Path layer Jl trace may be used to support discovery of the HOVC TCP-to-TCP adjacency. • LOVC layer: within the LOVC layer, the lower-order Path layer J2 trace may be used to support discovery of the LOVC TCP-to-TCP adjacency. The following mechanisms are applicable to OTN layer networks: • OTUk layer: within the OTUk layer, the SM section monitoring byte and the GCCO may be used to support discovery of the OTUk adjacency. Specifically, the SAPI subfield within the SM is used to carry the discovery message. • ODUk layer: within the ODUk layer, the PM path monitoring byte and the GCC-1 and GCC-2 bytes may be used to support discovery of the ODUk adjacency. Specifically, the SAPI subfield within the PM is used to carry the discovery message.
642
Chapter 16
16.11.1.2 Type 2 — Test-Signal Method In this method, test signal generators and receivers are used for directly discovering layer adjacencies. The test signals can encapsulate the necessary identification information. This method is utilized when the associations at the fabric port levels cannot be accomplished using the server trail overhead method because the trail is not available to do so. This approach is often useful in scenarios where there are two switches connected by intermediate pre-configured switches (or nailed-up connections), e.g., two SONET/SDH switches interconnected by a time-slot interchange device. In such cases, physical port connectivity does not directly provide information on the connectivity at the fabric port level. The test-signal method, therefore, discovers the TCP-to-TCP associations by means of test signal generators and receivers. Specific test signals are used for creating in-band events that effect associations between two TCPs directly, i.e., without discovering any server layer trails. For example, in the case of SONET/SDH, the testsignals themselves can include the identity information in the form of JO, J l , or J2 strings or in the form of Section and Line DCC. It is important to remember that this method can only be used prior to establishing a link connection. 16.11.1.3 LAD Protocol (G7714.1) The format for carrying discovery messages is defined in G.7714.L The following considerations influenced the definition of this format: • In order to foster interoperability, it was decided to use one single common message format for the discovery protocol independent of the mechanism used (e.g., J0/J1/J2 trace bytes or Section/Line DCC) at any specific layer to carry this message. Of the various mechanisms available to carry the discovery messages, the J0/J1/J2 trace bytes had the greatest number of limitations in terms of available message size and acceptable formats. Consequently, the requirements of the trace byte format were used to set limits to the format for the discovery messages. • It was decided to limit the discovery message format to only printable characters so as to allow reuse of existing text-oriented man-machine language (e.g., TLl) equipment commonly used for carrying other nondiscovery trace signals (e.g., Access Point Identifier, API). Use of arbitrary bit patterns for the discovery messages could also trigger unwarranted alarms from equipment expecting printable characters. • An important consideration in designing the discovery protocol was to provide a means in new equipment for distinguishing the discovery messages from the traditional trail trace messages. As a result, G.7714.1
Architecting the Automatically Switched Transport Network
643
includes a distinguishing character ("+") as the first non-CRC byte that will never appear as the first character of a TTI (trail trace identifier). • It was also decided that the discovery message should allow for the use of any address format (e.g. NSAP, IPv4, IPv6). Recommendation G.7714.1 makes use of an external name-server that allows translation of any address format into a specific format used by the discovery message. In summary, G.7714.1 limits the discovery messages to printable characters defined by T.50 [51], uses a special distinguishing character to set itself apart from other trace messages, and requires Base64 encoding for the TCP-ID and DA ID. External name-servers may be used to resolve the G.7714.1 TCP name, allowing the TCP to have an IP, NSAP, or any other address format. The ECC mechanism further supports messages using the raw LAPD data link protocol, or alternatively, using the Point-To-Point protocol (PPP).
16.12. METHODS AND PROTOCOLS — SIGNALING The following sections describe signaling protocol extensions developed to support the requirements of G.807, G.8080, and G.7713 for PNNI, GMPLS RSVP-TE, and GMPLS CR-LDP.
16.12.1 G.7713.1 PNNI Signaling Recommendation G.7713.1 [6] defines distributed call/connection management signaling based on the ATM PNNI signaling protocol [52]. PNNI [36] has been in use since the mid-1990s for control of ATM Switched Virtual Connections. It was designed for connection-oriented packet switched networks (CO-PSN) and matches fairly closely to the functionality required for ASON signaling. In particular, PNNI already supports a form of call and connection separation through the use of the Network Call Correlation Information Element in the PNNI signaling protocol. PNNI is also intrinsically designed for bidirectional connections, as are used in ATM networks, and already has extensions defined for explicit routing, soft permanent connections, connection setup and release, crankback, and connection query. Recommendation G.7713.1 adds new functionality into PNNI to support the following: • Signaling for SONET/SDH connections • Connection persistence • Transport over an IP control network infrastructure
644
Chapter 16
The main extensions involve the addition of a new Connection Identifier Information Element (IE) format for use with transport networks allowing the specification of new connection types, and to include the concepts of SNP and SNPP, and the addition of a new Traffic Descriptors IE format defined for use with transport networks that allows the specification of traffic descriptor attributes suitable for transport networks. No new protocol elements are required to support the connection persistence desired for ASON circuits, but some procedure changes are required to handle connection persistence under conditions of control plane failure. The ATM form of PNNI is designed to release a virtual connection in case of failure of the underlying signaling link layer protocol so as to allow the resources to be freed for future use. The opposite procedure is specified for ASON PNNI: the resources must be kept active for use by the transport connection, even in the event that the signaling link is terminated. Finally, transport over an IP control network infrastructure is specified in an Annex to G.7713.1. Use is made of an existing extension of the ATM Signaling Adaptation Layer protocol defined in Q.2111 [53] (SSCOP-MCE). This extension was originally defined for transport of messages over a link with connectionless characteristics, including potential multiplexing of traffic over multiple links.
16.12.2 G.7713.2 GMPLS RSVP-TE Signaling Recommendation G.7713.2 [7] defines GMPLS RSVP-TE-based distributed connection management for ASON. It is based on the requirements defined in G.7713 and specifies a number of extensions to the GMPLS RSVP-TE signaling defined in RFC 3473 [53]. The specifications in G.7713.2 were designed to use the same basic mechanisms as, and to be compatible with, RFC 3473 when using RFC 3473 as an Internal NetworkNetwork Interface (I-NNI) protocol. Recommendation G.7713.2 added new extensions onto RFC 3473 to support the following: • Call and connection separation • Call segments • SPC services • Explicit support for separation of control, transport, and DCN identifier spaces • Address space isolation across interdomain reference points (UNI, E-NNI transport resource names)
Architecting the Automatically Switched Transport Network 16.12.2.1
645
Support for Call and Connection Separation
Within this section, we describe the approach taken towards supporting separation of calls and connections in G.7713.2. It was recognized that the GMPLS RSVP-TE signaling protocol could be extended to handle the case where a call has one or more connections, as well as the case where a call has zero connections. Recommendation G.7713.2 takes the approach where call setup and teardown are signaled simultaneously along with connection setup and teardown (in other words, the call control protocol is "piggybacked" on the connection control protocol). The rationale for the approach that was taken is summarized below. Complete separation of call and connection is utilized within PNNI, which supports a call and connection separation based on Q.2982 [55]. Recommendation Q.2982 essentially reuses the "connection" messages as "call" messages, and allows for complete dissociation of the call operation (which is a separate flow) from the connection operation. These messages contain only call-specific information elements (call identifier, bearer identifier, call capability). It was recognized that ASON GMPLS RSVP-TE could take a similar approach to supporting the call and connection separation, i.e., to reuse the Path and Resv message to set up a call session (with an associated call identifier), followed by using the same Path and Resv message to set up connections within the call (using the previously set up call identifier to link the call and connection together). Alternatively, it was also recognized that the call functions could initially be supported via utilization of a logical call and connection separation. Using the above approach, all GMPLS RSVP-TE messages associated with connection operations would remain valid for call operations. Given the limited use of Q.2982-based signaling protocol, which provides the call and connection separation for the Q.2931 [56]-based signaling network, it was decided to first provide support within G.7713.2 for the simple case of logical call and connection separation. The implications of logical call and connection separation are described below: • A call with zero connections is considered a transient behavior (e.g., during restoration activities). A steady-state call will always have one or more connections associated with it. In such cases, since the call operation will always be part of a connection operation, only a unique CALL_ID is required to identify which of the operations are associated with a particular call. • A new call request may be initiated with a new call identifier, i.e., assuming call modification constraints are satisfied, any request with an existing call identifier signifies a call modification (in other words, the addition, deletion, or modification of a connection within the call).
646 •
•
Chapter 16
Subsequent requests for call modifications may be supported by using an existing call identifier within the set of RSVP-TE messages (e.g., Path and Resv messages). Since RSVP Session was scoped to connections within the same call segment, different connections established within the same RSVP Session share the same CALL_ID. Call release is achieved by releasing all the connections that have been set up within a particular call, i.e., that share the same CALL_ID.
16.12.2.2 Support for Signaling Control Domains In the modeling terminology of MPLS, the G.7713.2 extensions to RFC 3473 amount to using LSP stitching across signaling control domains, in that a separate RSVP session is created at the boundary between any two domains, and a separate session is used within a domain (assuming that it uses RSVP internally). As in LSP stitching, the sessions are stitched together in the control plane so that a single (end-to-end) G.805 network connection is achieved. As described in Section 16.2.5, in the transport plane it may be possible for a service to be transported in different ways across individual domains, using technologies such as virtual concatenation. Consistent with the stitching approach, G.7713.2 GMPLS RSVP-TE signaling has a different source and destination address within each signaling control domain and at the boundaries. At the call level, a single source and destination name pair is maintained end-to-end in the protocol. 16.12.2.3 Summary of G.7713.2 Extensions A limited number of extensions were made to the base RSVP-TE signaling defined in RFC 3473. This section summarizes the impact of these extensions. • Messages: No new messages were defined in G.7713.2. Use of some messages was considered not necessary for ASON, in particular, ResvTear and ResvErr messages (to be further discussed below). • Procedures: No changes to procedures were specified. (While usage of ResvErr and ResvTear was not deemed necessary, if used, the procedures remain the same.) • Objects: Three new call objects are provided in G.7713.2, namely, the Generalized_UNI object defined in the OIF UNI Implementation Agreement [36], a Call_ID object, and CallOPS (within an informative Appendix of G.7713.2). Having distinguished between call and connection objects, TLVs were designed so that they do not mix call and connection objects. This setup makes it easier for entities that only deal with connections.
Architecting the Automatically Switched Transport Network
647
•
C-Types: New Session Object C-Types were added for UNI and E-NNI. The new object and C-Type code points are allocated in RFCs 3474 [57] and 3476 [58]. It should be noted that G.7713 (and subtending protocolspecific Recommendations) distinguishes between attributes that are call related versus those that are connection related, since connection-handling entities in the network generally do not have to understand or deal with call objects. 16.12.23.1
Messages and Procedures (ResvTear/RevErr)
No case needing use of the ResvTear and ResvErr messages was identified in the call/connection control requirements for ASON in G.7713. Additionally, the procedures supported by ResvTear and ResvErr were felt to be inconsistent with the desired behavior of connection establishment in transport networks, since they result in a requirement for exchange of multiple messages in order to tear down the connection in case of failure during connection setup. If a need is identified in the future, the associated procedures can be added as an extension to G.7713.2. It should be noted that no protocol violation occurs if a ResvTear or ResvErr generates PathTear in the opposite