Cisco - Router Architecture

601601 1094_06F9_c4 1094_06F9_c1 Cisco Systems, Inc. Inc. 1094_06F9_c4 © 1999, © 1999, Cisco Systems,

1 1

Cisco Router Architecture Session 601

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

2

1

Agenda

• Router Fundamentals • Layered Switching • Router Architectures/Switching Paths • Optimized Network Design • Troubleshooting 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

3

Router Fundamentals

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

4

2

What Is a Router? • Routers perform two main functions

List of Reachable Networks

OSPF, EIGRP,BGP Static Routes, Etc…

• Control path routines • Data path control (switching) Packet Frame

Packet

Layer 3 Switch

Packet Frame Frame

Packet

Layer 2 Encapsulation TTL-1 Layer 3 Checksum 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

5

Routers (Operationally) • Maintain/manipulate routing information Listen for updates/update neighbors

• Classify packets for manipulation/queuing/permit-deny, etc. Compare packets to classification lists and perform control

• Perform Layer 3 switching Create outbound Layer 2 encapsulation Layer 3 checksum TTL/hop count update

• Management/billing (statistics) Interface statistics—Netflow export Telnet, SNMP, ping, trace route, HTTP 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

6

3

Router Functionality

System Level Tasks

Control Plane

• • • • • •

Run routing protocols Maintain routing tables Check for CLI commands Increment accounting counters ICMP Queue packets, etc…

CPU

Data Plane Packet Switching

601 1094_06F9_c4

Layer 1: Retime, Regenerate Signal Layer 2: Rewrite Header and CRC Layer 3: Decrement TTL and CRC Rewrite

© 1999, Cisco Systems, Inc.

7

Routers (Layer 3 Packet Functionally) • (Attempt to) switch packets Layer 3 switching based on routing information

• (Attempt to) transmit packets Access outbound media

• Manipulate packets Change contents of packet (CAR/NAT/compression/encryption)

• Consume packets Routing protocol updates etc…/services advertisements(SAP)/ICMP/SNMP

• Generate packets Routing protocol packets/SAPs/ICMP/SNMP Tunnels—GRE, IPSec, DLSw etc… 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

8

4

Generic Router Architectures

601 1094_06F9_c4

9

© 1999, Cisco Systems, Inc.

Shared Memory

CPU Memory

CPU

Routing Table

Shared Memory

601 1094_06F9_c4

Interface

Interface

Interface

Interface

Shared Bus

Electrical Limitations Limit of Shared Bus Switches <= 20 Gbps Thus 10 Gbps/No. Line Cards = Max Line Rate

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

10

5

SM Data Path (Conventional)

CPU Memory

CPU

Routing Table

Shared Memory

Interface

Interface

Interface

Interface

Shared Bus

One Packet/Switching Cycle 601 1094_06F9_c4

11

© 1999, Cisco Systems, Inc.

SM Data Path (Distributed Processors) CPU Memory CPU

Routing Table CPU

Interface CPU

Interface CPU

Interface CPU

Shared Memory 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

Interface

12

6

Cross Bar Data Path CPU Memory CPU

Routing Table CPU

Interface

i/p

CPU

Interface o/p

CPU

Interface CPU

Interface

601 1094_06F9_c4

13

© 1999, Cisco Systems, Inc.

Cross Bar Data Path CPU Memory CPU

Routing Table CPU

Interface

Bit Slicing Allows Multiple Switching Fabrics

CPU

Interface CPU

Interface CPU

Interface

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

14

7

Cisco Router Components

601 1094_06F9_c4

15

© 1999, Cisco Systems, Inc.

General Router Hardware

Flash

CPU

NVRAM ROM

Bus Interface

RAM

System Bus

601 1094_06F9_c4

Interface

Interface

Interface

Network Controller

Network Controller

Network Controller

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

16

8

Software Components • ROM monitor

• Device microcode

Startup diagnostic code Contains exception handling

Part of Cisco IOS that deals with network controllers Microcode deals with modular interface processors

• RxBoot Host mode Operating System Used for downloading full Cisco IOS®

• Cisco IOS Internetwork operating system Contains process scheduler, memory manager, parser Also contains protocolspecific code for packet handling 601 1094_06F9_c4

• Configuration register 16 Bits, specifies router startup parameters

• Configuration file Startup-config: Contains configuration info Running-config: Currently active configuration

17

© 1999, Cisco Systems, Inc.

Memory Usage

Boot Flash

PCMCIA Flash

RxBoot

Cisco IOS File

NVRAM

Cisco IOS Exec

Main

Running config

Routing Table

System Buffers

Data Structures

Layer 2/3 Cache

Interface Buffers

Startup-Config Config Reg. ROM ROM Monitor

601 1094_06F9_c4

CPU

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

RAM

I/O

18

9

Cisco Router Buffers and Queues

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

19

Router Interface Buffers • Interface FIFO A very small amount of buffer memory (for large MTUs not even one packet in size) used to store bits as they arrives from the wire and are dealt with by the interface driver These buffers are NOT configurable

• Interface Rx and Tx RING On some platforms these buffers are used by the interface driver for reception and transmission of packets Each interface has a FIFO Rx/Tx Ring Inbound (Rx) they are used to store a packet until an Interface buffer is available. Outbound (Tx) until the interface can transmit to the wire They can exist on the interface card or in shared memory These buffers are NOT normally configurable 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

20

10

Router Internal Buffers • Buffer Headers These data structures containing information about related buffer (e.g. location pointers, size, etc.). They are mostly located in main processor memory for all buffers. In some cases headers or particle headers may be stored in shared I/O memory for speed The purpose of buffer headers is to keep track of buffers and enqueue them for various processes

• Shared or main memory system buffers These buffers are used when packets are bound for the processor for either consumption or process switching System buffers are sized as small (104), middle (600), big (1,524), very big (4,520), large (5,024), and huge (18,024) The total number of buffers depends on available DRAM System buffers can grow or Trim on demand and can be configured These buffers are public. All interfaces can use them 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

21

Router Internal Buffers • Shared memory interface buffers These buffers are used to store packets between the interface driver and the switching path (not process switching) software (e.g optimum switching) These buffers are allocated at startup or after OIR. The number of buffers depends on the speed and MTU of the interfaces available These buffers are Interface specific These buffers are NOT configurable

• Shared memory interface particle buffers Used as Packet buffers on some platforms and VIP cards Particle Buffers are located in Shared I/O Memory Their size is 512 bytes (or 128 byte multicast/broadcast) The incoming packets are “scattered” into 512 byte particles and then “gathered” into a contiguous packets for transmission” These buffers are NOT configurable 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

22

11

Contiguous Buffers • Contiguous buffers Contiguous buffers (older platforms as well as RSP/RSM) store packets in buffers sized with respect to the interface media MTU and speed The amount of buffers is based on grouping the MTUs of the interfaces e.g. Ethernet = 1500b MTU, Token Ring/FDDI = 4500b MTU Let’s assume we have 6 Ethernet and 2 FDDI = 2 buffer pools. (all the Ethernet I/f share the 1500b pool and all the FDDI share the 4500b pool) Based on the aggregate bandwidth we get: Ethernet = (60/60+200) = 23% and FDDI = (200/60+200) = 77% We then apportion the available buffers (let’s say it’s a 7K SP) (.23*504K)/1500 = 79 buffers and (.77*504K)/4500 = 88 Buffers Therefore, 601 1094_06F9_c4

79/6 = 12 buffers/Ethernet interface and 88/2 = 44 buffers/FDDI interface 23

© 1999, Cisco Systems, Inc.

Contiguous Buffers • Contiguous buffers Packet buffers (contiguous) can be wasteful in terms of memory (Frames are rarely all full size), but more importantly are not as efficient as particles when it comes to packet replication In contiguous buffers the packets is treated as a whole, therefore to create a replication of the packet with a different output header requires a completely new packet to be created 1500

Ethernet Interface Buffer Pool 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

4500

FDDI Interface Buffer Pool 24

12

Particles vs. Contiguous Buffers • On some newer platforms (36xx, 72xx, 71xx, VIP2) packet memory is allocated in “particles”. Particles are either 1024b (36xx), 512b or 128b chunks of memory • Packets are divided into particle size blocks and stored in free particles • Particles have an associated Particle Buffer header which stores information as to which particles constitute an entire frame and or an?

I/P Buffer Header

Particle Headers Packet Memory Divided Into Particles O/P Buffer Header

Particle Headers 601 1094_06F9_c4

25

© 1999, Cisco Systems, Inc.

Particles vs. Contiguous Buffers • We have the ability to “clone” particle buffer headers • This allows us to “replicate” a packets particles a number of times without actually replicating the packet itself for every outbound interface. This significantly improved multicast performance I/P Buffer Header New Header New Header Particle Headers Cloned Particle Headers

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

Particle Headers Packet Memory Divided into Particles O/P Buffer Header O/P Buffer Header 26

13

Types of Queues • System interface queues Input hold queues Used to queue packets in system buffers for process switching These queues are based in Main Processor Memory The size of Input Hold queue is configurable per interface basis Use “Show Interface” to look at Input Queue statistics Output hold queues These queues are used for packets in System buffers after they have been process switched and are waiting to be transmitted by the interface driver These queues exist in main processor memory The size of input hold queue is configurable per interface basis Use “Show Interface” to look at output queue statistics 601 1094_06F9_c4

27

© 1999, Cisco Systems, Inc.

Types of Queues • Interface Queues Receive Queues The queues are used for incoming packets that wait in interface buffers for Fast Switching code Receive queue usually has a size limit (called RQL) which is calculated based upon various factors By default the RQL = Total buffers in a pool/no. interfaces in that pool (If for any reason this results in less than 16KB of buffers RQL= 16384/MTU) Transmit Queues These queues are used for outgoing packets that wait in interface buffers for transmission by outgoing interface driver code 601 1094_06F9_c4

Transmit queue usually has a size limit (called TQL) which is calculated based upon various factors © 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

28

14

Router Operating System Details

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

29

Cisco IOS Image and File Storage • BOOT ROM EPROM used for startup diagnostic code and ROM monitor (read only) and to load Cisco IOS

• NVRAM Used to store startup configuration (rewritable). Configuration register settings

• PCMCIA FLASH Portable storage of Cisco IOS image, configuration files etc… (rewritable)

• FLASH Onboard storage of full Cisco IOS (rewritable). In some cases Cisco IOS execution 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

30

15

Operational Storage • DRAM Used to store loaded Cisco IOS, running configuration, route tables, switching caches, processor and switched packets

• SRAM Used on platforms to deal with high-speed switching and high-speed interfaces

• FIFO First-In, First-Out memory used for interface buffering (not queued—circular buffer or RING) 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

31

Elements of Execution • ROM MONITOR (System Bootstrap, or Bootstrap code) Hardware configuration and system diagnostics performed at system startup

• RxBOOT (Boot Helper image, Helper Cisco IOS, or Bootstrap Image) A Subset of the Cisco IOS, used when a valid Cisco IOS image is not present allowing a user to download a full Cisco IOS image from the network

• Cisco IOS Cisco IOS normally resides in Flash or PCMCIA Flash card and is loaded into processor memory (DRAM) for execution In some platforms it may also run from Flash memory to save DRAM

• ROUTER CONFIG The Startup configuration is stored in NVRAM and a copy is loaded in DRAM at startup 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

32

16

Configuration Register Function • Force the system into the bootstrap program • Select a boot source and default boot filename • Recognition of break signal from console • Control broadcast addresses • Set the console terminal baud rate • Load operating software from ROM • Enable booting from a Trivial File Transfer Protocol (TFTP) server 601 1094_06F9_c4

33

© 1999, Cisco Systems, Inc.

Router Startup ROM Monitor Diagnostic Check, Console Setup, Memory Sizing, Config Register Check Loads Rxboot, or Stays in ROMMON [rommon>]

RxBoot Builds Basic Data Structures, Interface Setup, Host Mode Functionality, Startup-Config Check loads Cisco IOS, or Stays in RxBoot [router(boot)>]

Cisco IOS Interface Setup, Router Functionality, Allocate Buffers, Loads Startup-Config Boot Process Complete [router>] 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

34

17

How Routers Receive and Transmit Packets

601 1094_06F9_c4

35

© 1999, Cisco Systems, Inc.

Receiving Packet Shared Memory

Buffer Headers Process Scheduler

66

44

CPU

Packet Packet on on Interface Interface

22

Interface Interface Driver Driver Code Code

33

Frame Frame Check Check

22 33

Fail

x

55

11

Discard Frame Update Frame errors, e.g. “Runt, Giant, CRC”

OK 11

44

55

66

601 1094_06F9_c4

No

Yes

R

Interface FIFO Buffer

Assign Assign aa Free Free Buffer Buffer Header Header

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

Discard Frame Increment “Overrun” If CPU Too Slow

Increment “Ignore” Free Free Buffer Buffer No Increment “No Header Header Available Available Buffer” *Drop Packet* Yes Return to Scheduled Processes Move Move Packet Packet to to

Shared Shared Memory Memory

36

18

Receiving a Packet (Process Switching) Classify Classify Packet Packet

System Memory IP Process Input Hold Queue

Yes

Fast Cache

Can Can the the Packet Packet Be Be Fast Fast Switched Switched

88

Assign Assign System System Buffer Buffer Return Return Packet Packet Buffer Buffer to to Free Free List List

99

Enqueue Enqueue Packet Packet for for Process Process Switching Switching

If “Input Hold Queue” Full, *Drop Packet* Increment “Input Queue Drop”

Rx Rx Interrupt Interrupt Complete Complete Return Return CPU CPU to to Scheduled Scheduled Tasks Tasks

How Is This Box Related?

77

99

88

88

No

CPU

System System Buffers Buffers

Go to FastSwitching

77

Interface Rx Interrupt FIFO Buffer 601 1094_06F9_c4

37

© 1999, Cisco Systems, Inc.

Transmitting a Packet (Process Switching) System Memory Output Hold Queue

Forwarding Forwarding Decision Decision

11 11

10 10 12 12

Write Write New New Header Header Over Over Old Old Header Header

11 11

Packet Packet Is Is Enqueued Enqueued in in for for Output Output I/f I/f in in Output Output Hold Hold Queue Queue

How Is this Box Related?

12 12

Packet Packet Is Is Transmitted Transmitted

If Output Hold Queue Is Full, Packet Is Dropped With an “Output Queue Drop”

CPU

System System Buffers Buffers

Interface FIFO Buffer

601 1094_06F9_c4

10 10

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

38

19

Transmitting a Packet (Fast Switching)

88

77

Can Can the the Packet Packet be be Fast Fast Switched Switched

o/p

CPU

10 10 Packet Buffers

11 11

Interface FIFO Buffer

601 1094_06F9_c4

Yes

77

TQL i/p 99

Classify Classify Packet Packet

Fast Cache

Packet Memory

If No Output Buffer Available Drop Packet With “Output Buffer Failure” Note: If “Transmit Buffer Backing-Store” Is Enabled the Packet Will Be Moved to System Buffers and Dequeued as with Process Switching An Output Buffer Swap Is Incremented

© 1999, Cisco Systems, Inc.

Go to FastSwitching

88

Obtain Output Header From Fast Cache

99

Rewrite Header and Move Buffer Header or Move Packet to Output Interface Buffer

10 10

Enqueue Packet for Interface Transmit Queue

11 11 Transmit Packet 39

Packet Switching “Processes” and “Functions”

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

40

20

Processes and Functions • Process switching is a collection of functions based on a particular protocol that is being switched • The Scheduler gives processor time to each “Process” which may call a number of “Functions” before the “Process” is complete • Each “Process” is run to completion unless a higherpriority “Process” is invoked. The original “Process” is suspended until such time as the higher-priority “Process” is completed at which time the original “Process” will continue to be run

IP Input

Compression Compression Encryption Encryption

Process 601 1094_06F9_c4

I/P I/P ACLs ACLs

CAR CAR

Completed

Functions 41

© 1999, Cisco Systems, Inc.

Processes and Functions • For Fast and ½ Fast Switching there is no concept of a “Input Process” • The Fast Switching code, invoked after an Rx Interrupt, calls various functions, independently as defined by the configuration file for an interface or protocol • Some features are run in Rx Interrupt mode, but the packets are moved to system buffers depending on the requirements of the feature (e.g. NAT) MLPPP

NAT

IPSec

Function Calls Rx Interrupt 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

42

21

Processes and Functions • Dequeuing is the function used when a packet has been process switched and needs to be passed to the output interface, or when an interface has “fancy” queuing enabled and again packets are stored in system buffers until they are dequeued (based on queuing mechanism) System System Buffers Buffers

Output Output Hold Hold Q Q

Shared System Memory (DRAM)

Output I/f

Packet Packet Buffers Buffers “Fancy” “Fancy” Queued Queued Packets Packets

Shared Packet Memory (SRAM) 601 1094_06F9_c4

43

© 1999, Cisco Systems, Inc.

Processes and Functions • In the case of “fancy” queuing being applied to an interface, the Output Hold Queue represents either the (4) Priority, (>16) Custom or (>256) Weighted Fair queues • In Process switching this is a FIFO queue by default Process Switching Priority Queuing Custom Queuing

L

N

M H

0

1

2

3

Output Hold Q

FIFO

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

44

22

Process Switching Dequeue Dequeue Packet Packet from from Input Input Hold Hold Queue Queue

IP Process Input Hold Queue Input Processing

Compressed Compressed

System Memory

No

Encrypted Encrypted No

Input Input ACL ACL

CPU

Permit

Yes

Yes

Deny

Decompress Packet

Decrypt Packet

*Drop Packet*

Conform/Not Conform CAR Action

CAR CAR Is Is TTL>0 TTL>0

No

*Drop Packet*

Yes

TTL=TTL-1 TTL=TTL-1 NAT NAT Transform Transform

Yes

Out->In NAT

Attempt Attempt to to Forward Forward Packet Packet

= Packet Compared To Access List in Configuration 601 1094_06F9_c4

45

© 1999, Cisco Systems, Inc.

Process Switching IP Process Output Hold Queue Output Processing

Broadcast Broadcast No

System Memory

Policy Policy Route Route No

NAT NAT Transform Transform

CPU

Yes

Yes

Yes

Broadcast Packet

Policy Route

In->Out

No

Route Table

Look Look Up Up Route Route Output Output ACL ACL

Deny

*Drop Packet*

Permit

Encryption Encryption Compression Compression

Yes Yes

Encrypt Packet Compress Packet

= Packet Compared To Access List in Configuration 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

46

23

How Routers Make Switching Decisions

601 1094_06F9_c4

47

© 1999, Cisco Systems, Inc.

Routing Tables • Forwarding information populated by IGPs and EGPs RIP, OSPF, Statics, EIGRP etc… c3620#sh ip route Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default U - per-user static route, o - ODR Gateway of last resort is 0.0.0.0 to network 0.0.0.0 1.0.0.0/24 is subnetted, 2 subnets 1.1.1.0 is directly connected, Ethernet0/0 1.1.2.0 is directly connected, Loopback1 10.0.0.0/24 is subnetted, 1 subnets C 10.64.217.0 is directly connected, Ethernet0/1 S* 0.0.0.0/0 is directly connected, Ethernet0/1 C C

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

48

24

Adjacency Information • Discovered in various ways including ARP protocol c3620#sh arp Protocol Address Internet 1.1.1.1 Internet 1.1.1.5 Internet 32.97.105.46 Internet 1.1.1.6 Internet 171.68.225.9 Internet 209.17.176.120 Internet 204.71.200.74 Internet 128.32.18.166 Internet 152.163.241.223 Internet 38.15.254.206 Internet 206.79.171.51

601 1094_06F9_c4

Age (min) 118 153 4 145 142 91 156 161 144 153

Hardware Addr 0010.7b1f.4a61 00a0.c903.6077 0000.0caa.2350 00a0.c903.6064 0000.0caa.2350 0000.0caa.2350 0000.0caa.2350 0000.0caa.2350 0000.0caa.2350 0000.0caa.2350 0000.0caa.2350

Type ARPA ARPA ARPA ARPA ARPA ARPA ARPA ARPA ARPA ARPA ARPA

Interface Ethernet0/0 Ethernet0/0 Ethernet0/1 Ethernet0/0 Ethernet0/1 Ethernet0/1 Ethernet0/1 Ethernet0/1 Ethernet0/1 Ethernet0/1 Ethernet0/1

© 1999, Cisco Systems, Inc.

49

A Cached Table of Forwarding and Adjacency Information • Initialized after a packet has been successfully Process Switched and *Can* be cached… c3620#sh ip ca verb IP routing cache 3 entries, 516 bytes 232 adds, 229 invalidates, 0 refcounts Minimum invalidation interval 2 seconds, maximum interval 5 seconds, quiet interval 3 seconds, threshold 0 requests Invalidation rate 0 in last second, 0 in last 3 seconds Prefix/Length 1.1.1.5/32-24 1.1.1.151/32-24 171.69.0.0/16-0

601 1094_06F9_c4

Age Interface Next Hop 21:07:47 Ethernet0/0 1.1.1.5 14 00A0C903607700107B1F4A610800 00:06:22 Ethernet0/0 1.1.1.151 14 000039542C0800107B1F4A610800 01:16:08 Ethernet0/1 171.69.10.34 14 00000CAA235000107B1F4A620800

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

50

25

Finding an Entry in a Table • We could look sequentially through each entry until we find the one we are looking for • This could end up being very time consuming and (unless the tables were sorted inversely with time) probably end up with the most useful entries (i.e. the most recent ones) at the end • Also it is possible that a more specific and more optimized entry maybe missed if we take the first one we find sequentially Prefix/Length 1.1.1.5/32 1.1.1.151/32 171.68.0.0/16 171.68.1.0/24 199.2.54.0/24 601 1094_06F9_c4

Age Interface Next Hop 20:50:18 Ethernet0/0 1.1.1.5 00:08:20 Ethernet0/0 1.1.1.151 00:43:59 Ethernet0/1 171.70.10.70 This would come first 00:08:39 Ethernet0/2 171.68.11.34 This is more specific 00:03:54 Ethernet0/1 199.2.54.193

51

© 1999, Cisco Systems, Inc.

How Binary Trees Help Us Speed up Performance Linear List Look-Up Find X=4 X=n X=n X=n X=n

601 1094_06F9_c4

1 2 3 4 5 6 7 8 9

ü

This Is OK for Small Tables of Information Where the Maximum Look up Time Is the Number on Entries in the List

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

52

26

How Binary Trees Help Us Speed up Performance Binary List Look-Up Find X=4 Find Midpoint of the List

Find Midpoint of the List

1 3 4 6 8 9 11

An Entry in a Table W/2 Billion Entries Will Take 32 Iterations

1 3 4

ü

4

If X= -> L, Found, If X< -> L, look in first half of the list, else look in the second half

If X= -> L, Found, If X< -> L, look in first half of the list, else look in the second half 601 1094_06F9_c4

Or… 53

© 1999, Cisco Systems, Inc.

How Binary Trees Help Us Speed-Up Performance Binary List Look-Up Find X=4 LHS

RHS This Is a Node (In This Case the Root Node)

6

Compare X to Root Node If X = RN Done If X < RN Search in LHS If X > RN Search in RHS

3 1

9 4

8

11

Look-Up Time = (Max = Logn, Where N=no. Nodes) 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

54

27

Cisco Router Internals (Low-Mid Range)

601 1094_06F9_c4

55

© 1999, Cisco Systems, Inc.

Low-End Systems

Main DRAM Cisco IOS (Not 1600/2500) Running Config Data Structures

Shared DRAM Switch Cache Routing Tables Buffer Headers

SRAM 7200 (Npe150/200)

Tx and Rx Rings

601 1094_06F9_c4

Contiguous System Buffers (Small, Large, Huge, Etc.) Public, Dynamic, Configurable Contiguous Interface Buffers Private, Static, Configurable Particle Based in 7200 (512 B) 3600 (1524 B), Nonconfigurable

Particle-based Interface Buffer. (3 Hi BW PA Only), Nonconfigurable

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

Tx and Rx Descriptor Rings Private, Circular Linked Lists

56

28

Cisco 1000/2500 Family

601 1094_06F9_c4

57

© 1999, Cisco Systems, Inc.

Cisco 100x Series

I/O Buses Serial CSU/DSU BRI S/T, U

601 1094_06F9_c4

Boot ROM M 68360 SCC

Ethernet

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

CPU Bus

CPU Console

NVRAM PCMCIA DRAM

58

29

Cisco 160x Series

WIC

Boot ROM

WIC Slot

Console

I/O Buses Serial CSU/DSU BRI S/T, U

601 1094_06F9_c4

M 68360 SCC

CPU Bus

NVRAM

CPU

PCMCIA DRAM SIMM On Board DRAM

Ethernet

59

© 1999, Cisco Systems, Inc.

Cisco 25xx Series

Async 2509-2512 Hub Ports 2505, 2507 2516

Dual UART M 68030

Sys Ctrl ASIC

CPU Bus

Mgmt Card 2517-2519 Daughter and Hub Cards

WIC Slots 2524, 2525 System Bus

WIC

Boot ROM NVRAM PCMCIA Flash

Ether/TR WAN Intf

601 1094_06F9_c4

On Board DRAM

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

DRAM SIMM

60

30

Cisco 3600 Family

601 1094_06F9_c4

61

© 1999, Cisco Systems, Inc.

PCI Bridge

PCMCIA

Dual UART

R 4700 CPU Bus Sys Ctrl GT 64010

Boot ROM I/O Bus

Network Modules

PCI Bridge PCI Bus 0

Network Modules

PCI Bus 1 PCI Bus 2

Cisco 36x0 Series

NVRAM Flash DRAM

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

62

31

Cisco 4000 Family

601 1094_06F9_c4

63

© 1999, Cisco Systems, Inc.

DBus

NIMs

Dual UART M 68030

Sys Ctrl Logic Shared DRAM

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

CPU Bus

Control Bus

Cisco 4000 and 4000M

Boot ROM NVRAM Flash / EPROM Main DRAM

64

32

Cisco 4500 and 4700

Power Supply

601 1094_06F9_c4

Control Bus

R 4600 R 4700

Dual UART

CPU Bus Other Logic Shared DRAM

Sys Ctrl ASICs

I/O Bus

DBus

NIMs

Layer 2 Cache 4700 Only

Boot ROM NVRAM Flash

Main DRAM

Boot Flash

65

© 1999, Cisco Systems, Inc.

Low-Mid Router Comparison CPU Bus

4000

M68030

CISC

40 MHz

32 bit

NIM

-

4000M

M68030

CISC

40 MHz

32 bit

NIM

-

4500

R4600

RISC

100 MHz

64 bit

NIM

-

4500M

R4600

RISC

100 MHz

64 bit

NIM

-

4700

R4700

RISC

133 MHz

64 bit

NIM

512 KB

4700M

R4700

RISC

133 MHz

64 bit

NIM

512 KB

3620

R4700

RISC

80 MHz

64 bit

NM,WIC,VIC

-

3640

R4700

RISC

100 MHz

64 bit

NM,WIC,VIC

-

25xx

M68360

CISC

20 MHz

32 bit

Built in

-

17xx

?????

??????

BI,WIC,VIC

-

M68360

????? CISC

?????

160x

33 MHz

32 bit

BI,WIC,VIC

-

100x

M68360

CISC

25 MHz

32 bit

Built in

-

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

INTERFACES

4xxx

TYPE

36XX

PROCESSOR

601 1094_06F9_c4

CLOCK

Layer 2 CACHE

CPU

66

33

High-End Systems Main DRAM Switch Cisco IOS (Not 1600/2500) Cache Running Routing Configuration Tables Data Buffer Structures Headers Contiguous System Buffers Public, Dynamic, Configurable

DRAM

IProc

Microcode Local Buff Tx and Rx Rings 601 1094_06F9_c4

Micro Processor Network Controllers

SRAM

CPU RP SP/SSP

RSP SRAM

Contiguous Interface Buffers Shared, Static, Nonconfigurable

VIP

Particle Based 256 B Local Buffs Tx and Rx Rings

Micro Processor

DRAM

VIP Cisco IOS

Distributed Cache

PA Network Controllers

PA Network Controllers

© 1999, Cisco Systems, Inc.

67

Cisco 7200/7500 Family

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

68

34

PCI Bridge

PA 3

PCI Bridge

PA 1

PCI Bridge

PCI Bus 2

PA 5

PCI Bus 1

Cisco 720x Series PCI Bridge

PA 6

PCI Bridge

PA 4

PCI Bridge

PA 2

I/O Controller Fast Ether

PCMCIA

SRAM ! NPE-100

PCI Bus 0 CPU Bus

Sys Ctrl GT 64010

Dual UART

Boot ROM

I/O Bus NVRAM 601 1094_06F9_c4

Boot Flash

CPU NPE

PCI PCI Bridge Bridge

Midplane

DRAM

EEPROM

R 4700 R 5000 Layer 2 Cache NPE-200 69

© 1999, Cisco Systems, Inc.

SP/SSP EEPROM

Bit Slice Proc.

Multi Bus Intf Logic

Local Bus SRAM

CxBus Intf DMA Logic

Diag Bus Intf Logic

Multi Bus

Cisco 70x0—RP and SP/SSP I/O Devices

RP I/O Ctrl

M 68040

CPU Bus Multi Bus Intf Logic

Diag Bus Intf Logic

DRAM

Diag Bus Cx Bus Fan Tray

601 1094_06F9_c4

Intf Proc.

Intf Proc.

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

Cx Bus Arbiter

70

35

Cisco 70x0—RSP7000 Dual UART

EEPROM

Boot ROM

Envm Logic

NVRAM

DRAM I/O Bus

CI Board

PCMCIA

Diag Bus Intf Logic

Register FPGA Diag Bus FPGA

Boot Flash

RSP7K

Sys Ctrl ASICs

R 4600 CPU Bus SRAM

MemD Ctrl ASICs

QA ASIC

Diag Bus Cx Bus Fan Tray

601 1094_06F9_c4

IP/VIP

IP / VIP

Cx Bus Arbiter

71

© 1999, Cisco Systems, Inc.

Cisco 75xx Series Dual UART

NVRAM

I/O Bus

Boot ROM

RSP

PCMCIA

Register FPGA Diag Bus FPGA

Boot Flash

Sys Ctrl ASICs

DRAM Layer 2 Cache

R 4600 R 4700 R 5000

CPU Bus MemD Ctrl ASICs

SRAM QA ASIC

Diag Bus Cy Bus 1

Cy Bus 0 IP/VIP

601 1094_06F9_c4

IP/VIP

Cy Bus Arbiter

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

IP/VIP

IP/VIP

72

36

VIP PCI Bridge 2

PCI Bus 0

DRAM

PCI Bridge 1

Boot ROM SRAM PMA ASICs

DRAM Ctrl ASICs

I/O Ctrl ASIC

Layer 2 Cache

CYA ASICs

EEPROM

CBus

601 1094_06F9_c4

R 4600 R 4700 R 5000

CPU Bus Packet Bus

PA

PCI Bus 1

PA

PCI Bus 2

Cisco 75xx Series—VIP

Diag Bus

73

© 1999, Cisco Systems, Inc.

High End Router Comparison TYPE

RSP1

R4600

RISC

100 MHz

64 bit

IP,VIP1,VIP2

RSP2

R4600/R4700

RISC

100 MHz

64 bit

IP,VIP1,VIP2

-

RSP4

R5000

RISC

200 MHz

64 bit

IP,VIP1,VIP2

512 KB

VIP2-15

R4700

RISC

100 MHz

64 bit

PA

512 KB

VIP2-40

R4700

RISC

100 MHz

64 bit

PA

512 KB

VIP2-50

R4700

RISC

200 MHz

64 bit

PA

512 KB

NPE100

R4700

RISC

150 MHz

64 bit

PA,IO -FE

512 KB

NPE150

R4700

RISC

150 MHz

64 bit

PA,IO -FE

512 KB

NPE200

R5000

RISC

200 MHz

64 bit

PA,IO -FE

512 KB

RP

M68040

RISC

40 MHz

32 bit

IP, VIP1

-

RP

M68040

RISC

40 MHz

32 bit

IP, VIP1

-

RSP7K

R4600

RISC

100 MHz

64 bit

IP, VIP1,VIP2

-

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

INTERFACES

-

7500

CPU Bus

7200

CLOCK

Layer 2 CACHE

PROCESSOR

7000

CPU

74

37

GSR Family

601 1094_06F9_c4 1094_06F9_c1

75

© 1999, Cisco Systems, Inc.

GSR Switch Fabric Switch Fabric Cards CPU

I/f

CPU

I/f

CPU

I/f

CPU

I/f

CPU

I/f

CPU

I/f

CPU

I/f

CPU

I/f

CPU

I/f

CPU

I/f

CPU

CPU

RP

RP

Clock and Scheduler Cards 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

76

38

Each Card can Switch 15Gbps

GSR Throughput

Transceivers

CPU

I/f

CPU

RP 1.25Gbps per Trace 601 1094_06F9_c4

77

© 1999, Cisco Systems, Inc.

GSR RP

Layer 2 Cache

DRAM

SRAM

Tiger Asic

CSAR

R5000 CPU

FIA

SLI

Tranciever

FIA

SLI

Tranciever

R5000 CPU Executes Cisco IOS Software —200mhz RISC Processor

I/O BUS

Layer 2 CACH—512KB Write through Data and Instruction Cache

Ethernet

NVRAM

DRAM Memory for Main Storage Supporing up to 256MB

PCMCIA Console Flash SIMM

TIGER ASIC Connects RP to DRAM, CSAR I/O Bus

Boot ROM

CSAR Segments and Reassembles Packets SAR SRAM Provides a Cache for the CSAR FABRIC INTERFACE ASICS Manage Cisco Cells to and from the Fabric SLI ASICS Encode Packets for Transmission on the Fabric 8B/10B Encoding Similar to Gigabit Ethernet or FDDI

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

78

39

GSR Switch Processor Configurations

Mode

CSC

SFC

Bandwidth Clock Redundancy

Fabric Redundancy

Entry Level

1

0

622Mbps

none

none

Redundant Entry

2

0

622Mbps

1:1

1:1

High BW

1

3

2.4Gbps

none

none

High BW

2

2

2.4Gbps

1:1

none

Redundant High BW 2

3

2.4Gbps

1:1

1:N

601 1094_06F9_c4

79

© 1999, Cisco Systems, Inc.

GSR Interface Card

Tx

512Kb 512Kb Burst Burst Buffer Buffer

Rx

Processor

512Kb 512Kb Burst Burst Buffer Buffer

SQE 16-64MB 16-64MB Buffer Buffer Memory Memory

16-64MB 16-64MB Buffer Buffer Memory Memory

CPU

I/f

Fabric Interface

601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

80

40

Head of Line Blocking (HOL)

Blocked (grrrrr!!)

601 1094_06F9_c4

81

© 1999, Cisco Systems, Inc.

Output Queue CPU

C Interface

C

A

C

B

Delayed/ Dropped

B Interface

Congested

C

Interface

Shared Memory 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

82

41

“Lane Control” B

A

D

C

B A

C

B A

D

D A

B

C

C

D 601 1094_06F9_c4

83

© 1999, Cisco Systems, Inc.

All Destinations Have a Lane B

A

D

C

B A

C

B A

D

D A

B

C

C

No interface (Outbound) Can affect another Interface

D 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

84

42

Virtual Input Queuing CPU FIFO

C

FIFO

B

C

Interface

C

A

Delayed/ Dropped

B Interface

Congested

C

Interface

Shared Memory 601 1094_06F9_c4

85

© 1999, Cisco Systems, Inc.

Virtual Input Queue CPU FIFO

C

C

C

Queue Scheduler

FIFO Interface

B

A

B

Interface

Congested

C

Interface

Shared Memory 601 1094_06F9_c4

© 1999, Cisco Systems, Inc.

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

86

43

Please Complete Your Evaluation Form Session 601

601 1094_06F9_c4 1094_06F9_c1

© 1999, Cisco Systems, Inc.

87

601 1094_06F9_c1

© 1999, Cisco Systems, Inc.

88

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA. Presentation_ID.scr

44