|
|
This chapter presents basic WAN and serial line diagnostics, a series of common symptoms, possible causes of those symptoms, and general suggestions for identifying, isolating, and resolving those causes. The emphasis here is on symptoms and problems associated with WAN connectivity.
This chapter consists of the following sections:
The symptom/cause/action modules consist of the following sections:
When troubleshooting WAN problems, there are a variety of tools and techniques that can be applied to isolate the cause of interruptions. Some commonly used techniques are outlined in the discussion that follows. Where appropriate, pointers to other parts of this manual are included. In addition, when information specific to a particular technology, media, or protocol is needed, supporting notes are included in the text.
The following tools are universally applicable when gathering information to troubleshoot WAN links:
The application of these tools is described in the discussions that follow.
One of Cisco's most useful built-in diagnostic tools is the show interfaces EXEC command. The specific information displayed depends on the interface type being examined (serial, Ethernet, token ring, or FDDI) and the type of encapsulation to which the network is connected (such as X.25 or SMDS). This discussion focuses on information in the serial version of the display and on outlining the specific fields used to diagnose WAN/serial line connectivity problems.
Figure 7-1 illustrates the show interfaces serial number command output for a basic serial interface.
The interface depicted is not running packet-switched software. The fields presented in this display are detailed in your Router Products Configuration and Reference publication.
Information for analyzing the key diagnostic fields in this display is provided in the sections that follow. Several important diagnostic fields are presented in Figure 7-1 that help isolate WAN and serial line problems. These include:
There are five possible "problem" states that can be identified in the interface status line (see Figure 7-1) of the show interfaces serial display:
The causes and suggested actions associated with each of these conditions is summarized in Table 7-1.
| Status Line Display Output (Symptom) | Possible Causes and Suggested Actions | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Serial 0 is down, line protocol is down | Possible Causes:
| ||||||||||
|
| Suggested Actions:
Step 1: Inspect LED indicators for failure conditions (refer to your hardware installation and maintenance publication for LED descriptions). Step 2: Insert breakout box; check all control leads Step 3: Install correct applique if wrong Step 4: Swap faulty part(s) | ||||||||||
| Serial 0 is up, line protocol is down (DTE mode) | Possible Causes:
| ||||||||||
| Suggested Actions:
Step 1: Put modem or CSU/DSU in local loopback mode and determine if line protocol comes up in show interfaces output. | |||||||||||
| Step 2: Enable debug serial-interface | |||||||||||
| Step 3: If the line protocol does not come up in local loopback mode (keepalive counter does not increment), router hardware problem is likely; swap router interface hardware. | |||||||||||
| Step 4: If the keepalive counter increments and line comes up, problem is not in local router. Troubleshoot serial line per loopback and ping tests described later in this chapter. | |||||||||||
| Serial 0 is up, line protocol is down (DCE mode) | Possible Causes:
| ||||||||||
| Suggested Actions:
Step 1: Set DTE device to SCTE mode if possible. | |||||||||||
| Step 2: Set DCE applique to SCT mode if DTE does not support SCTE. | |||||||||||
| Step 3: If protocol is still down, then there is a possible hardware failure or cabling problem. Insert a breakout box and observe leads. | |||||||||||
| Step 4: Replace faulty part(s) as necessary. | |||||||||||
| Serial 0 is up, line protocol is looped | Possible Causes:
| ||||||||||
| |||||||||||
|
| Suggested Actions:
Step 1: Check router configuration using the write terminal command. Look for any instances of the loopback interface subcommand. | ||||||||||
| Step 2: If found, enter configuration mode and remove the loop using the no loopback interface subcommand for the appropriate interface. | |||||||||||
| Step 3: If not found, examine the DSU/CSUs to determine if they are configured in manual loopback mode. If they are, disable manual loopback. If not, contact the leased-line or other carrier service for line troubleshooting assistance. | |||||||||||
| Step 4: Reset one of the routers; inspect line status display; if protocol comes up, then Problem #2 is likely cause. No other action is needed. | |||||||||||
| Serial 0 is up, line protocol is disabled
| Possible Causes:
| ||||||||||
| Suggested Actions:
Step 1: Troubleshoot with serial analyzer and breakout box; look for hardware control leads toggling (CTS and DSR) Step 2: Swap out bad hardware as required (DSU/CSU, switch, local or remote router). | |||||||||||
| Serial 0 is administratively down, line protocol is down | Possible Causes:
| ||||||||||
| Suggested Actions:
Step 1: Check router configuration for shutdown command. Step 2: Remove command using the no shutdown interface subcommand. |
Five important error fields are provided in the show interfaces serial display output with respect to WAN and serial line troubleshooting:
Figure 7-2 illustrates the position of these information fields in the output of the show interfaces serial command.
Although these fields are described in your Router Products Configuration and Reference publication, the following discussions provide some additional details and outline how you can use them to diagnose connectivity problems for WAN and serial line connections.
When input errors appear in the show interfaces serial output, you must consider several possibilities in order to determine the source of those errors. Most likely problems are summarized in the list of possible causes that follows.
Symptom:
Increasing number of input errors in excess of one percent of total interface traffic.
Possible Causes:
Suggested Actions:
Step 1: Use a serial analyzer to isolate the source of the errors. If errors are detected by the serial analyzer, then some device external to the router is the source (bad hardware or clock mismatch).
Step 2: Use the loopback and ping tests described later in this chapter to the isolate specific problem source.
Table 7-2 details the meaning of CRC errors, framing errors, and aborts. These fields appear in the display shown in Figure 7-2.
Output drops appear in the show interfaces serial output when no buffers are available and the system is attempting to hand off a packet to a transmit buffer. The output drops count is illustrated in Figure 7-2.
Symptom:
Increasing output drops
Possible Cause:
Overused serial interface
Suggested Actions:
Step 1: Turn off fast switching on affected interfaces.
Step 2: Change output hold queue size.
Step 3: Implement priority queuing.
Input drops appear in the show interfaces serial command when too many packets from that interface are still being processed in the system. The input drops count is illustrated in Figure 7-2.
Symptom:
Increasing number of input drops
Possible Cause:
Overutilized interface
Suggested Actions:
Increase input hold queue size.
Interface resets appear in the show interfaces serial command when they result from missed keepalives. The interface resets count is illustrated in Figure 7-2.
Symptom:
Increasing interface resets
Possible Causes:
Suggested Actions:
When looking at interface resets, you must examine other fields as well to determine possible causes. Assuming an increase in interface resets is being recorded, examine the following fields:
Step 1: Check the carrier transitions field in the show interfaces display. If carrier transitions are high while interface resets are being registered, the problem is likely to be a bad link or bad CSU/DSU. Contact your leased line/carrier service and swap faulty equipment as necessary.
Step 2: Examine the input errors field in the show interfaces display. If input errors are high while interface resets are increasing, the problem also is probably a bad link or DSU/CSU. Contact your leased line/carrier service and swap faulty equipment as necessary.
Step 3: Examine the output drops field in the show interfaces display. If your system is running a version of software prior to Software Release 8.3, and a high number of output drops are registered along with interface resets, upgrade to Software Release 8.3 or higher.
Carrier transitions appear in the show interfaces serial command whenever there is an interruption in the carrier signal; for example, when there is an interface reset at the remote end of a link. The carrier transition count is illustrated in Figure 7-2.
Symptom:
Increasing carrier transitions
Possible Causes:
Suggested Actions:
Step 1: Check hardware at both ends of the link (attach breakout box or serial analyzer and test to determine source of problems).
Step 2: If analyzer or breakout box are unable to identify any external problems, check router hardware.
Step 3: Swap faulty equipment as necessary.
In addition to the basic diagnostic fields provided in all show interfaces serial displays, additional information is provided with different WAN encapsulations (such as X.25, SMDS, or Frame Relay). These displays are discussed in detail in your Router Products Configuration and Reference publication.
The discussion that follows is provided to illustrate the kind of supplemental information available through the show interfaces serial command for troubleshooting WAN connections. It focuses on displays provided with the X.25-specific show interfaces serial display.
The general serial diagnostics fields discussed in the preceding section apply to X.25 and other WAN connections as well. In addition, five additional error fields are provided in the show interfaces serial display output for X.25 internetworks. These fields provide the following information:
Figure 7-3 illustrates the position of each of these and other useful information fields in the X.25 version of the display output for the show interfaces serial command.
Symptom:
Recorded REJs, RNRs, FRMRs, RESTARTS, or DISCs in excess of 0.5 percent of IFRAMEs
Possible Causes:
Suggested Actions:
Step 1: Enable debug x25-events diagnostic command; evaluate "Cause and Diagnostic" information provided. Refer to Appendix A, "X.25 Cause and Diagnostics" for details concerning the meaning of the codes generated.
Step 2: Check hardware at both ends of the link using a serial analyzer. Use the analyzer to determine whether the SABMs are being responded to with UAs.
Step 3: If analyzer cannot identify any external problems, check router hardware.
Step 4: Swap faulty equipment as necessary.
Another important diagnostic tool is the show controllers command. For serial interfaces, use the show controllers mci command. If the electrical interfaces is displayed as "UNKNOWN" (instead of V.35, RS-449, or some other electrical interface type), then a likely problem is either a bad applique or a problem with the card's internal wiring. In addition, the corresponding display for the show interfaces serial command will show the interface and line protocol as down in the interface status line. Figure 7-4 illustrates this kind of display output for the show controllers mci command.
The output from debug commands provides diagnostic information concerning a variety of internetworking events relating to protocol status and network activity in general. The following are some debug commands that are useful when troubleshooting serial and WAN problems.
 | Caution Throughout this and other chapters, the use of debug commands is suggested for obtaining information about network traffic and router status. Use these commands with great care. In general, it is recommended that these commands only be used under the direction of your router technical support representative when troubleshooting specific problems. Enabling debugging can disrupt operation of the router when internets are experiencing high load conditions. When you finish using a debug command, remember to disable it with its specific undebug command or with the undebug all command. |
More information is provided in Chapter 10, "Debug Command Reference." In addition, when a specific debug command is recommended for a particular situation, appropriate reminders are included in the various scenarios and symptom modules.
In addition to the basic diagnostic capabilities provided with Cisco internetworking systems, there are a variety of supplemental tools and techniques that can be used to determine the conditions of cables, switching gear, modems, hosts, and remote internetworking hardware. Although complete discussions of these tools are beyond the scope of this publication, some hints about using these alternative tools are provided here. Consult your manufacturer's manuals (CSU/DSUs, serial analyzers, etc.) for more information and specific instructions.
If the output of the show interfaces serial command indicates that the serial line is up but the line protocol is down, use the DSU loopback tests to determine the source of the problem. Perform the local loop test first.
The following discussion outlines a general procedure for performing loopback tests in conjunction with built-in Cisco system diagnostic capabilities.
Step 1: Place the DSU in local loop mode. In local loop mode, the use of the line clock (from the T1 service) is terminated and the DSU is forced to use the local clock.
Step 2: Use the show interface serial EXEC command to determine if the line status changes state (from "line protocol is down" to "line protocol is up (looped)") or if it remains down.
If the show interfaces serial status line changes state when the CSU/DSU is in local loopback mode, it suggests that the problem is external to the router. If the status line does not change state, there is a possible problem in the router, connecting cable, or DSU.
Step 3: Use the debug serial-interface EXEC command to further evaluate the condition of the serial line.
Before enabling the CSU/DSU's local loop test, when the line protocol is down, the debug serial-interface output will indicate that keepalives are not incrementing.
Placing the CSU/DSU into local loop mode should cause the keepalives to begin incrementing. Specifically, the values for mineseen and yourseen keepalives will increment every 10 seconds. This information will appear in the debug serial-interface output. If they do not increment, there may be a timing problem on the interface card or on the network (refer to the subsequent section, "Troubleshooting Clocking Problems," for additional information).
If you are able to determine that the local hardware is functioning properly but cannot establish connections over the serial link, try using the remote loopback test that follows to isolate the problem cause.
Step 1: Put the CSU/DSU into remote loopback.
Step 2: Using the show interfaces serial EXEC command, determine whether the line protocol remains up, with the status line indicating "Serial n is up, line protocol is up (looped)."
If the line protocol remains up in remote loopback mode, the problem is probably at the far end of the serial connection. Perform both tests (local and remote) at the far end to determine the problem source.
If the line status displayed via show interfaces serial toggles to "Line protocol is down" when remote loopback mode is activated, the problem is in the local CSU/DSU or somewhere in the line to the CSU/DSU. Try replacing the local CSU/DSU. If this does not resolve the problem, contact your WAN network manager or the WAN service organization.
One of the most useful tests available on Cisco internetworking systems (as well as on many host systems) is the ping function. In the TCP/IP world, this diagnostic tool also is known as the ICMP Echo Request.
Cisco internetworking systems provide a mechanism to automate the sending of many ping packets in sequence. Figure 7-5 illustrates the menu used to specify extended ping options. This example specifies 20 successive pings; however, when testing the components on your serial line, you should specify a much larger number, such as 1000 pings.
In general, perform serial line ping tests as follows:
Step 1: Put CSU/DSU into local loopback mode.
Step 2: Configure the extended ping to send all zeros or all ones (Figure 7-5 illustrates specification of all ones in the "Data pattern [0xABCD]:" option field).
Step 3: Examine the serial interface statistics and determine whether input errors have increased. If input errors have not increased, the local hardware (DSU, cable, router interface card, and applique) are likely to be good.
Assuming this test sequence was prompted by the appearance of a large number of CRC and framing errors, a likely suspect is a clocking problem. Check the CSU/DSU for a timing problem. Refer to the discussion "Troubleshooting Clocking Problems" that follows immediately after this section.
Step 4: If you determine that the clocking configuration is correct and operating properly, put the CSU/DSU into remote loopback mode.
Step 5: Repeat the ping test and look for changes in the input error statistics.
Step 6: If input errors increase, there is either a problem in the WAN/serial line or on the CSU side of the CSU/DSU. Contact the WAN service provider and swap the CSU/DSU. If problems persist, consult your router technical support representative.
Clocking conflicts in serial connections can lead to either chronic loss of connection service or generally degraded performance. The following discussion addresses four issues regarding clocking problems:
In general, clocking glitches in serial WAN interconnections can be attributed to one of the four basic causes:
To detect clocking conflicts on your serial interface, look for input errors as follows:
Step 1: Use the show interfaces serial command on the routers at both ends of the link.
Step 2: Examine the display output for CRC and framing errors.
Step 3: If either of these steps indicate errors exceeding an approximate range of 0.5 to 1.0 percent of traffic on the interface, clocking problems are likely to exist somewhere in the WAN.
Step 4: Isolate the source of the clocking conflicts as outlined in the next procedure.
Once you have determined that clocking conflicts are the most likely cause of input errors, the following general steps will help you isolate the source of those errors:
Step 1: Perform a series of loopback and ping tests, both local and remote, as described in procedures that precede this section ("CSU/DSU Local and Remote Loopback Tests" and "Extended Ping Tests").
Step 2: Determine which end of the connection is the source of the problem, or if the problem is in the line. In "local" loopback mode, run different patterns and sizes in the ping tests (for instance 1500 bytes). Using a single pattern and packet size may not force errors to materialize, particularly when a serial cable to the router or CSU/DSU is the problem.
Step 3: Perform the show interfaces serial EXEC command and determine whether input errors counts are increasing and where they are accumulating.
If input errors are accumulating on both ends of the connection, clocking of the CSU is the likely problem.
If only one end is experiencing input errors, there is likely to be a DSU clocking or cabling problem.
If you see aborts on one end, it suggests that the other end is sending bad information or that there is a line problem.
Table 7-3 outlines suggested remedies for clocking problems, based on the source of the problem.
| Clocking Problem | Suggested Action |
|---|---|
| Incorrect CSU configuration | Step 1: Determine whether the CSUs at both ends are in agreement regarding the clock source (local or line). |
| Step 2: Configure both to agree if not already correctly configured (usually the line is the source). | |
| Incorrect DSU configuration | Step 1: Determine whether the DSUs at both ends have SCTE (terminal timing) enabled. |
| Step 2: Enable SCTE if not already correctly configured. | |
| (For any interface that is connected to a line of 128 Kbps or faster, SCTE must be enabled). | |
| Cable to router out of specification | Step 1: Use shorter cable if longer than 50 feet.
Step 2: Replace with shielded cable. |
Excessively high bandwidth utilization results in reduced overall performance and possibly intermittent failures. For example, DECnet file transmissions may be failing due to packets being dropped somewhere in the network. If the situation is bad enough, you must add bandwidth. However, adding bandwidth may not be necessary or immediately practical. One way to resolve marginal serial line overutilization problems is to control how the router uses data buffers.
You have three options to control how buffers are used:
The configuration commands associated with these options are fully described in the Router Products Configuration and Reference publication.
The following discussion focuses on identifying situations in which these options are likely to apply and defining how you can use these options to help resolve connectivity and performance problems in serial/WAN interconnections. Commands are discussed as appropriate.
There are two general buffer types on Cisco routers. These are referred to as hardware buffers and system buffers. Only the system buffers are directly configurable by system administrators.
The hardware buffers are specifically used as the receive and transmit buffers associated with each interface and (in the absence of any special configuration) are dynamically managed by the system software itself.
The system buffers are associated with the main system memory and are allocated to different size memory blocks. As of software release 8.3, these buffers can be configured using the buffers global configuration command.
The most useful parameters to set with the buffers command are min-free, initial, and max-free. Refer to your Router Products Configuration and Reference publication for more information concerning this command and its options.
The following procedure outlines a process for implementing system buffer changes:
Step 1: Determine whether adjusting system buffers may be useful. Use the show interfaces command and look for a high number of output drops on serial interfaces.
Step 2: Prepare the router for switching to using system buffers. Do this by disabling fast switching.
Use the write terminal command to determine what protocols are being run on the router and whether there are explicit instances of the various route-cache interface subcommands (such as ip route-cache). Disable these with the no versions of the command.
Since fast switching is enabled by default, you also must explicitly disable fast switching with these commands (such as no ip route-cache) for all protocols on affected serial interfaces.
Step 3: Identify which buffers are likely to be experiencing problems. Use the show buffers command and look for misses in the system buffers listed Table 7-4 lists the relative size of each system buffer type.
| Buffer Size Indicated | Packet Size Range Buffered | Typical Traffic Related to Drops |
|---|---|---|
| Small | 64 to 104 bytes | Virtual Terminal (DEC LAT, Telnet) |
| Middle | 105 to 600 bytes | Novell IPX |
| Big | 601 to 1524 bytes | File Transfer (FTP) |
| Large | 1525 to 5024 bytes | File Transfer (Token Ring) |
| Huge | 5025 to 12024 bytes | Fragmented packets destined for a router (routing updates associated with large internets) |
Step 4: Add a buffers global configuration command as appropriate, tailored to allow enough buffers to prevent misses for protocols where misses are of particular concern.
The following are guidelines for specifying the number of buffers when you see misses (output drops) occurring:
Hold queues are buffers used by the router for each interface to store outgoing or incoming packets. Use the hold-queue interface subcommand to increase the number of data packets queued before the router will drop packets.
Use this command to prevent packets from being dropped and to improve serial link performance under the following conditions:
Priority queuing is a list-based control mechanism that allows network administrators to prioritize traffic transmitted into networks on an interface-by-interface basis. In an manner that is analogous to Cisco's access list traffic control mechanisms, priority queuing involves two steps:
Step 1: Create a priority list, by protocol type and level of priority.
Step 2: Assign the priority list to a specific interface.
Both these steps use versions of the priority-list global configuration command (with the keywords protocol and interface, as appropriate). In addition, further traffic control can be applied by referencing access-list global configuration commands from priority-list specifications. Refer to your Router Products Configuration and Reference publication for examples of defining priority lists and details about command syntax associated with priority queuing.
Use priority queuing to prevent packets from being dropped and to improve serial link performance under the following conditions:
In general, start with the default number of queues (altered with the queue-limit keyword option of the priority-lists global configuration command) when implementing priority queues. After enabling priority queuing, monitor output drops with the diagnostic command show interfaces serial. If you notice that output drops are occurring in the traffic queue you have specified to be high priority, increase the number of packets that can be queued.
The symptom modules that follow pertain to serial and WAN problems. Unless otherwise indicated, each module is presented as a set of general problems applying to all WAN types (such as X.25, point-to-point serial, SMDS, or Frame Relay). Where there are special considerations associated with a specific network type, notes are included.
WAN connectivity symptoms discussed in this section include the following:
Symptom: Intermittent connectivity implies that connections randomly come and go. Connections can be made between some nodes, while others nodes cannot connect--with no apparent reason.
Table 7-5 outlines possible causes of intermittent connectivity in serial and WAN interconnections.
| Possible Cause | Suggested Action |
|---|---|
| Faulty interface card or cable | Step 1: Check status of interface with show interface serial and show controller commands. |
| Step 2: Look for line down condition and version level. | |
| Step 3: Upgrade microcode (firmware) if current code is older than 1.7. | |
| Step 4: Swap card or cable if nonoperational. | |
| Bad CSU/DSU | Step 1: Check for input errors in show interface serial output. |
| Step 2: Replace modem. | |
| Step 3: Observe behavior after modem change. | |
| Timing problem | Step 1: Check CSU/DSU configuration for SCTE/Terminal timing enabled; enable if not already enabled. |
| Step 2: If the CSU/DSU is properly configured, or if intermittent connectivity persists after enabling SCTE/Terminal timing on the CSU/DSU, verify that the correct network entity is generating the system clock; reconfigure nodes and modems if clocking is not properly configured. | |
| Step 3: If intermittent problems still persist, check cable length; if it is longer than 25 feet, you may need to invert clock on the MCI/SCI. | |
| Network generating invalid PRs (X.25) | Step 1: Run diagnostics at the switch.
Step 2: Swap switch hardware if necessary. |
| Router generating invalid PRs (X.25) | Step 1: Enable debug x25-events; examine cause and diagnostics output; consult Chapter 10 for more details. |
| Step 2: Upgrade router software to 8.3(5) or higher revision. Software release 9.0(3) is recommended. | |
| Serial line congestion | Step 1: Turn off fast switching. |
| Step 2: Manage buffers (must have Software 8.3 or greater). | |
| Step 3: Apply priority list. |
Symptom: The typical symptom is that users complain incessantly about lost connections at peak periods. One example of this problem is in an environment featuring bridged DEC Local Area Transport (LAT) traffic and multiple routed protocols. Data entry input from users (or other application requests) may be getting buffered at the end of an already long input queue; eventually one end of the connection will time out.
Table 7-6 outlines possible causes of load-related failures in serial and WAN interconnections.
| Possible Cause | Suggested Actions |
|---|---|
| Dirty serial line | Step 1: Determine if input errors are increasing. |
| Step 2: If input errors appear, diagnose serial line per discussion earlier in this chapter. | |
| Overutilized serial line | Step 1: If input errors do not appear, the problem is related to congestion. |
| Step 2: Turn off fast switching. | |
| Step 3: Include an appropriate priority queuing configuration statement. | |
| Step 4: Adjust buffer size (8.3 or more recent software needed). |
Symptom: This symptom is generally an example of connections dying under load. In this case, traffic on a serial link approaches saturation at specific times during the day, for instance, around 8:30 a.m., noon, and 5:30 p.m. The result is a loss of connections or the ability to make connections.
Table 7-7 outlines possible causes of load-related failures associated with time-of-day problems in serial and WAN interconnections.
| Possible Cause | Suggested Action |
|---|---|
| Overutilized bandwidth | Step 1: Check applications being run, especially for very large file transfers scheduled at particular times of day. |
| Step 2: If this is the case, set up a priority queue based on packet size to allow higher small-packet traffic (requires that the protocol allows flow control). | |
| Step 3: Rearrange file transfer timing by applications so that links are not overused during normal business hours. | |
| Step 4: Add bandwidth and consider dial backup over the new link for applications that are taking excessive bandwidth on existing links. | |
| Unshielded cable runs are too close to EMI sources | Step 1: Check show interfaces serial display for input errors.
Step 2: If loading is not the problem, and input errors are being registered, inspect cable runs for proximity to interference sources. |
| Step 3: Relocate or shield cables if found to be near E-M source. |
Symptom: When connections suddenly die after a period of relatively normal, error-free operation (and cannot be brought back), a hardware-related problem somewhere along the serial line is the likely culprit. However, there are other possible causes (discussed in the list that follows).
Table 7-8 outlines possible causes of sudden failures that occur following essentially error-free operations over serial and WAN interconnections.
| Possible Cause | Suggested Action |
|---|---|
| Hardware in the serial link died | Step 1: Use show interfaces serial to determine whether link is down.
Step 2: If link is down, troubleshoot serial line per CSU/DSU loopback tests and ping tests described earlier or with serial analyzer. |
| Routing tables are incorrect | Step 1: If the link is up, use the appropriate show protocol route command (example: show ip route or show apple route). |
| Step 2: Determine whether routes are correct; if not, look for source of bad routes (flapping link, backdoor bridge between the routed segments, or incorrect configuration of route redistribution between routing protocols). | |
| Buffer misses or other software problem | Step 1: If the routing table is correct and the link is up, use the show buffers command to evaluate buffer status. |
| Step 2: Refer to the discussion concerning buffer management earlier in this chapter for more information. | |
| Step 3: Modify buffers as necessary to prevent dropped connections. | |
| Step 4: Contact your technical support representative if all actions fail to resolve problem. |
Symptom: When no traffic of any kind is passing through a newly installed router interconnecting broadcast networks via an HDLC point-to-point link, look for problems associated with the new installation.
Table 7-9 outlines possible causes of connectivity problems that occur following new HDLC serial router installations.
| Possible Cause | Suggested Action |
|---|---|
| Link is dead | Step 1: Use show interfaces serial to determine whether link is down. |
| Step 2: If link is down, troubleshoot serial line per CSU/DSU loopback tests and ping tests described earlier or with serial analyzer. | |
| Keepalives not being received | Step 1: Use debug serial-inteface to determine status of keepalives. |
| Step 2: If keepalives are not incrementing (probably will not be, given this line status), troubleshoot serial line per the loopback and ping tests discussed earlier in this chapter. |
Symptom: When no traffic of any kind is passing through a newly installed router interconnecting broadcast networks via an X.25 WAN, look for problems associated with the new installation.This is especially true when local area networks (LANs) previously interconnected via the WAN continue to communicate with no disruption of service.
Table 7-10 outlines possible causes of connectivity problems that occur following new X.25 router installations.
| Possible Causes | Suggested Actions | ||||||
|---|---|---|---|---|---|---|---|
| Link is dead | Step 1: Use show interfaces serial to determine whether link is down. | ||||||
| Step 2: If link is down, troubleshoot serial line per CSU/DSU loopback tests and ping tests described earlier or with serial analyzer. | |||||||
| Switch is misconfigured | Step 1: Check configuration of switch; look for bad address specifications, incorrect VC parameter settings, or other configuration errors. | ||||||
| Step 2: If errors are found, modify configuration and check status of the line via show interface serial output. | |||||||
One of the following:
| Step 1: If the status line displays "Serial (n) is up, Line protocol is down," check the status of LAPB (state will probably be in CONNECT or SABMSENT).
Step 2: If the LAPB status is not CONNECT, attach a serial analyzer (probably will show SABMSENT). Step 3: Using the serial analyzer, look for unnumbered acknowledge (UA) packets sent in reply to SABMs. | ||||||
| Step 4: If UAs are not being sent, one of these problems is the likely cause. | |||||||
| Step 5: Reconfigure equipment or replace as required. | |||||||
| Step 6: If the status line displays "Serial (n) is up, Line protocol is up" and no connections can be made, check the router configuration with the write terminal EXEC command. | |||||||
| Step 7: Check the x25 map commands and ensure that the correct addresses are specified. | |||||||
| Step 8: Verify that the broadcast keyword is included in the x25 map command (if dynamic routing is being used in the network). | |||||||
| Step 9: Ensure that all router configuration options match switch settings. | |||||||
| Step 10: Modify configuration on router as needed to resume operation. |
Symptom: When no traffic of any kind is passing through a newly installed router interconnecting broadcast networks via a Frame Relay WAN, look for problems associated with the new installation.This is especially true when local area networks (LANs) previously interconnected via the WAN continue to communicate with no disruption of service.
Table 7-11 outlines possible causes of connectivity problems that occur following new Frame Relay router installations.
| Possible Causes | Suggested Actions |
|---|---|
| Frame Relay switch is misconfigured (dynamic DLCI and protocol address mapping) | Step 1: Check output of show interfaces serial to determine line status and whether LMI updates have been received.
Step 2: If LMIs have not been received, enable debug frame-relay-lmi; look for LMI information to determine whether switch and router are sending and receiving LMI packets. |
| Step 3: Check configuration of Frame Relay switch; make sure LMI is in use (dynamic mode). This is not a problem if using static mode. | |
| Router is misconfigured (wrong keepalive setting) | Step 1: Evaluate router configuration--use write terminal EXEC command; check for LMI keepalive setting (dynamic mode). |
| Step 2: Compare LMI keepalive setting with switch setting. | |
| Step 3: Make sure router is at least two seconds faster than switch. | |
| Router is misconfigured (dynamic DLCI and protocol address mapping) | Step 1: Check output of show interfaces serial to determine line status.
Step 2: Determine whether DLCI-to-protocol mapping is dynamic or static; set to correct mode if not correct. |
| Step 3: If dynamic mapping is implemented and interface status is line up/protocol up, but no connections can be made, get output of show frame-relay map. | |
| Step 4: Determine whether any of the far end networks were learned by the local router. | |
| Step 5: If far end networks were learned, ping nearest interface of remote router (if protocol supports ping). Verify that you can reach that point. | |
| Step 6: If you cannot successfully ping (with line up/protocol up status), the Frame Relay network is probably misconfigured. | |
| Step 7: If ping works, ping through to other side of router, working out to end stations. | |
| Step 8: Reconfigure equipment as necessary. (At the router, be sure that the remote DLCI number is mapped to the protocol address at the far end.) | |
| Step 9: If no far end networks are learned with show frame-relay map, enable debug frame-relay-events and execute the appropriate show route command (protocol dependent). | |
| Step 10: Determine what exchanges are occurring between router and switch and whether any routing protocol information is being picked up. | |
| Step 11: Make any necessary router configuration changes to address specifications or other Frame Relay configuration entries. | |
| Router misconfigured (static Frame Relay address mapping) | Step 1: Check output of show interfaces serial to determine line status.
Step 2: Determine whether DLCI-to-protocol mapping is dynamic or static; set to correct mode if not correct. |
| Step 3: If static mapping is implemented with line up/protocol up interface status, but no connections can be made, get output of show frame-relay map. | |
| Step 4: Status should be defined as "active"; if not, check the configurations on the switch and router, compare, and make sure they match. | |
| Step 5: If show frame-relay map indicates active status, get output of appropriate show route for protocol (determine whether routing information is accumulating); proceed to access list "Possible Causes" section. | |
| Router misconfigured (bad access list implementation) | Step 1: Evaluate router access lists at both ends of connection.
Step 2: Make sure there are no inadvertent access denials. |
| Step 3: Modify as needed or remove to test; rework to allow appropriate access. | |
| Cabling problem | Step 1: Inspect line/protocol status and check cabling per serial diagnostics discussions earlier in this chapter. |
| Step 2: Replace any incorrectly configured or failed cables. | |
| Dead hardware | Step 1: Inspect line/protocol status and perform loopback and ping tests as described earlier in this chapter to isolate specific problem equipment. |
| Step 2: Replace hardware as necessary. |
Symptom: When no traffic of any kind is passing through a newly installed router interconnecting broadcast networks via an SMDS WAN link, look for problems associated with the new installation.This is especially true when local area networks (LANs) previously interconnected via the WAN continue to communicate with no disruption of service.
Table 7-12 outlines possible causes of connectivity problems that occur following new SMDS router installations.
If you are having difficulty establishing connections over an SMDS cloud, obtain the following information as a preliminary step before beginning the problem isolation process:
| Possible Causes | Suggested Actions |
|---|---|
| SMDS switch is misconfigured | Step 1: Check router and switch configurations for address mismatch.
Step 2: Make sure SMDS switch is configured for multicast or static mapping (depending on intended network setup). |
| Router misconfigured (general SMDS) | Step 1: Evaluate router configuration--using write terminal EXEC command. |
| Step 2: Compare router configuration with switch requirements. Examples: bad address specification, wrong mode (multicast vs. static), smds encapsulation not included in configuration. | |
| Step 3: Modify configuration if necessary to make router match SMDS network requirement. | |
| Misconfigured SMDS interface or multicast addresses | Step 1: Use the show arp command to determine what other devices, if any, have been detected on the switch. |
| Step 2: If none are being detected, check to make sure that the interface SMDS address specified in the smds address configuration command matches the address of the attached switch. | |
| Step 3: Check to make sure that the SMDS multicast addresses specified in the smds multicast configuration command match the addresses configured on the switch. | |
| Step 4: Make sure that ARP is enabled with the smds enable-arp command so that higher layers learn about the router. | |
| Step 5: Check the router's configuration of static maps. Make sure that static maps are configured for all nonlearning protocols to allow the SMDS software to translate a destination address into a proper SMDS address for outgoing packets. | |
| Step 6: If SMDS data is still not being received, even if packets are being sent, check the cable/SDSU/AU/Switch connections for proper physical connectivity. | |
| Step 7: If the physical connections are operational, and packets still are not being received, check the SDSU configuration. | |
| Router misconfigured (static SMDS address mapping) | Step 1: Check output of show interfaces serial to determine line status. |
| Step 2: Using the show smds map command, determine whether applicable mode is multicast or static; configure correct mode if not correct. | |
| All network protocols, with the exception of IP and ISO CLNS, require static mapping from the protocol address to SMDS addresses. | |
| Step 3: If IP or ISO CLNS are being routed, check the multicast group specification. Make any necessary address changes. | |
| Step 4: If static mapping is implemented with line up/protocol up interface status, but no connections can be made, enable debug serial-interface. | |
| Step 5: Based on the debug output, determine whether the correct destination address is being used. | |
| Step 6: Make configuration changes as necessary to mapping, mode, or encapsulation specification. | |
| Router misconfigured (bad access list implementation) | Step 1: Evaluate router access lists at both ends of connection.
Step 2: Make sure there are no inadvertent access denials. |
| Step 3: Modify as needed or remove to test; rework to allow appropriate access. | |
| Cabling problem | Step 1: Inspect line/protocol status and check cabling per serial diagnostics discussions earlier in this chapter. |
| Step 2: Replace any incorrectly configured or failed cables. | |
| Dead hardware | Step 1: Inspect line/protocol status and perform loopback and ping tests as described earlier in this chapter to isolate specific problem equipment. |
| Step 2: Replace hardware as necessary |
Symptom: When some users or applications are able reach resources over a serial/WAN link through a router, while others cannot, the problem usually can be attributed to a configuration error.
Table 7-13 outlines possible causes of selective connectivity problems in serial and WAN interconnections.
| Possible Causes | Suggested Actions |
|---|---|
| Bad access list specification | Step 1: Check configuration using write terminal. Look for any access list specifications. |
| Step 2: If found, carefully compare access list with requirements. Be sure that no implicit access deny (or explicit access deny) is blocking required access. | |
| Step 3: Modify access lists as needed. | |
| Host configuration not set up to send ARPs | Step 1: Check host configuration.
Step 2: Modify host configuration to send ARPs. |
| Host configuration points at wrong router | Step 1: Check host configuration for gateway of last resort specification.
Step 2: Modify host configuration for gateway of last resort. |
| Discontinuous subnet addressing (IP) | Step 1: Check network configuration for discontinuous network address space assignment. |
| Step 2: If found, use secondary IP address to accommodate physical discontinuity. |
|
|