Internetworks come in a variety of topologies and levels of complexity--from single-protocol, point-to-point links connecting cross-town campuses to highly meshed, large-scale WANs traversing multiple time zones and international boundaries. The overall trend is toward increasingly complex environments, involving multiple media, multiple protocols, and sometimes interconnection to "unknown" networks. As a result, the potential for connectivity and performance problems in internets is often high, even when all elements of an environment appear to be fully operational. The objective of this publication is to help you identify potential problem sources in your internet and then to systematically resolve problems that arise.
Note In this release of
Troubleshooting Internetworking Systems, coverage focuses on AppleTalk, IBM SNA, Novell IPX, TCP/IP, and WAN/serial internets. Subsequent releases will cover additional protocols and technologies.
Failures in internets are characterized by certain symptoms (such as clients being unable to access specific servers). Each symptom can be diagnosed based on problems or causes using specific troubleshooting tools. Once identified, each cause can be remedied by implementing a series of actions.
Use this manual as a starting point to develop a problem-solving process for your internetwork. This publication aims to integrate the process of symptom definition, problem identification, and action implementation into an overall troubleshooting model. It illustrates how problems can be detected and diagnosed within the context of case environments.
With these broad objectives stated, it is equally important to outline topics that are beyond this publication's scope.
- This publication is not to intended to be the last word in troubleshooting. You will not find every "corner case" (or obscure anomaly) and subtle protocol problem. Instead, Troubleshooting Internetworking Systems is a roadmap that illustrates the common pitfalls and problems most frequently encountered by internetwork administrators.
- Troubleshooting Internetworking Systems is not a maintenance and repair guide; nor is it a reference guide. Refer to your hardware installation and maintenance publication for additional details regarding maintenance of Cisco hardware. Refer to the Router Products Configuration and Reference publication for configuration command details.
- This publication recommends actions for resolving a spectrum of common internetworking problems. In general, it assumes that routers are operational. However, several brief tables provided later in this chapter summarize typical router hardware problems.
- Finally, Troubleshooting Internetworking Systems is not a network troubleshooting publication. Although suggestions are provided in this chapter about troubleshooting certain media (including Ethernet, FDDI, serial, and Token Ring), the focus of the publication is not on troubleshooting media, per se. Several commercially available publications provide this information. One is Mark Miller's LAN Troubleshooting Handbook. Appendix E, "References and Recommended Readings," suggests some others.
What, then, does that leave? The discussions that follow outline how you can use this publication to resolve common internetworking problems.
The remainder of this overview addresses the following topics:
- Using this publication
- Using Cisco diagnostic tools
- Diagnosing Cisco hardware
- Using CiscoWorks to troubleshoot your internet
- Using third-party troubleshooting tools
- Troubleshooting media problems
Troubleshooting Internetworking Systems focuses on identifying failure symptoms and their associated causes, detecting and isolating those causes, and then resolving problems through specific actions. The symptom discussions and scenarios provided concentrate on issues pertaining to router configuration and the interoperation of nodes within a multivendor internetwork.
Within this context, use Troubleshooting Internetworking Systems as a guide to:
- Identify possible problem causes when your internet is down or slow
- Get direction about resolving problems
- See what kinds of problems have been encountered and resolved in the past
- Avoid falling into the same traps
- Develop your own processes for troubleshooting
To support these activities, this guide uses three key organizational elements (defined in the discussions that follow):
- General problem-solving model
- Symptom tables
- Troubleshooting scenarios
In addition, this overview provides guidelines for the following tasks:
- Using this guide to troubleshoot problems
n Using this guide as a tutorial
Before embarking on your troubleshooting effort, be sure to have a plan in place to identify prospective problems, isolate the likely causes of those problems, and then systematically eliminate each potential cause.
The problem-solving model that follows is not a rigid "cookbook" for solving internetworking problems. It is a foundation from which you can build problem-solving plans to suit your particular environment.
Figure 1-1 illustrates process flow for the general problem-solving model described in the steps that follow.
Figure 1-1: General Problem-Solving Flow Diagram
The following steps detail the problem-solving process outlined in Figure 1-1:
Step 1: Define problems in terms of a set of symptoms and associated causes.
Make a clear problem statement. You must recognize and define the problem/failure mode by identifying any associated general symptoms and then identifying the possible kinds of problems that result in the listed symptoms.
For example, certain hosts might not be responding to service requests from certain clients (a symptom). Possible causes include a misconfigured host, bad interface cards, or missing router commands.
Step 2: Gather facts.
Once your symptoms are listed and possible causes identified, collect facts. Fact gathering might involve obtaining network analyzer traces, serial line traces, stack dumps, core dumps, and output from a variety of show and debug commands. The definition of the problem will point to a more specific set of data to gather.
Step 3: Consider possibilities based on facts.
Armed with a working knowledge of the product, you should be able to eliminate entire classes of problems associated with system software and hardware. This way, you can narrow the scope of interest to only those portions of the product, media, or host problems that are relevant to the specific problem or failure mode.
Step 4: Create an action plan.
The action plan should be based on the set of possibilities you just derived. Your action plan must limit manipulation to one variable at a time. This approach allows you to reproduce a given solution to a specific problem. If you alter more than one variable simultaneously, you might solve the problem, but identifying the specific change that eliminated the symptom becomes more difficult.
Step 5: Implement action plan.
This phase consists of executing the action plan you just created. It is important to be very specific in creating the action plan (that is, identify a specific set of steps and then carefully implement each step).
Step 6: Observe results of each action.
After having manipulated a variable in an attempt to find a solution to a problem, be sure to gather results based on this action plan (obtain relevant traces, capture debug command data, examine output of show commands, etc.). This data can be used to fine-tune the action plan until the proper solution is achieved. It is during this phase that you must determine whether the problem has been resolved. This is the exit point of the iterative loop shown in Figure 1-1.
Step 7: Narrow possibilities based on results.
In order to reach a point where you can exit this problem/solution loop, you must strive to make continuous progress toward a smaller set of possibilities, until you are left with one.
Step 8: Iteratively apply problem-solving process.
After narrowing your possibility list, repeat the process, starting with a new action plan based on a new (possibly shorter or longer) list of possibilities. Continue the process until a solution is found. Problem resolution can consist of several modifications to hosts, routers, or media.
Note If you exhaust all the common causes and actions (either those suggested here or ones that you have identified for your environment), your last recourse is to contact your router technical support representative. Appendix B, "Technical Support Information List," outlines information needed by technical support representatives to troubleshoot internetworking problems. One objective of this publication is to help you develop your own processes for gathering data, resolving problems, and preventing problems from recurring (with a minimum of downtime and external intervention).
The symptom modules in this publication are not comprehensive case studies, but instead are brief snapshots of likely problems associated with a specific symptom. Use them as tools for compiling lists of candidate problems (by symptom). The connectivity and performance chapters (such as Chapter 3, "Troubleshooting Apple Connectivity") are organized around the symptom modules. These chapters are not meant to be read from beginning to end; rather, specific information in these symptom-oriented chapters is intended to be found as needed.
Each symptom module includes a brief summary statement and a table listing possible causes.A series of suggested actions is provided for each listed cause to help you determine whether the specific cause is actually the source of the symptom and then to resolve the problem.
The troubleshooting scenarios combine the problems and actions presented in symptom modules with the methods outlined in the "General Problem-Solving Model" within a context of integrated case studies.
Each scenario outlines a set of "observed" symptoms, an internetworking environment, and a list of likely problems for each symptom. Scenarios focus on the process of problem diagnosis (discovery), isolation, and resolution. Not all symptoms discussed in this publication are explored in the scenarios. Instead, selected multiple symptoms are addressed per scenario. An effort has been made to pick common, realistic problems.
When using this publication to troubleshoot your internet, follow these general steps:
Step 1: Identify symptoms being experienced on your internetwork.
Step 2: Eliminate Cisco hardware as a possible problem (refer to "Diagnosing Cisco Hardware") by either fixing any hardware problems or ruling out Cisco hardware as a possible cause.
Step 3: Scan the symptom modules provided in the various chapters addressing the technologies or protocols used in your internet to identify similar symptoms.
Step 4: Within identified symptom modules, evaluate problems listed in the associated "Possible Causes and Suggested Actions" section.
Step 5: Systematically and iteratively apply actions for each suspect problem until all symptoms are eliminated or the possible cause list is exhausted.
When using this guide as a tutorial, associated activities are a little less structured than when using it to troubleshoot a specific problem. Nonetheless, you can think of the learning process as a series of steps, as follows:
Step 1: Review "General Problem-Solving Model" earlier in this chapter to see how Cisco recommends approaching the troubleshooting process.
Step 2: Read through the scenarios presented in Chapter 2, "Connectivity Problem Scenarios," and Chapter 8, "Performance. Problem Scenarios."
Step 3: Characterize similarities or differences between these scenarios and your own internetworking environment.
Step 4: Review the symptom modules associated with protocols or technologies implemented in your internet.
Step 5: Develop a list of possible symptoms and problems that you encounter in your internet. Be as specific as possible. Keep this list on hand in a troubleshooting binder.
Step 6: When similar symptoms occur, use this list to start the troubleshooting process. Remember to modify your problem-solving procedures as you find subtleties associated with your implementation. The key to developing an effective response to problems in your environment is being able to identify the causes of those problems and then implement an action plan. Whatever you can do to preempt time spent in diagnosis will pay off in terms of reducing downtime.
Step 7: Periodically revisit this process to accommodate changes and additions to your internet.
Troubleshooting Internetworking Systems is one of several Cisco information resources that are essential for building Cisco-based internetworking environments. These resources include the following:
- Getting Started Guides--The getting started guides provide crucial information for first-time startup. Information includes a step-by-step initialization process. Separate manuals are provided for routers, communication servers, and protocol translators.
- Hardware Installation and Maintenance--The hardware installation and maintenance manuals are essential when bringing a new router on line. Information includes product overview, preinstallation information, installing the hardware (cabling, rack-mounting), troubleshooting for initial hardware configuration, user-configurable jumpers and switch settings, cabling instructions, and LED functional definitions.
- Router Products Configuration and Reference--The configuration and reference guide is the key system software reference publication for all information about configuring and monitoring your router. Information includes step-oriented task lists and complete reference material for configuration and system monitoring commands.
- System Error Messages--The error messages handbook lists and describes system error messages for routers, communication servers, and protocol translators. Not all messages indicate problems with a system. Some are informational, while others can help diagnose media, hardware, and software problems.
- Release Notes--Generally, a release note is the first place to look when configuring a new system. Information includes a summary of new software features, a listing and description of known bugs, descriptions of microcode revisions, a hardware/software compatibility matrix, hardware caveats, and a summary of new hardware features.
- Configuration Notes--Configuration notes are required reading for performing any upgrade or other change. Information includes installation and configuration instructions for spare or replacement and upgrade parts (such as cards, appliques, system software, and microcode).
- CiscoWorks User Guide--This guide provides detailed information about remote management of Cisco internetworking systems using the CiscoWorks network management application package.
- Customer Information Online (CIO)--Cisco's online information resource provides application information and current bug descriptions. This service is provided by Cisco's Customer Engineering (CE) organization. Contact your technical support representative for more information about accessing this database.
The following tools are universally applicable when gathering information to troubleshoot problems in Cisco internetworks:
- show EXEC commands
- debug diagnostic EXEC commands.
- ping (Echo Request/Echo Reply) and trace diagnostic tests
- exception dump and write core configuration commands
The discussions that follow summarize using these tools. Chapter 10, "Debug Command Reference," defines the debug commands for protocols and technologies discussed in this publication, and Appendix D, "Creating Core Dumps," describes the exception dump and write core commands.
The Router Products Configuration and Reference publication details the show, ping, and trace commands.
The router's show commands are among your most important tools for understanding the status of a router, detecting neighboring routers, monitoring the network in general, and isolating problems in the internet.
These commands are essential in almost any troubleshooting and monitoring situation. Use show commands for the following activities:
- Monitoring router behavior during initial installation
- Monitoring normal network operation
- Isolating problem interfaces, nodes, media, or applications
- Determining when a network is congested
- Determining the status of servers, clients or other neighbors
For some protocols, such as Novell IPX and AppleTalk, the methodical use of show commands is one the most reliable ways to create a topology map of your internetwork. To create a topology map, use the show commands as follows:
Step 1: Use the appropriate show protocol route command (such as show novell route) to determine which neighbors are directly connected.
Step 2: Record the names and network addresses of all directly connected neighbors.
Step 3: Open a connection to each of these directly connected neighbors and obtain the output of the show protocol route command at those neighbors.
Step 4: Continue this process for all routers in your internet.
The resulting map reflects all paths to the routers in your internet.
The debug EXEC commands can provide a wealth of information about the traffic being seen (or not seen) on an interface, error messages generated by nodes on the network, protocol-specific diagnostic packets, and other useful troubleshooting data. But beware! These commands often generate data that is of little use for a specific problem.
Use debug commands to isolate problems, not to monitor normal network operation. Do not use debug commands unless you are looking for specific types of traffic or problems and have narrowed your problems to a likely subset of causes.
This publication does not document every debug command that exists in the router code, but only those identified as especially useful for troubleshooting specific media and protocols.
 | Caution Throughout this publication, the use of debug commands is suggested for obtaining information about network traffic and router status. Use these commands with great care. In general, it is recommended that these commands only be used under the direction of your router technical support representative when troubleshooting specific problems. Enabling debugging can disrupt operation of the router when internets are experiencing high load conditions. When you finish using a debug command, remember to disable it with its specific undebug command or with the undebug all command. |
Two of the most useful internetworking diagnostic tools are the ping and trace features. The ping capability provides a simple mechanism to determine whether packets are reaching a particular destination. The trace capability allows you to determine the specific path taken to a destination and where packets are stopping. Together, these functions may be two of the most important troubleshooting tools available. Trace is supported with TCP/IP, ISO CLNS, and Banyan VINES on the router. Ping is supported with AppleTalk, TCP/IP, ISO CLNS, Novell IPX, and Banyan VINES.
The exception dump global configuration command and write core command are among the more obscure (although useful) diagnostic commands available in your router toolkit. When a router's system software fails, using the exception dump command to obtain a core dump is sometimes the only way to determine what happened. The write core command is useful if the router is malfunctioning but has not crashed.
| Caution Use these commands only in coordination with a qualified technical support representative. The resulting binary file must be directed to a specific syslog server and subsequently interpreted by qualified technical personnel. Appendix D, "Creating Core Dumps," briefly describes the process. |
Although this publication focuses on troubleshooting overall internetworking problems, the following tables provide some suggestions for diagnosing router hardware problems. Your installation and maintenance publication provides specific LED indicator information for system appliques and front panels.
This overview is not a step-by-step procedure. It is included as a "mental checklist" and should be used as a starting point for troubleshooting. The following discussion suggests a three-stage process:
- Physically inspecting your system
- Applying power and evaluating the system
- Testing and verifying operation
Each of these stages is discussed separately.
When initially evaluating a suspect system, keep the following three rules in mind:
- Contrast what should be happening with what is happening.
- Do not overlook the obvious.
- Do not alter anything before power-up; do not mask a possible failure.
At this stage, concentrate on problems that are obvious. Follow these inspection steps:
Step 1: Look for loose cards, cables, and appliques. Be sure to reseat any that are loose. When cards are new, sometimes a thin film of carbon or oxidation buildup prevents good contact. After reseating each board once or twice, you should achieve good contact.
Step 2: Remove the top of the chassis and inspect the interior. Are the wires to the power supply connected correctly? Are they burned?
Step 3: Look for burned cards, backplanes, and ribbon cables. Are there any visibly crimped or shorted wires or cables?
Step 4: Check for missing or loose parts, incorrectly connected cables, and anything that appears out of place. Does the unit need to be cleaned? Is there damage to the interior or exterior?
Note Do not change anything before powering up the system for evaluation. Making changes can mask other problems. Do not alter anything, even if it appears to be out of place, so that you can determine the source of suspected hardware problems during subsequent evaluation.
Once you have inspected the system, apply power to the unit and observe its behavior. When applying power to a unit, remember the following rules:
- Do not overlook the obvious (does this seem familiar?).
- Do not jump to conclusions or make unnecessary assumptions.
- Make the symptoms explain the problem.
If you suspect a hardware problem, follow these steps to evaluate operational conditions upon power-up:
Step 1: Power up the system (when system is off line).
Step 2: Use a voltmeter to ensure that all the power supply voltages are within specifications. Refer to the configuration note for your power supply model.
Note Configuration notes are only shipped with spares and replacement parts.
Step 3: Compare system behavior against symptoms outlined in Table 1-1.
Step 4: If a failure does not fit the examples in Table 1-1, verify that the software in the processor and the microcode in the various cards are labeled correctly, are in the appropriate order, and are compatible with the individual card revisions within the chassis. Refer to the release document provided with your system.
Step 5: If the system boots, use show controllers {token|mci|fddi|cbus} to ensure that the interface hardware addresses are nonzero. Hardware addresses of all zeros will cause problems in a network.
Note If the system boot-up sequence requires a password, the memory card and circuitry are working correctly. If the configuration in memory does not match the hardware configuration, problems can occur. Possible problems include hung ports, uninitialized ports, ping failures, local and multibus timeout errors, and reboots.
Power-up Problem Symptoms and Possible Causes
| Symptoms at Power-Up
| Possible Causes
|
|---|
| No response from chassis
| Fuse blown (3000, 4000, and I-, M-, and C-chassis)
Bad power supply
Bad switch
Bad backplane
Bad breaker (AGS/AGS+)
|
| No fan (MGS/CGS)
| Bad fan
Bad 12 V power supply
Shorted or broken wires
|
| No blower (A-type/AGS+)
| Bad blower
Bad breaker
Tripped breaker
Shorted or broken wires
Bad 110/220 capacitor
|
| No LEDs on at boot
| Bad 5 V power supply (no LEDs on card); box may boot
Shorted or broken wires
|
| System will not boot
| Bad power supply
Miswired power supply
Bad/disconnected console cable (system still boots; no monitor output)
Bad processor card or card is poorly seated
Bad software
Bad memory board
Shorted wires
|
| No cards show up in power-up message display
| Bad backplane
Bad processor/controller/interface card
Cards not seated in backplane
Bad power supply
|
| Breaker trips or fuse blows
| Bad power supply
Bad backplane
Shorted wires
Load too large on power supply
No load on power supply
Bad breaker
Bad blower
Bad card
|
| Constant or partial reboot
| Bad processor/controller/interface card
Bad backplane
Bad power supply
Bad software
Bad microcode
|
If replacing a part or card to remedy a suspected problem, remember the following rules:
- Make only one change at a time.
- Eliminate suspected problems one at a time.
- Think in terms of card replacement only.
- Keep track of any unrecorded failure symptoms or unexpected behaviors for future revisions of this guide.
- To test a system, start with a simple hardware configuration and add one card at a time until a failed interface appears or is isolated. Use a simple software configuration and test connectivty using a ping test.
Use Table 1-2 as the next step in evaluating hardware. The problems listed are not all of the possible failures for each product, but do represent commonly encountered symptoms. Where applicable, possible error messages associated with failure symptoms are also listed.
If you determine that a part or card replacement is required, contact your sales or technical support representative. Specific instructions concerning part or card installation are included with the configuration note provided with the replacement.
If a part replacement appears to solve a problem, make certain to reinstall the suspect part to verify the failure. Remember, if something seems too good to be true, it probably is; always double-check a repair.
Failure Symptoms by Card or Product Type
| Card Type or Part
| Failure Symptoms This Card May Cause
|
|---|
| CSC-ENVM
| System is down after running a short time; DC voltages off; blower on.
System will not power up; DC voltages off; blower on.
Configuration cannot be written to memory; loses memory over time
ENVM fails to shut system down even with excessive heat or DC voltage.
Error Messages--Bad checksum for configuration memory, configuration memory not set up, nonvolatile memory not present.
|
| CSC/4, CSC/3, and CSC/2
| System will not boot (any combination of LEDs lighted other than green LED lighted only).
Multibus cards are not seen.
The ciscoBus controller is not seen (CSC/4 and CSC/3).
Partial boot only.
Random reboot occurs after initial boot.
System will autoboot, but cannot boot manually.
System will reboot when configuration memory is written.
No response from keyboard.
Error Messages--Parity error, software versus hardware error, local timeout, bus error, wrong interface, emulation line error, software-forced crashes, checksum mismatch error.
|
| CSC-CCTL and
CSC-CCTL 2
| Some or all ciscoBus cards are not seen.
No LEDs light.
All LEDs light.
Wrong number of LEDs light--too many or too few.
Some or all Multibus cards are not seen.
Error Messages--MEMD failure, MEMA failure, ciscoBus daughter
controller failure.
|
| CSC-FCI, CSC-C2FCI,
CSC-C2FCI and
CSC-C2FCIT
| Not recognized by ciscoBus controller.
FDDI ring will not come up.
FDDI ring up, but no ping on FDDI ring; intermittent ping; only certain packet sizes will ping.
No keyboard response after FDDI ring comes up; lock-up.
Error Messages--Unknown data error, MEMD/MEMA failure, ciscoBus daughter controller failure.
|
| FDDI Applique (APP-LMM, APP-LMS, APP-LSM, and APP-LSS)
| FDDI ring will not come up.
LEDs are in wrong sequence.
FDDI ring will come up in "wrap-mode" only--wrap A or wrap B.
No ping through FDDI ring or to address of Unit Under Test (UUT); intermittent ping.
FDDI ring will intermittently or constantly transition.
|
| CSC-MEC
| Card is not seen by ciscoBus controller.
Unable to ping on any or some ports; intermittent ping; only certain packet sizes will ping.
All LEDs light.
No LEDs light.
Wrong number of LEDs light.
Error Messages--Multibus timeout, ciscoBus daughter controller failed, output hung.
|
| MCI and SCI
| Card is not seen by processor card.
No LEDs light.
All LEDs light.
No ping on any or some ports; DTE will ping and DCE will not ping (or vice versa); intermittent ping; only certain packet sizes will ping.
Ports will not initialize--some or all.
Will not netboot or ping to network; no ping to address of UUT.
MCI-3 cannot see nonvolatile memory (NVRAM) port .
Error Messages--Local timeout, MEMD failure, MEMA failure, output hang error, bus/ALU failure, configuration memory not set up, excessive input serial error, or Multibus timeouts.
|
| ciscoBus backplane and
Multibus backplane
| Cannot write configuration to memory; no memory access, memory access causes reboot.
The ciscoBus cards are not seen.
System will not boot or will reboot.
No DC voltages--some or all.
Bad power supply (caused by shorted backplane).
|
| CSC-R,CSC-R16M,
CSC-1R, CSC-2R, and
CSC-CTR
| Card is not seen by processor.
No ping to outside address or address of UUT; intermittent ping.
No hardware address seen.
Error Messages--Output hang, beaconing, local timeout, Open failed: lobe test, Multibus timeout.
|
| CSC-M, CSC-MT,
CSC-MC, and CSC-MC+
| NVRAM not seen by MCI-3 (CSC-MC).
Configuration cannot be written to memory.
Loses memory over time.
Configuration and/or Multibus memory wrong size (CSC-MT).
Error Messages--Bad checksum for configuration memory, configuration memory not set up, nonvolatile memory not present.
|
| Serial appliques
| Interface up but ping does not work, or intermittent ping functionality.
DTE will ping, DCE will not ping (or vice versa).
System reboots (with new V.35, suggests bad ground contact).
5V or 12V power supply LEDs indicate no power detected.
|
| IGS and 3000
| System will not boot.
Breaker trips or fuse blows.
Constant or partial reboot.
|
| 500-CS
| System will not boot.
|
| 4000
| System will not boot.
|
CiscoWorks is a router management tool that allows you to manage your internet from a central location. You can use CiscoWorks to monitor and troubleshoot complex internetworks. Because CiscoWorks uses the Simple Network Management Protocol (SNMP), it can monitor and control any SNMP device on an internet. CiscoWorks consists of five areas of operation: configuration management (which includes device management), fault management, accounting management, performance management, and security management.
In addition to the basic SNMP management functions, CiscoWorks provides a fully integrated relational database and uses built-in Sun Network Manager (SNM) capabilities to produce a dynamic, user-configurable, visual network map. The automatic map generation features associated with the CiscoWorks Path Tool capabilities can help you visually trace the routes to problem nodes.
Tools that can help you isolate connectivity and performance problems are outlined briefly in the following discussions. Refer to the CiscoWorks User Guide for complete details about using CiscoWorks to monitor and control your internetwork.
Use the following CiscoWorks fault management applications when troubleshooting connectivity problems in your internet.
- Device Monitor--Monitors specific devices for environmental and interface information. Sends event information to SNM that causes a glyph to change state.
- Path Tool--Graphically displays a route of the path from a source device to a destination device.
- Env. Monitor--Graphically displays the temperature and voltage data from an AGS+ router.
- Real-Time Graphs--Monitors the behavior of device interfaces or other network elements suspected of operating in a degraded mode and displays them in a graph.
- Show Commands--Enables you to view data similar to output from router EXEC show commands.
- Health Monitor--Provides information about the health of a device with access to several CiscoWorks applications on one window (including Show Commands and Real-Time Graphs) to monitor router activity.
- Contacts--Provides quick access to find your emergency contact person for a particular device.
- Log Manager--Enables you to store, query, and delete messages gathered from
CiscoWorks applications and Cisco Systems devices on the internetwork.
Use the following CiscoWorks performance management applications when troubleshooting performance problems in your internetwork:
- Device Polling--Probes and extracts data about the condition of your network devices.
- Polling Summary--Views polling data, stops and starts polling.
- Real-Time Graphs--Monitors the behavior of device interfaces or other network elements suspected of operating in a degraded mode and displays them in a graph.
- Path Tool--Graphically displays a route of the path from a source device to a destination device.
- Show Commands--Provides data similar to router EXEC show commands output.
- Sybase DWB--Allows you to access the Sybase Data Workbench application to write reports.
This publication emphasizes diagnostic tools provided with the router. However, other troubleshooting tools are also discussed in the symptom modules and scenarios.
In some cases, third-party diagnostic tools can be more useful that integrated tools. For example, enabling a debug command can be disastrous in any environment experiencing excessively high traffic levels. Attaching a network or serial analyzer to the suspect network is less intrusive and more likely to yield applicable information without exacerbating load problems for a router.
The following list summarizes some typical third-party troubleshooting tools:
- Time Domain Reflectometer (TDR)--A TDR transmits a short pulse of known amplitude and duration down a cable and measures the corresponding amplitude and time delay associated with resultant signal reflections. TDRs are available for all LAN types. Optical TDRs provide a similar test capability for fiber cable.
- Optical Power Source and Meter--This device employs an optical power source connected to one end of a fiber cable and a meter placed at the other end to measure optical power. Also called a "light meter," this device is a cost-effective alternative to an optical TDR.
- Oscilloscope--Scopes graphically display signal voltage per unit of time; commonly used to measure voltages on EIA-232 and EIA-422 interfaces
- Breakout Box (BOB)--A BOB displays and monitors status of EIA-232-D interface leads between Data Terminal Equipment (DTE) and Data Circuit-Terminating Equipment (DCE). BOBs are useful in reconfiguring interfaces.
- LAN Analyzer--LAN analyzers capture, record, and analyze frames transmitted on a LAN. Analyzers attach to a network just as any node does. These devices also are referred to as protocol analyzers and network analyzers. All analyzers support a range of physical interface specifications (including Ethernet, Token Ring, and FDDI), as well as a spectrum of network protocols (including TCP/IP, Novell IPX, IBM SNA, AppleTalk, DECnet, and ISO CLNS).
- WAN/Serial Line Analyzer--WAN protocol analyzers generally focus on WAN/serial line analysis, but can include LAN analysis capabilities. WAN analyzers support a range of physical interfaces (such as EIA-232, EIA-422, EIA-449, T1/E1, CCITT V.35, and CCITT X.21) and protocols (including HDLC, SDLC, Frame Relay, and ISDN).
Table 1-3 through Table 1-6 summarize general problem-solving guidelines for common media (Ethernet, serial/WAN, Token Ring and FDDI).
Suggested Actions for Ethernet Problems
| Media Problem
| Suggested Actions
|
|---|
| Errors or noise on Ethernet
| Step 1: Use a Time Domain Reflectometer (TDR) to find any unterminated Ethernet cables.
|
|
| Step 2: Check host cables and transceiver cables to determine whether any are incorrectly terminated, overly long, or damaged.
|
|
| Step 3: Look for a jabbering transceiver attached to a host (may require host-by-host inspection).
|
Suggested Actions for Serial Line Problems
| Media Problem
| Suggested Action
|
|---|
| Nonfunctional serial link
| Step 1: Use show interfaces serial number command to determine status of interface.
|
|
| Step 2: If show interfaces serial number indicates interface up/line protocol up, use the ping command between routers to test connectivity.
|
|
| Step 3: If routers do not respond to ping test, follow troubleshooting techniques as discussed in Chapter 7, "Troubleshooting WAN Connectivity."
|
Suggested Actions for Token Ring Problems
| Media Problem
| Suggested Action
|
|---|
| Nonfunctional Token Ring
| Step 1: Use show interfaces token number command to determine status of interface.
|
|
| Step 2: If status line indicates that the interface and line protocol are not up, check cable from router to MAU. Make sure that the cable is good; replace if necessary.
|
|
| Step 3: If show interfaces token number indicates interface up/line protocol up, use the ping command between routers to test connectivity.
|
|
| Step 4: If the remote router does not respond, check the ring specification on all nodes attached to the Token Ring backbone. Ring speed for all must be the same.
|
|
| Step 5: If necessary, modify ring speed specifications for clients, servers, and routers.
|
|
| Step 6: Use the ring speed command to modify ring speed configuration for IGS/TR. Change jumpers as needed for modular router platforms. Refer to your system's hardware installation and maintenance manual for more information about ring speed specification.
|
Suggested Actions for FDDI Problems
| Media Problem
| Suggested Actions
|
|---|
| Nonfunctional FDDI ring
| Step 1: Use the show interfaces fddi number command to determine status of interface.
|
|
| Step 2: If show interfaces fddi number indicates interface up/line protocol up, use the ping command between routers to test connectivity.
|
|
| Step 3: If interface is up and line protocol is up, make sure the MAC addresses of upstream and downstream neighbors is as expected.
|
|
| If all zeros appear in either of the address fields for these neighbors, a physical connection problem is likely.
|
|
| Step 4: In this case, (or if status line does not indicate interface up/line protocol up), check connections at patch panel or connectivity between using an optical TDR or light meter. Ensure that signal strength is within specification.
|
