This chapter focuses on common symptoms associated with poor performance in internetworks, possible causes of those symptoms, and general suggestions for identifying, isolating, and resolving causes.
This chapter consists of the following sections:
- Performance symptom list
- Symptom/cause/action modules
The symptom/cause/action modules consist of the following sections:
- Symptom statement--A specific symptom associated with the technology/media/protocol in which this module appears.
- Possible causes and suggested actions--A table for each symptom containing possible causes for the symptom and suggested actions for resolving each cause.
 | Caution Throughout this and other chapters, the use of debug commands is suggested for obtaining information about network traffic and router status. Use these commands with great care. In general, it is recommended that these commands only be used under the direction of your router technical support representative when troubleshooting specific problems. Enabling debugging can disrupt operation of the router when internets are experiencing high load conditions. When you finish using a debug command, remember to disable it with its specific undebug command or with the undebug all command. |
The symptom modules that follow pertain to performance-related problems for the various protocols and technologies addressed in this publication. Unless otherwise indicated, each module is presented as a set of problems associated with a specific technology or protocol. Where there are special considerations associated with a specific network type, notes are included.
Performance symptoms discussed in this section include the following:
- Sporadic service availability and poor AppleTalk internet performance
- Slow performance and intermittent loss of connections over Cisco RSRB
- Poor Novell server performance over router in LAN internet
- Poor Novell server performance over router in WAN
- Generally slow performance in TCP/IP internetworks
- Slow TCP/IP performance despite multiple paths
- Slow network or host response over WAN or serial link
- Loss of connections over WAN or serial link
Symptom: Connectivity to AppleTalk services over an internetwork is unpredictable and generally slow.
Table 9-1 outlines possible causes of unpredictable and slow performance in an AppleTalk internetwork.
Causes and Actions for Poor AppleTalk Internet Performance
| Possible Cause
| Suggested Actions
|
|---|
| ZIP storm
| Step 1: Use show appletalk traffic to look for number of ZIP Requests displayed; repeat after 30 seconds or so.
|
|
| Step 2: Compare resulting display output. If number is greater than 10 and increasing, there is probably a ZIP storm occurring.
|
|
| Step 3: Use show appletalk route to see whether a network shows up in the table, even though the zone indicates no zone set in the display.
|
|
| Step 4: If you find a network with this zone specification, a node in that network is probably not responding to ZIP requests, resulting in the ZIP storm.
|
|
| Step 5: Determine why the node is not responding to ZIP requests.
|
| Duplicate network numbers
| Step 1: The network where these symptoms are noticeable is likely to contain duplicate network numbers equidistant from the point where problems are observed.
|
|
| |
|
| Step 2: If you changed the network number on the interface, no further action is required. If not, change it now (make sure it is unique). Remember to reenter the zone name and any other interface configurations for Appletalk on that interface.
|
| Unexpected back door
| Step 1: Inspect internetwork for any bridges that may link networks that are routing AppleTalk.
|
|
| Step 2: If any bridges (including routers configured for bridging) are found, set all bridges to forward nonrouted protocols and filter routed protocols.
|
|
| Step 3: Monitor reporting of routes and neighbors with show appletalk route and show interfaces commands.
|
|
| Step 4: If networks continue to be associated with the wrong interfaces, consult your router technical support representative for more assistance.
|
Symptom: Users complain about connection loss at peak traffic periods when trying to connect to resources on the other side of a router configured for remote source-route bridging (RSRB).
Table 9-2 outlines possible causes and suggested actions when users experience intermittent connectivity in RSRB interconnections.
Causes and Actions for Load-Related RSRB Performance Problems
| Possible Cause
| Suggested Actions
|
|---|
| Busy router; high CPU utilization when using TCP encapsulation
| Step 1: Use show process EXEC command to determine CPU utilization. Look for CPU utilization higher than 50 percent.
|
|
| Step 2: Check configuration for keyword local-ack at the end of the source-bridge remote-peer global configuration command.
|
|
| Step 3: Add this additional keyword if missing.
|
|
| Step 4: Consider implementing Fast Sequenced Transport (FST) on the link, using the source-bridge fst-peername global configuration command. Refer to your Router Products Configuration and Reference publication for details about FST.
|
Symptom: Users complain about sessions dropping at peak traffic periods when trying to connect to resources on the other side of a router configured to route Novell IPX.
Table 9-3 outlines possible causes of poor file server response in a routed Novell IPX LAN internetwork.
Causes and Actions for Novell Performance Problems in LAN Internet
| Possible Cause
| Suggested Actions
|
|---|
| Excessive traffic; collisions causing session drops (Ethernet problem only)
| Step 1: Use protocol analyzer to examine traffic.
Step 2: Look for collisions in excess of normal acceptable conditions (varies for specific site).
|
|
| |
|
| Step 3: Examine output of protocol analyzer to determine bandwidth utilization. Analyzer information will provide more accurate reading of dynamic traffic information.
|
|
| Step 4: If bandwidth utilization detected by the analyzer is 15 to 20 percent (on average) or higher, you are likely to have a load-related performance problem.
|
|
| Step 5: If you see that collisions are increasing steadily with a higher-than-expected bandwidth utilization, consider segmenting the network with additional bridges or routers.
|
| Insufficient bandwidth on Token Ring to handle traffic
| Step 1: Upgrade from 4-Mbps to 16-Mbps Token Ring throughout network.
|
|
| Step 2: If performance is still inadequate, consider segmenting the network with additional bridges or routers.
|
Symptom: Users complain about sessions dropping at peak traffic periods when trying to connect to resources on the other side of a router configured to route Novell IPX over WAN or serial link.
Table 9-4 outlines possible causes of poor file server response in a routed Novell IPX internetwork.
Causes and Actions for Novell Performance Problems in WAN
| Possible Cause
| Suggested Actions
|
|---|
| Other protocol dominates CPU time
| Step 1: Use the show process command to look for large numbers appearing in the "Runtime (ms)" and "Invoked" fields for certain protocols. An example would be a protocol that has a value that is 10 times or greater than the value indicated for Novell traffic.
|
|
| |
|
| Step 2: Use the show interfaces command to look for a high level of output drops.
|
|
| Step 3: If you see output drops and a particular other protocol is dominating CPU time (per show process "Runtime (ms)" field), use priority queuing to force system to handle Novell traffic over other protocols. More information about priority queuing is provided in Chapter 7, "Troubleshooting WAN Connectivity."
|
|
| Step 4: If priority queuing does not work, add bandwidth by implementing a higher-speed line or adding additional lines (of same speed).
|
Symptom: TCP/IP internetwork performance is slow, with poor host response, spotty connection service, and generally slow file transfers. Packets may be dropped.
Table 9-5 outlines possible causes of generally slow performance in a TCP/IP internetwork.
Causes and Actions for Slow Performance in TCP/IP Internets
| Possible Cause
| Suggested Actions
|
|---|
| Bad network link; results in packets being dropped and lost
| Step 1: Ping out along entire length of path to determine where packets are being dropped.
Step 2: Refer to Chapter 1 for general hardware diagnostic information. Refer to Chapter 7, "Troubleshooting WAN Connectivity," for more specific information about serial debugging.
|
|
| Step 3: Perform serial debugging or other media debugging.
|
|
| Step 4: Replace hardware or add bandwidth as necessary.
|
| Access list applied to one link but not another (when there are multiple paths to a destination)
| Step 1: Refer to the symptom section following this section entitled "Slow TCP/IP Performance Despite Multiple Paths."
|
| Congested link
| Step 1: Determine whether the link is indeed congested.
|
|
| Step 2: Refer to Chapter 1 for general diagnostic information. Refer to Chapter 7, "Troubleshooting WAN Connectivity," for more information about serial debugging.
|
|
| Step 3: Apply priority queuing if feasible.
|
|
| Step 4: Add bandwidth or additional routers if priority queuing does not help.
|
Symptom: Despite multiple paths from one network to another and apparently sufficient bandwidth, performance over the links is poor and traffic does not appear to be getting through some of the links. Although this can be considered a connectivity problem, it manifests itself as a performance issue.
Table 9-6 outlines possible causes of performance problems in TCP/IP networks because some paths are being blocked.
Causes and Actions for Poor Performance Because of Blocked Paths
| Possible Cause
| Suggested Actions
|
|---|
| Misconfigured access lists where there are multiple paths and one or more access lists block access to one or more routes
| Step 1: Use the ping and trace EXEC commands to determine where traffic is stopping. Works best when standard access lists are used.
Step 2: If ping or trace packets are stopped along the way, check the specific router for access lists.
Step 3: If an access list is found, disable the list and monitor traffic through the router, using the ping and trace commands.
|
|
| Step 4: If ping and trace packets get through after removing the access list, you might need to add explicit permit statements to the access list to allow blocked traffic type.
|
|
| |
|
| Step 5: If ping packets get through, use a Sniffer along the path where problems occur to see where the dropped packet type was last seen. The next node is the most likely suspect.
|
|
| Step 6: As in the prior access list discussion, remove the access list and monitor traffic through the router using the protocol being blocked.
|
| Bad interface or media hardware
| Step 1: Refer to Chapter 1 for general hardware diagnostic information. Refer to Chapter 7, "Troubleshooting WAN Connectivity," for more information about serial debugging.
|
|
| Step 2: Perform serial debugging or other media debugging.
|
|
| Step 3: Replace hardware or add bandwidth as necessary.
|
| Load balancing problem (see Figure 9-1 that follows this table)
| Step 1: Use show interfaces and show ip traffic EXEC commands and ping out to destination to determine where traffic is being dropped.
|
|
| Step 2: At point of congestion, relieve traffic problems by adding a router in parallel or by increasing the bandwidth of the link.
|
|
| Step 3: If you cannot add a router or bandwidth, try the following:
|
|
| |
Figure 9-1 illustrates a situation where two routes may be equivalent in terms of hop count from Host-D to Far-Net, but due to the level of traffic (associated with Router-B and the Data Center), the alternative route (through Router-A) is administratively preferred. In this case, both routes look equally good to Host-D, so without any configuration modifications, Host-D can use either Router-A or Router-B to communicate with Far-Net. However, as outlined in the load balancing problem discussion in Table 9-6, several options are available to force traffic from Host-D (intended for Far-Net) to go through Router-A.
Figure 9-1: Load Balancing Problem Map
Symptom: As with similar loss of connection problems, users complain about very slow host and network responsiveness at peak traffic periods over a WAN or serial link.
In general, obtaining the following information will be useful in troubleshooting load-related connection problems:
- Observe output of show interface serial command on both ends of the serial line; evaluate error counters.
- If you do see input errors, refer to "Evaluating Input Errors" in Chapter 7, "Troubleshooting WAN Connectivity," for details about isolating the sources of input errors.
Table 9-7 outlines possible causes of load-related performance in serial and WAN interconnections.
Causes and Actions for Load-Related WAN Performance Problems
| Possible Cause
| Suggested Actions
|
|---|
| Dirty serial line
| Step 1: Determine whether input errors are increasing.
|
|
| Step 2: If input errors appear, diagnose serial line per discussion in Chapter 7, "Troubleshooting WAN Connectivity."
|
| Overutilized bandwidth
| Step 1: If input errors do not appear, the problem is related to congestion.
|
|
| Step 2: Turn off fast switching on affected interface.
|
|
| Step 3: Check applications being run, especially for very large file transfers scheduled at particular times of day.
|
|
| Step 4: If this is the case, set up priority queue (requires that the protocol allows flow control).
|
|
| Step 5: Rearrange file transfer timing by applications so that links are not overused during normal business hours.
|
|
| Step 6: Add bandwidth and consider using dial backup over the new link for applications that are taking excessive bandwidth on existing links.
|
|
| Step 7: Adjust buffer size (8.3 or more recent software needed).
|
| Hardware in the serial link is unreliable
| Step 1: Troubleshoot serial line per CSU/DSU loopback tests and ping tests described in Chapter 7 or with a serial analyzer.
|
|
| Step 2: Replace hardware as necessary.
|
| Carrier is automatically rerouting T1 trunk lines
| Step 1: Contact long line carrier service to determine whether this is happening.
|
|
| Step 2: Ensure that carrier provides dedicated circuit if automatic switching is causing performance problems.
|
Symptom: Users complain about dropped connections and the inability to make host connections at peak traffic periods. One example of this problem is in an environment featuring bridged DEC Local Area Transport (LAT) traffic and multiple routed protocols. Data entry input from users (or other application requests) may be getting buffered at the end of an already long input queue--eventually one end of the connection will time out.
Table 9-8 outlines possible causes of load-related connection drops in serial and WAN interconnections.
Causes and Actions for WAN Performance-Related Loss of Connections
| Possible Cause
| Suggested Actions
|
|---|
| Dirty serial line
| Step 1: Determine whether input errors are increasing.
|
|
| Step 2: If input errors appear, diagnose serial line per discussion in Chapter 7, "Troubleshooting WAN Connectivity."
|
| Overutilized bandwidth
| Step 1: If input errors do not appear, the problem is related to congestion.
|
|
| Step 2: Turn off fast switching on affected interface.
|
|
| Step 3: Check applications being run, especially for very large file transfers scheduled at particular times of day.
|
|
| Step 4: If this is the case, set up priority queue (requires that the protocol allows flow control).
|
|
| Step 5: When bridging LAT, consider implementing LAT compression to reduce bandwidth.
|
|
| |
|
| Step 6: Rearrange file transfer timing by applications so that links are not overused during normal business hours.
|
|
| Step 7: Add bandwidth and consider using dial backup over the new link for applications that are taking excessive bandwidth on existing links.
|
|
| Step 8: Adjust buffer size (8.3 or more recent software needed).
|
| Hardware in the serial link is unreliable
| Step 1: Troubleshoot serial line per CSU/DSU loopback tests and ping tests described in Chapter 7 or with a serial analyzer.
|
| Inadequate bandwidth
| Step 1: After checking all of the above, inspect the show interfaces serial display.
|
|
| Step 2: If after these actions, load is still indicated at about 80 percent, your line is inadequate for traffic requirements.
|
|
| Step 3: Add another serial line.
|
