Linux: How to see / change supported network bandwidth of NIC interface and get various eth network statistics with ethtool

Monday, 19th January 2015

linux-how-to-see-change-supported-network-bandwidth-of-NIC-interface-and-view-network-statistics
If you're a novice Linux sysadmin and inherited some dedicated servers without any documentation and hence on of the first things you have to do to start a new server documentation is to check the supported TCP/IP network speed of servers Network (ethernet) Interfaces. On Linux this is very easy task to verify the speed of LAN card supported Local / Internet traffic install ethtool (if not already preseont on the servers) – assuming you're dealing with Debian / Ubuntu Linux servers.

1. Install ethtool on Deb and RPM based distros

dedi-server1:~# apt-cache show ethtool|grep -i desc -A 3
Description: display or change Ethernet device settings
 ethtool can be used to query and change settings such as speed, auto-
 negotiation and checksum offload on many network devices, especially
 Ethernet devices.

dedi-server1:~# apt-get install –yes ethtool
..

ethtool should be installed by default on CentOS / Fedora / RHEL and  syntax is same like on Debs. If you happen to miss ethtool on any (SuSE) / RedHat / RPM based distro install it with yum

[root@centos:~] # yum -y install ethtool


2. Get ethernet configurations

To check the current eth0 / eth1 / ethX network (Speed / Duplex) and other network related configuration configuration:
 

dedi-server5:~# ethtool eth0

Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full

        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: off
        Supports Wake-on: pumbag
        Wake-on: g
        Current message level: 0x00000001 (1)
        Link detected: yes

Having a NIC configured to act as Duplex is very important as Duplex communication enables LAN card to communicate both sides (Sent / Receive) packets simultaneously.

full-duplex-half-duplex-explained-picture

Probably most interesting parameters for most admins are the ones that are telling whether the NIC UpLink is 10megabyte / 100 megabyte or 1Gigabyte as well as supported Receive / Send ( Transfer ) speeds of LAN, a common useful ethtool admin use to just show current LAN ethernet interface speed:

server-admin1:~# ethtool eth0 |grep -i speed
        Speed: 1000Mb/s

 

To get info about NIC (kernel module / driver) used with ethtool:

dedi-server3:~# ethtool -i eth0 driver: e1000e
version: 1.2.20-k2
firmware-version: 1.8-0
bus-info: 0000:06:00.0

3. Make LAN Card blink to recognize eth is mapped to which Physical LAN

Besides that ethtool has many other useful use cases, for example if you have a server with 5 lan or more LAN cards, but you're not sure to which of all different EthX interfaces correspond, a very useful thing is to make eth0, eth1, eth2, eth3, etc. blink for 5 seconds in order to identify which static IP is binded physically to which NIC , here is how:

ethtool -p eth0 5


Then you can follow the procedure for any interface on the server and map them with a sticker 🙂

Ethtool is also useful for getting "deep" (thorough) statistics on Server LAN cards, this could be useful to identify sometimes hard to determine broadcast flood attacks:
 

4. Get network statistics with ethtool for interfaces
 

dedi-server5:~# ethtool -S eth0|grep -vw 0
NIC statistics:
     rx_packets: 6196644448
     tx_packets: 7197385158
     rx_bytes: 2038559235701
     tx_bytes: 8281206569250
     rx_broadcast: 357508947
     tx_broadcast: 172
     rx_multicast: 34731963
     tx_multicast: 20
     rx_errors: 115
     multicast: 34731963
     rx_length_errors: 115
     rx_no_buffer_count: 26391
     rx_missed_errors: 10059
     tx_timeout_count: 3
     tx_restart_queue: 2590
     rx_short_length_errors: 115
     tx_tcp_seg_good: 964136993
     rx_long_byte_count: 2038559235701
     rx_csum_offload_good: 5824813965
     rx_csum_offload_errors: 42186
     rx_smbus: 383640020

5. Turn on Auto Negotiation and change NIC set speed to 10 / 100 / 1000 Mb/s

Auto-negotiation is important as an ethernet procedure by which two communication devices (2 network cards) choose common transmission parameters such as speed, duplex mode, and flow control in order to achieve maximum transmission speed over the network. On 1000BASE-T basednetworks the standard is a mandatory. There is also backward compatability for older 10BASE-T Networks.

a) To raise up NIC to use 1000 Mb/s in case if the bandwidth was raised to 1Gb/s but NIC settings were not changed:

dedi-server1:~# ethtool -s eth0 speed 1000 duplex half autoneg off


b) In case if LAN speed has to be reduced for some weird reason to 10 / 100Mb/s

 

dedi-server1:~# ethtool -s eth0 speed 10 duplex half autoneg off

dedi-server1:~# ethtool -s eth0 speed 100 duplex half autoneg off

c) To enable disable NIC Autonegotiation:

dedi-server1:~# ethtool -s eth0 autoneg on


6. Change Speed / Duplex settings to load on boot

a) Set Network to Duplex on Fedora / CentOS etc.

Quickest way to do it is of course to use /etc/rc.local. If you want to do it following distribution logic on CentOS / RHEL Linux:

Add to /etc/sysconfig/network-scripts/ifcfg-eth0

vim /etc/sysconfig/network/-scripts/ifcfg-eth0

ETHTOOL_OPTS="speed 1000 duplex full autoneg off"

To load the new settings restart networking (be careful to have physical access to server if something goes wrong 🙂 )

service network restart

b) Change network speed / duplex setting on Debian / Ubuntu Linux

Add at the end of /etc/network/interfaces

vim /etc/network/interfaces

post-up ethtool -s eth0 speed 100 duplex full autoneg off

7. Tune NIC ring buffers

dedi-server1:~# ethtool -g eth0

Ring parameters for eth0:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             256
RX Mini:        0
RX Jumbo:       0
TX:             256

As you can see the default setting of RX (receive) buffer size is low 256 and on busy servers with high traffic loads, depending on the hardware NIC vendor this RX buffer size varies.
Through increasing the Rx/Tx ring buffer size , you can decrease the probability of discarding packets in the NIC during a scheduling delay.
A change in rx buffer ring requires NIC restart so (be careful not to loose connection to remote server), be sure to have iLO access to it.

Here is how to raise Rx ring buffer size 4 times from default value:

ethtool -G eth0 rx 4096 tx 4069

Share this on:

Download PDFDownload PDF

Tags: , , , , , , , , , , , , , , , ,

Leave a Reply

CommentLuv badge