If you're a novice Linux sysadmin and inherited some dedicated servers without any documentation and hence on of the first things you have to do to start a new server documentation is to check the supported TCP/IP network speed of servers Network (ethernet) Interfaces. On Linux this is very easy task to verify the speed of LAN card supported Local / Internet traffic install ethtool (if not already preseont on the servers) – assuming you're dealing with Debian / Ubuntu Linux servers.
1. Install ethtool on Deb and RPM based distros
dedi-server1:~# apt-cache show ethtool|grep -i desc -A 3
Description: display or change Ethernet device settings
ethtool can be used to query and change settings such as speed, auto-
negotiation and checksum offload on many network devices, especially
Ethernet devices.
dedi-server1:~# apt-get install –yes ethtool
..
ethtool should be installed by default on CentOS / Fedora / RHEL and syntax is same like on Debs. If you happen to miss ethtool on any (SuSE) / RedHat / RPM based distro install it with yum
[root@centos:~] # yum -y install ethtool
…
2. Get ethernet configurations
To check the current eth0 / eth1 / ethX network (Speed / Duplex) and other network related configuration configuration:
dedi-server5:~# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: off
Supports Wake-on: pumbag
Wake-on: g
Current message level: 0x00000001 (1)
Link detected: yes
Having a NIC configured to act as Duplex is very important as Duplex communication enables LAN card to communicate both sides (Sent / Receive) packets simultaneously.
Probably most interesting parameters for most admins are the ones that are telling whether the NIC UpLink is 10megabyte / 100 megabyte or 1Gigabyte as well as supported Receive / Send ( Transfer ) speeds of LAN, a common useful ethtool admin use to just show current LAN ethernet interface speed:
server-admin1:~# ethtool eth0 |grep -i speed
Speed: 1000Mb/s
To get info about NIC (kernel module / driver) used with ethtool:
dedi-server3:~# ethtool -i eth0 driver: e1000e
version: 1.2.20-k2
firmware-version: 1.8-0
bus-info: 0000:06:00.0
3. Make LAN Card blink to recognize eth is mapped to which Physical LAN
Besides that ethtool has many other useful use cases, for example if you have a server with 5 lan or more LAN cards, but you're not sure to which of all different EthX interfaces correspond, a very useful thing is to make eth0, eth1, eth2, eth3, etc. blink for 5 seconds in order to identify which static IP is binded physically to which NIC , here is how:
ethtool -p eth0 5
Then you can follow the procedure for any interface on the server and map them with a sticker 🙂
Ethtool is also useful for getting "deep" (thorough) statistics on Server LAN cards, this could be useful to identify sometimes hard to determine broadcast flood attacks:
4. Get network statistics with ethtool for interfaces
dedi-server5:~# ethtool -S eth0|grep -vw 0
NIC statistics:
rx_packets: 6196644448
tx_packets: 7197385158
rx_bytes: 2038559235701
tx_bytes: 8281206569250
rx_broadcast: 357508947
tx_broadcast: 172
rx_multicast: 34731963
tx_multicast: 20
rx_errors: 115
multicast: 34731963
rx_length_errors: 115
rx_no_buffer_count: 26391
rx_missed_errors: 10059
tx_timeout_count: 3
tx_restart_queue: 2590
rx_short_length_errors: 115
tx_tcp_seg_good: 964136993
rx_long_byte_count: 2038559235701
rx_csum_offload_good: 5824813965
rx_csum_offload_errors: 42186
rx_smbus: 383640020
5. Turn on Auto Negotiation and change NIC set speed to 10 / 100 / 1000 Mb/s
Auto-negotiation is important as an ethernet procedure by which two communication devices (2 network cards) choose common transmission parameters such as speed, duplex mode, and flow control in order to achieve maximum transmission speed over the network. On 1000BASE-T basednetworks the standard is a mandatory. There is also backward compatability for older 10BASE-T Networks.
a) To raise up NIC to use 1000 Mb/s in case if the bandwidth was raised to 1Gb/s but NIC settings were not changed:
dedi-server1:~# ethtool -s eth0 speed 1000 duplex half autoneg off
b) In case if LAN speed has to be reduced for some weird reason to 10 / 100Mb/s
dedi-server1:~# ethtool -s eth0 speed 10 duplex half autoneg off
dedi-server1:~# ethtool -s eth0 speed 100 duplex half autoneg off
c) To enable disable NIC Autonegotiation:
dedi-server1:~# ethtool -s eth0 autoneg on
6. Change Speed / Duplex settings to load on boot
a) Set Network to Duplex on Fedora / CentOS etc.
Quickest way to do it is of course to use /etc/rc.local. If you want to do it following distribution logic on CentOS / RHEL Linux:
Add to /etc/sysconfig/network-scripts/ifcfg-eth0
vim /etc/sysconfig/network/-scripts/ifcfg-eth0
ETHTOOL_OPTS="speed 1000 duplex full autoneg off"
To load the new settings restart networking (be careful to have physical access to server if something goes wrong 🙂 )
service network restart
b) Change network speed / duplex setting on Debian / Ubuntu Linux
Add at the end of /etc/network/interfaces
vim /etc/network/interfaces
post-up ethtool -s eth0 speed 100 duplex full autoneg off
7. Tune NIC ring buffers
dedi-server1:~# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 256
RX Mini: 0
RX Jumbo: 0
TX: 256
As you can see the default setting of RX (receive) buffer size is low 256 and on busy servers with high traffic loads, depending on the hardware NIC vendor this RX buffer size varies.
Through increasing the Rx/Tx ring buffer size , you can decrease the probability of discarding packets in the NIC during a scheduling delay.
A change in rx buffer ring requires NIC restart so (be careful not to loose connection to remote server), be sure to have iLO access to it.
Here is how to raise Rx ring buffer size 4 times from default value:
ethtool -G eth0 rx 4096 tx 4069
