Ihave setup total four servers in cloud based data center. They run either CentOS or Ubuntu Linux. I need to make sure clustered file system performance is reasonable as my servers shared with other users. How do I check the speed between the two Linux servers using command line options for private LAN/VLAN?
You can test the network speed (throughput) between Ubuntu/CentOS/Debian/Fedora or Unix box using the iperf command. Iperf commands show info about bandwidth, delay, jitter, and datagram loss. From the man page:
iperf is a tool for performing network throughput measurements. It can test either TCP or UDP throughput. To perform an iperf test the user must establish both a server (to discard traffic) and a client (to generate traffic).
Here is my sample setup:
Fig.01 Test network speed between two Linux computers
Step 1 – Installation
You need to install iperf as per your Linux distro version. There are two versions of iperf:
iperf
iperf3
The iperf/iperf3 works on Linux, Unix, Windows, *BSD and other operating systems.
Install iperf on a Ubuntu/Debian Linux server
Type the following apt-get command/[nixmcd name=”apt”] to install iperf on both servers: $ sudo apt-get update $ sudo apt-get install iperf Sample outputs:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
iperf
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 51.0 kB of archives.
After this operation, 184 kB of additional disk space will be used.
Get:1 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf amd64 2.0.5+dfsg1-2[51.0 kB]
Fetched 51.0 kB in 0s (4921 kB/s)
Selecting previously unselected package iperf.
(Reading database ... 26108 files and directories currently installed.)
Preparing to unpack .../iperf_2.0.5+dfsg1-2_amd64.deb ...
Unpacking iperf (2.0.5+dfsg1-2) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up iperf (2.0.5+dfsg1-2) ...
A note about installing iperf3 on a Debian/Ubuntu Linux
It is also possible to install iperf3 (latest version) as follows: $ sudo apt-get install iperf3 Sample outputs:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libiperf0
The following NEW packages will be installed:
iperf3 libiperf0
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 58.5 kB of archives.
After this operation, 238 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://mirrors.linode.com/ubuntu xenial/universe amd64 libiperf0 amd64 3.0.11-1 [50.4 kB]
Get:2 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf3 amd64 3.0.11-1 [8090 B]
Fetched 58.5 kB in 0s (4796 kB/s)
Selecting previously unselected package libiperf0:amd64.
(Reading database ... 26120 files and directories currently installed.)
Preparing to unpack .../libiperf0_3.0.11-1_amd64.deb ...
Unpacking libiperf0:amd64 (3.0.11-1) ...
Selecting previously unselected package iperf3.
Preparing to unpack .../iperf3_3.0.11-1_amd64.deb ...
Unpacking iperf3 (3.0.11-1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libiperf0:amd64 (3.0.11-1) ...
Setting up iperf3 (3.0.11-1) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Install iperf on a Fedora Linux server
Type the following [nixmcd name=”dnf”] to install iperf on both servers: $ sudo dnf install iperf
A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux
Type the following command: $ sudo dnf install iperf3
Install iperf on a CentOS/RHEL/Oracle/Scientific Linux server
Type the following [nixmcd name=”yum”] to install iperf on both servers (first turn on EPEL repo for CentOS/RHEL): $ sudo yum install iperf Sample outputs:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: centos.mirror.snu.edu.in
* epel: mirror01.idc.hinet.net
* extras: centos.mirror.snu.edu.in
* updates: centos.mirror.snu.edu.in
Resolving Dependencies
--> Running transaction check
---> Package iperf.x86_64 0:2.0.8-1.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=====================================================================
Package Arch Version Repository Size
=====================================================================
Installing:
iperf x86_64 2.0.8-1.el7 epel 321 k
Transaction Summary
=====================================================================
Install 1 Package
Total download size: 321 k
Installed size: 396 k
Is this ok [y/d/N]: y
Downloading packages:
iperf-2.0.8-1.el7.x86_64.rpm | 321 kB 00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : iperf-2.0.8-1.el7.x86_64 1/1
Verifying : iperf-2.0.8-1.el7.x86_64 1/1
Installed:
iperf.x86_64 0:2.0.8-1.el7
Complete!
A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux
Type the following command: $ sudo yum install iperf3
Step 2 – Start an iperf server on serverA (server mode)
Type the following command to run iperf in server mode: iperf -s OR iperf3 -s Sample outputs:
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
A note about firewall config
Please note that you must open TCP port 5001 on the serverA. See how to use ufw to open a port on Ubuntu/Debian Linux: $ sudo ufw allow from 192.168.149.0/24 to 192.168.149.69 port 5001 proto tcp It is also possible to change the port by passing the -p option as follows (e.g. open and use TCP port 2323): iperf -s -p 2323
Step 3 – Start an iperf server on serverB (client mode)
Type the following command to run in client mode, connecting to 192.168.149.69: iperf -c {ip-address-of-serverA-here} iperf -c {ip-address-of-serverA-here} -p {tcp-port} iperf -c 192.168.149.69 -p 2323 iperf -c 192.168.149.69 OR iperf3 -c {ip-address-of-serverA-here} iperf3 -c {ip-address-of-serverA-here} -p {tcp-port} iperf3 -c 192.168.149.69 -p 2323 iperf3 -c 192.168.149.69 Sample outputs:
Fig.02: Linux network speed test result
How to use UDP rather than TCP
On the server side: iperf -s -u On the client side: iperf -c 192.168.149.69 -u
How to set number of parallel client threads to run (client mode options)
Pass the -P option: iperf -c 192.168.149.69 -P 3 You can also set time in seconds to transmit for (default 10 secs) as follows: iperf -c 192.168.149.69 -P 3 -t 30 Sample outputs:
------------------------------------------------------------
Client connecting to 192.168.149.69, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 5] local 192.168.181.40 port 54198 connected with 192.168.149.69 port 5001[ 3] local 192.168.181.40 port 54194 connected with 192.168.149.69 port 5001[ 4] local 192.168.181.40 port 54196 connected with 192.168.149.69 port 5001[ ID] Interval Transfer Bandwidth
[ 5]0.0-30.0 sec 2.26 GBytes 647 Mbits/sec
[ 3]0.0-30.0 sec 2.89 GBytes 828 Mbits/sec
[ 4]0.0-30.5 sec 1.91 GBytes 539 Mbits/sec
[SUM]0.0-30.5 sec 7.07 GBytes 1.99 Gbits/sec
Here is output from iperf3: ## on the server side ## iperf3 -s ## on the client side ## iperf3 -c 192.168.149.69 Sample outputs:
Type the following command: $ man iperf $ iperf --help
Usage: iperf [-s|-c host][options]
iperf [-h|--help][-v|--version]
Client/Server:
-f, --format [kmKM] format to report: Kbits, Mbits, KBytes, MBytes
-i, --interval # seconds between periodic bandwidth reports
-l, --len #[KM] length of buffer to read or write (default 8 KB)
-m, --print_mss print TCP maximum segment size (MTU - TCP/IP header)
-o, --output <filename> output the report or error message to this specified file
-p, --port # server port to listen on/connect to
-u, --udp use UDP rather than TCP
-w, --window #[KM] TCP window size (socket buffer size)
-B, --bind <host> bind to <host>, an interface or multicast address
-C, --compatibility for use with older versions does not sent extra msgs
-M, --mss # set TCP maximum segment size (MTU - 40 bytes)
-N, --nodelay set TCP no delay, disabling Nagle's Algorithm
-V, --IPv6Version Set the domain to IPv6
Server specific:
-s, --server run in server mode
-U, --single_udp run in single threaded UDP mode
-D, --daemon run the server as a daemon
Client specific:
-b, --bandwidth #[KM] for UDP, bandwidth to send at in bits/sec
(default 1 Mbit/sec, implies -u)
-c, --client <host> run in client mode, connecting to <host>
-d, --dualtest Do a bidirectional test simultaneously
-n, --num #[KM] number of bytes to transmit (instead of -t)
-r, --tradeoff Do a bidirectional test individually
-t, --time # time in seconds to transmit for (default 10 secs)
-F, --fileinput <name> input the data to be transmitted from a file
-I, --stdin input the data to be transmitted from stdin
-L, --listenport # port to receive bidirectional tests back on
-P, --parallel # number of parallel client threads to run
-T, --ttl # time-to-live, for multicast (default 1)
-Z, --linux-congestion <algo> set TCP congestion control algorithm (Linux only)
Miscellaneous:
-x, --reportexclude [CDMSV] exclude C(connection) D(data) M(multicast) S(settings) V(server) reports
-y, --reportstyle C report as a Comma-Separated Values
-h, --help print this message and quit
-v, --version print version information and quit
[KM] Indicates options that support a K or M suffix for kilo- or mega-
The TCP window size option can be set by the environment variable
TCP_WINDOW_SIZE. Most other options can be set by an environment variable
IPERF_<long option name>, such as IPERF_BANDWIDTH.
