FIO(Linux) performance test-Storage adapter

FIO --- fio is a tool that will spawn a number of threads or processes doing a particular type of io action as specified by the user. fio takes a number of global parameters, each inherited by the thread unless otherwise parameters given to them overriding that setting is given. The typical use of fio is to write a job file matching the io load one wants to simulate.        

Source ------ fio resides in a git repo, the canonical place is: git://git.kernel.dk/fio.git The http protocol also works, path is the same. Snapshots are frequently generated and they include the git meta data as well. 
You can download them here: http://brick.kernel.dk/snaps/ 

Pascal Bleser <guru@unixtech.be> has fio RPMs in his repository for SUSE variants, you can find them here: http://linux01.gwdg.de/~pbleser/rpm-navigation.php?cat=System/fio 

Dag Wieërs has RPMs for Red Hat related distros, find them here: http://dag.wieers.com/rpm/packages/fio/ 

Mandriva has integrated fio into their package repository, so installing on that distro should be as easy as typing 'urpmi fio'.

FIO Official website:   http://freshmeat.net/projects/fio/

1.FIO Installation
wget http://brick.kernel.dk/snaps/fio-2.2.5.tar.gz

yum install libaio-devel
tar -zxvf fio-2.2.5.tar.gz
cd fio-2.2.5
make
make install

2.FIO usage:

Random Write 🙁 Use 10 threads to write a 2GB file  into a hard disk(or SSD),random read for 60 seconds and then report to mytest)
fio -filename=/tmp/test_randread -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=512k -size=2G -numjobs=16 -runtime=60 -group_reporting -name=mytest

filename=/dev/sdb1        test file,choose the data directory ,not system
direct=1           bypass some buffer that machine used, get reasonable result
rw=randwrite             random write I/O test
rw=randrw                random read and write  I/O test
bs=16k                   each I/O block size is 16k
bsrange=512-2048         each data block size range
size=5g             test file size is 5GB,and test with 4kbs I/O each time
numjobs=30               test thread is 30.
runtime=1000            test time is 1000 seconds
ioengine=psync           I/O engine is pync
rwmixwrite=30            Under mixed read and write mode, write part is 30%
group_reporting          collect the tread info and give a reportlockmem=1g               use 1GB memory to test。
zero_buffers             use zero to initial the buffer。
nrfiles=8              the file quality that created by each thread

Note: It is recommended to set the following parameters for high throughput.

–numjobs=16 <number of cores>: Check the number of cores in the host using lscpu
# lscpu

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 62
Model name: Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz
Stepping: 4
CPU MHz: 1200.561
BogoMIPS: 4004.05
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0-15

Sequential Read:
fio -filename=/dev/sdb1 -direct=1 -iodepth 1 -thread -rw=read -ioengine=psync -bs=512k -size=2G -numjobs=16 -runtime=60 -group_reporting -name=mytest

Random Write :
fio -filename=/dev/sdb1 -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=512k -size=2G -numjobs=16 -runtime=60 -group_reporting -name=mytest

Sequential Write:
fio -filename=/dev/sdb1 -direct=1 -iodepth 1 -thread -rw=write -ioengine=psync -bs=512k -size=2G -numjobs=16 -runtime=60 -group_reporting -name=mytest

Mixed Random Read and Write:
fio -filename=/dev/sdb1 -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=70 -ioengine=psync -bs=512k -size=2G -numjobs=16 -runtime=60 -group_reporting -name=mytest -ioscheduler=noop

 

3.Test example:

[root@localhost ~]# fio -filename=/dev/sdb1 -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=70 -ioengine=psync -bs=16k -size=200G -numjobs=30 -runtime=100 -group_reporting -name=mytest1
mytest1: (g=0): rw=randrw, bs=16K-16K/16K-16K, ioengine=psync, iodepth=1

mytest1: (g=0): rw=randrw, bs=16K-16K/16K-16K, ioengine=psync, iodepth=1
fio 2.0.7
Starting 30 threads
Jobs: 1 (f=1): [________________m_____________] [3.5% done] [6935K/3116K /s] [423 /190  iops] [eta 48m:20s] s]
mytest1: (groupid=0, jobs=30): err= 0: pid=23802
  Important :read : io=1853.4MB, bw=18967KB/s, iops=1185 , runt=100058msec
clat (usec): min=60 , max=871116 , avg=25227.91, stdev=31653.46
lat (usec): min=60 , max=871117 , avg=25228.08, stdev=31653.46
clat percentiles (msec):
|  1.00th=[    3],  5.00th=[    5], 10.00th=[    6], 20.00th=[    8],
| 30.00th=[   10], 40.00th=[   12], 50.00th=[   15], 60.00th=[   19],
| 70.00th=[   26], 80.00th=[   37], 90.00th=[   57], 95.00th=[   79],
| 99.00th=[  151], 99.50th=[  202], 99.90th=[  338], 99.95th=[  383],
| 99.99th=[  523]
bw (KB/s)  : min=   26, max= 1944, per=3.36%, avg=636.84, stdev=189.15
 Important : write: io=803600KB, bw=8031.4KB/s, iops=501 , runt=100058msec
clat (usec): min=52 , max=9302 , avg=146.25, stdev=299.17
lat (usec): min=52 , max=9303 , avg=147.19, stdev=299.17
clat percentiles (usec):
|  1.00th=[   62],  5.00th=[   65], 10.00th=[   68], 20.00th=[   74],
| 30.00th=[   84], 40.00th=[   87], 50.00th=[   89], 60.00th=[   90],
| 70.00th=[   92], 80.00th=[   97], 90.00th=[  120], 95.00th=[  370],
| 99.00th=[ 1688], 99.50th=[ 2128], 99.90th=[ 3088], 99.95th=[ 3696],
| 99.99th=[ 5216]
bw (KB/s)  : min=   20, max= 1117, per=3.37%, avg=270.27, stdev=133.27
lat (usec) : 100=24.32%, 250=3.83%, 500=0.33%, 750=0.28%, 1000=0.27%
lat (msec) : 2=0.64%, 4=3.08%, 10=20.67%, 20=19.90%, 50=17.91%
lat (msec) : 100=6.87%, 250=1.70%, 500=0.19%, 750=0.01%, 1000=0.01%
cpu          : usr=1.70%, sys=2.41%, ctx=5237835, majf=0, minf=6344162
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=118612/w=50225/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=1853.4MB, aggrb=18966KB/s, minb=18966KB/s, maxb=18966KB/s, mint=100058msec, maxt=100058msec
WRITE: io=803600KB, aggrb=8031KB/s, minb=8031KB/s, maxb=8031KB/s, mint=100058msec, maxt=100058msec

Disk stats (read/write):
sdb: ios=118610/50224, merge=0/0, ticks=2991317/6860, in_queue=2998169, util=99.77%