Projects / fio / Comments

Comments for fio

29 May 2014 04:25 paragkamble

Kindly help me to get FIO for solaris 10/Solaris11

07 Oct 2013 13:56 diatta

I have written a script using fio the benchmark filesystems that I would like to share:

fio.pl

#!/usr/bin/perl
# Program name : my_fio_script_generator.pl
# Author : Mamadou Lamine Diatta
# Email: diatta at post dot harvard dot edu
# Purpose : Generate job files used by Linux command fio.
# ==========================================================================
# Here few example how to use fio at the command line without a job file.
#
#Storage subsystem:
# IBM EXP300 - 14 Disk Fiber Expansion, 17G - 15K RPMS
# Host: AMD Barcelona, 2 socket, 8G RAM
# HBA: QLogic Corp. ISP2312-based 2Gb Fibre Channel to PCI-X HBA (rev 02)
#
#Benchmark[1]:
# fio --name=guestrun --filename=/dev/mapper/volumes-fibre \
# --rw=randwrite --bs=16k --ioengine=libaio --direct=1 \
# --norandommap --runtime=120 --time_based --numjobs=1 \
# --group_reporting --thread --size=25g --write_lat_log \
# --write_bw_log --iodepth=74
#
#Qemu parameters:
# -m 1024 \
# -drive file=/images/npt2-guest-virtio.qcow2,if=ide,boot=on,snapshot=off \
# -drive file=/dev/mapper/volumes-fibre,if=virtio,cache=(on|off) \
# -drive file=/dev/mapper/volumes-npt2--dom1,if=virtio,cache=off
# -net nic,macaddr=00:FF:FF:00:00:01,model=rtl8139 -net tap -vnc :123 \
# -monitor stdio
#
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Few functions here.
# ***************************************************************************
sub usage {
print<<END;

--------------------------------------------------------------------
Usage: $0 <directory>
Where: <directory> is a full path access read and write directory
NOTE: If the directory doesn't exist, it will be created if possible
i.e: $0 /asm01/fio/io1
--------------------------------------------------------------------

END
}
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sub generate_filename {
my $range = 36;
my $minimum = 0;
@ALPHA=('q','w','e','r','0','t','y','1','u','i','2','o','p','3','a','s','4','d','f','5','g','h','j','6','k','l','7','z','x','8','c','v','9','b','n','m');
$filenamesize=10;
$line=1;
$filename="";
while(1){
my $random_number = int(rand($range)) + $minimum;
$filename=$filename . $ALPHA[$random_number];
if ($line == $filenamesize) {
return $filename;
}
$line++;
}
}
# ***************************************************************************
sub generate_seconds {
my $range = 60;
my $minimum = 1;
my $seconds = int(rand($range)) + $minimum;
return $seconds;
}
# ***************************************************************************
sub generate_percent {
my $range = 100;
my $minimum = 1;
my $percent = int(rand($range)) + $minimum;
return $percent;
}
# ***************************************************************************
sub generate_numjobs {
my $range = 10;
my $minimum = 0;
my $numjobs = int(rand($range)) + $minimum;
return $numjobs;
}
# ***************************************************************************
sub generate_runtime {
my $range = 3600;
# Run between 1s or 1 hour = 60*60 seconds
my $minimum = 0;
my $runtime = int(rand($range)) + $minimum;
return $runtime;
}
# ***************************************************************************
sub generate_nrfiles {
my $range = 5000;
my $minimum = 1;
my $ioload = int(rand($range)) + $minimum;
return $ioload;
}
# ***************************************************************************
sub generate_filesize {
my $rangesize = 9;
my $rangebyte = 3;
my $minimum = 0;
@BYTE=('k','m','g');
@SIZEK=('1024','2048','4092','4096','5120','6144','7168','8192','9216');
@SIZEM=('100','200','300','400','500','600','700','800','900');
@SIZEG=('1','2','3','4','5','6','7','8','9');
$filesize="";

my $random_number = int(rand($rangebyte)) + $minimum;
my $byte=$BYTE[$random_number];
if ( $byte eq "k" ){
my $random_number = int(rand($rangesize)) + $minimum;
$filesize=$SIZEK[$random_number] . $byte;
}
elsif ( $byte eq "m" ){
my $random_number = int(rand($rangesize)) + $minimum;
$filesize=$SIZEM[$random_number] . $byte;
}
else {
my $random_number = int(rand($rangesize)) + $minimum;
$filesize=$SIZEG[$random_number] . $byte;
}
return $filesize;
}
# ***************************************************************************
sub generate_write_read_options {
my $rangerw = 4;
my $minimum = 0;
@RW=('read','randread','write','randwrite');
my $random_number = int(rand($rangerw)) + $minimum;
$rw=$RW[$random_number];
return $rw;
}
# ***************************************************************************
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Main function
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#print "#ARGV=$#ARGV\n";
if ( $#ARGV != 0) {
&usage;
exit 1;
}
my $workingdir=$ARGV[0];
if ( ! -d "$workingdir" ) {
mkdir ($workingdir, 0755) ||
die "Can't make directory: $workingdir: $!\n";
}
else {
print "Directory: $workingdir already exist!\n";
}
while(1) {
$FIO="/usr/local/bin/fio";
$RM="/bin/rm -fr";
$CAT="/bin/cat";
my $scriptdir="/tmp";

my $fio_script= &generate_filename . ".fio";
open(F ,">$scriptdir/$fio_script") ||
die "Can't open file : $scriptdir/$fio_script : $!\n";

my $rw= &generate_write_read_options;
my $filename= &generate_filename;
my $filesize= &generate_filesize;
my $rwmixread= &generate_percent;
my $rwmixwrite= 100 - $rwmixread;
# Note: rwmixread and rwmixwrite are related and must
# add up to 100%.

my $nrfiles= &generate_nrfiles;
my $runtime= &generate_runtime;
my $numjobs= &generate_numjobs;
print F "; $rw $filesize with rwmixread=$rwmixread and rwmixwrite=$rwmixwrite\n";
print F "[${rw}-${rwmixread}-${rwmixwrite}]\n";
print F "rw=$rw\n";
print F "directory=$workingdir\n";
print F "filename=$filename\n";
print F "size=$filesize\n";
print F "rwmixread=$rwmixread\n";
print F "rwmixwrite=$rwmixwrite\n";
print F "nrfiles=$nrfiles\n";
print F "runtime=$runtime\n";
print F "description=${rw}_${filename}_${filesize}_${rwmixread}_${rwmixwrite}_${nrfiles}_${runtime}\n";
print F "unlink=1\n";
# If set unlink files when it's done.
print F "numjobs=$numjobs\n";
print F "write_bw_log\n";
# To write bandwidth log.
print F "write_lat_log\n";
# To write latency log.
print F "; ----------------------------------------------------\n\n\n";
close(F);
print "---------------------- Running $FIO $scriptdir/$fio_script\n";
open(R, "$scriptdir/$fio_script") ||
die "Can't open $scriptdir/$fio_script : $!\n";
while(<R>){
chop $_;
print "$_\n";
}
print "\n";
print "\n";
close(R);
`cat $scriptdir/$fio_script > ${fio_script}.log`;
`$FIO $scriptdir/$fio_script >> ${fio_script}.log 2>&1`;
`$RM $scriptdir/$fio_script`;
sleep 10;
}
# ***********************************************************************************
# #######################################################

30 Nov 2012 18:10 bslade

Getting 'stat64 already defined' type errors when trying to "make" fio on RedHat/Centos 6.3 (linux kernel 2.6.18-308). Nothing in the http://www.spinics.net/lists/fio mailing list archives about it. No response to my signup request to the fio@vger.kernel.org mailing list, so I'm posting this problem here.

Has anybody else seen a problem like this?

> make
DEP .depend
CC gettime.o
: Assembler messages:
:354: Error: symbol `fstatat64' is already defined
:388: Error: symbol `fstat64' is already defined
:418: Error: symbol `lstat64' is already defined
:448: Error: symbol `stat64' is already defined
:1704: Error: symbol `pread64' is already defined
make: *** [gettime.o] Error 1

25 Oct 2012 16:44 dschulz

Id appreciate it if anyone here could let me know how to find that information.

25 Oct 2012 16:43 dschulz

When was the switch made to version 3 from version 2 of the headers for --minimal output and was something other than the order of the FIO Version/Jobname/Groupid/Error changed?

moving between 1.5.8 and 2.0.10 it went from:
fio 1.50
2 job 0 0

to

3 fio-2.0.10 job 0 0

10 Oct 2012 17:30 KeithLocke

I have a couple of test requirements that I would appreciate guidance on how to perform with fio.

1. 80/20 mix of random/sequential IO.
2. 80% Random, 10% '00', & 10% 'FF' mixture of data patterns

11 Jun 2012 12:09 anakon

Hi Guys,

I'm having an issue and I'd appreciate if someone could lend a hand.

I am trying to generate latency logs and bandwidth logs with fio, but am having no luck.

The settings I'm using are based off of the 1mbs_clients test :

[global]
size=32m
rw=read
directory=d:/tmp
rate=1250
ratemin=1024
write_lat_log
write_bw_log

[file1]
startdelay=0
filename=file1test

[file2]
startdelay=4

[file3]
startdelay=8

[file4]
startdelay=12


with the following command line (I have tries many other variations) :

fio examples/1mbs_clients --section=file1 --output=d:/tmp/output.log

but whatever I do, I can not generate the latency-log or bandwidth log.


Any help would be greatly appreciated.

Many Thanks,

Adrian.

13 Apr 2012 10:30 axboe

nitink, that should be fixed. Try upgrading, 2.0.7 was just tagged yesterday.

In general, please report issues on the fio mailing list, fio@vger.kernel.org
I don't watch this space very closely, evidenced by the month long reply time :-)

01 Mar 2012 14:03 nitink

I have tried to compile latest fio code on windows if fails with bellow error message. please help to solve this issue

Administrator@win2k8-nitin ~/Documents/fio-2.0.5
$ make
DEP .depend
CC gettime.o
In file included from gettime.c:10:0:
fio.h:8:26: fatal error: sys/resource.h: No such file or directory
compilation terminated.
make: *** [gettime.o] Error 1

Administrator@win2k8-nitin ~/Documents/fio-2.0.5
$ ls
COPYING cgroup.h fifo.c fio_version.h io_u.c memory.c rbtree.c trim.c
HOWTO client.c fifo.h flist.h ioengine.h mutex.c rbtree.h trim.h
Makefile compiler file.h flow.c ioengines.c mutex.h server.c verify.c
README crc filehash.c flow.h iolog.c options.c server.h verify.h
REPORTING-BUGS debug.c filehash.h gettime.c iolog.h options.h smalloc.c
SERVER-TODO debug.h filesetup.c gettime.h lib os smalloc.h
arch diskutil.c fio.1 hash.h libfio.c parse.c stat.c
backend.c diskutil.h fio.c helpers.c log.c parse.h stat.h
blktrace.c engines fio.h helpers.h log.h profile.c t
blktrace_api.h eta.c fio_generate_plots init.c memalign.c profile.h time.c
cgroup.c examples fio_generate_plots.1 io_ddir.h memalign.h profiles time.h

Administrator@win2k8-nitin ~/Documents/fio-2.0.5

14 Oct 2011 10:47 axboe

vjacheslav, there is synchronization support. See the stonewall/wait_for_previous primitive. You might also be interested in using exit_all to terminate the write part when the read part finishes.

Screenshot

Project Spotlight

ReciJournal

An open, cross-platform journaling program.

Screenshot

Project Spotlight

Veusz

A scientific plotting package.