我正在
Linux服务器(内核2.6.37,16内核,32G内存)上的2个文件系统之间复制大文件(3 x 30G),但性能很差.我怀疑缓冲区缓存的使用正在扼杀I / O性能.
为了尝试缩小问题范围,我直接在SAS磁盘上使用fio来监控性能.
这是2个fio运行的输出(第一个是direct = 1,第二个是direct = 0)：
配置：
- [test]
- rw=write
- blocksize=32k
- size=20G
- filename=/dev/sda
- # direct=1
运行1：
- test: (g=0): rw=write,bs=32K-32K/32K-32K,ioengine=sync,iodepth=1
- Starting 1 process
- Jobs: 1 (f=1): [W] [100.0% done] [0K/205M /s] [0/6K iops] [eta 00m:00s]
- test: (groupid=0,jobs=1): err= 0: pid=4667
- write: io=20,480MB,bw=199MB/s,iops=6,381,runt=102698msec
- clat (usec): min=104,max=13,388,avg=152.06,stdev=72.43
- bw (KB/s) : min=192448,max=213824,per=100.01%,avg=204232.82,stdev=4084.67
- cpu : usr=3.37%,sys=16.55%,ctx=655410,majf=0,minf=29
- IO depths : 1=100.0%,2=0.0%,4=0.0%,8=0.0%,16=0.0%,32=0.0%,>=64=0.0%
- submit : 0=0.0%,4=100.0%,64=0.0%,>=64=0.0%
- complete : 0=0.0%,>=64=0.0%
- issued r/w: total=0/655360,short=0/0
- lat (usec): 250=99.50%,500=0.45%,750=0.01%,1000=0.01%
- lat (msec): 2=0.01%,4=0.02%,10=0.01%,20=0.01%
- Run status group 0 (all jobs):
- WRITE: io=20,aggrb=199MB/s,minb=204MB/s,maxb=204MB/s,mint=102698msec,maxt=102698msec
- Disk stats (read/write):
- sda: ios=0/655238,merge=0/0,ticks=0/79552,in_queue=78640,util=76.55%
运行2：
- test: (g=0): rw=write,iodepth=1
- Starting 1 process
- Jobs: 1 (f=1): [W] [100.0% done] [0K/0K /s] [0/0 iops] [eta 00m:00s]
- test: (groupid=0,jobs=1): err= 0: pid=4733
- write: io=20,bw=91,265KB/s,iops=2,852,runt=229786msec
- clat (usec): min=16,max=127K,avg=349.53,stdev=4694.98
- bw (KB/s) : min=56013,max=1390016,per=101.47%,avg=92607.31,stdev=167453.17
- cpu : usr=0.41%,sys=6.93%,ctx=21128,minf=33
- IO depths : 1=100.0%,short=0/0
- lat (usec): 20=5.53%,50=93.89%,100=0.02%,250=0.01%,500=0.01%
- lat (msec): 2=0.01%,4=0.01%,20=0.01%,50=0.12%
- lat (msec): 100=0.38%,250=0.04%
- Run status group 0 (all jobs):
- WRITE: io=20,aggrb=91,minb=93,455KB/s,maxb=93,mint=229786msec,maxt=229786msec
- Disk stats (read/write):
- sda: ios=8/79811,merge=7/7721388,ticks=9/32418456,in_queue=32471983,util=98.98%
我不太了解fio来解释结果,但我不认为使用缓冲区缓存的整体性能比O_DIRECT低50％.
有人可以帮我解释fio输出吗？
是否有任何内核调整可以解决/最小化问题？
非常感谢,
