Hi Sam.<br>Well... thanks for your advice. By now I'm running a IOzone test with a few several changes:<br><br>- Without the -U option.<br>- Running only read tests.<br>- Using 1 Metadata server, 4 IO servers and only 1 client (where the test is running).
<br><br>And about the new pvfs2 feature; the flag for only read files, is a interest new feature, and it would be very useful for video streaming applications.<br><br>I will let you know the new test results. <br>Cheers<br>
<br><br>Gonzalo <br><br><br><div class="gmail_quote">On Nov 4, 2007 9:24 PM, Sam Lang <<a href="mailto:slang@mcs.anl.gov">slang@mcs.anl.gov</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>Gonzalo,<br><br>I haven't been able to reproduce your results. iozone runs to<br>completion for me with a similar setup and iozone parameters.<br><br>One thing about the iozone command line you're using though, the -U
<br>option will remount the filesystem after every test run, which for<br>PVFS isn't necessary. Also, I doubt that it matches any of the<br>workloads you intend. I'm not sure that's the cause of the problem<br>
you're seeing, but could you try it without the -U?<br><br>On the workload topic, it sounds like you're going to be doing a lot<br>of read-heavy I/O operations (video-streaming is pretty much read-<br>only, with large block sizes). Keep in mind that while iozone is a
<br>good overall filesystem benchmark, it won't be a good benchmark for<br>your large, read-heavy workloads, unless you specify to only run<br>tests that do reads, such as 1,7, or 10.<br><br>Also, while PVFS doesn't do client-side caching normally (Lustre
<br>does), in our upcoming release we will have a flag that can be set on<br>a file, making it immutable. This might be especially useful for<br>you, where you can set the immutable flag once the video has been<br>uploaded, allowing caching to take place on the clients. I can keep
<br>you appraised on the status of that release and how to set the<br>immutable flag if you're interested in that.<br><font color="#888888"><br>-sam<br></font><div><div></div><div class="Wj3C7c"><br><br>On Nov 2, 2007, at 6:43 PM, Gonzalo Soto Subiabre wrote:
<br><br>> Hi to everyone.<br>><br>> This is my first post and regretfully is to ask for help.<br>> I'm doing a performance evaluation of Distributed Parallel<br>> Filesystems, using iozone. The filesystems tested are Lustre, PVFS2
<br>> and GlusterFS. This evaluation has an aim: to choose a parallel<br>> filesystem for a video streamig system storage server.<br>> But I've a problem; when iozone test is running over pvfs2 it fails<br>> at a point where the file size is 2 GB and the Record Lenght 64 Kb.
<br>><br>><br>> My tested is this:<br>><br>> [root@redhat03 ~]# pvfs2-statfs -h -m /mnt/pvfs2/<br>> aggregate statistics:<br>> ---------------------------------------<br>><br>> fs_id: 1965017224
<br>> total number of servers (meta and I/O): 4<br>> handles available (meta and I/O): 4294967286<br>> handles total (meta and I/O): 4294967290<br>> bytes available:
111.3G<br>> bytes total: 111.3G<br>><br>> NOTE: The aggregate total and available statistics are calculated<br>> based<br>> on an algorithm that assumes data will be distributed evenly; thus
<br>> the free space is equal to the smallest I/O server capacity<br>> multiplied by the number of I/O servers. If this number seems<br>> unusually small, then check the individual server statistics below<br>> to look for problematic servers.
<br>><br>> meta server statistics:<br>> ---------------------------------------<br>><br>> server: tcp://redhat02:3334<br>> RAM total : 756.9M<br>> RAM free : 372.1M<br>> uptime : 2 hours, 12 minutes
<br>> load averages : 0 0 0<br>> handles available: 1717986912<br>> handles total : 1717986916<br>> bytes available : 28.2G<br>> bytes total : 34.3G<br>> mode: serving both metadata and I/O data
<br>><br>><br>> I/O server statistics:<br>> ---------------------------------------<br>><br>> server: tcp://redhat02:3334<br>> RAM total : 756.9M<br>> RAM free : 372.1M
<br>> uptime : 2 hours, 12 minutes<br>> load averages : 0 0 0<br>> handles available: 1717986912<br>> handles total : 1717986916<br>> bytes available :
28.2G<br>> bytes total : 34.3G<br>> mode: serving both metadata and I/O data<br>><br>> server: tcp://redhat03:3334<br>> RAM total : 488.2M<br>> RAM free :
10.4M<br>> uptime : 2 hours, 09 minutes<br>> load averages : 2784 10592 5536<br>> handles available: 858993458<br>> handles total : 858993458<br>> bytes available :
27.8G<br>> bytes total : 33.9G<br>> mode: serving only I/O data<br>><br>> server: tcp://redhat04:3334<br>> RAM total : 1003.6M<br>> RAM free : 605.3M<br>
> uptime : 2 hours, 07 minutes<br>> load averages : 4096 1312 224<br>> handles available: 858993458<br>> handles total : 858993458<br>> bytes available :
64.3G<br>> bytes total : 70.4G<br>> mode: serving only I/O data<br>><br>> server: tcp://redhat05:3334<br>> RAM total : 488.2M<br>> RAM free : 96.7M<br>
> uptime : 2 hours, 05 minutes<br>> load averages : 0 0 0<br>> handles available: 858993458<br>> handles total : 858993458<br>> bytes available : 27.8G
<br>> bytes total : 33.9G<br>> mode: serving only I/O data<br>><br>> ######################################################################<br>> #########<br>><br>> Therefore my cluster has 1 metadata server, 4 IO servers and 1 client.
<br>><br>> The iozone test command is this:<br>> [root@redhat02 ~]# iozone -Rab /home/pvfs2/Desktop/salida-redhat02-<br>> test03.xls -g 4G -f /mnt/pvfs2/iozone-file.tmp -U /mnt/pvfs2<br>><br>> and the message error is this:
<br>> fsync: Bad file descriptor<br>><br>> ######################################################################<br>> #########<br>><br>> Why I did chose IOzone? because his variety of tests and the excel
<br>> output options.<br>> If anyone can help me telling me why i've this message error and<br>> how can i resolve this, it would be great.<br>> Thanks a lot, cheers.<br>><br>><br>> --<br>> Gonzalo Soto Subiabre
<br>> Computer Engineer. FVT Chile Ltda.<br>> <a href="mailto:gsotosubiabre@gmail.com">gsotosubiabre@gmail.com</a><br>> msn: <a href="mailto:gonza_101@hotmail.com">gonza_101@hotmail.com</a><br></div></div><div><div>
</div><div class="Wj3C7c">> _______________________________________________<br>> Pvfs2-users mailing list<br>> <a href="mailto:Pvfs2-users@beowulf-underground.org">Pvfs2-users@beowulf-underground.org</a><br>>
<a href="http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users" target="_blank">http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users</a><br><br></div></div></blockquote></div><br><br clear="all"><br>--
<br>Gonzalo Soto Subiabre<br><a href="mailto:gsotosubiabre@gmail.com">gsotosubiabre@gmail.com</a><br>msn: <a href="mailto:gonza_101@hotmail.com">gonza_101@hotmail.com</a>