[Pvfs2-developers] TroveSyncData settings

[Rob Ross]

This is similar to using O_DIRECT, which has also shown benefits.

With alt aio, do we sync in the context of the I/O thread?

Thanks,

Rob

Phil Carns wrote:
> 
>> One thing that we noticed while testing for storage challenge was that 
>> (and everyone correct me if I'm wrong here) enabling the data-sync 
>> causes a flush/sync to occur after every sizeof(FlowBuffer) bytes had 
>> been written.  I can imagine how this would help a SAN, but I'm 
>> perplexed how it helps localdisk, what buffer size are you playing with?
>> We found that unless we were using HUGE (~size of cache on storage 
>> controller) flowbuffers that this caused way too many syncs/seeks on 
>> the disks and hurt performance quite a bit, maybe even as bad as 50% 
>> performance because things were not being optimized for our disk 
>> subsystems and we were issuing many small ops instead of fewer large 
>> ones.
>>
>> Granted I havent been able to get 2.6.0 building properly yet to test 
>> the latest out, but this was definitely the case for us on the 2.5 
>> releases.
> 
> You are definitely right about the data sync option causing a flush/sync 
> on every sizeof(FLowBuffer).  I don't really have a good explanation for 
> why this doesn't seem to burn us anymore on local disk.  Our settings 
> are standard, except for:
> 
> - 512KB flow buffer size
> - alt aio method
> - 512KB tcp buffers (with larger /proc tcp settings)
> 
> This testing was done on some version prior to 2.6.0 also (I think it 
> was a merge of some in-between release, so it is hard to pin down a 
> version number).
> 
> It may also have something to do with the controller and local disks 
> being used?  All of our local disk configurations are actually hardware 
> raid 5 with some variety of the megaraid controller, and these are 
> fairly new boxes.
> 
> -Phil
> 
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>