[Pvfs2-users] File affinity and CPU resource scheduling concerns when using compute node as a PVFS server

[Kyle Schochenmaier]

Chris -

A lot of work has been done with pvfs2 and infiniband.
Pete Wyckoff has many publications with it -  (I cant locate them
right now from my old link to his publications)
There are some documents from my work at AmesLab floating around, and
a careful search of the mailing list archives here will yield some
user-contributed data.

If you specify what type of performance metrics you are looking for I
might be able to help further.  We can take that offline.

Hope that helps.

Kyle Schochenmaier



On Wed, Oct 21, 2009 at 1:22 PM, Rob Ross <rross at mcs.anl.gov> wrote:
> Hi Chris,
>
> Yes, you can certainly run PVFS servers on compute nodes. That's what we
> used to do all the time back in "the day", but it's still possible.
>
> The file system does not migrate files. In part this is because it would
> dramatically complicate metadata management, in part because we are
> frequently running in environments where there's no locality to exploit.
>
> You can choose to distribute over fewer (or just one) node, and there is
> functionality to discover where a file is stored, so you can map your
> computation onto the storage.
>
> Obviously yes, doing work on some other node will slow down computation on
> that node, how much depends on many things.
>
> Rob
>
> On Oct 21, 2009, at 12:40 PM, Chris Worley wrote:
>
>> I haven't looked at PVFS for nearly a decade, so my understanding
>> could be way out of date.  Sorry if this is a stupid question; I'm
>> hoping PVFS might fill some requirements I'm looking at.
>>
>> Is it typical to use compute nodes as PVFS file servers?  Is it even
>> possible (i.e. it isn't w/ Lustre)?
>>
>> If so, while striping across I/O nodes looks to be the default, is
>> there any provision for file affinity... i.e. files migrating to the
>> nodes using them?
>>
>> If striping is the only choice, and compute nodes serve the I/O too,
>> then a job running on one set of nodes could be using files striped
>> across another set of nodes in use by another job, causing both jobs
>> to loose performance, correct?
>>
>> Are there any links to PVFS performance over IB?
>>
>> Thanks,
>>
>> Chris
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>