[Pvfs2-users] Event tracing

Tue Nov 18 13:15:56 EST 2008

Ok this is working but I want to make sure.

First I run the pvfs2-set-eventmask *on the server* with the API and
Operations flag

I perform the I/O, in my case this would be running a code or benchmark

I turn off the pvfs2-set-eventmask on the server passing the NULL values
to the API and operations flag

Finally on the client, I run the pvfs2-event-mon-example and get the
output.

For one- is this process correct, since Im confused where I must run the
pvfs2-set-eventmask utility - it works fine at both ends

For another, the pvfs2-event-mon-example output is a trace of I/O at all
the servers? Does this mean that the tracing occurs at each server and
is sent over the network to the client which runs the
pvfs2-event-mon-example utility?

Could I also understand how to interpret this output? There are 8
columns of numbers. For example:
3 2 1 65536 0 2 1227047834 726432

Nirmal

Sam Lang wrote:
>
> You need to run the pvfs2-event-mon-example app on the client.  The
> total process would be:
>
> pvfs2-set-eventmask -m /pvfs2mnt/ -a 0xFFFF -o 0xFFFF
> < perform file system workload >
> pvfs2-set-eventmask -m /pvfs2mnt/ -a 0 -o 0
>
> pvfs2-event-mon-example -m /pvfsmnt/
>
> The output of that tool is to stdout.
>
> -sam
>
> On Nov 18, 2008, at 11:33 AM, Nirmal Thacker wrote:
>
>> Yes Im looking at /tmp/pvfs2-server.log . Where would the events tracing
>> output be recorded?
>> Nirmal
>> Sam Lang wrote:
>>>
>>> On Nov 18, 2008, at 11:17 AM, Nirmal Thacker wrote:
>>>
>>>> Thanks for this, Sam.
>>>>
>>>> Although I've noticed that the events are recorded as HH:MM .
>>>
>>> They are seconds, microseconds.  Are you looking at the PVFS logging
>>> output instead of the PVFS events tracing output?
>>> -sam
>>>
>>>> I figure
>>>> that if I need more fine grained timings, I would have to add this
>>>> myself?
>>>>
>>>> Yes client side tracing would be helpful- has anyone performed any
>>>> kind
>>>> of tracing through the VFS- is it possible to capture maybe the VFS
>>>> related operations with a more VFS generic tracing tool and hence this
>>>> might be the reason client tracing is not developed in PVFS?
>>>>
>>>> Nirmal
>>>> Sam Lang wrote:
>>>>>
>>>>> Nirmal,
>>>>>
>>>>> We don't have a tracing component for the VFS.  Component tracing on
>>>>> the server is enabled for each component with:
>>>>>
>>>>> pvfs2-set-eventmask -a 0x4 -o 0x9
>>>>>
>>>>> That enables the TROVE api (disk accesses), specifically the
>>>>> WRITE_LIST operation is only enabled.  The values of the api and
>>>>> operation must be hexidecimal.  See the list of apis and operations
>>>>> below.
>>>>>
>>>>> The current version of PVFS doesn't support event tracing on the
>>>>> client.  We hope to have that working in a future version though.
>>>>>
>>>>> -sam
>>>>>
>>>>> APIs:
>>>>>
>>>>> BMI     0x2
>>>>> TROVE     0x4
>>>>> SM         0x80
>>>>>
>>>>>
>>>>>   Operations:
>>>>>
>>>>>    BMI_SEND = 1,
>>>>>    BMI_RECV = 2,
>>>>>    FLOW = 3,
>>>>>    TROVE_READ_AT = 4,
>>>>>    TROVE_WRITE_AT = 5,
>>>>>    TROVE_BSTREAM_FLUSH = 6,
>>>>>    TROVE_KEYVAL_FLUSH = 7,
>>>>>    TROVE_READ_LIST = 8,
>>>>>    TROVE_WRITE_LIST = 9,
>>>>>    TROVE_KEYVAL_READ = 10,
>>>>>    TROVE_KEYVAL_READ_LIST = 11,
>>>>>    TROVE_KEYVAL_WRITE = 12,
>>>>>    TROVE_DSPACE_GETATTR = 13,
>>>>>    TROVE_DSPACE_SETATTR = 14,
>>>>>    TROVE_BSTREAM_RESIZE = 15,
>>>>>    TROVE_KEYVAL_REMOVE = 16,
>>>>>    TROVE_KEYVAL_ITERATE = 17,
>>>>>    TROVE_KEYVAL_ITERATE_KEYS = 18,
>>>>>    TROVE_DSPACE_ITERATE_HANDLES = 19,
>>>>>    TROVE_DSPACE_CREATE = 20,
>>>>>    TROVE_DSPACE_REMOVE = 21,
>>>>>    TROVE_DSPACE_VERIFY = 22,
>>>>>    TROVE_BSTREAM_VALIDATE = 23,
>>>>>    TROVE_KEYVAL_VALIDATE = 24,
>>>>>    TROVE_KEYVAL_WRITE_LIST = 25,
>>>>>    TROVE_KEYVAL_GET_HANDLE_INFO = 26,
>>>>>    TROVE_DSPACE_GETATTR_LIST = 27,
>>>>>    TROVE_KEYVAL_REMOVE_LIST = 28,
>>>>>
>>>>>
>>>>> On Nov 18, 2008, at 10:24 AM, Nirmal Thacker wrote:
>>>>>
>>>>>> That would be great!
>>>>>> I'd like to probably trace each of those separately and if there is
>>>>>> any
>>>>>> tracing possible on the client side as well
>>>>>> Particularly the client side VFS tracing when I go through the VFS
>>>>>> would
>>>>>> be interesting to note. I'd also like the server operation and disk
>>>>>> operation traces and network traces to discount for the time I
>>>>>> spend on
>>>>>> the network
>>>>>>
>>>>>> Nirmal
>>>>>> Sam Lang wrote:
>>>>>>>
>>>>>>> Hi Nirmal,
>>>>>>>
>>>>>>> You can restrict it to trace only network events, only disk
>>>>>>> events, or
>>>>>>> only server operation events.  The default event driver just
>>>>>>> keeps a
>>>>>>> ring of events until they're requested with
>>>>>>> pvfs2-event-mon-example.
>>>>>>> Unfortunately, there's no good documentation for it.  I can give
>>>>>>> you
>>>>>>> more detail if you need it.  What are you are looking to do?
>>>>>>>
>>>>>>> -sam
>>>>>>>
>>>>>>> On Nov 13, 2008, at 12:54 PM, Nirmal Thacker wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I am trying to perform event tracing with PVFS- before I maybe
>>>>>>>> perform
>>>>>>>> my own instrumentation, I wanted to check out pvfs2-set-eventmask
>>>>>>>>
>>>>>>>> I mostly understood its one form of usage from a thread on the
>>>>>>>> list
>>>>>>>> archives
>>>>>>>> [http://www.beowulf-underground.org/pipermail/pvfs2-developers/2006-January/001774.html]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I'd like to know if there are more variations to the usage. For
>>>>>>>> example
>>>>>>>> are there more levels of tracing? Can the tracing be restricted to
>>>>>>>> only
>>>>>>>> what I require?
>>>>>>>>
>>>>>>>> Is this documented in more detail somewhere ?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Nirmal
>>>>>>>> _______________________________________________
>>>>>>>> Pvfs2-users mailing list
>>>>>>>> Pvfs2-users at beowulf-underground.org
>>>>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>