[Pvfs2-users] Event tracing

Sam Lang slang at mcs.anl.gov
Tue Nov 18 16:21:03 EST 2008 

On Nov 18, 2008, at 12:15 PM, Nirmal Thacker wrote:

> Ok this is working but I want to make sure.
>
> First I run the pvfs2-set-eventmask *on the server* with the API and
> Operations flag
>
> I perform the I/O, in my case this would be running a code or  
> benchmark
>
> I turn off the pvfs2-set-eventmask on the server passing the NULL  
> values
> to the API and operations flag
>
> Finally on the client, I run the pvfs2-event-mon-example and get the
> output.
>
> For one- is this process correct, since Im confused where I must run  
> the
> pvfs2-set-eventmask utility - it works fine at both ends

It doesn't matter where you run it.

>
>
> For another, the pvfs2-event-mon-example output is a trace of I/O at  
> all
> the servers? Does this mean that the tracing occurs at each server and
> is sent over the network to the client which runs the
> pvfs2-event-mon-example utility?

Yes.

>
>
> Could I also understand how to interpret this output? There are 8
> columns of numbers. For example:
> 3 2 1 65536 0 2 1227047834 726432

It prints a format at the beginning.

-sam

>
>
> Nirmal
>
>
>
>
> Sam Lang wrote:
>>
>> You need to run the pvfs2-event-mon-example app on the client.  The
>> total process would be:
>>
>> pvfs2-set-eventmask -m /pvfs2mnt/ -a 0xFFFF -o 0xFFFF
>> < perform file system workload >
>> pvfs2-set-eventmask -m /pvfs2mnt/ -a 0 -o 0
>>
>> pvfs2-event-mon-example -m /pvfsmnt/
>>
>> The output of that tool is to stdout.
>>
>> -sam
>>
>> On Nov 18, 2008, at 11:33 AM, Nirmal Thacker wrote:
>>
>>> Yes Im looking at /tmp/pvfs2-server.log . Where would the events  
>>> tracing
>>> output be recorded?
>>> Nirmal
>>> Sam Lang wrote:
>>>>
>>>> On Nov 18, 2008, at 11:17 AM, Nirmal Thacker wrote:
>>>>
>>>>> Thanks for this, Sam.
>>>>>
>>>>> Although I've noticed that the events are recorded as HH:MM .
>>>>
>>>> They are seconds, microseconds.  Are you looking at the PVFS  
>>>> logging
>>>> output instead of the PVFS events tracing output?
>>>> -sam
>>>>
>>>>> I figure
>>>>> that if I need more fine grained timings, I would have to add this
>>>>> myself?
>>>>>
>>>>> Yes client side tracing would be helpful- has anyone performed any
>>>>> kind
>>>>> of tracing through the VFS- is it possible to capture maybe the  
>>>>> VFS
>>>>> related operations with a more VFS generic tracing tool and  
>>>>> hence this
>>>>> might be the reason client tracing is not developed in PVFS?
>>>>>
>>>>> Nirmal
>>>>> Sam Lang wrote:
>>>>>>
>>>>>> Nirmal,
>>>>>>
>>>>>> We don't have a tracing component for the VFS.  Component  
>>>>>> tracing on
>>>>>> the server is enabled for each component with:
>>>>>>
>>>>>> pvfs2-set-eventmask -a 0x4 -o 0x9
>>>>>>
>>>>>> That enables the TROVE api (disk accesses), specifically the
>>>>>> WRITE_LIST operation is only enabled.  The values of the api and
>>>>>> operation must be hexidecimal.  See the list of apis and  
>>>>>> operations
>>>>>> below.
>>>>>>
>>>>>> The current version of PVFS doesn't support event tracing on the
>>>>>> client.  We hope to have that working in a future version though.
>>>>>>
>>>>>> -sam
>>>>>>
>>>>>> APIs:
>>>>>>
>>>>>> BMI     0x2
>>>>>> TROVE     0x4
>>>>>> SM         0x80
>>>>>>
>>>>>>
>>>>>>  Operations:
>>>>>>
>>>>>>   BMI_SEND = 1,
>>>>>>   BMI_RECV = 2,
>>>>>>   FLOW = 3,
>>>>>>   TROVE_READ_AT = 4,
>>>>>>   TROVE_WRITE_AT = 5,
>>>>>>   TROVE_BSTREAM_FLUSH = 6,
>>>>>>   TROVE_KEYVAL_FLUSH = 7,
>>>>>>   TROVE_READ_LIST = 8,
>>>>>>   TROVE_WRITE_LIST = 9,
>>>>>>   TROVE_KEYVAL_READ = 10,
>>>>>>   TROVE_KEYVAL_READ_LIST = 11,
>>>>>>   TROVE_KEYVAL_WRITE = 12,
>>>>>>   TROVE_DSPACE_GETATTR = 13,
>>>>>>   TROVE_DSPACE_SETATTR = 14,
>>>>>>   TROVE_BSTREAM_RESIZE = 15,
>>>>>>   TROVE_KEYVAL_REMOVE = 16,
>>>>>>   TROVE_KEYVAL_ITERATE = 17,
>>>>>>   TROVE_KEYVAL_ITERATE_KEYS = 18,
>>>>>>   TROVE_DSPACE_ITERATE_HANDLES = 19,
>>>>>>   TROVE_DSPACE_CREATE = 20,
>>>>>>   TROVE_DSPACE_REMOVE = 21,
>>>>>>   TROVE_DSPACE_VERIFY = 22,
>>>>>>   TROVE_BSTREAM_VALIDATE = 23,
>>>>>>   TROVE_KEYVAL_VALIDATE = 24,
>>>>>>   TROVE_KEYVAL_WRITE_LIST = 25,
>>>>>>   TROVE_KEYVAL_GET_HANDLE_INFO = 26,
>>>>>>   TROVE_DSPACE_GETATTR_LIST = 27,
>>>>>>   TROVE_KEYVAL_REMOVE_LIST = 28,
>>>>>>
>>>>>>
>>>>>> On Nov 18, 2008, at 10:24 AM, Nirmal Thacker wrote:
>>>>>>
>>>>>>> That would be great!
>>>>>>> I'd like to probably trace each of those separately and if  
>>>>>>> there is
>>>>>>> any
>>>>>>> tracing possible on the client side as well
>>>>>>> Particularly the client side VFS tracing when I go through the  
>>>>>>> VFS
>>>>>>> would
>>>>>>> be interesting to note. I'd also like the server operation and  
>>>>>>> disk
>>>>>>> operation traces and network traces to discount for the time I
>>>>>>> spend on
>>>>>>> the network
>>>>>>>
>>>>>>> Nirmal
>>>>>>> Sam Lang wrote:
>>>>>>>>
>>>>>>>> Hi Nirmal,
>>>>>>>>
>>>>>>>> You can restrict it to trace only network events, only disk
>>>>>>>> events, or
>>>>>>>> only server operation events.  The default event driver just
>>>>>>>> keeps a
>>>>>>>> ring of events until they're requested with
>>>>>>>> pvfs2-event-mon-example.
>>>>>>>> Unfortunately, there's no good documentation for it.  I can  
>>>>>>>> give
>>>>>>>> you
>>>>>>>> more detail if you need it.  What are you are looking to do?
>>>>>>>>
>>>>>>>> -sam
>>>>>>>>
>>>>>>>> On Nov 13, 2008, at 12:54 PM, Nirmal Thacker wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I am trying to perform event tracing with PVFS- before I maybe
>>>>>>>>> perform
>>>>>>>>> my own instrumentation, I wanted to check out pvfs2-set- 
>>>>>>>>> eventmask
>>>>>>>>>
>>>>>>>>> I mostly understood its one form of usage from a thread on the
>>>>>>>>> list
>>>>>>>>> archives
>>>>>>>>> [http://www.beowulf-underground.org/pipermail/pvfs2-developers/2006-January/001774.html 
>>>>>>>>> ]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'd like to know if there are more variations to the usage.  
>>>>>>>>> For
>>>>>>>>> example
>>>>>>>>> are there more levels of tracing? Can the tracing be  
>>>>>>>>> restricted to
>>>>>>>>> only
>>>>>>>>> what I require?
>>>>>>>>>
>>>>>>>>> Is this documented in more detail somewhere ?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Nirmal
>>>>>>>>> _______________________________________________
>>>>>>>>> Pvfs2-users mailing list
>>>>>>>>> Pvfs2-users at beowulf-underground.org
>>>>>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2- 
>>>>>>>>> users
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

More information about the Pvfs2-users mailing list