[Pvfs2-developers] protocol encoding

[Sam Lang]

On Jan 30, 2008, at 11:39 AM, Murali Vilayannur wrote:

> Hi Walt,
>> In src/proto/pvfs2-req-proto.h when you define a new request you  
>> create
>> a struct and then use a macro to create an encoding function for the
>> struct (endecode_fields_X_struct).  Sometimes, in the args to those
>> macros you insert a skip4,, which I gather is used to align  
>> something.
>> Can someone explain the rules for when and where you place this?
>
> ia64 and/or x86_64 likes pointers aligned on 8 byte boundaries.
> Unaligned access to memory via a pointer can cause one or more of the
> following I think
> - slower performance
> - segmentation faults
> skip4 is needed only when pointers are involved as far as I
> understand. You don't need those when accessing scalar types.
>
>>
>> There is also some confusion as to the naming of those macros, in  
>> that
>> some of them seem to count the skip4,, and some don't.  In  
>> particular,
>> if there are 3 scalar arguments, but we need one skip, we use the
>> endecode_fields_4_struct macro - so we DO count the skip (3 args +  
>> skip
>> = 4) but if there is an array, say 3 scalars plus an array, we use  
>> the
>> endecode_fields_3a_struct macro - so we DO NOT count the skip.  Some
>> array macros have one, some have two skips.  Any words of wisdom,  
>> or do
>> we just have to look it up in the code?
>
> yeah.. I think that needs to be fixed. If I am not mistaken, the
> number in the naming convention must include the skip4 also.
> As regards to why some array macros have one and some have two skips
> is because the former embeds only 1 pointer while the latter embeds
> two sets of arrays along with the count.
> Since the count is typically 4 bytes, we insert a skip4 and then drop
> the array after that.

Murali, I agree with your explanation, but its not clear what you  
think needs to be fixed.  The macros match the naming convention you  
described.

Maybe part of the confusion that Walt&Co have is that the an array  
macro (3a_struct) always includes an extra uint32_t for the length of  
the array, but that field is not counted in the number used in the  
name of the macro.  For example:

endecode_fields_3a_struct(
     PVFS_servresp_readdir,
     PVFS_ds_keyval, token,
     uint64_t, directory_version,
     skip4,,
     uint32_t, dirent_count,
     PVFS_dirent, dirent_array)

There are 3 fields and a struct here.  The 3 fields are token,  
directory_version, and the skip4.  The array is the count and the  
dirent_array.  The skip is needed because of the 32 bit count field,  
but in some cases it wouldn't be:

endecode_fields_2a_struct(
	my_op,
	uint64_t, var1,
	uint32_t, var2,
	uint32_t, array_count,
	uint32_t, array)

You might argue that we should always just pad the array count, or use  
a 64 bit value for it, but I don't think Pete wanted to waste bytes in  
the request unless necessary.  Its hard to quibble about 4 bytes, but  
that design focus does help keep request messages under the eager  
message sizes of our protocols.

-sam


>
>
>> For those who are interested, the first thing we are working on is
>> Phil's server-to-server enabled file create.  In the first step we  
>> are
>> migrating the client create syscall functionality to the server,  
>> then we
>> will work on implementing collective communication.  Right now we are
>> trying to figure out to what extent we can use the new state machine
>> features to simplify that by essentially starting a client state  
>> machine
>> on the server.  Any input on that activity is encouraged.
> Awesome!
> Thanks,
> Murali
>>
>> Walt
>> --
>> Dr. Walter B. Ligon III
>> Associate Professor
>> ECE Department
>> Clemson University
>> _______________________________________________
>> Pvfs2-developers mailing list
>> Pvfs2-developers at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>