[Pvfs-users] pvfs_read beyond EOF

[Rob Ross]

Ah, my bad!  I need to pay closer attention!  Or not answer emails late
in the evening.  But you should be using PVFS2!

I've had a quick look back through the PVFS1 code.  There's a function
called prune_alist_and_zero_mem() in lib/prune.c that is doing this.
When we receive less data from a server than we expect, we zero the
regions of the buffer that we did not receive data for.  This is to
address the same issue that we were addressing in PVFS2 using the same
algorithm.

Same discussion of POSIX applies of course :).

Regards,

Rob

Wei-keng Liao wrote:
> Rob,
> 
> My question was sent to pvfs-user mailing list and was intended for PVFS 1.
> I did not test PVFS 2 for this case, but am interested in how PVFS 2 
> would behave.
> 
> I am using PVFS 1.6.3.
> 
> Wei-keng
> 
> 
> On Mon, 23 Jan 2006, Rob Ross wrote:
> 
>> Hi Wei-keng,
>>
>> PVFS2 isn't doesn't hold to the POSIX standard for all cases.  In the 
>> past we have cleared the entire read buffer in order to solve the 
>> problem of sparse file data access (servers with no data don't return 
>> anything).  In the general case (noncontiguous buffers and I/O regions 
>> that might be read in any order), this is a hard problem to solve. 
>> That's why you're seeing the behavior that you are seeing.
>>
>> However, I think that this behavior has changed recently and is likely 
>> closer to what you are expecting.  Sam, what's the latest behavior in 
>> this case? Wei-keng, what version of PVFS2 are you using?
>>
>> For my own edification, I found a copy of the POSIX spec here:
>>  http://www.opengroup.org/onlinepubs/009695399/toc.htm
>>
>> Here's the closest text that I found describing EOF behavior in that 
>> spec:
>>
>>  No data transfer shall occur past the current end-of-file. If the
>>  starting position is at or after the end-of-file, 0 shall be returned.
>>  If the file refers to a device special file, the result of subsequent
>>  read() requests is implementation-defined.
>>
>> That's ambiguous.  "data transfer" is vague.  They probably intended 
>> your interpretation, but they should have specified the condition of 
>> the buffer explicitly, perhaps in the rationale section.  I wouldn't 
>> count on that behavior on other systems either.
>>
>> Anyway, Sam will fill us in on the current semantic.  Given how much 
>> trouble it has been to get to where we are, I honestly doubt we'll 
>> adjust that code again in the near future.
>>
>> Regards,
>>
>> Rob
>>
>> Wei-keng Liao wrote:
>>>
>>> What will happen if a read (pvfs_read) reads beyond the current end 
>>> of file (EOF)? According to POSIX, read() should stop at EOF and 
>>> return the number of bytes read to the buffer. The buffer beyond EOF 
>>> should not be touched. (Please correct me if this is wrong.)
>>>
>>> But, I found the pvfs_read will set the buffer contents to null for 
>>> the beyond part. Here is the example code I tested: 1) create a new 
>>> file, 2) initialize read buffer, 3) read 10 bytes, 4) print out 
>>> buffer. (I am using PVFS 1.6.3) Is this the way PVFS behaves for 
>>> reading beyond EOF?
>>>
>>> Wei-keng Liao
>>> ---------------------------------------------------------------------
>>>
>>>
>>> #include <string.h>
>>> #include <sys/types.h>
>>> #include <sys/stat.h>
>>> #include <fcntl.h>
>>> #include <pvfs.h>
>>>
>>> #define PVFS_FILE_NAME "/pvfs/wkliao/testfile.dat"
>>> #define UNIX_FILE_NAME "testfile.dat"
>>>
>>> #define TEST_FILE(filename,ofunc,cfunc,rfunc)    {               \
>>>     fd = ofunc(filename, flag, 0600, 0, 0);                      \
>>>                                                                  \
>>>     readsize = rfunc(fd, buf, 10);                               \
>>>     cfunc(fd);                                                   \
>>>                                                                  \
>>>     printf("readsize for %s = %d bytes\n", filename, readsize);  \
>>>     printf("buf[*]= ");                                          \
>>>     for (i=0; i<10; i++) printf(" '%c'", buf[i]);                \
>>>     printf("\n");                                                \
>>> }
>>>
>>> int main(int argc, char **argv) {
>>>     int  i, readsize, flag;
>>>     int  fd;
>>>     char buf[10];
>>>
>>>     flag = O_TRUNC | O_CREAT | O_RDWR;
>>>
>>>     /* test pvfs file -------------------------------------------*/
>>>     for (i=0; i<10; i++) buf[i] = '1';
>>>     TEST_FILE(PVFS_FILE_NAME, pvfs_open, pvfs_close, pvfs_read);
>>>
>>>     /* test unix file -------------------------------------------*/
>>>     for (i=0; i<10; i++) buf[i] = '1';
>>>     TEST_FILE(UNIX_FILE_NAME, open, close, read);
>>>
>>>     return 1;
>>> }
>>
> _______________________________________________
> Pvfs-users mailing list
> Pvfs-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs-users
>