Hello all,<br><br>I have been trying to install PVFS 2.8.1 on Ubuntu server, Centos4 and <br>Scientific Linux 4. I compile it and can run it on a "single host" configuration<br>without any problems.<br><br>However, when I add more nodes to the configuration(always using the <br>
pvfs2-genconfig defaults ) I have the following problem:<br><br><b>On the metadata node I get these messages:</b><br><font size="1">[E 04/02 20:16] batch_create request got: Invalid argument<br>[E 04/02 20:16] batch_create request got: Invalid argument<br>
[E 04/02 20:16] batch_create request got: Invalid argument<br>[E 04/02 20:16] batch_create request got: Invalid argument</font><br><br><br><b>In the IO nodes I get:</b><br><font size="1">[root@wn140 ~]# tail -50 /tmp/pvfs2-server.log <br>
[D 04/02 23:53] BMI_testcontext completing: 18446744072456767880<br>[D 04/02 23:53] [SM Entering]: (0x88f8b00) msgpairarray_sm:complete (status: 1)<br>[D 04/02 23:53] [SM frame get]: (0x88f8b00) op-id: 37 index: 0 base-frm: 1<br>
[D 04/02 23:53] msgpairarray_complete: sm 0x88f8b00 status_user_tag 1 msgarray_count 1<br>[D 04/02 23:53] msgpairarray: 1 operations remain<br>[D 04/02 23:53] [SM Exiting]: (0x88f8b00) msgpairarray_sm:complete (error code: -1073742006), (action: DEFERRED)<br>
[D 04/02 23:53] [SM Entering]: (0x88f8b00) msgpairarray_sm:complete (status: 0)<br>[D 04/02 23:53] [SM frame get]: (0x88f8b00) op-id: 37 index: 0 base-frm: 1<br>[D 04/02 23:53] msgpairarray_complete: sm 0x88f8b00 status_user_tag 0 msgarray_count 1<br>
[D 04/02 23:53] msgpairarray: all operations complete<br>[D 04/02 23:53] [SM Exiting]: (0x88f8b00) msgpairarray_sm:complete (error code: 190), (action: COMPLETE)<br>[D 04/02 23:53] [SM Entering]: (0x88f8b00) msgpairarray_sm:completion_fn (status: 0)<br>
[D 04/02 23:53] [SM frame get]: (0x88f8b00) op-id: 37 index: 0 base-frm: 1<br>[D 04/02 23:53] (0x88f8b00) msgpairarray state: completion_fn<br>[E 04/02 23:53] Warning: msgpair failed to tcp://wn141:3334, will retry: Connection refused<br>
[D 04/02 23:53] *** msgpairarray_completion_fn: msgpair 0 failed, retry 1<br>[D 04/02 23:53] *** msgpairarray_completion_fn: msgpair retrying after delay.<br>[D 04/02 23:53] [SM Exiting]: (0x88f8b00) msgpairarray_sm:completion_fn (error code: 191), (action: COMPLETE)<br>
[D 04/02 23:53] [SM Entering]: (0x88f8b00) msgpairarray_sm:post_retry (status: 0)<br>[D 04/02 23:53] [SM frame get]: (0x88f8b00) op-id: 37 index: 0 base-frm: 1<br>[D 04/02 23:53] msgpairarray_post_retry: sm 0x88f8b00, wait 2000 ms<br>
[D 04/02 23:53] [SM Exiting]: (0x88f8b00) msgpairarray_sm:post_retry (error code: 0), (action: DEFERRED)<br>[D 04/02 23:53] [SM Entering]: (0x89476c0) perf_update_sm:do_work (status: 0)<br>[P 04/02 23:53] Start times (hr:min:sec): 23:53:11.330 23:53:10.310 23:53:09.287 23:53:08.268 23:53:07.245 23:53:06.225<br>
[P 04/02 23:53] Intervals (hr:min:sec) : 00:00:01.026 00:00:01.020 00:00:01.023 00:00:01.019 00:00:01.023 00:00:01.020<br>[P 04/02 23:53] -------------------------------------------------------------------------------------------------------------<br>
[P 04/02 23:53] bytes read : 0 0 0 0 0 0<br>[P 04/02 23:53] bytes written : 0 0 0 0 0 0<br>
[P 04/02 23:53] metadata reads : 0 0 0 0 0 0<br>[P 04/02 23:53] metadata writes : 0 0 0 0 0 0<br>
[P 04/02 23:53] metadata dspace ops : 0 0 0 0 0 0<br>[P 04/02 23:53] metadata keyval ops : 1 1 1 1 1 1<br>
[P 04/02 23:53] request scheduler : 0 0 0 0 0 0<br>[D 04/02 23:53] [SM Exiting]: (0x89476c0) perf_update_sm:do_work (error code: 0), (action: DEFERRED)<br>
[D 04/02 23:53] [SM Entering]: (0x8948810) job_timer_sm:do_work (status: 0)<br>[D 04/02 23:53] [SM Exiting]: (0x8948810) job_timer_sm:do_work (error code: 0), (action: DEFERRED)<br>[D 04/02 23:53] [SM Entering]: (0x89476c0) perf_update_sm:do_work (status: 0)<br>
[P 04/02 23:53] Start times (hr:min:sec): 23:53:12.356 23:53:11.330 23:53:10.310 23:53:09.287 23:53:08.268 23:53:07.245<br>[P 04/02 23:53] Intervals (hr:min:sec) : 00:00:01.020 00:00:01.026 00:00:01.020 00:00:01.023 00:00:01.019 00:00:01.023<br>
[P 04/02 23:53] -------------------------------------------------------------------------------------------------------------<br>[P 04/02 23:53] bytes read : 0 0 0 0 0 0<br>
[P 04/02 23:53] bytes written : 0 0 0 0 0 0<br>[P 04/02 23:53] metadata reads : 0 0 0 0 0 0<br>
[P 04/02 23:53] metadata writes : 0 0 0 0 0 0<br>[P 04/02 23:53] metadata dspace ops : 0 0 0 0 0 0<br>
[P 04/02 23:53] metadata keyval ops : 1 1 1 1 1 1<br>[P 04/02 23:53] request scheduler : 0 0 0 0 0 0<br>
[D 04/02 23:53] [SM Exiting]: (0x89476c0) perf_update_sm:do_work (error code: 0), (action: DEFERRED)<br>[D 04/02 23:53] [SM Entering]: (0x8948810) job_timer_sm:do_work (status: 0)<br>[D 04/02 23:53] [SM Exiting]: (0x8948810) job_timer_sm:do_work (error code: 0), (action: DEFERRED)</font><br>
<br><br>The metadata node keeps asking for something that the IO nodes cannot give <br>the right way. So it complains. This makes the nodes and the metadata node <br>not to work.<br><br>I have installed those services many times. I have tested this using berkeley <br>
db 4.2 and 4.3 on Redhat systems(centos, scientific linnux) and on Ubuntu server. <br><br>I have also tried the PVFS version 2.6.3 and I get the same problem.<br><br><b>My config files look like:</b><br>[root@wn140 ~]# more /etc/pvfs2-fs.conf <br>
<font size="1"><Defaults><br> UnexpectedRequests 50<br> EventLogging all<br> EnableTracing no<br> LogStamp datetime<br> BMIModules bmi_tcp<br> FlowModules flowproto_multiqueue<br> PerfUpdateInterval 1000<br>
ServerJobBMITimeoutSecs 30<br> ServerJobFlowTimeoutSecs 30<br> ClientJobBMITimeoutSecs 300<br> ClientJobFlowTimeoutSecs 300<br> ClientRetryLimit 5<br> ClientRetryDelayMilliSecs 2000<br> PrecreateBatchSize 512<br>
PrecreateLowThreshold 256<br><br> StorageSpace /pvfs<br> LogFile /tmp/pvfs2-server.log<br></Defaults><br><br><Aliases><br> Alias wn140 tcp://wn140:3334<br> Alias wn141 tcp://wn141:3334<br></Aliases><br>
<br><Filesystem><br> Name pvfs2-fs<br> ID 320870944<br> RootHandle 1048576<br> FileStuffing yes<br> <MetaHandleRanges><br> Range wn140 3-2305843009213693953<br> Range wn141 2305843009213693954-4611686018427387904<br>
</MetaHandleRanges><br> <DataHandleRanges><br> Range wn140 4611686018427387905-6917529027641081855<br> Range wn141 6917529027641081856-9223372036854775806<br> </DataHandleRanges><br>
<StorageHints><br> TroveSyncMeta yes<br> TroveSyncData no<br> TroveMethod alt-aio<br> </StorageHints><br></Filesystem></font><br><br><br>My setup is made from two nodes that are both IO and Metadata nodes. I have also tried<br>
a 4 node setup with 2I/O - 2 MD nodes resulting in the same thing.<br><br>Any suggestions?<br><br>thank you in advance,<br><font color="#888888">--<br clear="all">Asterios Katsifodimos<br>High Performance Computing systems Lab<br>
Department of Computer Science, University of Cyprus<br>
<a href="http://www.asteriosk.gr/" target="_blank">http://www.asteriosk.gr</a></font>