Re: LAM: Nonblocking communication issue

This sounds right.  It's been eons since I've looked at the
GM code  
in LAM, but the short sends are eager and the long sends use
a  
rendezvous protocol.  Specifically, the main content of the
message  
won't be sent for a "long" message until the
receiver ACKs that a  
matching MPI receive has been posted.

This is, among other reasons, to help prevent resource
exhaustion at  
the receiver.

I believe that there is an SSI parameter to change the size
of  
short / long messages, but I don't recall what it is
offhand.  Did  
you look in the LAM/MPI User's Guide?  I seem to recall
documenting  
all such things in there...


On Mar 22, 2007, at 6:03 AM, Vartan Padaryan wrote:

> Hello everybody.
>
> I'm exploring performance of various communication
patterns on
> Linux-cluster with LAM-7.1.1 over GM. A simple example
with  
> nonblocking
> communication produced strange results.
>
> Proc0: Isend; Some computation; Wait;
> Proc1: Recv;
>
> In case of buf_size <= 16KB that's OK. Execution
time of Recv is  
> stable
>   ~100us. for 4KB and any communication mode of send
(Isend, Issend,
> Ibsend, Irsend). But if buf_size > 16KB, execution
time of Recv  
> depends
> on weight of computation in Proc0, i.e. the more
computations Proc0
> does, the longer Recv goes on.
> Is it correct behavior?
> Does LAM change protocol for nonblocking sends, if
buf_size more  
> than 16K?
> How can I get the same behavior for large buffer?
> Maybe my code isn't right (i use pselect for modeling
delay between
> Isend and Wait)?
>
> WBR, Vartan Padaryan.
> _______________________________________________
> This list is archived at http://www.l
am-mpi.org/MailArchives/lam/


-- 
Jeff Squyres
Cisco Systems

_______________________________________________
This list is archived at http://www.l
am-mpi.org/MailArchives/lam/

Jeff Squyres wrote:
> This sounds 
> right.  It's been eons since I've looked at the GM code
in LAM, but the 
> short sends are eager and the long sends use a
rendezvous protocol.  
> Specifically, the main content of the message won't be
sent for a "long" 
> message until the receiver ACKs that a matching MPI
receive has been 
> posted.
> 
> This is, among other reasons, to help prevent resource
exhaustion at the 
> receiver.
> 
> I believe that there is an SSI parameter to change the
size of short / 
> long messages, but I don't recall what it is offhand. 
Did you look in 
> the LAM/MPI User's Guide?  I seem to recall documenting
all such things 
> in there...
> 
> 
> --Jeff Squyres
> Cisco Systems
> 
> 

Yes, I know about param rpi_gm_tinymsglen. But the point is,
Recv 
depends on completion of Wait in proc-sender, whereas data
transmission 
had been finished a long time ago - there is large enough
space between 
Isend and Wait. In my opinion, the natural behavior of such

communication pattern is completion of Recv as soon as data
has arrived. 
But in practice, in case of large buffer, receiver can be
paused for 
arbitrary period.

WBR, Vartan Padaryan.
_______________________________________________
This list is archived at http://www.l
am-mpi.org/MailArchives/lam/