LAM: using ssi rpi parameter 'rpi_tcp_sockbuf'

hello,

I have a question about changing tcp socket sizes 
with 'rpi_tcp_sockbuf'
at run time, the user guide says that if it is not the
default value
of '-1' then I can set the socket parameter to 0 instead
with: 

$ mpirun -ssi rpi_tcp_sockbuf 0 N <executable name> 

Note that in my program I set the send and receive socket
buffers
with the setsockopt(2) function as follows setting size to
131072.
 
          int val = 131072;
     sock = socket( PF_INET, SOCK_STREAM, 0 );
            ierr_setsock = setsockopt(sock,0, SO_SNDBUF,
&val, 
131072) ;
            ierr_setsock2 = setsockopt(sock,0, SO_RCVBUF,
&val, 
131072) ;


Is this correct?  I am not sure it works as my performance
figures
dont indicate any change so perhaps they are not being
resized at
all.  Any suggestions regarding how to do this would be
great.

Also - what is the rpi_tcp_priority for? Can it be changed? 

Is the shared memory parameter used with tcp collective
communications
across several nodes?  If so would it be useful to tune it?

thanks.
_______________________________________________
This list is archived at http://www.l
am-mpi.org/MailArchives/lam/

On Sat, 30 Jun 2007, Maureen O Flynn wrote:

> I have a question about changing tcp socket sizes with

> 'rpi_tcp_sockbuf' at run time
> ...
> Note that in my program I set the send and receive
socket buffers
> with the setsockopt(2) function as follows setting size
to 131072.

After reading the above 2 phrases, it's totally unclear to
me what you 
are trying to do. Are you modifying the LAM/MPI library or
deamons ? 
Or you are just running a MPI program ?

If it's the latter, it's very uncommon what you are trying
to do, so 
maybe more details will help us understand better. In
principle, a MPI 
program should just use the LAM/MPI library as is, by
calling the MPI 
functions, independent of the underlying message passing
mechanism. By 
creating sockets yourself you are actually using a second
data 
transmission mechanism between the processes that belong to
your job, 
independent of the LAM/MPI one; in this case, if you are not
calling 
the MPI functions, there is no involvement of the MPI
library in your 
communication.

-- 
Bogdan Costescu

IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches
Rechnen
Universitaet Heidelberg, INF 368, D-69120 Heidelberg,
GERMANY
Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
E-mail: Bogdan.CostescuIWR.Uni-Heidelberg.De
_______________________________________________
This list is archived at http://www.l
am-mpi.org/MailArchives/lam/

On Jun 30, 2007, at 9:13 PM, Maureen O Flynn wrote:

> I have a question about changing tcp socket sizes
> with 'rpi_tcp_sockbuf'
> at run time, the user guide says that if it is not the
default value
> of '-1' then I can set the socket parameter to 0
instead with:
>
> $ mpirun -ssi rpi_tcp_sockbuf 0 N <executable
name>
>
> Note that in my program I set the send and receive
socket buffers
> with the setsockopt(2) function as follows setting size
to 131072.
>
>           int val = 131072;
>      sock = socket( PF_INET, SOCK_STREAM, 0 );
>             ierr_setsock = setsockopt(sock,0,
SO_SNDBUF, &val,
> 131072) ;
>             ierr_setsock2 = setsockopt(sock,0,
SO_RCVBUF, &val,
> 131072) ;

Is there any reason you don't want to set the
rpi_tcp_sockbuf  
parameters to 131072 instead of setting them yourself? 
These values  
should be set for you during MPI_INIT.

Taking a quick peek at the code in LAM (it's been years
since I've  
looked at this -- I *think* my analysis is correct), it
looks like we  
unconditionally set the parameter regardless of its value.

You should be able to getsockopt() to see what values LAM
actually set.

> Is this correct?  I am not sure it works as my
performance figures
> dont indicate any change so perhaps they are not being
resized at
> all.  Any suggestions regarding how to do this would be
great.

I know that there's some funniness about the TCP socket
buffer  
parameters -- I have a dim recollection that there's a max
enforced  
by the kernel, etc.  You might want to google around about
this.

> Also - what is the rpi_tcp_priority for? Can it be
changed?
> Is the shared memory parameter used with tcp collective
communications
> across several nodes?  If so would it be useful to tune
it?

It's used for choosing which RPI plugin to use.  The
shmem+tcp  
modules should all have higher priority than the plain tcp
parameter,  
so they should automatically get chosen.

However, the code in the TCP RPI is always used (even if you
use one  
of the two shmem+tcp modules), so if you make changes there,
they  
will always apply.

-- 
Jeff Squyres
Cisco Systems

_______________________________________________
This list is archived at http://www.l
am-mpi.org/MailArchives/lam/