add shmat() and shmdt() to IPC::SysV

On Sep 24, 2007, at 1:25 PM, Marcus Holland-Moritz wrote:
> Graham, if you're not interested in maintaining the
CPAN version of
> IPC::SysV, I can offer you to take over that module.

Marcus,

If you want to take over IPC::SysV tthen I can transfer it
to you on  
PAUSE. What is your CPAN id ?

the CPAN distribution has not bee updated in nearly 10
years. I  
suspect the version distributed with Perl has had many
updates so I  
would start with that.

Graham.

>
> However, it would be cool to get *any* reply this
time.
>
> Thanks,
> Marcus
>
>
> Begin forwarded message:
>
> Date: Wed, 13 Apr 2005 22:18:26 +0200
> From: Marcus Holland-Moritz <mhx-perlgmx.net>
> To: gbarrpobox.com
> Subject: Fw: [PATCH] add shmat() and shmdt() to
IPC::SysV
>
>
> Hi Graham,
>
> last month I sent the following message concerning
IPC::SysV to
> you as the maintainer and p5p, but never got a reply.
>
> It would be kind if you could have a brief look.
>
> Thanks,
> Marcus
>
>
> Begin forwarded message:
>
> Date: Mon, 14 Mar 2005 21:22:01 +0100
> From: Marcus Holland-Moritz <mhx-perlgmx.net>
> To: p5p <perl5-portersperl.org>, gbarrcpan.org
> Subject: [PATCH] add shmat() and shmdt() to IPC::SysV
>
>
> The attached patch adds shmat() and shmdt() support to
IPC::SysV.
>
> Currently, shared memory can only be handled with
shm(get|ctl|read| 
> write).
>
> The problem is that shared memory created by shmget is
persistent,  
> even
> after the creating process terminates. The process can
and should  
> of course
> delete the shared memory using shmctl. But if it's
killed by an  
> untrappable
> signal we have a shared memory leak.
>
> AFAIK, the only way to avoid this is to attach to the
shared memory  
> with
> shmat() directly after creating it and to delete it
immediately.  
> The shared
> memory is then marked as 'destroyed', but as long as at
least one  
> process
> is still attached, all shared memory operations can
still be  
> carried out.
> When the process terminates, the shared memory is
automatically  
> detached
> and if the attach count drops to zero the shared memory
is deleted.
>
> If this patch turns out to be good, the new version of
IPC::SysV  
> should
> also go on CPAN. The current CPAN version is already
out of date with
> regard to bleadperl.
>
> Marcus
> <ipcsysv.diff>

Hi Graham,

thanks for the reply!

On 2007-09-24, at 19:20:16 -0500, Graham Barr wrote:

> On Sep 24, 2007, at 1:25 PM, Marcus Holland-Moritz
wrote:
> > Graham, if you're not interested in maintaining
the CPAN version of
> > IPC::SysV, I can offer you to take over that
module.
> 
> Marcus,
> 
> If you want to take over IPC::SysV tthen I can transfer
it to you on  
> PAUSE. What is your CPAN id ?

That would be fine with me. The id is "MHX".

> the CPAN distribution has not bee updated in nearly 10
years. I  
> suspect the version distributed with Perl has had many
updates so I  
> would start with that.

Sure, and I'll also need to have a look at the usual
backwards
compat stuff.

Thanks,
Marcus

-- 
There are two ways to write error-free programs; only the
third one works.

On 24/09/2007, Marcus Holland-Moritz said:
> The attached patch adds shmat() and shmdt() support to
IPC::SysV.
[...]
> If this patch turns out to be good, the new version of
IPC::SysV should
> also go on CPAN. The current CPAN version is already
out of date with
> regard to bleadperl.

I welcome this addition, and the backporting of this module
to the
CPAN. However, given that this is new functionality, heavily
system
dependent, maybe shouldn't it go right away in the core ? I
suspect
possible smoke breakages, and the trunk is getting more
stable for
5.10 every day 

On 2007-09-25, at 16:01:05 +0200, Rafael Garcia-Suarez
wrote:

> On 24/09/2007, Marcus Holland-Moritz said:
> > The attached patch adds shmat() and shmdt()
support to IPC::SysV.
> [...]
> > If this patch turns out to be good, the new
version of IPC::SysV should
> > also go on CPAN. The current CPAN version is
already out of date with
> > regard to bleadperl.
> 
> I welcome this addition, and the backporting of this
module to the
> CPAN. However, given that this is new functionality,
heavily system
> dependent, maybe shouldn't it go right away in the core
?

Right. As I said earlier, I was primarily interested in a
CPAN
release. And as it seems, I'll be responsible for that
now... 

> I suspect possible smoke breakages, and the trunk is
getting more
> stable for 5.10 every day 

Sure! 

Marcus

-- 
Any given program will expand to fill available memory.

On 2007-09-25, at 14:40:56 +0000, H.Merijn Brand wrote:

> On Tue, 25 Sep 2007 16:01:05 +0200, "Rafael
Garcia-Suarez"
> <rgarciasuarezgmail.com> wrote:
> 
> > On 24/09/2007, Marcus Holland-Moritz said:
> > > The attached patch adds shmat() and shmdt()
support to IPC::SysV.
> > [...]
> > > If this patch turns out to be good, the new
version of IPC::SysV should
> > > also go on CPAN. The current CPAN version is
already out of date with
> > > regard to bleadperl.
> > 
> > I welcome this addition, and the backporting of
this module to the
> > CPAN. However, given that this is new
functionality, heavily system
> > dependent, maybe shouldn't it go right away in the
core ? I suspect
> > possible smoke breakages, and the trunk is getting
more stable for
> > 5.10 every day 
> 
> True.
> 
> As of day 1, I missed ftok () and shmstats (), which I
added in my own
> XS module to use when I need it:

I'll have a look at if these can be easily added as well.

Marcus

-- 
consultant, n.:
	Someone who knowns 101 ways to make love, but can't get a
date.

On 2007-09-25, at 14:40:56 +0000, H.Merijn Brand wrote:

> As of day 1, I missed ftok () and shmstats (), which I
added in my own
> XS module to use when I need it:

Actually, ftok() was added to IPC::SysV in the last
millennium:

  Change 135 on 1998/05/18 by <gbarrpobox.com> (Graham Barr)
  
         applied changes from Jarkko Hietaniemi <jhiiki.fi> to add
         new constants and ftok

Marcus

-- 
At the source of every error which is blamed on the computer
you will find
at least two human errors, including the error of blaming it
on the computer.

On Tue, 25 Sep 2007 22:25:19 +0200, Marcus Holland-Moritz
<mhx-perlgmx.net>
wrote:

> On 2007-09-25, at 14:40:56 +0000, H.Merijn Brand
wrote:
> 
> > As of day 1, I missed ftok () and shmstats (),
which I added in my own
> > XS module to use when I need it:
> 
> Actually, ftok() was added to IPC::SysV in the last
millennium:
> 
>   Change 135 on 1998/05/18 by <gbarrpobox.com> (Graham Barr)
>   
>          applied changes from Jarkko Hietaniemi
<jhiiki.fi> to add
>          new constants and ftok

Ahh, yes, but as I already had my version in since October
1996, I
never switched. Reading on what I wrote then, I also note
now why
I didn't switch. With my version the second argument is
optional
and defaults to 'A' or 65. In hindsight that might be handy
for
the cases where I use(d) it, but not good at all.

I might as well switch to the IPC::SysV version ...

-- 
H.Merijn Brand         Amsterdam Perl Mongers (http://amsterdam.pm.org/
)
using & porting perl 5.6.2, 5.8.x, 5.9.x   on HP-UX
10.20, 11.00, 11.11,
& 11.23, SuSE 10.0 & 10.2, AIX 4.3 & 5.2, and
Cygwin. http://qa.perl.org
http://mirrors.de
velooper.com/hpux/            http://www.test-smoke.org
                        http
://www.goldmark.org/jeff/stupid-disclaimers/

On 2007-09-28, at 09:24:44 +0000, H.Merijn Brand wrote:

> On Tue, 25 Sep 2007 22:25:19 +0200, Marcus
Holland-Moritz <mhx-perlgmx.net>
> wrote:
> 
> > On 2007-09-25, at 14:40:56 +0000, H.Merijn Brand
wrote:
> > 
> > > As of day 1, I missed ftok () and shmstats
(), which I added in my own
> > > XS module to use when I need it:
> > 
> > Actually, ftok() was added to IPC::SysV in the
last millennium:
> 
> But, it doesn't work
> 
> %set prompt="% "
> % echo $DBPATH
> /pro/dbu/lep/0392
> % perl -MIPC::SysV -le'print ftok ($ENV,
"A")'
> % perl -MPROCURA   -le'print ftok ($ENV,
"A")'
> 1093496642
> %

Well, it does work. However, you (and your implementation

assume that passing a string as the second argument is ok.

However, ftok() takes an int:

  key_t ftok(const char *pathname, int proj_id);

That's exactly the interface also provided by the IPC::SysV
implementation. In your implementation, proj_id defaults to
65, and if you don't pass an integer argument, it uses the
value of the first byte in the stringified version of the
argument. (Which may cause surprises if you pass NVs or
multi-character PVs and get unexpected results.)

If you had run your test case above with -w, you'd have
gotten this warning:

  mhxr2d2 $ perl -MIPC::SysV=ftok -wle'print
ftok("/tmp", "A")'
  Argument "A" isn't numeric in subroutine entry
at -e line 1.

So, the second argument passed to ftok() was zero, and as
the manpage says:

  "... the least significant 8 bits of proj_id (which
must
   be non-zero) ..."

Anyway, I think from the interface side, IPC::SysV is
mostly ok. I don't like the defaulting to 65, but allowing
a character is probably useful.

Please let me know if IPC::SysV's ftok() works for you if
you pass it an integer.

Marcus

-- 
10.0 times 0.1 is hardly ever 1.0.

On 2007-09-25, at 16:01:05 +0200, Rafael Garcia-Suarez
wrote:

> However, given that this is new functionality, heavily
system
> dependent [...]

Ooooh, how true this is!

While my attempt to add shmat() and shmdt() made perfect
sense
to me and worked flawlessly on Linux, I got failures on
*BSD.

The reason is that I'm doing the following (to make sure
global shared memory is cleaned up automatically after the
last attached process exits):

  - create a shared memory segment  (shmget)
  - attach to it  (shmat)
  - remove it immediately after attaching  (shmctl
IPC_RMID)
  - access it using shmread / shmwrite

The problem is that calls to shmread/shmwrite attach and
detach the shared memory segment with each invokation.

Doing so is valid (for historical reasons, as it seems) on
Linux...

	case IPC_RMID:
	{
		/*
		 *	We cannot simply remove the file. The SVID states
		 *	that the block remains until the last person
		 *	detaches from it, then is deleted. A shmat() on
		 *	an RMID segment is legal in older Linux and if 
		 *	we change it apps break...

...but not on *BSD (the code is OpenBSD 4.1):

	struct shmid_ds *
	shm_find_segment_by_shmid(int shmid)
	{
		int segnum;
		struct shmid_ds *shmseg;
	
		segnum = IPCID_TO_IX(shmid);
		if (segnum < 0 || segnum >= shminfo.shmmni ||
		    (shmseg = shmsegs[segnum]) == NULL ||
		    shmseg->shm_perm.seq != IPCID_TO_SEQ(shmid))
			return (NULL);
		if ((shmseg->shm_perm.mode &
(SHMSEG_REMOVED|SHMSEG_RMLINGER)) == SHMSEG_REMOVED)
			return (NULL);
		return (shmseg);
	}
        [...]
	case IPC_RMID:
		if ((error = ipcperm(cred, &shmseg->shm_perm,
IPC_M)) != 0)
			return (error);
		shmseg->shm_perm.key = IPC_PRIVATE;
		shmseg->shm_perm.mode |= SHMSEG_REMOVED;

Since I still like the idea, the only backwards-compatible
solution
that comes to my mind is to add two functions to IPC::SysV
that allow
reading and writing of memory, with an interface similar to
shmread
and shmwrite, but with an address instead of a shmid. Then
all processes
could attach before the shared memory segment is marked for
removal and
could still access the shared memory.

shmat would return an address in the same format as
pack('P'), and
shmdt and the new functions (called 'memread' and
'memwrite') would
take such an address. This will allow to also read the
memory using
unpack("P") and hopefully make it a little heavier
to misuse these
functions.

Here's a little example:

    use IPC::SysV qw(shmat shmdt memread memwrite);
    $shm = shmget(IPC_PRIVATE, 4, S_IRWXU);
    $addr = shmat($shm, 0, 0);
    shmctl($shm, IPC_RMID, 0);
    memwrite($addr, pack("N", 0xbadc0de5), 0, 4);
    memread($addr, $bytes, 0, 4);
    $var = unpack "N", $bytes;
    $var = unpack "N", unpack "P4",
$addr;
    shmdt($addr);

I'd like to receive feedback if anyone thinks that this is a
really
bad idea. Otherwise it will most probably go into the CPAN
release.

Marcus

-- 
God is real, unless declared integer.