dtrace: failed to grab pid 16537: process is traced

I'm getting an odd behavior while trying to trace a mysql
process using 
the -p flag on a Sun Cluster.

This fails

# dtrace -n 'pid$target:mysql_parse*
:entry {}' -p `pgrep -x mysqld`
dtrace: failed to grab pid 16537: process is traced

Yet this works
# dtrace -n 'pid$1:mysql_parse*
:entry {}' `pgrep -x mysqld`
dtrace: description 'pid$1:mysql_parse*
:entry ' matched 1 probe

All processes managed within the Sun Cluster agent framework
appear to 
exhibit this behavior yet if they aren't managed by cluster
they appear 
to work.  Has anyone seen this problem? Also any pointers to
investigate 
this further?

# uname -a
SunOS wolverine 5.10 Generic_118833-36 sun4u sparc
SUNW,Sun-Fire


Thanks

Dave
_______________________________________________
dtrace-discuss mailing list
dtrace-discussopensolaris.org

On Fri, Mar 21, 2008 at 01:01:14PM -0500, David Blasingame,
Area Systems Support Engineer wrote:
> I'm getting an odd behavior while trying to trace a
mysql process using 
> the -p flag on a Sun Cluster.
> 
> This fails
> 
> # dtrace -n 'pid$target:mysql_parse*
:entry {}' -p `pgrep -x mysqld`
> dtrace: failed to grab pid 16537: process is traced
> 
> Yet this works
> # dtrace -n 'pid$1:mysql_parse*
:entry {}' `pgrep -x mysqld`
> dtrace: description 'pid$1:mysql_parse*
:entry ' matched 1 probe
> 
> All processes managed within the Sun Cluster agent
framework appear to 
> exhibit this behavior yet if they aren't managed by
cluster they appear 
> to work.  Has anyone seen this problem? Also any
pointers to investigate 
> this further?
> 
> # uname -a
> SunOS wolverine 5.10 Generic_118833-36 sun4u sparc
SUNW,Sun-Fire
> 
> 
> Thanks
> 
> Dave

Sun Cluster has a long-standing design limitation/flaw that
it uses /proc
to trace processes to effect their restart.  This interferes
with the
ability of other debuggers like dbx, DTrace, and proc(1)
tools to work on them.
It is for this reason that when we invented SMF in Solaris
10, we implemented
a proper kernel mechanism for managing process restart,
called contracts,
independent of /proc, to handle notification events etc. for
restart.
Sun Cluster needs to be converted to use it.  Hopefully
someone from
that team can comment on the software release schedule for
that change.

-Mike

-- 
Mike Shapiro, Sun Microsystems Fishworks.
blogs.sun.com/mws/
_______________________________________________
dtrace-discuss mailing list
dtrace-discussopensolaris.org