Python/MI/STL visualization

Hi,
I've just implement the logic for computing the children of
MI varobj
using Python scripts. Using the attached .gdbinit, I can do
the following:

  -var-create V * v
 
^done,name="V",numchild="1",value="
{...}",type="std::vector<int,std::allocator<
int> >"
  (gdb)
  -var-set-visualizer V VectorVisualizer
  ^done
  (gdb)
  -var-list-children --all-values V
  ^done,numchild="2",children=[
 
child={name="V.0",exp="0",numchild="
;0",value="1",type="int"},
 
child={name="V.1",exp="1",numchild="
;0",value="2",type="int"}]

Previously, we discussed how to best report the case where
the number of children of
varobj changes. The approach I've implemented is for
-var-update to report the varobj
that had the number of children changed, and include the new
'children' attribute
for that varobj. So, if I had a vector of 1 element and push
another other, I get
this:

  -var-update V
 
^done,changelist=[{name="V",in_scope="true&qu
ot;,type_changed="false",
    
children=[{name="V.0",exp="0",numchild=&
quot;0",type="int"},
              
{name="V.1",exp="1",numchild="0&quo
t;,type="int"}]}]

I like this approach because it does not assume that
children are added or removed
at back -- if the list of children change, we report the
entire new list, and can
put new varobj in the middle.

On Python level, visualization is handled by a Python class
instance -- one instance
per varobj. This approach, as opposed to a function, allows
Python code to do
whatever caching it sees fit.

Of course, there are quite some issues and questions:

1. Vectors can get large, and getting them can get slow. Do
we want to have
incremental fetch of some kind? On UI level, I'm thinking of
something
like KDevelop's incremental fetch of frames, see 
http://vladimir_prus.blogspot.com/20
07/02/debugger-stories-stack-widget.html
But we also need MI level support.

2. Presumably, it's better to automatically assign
visualizers to varobjs
of specific types. What's the best way to specify 'all
vectors'. Does
using regexps seem good enough?

3. One can have vector of vectors. However, present code
requires the visualizers
be explicitly set for each element of the outer vector.
Should be have some way to
set visualizers on all future children of a given varobj?
I'm not quite sure
how std::map will be presented, but probably we will have
children of different
types. Then, should we have a way to set visualizers on all
future children of 
specific type.

4. We still have the problem that GCC debug information will
say that variable
exists even before the constructor for that variable has
run. So, creating 
robust visualizer is rather hard. Anybody knows if we can
workaround this?
I think even comparing declaration line with the current
line is better than
nothing.

- Volodya

On Tuesday 22 April 2008 22:06:54 Vladimir Prus wrote:
> Hi,

Hi Vladimir.

> I've just implement the logic for computing the
children of MI varobj
> using Python scripts. Using the attached .gdbinit, I
can do the following:
> 
>   -var-create V * v
>  
^done,name="V",numchild="1",value="
{...}",type="std::vector<int,std::allocator<
int> >"
>   (gdb)
>   -var-set-visualizer V VectorVisualizer
>   ^done
>   (gdb)
>   -var-list-children --all-values V
>   ^done,numchild="2",children=[
>  
child={name="V.0",exp="0",numchild="
;0",value="1",type="int"},
>  
child={name="V.1",exp="1",numchild="
;0",value="2",type="int"}]

Nice.

[Btw, one thing I do when dumping large MI-style data is to


  
^done,numchild="2",childtyp="int",childn
umchild="0",children=[
  
child={name="V.0",exp="0",value="1&
quot;},
  
child={name="V.1",exp="1",value="2&
quot;}]

Not exactly good syntax but reduces communication effort
which
shows when dumping large amount of data over a slow
connection
especially if the 'childtype' is something lengthy . I'd
argue it makes
even the output better readable for human inspectors.]

> Previously, we discussed how to best report the case
where the number of children of
> varobj changes. The approach I've implemented is for
-var-update to report the varobj
> that had the number of children changed, and include
the new 'children' attribute
> for that varobj. So, if I had a vector of 1 element and
push another other, I get
> this:
> 
>   -var-update V
>  
^done,changelist=[{name="V",in_scope="true&qu
ot;,type_changed="false",
>     
children=[{name="V.0",exp="0",numchild=&
quot;0",type="int"},
>               
{name="V.1",exp="1",numchild="0&quo
t;,type="int"}]}]
>
> I like this approach because it does not assume that
children are added or removed
> at back -- if the list of children change, we report
the entire new list, and can
> put new varobj in the middle.
> 
> On Python level, visualization is handled by a Python
class instance -- one instance
> per varobj. This approach, as opposed to a function,
allows Python code to do
> whatever caching it sees fit.
> 
> Of course, there are quite some issues and questions:
> 
> 1. Vectors can get large, and getting them can get
slow. Do we want to have
> incremental fetch of some kind? On UI level, I'm
thinking of something
> like KDevelop's incremental fetch of frames, see 
> http://vladimir_prus.blogspot.com/20
07/02/debugger-stories-stack-widget.html
> But we also need MI level support.

Specifying a 'child range' might do the trick. Maybe

  -var-set-child-range V <from> <to>
  -var-set-child-range V *      /* reset */

or such, and  whenever data is reported, only children in
the range
 <from>...<to> are reported. 

> 2. Presumably, it's better to automatically assign
visualizers to varobjs
> of specific types. What's the best way to specify 'all
vectors'. Does
> using regexps seem good enough?

I would think so. It is nice when one can distinguish three
cases
(simple type, pointers, "the rest"), and this
should be possible with
regexps.
 
> 3. One can have vector of vectors. However, present
code requires the visualizers 
> be explicitly set for each element of the outer vector.
Should be have some way to 
> set visualizers on all future children of a given
varobj? I'm not quite sure
> how std::map will be presented, but probably we will
have children of different
> types. Then, should we have a way to set visualizers on
all future children of 
> specific type.

I think visualizers are best set _by type_, not by var
object. It's hard to
imagine a scenario where this is not wanted.

> 4. We still have the problem that GCC debug information
will say that variable
> exists even before the constructor for that variable
has run. So, creating 
> robust visualizer is rather hard. Anybody knows if we
can workaround this?

Well, I run "visualizer code" injected into the
inferior and trigger it with
gdb's "call". I do not even try to make them
robust after noticing that 
there is not enough information available. But simply
letting them crash
and handling the segfault works rather nicely as far as I
can tell 

> I think even comparing declaration line with the
current line is better than
> nothing.

"Nothing" might be ok 

Andre'

On Wednesday 23 April 2008 09:46:07 Vladimir Prus wrote:
> >> 1. Vectors can get large, and getting them can
get slow. Do we want to have
> >> incremental fetch of some kind? On UI level,
I'm thinking of something
> >> like KDevelop's incremental fetch of frames,
see
> >> http://vladimir_prus.blogspot.com/20
07/02/debugger-stories-stack-widget.html
> >> But we also need MI level support.
> > 
> > Specifying a 'child range' might do the trick.
Maybe
> > 
> >   -var-set-child-range V <from> <to>
> >   -var-set-child-range V *      /* reset */
> > 
> > or such, and  whenever data is reported, only
children in the range
> >  <from>...<to> are reported.
> 
> Or adding range parameter to -var-list-children. In
either case, I'm not yet sure 
> how to make Python interface support this. Of course,
we can always make python
> side use iterators, which will work for the case where
we fetch more children at
> the end.

Or that 

>  [...]
> >> 4. We still have the problem that GCC debug
information will say that variable
> >> exists even before the constructor for that
variable has run. So, creating
> >> robust visualizer is rather hard. Anybody
knows if we can workaround this?
> > 
> > Well, I run "visualizer code" injected
into the inferior and trigger it with
> > gdb's "call". I do not even try to make
them robust after noticing that
> > there is not enough information available. But
simply letting them crash
> > and handling the segfault works rather nicely as
far as I can tell 
> 
> Well, segfaul is not scary. What is scary is if due to
uninitialized data,
> the amount of the children the visualizer wants to
fetch end up been 10000 

Indeed.

My "solution" here is to wildly poke around in the
structure hoping to 
trigger a segfault early and interpret such segfault as
"uninitialized object".

So if there's a std::vector<> with size() reporting a
non-0 value, try
to access the first and last element and one from the middle
and 
hope that at least one of these addresses are
"bad" 

In practice, this works rather nicely. The bigger the
reported size()
gets the more potential harm would actually listing the
children do,
but the more likely it also is to recognize the object as
uninitialized.

Of course, not having to play such "games" would
be much nicer ;-}

Andre'