List Info

Thread: interesting push parser use case




interesting push parser use case
user name
2007-09-08 17:05:42
Hi,

I found an interesting issue when converting my program from
the pull
parser to the push parser. I used to generate a parser, and
then write a
thin "main" around the parser. Then I would feed
the program a
particular file and let it parse it.

The file contained a list of gdb/mi output commands. The
grammar is
capable of parsing one or more gdb/mi output commands.

The grammar would look like,

  %start opt_output_list
  %%

  opt_output_list: {
    tree = NULL;
  };

  opt_output_list: output_list {
    tree = $1;
  };

  output_list: output {
    $$ = $1;
  };

  output_list: output_list output {
    $$ = append_gdbmi_output ($1, $2);
  };

which would allow me to parse a list of commands. The
"output" rule is
the rule that parses a single gdb/mi output command. So, as
you can see,
the "output_list" rule would allow you to parse 1
or more gdb/mi
commands.

Here is the issue, when I convert this to a push parser, the
parser
_always_ thinks that it can parse more data. It never thinks
it is
finished. Since it never realizes it's finished, it never
gets from 
the top "output_list" rule to the
"opt_output_list" rule. This causes my
global variable "tree" to never get the result of
the parse tree.

The first issue is that I probably need a way to tell the
parser that
I'm done giving it tokens. That way, it will finish all of
it's rules.
Is there already a way to do this?

The second issue is slightly more fuzzy. Essentially, after
each token I
give to the parser, it would probably be useful to know if
it just
finished a particular rule, and if so, which rule. That way,
the
application would know when it would want to tell the push
parser that
it is all done giving it tokens. Basically, I'd want to tell
the parser
that it's done after I've finished an
"output_list" rule.

It is possible that I can workaround these issues by
changing the
grammar or moving where I do the assignment to the 'tree'
pointer.
However, these issues will probably come up again from
someone else.

Any ideas?

Thanks,
Bob Rossi


_______________________________________________
help-bisongnu.org http
://lists.gnu.org/mailman/listinfo/help-bison

Re: interesting push parser use case
country flaguser name
United States
2007-09-08 18:05:54
On Sat, 8 Sep 2007, Bob Rossi wrote:

> The first issue is that I probably need a way to tell
the parser that
> I'm done giving it tokens. That way, it will finish all
of it's rules.
> Is there already a way to do this?

Yacc-generated parsers expect the last token in the input
stream to be 
token 0.  In the .output file, it's called $end.

> The second issue is slightly more fuzzy. Essentially,
after each token I
> give to the parser, it would probably be useful to know
if it just
> finished a particular rule, and if so, which rule.

Could you use %parse-param to pass a pointer to a variable
to be set by 
the semantic action of that rule?



_______________________________________________
help-bisongnu.org http
://lists.gnu.org/mailman/listinfo/help-bison

Re: interesting push parser use case
country flaguser name
United States
2007-09-08 18:23:32
On Sat, 8 Sep 2007, Joel E. Denny wrote:

> On Sat, 8 Sep 2007, Bob Rossi wrote:
> 
> > The first issue is that I probably need a way to
tell the parser that
> > I'm done giving it tokens. That way, it will
finish all of it's rules.
> > Is there already a way to do this?
> 
> Yacc-generated parsers expect the last token in the
input stream to be 
> token 0.  In the .output file, it's called $end.

I should be more careful.  A negative token number should be
equivalent to 
zero.  $end is the usual representation, but I don't see
$end in the Open 
Group specification of Yacc.

> > The second issue is slightly more fuzzy.
Essentially, after each token I
> > give to the parser, it would probably be useful to
know if it just
> > finished a particular rule, and if so, which
rule.
> 
> Could you use %parse-param to pass a pointer to a
variable to be set by 
> the semantic action of that rule?


_______________________________________________
help-bisongnu.org http
://lists.gnu.org/mailman/listinfo/help-bison

Re: interesting push parser use case
user name
2008-02-20 09:54:49
ON SUN, SEP 09, 2007 AT 11:03:13AM +0200, LFINSTO1GWDG.DE
WROTE:
> > ON SAT, 8 SEP 2007, BOB ROSSI WROTE:
> >> THE SECOND ISSUE IS SLIGHTLY MORE FUZZY.
ESSENTIALLY, AFTER EACH TOKEN I
> >> GIVE TO THE PARSER, IT WOULD PROBABLY BE
USEFUL TO KNOW IF IT JUST
> >> FINISHED A PARTICULAR RULE, AND IF SO, WHICH
RULE.
> >
> > COULD YOU USE %PARSE-PARAM TO PASS A POINTER TO A
VARIABLE TO BE SET BY
> > THE SEMANTIC ACTION OF THAT RULE?
> >
> 
> YES.  PERSONALLY, I THINK THIS WOULD BE OVERKILL IN
MOST SITUATIONS, BUT
> IF IT'S USEFUL TO YOU I DON'T SEE ANY PROBLEM WITH IT. 
I DO RECOMMEND
> USING A POINTER TO AN OBJECT OF A `STRUCT' OR `CLASS'
TYPE, SO YOU COULD
> USE THE OBJECT PASSED AS A PARAMETER TO `YYPARSE' (AND
`YYLEX') FOR OTHER
> PURPOSES, TOO.
> 
> I WOULD ALSO WRAP THE CODE FOR SETTING IT IN #IFDEF
WHATEVER ... #ENDIF,
> SO THAT IT'S CONDITIONALLY COMPILED.  THIS WAY, YOU
COULD BUILD THE
> PROGRAM WITHOUT THE OVERHEAD OF THIS FEATURE, IF YOU
DON'T ALWAYS NEED IT.

HI LARUENCE,

SORRY FOR THE EXTREMELY LONG DELAY. I'M TRYING OUT YOUR
SUGGESTION NOW.
WHENVER I PUT A STRUCT IN THE PARSE PARAM LIKE SO,
  %PARSE-PARAM { STRUCT GDBMI_PDATA *GDBMI_PDATA }
I GET A COMPILER WARNING, 
../../../../CGDB/LIB/GDBMI/SRC/GDBMI_GRAMMAR.H:127: WARNING:
€˜STRUCT
GDBMI_PDATA€™ DECLARED INSIDE PARAMETER LIST

DO YOU KNOW HOW TO AVOID THAT? IT SEEMS LIKE THE HEADER FILE
GENERATED
HAS THE STRUCT IN THE PARAMETER LIST BUT THE DEFINITION IS
NOT IN GLOBAL
SCOPE.

THE MORE I THINK ABOUT IT, THIS MIGHT HAVE BEEN INTRODUCED
WITH THE PUSH
PARSER, SINCE WE PUT THE DECLARATIONS IN THE .H FILE NOW.

DOES ANYONE HAVE ANY SUGGESTIONS ON HOW TO FIX THIS? ONE
OBVIOUS
SOLUTION WOULD BE TO HAVE A SECTION IN THE BISON GRAMMAR
FILE THAT WOULD
GO INTO THE HEADER FILE, TO ALLOW THE DEFINITION OR
PROTOTYPE OF CERTAIN
TYPES.

BOB ROSSI



_______________________________________________
HELP-BISONGNU.ORG
HTTP://LISTS.GNU.ORG/MAILMAN/LISTINFO/HELP-BISON

Re: interesting push parser use case
country flaguser name
United States
2008-02-20 12:25:36
On Wed, 20 Feb 2008, Bob Rossi wrote:

> Sorry for the extremely long delay. I'm trying out your
suggestion now.
> Whenver I put a struct in the parse param like so,
>   %parse-param { struct gdbmi_pdata *gdbmi_pdata }
> I get a compiler warning, 
> ../../../../cgdb/lib/gdbmi/src/gdbmi_grammar.h:127:
warning: ‘struct
> gdbmi_pdata’ declared inside parameter list
> 
> Do you know how to avoid that? It seems like the header
file generated
> has the struct in the parameter list but the definition
is not in global
> scope.
> 
> The more I think about it, this might have been
introduced with the push
> parser, since we put the declarations in the .h file
now.
> 
> Does anyone have any suggestions on how to fix this?
One obvious
> solution would be to have a section in the bison
grammar file that would
> go into the header file, to allow the definition or
prototype of certain
> types.

You want something like:

  %code requires { struct gdbmi_pdata; }
_______________________________________________
help-bisongnu.org http
://lists.gnu.org/mailman/listinfo/help-bison
Re: interesting push parser use case
country flaguser name
Germany
2008-02-21 03:09:37
On Wed, 20 Feb 2008, Bob Rossi wrote:

> Hi Laruence,
> 
> Sorry for the extremely long delay. 

No need to apologize.

> I'm trying out your suggestion now.
> Whenver I put a struct in the parse param like so,
>   %parse-param { struct gdbmi_pdata *gdbmi_pdata }
> I get a compiler warning, 
> ../../../../cgdb/lib/gdbmi/src/gdbmi_grammar.h:127:
warning: ‘struct
> gdbmi_pdata’ declared inside parameter list

I don't have a `%parse-param' declaration in my grammar.  I
just pass 
the argument to `yyparse' by means of a `void*' and cast it
to the 
correct type, where needed.  It isn't always needed.  

I don't know when the `%parse-param' feature was introduced.
 It might
not have existed when I started my grammar.  Otherwise, I
probably
would have used it.

> Do you know how to avoid that? It seems like the header
file generated
> has the struct in the parameter list but the definition
is not in global
> scope.

I would expect that it would work to declare it in a header
file that's 
included by your Bison input file or in the first section
("prologue") 
of your grammar, where the header files are included. 
However, I
haven't tried this and I don't know how or when Bison
processes the
`%parse-param' declaration.  However, it's really not much
of a
nuisance to cast it.  I just do it automatically without
thinking
about it.

Laurence
_______________________________________________
help-bisongnu.org http
://lists.gnu.org/mailman/listinfo/help-bison
Re: interesting push parser use case
user name
2008-02-22 10:02:11
ON WED, FEB 20, 2008 AT 01:25:36PM -0500, JOEL E. DENNY
WROTE:
> ON WED, 20 FEB 2008, BOB ROSSI WROTE:
> 
> > SORRY FOR THE EXTREMELY LONG DELAY. I'M TRYING OUT
YOUR SUGGESTION NOW.
> > WHENVER I PUT A STRUCT IN THE PARSE PARAM LIKE
SO,
> >   %PARSE-PARAM { STRUCT GDBMI_PDATA *GDBMI_PDATA
}
> > I GET A COMPILER WARNING, 
> >
../../../../CGDB/LIB/GDBMI/SRC/GDBMI_GRAMMAR.H:127: WARNING:
€˜STRUCT
> > GDBMI_PDATA€™ DECLARED INSIDE PARAMETER LIST
> > 
> > DO YOU KNOW HOW TO AVOID THAT? IT SEEMS LIKE THE
HEADER FILE GENERATED
> > HAS THE STRUCT IN THE PARAMETER LIST BUT THE
DEFINITION IS NOT IN GLOBAL
> > SCOPE.
> > 
> > THE MORE I THINK ABOUT IT, THIS MIGHT HAVE BEEN
INTRODUCED WITH THE PUSH
> > PARSER, SINCE WE PUT THE DECLARATIONS IN THE .H
FILE NOW.
> > 
> > DOES ANYONE HAVE ANY SUGGESTIONS ON HOW TO FIX
THIS? ONE OBVIOUS
> > SOLUTION WOULD BE TO HAVE A SECTION IN THE BISON
GRAMMAR FILE THAT WOULD
> > GO INTO THE HEADER FILE, TO ALLOW THE DEFINITION
OR PROTOTYPE OF CERTAIN
> > TYPES.
> 
> YOU WANT SOMETHING LIKE:
> 
>   %CODE REQUIRES { STRUCT GDBMI_PDATA; }

THANK YOU, BY USING THE STRUCT SOLUTION, I'M NOW ABLE TO
REMOVE A GLOBAL
VARIABLE THAT I HAD IN THE PARSER. THIS WORKED BTW.

BOB ROSSI



_______________________________________________
HELP-BISONGNU.ORG
HTTP://LISTS.GNU.ORG/MAILMAN/LISTINFO/HELP-BISON

[1-7]

about | contact  Other archives ( Real Estate discussion Medical topics )