Question

grammar Test;

IDHEAD: ( a .. z  |  A .. Z  |  _ );
IDTAIL: (IDHEAD |  0 .. 9 );
ID:     (IDHEAD IDTAIL*);
fragment
TYPE:   ( text  |  number  |  bool );

define:  define  ID  as  TYPE;

The problem is that the define rule matches the tokens define, ID, as, but wont match TYPE. I m yielding a MissingTokenException.

If I inline the TYPE, as follows, it works as I m intending:

grammar Test;

IDHEAD: ( a .. z  |  A .. Z  |  _ );
IDTAIL: (IDHEAD |  0 .. 9 );
ID:     (IDHEAD IDTAIL*);
fragment
TYPE:   ( text  |  number  |  bool );

define:  define  ID  as  ( text  |  number  |  bool );

Update: The fragment keyword was added in an effort to resolve another conflict: The following token definitions can never be matched because prior tokens match the same input: TYPE.

Answer 1

In the combined grammar, place your parser rules above your lexer rules. Also, remember that the lexer runs first and only after it s finished the parser runs. The TYPE (lexer) token must be matched before it s known that the define (parser) rule needs it.

Fragment lexer rules do not create tokens, but they can be composed into non-fragment rules that do create tokens. In your example, IDHEAD and IDTAIL are not tokens - they are just used to describe the parts of ID. As such, TYPE and ID are your non-fragment rules, and IDHEAD and IDTAIL are fragment rules.

grammar Test;

define:  define  ID  as  TYPE;

/*
 * Lexer rules only below here
 */

TYPE:   ( text  |  number  |  bool );
ID:     (IDHEAD IDTAIL*);

fragment
IDHEAD: ( a .. z  |  A .. Z  |  _ );

fragment
IDTAIL: (IDHEAD |  0 .. 9 );

Answer 2

I thought the lexer rules have a priority of how they are listed. So if you want token TYPE to actually be created, move it above all the other lexer rules.

grammar Test;

fragment
TYPE:   ( text  |  number  |  bool );
IDHEAD: ( a .. z  |  A .. Z  |  _ );
IDTAIL: (IDHEAD |  0 .. 9 );
ID:     (IDHEAD IDTAIL*);

define:  define  ID  as  TYPE;

Answer 3

Isn t it because TYPE is defined as fragment?

Can t test right now, but try and remove fragment, and that should do the trick, plus give you a token to boot.

Answer 4

What is the line with fragment supposed to be doing? I think it should work as you expect if you remove it.

友情链接