I've ended up with something like:
header = some(lambda tok: tok.type == "HEADER")
data = some(lambda tok: tok.type == "DATA")
empty_line = some(lambda tok: tok.type == "EMPTY")
body = many(data | empty_line)
segment = header + body
segments = segment + many(skip(empty_line) + segment)
This ends up with an unexpected token error for the second HEADER token with a token stream like HEADER BODY EMPTY HEADER BODY as the EMPTY gets consumed by body and hence it cannot be consumed by segments.
In Haskell I'd solve this with something like body = data <|> (try ( do { empty_line ; notFollowedBy header } )). As far as I can tell, there's nothing comparable to try or notFollowedBy. Is there any sensible way to define such a grammar?
I've ended up with something like:
This ends up with an unexpected token error for the second HEADER token with a token stream like
HEADER BODY EMPTY HEADER BODYas the EMPTY gets consumed bybodyand hence it cannot be consumed bysegments.In Haskell I'd solve this with something like
body = data <|> (try ( do { empty_line ; notFollowedBy header } )). As far as I can tell, there's nothing comparable totryornotFollowedBy. Is there any sensible way to define such a grammar?