Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple?

On Thursday, July 14, 2022 at 8:10:56 AM UTC-7, Roger L Costello wrote:


> A common example in books on Lex/Flex and Yacc/Bison is evaluating arithmetic
> expressions. When the lexer encounters an integer lexeme, it casts the lexeme
> to a binary integer and returns the value to the parser. The lexer contains a
> rule that looks something like this:


> {INTEGER} { yylval.intval = atoi(yytext); return NUMBER; }


A common example is an RPN or algebraic calculator.
(Which evaluates numerical expressions.)


It is a nice simple example, compared to a complete programming language.


For algebraic, you get into precedence and such, so real parsing problems,
but in a very simple, and easy to follow, case.


The other things, is that it isn't so easy to return character strings in C.
You would malloc() it and copy the value over, and then hope that
somewhere later it is free()d. (I suspect that is done more in Java.)


In the STEP processor that I previously wrote about, the only return
type is character string, and the processor keeps track of allocation.
Macros that do arithmetic convert to numerical type, evaluate an
expression, and convert back to characters.


The code for the processor itself, as usual for compilers, is written
using itself, has a macro to evaluate a constant integer
expression. That is useful, as Fortran 66 (and Fortran 77)
don't do that.

Return to the comp.compilers page.
Search the comp.compilers archives again.