freenode/#lisp - IRC Chatlog
Search
9:25:50
LdBeth
If a language is very terse (TECO), where almost every token is one character, is there any possibility benefiting from using a parser generator?
9:45:43
beach
Then it has nothing to do with the tokens themselves, but with the fact that the grammar is very simple too.
9:49:37
beach
Implementing a typical language often consists of defining two separate things, namely a lexer and a parser. The lexer turns sequences of characters into tokens, and the parser turns the sequence of tokens into a parse tree (usually). If the tokens are simple, the lexer is simple. If the grammar for combining tokens into parse trees is simple, then the parser is simple. The two are orthogonal.
9:55:18
aeth
I wonder where optimizations fit. These are especially key when you have a minimal command language where you're supposed to compose things. One case would be +++++ which in a hypothetical simple language taken literally would mean (progn (incf x 1) (incf x 1) (incf x 1) (incf x 1) (incf x 1)) but can be optimized to (incf x 5)
9:55:34
aeth
It seems like this kind of thing could be done at either the lexer or the parser, and would complicate even a seemingly simple language.
9:57:57
beach
The role of the lexer and that of the parser are often blurred. In most real languages, a lexer is in fact not possible as a first step. It has to know the context as defined by the grammar. Also, there are parsing techniques that do not need a lexer at all.
10:29:58
White_Flame
aeth: generally the parser does not optimize. Once you have the parse tree or AST or whatever, then you perform that sort of analysis in the compiler
11:32:18
White_Flame
I guess that depends on if that concatenation is a lexical construct, a parser construct, or neither
11:33:10
White_Flame
obviously your language definition could demand that at any level, or leave it optional for the compiler to optimize. It would be fairly transparent to the actual source code, but not to implementers
12:29:48
ck_
it reminded me to see whether the source is available, or somebody ported it to today's CL
18:28:34
pfdietz_
For radix > 10, is there a way to get the printer to print the alphabetic digits in lower case?
18:34:34
ck_
does your lisp implementation offer anything, pfdietz_? Allegro CL has something called Case Mode, but I don't have one and can't check whether it applies to numbers
18:34:35
Bike
i don't think there's a flag you can use, if you're worried about doing this in the middle of some structure
18:36:37
sjl
if you're pretty printing you might be able to hack something together with http://clhs.lisp.se/Body/v_pr_ppr.htm