freenode/#sicl - IRC Chatlog
Search
10:17:09
beach
So a compiled file is just an external version of the AST, and we decided that the form the AST takes there is independent of the backend.
10:18:35
beach
So constants in the code have not yet been segregated into immediate values that the native code can handle and constants that must be created at "load time".
10:19:00
beach
However, once the AST is read from file, everything from then on is specific to a backend.
10:19:25
beach
So I don't see any reason to avoid doing transformations on the AST that depend on the backend, before that AST is turned into HIR.
10:20:30
beach
Since it is easier to do transformations on the AST than it is to do them on the HIR code, it seems like I should move some transformations that I currently do in HIR to the AST level.
10:22:28
beach
Perhaps even things like argument processing. All those transformations are hard in HIR.
10:23:43
heisig
Every Lisp programmer can write AST transformations, but not every Lisp programmer can write HIR transformations.
10:27:43
beach
But my main work for today is to figure out what I mean by "the compiler" in the context of SICL. It is clearly neither COMPILE-FILE, COMPILE, EVAL, nor LOAD. COMPILE-FILE is just creating an AST so it is not really part of "the compiler". I want "the compiler" to mean what is done to an AST, so that "the compiler" can be used by COMPILE, EVAL, and LOAD.
10:28:02
beach
Then I have to figure out how "the bootstrapping compiler" differs from "the native compiler".
10:28:30
beach
I know, it is just a matter of terminology, but the right terminology will determine the modular structure of the entire thing.
10:29:44
beach
Indeed. And I think that failing to define what "the compiler" means has prevented me from accomplishing some code factoring that is now urgently needed.
10:40:03
heisig
beach: Oh, here is one more thought: Although COMPILE-FILE only turns the file into an AST, it should nevertheless perform some kind of analysis to emit warnings.
10:43:55
beach
Also, thinking about this, I no longer understand why my ultimate top-level function has non-trivial constants in the static environment, whereas it has an argument for finding functions in the environment being "tied" to.
10:46:27
beach
So "the compiler" might mean something that takes an AST, and turns it into a function with no arguments that, when executed, creates the effect of "evaluating" the original AST.
10:47:09
beach
The static environment of that function contains constants and function cells from the environment being "tied" to.
10:47:50
beach
Or is the code "tied to" the environment before the function is executed? Another term to be defined precisely.
10:50:31
beach
Oh, and doing more transformations before turning the AST into HIR might make it easier to preserve source information as well.
12:56:40
froggey
I think my SICL adventure has come to an end. "No applicable methods to generic function CLUSTER::CODE-NUMBER when called with (4)" after starting phase 6
12:56:47
froggey
the same error occurs under SBCL, so I assume I've reached something that hasn't been implemented yet
13:23:33
froggey
interestingly most of the trouble was with the pretty printer, it had some exponential behaviour that lead to file complication being unreasonably slow
14:05:01
beach
If I include "relocation" information in the code object, I can define "the AST compiler" (including "AST" to avoid confusion) to mean turning an AST into a code object.
14:05:46
beach
Hmm, no that's not enough. I need the contents of the static environment of the top-level function as well.
14:12:53
beach
One more radical idea (one that has been suggested in a different context in the past). I am using a HIR interpreter during bootstrapping so as to allow for the host to execute SICL code. And I have been systematically applying HIR transformations early so that they will be tested during bootstrapping. But once this thing works, it is probably easier to interpret the AST rather than the HIR code.
14:13:25
beach
The HIR interpreter has to do some strange things with the CATCH-INSTRUCTION and the UNWIND-INSTRUCTION.
14:15:29
Bike
compile time evaluation - since the compiler is slow, if we use an interpreter instead for simple stuff compilation is faster
14:16:07
Bike
oh, but it's not complete though. because of that use case, for some of t he more complicated stuff we just give up and use the compiler
14:18:44
Bike
i don't remember what it doesn't handle... it does blocks, tagbody, closures, multiple value stuff...
14:19:26
Bike
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/cleavir/ast-interpreter.lisp here it is. hopefully it could be a starting point if nothing else. it's pretty simplistic
14:23:05
Bike
geez, why did i write my own lambda list parser... clasp has to have like five of those by now...
14:23:55
Bike
environments are hash tables... if you use this for substantial code you may hti performance problems
14:25:22
beach
Well, another idea would be to turn the AST into host code for compilation. I abandoned it in the HIR compiler, because the code became huge. But the result of turning ASTs into host code ought to be significantly smaller.
16:31:57
froggey
alright, after pulling the Cluster changes it gets through (boot) without any problems
17:04:20
beach
Have you also tried to understand how the boot procedure works, and if so, do you have opinions about it?
17:11:07
froggey
only very superficially. it's surprisingly fine grained, with each phase importing exactly what it needs from the host
17:12:28
beach
Yes, it didn't used to be that way. I used to import pretty much everything, but it became harder to find certain bugs then.
17:20:14
froggey
the HIR interpreted turns backtraces into a soup of INTERPRET-HIR calls, which makes them difficult to understand. that's probably more a problem with CL debuggers not being extensible enough
17:25:14
beach
I hesitate to add specific tools to the bootstrapping environments, because it's a lot of work, but if I can find a way of writing those tools in a "natural" way and reuse them in the bootstrapping environments, that would be good. A backtrace would be such a thing. I did write some customization code for Clouseau so that I can inspect the objects in the bootstrapping environments.
17:36:33
froggey
it did get me thinking about how backtraces could be made extensible. right now there's code in mezzano's debugger to identify generic function discriminator functions in the trace and replace them with the generic function itself
17:37:36
beach
OK, see you. My (admittedly small) family is going to call me to dinner very soon as well.
17:39:09
beach
My tentative plan is to show source locations in the backtrace. That way, I shouldn't have to care much about the objects. Or at least that is what I am hoping.