freenode/#sicl - IRC Chatlog
Search
4:58:12
beach
I am contemplating a question of elegance and performance during bootstrapping. One of the first-class global environments I use, like E4 or E5 is going to be the template for the final system, in that it will contain no functions imported from the host. It will still contain host objects such as CONSes, symbols, numbers, and probably also packages.
4:58:21
beach
When the binary executable contents is written, those host objects will be replaced by SICL equivalents. Now, during bootstrapping, I often have a choice between a host function and a SICL function. In particular, for functions on lists, I can import the host version of something like MAPCAR, or I can load the SICL version from a SICL FASL file.
4:58:29
beach
The SICL version will be significantly slower, because it is executed using the HIR interpreter. So the first question is: Should I use SICL versions of or host versions in environments other than the last?
4:58:40
beach
The second question is: Should I use SICL versions exclusively in the last environment, or should I use host versions during bootstrapping (for performance reasons) and only load SICL versions right before the final executable is generated?
4:58:47
beach
The third question is: Should I import only those host functions that I absolutely need (as I do now), carefully justifying why they are needed, or should I just import a lot of host functions, like all the functions on lists?
4:58:51
beach
The fourth question is: In the last environment, should I load only those SICL functions that are required in order for the executable to function, and load the rest from the executable itself, or should I load lots of other functions as well?
4:59:19
beach
I mean, I am fully capable of making a decision myself, but it would be interesting to hear opinions.
5:01:42
beach
So for the first question, I have a tendency to think that I should use host functions, because I am likely to add more work during bootstrapping, and it would be silly to take a performance hit just because I can't use compiled versions of SICL functions during bootstrapping.
5:02:50
beach
For the second question, I think the reason is the same. There is no reason to use SICL functions during bootstrapping when there are host functions that will do the same thing, except faster.
5:03:46
beach
For the third question, I think I want to limit the number of host functions I import to the strict minimum, justifying each one with a comment. But that's a lot of work.
5:06:20
beach
For the fourth question, it is clear that there will be SICL stuff that I can't create in the host. In particular, I am thinking of hash tables. EQ hashing is going to depend on address, and that won't be known until the last minute.
5:06:21
beach
I plan to implement a simple version of the hash-table interface, but that uses lists instead, and use it during bootstrapping. In the final system, those hash tables will need to be replaced by real ones.
5:11:23
beach
The advantage of using SICL versions during bootstrapping is that those functions will then be exercises more, but I can write a bunch of tests running in the last environment after all the SICL versions have been loaded.
8:41:06
MichaelRaskin
Is it likely that these decisions will later affect cross-compilation work? (My immediate reaction is that one would want to use host functions for host data structures, though, so I guess this doesn't chage the conclusion anyway)
9:26:49
beach
So most cross compilation will be done in that "final" first-class global environment I was referring to, in that I need to compile everything that must be present in the executable from the start, including the reader and the compiler.
9:28:54
beach
In other environments, I only need to cross compile the SICL-specific code that is needed for the bootstrapping procedure itself. That's not a small amount though, because it involves the entire MOP machinery to create generic functions, methods and classes, to compute discriminating functions, to finalize inheritance, to create and initialize instances, etc., etc.
9:31:08
beach
Recall that creating a SICL FASL file is very easy. I am using a printed version of the AST as the FASL format, so all that is needed is to use Eclector to read the code, and to use CST-to-AST against a first-class global environment with enough macro definitions in it.
9:33:07
beach
The downside of that FASL format is of course, that to load a FASL file, I need most of the compiler, including AST-to-HIR, HIR transformations, HIR-to-MIR, MIR-to-LIR, LIR-to-"assembly", and "assembly"-to-binary.
9:34:36
beach
But all that stuff is "simple" in that it involves transformations that can be done either by native code or by host code.
9:35:55
beach
There are two hard parts. One is creating the AST, but the hard part of that creation is taken care of by the use of first-class global environments.
9:38:11
MichaelRaskin
And in a sense, does SICL use different implementations of CDR on host and in the already-bootstrapped native mode (probably same HIR but using different representations of cons cells for host and for final version)?
9:39:53
beach
Yes, they are different. I mean, there is a CDR-INSTRUCTION in HIR, and in the host, when I interpret that instruction, I call the CDR function of the host. In parallel, I translate that HIR instruction to MIR that exposes memory references and such.
9:42:32
MichaelRaskin
So in a sense, during the bootstrap SICL code runs under different conditions than normally. (I guess running a test suite in the final host-supported environment might be useful for checking for implicit assumptions then)
9:43:45
beach
The majority of such assumptions would be the existence of a function in the final executable.
9:45:00
beach
Suppose during bootstrapping, I call MAPCAR, but I use the host version. But I forget to include the SICL version of it in the final environment. Then the executable generated from that environment very likely won't work.
9:45:51
beach
So one test I need to do is to traverse all the code referred to by that final environment and make sure it is all SICL code.
9:47:46
beach
I may also need to check that important classes are finalized, and that important generic functions are functional, i.e., they have the required methods and a discriminating function has been computed and installed.
9:50:39
MichaelRaskin
I assume you want to run a full test for the resulting native SICL executable anyway
10:25:00
beach
I also plan to configure Clouseau so that I can inspect host representations of SICL objects in a slightly more convenient way.
12:07:29
beach
The other thing I should do is this: Currently, all my first-class global environments are host objects, and so are the data structures that they contain for organizing classes, functions, types, etc. But ultimately, I am going to need a first-class global environment represented as a SICL object, i.e. with a header and a rack. And it would be preferable if I can create that in the host before I create the final executable file.
12:08:18
beach
So in a first-class global environments, I should load files containing definitions of a first-class global environment and its accessor functions.
12:09:08
beach
I should then create an instance of that class, and use that instance to contain all the final code to be used to create the executable file.
12:10:13
beach
Many of the objects in that last environment could be directly imported from the last of the first-class global environments represented as host objects, I hope.
12:17:07
beach
The other thing that had me baffled for a long time was how to "tie the knot", i.e. how to make the class of objects (including classes) in environment En be classes also in En and not in En-1 as is the case during bootstrapping.
12:20:03
beach
Under normal circumstances, accessor generic functions do not consult the class object of their arguments. They consult the stamp in the instance. So if everything is finalized, satiated, etc., then it should not really matter what the "class" slot of the header refers to. Therefore, I should be able to alter that slot without changing the way the code works.
12:20:29
beach
The other thing I need to do is to modify some of the definitions that I introduced during bootstrapping.
12:21:21
beach
For example, I have a FIND-METACLASS function, that, under normal circumstances, should just call FIND-CLASS. But during bootstrapping, it is defined to look for the class in En-1.
12:22:12
beach
Otherwise, the rest of the code just calls accessors to get the job done, so as long as the accessors work correctly, then there should be no problem.