freenode/#clasp - IRC Chatlog
Search
4:12:53
drmeister
I will do most of it - looking at the llvm-IR and stepping through the machine code and comparing that to what you get when you step through the sbcl code.
4:13:14
beach
Are you saying that your implementation of my technique is slower that SBCL's implementation of the PCL technique?
4:14:22
beach
That would be hard for me to figure out why, I think. When my technique is implemented in SBCL it is faster than the PCL technique, so it must have something to do with the way you represent objects, or with the way you generate LLVM code.
4:16:17
drmeister
I put a lot of time into the fastgf and then put it aside for a while - but I need to go back to it and look at it with fresh eyes.
4:17:27
drmeister
The LLVM-IR is pretty tight. Now that I think about it again - most of the time is spent outside of the dispatch function.
4:18:18
beach
So it probably has more to do with the way you pass arguments, call effective methods, etc.
4:19:24
beach
That would be a very large effort on my part to understand in sufficient detail that I would be able to help, I would think.
4:19:50
drmeister
Yeah - so I probably don't need your help - it's the other details of Clasp's argument handling and runtime.
4:21:17
drmeister
Ok - never mind then. I was planning to invest some time in it to squeeze out more wasted cycles and then generate data comparing it to sbcl for part of an ELS paper.
4:26:11
drmeister
The last thing I did was switch to using as many registers as were available for the dispatch.
4:28:06
beach
What does it mean to "switch to" that? If you are not using registers, you are making memory accesses, which is what my technique is trying to avoid.
4:31:51
drmeister
Method calls take two arguments, a list of arguments and a list of next methods. When I started I took that literally and the dispatchers used the list of arguments. I've taken a lot of pains to hack the C calling convention so that I can access the arguments in registers directly. The last thing I did with the fastgf implementation is switch to using the first four arguments in registers for gf dispatch and when
4:32:31
drmeister
Right - and this is the rub. Calling the effective methods requires creating the list of arguments and passing that to the effective methods.
4:34:26
drmeister
But for cases where the effective method doesn't call call-next-method or next-method-p, we don't need to spill the registers to memory and call the effective method that way. We could generate effective methods that take register arguments and pass them through the discriminating function.
4:35:37
drmeister
That optimization is awaiting Bike's work with first class top level environments and the new bootstrapping system so that we can make that optimization - because the current way of bootstrapping gets in the way.
4:36:59
drmeister
That's why it's still slower than sbcl using PCL. I have to spill the arguments and pass them as a varargs structure and then the effective method has to destructure them.
4:53:36
drmeister
cclasp (the Cleavir compiler in Clasp with first class top level environments) compiling the cclasp code.
4:55:14
drmeister
Bike is setting things up so that macro definitions and whatnot modify the compile-time environment but not the runtime environment. So compiling cclasp with cclasp can not damage the running compiler. But the new definitions for macros and symbol macros and everything else will be used in the compilation.
4:56:09
drmeister
It gives us the kind of protection that sbcl gets with tricky package manipulation.
4:57:52
beach
And I thought that idea was rejected because it was too hard to set up a Clasp environment in SBCL. Not that I understand why.
4:59:00
drmeister
Everything that cclasp needs from the C++ is summarized in a few data structures. We can generate that from a quick run of a script in the Clasp interpreter - and then give that to a full cclasp compiler to generate bitcode for a new cclasp compiler.
5:00:29
drmeister
So 15 min to compile the C++ code, 15 min for a cclasp to compile another cclasp based on the configuration from the C++ code and then a few min to link the C++ bitcode with the CL bitcode.
5:01:24
drmeister
And we might be able to get the configuration info out of the C++ source code so that we can compile the C++ code in parallel with the CL code.
5:02:45
drmeister
Then we can apply optimizations that right now run afoul of the bootstrapping system.
5:08:45
beach
If you introduce a defect at some point, and that defect goes unnoticed for a few generations of bootstrapping, it is sometimes impossible to get back to a correct version.
5:10:22
beach
I don't have any universal method. I used that in the compiler I wrote for my theses, and never found a good way out, other than being extremely careful. That is why, for SICL, I only require a full working Common Lisp implementation plus the MOP.
5:11:07
drmeister
So - once we have it working for Clasp - Bike want's to extend it to sbcl and other Common Lisp compilers.
5:12:10
beach
But then, you lose the advantage you said: "It's much less work to do it with cclasp than it is with sbcl."
5:12:11
drmeister
In that approach we have a backend that generates an intermediate representation that the C++ interpreter would use to generate llvm bitcode.
5:13:07
drmeister
I see it more as an evolution. We can get something up using cclasp and then migrate it to sbcl by writing this backend.
5:15:13
drmeister
Then the build system becomes (1) extract the configuration info from the C++ code (2) compile the Common Lisp code using sbcl -> generate S-expressions that describe the intermediate representation (3) Run code in the Clasp interpreter that translates the intermediate representation to llvm-IR and generate bitcode (4) link the CL bitcode with the C++ bitcode.
5:16:51
drmeister
The configuration info includes things like the stamp values of all of the C++ classes.
5:18:31
drmeister
These are almost all of the constants that the CL code needs to know to generate code that works with the runtime in C++.
5:24:03
drmeister
Bike is the one that impressed on me that we want to run this in sbcl as soon as possible.
13:40:59
Bike
beach: i changed my mind about sbcl being too hard. it's still work, but it seems preferable to the current system, or to relying on having a previous clasp binary
13:41:21
Bike
i haven't started on it, though. we'll have to rewrite the source so that it's written more conformingly
13:43:07
Bike
i also think we can probably have the cleavir-in-sbcl thing only go to ASTs and not bother with HIR, unless i'm missing something
13:45:19
Bike
i'm hoping after some time we can strip things down so that the interpreter knows how to load fasls and call C++ and not much else, so we can have things defined in CL instead of C++. but that's farther off
13:53:37
beach
I think going from source code (or CST) to AST is the hard part, because the environment must contain all the implementation-specific macros, types, etc.
13:54:04
beach
From AST to HIR should be easier. There could be some specialization on the implementation, but it shouldn't be too hard.
13:59:26
Bike
I have some functions set up that make an environment for cleavir that only has the functions available in the interpreter, plus cleavir primops and such. It has a cleavir-env:eval method that evaluates things in a different global environment that has a full CL
14:58:52
drmeister
I presume that trace is binding *print-pretty* to T and the traces are unreadable because the lines are very, very long
15:02:52
Bike
(princ (with-output-to-string (*trace-output*) ...) *trace-output*) haha, what the fuck