freenode/#clasp - IRC Chatlog
Search
23:51:41
drmeister
::notify karlosz Could you compile alexandria and time it? I'm dying to compare timing values between Clasp and your setup.
0:32:56
Colleen
karlosz: drmeister said 41 minutes, 15 seconds ago: Could you compile alexandria and time it? I'm dying to compare timing values between Clasp and your setup.
0:34:06
karlosz
caveat: this is with cleavir compiled with clisp. it shouldn't matter too much, but im unable to use cleavir entirely compiled with cleavir at the moment because it segfaults while self compiling somewhere in generate ast
0:37:44
karlosz
theres a lot of low hanging fruit on my end in terms of bringing the compile time down
0:38:12
karlosz
using a real basic block representation would shave a lot of linear lookups for successors
0:40:01
karlosz
in fact ill go ahead and implement that right now to see how much of a difference it makes...
1:09:48
karlosz
partially cleavir compiled cleavir and clisp compiled cleavir dont seem to differ much in terms of compile time
1:17:22
drmeister
karlosz: Could I get a copy of what you are running at some point to compare to clasp?
1:22:52
karlosz
into clisp and switching cleavir-clisp::*use-cleavir* on and compiling forms, if you actually want to run it
1:23:25
karlosz
to load alexandria you have to load it once with clisp to get the macros in and then (let ((sys::*load-compiling* t)) (asdf:load-system :alexandria :force t)) to compile it with cleavir
1:31:36
karlosz
you don't have to compile clisp yourself. i think a binary copy from whatever package manager works well enough
1:33:33
karlosz
but i need to load it once and then manually have asdf force compile it with cleavir
1:34:17
karlosz
make sure you load alexandria with clisp first -- some macros are declared in the same file they are used in alexandria
1:45:10
karlosz
the compiler is written in lisp, but it compiles to a bytecode stack virtual machine that is implemented with C
1:49:13
balrog
ugh, this box seems to hang when compiling large projects. might be time to replace it :|
2:36:16
karlosz
i was able to cut the time by more than 2/3rds just by switching to a better basic block representation
2:40:38
karlosz
drmeister: sorry to pull the rug right under you, but i just made cleavir compile things >3x faster
2:44:12
karlosz
there's an interpreter/compielr dichotomy in clisp, if thats what youre talking about
2:47:32
karlosz
but the drawback is it takes 30 minutes for sbcl to compile itself with that generated code
2:51:40
karlosz
anyway, i see promising results from this new block representation. do you think cleavir uses basic blocks enough to benefit?
2:53:19
karlosz
but if i keep track of each block's successors in the block itself it speeds things up
3:09:35
drmeister
I have to use (return-from whatever-function-im-in) to guarantee I'm returning from the current function - right?
3:09:55
drmeister
I'm going to short circuit the compiler and cut everything about LLVM out of the loop.
3:20:57
karlosz
oh great. its not the stack that overflows when self compiling now, its the heap that runs away, eating up all the ram i have
3:23:23
drmeister
I defined a special variable that causes the compiler to short circuit before it starts generating instructions and lowering it to native code.
3:25:07
drmeister
Maybe I screwed up the previous timing? It's kind of tricky to count all of the time spent in llvm.
3:31:55
drmeister
I set this special variable and COMPILE returns immediately in layout-basic-block and then returns immediately from everything that calls it.
3:42:43
Bike
first it means we should do some more comprehensive examination, because i thought this was the case before and you did some timing and hey it wasn't
3:46:29
drmeister
I'm thinking aloud now. I wrap (catch :dont-generate-code ...) around the body of COMPILE.
3:47:30
drmeister
And the first form in (defun translate (...) (when *dont-generate-code* (throw :dont-generate-llvm nil) ... ))
3:59:52
drmeister
I move the (when *dont-generate-code* (throw :dont-generate-llvm nil)) to after the call to (translate ...)
4:00:10
drmeister
(time (let ((clasp-cleavir::*dont-generate-code* nil)) (ccomp 1000))) --> 23.2 secs
4:05:28
drmeister
The time definitely keeps increasing each time I (time (ccomp 1000)) it increases by 2-3 seconds
4:17:20
drmeister
The llvm time in TIME is off by a few seconds but not off by an order of magnitude.
4:18:26
drmeister
So it's generating llvm instructions - but doesn't optimize the module or lower it to native code
4:19:17
drmeister
With full compile (code generation) --> 28.9 seconds and LLVM time reports 14.6 seconds
4:24:55
drmeister
This means 90% of the time is spent optimizing and generating code for the module.
4:25:52
drmeister
Last one for the night - I'm moving the throw right before the jit-finalize-repl-function
4:45:23
drmeister
Found it - I was compiling a function in jit-setup.lsp and it doesn't include an (in-package :cmp) because that macro isn't defined that early
4:50:48
drmeister
I'm running 100 truncated compiles and moving the throw around the jit-add-module-return-function
8:25:36
heisig
I made good progress with the hoisting this weekend. I think I need about two more weekends until completion.
8:42:02
heisig
It is a fun problem with lots of recursion on circular data. The fast pace of karlosz has motivated me to hurry up.