Search
Monday, 11th of June 2018, 14:56:14 UTC
16:32:29
Bike
drmeister: speaking of the condition printing stuff - if i do %a in bformat will that be like ~a?
17:46:59
drmeister
Bike: No - there is a short list of format specifiers that translate to C++ boost::format specifiers - I don't think ~a -> %a is one of them.
17:47:38
Bike
Just wondering if we could fix that thing where errors during build don't display correctly until you quit.
17:48:24
drmeister
Actually ~a is one of them.
17:48:51
drmeister
https://github.com/clasp-developers/clasp/blob/dev/src/core/bformat.cc#L148
17:49:20
drmeister
Yes we can fix that - do you know why its happening?
17:49:58
Bike
No. I just noticed the initial printing of the condition is done with %s
19:50:14
karlosz
is there a way to tell if there's a path between two instructions? following the directed edges
19:52:09
Bike
I mean, as a library function. Obviously you can loop yourself.
19:52:30
karlosz
yeah, i guess ill have to do that
19:53:08
karlosz
apparently the catch instruction doesn't necessarily have to dominate all instructions before its two branches merge again
19:53:14
karlosz
whcih i thought was the case
19:53:28
Bike
I'm not sure what you mean.
19:53:40
Bike
Also the two branches don't actually have to merge.
19:53:59
karlosz
well they will eventaully at a ret no matter what, right?
19:54:12
Bike
(block nil (loop)) for instance.
19:54:37
Bike
some of my early bugs working on cleavir were based on the same assumption
19:54:44
karlosz
thers no catch instrcution there though
19:57:35
karlosz
maybe you meant (lambda () (block nil (loop (print (lambda (x) (return))))))?
20:01:29
Bike
there is a catch instruction, it's just that it can be deleted. but yeah you get it.
20:02:01
Bike
additionally, if no paths terminate there might nto be a return instruction at all
20:04:50
karlosz
is it still necessarily true that catch-instruction dominates everything that succeeds it?
20:05:04
karlosz
i mean, up until the merge point, if there is one
20:06:10
Bike
it seems likely, but i couldn't prove it.
20:06:49
Bike
(1+ (if (foo) (block nil ...) ...))
20:08:57
karlosz
i am having trouble inserting catch-close instructions
20:09:06
karlosz
not that i particualrly like dealing with non local exits, but clisp requires it
20:09:16
karlosz
also, in your example, its still true
20:09:31
karlosz
the merge point is not dominated by catch, but everything before it still is
20:09:54
karlosz
i mean like a case where you have some directed edge into a block succeeding catch but not itself dominated by catch
20:10:17
Bike
i see. no, i don't think that's possible.
20:10:29
Bike
you need to have this catch close instruction at the point the block is no longer valid, right?
20:10:45
karlosz
im inserting catch close instruction at the predecessor dominated by the first successor of catch
20:11:00
karlosz
but thats not working for some convoluted cases, hence why i asked by using the weaker path condition instead
20:11:01
Bike
you might want to just customize ast-to-hir
20:11:32
Bike
though it'll be annoying for tagbody
20:12:05
karlosz
and maybe too many hir transofrmations will screw it up
20:13:54
Bike
i don't like to have to reconstruct things that are obvious before a conversion, so
20:14:07
Bike
anyway, does it cover the nonlocal exit case?
20:14:50
karlosz
does what cover the nonlocal exit case?
20:15:12
Bike
I mean, if you have like (block inner (return-from outer)) how does the inner block get closed?
20:15:13
karlosz
what i have works for the majority of non local exits. this is some voodoo coming from restart case
20:15:54
karlosz
the inner one gets closed dynamically
20:16:17
karlosz
the virtual machine marks intervening blocks invalid
20:16:28
karlosz
when hitting a return-from
20:16:33
Bike
when it unwinds, you mean. guess that figures.
20:17:33
karlosz
right, this catch close is just for deallocating the block frame allocated onto the stack
20:17:37
karlosz
(containing the frame pointer)
20:20:43
karlosz
ooh, that explains it. one of the instructions doesn't have a predecessor...
20:21:22
karlosz
these IR consistency issues are not fun
21:38:31
karlosz
cool, alexandria compiles successfully
23:51:41
drmeister
::notify karlosz Could you compile alexandria and time it? I'm dying to compare timing values between Clasp and your setup.
23:51:41
Colleen
drmeister: Got it. I'll let karlosz know as soon as possible.
0:32:56
karlosz
drmeister: alexandria took a minute to compile
0:32:56
Colleen
karlosz: drmeister said 41 minutes, 15 seconds ago: Could you compile alexandria and time it? I'm dying to compare timing values between Clasp and your setup.
0:33:10
karlosz
for reference, clisp takes a second
0:33:23
karlosz
this is with (asdf:load-system :alexandria :force t)
0:34:06
karlosz
caveat: this is with cleavir compiled with clisp. it shouldn't matter too much, but im unable to use cleavir entirely compiled with cleavir at the moment because it segfaults while self compiling somewhere in generate ast
0:34:35
karlosz
how long does clasp take?
0:37:00
karlosz
like one minute million years or like million years million years?
0:37:20
Bike
i don't have a number though
0:37:44
karlosz
theres a lot of low hanging fruit on my end in terms of bringing the compile time down
0:38:12
karlosz
using a real basic block representation would shave a lot of linear lookups for successors
0:40:01
karlosz
in fact ill go ahead and implement that right now to see how much of a difference it makes...
1:08:56
drmeister
karlosz: This is clisp compiled cleavir compiling alexandria?
1:09:48
karlosz
partially cleavir compiled cleavir and clisp compiled cleavir dont seem to differ much in terms of compile time
1:09:56
karlosz
but clisp utterly destroys both so
1:10:08
karlosz
theres still a ways to go
1:10:26
drmeister
"but clisp utterly destroys both so"?
1:10:58
karlosz
as in clisp just compiles much much faster
1:12:19
karlosz
do you have timings for clasp? and is it a cleavir or llvm problem?
1:17:22
drmeister
karlosz: Could I get a copy of what you are running at some point to compare to clasp?
1:21:31
karlosz
drmeister: https://gitlab.com/karlosz/clisp
1:21:55
karlosz
very WIP, may break at any moment
1:22:04
karlosz
but the gist is loading it with clisp/src/load.lisp
1:22:52
karlosz
into clisp and switching cleavir-clisp::*use-cleavir* on and compiling forms, if you actually want to run it
1:23:25
karlosz
to load alexandria you have to load it once with clisp to get the macros in and then (let ((sys::*load-compiling* t)) (asdf:load-system :alexandria :force t)) to compile it with cleavir
1:30:13
drmeister
Thank you very much! Trying now...
1:31:36
karlosz
you don't have to compile clisp yourself. i think a binary copy from whatever package manager works well enough
1:31:45
drmeister
This gives me the itch to start profiling everything again.
1:31:54
drmeister
Must. Restrain. Myself.
1:32:12
karlosz
heh! i think that minutes to compile alexandria is justified
1:32:19
drmeister
Can you give me a bit of background? clisp compiles clisp?
1:32:33
karlosz
i think clisp bootstraps from c
1:32:41
karlosz
but you can just use a binary
1:32:50
karlosz
all the stuff i added can just be loaded into aan existing system
1:32:57
drmeister
karlosz: Do you compile alexandria using quicklisp?
1:33:12
karlosz
well i used quicklisp to get it
1:33:31
drmeister
Oh - I see - you said what you did.
1:33:33
karlosz
but i need to load it once and then manually have asdf force compile it with cleavir
1:33:41
drmeister
(let ((sys::*load-compiling* t)) (asdf:load-system :alexandria :force t))
1:33:49
drmeister
Sorry - my eyes skipped over that.
1:34:17
karlosz
make sure you load alexandria with clisp first -- some macros are declared in the same file they are used in alexandria
1:40:18
drmeister
The :force t forces compilation I wonder?
1:42:01
karlosz
yeah, if you dont have it it will just load preexisting fasls
1:42:21
karlosz
it might not be strictly necessary but it doesnt hurt
1:43:57
drmeister
How does clisp work? Does it have a CL compiler written in C?
1:45:10
karlosz
the compiler is written in lisp, but it compiles to a bytecode stack virtual machine that is implemented with C
1:45:42
drmeister
How does it bootstrap the compiler written in Common Lisp?
1:45:53
karlosz
i think with an interpreter
1:46:14
karlosz
im not quite sure either, i haven't needed to delve into those guts
1:47:00
drmeister
I'm building from scratch
1:49:13
balrog
ugh, this box seems to hang when compiling large projects. might be time to replace it :|
2:14:21
drmeister
So - I do this...
2:14:35
drmeister
(defun foo (x y) (+ x y))
2:14:54
drmeister
Then... (time (loop for x below 1000000 do (foo 1 2)))
2:15:58
drmeister
clisp 26.1 seconds
2:16:10
drmeister
Clasp 0.11 seconds
2:16:54
karlosz
is that with cleavir on clisp?
2:17:43
karlosz
my laptop does it in less than a second on clisp
2:18:12
drmeister
No - that's the regular clisp compiler.
2:18:31
karlosz
weird. 26.1 seconds is way too much
2:18:37
karlosz
i get 0.14 seconds with clasp
2:18:41
karlosz
and 0.93 seconds with clisp
2:19:35
karlosz
sure, native code is fast - but it shouldn't be that much faster
2:21:56
karlosz
maybe try compiling that loop?
2:22:02
karlosz
(compile nil (lambda () ...))
2:24:34
karlosz
or try compiling foo
2:27:55
drmeister
I had to compile the two functions in clisp
2:29:55
karlosz
okay, that makes sense
2:35:38
karlosz
oh, cool. now i compiled alexandria in 19 seconds
2:36:16
karlosz
i was able to cut the time by more than 2/3rds just by switching to a better basic block representation
2:36:33
karlosz
lots of low hanging fruit still
2:37:11
karlosz
1.11 to 0.19 isnt so bad
2:40:38
karlosz
drmeister: sorry to pull the rug right under you, but i just made cleavir compile things >3x faster
2:41:10
drmeister
It's ok, I'm dealing with the 100x slower compilation of Clasp.
2:41:53
Bike
i don't understand why karlosz has times two hundred faster to begin with.
2:41:59
Bike
just with clisp i mean.
2:42:33
karlosz
my lack of optimizations?
2:42:38
karlosz
im still on generate-ast
2:42:38
Bike
llvm isn't involved with just clisp, is it?
2:42:52
karlosz
maybe llvm is slowing things down for you
2:43:00
Bike
llvm takes time, but cleavir is slow too.
2:43:15
Bike
but what i'm concerned with is how you get 0.93s with clisp and meister gets 26s.
2:43:37
karlosz
no, he said that he fixed it by actually compiling them
2:43:52
karlosz
he had interpreted the code, from what it looks liek
2:44:12
karlosz
there's an interpreter/compielr dichotomy in clisp, if thats what youre talking about
2:44:13
drmeister
(defun ccomp (n) (loop for i below n do (compile 'nil '(lambda (x y) (+ x y)))))
2:44:46
drmeister
(time (ccomp 100))
2:44:59
drmeister
https://www.irccloud.com/pastebin/lp8OOyvO/
2:45:13
drmeister
llvm is taking a lot of time - 1.47 seconds of the 1.943 total
2:45:30
Bike
...but failing to compile it was why it was slower in clisp?
2:46:08
drmeister
clisp ... 0.017 seconds.
2:46:15
karlosz
Bike: interpreted clisp code is much slower
2:46:24
karlosz
than compiled clisp code
2:46:41
karlosz
drmeister got 29 seconds with interpreted, and compiled is 0.97
2:46:57
karlosz
clisp also just has a really fast compiler
2:47:03
Bike
cleavir also runs very quickly in sbcl when i tried doing that.
2:47:16
karlosz
it can compile host 1 of sbcl in like 2 seconds
2:47:32
karlosz
but the drawback is it takes 30 minutes for sbcl to compile itself with that generated code
2:48:10
karlosz
little/no optimizations = better compile time
2:48:32
Bike
yeah. like ccl but moreso.
2:51:40
karlosz
anyway, i see promising results from this new block representation. do you think cleavir uses basic blocks enough to benefit?
2:51:57
Bike
i'm not sure w hat you mean by "uses".
2:52:02
Bike
it certainly traverses the hir a lot.
2:52:33
karlosz
well for example, i use basic blocks for dominator, df, and ssa calculations
2:52:56
karlosz
and before the successor fun was really bad
2:53:19
karlosz
but if i keep track of each block's successors in the block itself it speeds things up
2:54:03
Bike
sure, clasp does a lot of traversals too.
Tuesday, 12th of June 2018, 2:56:14 UTC