freenode/#clasp - IRC Chatlog
Search
22:48:31
karlosz
this graph shows the compilation of cleavir/covert-special.lisp with the new PM: ocf.io/~karlos/flame12.svg
22:48:51
karlosz
this explains why switching pass managers for optimizeModule doesn't seem to make much of a difference
22:53:08
karlosz
aha, i see a way to improve the speed from looking at the graph. IR module optimizations already include IR function optimizations, so doing both is just redundant
22:57:41
karlosz
well, i'm not sure if the current way in jit-setup is actually doing it twice (it does populate both the function pipeline and the module pipeline), but doing so in the new PM is redundant
22:58:12
drmeister
Another thing that I've thought about is cleavir's map-instructions does a lot of consing of hash-tables. Could we estimate the number of instructions and set the initial size of the hash-table used to do map-instructions?
23:00:15
karlosz
we can cache the number of instructions in the graph somewhere as an approximation of the initial size, if the hash table resizing is actually a bottleneck
23:16:35
Bike
what should be happening is that with-early-accessors macrolets generic-function-call-history, since it's a slot in standard-generic-function
23:17:05
Bike
but in that backtrace it seems like there's no generic-function-call-history macro, because if there was it wouldn't be calling default-cas-expansion
23:19:27
drmeister
Where should the generic-function-call-history macro be defined? I don't see it anywhere. In my code or yours.
23:20:51
Bike
https://github.com/clasp-developers/clasp/blob/master/src/lisp/kernel/clos/hierarchy.lsp#L146 here's the slot.
23:21:17
Bike
https://github.com/clasp-developers/clasp/blob/master/src/lisp/kernel/clos/std-slot-value.lsp#L66-L81 here's with-early-accessors, which as you can see should expand into a macrolet.
23:39:45
drmeister
I'm having trouble with ediff-directories as well - it says there are differences between directories but the files within them are all the same.
23:46:49
Bike
like does it not know about the with-early-accessors macro, so it's compiling it as a function call?
23:48:18
Bike
because that would explain the problem, though i don't know how it would miss that macro.
0:13:32
drmeister
So a test discriminator compile takes a little bit longer and the overall build time is unchanged
0:14:18
drmeister
Maybe the thing is going nuts when we turn off the interpreter and it goes more nuts if the discriminating function compiler is even a little slower.
0:14:55
Bike
but you're seeing that the build time is unchanged if you keep this in but keep the interpreter on? that's not what i just saw
0:16:38
Bike
yeah. normal build = like nineteen minutes. build with interpreter still on but this discriminator code = like 23 minutes
0:18:25
drmeister
I've been watching this guy "kitboga" on twitch. He calls internet scammers and strings them along for as long as possible.
0:43:45
drmeister
Here's the sorted list of the files that take the longest time to build across aclasp/bclasp/cclasp - explanation follows...
0:45:13
drmeister
The numbers on each line are... <pid> <seconds-to-build> <bytes-consed> <total-num-files-aclasp/bclasp/cclasp-58/141/473> <file-name>
0:49:47
drmeister
cl-wrappers.lisp takes almost 2x longer. We should be able to pick that up with a flame graph.
0:55:48
drmeister
Discriminator functions will get compiled as the interpreted discriminator functions age out.
0:56:48
Bike
https://github.com/clasp-developers/clasp/blob/master/src/core/funcallableInstance.cc#L371-L376
0:58:56
Bike
it seems like the improvement in the speed of discrimination is generally dwarfed by the compile time.
1:01:34
drmeister
If it's all interpreted generic functions and satiated ones - then the satiated functions must be slower than the old satiated functions.
1:01:36
Bike
since it's turned off in cleavir, i think this is the only place the compiler is used https://github.com/clasp-developers/clasp/blob/master/src/lisp/kernel/clos/satiation.lsp#L451-L618
1:02:03
Bike
it's hard to imagine these clos functions being so much slower that it doubles compile time
1:02:17
Bike
especially when in simple comparisons it doesn't look like the new code generated is in fact slower
1:03:10
Bike
like i said, my first suspicion is that something is dispatch missing a lot, but it's hard to tell
1:03:15
drmeister
How about we count the number of gf discriminating functions being compiled and report that with Time and consed in the build output.
1:08:40
drmeister
Here are the slowest ones - they look like the first ones I posted - with the old discriminating function compiler.
1:09:34
drmeister
cracauer told me that when he used to do timing at Google that he took a lot of trouble to get the machines stable for timing measurements.
1:37:12
Bike
anyway, if this is legit, cool i guess. i can do this for the dtree interpreter and then where tags are unneeded. and i can work on something else.
1:58:23
drmeister
The llvm-ir for Instance_O vs FuncallableInstance_O looks identical - is the rack at the same offset at the moment?
1:59:41
Bike
but that could change, or we coudl use the same thing for derivables where the rack is at a different offset.
2:01:01
Bike
splitting it up also means that we do one 3-switch on the header and then a 2- or 3- switch on the rack stamp, rather than a switch on the where tag followed by a 5-switch on the rack stamp.
2:07:14
Bike
should do some metrics on the average spread, at some point when i'm less sick of this code
2:09:18
Bike
i gues a histogram of call history lengths would probably be a good start and easy to do
2:22:46
Bike
bclasp generally. if you put it in a file and you're in cclasp it'll be cleavir, though.
2:24:24
Bike
actually it might be possible to do so now since dispatch misses don't invalidate the function, but i dunno, i don't like going down that rabbit hole
2:27:06
Bike
cleavir-ir:successors is a function i use for this sometimes, since it's inevitably got a call history that's like 60 long
2:28:22
Bike
i usually pick out the form and call macroexpand myself. like (macroexpand-1 (third (fourth (third (third (fourth form)))))) in this case
2:28:53
Bike
like i said. just put #.(clos::generate-discriminator whatever) in an otherwise empty file and compile that.
2:29:29
drmeister
You said "quickest way would probably be #.(clos::generate-discriminator #'generic-function)"
2:29:58
drmeister
The quickest way would probably be to put #.(clos::generate-discriminator #'generic-function) into a file and compile-file it.
2:31:32
drmeister
quickest way would probably be #.(clos::generate-discriminator #'generic-function)
2:33:50
Bike
uh, maybe do #.(clos::compile-time-discriminator #'initialize-instance (clos::generic-function-call-history #'initialize-instance))
2:39:45
Bike
your ir will have some fixnum tests it doesn't need to. i cut those out locally to see if they were the problem