freenode/#clasp - IRC Chatlog
Search
21:50:48
karlosz
i'll keep in mind to play with the heap size next time i get a floating point exception!
21:55:24
selwyn
karlosz: surely it's possible to do https://github.com/clasp-developers/FlameGraph with perf_events on linux?
21:55:58
karlosz
the code drmeister wrote to help make the traces from cleavir more readable are dtrace specific
21:56:43
karlosz
selwyn: but feel free to take a crack at it with perf_events. i personally don't understand either dtrace or perf_events
23:07:26
kpoeck
drmeister: I made a flamegraph in your new layout with latest cleavir patches from karlosz compiling cl-bench
23:08:22
kpoeck
Sicl from yesterday, but including karlosz patches from https://github.com/robert-strandh/SICL/pull/131
23:08:44
drmeister
Yes - it looks very nice - reinitialize-data is gone and there is a new thing compute-destinies
23:10:49
karlosz
drmeister: do a ./waf configure and then checkout my rui-v2 branch from src/lisp/kernel/contrib/sicl/
23:28:01
selwyn
Probably a BUG: slot SELFWARD-OPERATION in #<The STANDARD-CLASS ASDF/LISP-ACTION:COMPILE-OP> stopped existing between compile and load - from loading clim
23:37:12
kpoeck
Yesterday we got in a strange discussion about the circumstances this bug occured and somehow lost focus on the problem
23:55:20
karlosz
i distinctly remember coming up with an incremental way of updating the function dag. i think the main idea was to create sub dags for newly copied functions and to attach them to the parent dags. then when a function node is removed by remove-useless-instructions, it would be straightforward to remove the subdag rooted at the removed function from the main dag
23:55:36
karlosz
all remove-useless-instructions needs to do is return a list of instructions it removes
23:56:11
karlosz
and then if all references to a closure in that list are gone, then the subdag corresponding to that function can go away
23:57:28
karlosz
in theory, there should be no reason why the function dag needs to be entirely recomputed when something gets inlined
3:11:09
Bike
partial dags are something i tried before but i boned it somehow. i'm sure it's possible to do correctly, though.
3:11:54
Bike
for the slot disappearing thing, that's probably on me - we just started using my fastgf compiler in earnest, and i didn't think very hard about class slots. i suspect we could get a more reproducible error just fucking with class slots a bit.
3:14:30
Bike
drmeister, could you use all that data and see what CL or CORE functions show up a lot? today i noticed zerop wasn't inlined and i'm wondering if there aren't some other gains to be made in that area
3:15:43
Bike
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/cmp/cmpfastgf.lsp#L487 oh. yeah. this is stupid and broken
3:35:38
Bike
i'm specifically thinking i know the ins and outs of booting well enough that i can try the make-instance thing again
4:05:24
drmeister
C++ifying map-instructions-xxxx shaving 25 min off the build suggests the value of tightening up the code
4:07:25
Bike
i thought the point of your C++ified code was using slots in the instructions rather than allocating hash tables.
4:08:22
drmeister
It's two things - adding a 'touched' slot in the instructions and reducing the overhead of map-instructions-xxx
4:10:20
Bike
the only thing in map-instructions-arbitrary-order that seems like it could have a type check elided is the car in (pop instructions-to-process)
4:11:09
Bike
the compiler doesn't use much arithmetic or vectors, which are things where removing type checks would really help. if type inference improves compiler efficiency, it's because it would reduce the amount of code to be compiled
4:11:56
drmeister
You may be right - we can compare the llvm-ir for the C++ified versions and the Common Lisp versions.
4:14:51
Bike
whereas with the make instance thing, i know we have shitloads of make-instance calls and if i do it right they'll skip most of the work they do currently
4:18:21
drmeister
I've added karlosz changes on top of mine. Building -- I'll have new timing tomorrow. But this may bring us to parity with the AST compiler
4:22:54
Bike
i can probably also do a satiation thing where the ctors are in an initial state where they use functions compiled beforehand
4:29:20
Bike
the former of which has a variable class, so there's not a lot that can be done, which is even explained in a comment
4:29:44
drmeister
::notify kpoeck In the cl-bench tests - is there a way to get a spreadsheet view? It's difficult to scroll around. It would be a good idea to run it a few times to ensure all generic function discriminators are compiled.
4:30:50
drmeister
No - every generic function has its own counter - cleavir triggers them in batches
4:32:43
drmeister
it will pause in the middle a couple of times. If you do it again it will pause less and again - not at all.
4:33:48
drmeister
There is a lot to be said for beach's suggestion to put a somewhat random timer in there.
4:34:13
Bike
should put in a mode that's like the fastgf log except all it does is dump something when there's a compile
4:37:52
Bike
literally all you have to do to add a satiation case is add a list of the actual classes to the satiation form in satiation.lisp
4:38:40
drmeister
That would be very attractive - an automated way to set up satiation so that startup is faster.
4:39:06
Bike
https://github.com/clasp-developers/clasp/blob/master/src/lisp/kernel/cleavir/satiation.lisp not really automatable, since i do a ton of macrology in this file
4:41:47
Bike
but i mean, i guess? satiation overwrites the funcallable instance function, so the interpreter is unceremoniously dumped
4:42:33
Bike
last time i tuned up satiation the only calls that were compiling were like, some stuff in eclector
4:42:55
drmeister
Well, I love the idea of generating satiation code automatically by running clasp without it and then using the input for another build.
4:43:44
Bike
i mentioned it more as a cute idea than as something we should actually do. the main problem with it is that starting the system doesn't actually stress cleavir enough
4:44:18
Bike
you want to satiate the whole compiler, not just what we happen to use, or else the user will see random long compiles
4:44:53
drmeister
I'm building now with karlosz's latest changes. Then tomorrow I'll switch back to CL map-instructions-xxx with the 'touched' slot in instructions and see how that impacts things.
4:45:43
karlosz
if i remember correctly, beach would even be okay with adding a touched slot if it were done with stealth mix-ins
4:47:39
Bike
anyway, if we take a look at what dispatch misses are happening we can evaluate things better.
4:48:09
Bike
oh and a totally unrelated thing, i realized during brahms that if we do the multiple entry points fastcall thing we can use that to do tail calls.
8:21:14
Colleen
kpoeck: drmeister said 3 hours, 51 minutes ago: In the cl-bench tests - is there a way to get a spreadsheet view? It's difficult to scroll around. It would be a good idea to run it a few times to ensure all generic function discriminators are compiled.
8:22:08
kpoeck
karlosz: https://github.com/robert-strandh/SICL/pull/131 make the cst build another hour faster for me
8:22:33
kpoeck
My timings are Compilation finished in 5:24:51, Compilation finished in 4:03:12, Compilation finished in 2:58:46
8:23:13
kpoeck
First number baseline, second number your first pull request, third number your second pull request
8:23:40
karlosz
just need to run another build and ill ask you all to help me benchmark again, probably
8:45:48
kpoeck
I might no be getting this right, code for read is in https://github.com/clasp-developers/clasp/blob/dev/src/core/hashTable.cc#L625
8:46:03
beach
Since hash tables are used so much in Cleavir, it might be worthwhile giving some thought to their implementation in Clasp.
9:23:32
karlosz
but first i should run a build overnight for the build-function-dag removal and go to sleep
9:25:20
kpoeck
In my microbenchmark for hash-tables (only read), ccl 96 seconds, sbcl 116 seconds, clasp 113 seconds