freenode/#clasp - IRC Chatlog
Search
18:47:44
drmeister
Bike: We have a _SetfFunction slot - do you know if I use eval::funcall(_sym_touched->_SetfFunction,value,object) - will that set the value of the 'touched' slot?
18:49:10
drmeister
I think that's how it's supposed to work if I want to set the value of an objects slot.
18:49:36
kpoeck
karlosz: A flamegraph compiling asdf with your fixes that are already merged to sicl
18:57:26
drmeister
I know the name of the accessor is touched - so the setf function is (setf touched).
18:58:35
drmeister
I've rewritten map-instructions, map-instructions-with-owner and map-instructions-arbitrary-owner in C++ and switched from using a hash-table to using a slot called 'touched' in each instruction.
19:00:06
drmeister
I want to see if my hypothesis that overhead and using hash-tables to keep track of what instructions have been seen is responsible for a lot of the time taken by inlining.
19:20:50
drmeister
Bike: Did you add automatic generation of xxxx-instruction-p predicates? I thought you did but I don't see enclose-instruction-p
19:22:20
Bike
i guess with the dtree interpreter in place there's not as much worry of weird slowdowns, so i may as well
19:25:05
drmeister
selwyn: My new linux machine at home builds AST in about the same amount of time.
19:25:11
Bike
itll take me a bit to figure out since it involves anonymous generic functions and stuff
19:26:02
drmeister
FYI startup is a bit slower now because there are some compilations happening even when it only compiles the 1024th time a gf is called.
19:26:37
drmeister
Also - once you start compiling things there are big slowdowns as all the compilation chickens come home to roast.
19:31:09
Bike
like, i can put the thing to set up the predicate in initialize-instance for class, but that doesn't work for anything built in
19:32:05
drmeister
It's ok - I'm just hacking here - I added an enclose-instruction-p and enter-instruction-p predicate.
19:32:43
Bike
don't push it to the repo or anything, that caused me a lot of rebasing problems before
19:33:39
drmeister
I don't touch sicl - that's beach's and your domain - this is an experiment that I would be perfectly happy to see fail.
19:34:08
selwyn
speaking of chickens: at ELS i found out there is an attempt at chicken scheme <-> c++ integration
20:06:31
drmeister
It's not directly comparable to the other flame graphs I've posted because this is single threaded.
21:01:27
drmeister
Well, I'll be hornswoggled - the C++ version doesn't appear to offer any benefits whatsoever.
21:01:58
drmeister
I'm not sure - because I have to time an entire build - but the flame graphs look exactly the same!
21:02:40
drmeister
Yeah - they are different in the respects where I changed them - that's different.
21:06:17
Bike
it did decrease the reinitialize-data fraction by a percent or two. but yeah, doesn't seem to have helped much
21:09:21
karlosz
Bike: i'd like to investigate what you noted about predecessors and reinitialize data not being able to hook up when inlining functions that don't return
21:10:06
karlosz
selwyn: you pulled with the pull request i sent this morning right? do you think its possible you can make a flame graph?
21:13:00
Bike
off the top of my head, if you had like (progn ((lambda () (return))) (funcall f)) it could inline the first call and then try to inline the second
21:15:34
drmeister
Ah - yeah - I can make register_if_unvisited simpler - I wasn't thinking about improving the code - just C++ifying it.
21:16:26
drmeister
selwyn: Did it build ASDF? Or did it die when trying with a Floating point Exception?
21:18:08
Bike
i mean, like, what did you make it do? i thought setting a slot would be easier than setf gethash
21:18:12
drmeister
Actually - no - I can't make register_if_unvisited simpler - it has to conditionally build an instruction list.
21:19:34
selwyn
i think the second. it prints 'Floating point exception' and exits straight away. do you think it's worth running with the debugger?
21:19:59
drmeister
The C++ version has less GC going on - that's probably because there is less rehashing.
21:21:09
drmeister
selwyn: It might be the problem that Bike and karlosz are talking about - in which case lldb/gdb will not be very illuminating.
21:23:33
Bike
https://github.com/clasp-developers/clasp/commit/7f64024a81db6db8357dfb6e9658c474c981a981#diff-1f35b120724d1751acd6cd263fc20505R431
21:24:17
karlosz
i like to log other in magit and then just cherry pick the patches i want in case someone else broke dev
21:25:24
drmeister
selwyn: Then it might be a very good idea to run it under gdb/lldb and print a backtrace.
21:26:01
karlosz
kpoeck: then do git pull <remote name>/rui-v2 all while in src/kernel/lisp/contrib/sicl/
21:27:21
kpoeck
After an innocent (ql:quickload :cl-opengl), I get again Probably a BUG: slot SELFWARD-OPERATION in #<The STANDARD-CLASS ASDF/LISP-ACTION:COMPILE-OP> stopped existing between compile and load
21:27:30
Bike
so if the difficulty in map-instructions-arbitrary-order is the worklist that's going to be difficult to deal with
21:50:48
karlosz
i'll keep in mind to play with the heap size next time i get a floating point exception!
21:55:24
selwyn
karlosz: surely it's possible to do https://github.com/clasp-developers/FlameGraph with perf_events on linux?
21:55:58
karlosz
the code drmeister wrote to help make the traces from cleavir more readable are dtrace specific
21:56:43
karlosz
selwyn: but feel free to take a crack at it with perf_events. i personally don't understand either dtrace or perf_events
23:07:26
kpoeck
drmeister: I made a flamegraph in your new layout with latest cleavir patches from karlosz compiling cl-bench
23:08:22
kpoeck
Sicl from yesterday, but including karlosz patches from https://github.com/robert-strandh/SICL/pull/131
23:08:44
drmeister
Yes - it looks very nice - reinitialize-data is gone and there is a new thing compute-destinies
23:10:49
karlosz
drmeister: do a ./waf configure and then checkout my rui-v2 branch from src/lisp/kernel/contrib/sicl/
23:28:01
selwyn
Probably a BUG: slot SELFWARD-OPERATION in #<The STANDARD-CLASS ASDF/LISP-ACTION:COMPILE-OP> stopped existing between compile and load - from loading clim
23:37:12
kpoeck
Yesterday we got in a strange discussion about the circumstances this bug occured and somehow lost focus on the problem
23:55:20
karlosz
i distinctly remember coming up with an incremental way of updating the function dag. i think the main idea was to create sub dags for newly copied functions and to attach them to the parent dags. then when a function node is removed by remove-useless-instructions, it would be straightforward to remove the subdag rooted at the removed function from the main dag
23:55:36
karlosz
all remove-useless-instructions needs to do is return a list of instructions it removes
23:56:11
karlosz
and then if all references to a closure in that list are gone, then the subdag corresponding to that function can go away
23:57:28
karlosz
in theory, there should be no reason why the function dag needs to be entirely recomputed when something gets inlined
3:11:09
Bike
partial dags are something i tried before but i boned it somehow. i'm sure it's possible to do correctly, though.
3:11:54
Bike
for the slot disappearing thing, that's probably on me - we just started using my fastgf compiler in earnest, and i didn't think very hard about class slots. i suspect we could get a more reproducible error just fucking with class slots a bit.
3:14:30
Bike
drmeister, could you use all that data and see what CL or CORE functions show up a lot? today i noticed zerop wasn't inlined and i'm wondering if there aren't some other gains to be made in that area
3:15:43
Bike
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/cmp/cmpfastgf.lsp#L487 oh. yeah. this is stupid and broken
3:35:38
Bike
i'm specifically thinking i know the ins and outs of booting well enough that i can try the make-instance thing again
4:05:24
drmeister
C++ifying map-instructions-xxxx shaving 25 min off the build suggests the value of tightening up the code
4:07:25
Bike
i thought the point of your C++ified code was using slots in the instructions rather than allocating hash tables.
4:08:22
drmeister
It's two things - adding a 'touched' slot in the instructions and reducing the overhead of map-instructions-xxx
4:10:20
Bike
the only thing in map-instructions-arbitrary-order that seems like it could have a type check elided is the car in (pop instructions-to-process)
4:11:09
Bike
the compiler doesn't use much arithmetic or vectors, which are things where removing type checks would really help. if type inference improves compiler efficiency, it's because it would reduce the amount of code to be compiled
4:11:56
drmeister
You may be right - we can compare the llvm-ir for the C++ified versions and the Common Lisp versions.
4:14:51
Bike
whereas with the make instance thing, i know we have shitloads of make-instance calls and if i do it right they'll skip most of the work they do currently
4:18:21
drmeister
I've added karlosz changes on top of mine. Building -- I'll have new timing tomorrow. But this may bring us to parity with the AST compiler
4:22:54
Bike
i can probably also do a satiation thing where the ctors are in an initial state where they use functions compiled beforehand
4:29:20
Bike
the former of which has a variable class, so there's not a lot that can be done, which is even explained in a comment
4:29:44
drmeister
::notify kpoeck In the cl-bench tests - is there a way to get a spreadsheet view? It's difficult to scroll around. It would be a good idea to run it a few times to ensure all generic function discriminators are compiled.
4:30:50
drmeister
No - every generic function has its own counter - cleavir triggers them in batches
4:32:43
drmeister
it will pause in the middle a couple of times. If you do it again it will pause less and again - not at all.
4:33:48
drmeister
There is a lot to be said for beach's suggestion to put a somewhat random timer in there.
4:34:13
Bike
should put in a mode that's like the fastgf log except all it does is dump something when there's a compile
4:37:52
Bike
literally all you have to do to add a satiation case is add a list of the actual classes to the satiation form in satiation.lisp
4:38:40
drmeister
That would be very attractive - an automated way to set up satiation so that startup is faster.
4:39:06
Bike
https://github.com/clasp-developers/clasp/blob/master/src/lisp/kernel/cleavir/satiation.lisp not really automatable, since i do a ton of macrology in this file
4:41:47
Bike
but i mean, i guess? satiation overwrites the funcallable instance function, so the interpreter is unceremoniously dumped
4:42:33
Bike
last time i tuned up satiation the only calls that were compiling were like, some stuff in eclector
4:42:55
drmeister
Well, I love the idea of generating satiation code automatically by running clasp without it and then using the input for another build.
4:43:44
Bike
i mentioned it more as a cute idea than as something we should actually do. the main problem with it is that starting the system doesn't actually stress cleavir enough
4:44:18
Bike
you want to satiate the whole compiler, not just what we happen to use, or else the user will see random long compiles
4:44:53
drmeister
I'm building now with karlosz's latest changes. Then tomorrow I'll switch back to CL map-instructions-xxx with the 'touched' slot in instructions and see how that impacts things.
4:45:43
karlosz
if i remember correctly, beach would even be okay with adding a touched slot if it were done with stealth mix-ins
4:47:39
Bike
anyway, if we take a look at what dispatch misses are happening we can evaluate things better.
4:48:09
Bike
oh and a totally unrelated thing, i realized during brahms that if we do the multiple entry points fastcall thing we can use that to do tail calls.