freenode/#clasp - IRC Chatlog
Search
12:18:51
drmeister
I changed the file that I compile-file for this flame graph - I switched to format.lsp - it has a really long pause at the end when it is compiling the format compiler-macr
12:22:42
drmeister
I have a suggestion: (1) We temporarily add a slot to instructions that stores a 'visited' mark and write a new version of of map-instructions-xxxx that uses the 'visited' mark rather than a hash table to keep track of what instructions have been visited. I believe that this will cut inlining time by quite a lot.
12:23:09
drmeister
(2) We work on value numbering, type inference and removing useless code paths to tighten up the code.
15:53:29
kpoeck
karlosz: Changing to new sicl - with your patches- reduced build time for clasp - after wiping build - from Compilation finished in 5:24:51 to Compilation finished in 4:03:12, quite an improvement
16:56:16
karlosz
hello everyone. I have some more inlining improvements here: https://github.com/robert-strandh/SICL/pull/131
16:56:50
karlosz
i know you guys just did another rebase, but i observed another 2-3 hours improvement with these patches
16:57:12
Bike
the problem i had with set-predecessors is that we could inline a function that doesn't return, and that will cut off whatever the call dominates
16:58:07
karlosz
could you maybe take a look? i did some instrumenting and thought i got rid of most of that
17:00:18
Bike
well it's basically the same issue. we don't know what instructions we have to unhook from predecessors or definers/users
17:30:27
selwyn
drmeister: i'll find out the hard way - building with https://github.com/robert-strandh/SICL/pull/131 now
18:47:44
drmeister
Bike: We have a _SetfFunction slot - do you know if I use eval::funcall(_sym_touched->_SetfFunction,value,object) - will that set the value of the 'touched' slot?
18:49:10
drmeister
I think that's how it's supposed to work if I want to set the value of an objects slot.
18:49:36
kpoeck
karlosz: A flamegraph compiling asdf with your fixes that are already merged to sicl
18:57:26
drmeister
I know the name of the accessor is touched - so the setf function is (setf touched).
18:58:35
drmeister
I've rewritten map-instructions, map-instructions-with-owner and map-instructions-arbitrary-owner in C++ and switched from using a hash-table to using a slot called 'touched' in each instruction.
19:00:06
drmeister
I want to see if my hypothesis that overhead and using hash-tables to keep track of what instructions have been seen is responsible for a lot of the time taken by inlining.
19:20:50
drmeister
Bike: Did you add automatic generation of xxxx-instruction-p predicates? I thought you did but I don't see enclose-instruction-p
19:22:20
Bike
i guess with the dtree interpreter in place there's not as much worry of weird slowdowns, so i may as well
19:25:05
drmeister
selwyn: My new linux machine at home builds AST in about the same amount of time.
19:25:11
Bike
itll take me a bit to figure out since it involves anonymous generic functions and stuff
19:26:02
drmeister
FYI startup is a bit slower now because there are some compilations happening even when it only compiles the 1024th time a gf is called.
19:26:37
drmeister
Also - once you start compiling things there are big slowdowns as all the compilation chickens come home to roast.
19:31:09
Bike
like, i can put the thing to set up the predicate in initialize-instance for class, but that doesn't work for anything built in
19:32:05
drmeister
It's ok - I'm just hacking here - I added an enclose-instruction-p and enter-instruction-p predicate.
19:32:43
Bike
don't push it to the repo or anything, that caused me a lot of rebasing problems before
19:33:39
drmeister
I don't touch sicl - that's beach's and your domain - this is an experiment that I would be perfectly happy to see fail.
19:34:08
selwyn
speaking of chickens: at ELS i found out there is an attempt at chicken scheme <-> c++ integration
20:06:31
drmeister
It's not directly comparable to the other flame graphs I've posted because this is single threaded.
21:01:27
drmeister
Well, I'll be hornswoggled - the C++ version doesn't appear to offer any benefits whatsoever.
21:01:58
drmeister
I'm not sure - because I have to time an entire build - but the flame graphs look exactly the same!
21:02:40
drmeister
Yeah - they are different in the respects where I changed them - that's different.
21:06:17
Bike
it did decrease the reinitialize-data fraction by a percent or two. but yeah, doesn't seem to have helped much
21:09:21
karlosz
Bike: i'd like to investigate what you noted about predecessors and reinitialize data not being able to hook up when inlining functions that don't return
21:10:06
karlosz
selwyn: you pulled with the pull request i sent this morning right? do you think its possible you can make a flame graph?
21:13:00
Bike
off the top of my head, if you had like (progn ((lambda () (return))) (funcall f)) it could inline the first call and then try to inline the second
21:15:34
drmeister
Ah - yeah - I can make register_if_unvisited simpler - I wasn't thinking about improving the code - just C++ifying it.
21:16:26
drmeister
selwyn: Did it build ASDF? Or did it die when trying with a Floating point Exception?
21:18:08
Bike
i mean, like, what did you make it do? i thought setting a slot would be easier than setf gethash
21:18:12
drmeister
Actually - no - I can't make register_if_unvisited simpler - it has to conditionally build an instruction list.
21:19:34
selwyn
i think the second. it prints 'Floating point exception' and exits straight away. do you think it's worth running with the debugger?
21:19:59
drmeister
The C++ version has less GC going on - that's probably because there is less rehashing.
21:21:09
drmeister
selwyn: It might be the problem that Bike and karlosz are talking about - in which case lldb/gdb will not be very illuminating.
21:23:33
Bike
https://github.com/clasp-developers/clasp/commit/7f64024a81db6db8357dfb6e9658c474c981a981#diff-1f35b120724d1751acd6cd263fc20505R431
21:24:17
karlosz
i like to log other in magit and then just cherry pick the patches i want in case someone else broke dev
21:25:24
drmeister
selwyn: Then it might be a very good idea to run it under gdb/lldb and print a backtrace.
21:26:01
karlosz
kpoeck: then do git pull <remote name>/rui-v2 all while in src/kernel/lisp/contrib/sicl/
21:27:21
kpoeck
After an innocent (ql:quickload :cl-opengl), I get again Probably a BUG: slot SELFWARD-OPERATION in #<The STANDARD-CLASS ASDF/LISP-ACTION:COMPILE-OP> stopped existing between compile and load
21:27:30
Bike
so if the difficulty in map-instructions-arbitrary-order is the worklist that's going to be difficult to deal with
21:50:48
karlosz
i'll keep in mind to play with the heap size next time i get a floating point exception!
21:55:24
selwyn
karlosz: surely it's possible to do https://github.com/clasp-developers/FlameGraph with perf_events on linux?
21:55:58
karlosz
the code drmeister wrote to help make the traces from cleavir more readable are dtrace specific
21:56:43
karlosz
selwyn: but feel free to take a crack at it with perf_events. i personally don't understand either dtrace or perf_events
23:07:26
kpoeck
drmeister: I made a flamegraph in your new layout with latest cleavir patches from karlosz compiling cl-bench
23:08:22
kpoeck
Sicl from yesterday, but including karlosz patches from https://github.com/robert-strandh/SICL/pull/131
23:08:44
drmeister
Yes - it looks very nice - reinitialize-data is gone and there is a new thing compute-destinies
23:10:49
karlosz
drmeister: do a ./waf configure and then checkout my rui-v2 branch from src/lisp/kernel/contrib/sicl/
23:28:01
selwyn
Probably a BUG: slot SELFWARD-OPERATION in #<The STANDARD-CLASS ASDF/LISP-ACTION:COMPILE-OP> stopped existing between compile and load - from loading clim