freenode/#clasp - IRC Chatlog
Search
22:21:59
drmeister
Ha ha - certainly not. They were excited about a 3% increase in performance with BotW when they switched to clang from their proprietary C++ compiler.
22:26:06
karlosz
3% is quite a lot from just changing C compilers... did they use gcc before or something?
22:27:02
drmeister
karlosz: They said they had proprietary tools. They did complain that they didn't like losing access to lexical variables in debugging.
22:27:15
cracauer
I'm going nuts. I have 4 different machines testing different variants and wanting modules and quicklisp locations. I had to start all of them over fro scratch
22:28:03
drmeister
Are we going to have to do some genetic programming to figure out the optimal configuration?
22:30:19
karlosz
especially optimizations which merge instructions and cant merge the line information cleanly
22:31:06
cracauer
that's not what I meant. 27 or 54 minutes is the same damage. The problem is broken trees.
22:37:36
drmeister
We gotta get ASDF to do parallel compilation. It's just way too slow compiling files one after the other.
22:53:51
karlosz
even better: has anyone made flamegraphs for generate-ast vs cst compiler to compare?
22:54:50
drmeister
I'm talking with an engineer about the gdb/jit interface on linux - I suspect but don't know that if I get that working that perf should work as well.
22:57:38
karlosz
anytime anyone complains about the compiler speed i get an itch to look at it again - just because i know it doesn't have to be so
23:11:02
karlosz
well, CST-TO-AST is taking a long time. if i remember correctly, CSTs represent non macro expanded code
23:12:36
karlosz
macroexpansion happens somewhere in between the creation of the original cst and the final ast level
23:12:46
cracauer
I was able to edit esrap-20190521-git/src/evaluator.lisp between it being retried but before compilation bombed.
23:14:13
drmeister
If I raise the crop level to 800 frames I can't render it. Chrome is working very, very slowly.
23:15:11
karlosz
nothing immediately algorithmic jumps out at me from the flamegraph, since it seems like its just walking a tree. so it looks like clos could be slow? it really shouldnt take so much time to convert csts to asts
23:15:53
Bike
is it macro EXPANSION that takes time, or building the macroexpanders themselves, because the latter has been a recurring thing
23:19:21
karlosz
getting rid of call-with-variable-bound is also going to help a lot with clearing up those frames
23:19:48
karlosz
but this is good. i never saw this when i profiled cleavir with clisp because i use clisp's own macroexpander
23:21:21
karlosz
its also a good candidate for the bottleneck precisely because we haven't seen the same issue with generate-ast, which doesnt have cst-to-ast at all
23:24:07
karlosz
cst-to-ast seems to be more than 50% of compile time for setf.lsp if im interpreting it correctly
23:31:25
karlosz
its bizarre especially because im pretty sure this was not in previous flame graphs
23:32:15
karlosz
with the previous flamegraphs my-hir-transformations was the only thing you could even see
23:39:54
drmeister
Bike: link-inline-remove-builtins is 14% - it's good that we are getting rid of it.
23:42:53
karlosz
but i mean yeah, i think we know now at least that cst-to-ast as a whole takes >50% of the compilation time for macro-heavy setf.lsp, so that's something
23:48:27
karlosz
yeah isolating macro functions and getting a minimal test case to profile would be good
23:51:28
karlosz
another general suggestion: this experiment has made me think that the cst-compiler isnt actually slower than generate-ast for the majority of files in the self build process
23:52:01
karlosz
only a few select files in the cst-compiler really blow up in compile time, due to it handling certain things differently
23:52:24
karlosz
so one thing to do would be to add compilation times to each compilation file built during the build process
23:52:44
karlosz
and compare the build times for each individual file to see which files slow down the most
23:53:14
karlosz
my hypothesis is that for most files, there isnt actually much slow down, but we will see a couple of outliers where cst takes much more time
23:54:18
karlosz
but i think the macro expanding thing would be a good first construct to investigate
0:20:48
drmeister
Sounds good - I have an emergency with a broken vacuum cleaner - I have to run to buy some screws.
0:34:35
kpoeck
my pick on adding compilation times to the build is in https://github.com/clasp-developers/clasp/pull/788
0:39:10
kpoeck
tomorrow - late here - i can do what karlosz prosed, compare compilation-times in the build for ast and cst
0:42:23
Bike
drmeister: could you look at https://github.com/clasp-developers/clasp/pull/772 ? i want to merge it but don't understand the problem
6:32:17
karlosz
::notify Bike do you mind reviewing the pull request i sent in to sicl about inlining i sent in a while ago? some feedback on the general approach would be nice because its been sort of a long standing issue in the compiler for a bit