freenode/#clasp - IRC Chatlog
Search
12:41:14
kpoeck
Some exiting progress of Bike and karlosz regarding Cleavir (Version 1). For months I coudn't compile maxima, but now that works again due to improved inlining
12:42:44
kpoeck
It seems to discussions of Bike and karlosz are very productive (although hard to follow)
12:44:14
kpoeck
Regarding performance compiling "generated-encodings.lsp". I redid a flamegraph on macosx, https://kpoeck.github.io/out-37324.svg
12:45:15
kpoeck
Whatever the reason is, I only get helpfull flamegraph if I use "CLASP_BUILD_MODE = 'object'"
12:46:24
kpoeck
drmeister said it should work for CLASP_BUILD_MODE = 'faso', but it doesn't really work for me, svg are generated, but the content is reduced (the lisp part seem to be missing)
12:48:07
kpoeck
I also tried metering with Swank-metering. The results says that most of the compilation time is spent in CLASP-CLEAVIR::SIMPLE-EVAL-CST
12:50:04
Bike
equivalent to saying most compilation time is in cst-eval, aka evaluating things in eval-when, i think
12:53:49
kpoeck
(time (ext::process-encodings-file)) -> Time real(2.997 secs) run(2.997 secs) consed(384374048 bytes) unwinds(69414)
12:59:30
drmeister
kpoeck: There is another option that you need to provide to the build to get symbols in backtraces.
13:00:41
drmeister
That is because the unix 'backtrace(...)' function doesn't have access to any info for JITted frames and interpreted frames.
13:01:31
drmeister
So DEBUG_JIT_LOG_SYMBOLS writes a file to /tmp/perf-<pid>.map and the flame graph tool uses it to fill in missing symbol information.
13:02:00
drmeister
It's an extra file that clasp generates and so only developers will need to turn it on.
13:03:56
drmeister
I ran into a problem building cando - it hangs when it's installing it - I'm trying to figure out why. (sigh)
13:06:05
drmeister
To integrate cando into clasp someone would clone https://github.com/cando-developers/cando.git extensions/cando
13:06:58
yitzi
I know...the clasp stuff used to be on separate RUN lines. I just left the cando line there as a reminder to do it at some point.
13:06:59
drmeister
./waf build_cboehm needs cando installed in the clasp directory hierarchy before you build everything - it will build the extension into clasp automatically.
13:08:29
yitzi
Have you committed your changes? Maybe I can try after I get back from my morning walk if you haven't figured it out yet.
13:10:27
drmeister
This is what I've done so far - I added the lines to install cando. It's hanging on: RUN ./waf install_cboehm
13:11:20
drmeister
I think it's trying to build the quicklisp code for cando but 'waf' - our build system that we all love to hate suppresses the output and so we have no idea where it's hanging. Thanks 'waf'.
13:15:52
yitzi
It could? No idea really. The installed .clasprc references the quicklisp installed by SBCL earlier in the dockerfile
13:16:07
drmeister
Also cando doesn't need any quicklisp local packages at that stage - it supplies it's own quicklisp systems and downloads others from the standard quicklisp repositories.
14:15:42
kpoeck
Bike cleavir-env:cst-eval seems to be called for every top-level form, see https://gist.github.com/kpoeck/c7124d975e99806570262041940b9f9c
14:16:24
Bike
that's not every toplevel form, it's the eval-whens in the toplevel forms in the file you picked
14:19:52
Bike
all of thse hopefully don't even get to the ast interpreter, sincce they're just progn, quote, and function calls
14:20:37
kpoeck
Didn't take implicit eval-when into account, so now I finally understand what you said
14:56:50
drmeister
It looks like we haven't submitted a pull request for bordeaux-threads. I'm going to submit one.
14:59:26
Bike
alright, if i have the ast interpreter check for interpretability ahead of time, the compile time drops from "100s-120s" according to karlosz to like 80
14:59:50
Bike
for asdf. still more than the compiler, so it's still stupid. if i can get the compiler to work on an ast directly that ought to be fixable
15:10:38
kpoeck
drmeister I honestly believe you made a pull request to bordeaux-threads with things there are already in there
15:14:16
kpoeck
drmeister your pull request only touches the file bordeaux-threads.asd adding clauses that are already there
15:14:48
kpoeck
Please look at https://github.com/sionescu/bordeaux-threads/blob/master/bordeaux-threads.asd
15:14:56
drmeister
Yeah - I'm looking at it and I don't get why github thinks those changes need to be made.
15:19:16
Bike
i don't understand. the clasp change is in sionescu's repo, right? and that's what quicklisp uses.
15:19:50
kpoeck
But in quicklisp for every single component one can specify whether a release is taken, or current git
15:20:47
drmeister
Excellent - something good came out of this. As kpoeck said - Stelian said one minute ago that he'd do a release.
15:21:19
yitzi
drmeister: so moving the clone of bordeaux-threads earlier in the Docker should fix it?
15:21:45
kpoeck
Look at https://github.com/quicklisp/quicklisp-projects/blob/master/projects/bordeaux-threads/source.txt
15:22:01
kpoeck
There it says latest-github-release https://github.com/sionescu/bordeaux-threads.git
15:22:20
drmeister
yitzi: The issue is that bordeaux-threads on quicklisp is out of date and so we always have to install a fork of it. I. GET. BIT. BY. THIS. EVERYTIME. I. INSTALL. CANDO. ANYWHERE.
15:24:35
Bike
"release" being like a specific thing sionescu has to do. quicklisp isn't just using the latest commit in the repo.
15:25:27
kpoeck
Shinmera software si configured to use the lates commit on git, I like that better e,g. https://github.com/quicklisp/quicklisp-projects/blob/master/projects/dissect/source.txt
15:26:01
drmeister
Ok. Now I understand. How do we figure out what the latest release looks like other than wiping out my local-projects/bordeaux-threads and pulling one from quicklisp.
15:26:19
drmeister
It's still going to take days or weeks for this to resolve - even if he releases - right?
15:28:18
kpoeck
in my local project of bordeau-threads i did git remote add upstream https://github.com/sionescu/bordeaux-threads.git
15:29:43
drmeister
yitzi: Right now I'm putting everything after building cando because that takes a couple of hours.
15:30:23
kpoeck
Unfortunately no, works fine on macosx and your linux box, but is freaking slow on the buildbot
15:33:11
drmeister
Yes - I'm working on a Dockerfile to build cando with jupyter notebook/jupyterlab
15:34:23
yitzi
drmeister: I've started rebuilding on my end by moving the clone of bordeaux-threads earlier to see if that works.
16:28:28
drmeister
yitzi: Excellent - thank you. I tend to knock these together to avoid long delays and then rearrange them once it does what I want.
16:51:45
kpoeck
drmeister Bike as an immediate solution to the "generated-encodings.lsp" problem I could lazily load the data. That should make the build much faster, but would be a 3 seconds delay on first use