freenode/#clasp - IRC Chatlog
Search
13:16:36
drmeister
::notify Bike If the gfvm is still slower than the version that I wrote there are some ways to implement interpreters with jump tables in C++ that could improve performance. It's not standard C++ but gcc and clang support arrays of goto labels that I think can speed up implementation of bytecode interpreters. Also we could switch to byte code rather than the simple-vector's that you are currently using.
14:05:42
drmeister
That looks like it goes beyond what I said previously in terms of writing efficient interpreters.
14:06:29
Colleen
Bike: drmeister said 49 minutes, 53 seconds ago: If the gfvm is still slower than the version that I wrote there are some ways to implement interpreters with jump tables in C++ that could improve performance. It's not standard C++ but gcc and clang support arrays of goto labels that I think can speed up implementation of bytecode interpreters. Also we could switch to byte code rather than the simple-vector's that you are currently using.
14:06:29
Colleen
Bike: drmeister said 1 minute, 33 seconds ago: http://www.emulators.com/docs/nx25_nostradamus.htm
14:07:21
drmeister
That means that build times are really sensitive to gf dispatch efficiency - right?
14:08:03
drmeister
Oh yes - and I'd like you to turn that back on because startup times in cando/cleap are long.
14:08:36
Bike
anyway, now that build time is back down and things seem to be working, should i merge this into dev?
14:13:23
drmeister
Check out that nostrodamus blog posting. They say a lot more about emulators after computed goto.
14:14:01
drmeister
They say when you look at the generated code for gcc it wipes out the advantage that computed goto is supposed to deliver.
14:15:35
Bike
honestly i'm pretty good with 33 minutes and hopefully turning the build satiation back on. don't want to do anything too extreme
15:05:31
drmeister
::notify scymtym: I ran into a problem with esrap when building cando. The latest version of esrap appears to have a bug where a (declare ignore ...) expands in a macrolet to somewhere other than at the top of a form. I posted an issue on github.
15:06:17
drmeister
I have a quick fix to comment out a line. Once I hear back from scymtym I'll decide how to proceed - because this breaks the build.
15:12:21
drmeister
The Dockerfiles for setting up the docker image are here. They may be a little out of date - but not too much.
15:13:49
drmeister
clasp/tools/dockerfiles/* . - well, they are more than a little out of date - but with a little time investment you could bring them up to date.
15:14:11
wuehlmaus
perhaps i have to learn how to roll my own clasp without docker, i would like it even a bit more on my Mac
15:17:34
drmeister
Yeah - we are a bit short handed here and have a lot on our plate. But if you have specific questions - feel free to ask.
15:19:12
drmeister
Currently we use llvm6 and you can install it with a package manager. Check the github wiki page on installation.
15:19:14
Bike
I got "1 is not of type (integer 0 0)" while compiling architecture.builder-protocol while starting cando
15:20:53
drmeister
Bike: That is the &rest think. Delete your copy of architecture.builder-protocol and let quicklisp pull in a new one.
15:41:31
drmeister
I'd say write two special operators core:read-header-stamp . core:read-complex-stamp
15:42:21
drmeister
Then we implement llvm-ir generators and run them alongside the C++ ones and match them with each other. Once we are sure they work then we remove the C++ ones.
15:43:21
drmeister
We also need special operators to get the next argument and rewind the vaslist - right?
15:44:26
drmeister
I'd say we write intrinsics for everything first and then generate llvm-ir. I'd like something that works first and then generate llvm-ir and match it to the stuff that works. I'm done with segfaulting due to bad llvm-ir.
15:45:17
drmeister
It's more work - but generating llvm-ir in the first go is guaranteed to cause headaches - I've never done it right the first time.
15:45:48
Bike
ok, well, that should be pretty easy then, since the read stamp code already exists in C++
15:46:29
drmeister
So I've started writing a prototype in C++ and then llvm-ir and then generating them BOTH into the code at the same time and taking the results and comparing them to each other with cc_match
15:46:52
drmeister
If they don't match I print a message and use the old C++ value that we know works.
16:03:47
cracauer
It will also print what is happening during quicklisp compilation during install_cboehm
16:04:49
cracauer
the structure for fetching git trees does not kill local modifications unless you switch branches
16:06:40
drmeister
Should we do that. Even when scymtym fixes it (I haven't heard from him yet) it will take weeks for it to be incorporated into quicklisp.
16:07:42
cracauer
we could also retrieve an older, working version via the git checkouts and put that into local-projects
16:11:45
cracauer
Right now it works as users would expect when cando/jupyter people use ./build.sh and other people use ./waf directly
16:14:53
cracauer
3: whether quicklisp compilation (if any) goes into a static directory (not ~/.cache)
16:15:39
cracauer
when it is acting as a distribution builder is uses /opt/clasp/lib/quicklisp and no relocation of fasl files
16:16:33
cracauer
even if you use it as a distribution builder you can request homedir and .cache use by putting CLASP_QUICKLISP_HOMEDIR=1
16:17:22
cracauer
well, I think for now we are good. People who don't want cando and jupyter just use waf, which works as before
16:17:56
cracauer
people who want cando and/or jupyter use build.sh which pulls in both by default, and gets configured via a file
16:18:58
cracauer
As far as defaults go, I think just renaming build.sh to a name indicating that it is for pulling in more stuff will do
16:21:26
cracauer
I guess I need to enforce that the proper env variables are set when the leap executable is being called, though.
16:22:10
drmeister
Can we set things from within Clasp - so we don't have to rely on environment variables?
16:24:01
cracauer
It's what chrome, firefox etc use, too. It is just much easier to debug and if needed to adjust (compared to being inside the binary)
16:25:52
cracauer
for the jupyter builds this is already somewhere in jupyter startup. That's how the docker jupyter has been working for a long while.
18:52:39
drmeister
cracauer: Cool - so can I pull the deploy script and build an /opt/clasp directory on my system?
19:04:50
cracauer
oh and we don't have a shellscript wrapper to set env variables around the lisp executables yet
19:16:48
drmeister
an engineer at Apple or Google (I dunno) - but he's contributing patches to get lldb to work with the gdb/jit API that I haven't gotten to work yet.
19:17:21
drmeister
He's got some demo code that I need to build because it should work. We could check if that also means that profiling its jitted code on linux works.
21:28:03
drmeister
cracauer: Do you know what the expression in esrap that miscompiled with that declare looks like? I didn't have time to debug it and I should provide some more details to scymtym.
21:28:53
drmeister
I did see it crash - it printed blank lines in an endless loop. I did comment out the (declare ignore ...) and my build is working because of it.
22:10:00
drmeister
Sorry - it's been pretty consistent for the past couple of months since we switched to the CST compiler. Bike made some changes to the GF dispatch (he implemented a little generic function virtual machine gfvm).
22:14:08
drmeister
I'm repeating myself - but Nintendo switched to clang and llvm in the past two years.
22:21:59
drmeister
Ha ha - certainly not. They were excited about a 3% increase in performance with BotW when they switched to clang from their proprietary C++ compiler.
22:26:06
karlosz
3% is quite a lot from just changing C compilers... did they use gcc before or something?
22:27:02
drmeister
karlosz: They said they had proprietary tools. They did complain that they didn't like losing access to lexical variables in debugging.
22:27:15
cracauer
I'm going nuts. I have 4 different machines testing different variants and wanting modules and quicklisp locations. I had to start all of them over fro scratch
22:28:03
drmeister
Are we going to have to do some genetic programming to figure out the optimal configuration?
22:30:19
karlosz
especially optimizations which merge instructions and cant merge the line information cleanly
22:31:06
cracauer
that's not what I meant. 27 or 54 minutes is the same damage. The problem is broken trees.
22:37:36
drmeister
We gotta get ASDF to do parallel compilation. It's just way too slow compiling files one after the other.
22:53:51
karlosz
even better: has anyone made flamegraphs for generate-ast vs cst compiler to compare?
22:54:50
drmeister
I'm talking with an engineer about the gdb/jit interface on linux - I suspect but don't know that if I get that working that perf should work as well.
22:57:38
karlosz
anytime anyone complains about the compiler speed i get an itch to look at it again - just because i know it doesn't have to be so