freenode/#clasp - IRC Chatlog
Search
19:03:53
drmeister
stassats: Thank you for that advice - I can recover now from child threads dying.
19:06:10
drmeister
It compiles the AST's serially and then launches threads to do the AST->HIR->MIR->LLVM-IR->native-code in parallel.
19:06:51
drmeister
There is clearly some competition going on by the threads - but I don't know what. I'm going to try to swap in tcmalloc for the regular malloc.
19:08:04
drmeister
We still have to implement a bytecode interpreter for literals - we have it figured out - we just need to implement it. That will improve things as well.
19:08:31
drmeister
I came up with a scheme where the C++ byte code interpreter will be automatically generated by the Common Lisp code.
19:12:14
drmeister
I've got one really ugly thing in this parallel compile-file business - it's to initialize child threads special variables with the value from the parent. It has a bad code smell to it but I'm not sure what else I could have here.
19:17:15
drmeister
Hmm that might work. My understanding is that the forms get evaluated in the child thread - does what you suggest work with that?
19:18:24
drmeister
The bordeaux-threads documentation on *default-special-bindings* isn't clear about where the forms get evaluated.
19:23:17
drmeister
Bah! I get an assertion failure in the parallel compile-file when compiling babel.
19:24:20
drmeister
There's more to this than I have time to deal with right now. I thought I could get it working quickly and benefit from faster compiles but I'm getting anxious about how much time this is taking and how deep I seem to be getting into the compiler rabbit hole.
19:24:56
Bike
if you want to take a break i could maaybe look at it, probably after the loading machine
19:25:48
drmeister
Well, I thought it would be straightforward to go from the AST on in parallel - but there is stuff going on with load-time-values.
19:27:07
drmeister
https://github.com/Bike/SICL/blob/master/Code/Cleavir/HIR-transformations/segregate-lexicals.lisp#L212
19:27:25
drmeister
https://github.com/Bike/SICL/blob/master/Code/Cleavir/HIR-transformations/segregate-lexicals.lisp#L176
19:34:42
drmeister
I implemented your suggestion above and I'll try it running in serial and see if the same problem shows up. It would be illuminating to know if it's some kind of out of order thing or a general problem with the way the compilation is broken up now.
19:35:31
drmeister
I'll try serial compilation, then threaded but one thread at a time and then all threads going nuts.
19:36:54
drmeister
What was that thing that you mentioned about telling asdf that systems are loaded when they are loaded a different way than how asdf would load them?
20:04:38
drmeister
Huh - that's surprising - compile-file-parallel works fine if I run the ast->hir->...->native-code in the parent thread - so breaking up the compile-file this way is not the problem.