freenode/#clasp - IRC Chatlog
Search
19:34:36
drmeister
I'm surprised - I thought that ast compilation would slow things down enough so that maybe 20-30 threads were created at a time.
21:23:16
drmeister
No - that's not what you are saying - there are no more than 10 threads at a time - that's been my experience.
21:25:25
drmeister
kpoeck_: I've been looking at compile-file-parallel - I see two things slowing it down (1) invocation of compile during AST generation (2) A still mysterious lock contention between the AST->native code threads and AST generation.
21:26:05
drmeister
I put it aside for a bit while I'm working on chemistry code and thinking about it.
21:28:04
kpoeck_
I assumed creating that many threads is expensive, but i committed the sin of not measuring it
21:56:53
drmeister
Since we did the x.characterp() test we don't need to use gc::As<Character_sp>(x)
21:58:06
drmeister
gc::As_unsafe<Character_sp>(x) is for when you KNOW x is a Character_sp and you just want the compiler to treat it like one - it's a zero cost transformation.
22:15:52
drmeister
And so now we should be able to compile the stock Mcclim - I'm building and testing now.
1:57:50
drmeister
I think that's what the problem is. If you run a tight loop that generates lots of threads and joins them I don't think the overhead is that high.
2:06:56
drmeister
So that's going to be the time it takes for the threads that are still compiling to finish.
2:08:35
drmeister
The contention that I'm seeing is a little hard to demonstrate. If you profile compile-file-parallel you will see that the top thread that compiles the ASTs spends a lot of time in llvm.
2:09:30
drmeister
So then I tell it to use the interpreter rather than the compiler when generating ASTs.
2:16:30
drmeister
So THEN I tell it to not launch threads to compile the ASTs to native code and then the AST generation is 10x faster
2:17:47
drmeister
Bike: I'm still having trouble saving CLOS objects with the printer in a readable way- what did I need to do to add that for an object?