freenode/#clasp - IRC Chatlog
Search
13:32:00
Colleen
Bike: drmeister said 7 hours, 9 minutes ago: I fixed collect-macro-forms in clasp's version of swank.
13:32:00
Colleen
Bike: drmeister said 7 hours, 8 minutes ago: On discord they said that SJLJ does honor cleanups - really!?!? What do you make of that?
15:51:54
cracauer
`Symbol named "LOCK" is not external in the MP package.` when compiling bordeau threads.
16:11:22
Bike
i don't think it was even working before. mp:lock existed as a type, but the type of mutexes was actually mp:mutex
16:36:15
drmeister
I use dtrace to save the backtrace for every call to __cxa_throw. Then I invert the backtrace and truncate it at say 20 frames. Then I generate a flame graph.
16:37:32
drmeister
https://github.com/Bike/SICL/blob/master/Code/Cleavir/CST-to-AST/convert.lisp#L32
16:41:19
drmeister
I fixed collect-macro-forms in slime/swank/clasp.lisp last night - pull the latest slime to get the change.
17:37:05
beach
Sorry, I can't wait for the answer. It is time for me to go fix dinner for my (admittedly small) family.
18:04:56
drmeister
beach: Sorry - we were rearranging some code. There are cases where cleavir relies on stack unwinding for the normal code path where with a little rearranging it can use normal returns.
18:04:56
Bike
beach: What he pasted is just one of the unspecialized methods in Cleavir. Macroexpanded, of course
18:19:04
Bike
so meanwhile in personality land, the dwarf eh tables parsing is working, miraculously enough, but there's something weird with the type IDs. The ID we get from the vtable is like 1, but the IDs generated in the code are like 4552729896
18:29:25
drmeister
Bike: Now we are down from several thousand throws to 67 in one second of profiling.
18:35:19
Bike
one-potential-inline mapcs a function that unwinds. i don't know why the flame graph is wrong
18:35:49
Bike
one-potential-inline does basically a tree search and i think unwinding is appropriate there
19:03:01
Bike
Do you think you could just cut paste it for now? I'm in the middle of something else.
19:25:30
Bike
and, as i said, i think the one-potential-inline code is unwinding in a reasonable way, to exit early from a tree search
19:28:14
Colleen
kpoeck: drmeister said 13 hours, 5 minutes ago: I fixed the bordeaux-threads problem (I hope) I changed mp:lock to mp:mutex. I pushed it to clasp's local-projects version of bordeaux threads.
19:28:59
kpoeck
I did that change too, but now have trouble running the test, basically the test condition-variable seem to hang
19:34:36
Bike
the change i made to condition variables was giving them a __repr__ that displayed their name
19:38:35
Bike
well, it's a basic condition variable. It hangs until the condition variable is notified (with `pthread_cond_notify` or `pthread_cond_broadcast`, I think)
19:42:41
kpoeck
There is a bug report that the test for condition variable is bad https://github.com/sionescu/bordeaux-threads/issues/65
19:45:06
kpoeck
https://github.com/sionescu/bordeaux-threads/blob/master/test/bordeaux-threads-test.lisp#L129
19:45:41
kpoeck
This (loop until (= i *shared*) do (condition-wait *condition-variable* *lock*)) looks wrong
19:46:21
Bike
that's the usual pattern for using condition variables. the problem is that the test expects the threads operate in a particular order
20:03:53
Bike
i have the personality function working to the point i can start the system, but it's still kinda borked
20:07:26
kpoeck
The MP:CONDITION-VARIABLE-WAIT only returns, if mp:condition-variable-signal is called?
20:09:30
Bike
::notify kpoeck our condition-variable-wait, -signal, and -broadcast functions implement this https://en.wikipedia.org/wiki/Monitor_(synchronization)#Condition_variables_2
20:12:00
Colleen
kpoeck: Bike said 2 minutes, 30 seconds ago: our condition-variable-wait, -signal, and -broadcast functions implement this https://en.wikipedia.org/wiki/Monitor_(synchronization)#Condition_variables_2
20:40:16
drmeister
We are probably already there in terms of what compile-file-parallel can get us. On Mac it’s 2.5-3.5x speed up
21:43:51
drmeister
dtrace is also a great tool here. If you have a situation where there are thousands of calls to something and then it fails. You can have it store backtraces for each call and then look at just the last one.
21:45:40
Bike
if std::terminate gets called thousands of times we're in a pretty weird situation, i think
21:45:47
drmeister
clasp/src/profiler/scripts/profile_throw whos you how to do it. Change the *cxx_throw*:entry to *whatever*:entry and it will save a backtrace everytime any function that contains "whatever" in the name is entered.
21:46:29
drmeister
I was thinking of the situation where the stack unwinds and you lose the information about whatever was doing a throw.
21:47:16
drmeister
I've setup wscript now so that we can build clasp in "object" mode but compile-file-parallel takes over once Cleavir is loaded.
21:57:12
drmeister
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/lsp/setf.lsp#L581
22:01:22
Bike
multiple-value-bind shouldn't make a function boundary. with the macroexpansion it will, but in cleavir we use a compiler macroexpansion instead.
22:01:31
drmeister
It did unwind during macroexpansion. I was thinking about babel's monster macroexpansion.
22:03:18
Bike
So you're saying the push macroexpander is unwinding. Not multiple-value-bind's or anything.
22:03:47
Bike
But it shouldn't be, because like I said, multiple-value-bind compiler macroexpands into something that doesn't involve any function boundaries.
22:06:29
drmeister
No - I'm building bclasp and for yuks I pointed do-flame-throw at it. I saw what I pasted above and then I started thinking about macroexpanders throwing exceptions and now I see that this wouldn't be a problem for babel for the reason you just described.
22:07:50
drmeister
I am poking around with this new tool to see if there are hidden speed bumps due to too much unwinding. Unwinding is bad in parallel - but it's also slow in serial code. You are working to improve that situation.
22:10:05
Bike
which i guess suggests that funwind protect is trying to rethrow an exception that doesn't exist
22:11:56
Bike
"Nested foreign exceptions, or rethrowing a foreign exception, result in undefined behaviour." oh.
22:12:35
Bike
Guess I'll have to do unwind protect with destructors instead. wasn't that how it went before? i'll check the commit log first
22:16:34
drmeister
If I remember correctly getting the semantics right was tricky. Destructors didn't... quite... work? I don't want to freak you out though - I may be wrong.
22:17:05
Bike
well i mean the existing unwind protect implementation works fine with the new throw, for whatever reason
22:17:32
Bike
oh, right. destructors wouldn't work since the unwind protect cleanup might itself throw, but a destructor throwing is N.G.
22:23:08
drmeister
Itanium exception handling is a bit more flexible than C++ exception handling. You need a 'finally' clause - right?
22:24:04
Bike
I don't know what I need. I don't understand why things are working or not working. The Itanium ABI doc seems to say rethrow of a foreign exception (e.g. by "catch (...) { ... throw; ... }") is illegal, but (a) that's not what I observe, and (b) that would be really stupid
22:26:01
Bike
Maybe. I already dumped in a big block of text describing our problems and nobody read it.
22:27:59
Bike
But since I don't actually understand what the problem is here I'm not sure what to ask
22:30:10
Bike
It's not just because the exception is foreign now - because then unwind-protect would never work at all. But it is.
22:31:17
drmeister
You mean core__funwind_protect is catching the foreign exception - but it looks like it just can't rethrow it.
22:32:42
drmeister
Could you drop by my office and explain this? I've lost track of what case is failing.
22:43:04
drmeister
Building cl-ppcre on linux with compile-file-parallel and I'm getting between 150% and 450% CPU.
22:47:12
drmeister
So I still didn't get it - can we use the SJLJ stuff? Can we make it work with C++?
22:49:31
Bike
there's the SJLJ that means the setjmp and longjmp operators, and there's the SJLJ that means implementing C++ try/catch using setjmp/longjmp
22:49:49
Bike
the latter is kind of irrelevant to us, but it's what that guy was referring to yesterday.
22:51:06
drmeister
How does one implement C++ try/catch with setjmp/longjmp? That's a compiler switch?
22:51:32
Bike
yeah, -fsjlj-exceptions, and you also have to rebuild a bunch of system libraries like libunwind
22:52:17
drmeister
I see - you can't load a dynamic library that was not built with -fsjlj-exceptions.
22:53:11
Bike
i mean, probably. but this is basically irrelevant to us because we're not implementing C++
22:54:33
drmeister
I thought it could be relevant if we restricted ourselves to only use code that was compiled from C++ source using -fsjlj-exceptions and we changed our exception handling to use llvm sjlj intrinsics.
22:55:39
Bike
It's not relevant to us because we'd still need to call exception handlers and everything.
23:00:56
cracauer
Bike: did you ever remove or rename those files with '@' in the filename in your copy of the SICL tree? That keeps us from running with some customers.
23:01:48
drmeister
cracauer: We are seeing the 2-3x speedup with compile-file-parallel now. Once Bike pushes a change to sicl we should be able to start the buildbot with the new changes.
23:04:39
cracauer
The @ is actually not the dealbreaker, but either way it would be good to have the filenames normalized with no special characters, and maybe even no sppaces.
23:05:09
Bike
alright. right now i'm building with the sicl changes to double check everything's fine. I'll just delete that entire directory in my branch
23:14:16
drmeister
cracauer: Do you have a wscript.config file on the buildbot or are you using the default settings?
23:20:56
drmeister
Ok. I've made the default on linux and freebsd CLASP_BUILD_MODE = "object" . USE_COMPILE_FILE_PARALLEL=True
23:21:59
drmeister
Lang Hames is taking the linker in a very nice direction. The runtime linker is going to behave like the system linker.
23:26:16
drmeister
What it's building now will be the first time we had the buildbot building the quicklisp code with compile-file-parallel in a way that could speed up the build.