freenode/#clasp - IRC Chatlog
Search
14:36:34
selwyn
i haven't had time yet to look at the flamegraphs - could the slow backtraces have been responsible for the poor flamegraph output? how do they look now?
14:55:41
Bike
drmeister: https://github.com/slime/slime/blob/master/swank/clasp.lisp#L710 do you remember why the swank/clasp locks are recursive? things seem to work fine with non recursive locks
15:15:49
Bike
there's an entire other thing for working atomically that i didn't know about. AtomicTHolder. cool
15:28:43
Bike
if i do (nth-value 1 (ignore-errors (funcall (block nil (lambda () (return)))))) clasp still dies aaaaagh
16:12:46
Bike
drmeister: https://github.com/slime/slime/blob/master/swank/clasp.lisp#L710 do you remember why the swank/clasp locks are recursive? things seem to work fine with non recursive locks
16:19:57
drmeister
I don't recall - I think since I didn't have good data at the time I just played it safe.
16:21:15
drmeister
I've got to tell you about our new compiler memory management scheme. We were leaking llvm compiler objects. We aren't leaking them now - but there is some danger that hasn't presented itself yet.
16:21:30
Bike
I ask because while I'm doing this I'm normalizing some interfaces and removed the `:recursive` keyword.
16:21:50
Bike
so if i merge it will break slime, which is bad, unless we remove that from swank/clasp.lisp
16:23:03
Bike
i mean what i did is look at the other implementation's swanks, and they're not recursive locks
16:24:24
drmeister
Hmm, I just realized our llvm C++ object cleanup scheme is more robust than I feared.
16:25:16
drmeister
Nothing can get collected until we run (gctools:thread-local-cleanup) and then only if the GC collected its wrapper.
16:27:34
drmeister
We have some quicklisp systems for which compile-file-parallel is pathologically slow. I'm trying to figure out which ones.
16:32:00
Bike
also things like run-dsymutil that i'm skeptical of the use of for programmers. but i can just not put those in the manual for now
17:51:50
kpoeck
in https://kpoeck.github.io/report.html are cl-bench results of different lisps/clasp-releases
17:53:21
kpoeck
The two one from the left are clasp as of 3rd of january with the bignum gc fix and without
17:56:45
kpoeck
Will try to bisect to find out where that started, will first go back to monday morning last week, since a lot was merged that day
19:03:33
Bike
(char-name (code-char 161)) => "UA1". sbcl gives "INVERTED_EXCLAMATION_MARK", which I'd say is better. Of course that would mean including a couple kilobytes of UnicodeData.txt or whatnot
19:53:47
Bike
drmeister: https://pastebin.com/L3NgX0Ck manual sketch. Mostly just wanna see if it has all the extensions Clasp has - I can't think of more. You can throw it at https://markdownlivepreview.com/ or something to see it rendered
20:28:00
kpoeck
Perhaps add profiling with flamegraphs? Debugging with (core:btcl), (core:safe-backtrace) & (CORE:BT-ARGUMENT <frame> <index>)
20:29:54
Bike
we ought to have an actual backtrace interface, like a backtrace object and navigating frames
20:31:46
kpoeck
I was starting to implement Shinmera dissect , but got distracted. He also has this nice overview table with capabilities of the different cl implementations
20:39:32
kpoeck
in revision b4c487592b91cd3ef1a24908320d893148dc8311 ctak is still fast, but CLOS/methodcalls either are really slow or hang
20:50:29
Shinmera
kpoeck: Oh, I was going to do dissect at some point myself, but I'd be very happy if I didn't have to! :)
22:14:16
drmeister
I've got this Unwind_Find_FDE issue under a microscope - if you are still here for a few min - could you drop by and take a look at it with me?
22:15:01
drmeister
Backtraces are truncated. flame graphs don't work. Functions I can figure out are all called LAMBDA
23:53:18
drmeister
I also write a C++ example program and look at the speed of unwinding and it's nowhere near as bad as clasp.
0:18:47
drmeister
Bike: You understand the unwinding stuff better than I do - do you have an idea for how to fix this?
0:19:36
Bike
doesn't this just indicate that whatever unwind.cc is isn't doing unwinding in the same way clasp is?
0:22:01
drmeister
There doesn't appear to be a difference between libstdc++ and libc++ with this demo - not a significant one.
0:22:24
Bike
right. so i'm saying it seems likely to me that this means the demo isn't an accurate reflection of clasp.
0:23:11
drmeister
Yes - I'm with you there . That's where my vague "But I’m wondering if we are doing even worse than that" came from.
0:23:11
Bike
unless there's some other reason for there to be this gulf between linux and mac performance
0:23:34
Bike
like mac performance seems okay to me. it's not amazing or anything but there hasn't been this botteleneck, yeah?
0:24:22
drmeister
I can't link libc++ with clasp on linux - it throws up all kinds of missing symbols - I'm not ssure why.
0:25:42
Bike
and like, i've been talking about writing our own unwinder - and i still think that has some advantages - but Unwind_Find_FDE, the big offender, is called by Unwind_RaiseException, isn't it?
0:25:55
Bike
Unwind_RaiseException is exactly the underlying function our unwinder would still be calling
0:30:42
Bike
https://github.com/gcc-mirror/gcc/blob/master/libgcc/unwind-seh.c#L328-L330 and here's the source
0:33:24
drmeister
https://gcc.gnu.org/onlinedocs/gccint/Exception-handling-routines.html#Exception-handling-routines
0:33:58
drmeister
Right - but any stack unwinding has to call these - are they the reason for the slow down.
0:36:24
Bike
there are two problems with unwinding. one is general slowness, independent of OS or implementation. that is what i have in mind when i talk about writing an unwinder.
0:36:54
Bike
then there's this Mac/Linux thing, which is apparently due to Linux's implementation having this stupid linear search in it. Writing our own unwinder wouldn't help with that, probably.
0:40:13
drmeister
(defun foo (num) (dotimes (i num) (block hhh (funcall (lambda () (return-from hhh))))))
0:58:35
drmeister
(defun foo (num) (dotimes (i num) (block hhh (funcall (lambda () (declare (core:lambda-name inner-foo)) (return-from hhh))))))
1:06:33
drmeister
Question though - the consing - are we allocating a closure on the heap or on the stack in this case?
1:08:27
drmeister
Here's a thought - we are doing a lot of unwinding. Unwinding is allocating closures on the heap. Allocating on the heap involves a mutex in the GC.
1:09:54
drmeister
Unwinding requires a closure to pass the closed over handle for the the unwind target?
1:10:28
Bike
ok like, this is all deep and confusing stuff and we need to be exact. when you say "unwinding" i think like, what happens, at runtime, when you hit a return-from
1:12:08
Bike
anyway, although the underlying "unwind tag" has dynamic extent, the closure itself can still have unlimited extent, so we can't stack allocate it.
1:12:21
Bike
we could stack allocate the particular cell for the "unwind tag", but we shouldn't even have a cell anyway really
1:45:34
drmeister
On macOS (time (foo 10000)) -> Time real(3.058 secs) run(3.057 secs) consed(0 bytes)