libera/#clasp - IRC Chatlog
Search
14:45:12
drmeister
We can't fully get rid of LambdaListHandler_O until we fix the cases where its use has leaked out into the Common Lisp code.
14:46:30
drmeister
https://github.com/clasp-developers/clasp/blob/vm-boehm/src/lisp/kernel/lsp/defstruct.lisp#L361
14:46:51
drmeister
https://github.com/clasp-developers/clasp/blob/vm-boehm/src/lisp/kernel/lsp/defvirtual.lisp#L23
14:48:25
drmeister
Bike: I am removing LambdaListHandler_O carefully. I'm setting everything up with alternate code that works with bytecode wrappers - I'll do the methods.
14:49:06
drmeister
I want to set it up so we can switch from LambdaListHandler to BytecodeWrapper and then test everything to make sure it works with BytecodeWrapper before throwing the switch and removing the LambdaListHandler code.
14:49:34
Bike
i missed that we use lambda list handlers in defstruct and defvirtual, let me see about removing that
14:49:45
drmeister
yitzi: We need to add a static analyzer run to the github Actions. It's too important to leave for occasional running and then hitting problems like I'm trying to debug.
14:50:23
Bike
in defstruct we just use it to check for correctness. that's relatively easy to replace, at least
14:51:14
drmeister
Bike: Yes - that would be a good idea - that's more in your wheelhouse. It does interact with core:ensure-single-dispatch-method - so we will probably need to discuss it.
14:53:28
drmeister
The `:selection-pattern "commandLineOptions"` - that lets us limit the static analyzer to files that contain that substring.
14:54:05
drmeister
It still generates huge amounts of output and takes a while. The action will need to deal with that.
14:57:45
drmeister
And it's segfaulting deep inside the clang ASTMatcher code inside of private classes.
14:58:24
drmeister
It's awful. They don't have a common root class for everything (C++) so on the outside it looks simple - but inside they have all sorts of `if` statements and dispatching going on. Awful, awful, awful.
15:01:27
drmeister
So - I give it a `DynTypedMatcher &NodeMatch` and then it adds it to different std::vector objects based on what that `NodeMatch` can be type cast to.
15:01:53
drmeister
One (or more) of the std::vector objects that hold these appears to be getting stomped on.
15:35:22
Bike
it looks like the lambda list handler for ensure-single-dispatch-method is only used for a sanity check as to whether the method and gf have the same required arguments
15:37:23
Bike
that said, the SD generic functions also have a lambda list handler, though i'm not clear on where they get it from
17:30:35
Bike
looks like i can delete it ok. though it's another thing we'll need to run the analyzer on.
17:34:02
drmeister
There is a MatchFinder object that is being garbage collected when I don't think it should be.
17:35:03
drmeister
By adding print statements to the clang code I found what address in memory was corrupt.
17:35:49
drmeister
I can see that address being modified by our finalizer code and then before that when we are allocating a MatchFinder object.
17:36:18
drmeister
The MatchFinders are all put into a linked list and the head of that list should be on the stack.
17:36:42
drmeister
Maaaybe - then that would put us all the way to "How the heck does anything work".
17:37:28
drmeister
But I've been there so many times that I've set up a chair and umbrella to make myself comfortable.
17:38:31
drmeister
It will take more time to sort this out. I may have to write more Python debugging tools - like a conservative memory marker - bleh.
17:39:40
Bike
i know sbcl has some fancy tools to track memory leaks, but that's kind of the opposite of this problem
18:25:03
Bike
it still uses lambda list handlers... in a way that can be replaced, but it'll be a little messy
18:41:02
yitzi
Bike: The new list is here https://github.com/clasp-developers/clasp/blob/vm-boehm/src/lisp/cscript.lisp
18:46:55
drmeister
Bike: What direct-calls.lisp did will be replaced with the bytecode wrappers. The BytecodeModule_O will carry the form that generated the bytecode wrapper with them and we will be able to compile them to native code at any time simply using COMPILE.
18:47:28
drmeister
bytecode wrappers will be considered (compiled-function-p #'bytecode-wrapper) -> NIL
21:05:31
drmeister
There is `(defun print-object (object stream) (print-unreadable-object (object stream) ...))` in kernel.lisp
21:07:47
drmeister
Now that I put the question to words I realize I should add a method for what I really want to print, which is wrapped-pointer
22:54:28
drmeister
We got bit by the worst C++ interop problem. A C++ wrapped object went out of scope when it shouldn't have, got finalized and then a use after free.
23:00:26
karlosz
i thought about how to do the local GO/RETURN -FROM thing some more actually and i think its probably going to be better to just focus on making vm_enty and vm_exit faster by reworking the dynenv represntation like we were talking about
23:01:01
karlosz
that has the bonus of improving cclasp performance as well as fundamentally removing ocnsing for a whole bunch of stuff in the bytecode compiled coe
23:01:22
Bike
I did not. I was mostly doing that to try to lose the funwind protect rather than for efficiency, but then i already had it written up, sooo
23:05:22
karlosz
i just figured if its not too difficult we could improve a lot of things more generally by making entry not cons
23:06:57
karlosz
to just do the specila case of no-exit entry elision is just anotherfixp class. i mean its already kinda there with the borken optimization in the reference compilerat the moment
23:07:16
karlosz
(which should probably get reverted since special bindings don't work with itbecuase of a lack of appropriate local unwinding)
23:26:49
drmeister
I think I've got the static analyzer working again. We should look into getting the sanitizers working.
23:29:59
drmeister
(1) Very complicated code (2) segfault deep in clang ASTMatcher code (3) object quietly went out of scope and was garbage collected (4) use after free
0:22:18
drmeister
yitzi: I got this when I tried to run the analyze script - does it tell you anything about what could be wrong? I thought I unwound all of my hacks to debug this problem...
0:27:51
yitzi
I would skip the analyze script for now. I'll run on the vm-boehm-simp branch after you push. I don't think the cando build is working on vm-boehm
0:31:15
yitzi
you don't need the analyze script....that is for automating the analyzer on clasp and then doing the same thing on cando.
0:32:24
yitzi
The idea behind the analyzer script is to avoid dirtying your working tree with a cando build when you are working on clasp. It builds everything in a separate directory, first clasp, then cando.
0:44:36
yitzi
You can run `ninja -C build analyze` in a completely bare clone and it will everything you need. :)
0:48:53
drmeister
I don't know if it's possible with a garbage collector and our own code generating compiler
0:52:29
yitzi
Ok....well I stuck it in koga in any case...so that is a start. https://github.com/clasp-developers/clasp/blob/61d776cf9a753c9c85c7aa34b1cdc02c3299e05b/src/koga/units.lisp#L205-L207
1:26:39
drmeister
We need the sanitizers because nobody is going to have the patience to debug something like this.
1:34:24
drmeister
Bike: There is an issue though - I actually started trying to debug the static analyzer problem just using bytecode. When I `ninja -C build load_cclasp-boehm` it dies trying to unwind from the first error.
1:35:25
Bike
i mentioned it earlier, but i have something that should be cheaper than the unwind protect
1:36:01
Bike
i'm not sure how to do it that way correctly. i think the unwinder needs to know about bytecode frames some way or another.