libera/#clasp - IRC Chatlog

13:06:48 drmeister What is the `target` and what is the `frame` in a LexDynEnv_O?

13:07:26 drmeister I think `target` points into stack frames - is that correct?

13:08:19 Bike target is a jump_buf* that we can longjmp to

13:08:46 Bike frame is a frame base pointer for when we need to use the C++ unwinding instead.

13:12:15 drmeister So they both point onto the stack within the stack frame of a particular function?

13:13:15 Bike i believe so.

13:14:46 drmeister I want to enhance the new `lde` command - it prints the dynamic environment stack so that for each entry it tells me what stack frame (matching the `backtrace` command) it points into.

13:14:50 drmeister https://www.irccloud.com/pastebin/uwnZLhQ0/

13:24:33 drmeister https://www.irccloud.com/pastebin/As3cj3VP/

13:25:30 drmeister Now, will those values ALWAYS fall between a $rbp of one stack frame and the next?

13:54:50 drmeister I just learned about tmux "copy mode" - this guys face is how I feel.

13:54:52 drmeister https://waylonwalker.com/tmux-copy-mode/#:~:text=tmux%20copy%2Dmode%20is%20a,enter%20to%20copy%20the%20selection.

13:56:59 Bike i've yet to figure out copying

14:01:23 drmeister I just want to scroll back.

14:01:56 drmeister I'm trying to figure out how to get the frame pointer for each stack frame in a Python list.

14:02:03 yitzi iterative building! https://plaster.tymoon.eu/view/3444

14:03:09 drmeister Actually - I want the stack frame value and the frame index (as given by the `backtrace` gdb command) together.

14:03:43 drmeister The gdb Frame api SAYS it has a Frame.index() function - but it doesn't exist.

14:03:51 drmeister https://sourceware.org/gdb/onlinedocs/gdb/Frames-In-Python.html

14:04:22 drmeister So I try counting them myself but they don't match. Why, why, why, WHY?

14:04:45 drmeister Uh - it's Frame.level() - hang on...

14:05:53 drmeister Uh no.

14:05:55 drmeister https://www.irccloud.com/pastebin/7l5yK0lO/

14:06:13 drmeister yitzi: Nice!

14:07:08 yitzi I am getting closer to being about to share fasos between cclasp and eclasp. This will result in cutting out the compilation of ~400 files during eclasp.

14:07:14 yitzi s/about/able/

14:19:24 yitzi Loading babel's string.lisp takes 7 minutes on zeus. Just to give y'all an idea of happens to load times in eclasp.

14:19:45 yitzi what happens

14:33:51 drmeister Yeah

14:34:20 drmeister We are coming to fix that problem with a whole bunch of new capabilities.

14:37:13 yitzi Gotcha

15:08:45 drmeister Bike: What is the UnwindProtectDynEnv target pointing at?

15:21:56 drmeister This should describe everything about the broken sjlj_unwind.

15:21:58 drmeister https://www.irccloud.com/pastebin/pzEO8KnK/

15:23:05 drmeister Explanation..

15:23:30 drmeister Line 1 - we are in the frame of the broken sjlj_unwind and we print 5 frames of backtrace.

15:23:51 drmeister Line 8: We print the destination of the sjlj_unwind

15:24:30 drmeister ...it wants to unwind to the LexDynEnv_O at 0x7f3c709fa849

15:25:09 drmeister Line 10: I inspect that object and it shows the target 0x7ffe8b37f300 and frame 0x7ffe8b37f530

15:25:19 drmeister Line 19: New magic!

15:25:35 drmeister ...dump the first 10 entries in the DynEnv stack

15:26:16 drmeister It prints the tagged pointer for the DynEnv, the class of the DynEnv, and the fields of the DynEnv.

15:27:25 drmeister In parenthesis is the backtrace frame level of whatever fields of the DynEnv are pointing into the stack. eg: (#2 )

15:27:34 drmeister Line 30 I display more stack frames.

15:28:30 drmeister Ok, now I'll evaluate the first sjlj_proceed and see what everything looks like right after that sjlj_proceed.

15:33:58 drmeister Bike: Is this what you intended to do?

15:33:58 drmeister https://github.com/clasp-developers/clasp/blob/vm-boehm/src/core/unwind.cc#L90

15:34:26 drmeister You do `thread->dynEnvStackSet(iter)`

15:34:39 drmeister At this point the dynEnvStack is already equal to iter.

15:35:49 drmeister It's probably perfectly fine - but something else I could guess would be done here is to advance the dynEnv stack - but I don't know all the details.

15:35:56 drmeister Call this ^^^ Question#1

15:39:32 drmeister The answer to Question#1 is probably "The code is correct"

15:57:17 Bike the unwind protect target should be a setjmp that runs the cleanup code and then resumes unwinding

15:57:40 Bike you can see it in funwind_protect in unwind.h

15:58:35 Bike i don't see how the dynenvstack is already iter there

16:12:08 drmeister At this point it's still unwinding the TagbodyEnv

16:12:22 drmeister I'm single stepping machine instructions through the unwind - learning how this works.

16:12:31 drmeister I'll have a more intelligent question in a bit.

16:33:49 drmeister It looks like you aren't updating _UnwindDest properly.

16:33:55 drmeister Bike: ^^^

16:34:22 drmeister There is an `sjlj_unwind` to 0x7f3c709fa849

16:34:38 drmeister The `sjlj_unwind_proceed` does its thing and unwinds.

16:35:07 drmeister And the 0x7f3c709fa849 TagbodyDynEnv_O is removed from the DynEnv stack

16:35:47 drmeister Then there is another `sjlj_unwind_proceed` to that 0x7f3c709fa849 TagbodyDynEnv_O DynEnv again.

16:35:59 drmeister At this point it's off the DynEnv stack and it can't be found.

16:41:26 drmeister The bad `sjlj_unwind_proceed` is called by `sjlj_continue_unwinding` from the cleanup clause of a funwind_protect

16:55:40 drmeister I'm thinking that `my_thread->_UnwindDest` and `my_thread->_UnwindDestIndex` are not being updated after an unwind.

16:59:17 drmeister They are only ever set here when we enter `sjlj_unwind_proceed` and then they are set to the dest and index that we are currently unwinding to.

16:59:18 drmeister void sjlj_unwind_proceed(DestDynEnv_sp dest, size_t index)

16:59:32 drmeister https://github.com/clasp-developers/clasp/blob/vm-boehm/src/core/unwind.cc#L75

17:06:51 drmeister I put a watchpoint on my_thread->_UnwindDest and as expected, it is not changed other than at the top of the first `sjlj_unwind_proceed`

17:29:31 drmeister Or `sjlj_continue_unwinding` isn't supposed to just invoke `sjlj_unwind_proceed(gc::As_unsafe<DestDynEnv_sp>(my_thread->_UnwindDest), my_thread->_UnwindDestIndex);` and it should do something that accounts for what has already been unwound?

17:46:24 drmeister I'm pretty sure the problem is related to this.

17:49:16 drmeister ::notify Bike It looks like _UnwindDest and _UnwindDestIndex are not being updated properly when functions return and the stack unwinds. sjlj_continue_unwinding invokes sjlj_unwind_proceed( gc::As_unsafe<DestDynEnv_sp>(my_thread->_UnwindDest), my_thread->_UnwindDestIndex) but my_thread->_UnwindDest and my_thread->_UnwindDestIndex are never set anywhere other than sjlj_unwind_proceed. This means a stale DynEnv is proceeded from from

17:49:17 drmeister sjlj_continue_unwinding.

17:49:17 Colleen drmeister: Got it. I'll let Bike know as soon as possible.

17:50:36 drmeister I'll be AFK for a while running an errand. Call me if you have time to chat about it.

17:53:55 drmeister I realize it's kind of a dick move to ask all these questions on a Saturday, ...but it's the weekend mornings I can finally focus on debugging tricky problems. I appreciate the attention that you put into this on the weekends. It's probably best to let me spin my wheels for a while narrowing down the problem.

18:29:46 Bike wait, hang on now

18:29:46 Colleen Bike: drmeister said 40 minutes, 30 seconds ago: It looks like _UnwindDest and _UnwindDestIndex are not being updated properly when functions return and the stack unwinds. sjlj_continue_unwinding invokes sjlj_unwind_proceed( gc::As_unsafe<DestDynEnv_sp>(my_thread->_UnwindDest), my_thread->_UnwindDestIndex) but my_thread->_UnwindDest and my_thread->_UnwindDestIndex are never set anywhere other than sjlj_unwind_proceed. This means a stale DynEnv is proceeded from from

18:30:03 Bike unwind dest and index aren't _supposed_ to be updated, i think. they're the ultimate destination and index

18:30:15 Bike you set them when you start unwinding and read them when you finish and that's it

18:34:58 Bike if i remember correctly

18:35:35 Bike yes, that seems to be the case

18:36:11 Bike continue_unwinding should onlye ver be invoked in the middle of unwinding to some higher destination, i.e. _UnwindDest should always be on the stack from there

19:15:47 drmeister Then we shouldn’t be invoking continue_unwinding because the destination was already past.

19:22:13 Bike yeah, something like that

19:27:31 yitzi drmeister: I've succeeded in merging the cclasp and eclasp fasl folders. No more need for names like `SRC:LIB;ECLASP-BITCODE-BLA;` ... and eclasp reuses the fasos from the cclasp image. 400 less file compilations.

19:34:06 drmeister Excellent

20:47:42 drmeister Do you have any recommendations for additional debugging info? Can we trace entry into sjlj_unwind and leaving setjmp to try/catch?

20:49:59 Bike you mean how to get more now, or more like the compiler stuff i was talking about

21:09:39 Bike it sounds like the problem is unwinding past where we should be, so i guess it would make sense to trace entry into sjlj_unwind_proceed and figure out whether the destination is actually live when we start each sub-exit