freenode/#clasp - IRC Chatlog
Search
17:12:42
drmeister
Occasionally, when you know that no thread is reading the hash-table you pause, apply the fixups to the hash-table and clear the fixup list.
17:13:31
drmeister
We can control how long we let the fixup list grow before you want to pause and fix hash-tables.
17:14:01
drmeister
The implementation of the fixup list would probably be a vector for faster reads.
17:14:49
drmeister
Oh - you know what - you can' use a vector - because that can't be updated atomically without a mutex.
17:15:33
Bike
you can update a vector atomically. you just atomically increment the fill pointer, then store whatever at the old index. right?
17:16:23
Bike
i guess there'd be another problem if another thread is simultaneously reading the vector up to the fill pointer, and gets the uninitialized entry
17:17:29
drmeister
Hmm, I'll have to think about that. The atomic linked list is definitely safe and flexible.
17:18:06
drmeister
You could keep a pool of cons cells to draw from to add to the atomic linked list.
17:19:07
drmeister
So (setf gethash) would grab a cons cell and a fixup record from a pool and fill the fixup record and push it to the head of the atomic fixup list.
17:20:27
drmeister
The yieldpoints would check a bitmap of different kinds of yields against a thread local bitmap of what yields are permitted.
17:21:04
drmeister
Then we would have suppress-yield and allow-yield calls that would modify the thread-local bitmap of yields that are allowed.
17:24:31
Bike
those kind of worklist things are usually how wait-free algorithms work (i think) but they don't need safepoints
17:25:21
drmeister
We already have safe points - we check a global variable to see if a signal handler has been activated.
17:26:30
drmeister
If you don't have safepoints - you use signals? If you use signals - you have to poll occasionally (at a safepoint) to see if a signal came in.
17:30:08
drmeister
Ok - I think we can put this on a firmer foundation. Can you take a look at the safepoint/yieldpoint injection pass in llvm?
17:30:56
drmeister
You could generate a .ll file using compile-file and then run the pass on it with opt --place-backedge-safepoints-impl
17:31:18
drmeister
Then what does it look like in the new llvm-IR and what does it look like in the disassembled object file.
17:31:49
drmeister
I added a facility to dump every JIT generated object file to /tmp/ one after the other.
17:32:19
drmeister
I can do the same with modules and then absolutely everything the compiler generates can be inspected.
17:45:16
drmeister
They are mostly straightforward - like pathname literals - but they have a twist.
17:45:42
drmeister
They contain function entry points (currently only one) and a ObjectFile_sp tagged pointer.
17:45:50
drmeister
The ObjectFile_sp will be the same for every FunctionDescription_O in the current object file.
17:46:32
drmeister
I think I can get the ObjectFile_sp from a thread local slot that stores the current ObjectFile_sp that is being load-time evaluated.
17:48:45
drmeister
https://github.com/clasp-developers/clasp/blob/future/include/clasp/core/functor.h#L82
17:49:48
drmeister
I have all of the function entry points in a vector within the Module - so I can reference them using an integer.
17:51:47
drmeister
I think I need to set up a call to a function like ltvc_make_function_description that is like ltvc_make_pathname
17:51:48
drmeister
https://github.com/clasp-developers/clasp/blob/future/src/llvmo/link_intrinsics.cc#L396
17:52:49
drmeister
It will take an integer argument for each entry point function and no argument for the ObjectFile_O object because it will get it from a thread local slot.
17:53:28
drmeister
The rest of the info like sourcePathname, functionName, lambdaList, docstring, lineno, column, filepos - those will all be regular literals.
17:54:06
drmeister
I could argue that I don't need some of these because they are made redundant by pulling the info out of the DWARF metadata.
17:54:50
drmeister
multiple entry points will be handled in the future by adding more integer arguments, one for each entry point.
18:23:52
Bike
i think the problem was that sometimes we get an offset, and sometimes we get a lineno + column
18:24:38
drmeister
Oh wait - it is redundant. This is the filepos of the function itself. Most of the time we need the filepos of a return address - that we only get from DWARF - and there all we get is what is in the line tables.
18:26:43
drmeister
Is it the case that we get the offset only from the FunctionDescription objects. If we got rid of filepos maybe that would be best?
18:27:01
drmeister
As in filepos is a complicating feature that doesn't bring a lot of value. Adds complexity with little value.
18:27:08
Bike
mostly i remember that we really should have one single structure representing source pos info
18:27:46
drmeister
Ok and DWARF kind of makes our choice for us - lineno+column is what DWARF gives us.
18:28:58
Bike
for example swank's compiler-condition class has a "location" that i think has to be the offset
18:40:47
Bike
https://github.com/slime/slime/blob/master/slime.el#L3353-L3358 i guess this is what we have. so lineno column is fine. maybe.
19:20:29
kpoeck
drmeister Earlier today you said "- we check a global variable to see if a signal handler has been activated. ". Could you point me to that code? As Bike mentioned, at least fpe handling seems to be delayed and I would like to look into that.
19:24:08
drmeister
kpoeck: I think it's in the gcalloc.h file - every call to handle_all_queued_interrupts();
19:24:57
drmeister
https://github.com/clasp-developers/clasp/blob/master/src/gctools/interrupt.cc#L209
19:29:05
drmeister
The queue_signal_or_interrupt is what adds a handler to the thread local list of interrupts to handle.
19:29:56
drmeister
Yeah - and that gets called from handle_or_queue_signal and that is a signal handler.
19:31:18
drmeister
So a thread gets a signal, handle_or_queue_signal is called. That may handle it or it pushes an interrupt onto the current threads list. Then handle_all_queued_interrupts gets called after almost every allocation to handle any interrupts that came in asynchronously.
19:32:14
drmeister
https://github.com/clasp-developers/clasp/blob/master/src/gctools/interrupt.cc#L252
19:33:07
drmeister
I think this means we are invoking our code from a signal handler - I think this is dangerous. Maybe we do it properly - maybe we don't.
19:34:20
drmeister
My understanding is that there is very little that we are allowed to do in a signal handler.
19:35:15
Bike
yeah so the problem is we kind of have to unwind out of them to use them for conditions.
19:36:21
drmeister
We can't unwind out of them is my understanding. Unix signals and C++ exceptions are fundamentally incompatible.
19:37:00
drmeister
What we can do is insert yieldpoints in the code and check if a signal came in and then unwind from there.
19:38:00
drmeister
Hmm then there are problems with threads. Does the thread that generated the FPE get the FPE signal?
19:44:15
drmeister
Then we should be able to add a thread-local flag that an FPE occurred and inject a test for that into the arithmetic code.
19:48:07
Bike
if we wanted to check flags on every single floating point operation we could turn off the signal handling and do that
20:45:55
Bike
also if we leave NaNs around or want particular rounding i think it really is every operation
20:48:04
kpoeck
for an fpe, we run directly this: https://github.com/clasp-developers/clasp/blob/master/src/gctools/interrupt.cc#L316
20:50:41
Bike
i looked it over and as far as i can tell, sbcl does unwind out of signal handlers and that works fine for it
20:51:39
Bike
because the exceptions library does all kidns of goofy shit that probably not reentrant
20:52:23
drmeister
sbcl doesn't use C++ exception handling - it may be leaving something in the stack frames to let them unwind out of a signal handler but from everything that I've been told we cannot do this with C++ exception handling.
20:53:16
Bike
i don't really understand why unwinding out of a signal handler in particular is not allowed. longjmp ought to work, i think
20:54:07
Bike
assuming longjmp is reentrant i guess. maybe all the dumb shit stuffed into jmp_bufs makes that hard
21:00:03
drmeister
A signal can happen between any two instructions - right? Is the stack frame that the signal handler sets up able to synchronize with the itanium unwinding machinery.
21:03:07
Bike
my impression was that within a handler you still have, like, a normal stack with frames and everything
21:03:25
Bike
above your handler function is whatever operating system machinery, and then above that is the frame of the function that was running when the signal interrupted
21:07:58
kpoeck
apart from what are discussing, I believe we have 1 more issue. Just try (mod 1 0) in clasp, sometimes, we get the fpe, sometimes clasp hangs with 99,9% cpu in core::clasp_truncate. lldb knows that an exception happens, but handle_fpe is not called.
21:10:43
Bike
"the main problem with unwinding is dlopen and dlclose" i really hate the C++-brain on this topic, man
21:13:17
drmeister
I don't have the mental bandwidth to follow the whole talk - but there may be something about signal handlers in there.
21:21:03
Bike
looking through stack frames and stuff is no problem. so in a lisp-style unwinding setup, unwinding would be no issue, since all the cleanup and destination info is on the stack. on the other hand, getting the corresponding dwarf info from disc/wherever is not signal safe
21:21:30
Bike
i guess if we like, receive a signal in the middle of dl_iterate_phdr, or something weird like that
21:28:03
Bike
my understanding though is that this stuff should be less of a problem for _synchronous_ signals like sigfpe, which is after all signaled at only particular places (where we do a floating point operation)
21:41:18
drmeister
Does the unwinding happen with a library that we can get debug information for on linux?
22:02:20
Bike
i could try. at the moment it's just crashing outright instead of just signaling weird errors like it used to, though
22:14:32
Bike
and then if you try to abort you get the same out-of-extent-unwind so you're stuck in the debugger forever.
23:12:39
drmeister
Do we need to be able to get the declarations for a function from the function-description?
23:29:54
drmeister
I have made FunctionDescription_sp objects literals - but I don't set the entry point in the literal.
23:30:12
drmeister
I create the FunctionDescription_sp object at load time and then I set the entry point using a function pointer.
23:31:04
drmeister
Once I get it working I'll look closely at it and see if I can set the entry point in the FunctionDescription_sp object directly.
23:40:54
drmeister
The solution is going to involve the ltv/function-description - I need to write out a reference to the entry-point.
0:18:25
drmeister
FunctionDescriptions used to be raw blocks of data with a name appended with "^DESC". That's gone now and they are first class objects.