freenode/#clasp - IRC Chatlog
Search
20:59:25
stassats
my_thread->_BFormatStringOutputStream seems to be indeed different, but the vector to which it push is the same
21:01:30
drmeister
And the main thread doesn't appear to call my_thread->initialize_thread() searching...
21:11:18
drmeister
Hmm, no - that's not it. The ThreadLocalState does not invoke the MPS allocators - so the order is fine and the order I had in start_thread previously was fine.
21:21:32
drmeister
With this: (loop repeat 1000 do (mp:process-run-function nil #'(lambda () (core:bformat nil "module%s" 10))))
21:23:55
stassats
i'm currently using (loop repeat 100 do (mp:process-run-function nil #'(lambda () (catch 'x (eval '(throw 'x 10))))))
21:26:23
drmeister
There are differences - but allocation points are created before allocations take place.
21:29:28
drmeister
That takes more work - and I have to share the start_thread code - that is possibly giving us trouble.
21:30:51
drmeister
The main thread code isn't a simple function like start_thread - it's more scattered and comes in from main(...)
21:44:40
stassats
just need to restrict the number of backtrace frames printed, otherwise i'm losing the numbers out of sight
21:45:19
stassats
crashing at this->_Data->rowMajorAset(idx+this->_DisplacedIndexOffset,newElement);
21:59:27
drmeister
I'm single stepping through the initialize_thread method where the _BFormatStringOutputStream is created - I noticed there is a thread unsafe increment of a counter that I put in that counts whenever a class is allocated - I will make that thread safe using an atomic variable
22:18:00
stassats
(mp:process-run-function nil #'(lambda () (let ((x (make-string-output-stream))) (write-char #\a x) (loop (assert (find #\a (get-output-stream-string x)))))))
22:30:00
stassats
(mp:process-run-function nil #'(lambda () (loop (core:bformat "a%p" 10)))) crashes on its own
22:51:39
drmeister
If you want a simpler test case - I can get it to crash in the interpreter - with no loaded Common Lisp.
22:52:07
drmeister
(let ((c 1000)) (tagbody top (mp:process-run-function nil #'(lambda () (eval '(list 1 2 3 4)))) (setq c (- c 1)) (if (> c 0) (go top))))
22:54:51
drmeister
This means the fault happened in frame #4? Or can the signal handler be caused by a fault in one of the other 6 threads?
22:56:26
drmeister
Interesting - now it created about 100 threads and then failed with a different error
22:59:07
stassats
the thread struct, with local bindings, bformatStringOutputStream, should be a root itself
23:02:29
stassats
even then, &stack_base isn't guaranteed to be the bottom of the stack where you put your other stuff
23:03:42
drmeister
Right - it's in the frame - but where in the frame relative to the ThreadLocalState is anybodies guess.
23:07:21
stassats
https://stackoverflow.com/questions/1102049/order-of-local-variable-allocation-on-the-stack?answertab=votes#tab-top
23:09:52
drmeister
Sorry - the Process_O::make_process call is here: https://github.com/drmeister/clasp/blob/dev/src/core/mpPackage.cc#L283
23:10:13
drmeister
Process_O::enable() is what calls pthread_create: https://github.com/drmeister/clasp/blob/dev/include/clasp/core/mpPackage.h#L140
23:11:38
drmeister
In start thread I declare stack_base on the stack - but who knows where that ends up on the stack relative to the ThreadLocalState - which must, MUST be above it (on X86_64 below it because the stack grows down).
23:21:03
drmeister
They say you find it the way I do it for the main thread - in a function that calls the function that calls mps_root_create_thread_tagged and that you should ensure that the inner function that calls mps_root_create_thread_tagged is not inlined.
23:25:13
drmeister
I'll add __attribute__((noinline)) to initializeMemoryPoolSystem and initializeBoehm
23:27:29
drmeister
I have all of the Boehm code that would call the API to set the stack base commented out and I recall that Boehm was pretty smart about this stuff.
23:29:18
stassats
so, what i think should happen to be totally robust, no allocation before mps_root_create_thread is called
23:29:20
drmeister
So what did you do to find the cold end of the stack? I was girding my loins to copy that code you pasted from sbcl.
23:29:45
stassats
all allocations is then coming from a noninlined function called after mps_root_create_thread
23:29:48
drmeister
Ok, I am certain that I don't do any allocation before mps_root_create_thread is called.
23:33:37
stassats
since you can probably avoid allocating claspProcess, but you can't really avoid passing the function
23:35:49
drmeister
Ok, so (1) copy the code from sbcl to find the cold end of the stack. (2) move everything that allocates into a separate noinline function and call that from start_thread
23:37:11
drmeister
Just having a pointer on the stack to a thing pins it down. So I just need to make sure the thread has initialized before I leave the Process_O::enable()
23:37:44
drmeister
Right - option (2) is like what the MPS docs suggest and then I don't need to do (1).
23:39:01
drmeister
A mutex? Or a condition variable? I need some way of telling the parent thread that the child thread is ready to go.
23:40:00
drmeister
Besides getting this working - is it a reasonable idea to put the ThreadLocalState struct at the top of the stack like this?
23:40:26
drmeister
Or should I declare it as a collection of roots. I know I need to get the cold end of the stack correctly regardless.
23:41:25
drmeister
Is it a reasonable idea to use conservative GC to keep the ThreadLocalState alive.
0:26:27
drmeister
I went with option (2) and I declared: std::atomic<ProcessPhase> _Phase; and I use a spin lock in Process_O::enable() to check for when the process becomes Active.
0:32:47
drmeister
I will - I'm just going through them - I did some on the linux machine and some on my machine - I have to get everything resolved.
1:10:33
drmeister
Yeah - you need to register the "cold end of the stack" and all roots need to be above it.
1:11:25
drmeister
I'd forgotten that to do this properly you need to do it carefully, (1) get the address of a local variable on the stack, (2) pass it to a noinline function that does all allocation.
1:12:15
drmeister
I was declaring the local variable and allocating objects in the same function and the compiler rearranged them so that roots were below what it thought was the cold end of the stack.
1:12:54
drmeister
And so very important thread local data structures were being collected and everything broke.
1:16:35
stassats
a function call should be separating the point of registering stack roots and using them
1:16:59
drmeister
The function that set the stack_base/cold_end_of_the_stack also allocated the ThreadLocalState on the stack - the order is determined by the compiler and not the order I expected or wanted.
1:17:29
Bike
oh, so it reordered the stack frame so the stack wasn't where you thought it was, i really do see.
1:17:45
drmeister
Yes - and I did that for the main thread for years - but the multithreading is new and I forgot that rule.
1:18:52
drmeister
stassats: An atomic-incf function - it should take a symbol and operate on the symbol-value of that symbol - correct?
1:20:09
drmeister
A place - how do I do it on a place? That needs a macro and I do atomic things in C++.
1:21:15
Bike
you could have a macro that expands into whatever based on the place. like setf. i think sbcl does this.
1:22:10
drmeister
So we need a new macro facility - like setf. I can see the value of it. We should steal sbcl's
1:51:37
drmeister
Is a Common Lisp "place" the same thing as a C++ "reference"? I think they are similar.
1:53:34
Bike
no, a reference is like, an actual thing. places are something only setf knows about, they're not a runtime deal (outside of eval and friends)
1:54:13
Bike
sbcl has a setf-like thing for cas, and the docstring for it has an incf implementation. but then atomic-incf is separate. hokay.
2:06:25
drmeister
I'm still waiting for cclasp to finish building - but slime in bclasp+mps works fine with multithreading.
2:21:14
drmeister
They guess they must establish a restart context and then unwind to that and return?
2:24:15
Bike
you signal an error. if it's handled by something no problem, it's just control flow. otherwise it starts the debugger, which opens an sldb window or whatever, and maybe that restarts, but it's still fundamentally thread local.
2:25:18
drmeister
Understood. I really liked working with core files on linux today. I'm going to start generating them on OS X.
2:25:47
drmeister
Yes - It's so much more convenient than starting everything up again in the debugger and trying to reproduce the error.
2:27:55
drmeister
Hmm, no problems with quitting from sldb with cclasp - I cleaned out the .slime/fasls - maybe that was the problem.
3:38:49
drmeister
(loop repeat 8 do (mp:process-run-function 'foo #'(lambda () (fibn 100000000 78))))
4:29:17
drmeister
Do you know about the Memory Pool System library? It's got a lot of the features that you have been talking about. Non moving pools, mark and sweep pools, compacting pools etc.
4:35:15
Bike
like you can specify "allocation pattern" hints, and schedule collections in a soft real time way
4:38:39
drmeister
Bike: Did you see these functions? https://github.com/drmeister/clasp/blob/dev/src/gctools/gcFunctions.cc#L471
4:41:07
Bike
i think room is slightly dangerous. the documentation says you can't map objects (with mps_amc_apply) without parking the arena first.
4:45:39
drmeister
Yeah - I went looking for the call that parked the arena in cl__room and didn't see it - why did I leave that out?
4:51:52
beach
drmeister: As a general principle, I am not interested in trying to keep up with external software. I see every day how Clasp maintainers try to debug problems with unknown code, try to contact people in charge, try to figure what will break after the next release of such code, etc.
4:51:58
beach
I am even less interested if this software happens to be written in C or some other low-level language. When it comes to memory management, I would like to try out my own ideas first, because they are made possible by the uniform representation of SICL objects.
4:51:59
beach
It may not be worthwhile trying to keep up with a complex system such as MPS when I only need a tiny fraction of what it can do. I am willing to change my mind, of course, but only after I try my own ideas.
4:55:31
drmeister
I know - I'm just making small talk - and I'm happy for the help that stassats gave me.