freenode/#clasp - IRC Chatlog
Search
16:05:33
drmeister
https://stackoverflow.com/questions/65739437/how-do-i-keep-track-of-marking-work-in-my-own-mark-procedure-for-boehm-gc-and-wh
16:05:53
drmeister
Ivan, the maintainer wanted me to ask questions like that - let's see how well it works.
16:10:52
kpoeck
Bike or karlosz ; Could you commit yesterdays fix for ltvc_read_bignum or ltvc_write_bignum? You might only have changed that on BigMac
17:32:17
drmeister
jackdaniel: Do you know if it's guaranteed that each object that gets passed to a mark procedure MUST be valid? I'm getting obviously invalid objects passed to my mark procedure from boehm marking threads (not the main thread). These objects are not coming from my objects that I push to the mark stack.
17:47:05
drmeister
It has a header containing the stamp and some tag bits and is followed by a fully constructed object.
17:47:58
drmeister
When I turn on guards a big header gets added with a couple of guard words that are set to things like 0xFEEAFEEBDEADFEEF and 0xAAAAAAAAAAAAAAAA.
17:49:00
drmeister
I am assuming that only valid objects can get passed to a mark procedure unless something is seriously wrong.
17:49:37
frgo_
drmeister: Have you seen https://github.com/ivmai/bdwgc/blob/57b97be07c514fcc4b608b13768fd2bf637a5899/typd_mlc.c#L364 - I don't actually know what's going on there but the comment connects to the stackoverflow question for me.
17:49:49
beach
If stack scanning is conservative, the stack scanner has no way of knowing what is an object and what is not.
17:51:00
beach
Oops, my (admittedly small) family just announced that dinner is served. So I'm off for today.
17:57:05
drmeister
That leaves me with even more questions. They DO seem to be using 'env' to keep track of the work done.
18:02:55
frgo
You shouldn't have to. If you set GC_ext_descriptors[env].ed_continued to true the machinery is there. Now, I don't know how to do that as I haven't looked into clasp GC code for ages.
18:04:01
drmeister
I see the GC_ms_entry defined as an opaque pointer in gc_mark.h and the actual definition in gc_priv.h - so - there be dragons.
18:06:24
drmeister
Is that consistent with your thinking? Or am I missing something. The GC_ms_entry and GC_ext_descriptors are in gc_priv.h that leads me to think I shouldn't need to use them.
18:07:20
drmeister
On the other hand - the API may be incomplete because I'm messing with things that no sane person would mess with?
18:08:20
drmeister
I am a bit heartened to see that they use 'env' to keep track of work - I was about to add a field to the end of any container that stores pointers. Perhaps I don't need to.
18:10:58
drmeister
https://www.bountysource.com/issues/56317406-can-custom-mark-procedures-be-incremental
18:12:25
frgo
So there's an API for making a descriptor: https://github.com/ivmai/bdwgc/blob/master/include/gc_typed.h#L52
18:15:27
frgo
I say it again: Complete guesswork here from me. But given your amazing code reading capability I thought it worth throwing in.
18:17:43
drmeister
I mean it might be a good time to include <gc/private/gc_private.h> but - I have butterflies in my stomach.
18:21:09
drmeister
1. I know all the offsets of all pointers that MPS needs to fix and that precise boehm needs to mark - and I use this information in both GC's
18:22:22
drmeister
3. I'm using very similar code between MPS and boehm. In fact I learned about using bitmaps to indicate pointers in small objects from boehm and I switched MPS to use the same bitmaps. They work great.
18:23:08
drmeister
4. When I turn on precise mode for boehm objects get allocated and mark procedures get called - but very early a boehm thread calls a mark procedure with an invalid object that was never allocated.
18:24:05
drmeister
I BELIEVE that only valid objects should be passed to the mark procedure. I wrote the code to error out if an invalid object is passed.
18:24:49
drmeister
Yeah - one of my assertions triggers when it sees a bad header. In this case the header is full of zero bytes.
18:25:22
drmeister
The address looks like a reasonable one - it's got the right range, and I can read the memory - it's not completely crazy.
18:26:04
frgo
Hm - zeroes means allocated but not initialized with metadata, so not recognizable as an "object".
18:26:12
drmeister
And - this is a clue but I don't know what it means - it's a boehm marking thread that calls the mark procedure.
18:26:19
drmeister
Yes, zeroes means allocated but not initialized with metadata, so not recognizable as an "object".
18:27:02
drmeister
I print all the addresses that I've allocated (this is happening very early) - and the address does not belong to an allocated object.
18:30:16
frgo
I assume you do know this: https://www.hboehm.info/gc/scale.html - and I can't see in this text if that's still valid info. It describes thread handling for marking.
18:32:36
frgo
Have you reached out to Hans-J. Boehm? (it's a former colleague- we've both been at HP)
18:58:10
drmeister
frgo: I read all the boehm stuff long ago - I've been running on my working knowledge and imagination - I'll read that again.
18:58:48
drmeister
I did not reach out to Hans-J.Boehm - I didn't imagine that that would be productive. That you know him is interesting.
18:59:50
drmeister
I did reach out to Ivan Maidanski - he is the maintainer of the boehm library. He suggested that I ask questions on StackExchange and post feature requests to the github.
19:02:52
drmeister
The guidance so far https://www.hboehm.info/gc/scale.html is to build boehm with flag settings for parallel collection. We have not done that yet. We have been using the stock boehm libraries for debian (apt-get) and macOS (homebrew).
19:09:28
drmeister
frgo: If you are on good terms with him - could you introduce me to Hans-J. Boehm? He might be able to direct us to people who could answer questions.
20:17:55
kpoeck
drmeister regarding boehmgc, I thought we should use the version from https://github.com/clasp-developers/clasp-boehm instead of the version used by brew. Is this no longer the case?
20:19:27
kpoeck
Especially have a look at https://github.com/clasp-developers/clasp-boehm/blob/master/makefile#L12
21:36:12
drmeister
kpoeck: It's been so long that I don't recall about the clasp-boehm repo - but it's good that you reminded me. I'm building a version I got from cracaeur at the moment - I'll take a look at clasp-boehm
22:22:58
drmeister
Argh - I read these long comments - but my brain switches off near the end of them.
23:06:29
drmeister
I think there is missing functionality in boehm. There is no way to push partial work onto the mark stack.