freenode/#clasp - IRC Chatlog
Search
17:55:27
drmeister
I've refreshed my memory on how clasp's C++ interop works. I have two kinds of code pointers to deal with.
17:56:18
drmeister
BuiltinClosure_O is a base class for a whole lot of different template classes that wrap function pointers and method pointers that I need to deal with.
17:57:29
drmeister
To get the size of the object we use the templatedSizeof() method - the size can vary depending on what the templated descendent class stores.
17:58:41
drmeister
If they wrap a method pointer that adds 16 bytes. Method pointers in C++ are 16 bytes. It's a function pointer and an offset for adjusting the "this" pointer. Supposedly this is dictated by the ABI - but I can't find any mention in the X86-64 ABI document.
17:59:27
drmeister
All of our "this" offsets should be zero - and I'll verify that as a sanity check during image save/load if that is the case.
18:01:33
drmeister
Image save/load is a complex problem. There are many (thousands to hundreds of thousands) of code pointers that need to be carefully fixed-up/saved and then loaded/fixed-up to make it work.
18:01:58
drmeister
They are all one direction though - from GC managed memory to C++ code in the executable and libraries.
18:02:30
drmeister
I'm not supporting pointers that go the other way or pointers that go from GC managed memory to other GC managed memory.
18:02:58
drmeister
We know that's not a problem because clasp has worked with MPS moving GC and that would scramble those kind of pointers.
18:03:56
drmeister
My big challenge at the moment is taking a function pointer and converting it into something that on load can be restored to a relocated function pointer.
18:04:30
drmeister
My plan is to turn <function-pointer> into <library-ID | function-pointer-offset>
18:05:41
drmeister
A 8 byte function pointer will become a 4 byte library-ID and a 4 byte function-pointer-offset.
18:08:45
drmeister
Then there would be a limit of 65536 different libraries and they could contain 281,474,976,710,656 bytes of code.
18:09:40
drmeister
Hmm, well the scope of this coding scheme is limited to image save/load - so I can adjust it in the future.
18:13:37
drmeister
I think MMTk is going to be the hot new thing in memory management - but it's not clear yet.
18:17:52
froggey
Yes, image save/load is implemented. it works a bit like hibernate/suspend to disk, so probably not useful for you. interestingly the lisp continues running after saving an image (save-lisp-and-keep-going), which seems different to other implementations
18:18:37
froggey
memory management is a very broad topic, everything is custom. custom gc, etc. it's fairly tightly integrated with the low-level kernel-like code
18:18:41
drmeister
Is running after saving an image tremendously useful? I have a way to implement that - but I can't see a reason to expend the effort.
18:20:04
froggey
probably not for you, but for me it is. it's less about saving an image and more about persisting the state of the system. Like SICL, I think
18:20:38
drmeister
I hope a benefit of MMTk will be a community around the MMTk library and advanced memory management code being developed and maintained by that community.
20:55:06
drmeister
C++ method pointers are 16 bytes, two qwords. The second qword is always zero in my tests - as I hoped.
21:17:26
drmeister
freemint: Just a bit - I can save simple images and load them in and fixup vtable pointers.
21:19:20
freemint
Julia is not gonna work on such functionality for some time and maybe it is technocally impossible without a major rewrite.
21:19:48
drmeister
Yeah - I've been mucking around with clasp internals for the last three months. It's not easy to implement this.
21:20:22
drmeister
I'm still not certain it's possible. There are lots of places where it could go off the rails.
21:21:50
drmeister
I'm working on reconnecting function and method pointers at the moment. I'm in the middle of a heart stopping moment where I have function pointers like 0x21. I'm working on figuring out where they are coming from.
21:22:21
freemint
Julia could use some work on GC but it is good enough for the fast non allocating you tend to write in Julia.
21:32:09
drmeister
Bike: Yeah - I'm doing an inventory of all the code pointers that I need to save/load. I'm identifying what library they belong to.
21:32:40
drmeister
I found about two dozen that look bogus - they have function pointers that look like 0x21 and 0x1dc - small values like that.
21:33:26
drmeister
So I added some code to trap and print a message when a bogus function pointer like that comes through. The first one I looked at was DoubleFloat_O::castToInteger.
21:36:56
drmeister
And it does this weird thing - but that may not be related to its weird function pointer.
21:38:37
drmeister
ABI specification documents? We don't need no stinkin' ABI specification documents!
21:41:07
drmeister
But on that, I was perusing the x86-64 ABI specification this morning before breakfast and I see that they give x86-64 code examples for different types of calls. Small code model calls, medium code model calls, large code model calls.
21:43:02
drmeister
I'll have to look again but I didn't see any conditional tests in there - it's straight up load registers and calls. So a function pointer like 0x21 may be a bug in clasp that we haven't encountered? Or is there code down at the bottom of memory?
21:52:36
drmeister
I don't understand how these methods are callable - their function pointer values are messed up but they appear to work.
21:53:30
drmeister
I have like 3500 function/method pointers that are fine. I have about 25 that are messed up with these low values - but I am able to call them - weird.
21:53:59
drmeister
I'd like to understand how they work before I go implementing image save/load and do what with them? Ignore them?