freenode/#clasp - IRC Chatlog
Search
20:39:36
drmeister
I am unable yet to generate a symbol list automatically that is anywhere as good as the one I constructed by hand. It's very frustrating.
20:40:07
drmeister
A whole new stupid problem to solve - because C++ doesn't let you explicitly control attributes of the vtable symbol.
20:41:09
drmeister
If I take control of the exported symbols list I have to figure out every symbol that the compiler and linker would make exported.
20:41:16
frgo
drmeister: Another idea: Why not have a special variant of dlsym that doesn't check for symbol "visibility" and just returns the address just like on unix?
20:42:27
drmeister
Then I have to have go back to scraping ELF files and macho files or running 'nm' at startup on macos.
20:43:52
frgo
If that's the case then I don't understand the process on linux. I seem to recall that you just had to call dlsym and that's it.
20:46:50
drmeister
No - that's not the problem. I don't actually have a problem on linux using dladdr/dlsym
20:49:07
frgo
I know. The reason why this is working on Linux is that dlsym doesn't check for the symbol being local. On macOS it does. So, if we replace the stock dlsym call to a call to "our" dlsym where just one line of code is changed compared to the stock dlsym - then I assume this should work on macOS.
20:59:31
drmeister
I kind of have a solution for macOS - I have a symbol list that I put together by hand that contains 12,210 symbols - if I could figure out how to recreate that list automatically - I'd be done.
21:00:54
drmeister
My attempts up until now generate lists that are >10x larger and cause the linker to spit out tens of thousands of warnings.
21:02:18
frgo
Hm - a ton of questions arising for me - I don't want to make it even more work for you.
21:03:23
drmeister
Feel free to ask away. The "reimplement dlsym to work with internal symbols" idea is one idea - we could revisit it.
21:05:30
drmeister
I have to run the iclasp-boehmprecise executable a couple of times and extract information from it and from error messages that come up when I try to save a snapshot.
21:06:01
drmeister
It basically tells me what symbols are not working with dlsym and I build up a list of those.
21:13:39
drmeister
Right now I am trying to take a regularly linked iclasp-boehmprecise executable, extract external symbols from that and then add in the vtable symbols and then link a new executable.
21:14:35
drmeister
But if I have to build a pile of horsesh*t to get snapshot save/load to work - I will.
21:15:41
drmeister
Well, the part that is causing me trouble is I have thousands of vtable pointers and function pointers that I need to update within objects when the objects load.
21:16:14
drmeister
I started with an approach where I used the library-start/byte-offset of each symbol - but that is extremely brittle because the only executable that can run a snapshot is the one that created it.
21:16:30
drmeister
So I couldn't embed a snapshot within the executable - because that requires re-linking.
21:17:04
drmeister
So I switched to using dladdr on addresses to get symbols at save time, building a symbol table and using dlsym at load-time to lookup the absolute addresses.
21:19:20
drmeister
I have this list above of 12,210 symbols that works. I'll extract the external symbols from a regularly linked executable and then take a set difference and see what's missing...
21:20:02
drmeister
I've been trying to guess what I need, hoping if I cast a wide enough net that it would work - but I get all these warnings if I include the wrong symbols.
21:41:16
frgo
So, macOS does things differently. I mean, really differently. Been reading the Dynamic Loader source code - it's a whole new world at that (low) level (for me).
21:44:51
frgo
In https://opensource.apple.com/source/dyld/dyld-832.7.3/dyld3/MachOLoaded.cpp.auto.html the method MachOLoaded::findExportedSymbol would need o be modified.
21:48:41
frgo
No - won't work. macos seems to behave different during link time. The so-called Exports Trie needs to be built at link time. If the symbol is not in that structure then no way to find it.
21:55:39
drmeister
I'm going to embrace the idea that I can get the hard to find symbols out of cando itself.
21:56:24
drmeister
I'm adding a command line option that will cause dladdr to be evaluated on every function/method pointer that is exposed.
21:57:23
drmeister
What I can't get this way is the vtable symbols - I'll have to extract those from object files and merge the results.
21:59:38
drmeister
dladdr doesn't care if a symbol is external or local - it will return it whatever it is.