freenode/#clasp - IRC Chatlog
Search
18:08:47
drmeister
I figured out how to recover info from stackmaps and answers to all of the questions we had on the weekend.
18:09:16
drmeister
I really appreciate the help you gave on Saturday - that really helped push me in a productive direction.
18:10:41
drmeister
Ok, so here's the poop: the _mh_bundle_header symbol is an internal symbol within the bundle(dylib) it points to the start of the library - but it's troublesome to use it because it isn't available if I load bitcode directly and it won't be available in linux.
18:11:41
drmeister
It was much easier and more portable to use 'dladdr' - with the address of any function in the library the Dl_info struct will return the address of the start of the library.
18:12:38
drmeister
Then on macOS we are kind of on our own. There are the header files and structs in /usr/include/mach-o but there don't appear to be any library functions to work with dylib files. the getsectionaddr is a helper function written to support otool and a few other tools.
18:13:01
drmeister
It was easier to copy and paste the getsectionaddr code into clasp and modify it a bit.
18:14:11
drmeister
I got the LLVM_STACKMAPS section/segment (whatever) and I run it through the parser that I wrote based on the llvm stackmaps documentation.
18:14:54
drmeister
I know a lot more about object files and how to use them with llvm. This will help us with startup time.
18:17:06
drmeister
Well, not so much for libraries - but there is for the ORC JIT. Basically it constructs an object file in memory and then I can subclass the memory manager and capture when it allocates data sections and I just figured out the method to overload that is called when the object file contents have been set up and relocated into memory. Now I can get the __llvm_stackmaps data in its final, relocated form.
18:18:38
drmeister
This has really helped to demystify the underlying object file and dynamic library machinery.
18:46:33
drmeister
I think we are in business now with stackmaps working for fasl files and JITted code.
19:48:31
frgo_
Man, it took me a second look to get what I see. Real backtraces with real info in them. YES!
19:58:22
drmeister
With new stackmaps cclasp bitcode --> 25,113,174 lines of llvm-ir and 147 landingpads
20:03:06
drmeister
The good thing about all the complicated shadow stack and interweaving of the shadow stack with the real stack is that I can still use it all - and it merges interpreter frames with real frames.
21:52:39
drmeister
I haven't pushed to dev yet because I stashed the changes and I'm building with without the changes to see the impact on the number of lines of llvm-ir and landingpads
21:53:13
drmeister
Building cclasp [81 of 439] - as soon as it's done 10-15 min I'll pop the stash and push the changes.
21:54:48
drmeister
At [81 of 439] there are more than 8,000 landingpads - but we knew that it would be high.
23:22:09
drmeister
These new numbers are better than my previous estimates because they are from clasp with the new stackmap changes and then without the stackmap changes. It's as close a comparison as there can be.
3:02:46
drmeister
Wow! Fun fact - you can productively 'ediff' .ll files generated by different versions of clasp.
3:03:49
Colleen
Bike: drmeister said 21 hours, 24 minutes ago: After eliminating the shadow stack and the landingpads that were needed to support them I reduced the number of landing pads in cclasp from 55,562 to 147.
3:03:49
Colleen
Bike: drmeister said 20 hours, 53 minutes ago: Start keeping track of the total number of bitcode lines. I used (wc `find . -name '*.ll' -print`). Its a quick, rough measure of how much code we are generating.
3:06:05
drmeister
Yeah - there is no mystery - yesterday I was comparing to an old version of cando that appears to be very out of date and earlier today I was comparing to clasp before it had finished generating bitcode for 'inline.lisp'
3:15:20
drmeister
Also I have a question about the removal of temporary alloca's - they are replaced with llvm variables - do you give them names using gensym's? Or do you let llvm name them?
3:16:01
Bike
let llvm name them. i'd have to change things in translate-instruction to give them proper names.
3:17:05
drmeister
Because I am now thinking we should leave the naming to llvm - it might lead to better 'ediff'ability of the generated llvm-ir. That would help with improving code quality.
3:18:23
drmeister
Do you have any ideas of how to change literals to reduce the code size for inline.lsp?
3:27:41
drmeister
It's not too much to ask. We are essentially spitting out instructions for a little virtual machine.
3:37:37
drmeister
We should think this through with the new things we've learned over the last weeks.
3:50:58
drmeister
It's a bit fussy because I don't know the size of functions loaded from fasl files.