libera/#clasp - IRC Chatlog
Search
18:42:37
bike
we have a file llvmo/jit.cc, but it's not in the cscript, or #included anywhere as far as i can tell. and it uses a lot of a class ClaspLinkerJIT that isn't actually defined anywhere. is this file just not used at all
18:45:04
bike
i'm looking a bit on including machine code in bytecode FASLs, so i'm diving into some of the jit stuff
18:48:41
bike
not totally clear on why we're using "JIT" stuff for fasos, really, since we're not jit compiling anything. i guess there's no easier way to load up some memory and say hey this is a function?
18:59:01
drmeister_
I'm pretty certain that jit.cc is old code that can be removed. Everything important was moved to runtimeJit.cc
18:59:53
drmeister_
Can you elaborate on your second question? "not totally clear on why we're using "JIT" stuff for fasos, really, since we're not jit compiling anything. i guess there's no easier way to load up some memory and say hey this is a function?"
19:00:31
drmeister_
JITted code generates ELF/MachO object files in memory and those are written out to faso files.
19:01:45
drmeister_
We have an older ahead-of-time compilation that generates object files that get written into faso's.
19:02:08
drmeister_
JITted code generated ELF/MachO object files that can get written into snapshot files.
19:03:07
bike
yeah, i mean if we compile a file, and then load the faso into a new image, it's not really much different from loading a shared object is all
19:03:31
bike
except of course we pack a bunch of object files into each faso, so it's not as simple as setting the dynamic loader on it
19:06:12
drmeister_
Dynamic libraries also mess with debug info don't they? I kinda like object files because they are less "processed" than dynamic libraries. We used to generate dynamic libraries.
19:07:41
bike
oh, right, and we had that whole fiasco where mac wanted dwarf in separate files or something
19:07:48
drmeister_
Linking object files strips and consolidates DWARF info. In MacOS they put it in a separate file.
19:09:52
drmeister_
Also, the Orc/LLJIT LLVM JIT framework lets you add object files to it. I don't think there is any way or need or if it even makes sense to add dynamic libraries.
19:10:36
bike
i'm getting into the jit stuff because i think putting object code in bytecode fasls should work a bit differently. the fasos we have now are kind of all consuming - we have one machine code function that builds all the constants and sets up all the functions - i don't think that's going to work, or at least any way we could make it work would involve plenty of unnecessary duplication
19:11:13
bike
it would be nice to have a thing in the bytecode fasl that's a whole object file, and then individual bytecode functions could just be tagged with a pointer (or symbol name) for their XEP/local functions
19:13:21
bike
so i would think, but it's deeply tied into this jit stuff, and that has confusing bits like this huge unused file.
19:13:47
drmeister_
Hmm, there is a bit more to it - there is the symbol that you use to "materialize"/link the object file.
19:13:49
bike
this might incidentally be somewhere we could actually use that delayed jit symbol resolution lookup gizmo you were asking me about way back when
19:14:12
bike
since we don't actually need to resolve a function's native code before it's actually called. but that's tangential
19:14:28
bike
nd if we have the binary anyway it's not like we'd be delaying real compilation work...
19:22:20
drmeister_
Object files also let us use RIP relative addressing. GC managed memory for lexicals needs to be within 4GB of the code that manipulates them. I was able to control where relocated code from object files was placed and so I was able to get it into GC managed memory CodeBlock_O objects.
19:24:26
drmeister_
I was pleasantly surprised that things work as well as they do given we don't have much control over the Boehm GC.
19:26:08
bike
do you happen to know how much performance improvement we get from rip relative literals?
19:30:29
drmeister_
I don't want to imagine a world where we didn't have RIP-relative addressing. I can imagine a better world where we embed GC managed pointers inside of code but anything worse than RIP-relative addressing would be way, way worse.
19:31:34
drmeister_
Watch that video I linked above. The Orc JIT supports materializing code in user defined formats - we could materialize bytecode as native code with the B2B compiler.
19:32:58
bike
the thing with rip relative addressing isn't that i have any better ideas, just that i'm wondering how much it matters how fast literal access is
19:36:28
drmeister_
I've worked with pure bytecode builds and faso-bytecode for weeks on end. I can't tolerate pure bytecode on the cluster - I can feel the drag on everything.
19:39:44
bike
i want to remove the faso format and just stuff native code into our single fasl format. simpler
19:39:58
bike
i guess first i just need to figure out how to make a lisp function from a blob of code, without running any startup function stuff
19:41:03
bike
might need to also figure out how to initialize the rip-relative literals vector from outside of a startup function
20:15:43
bike
so JITDylibs... the underlying LLVM object has a name field, and our wrapper objects have a separate name, that's a lisp string
20:16:20
bike
our name and the llvm name are apparently not synced, because if i change the snapshotSaveLoad test to use the llvm name snapshots stop working
20:22:36
drmeister_
Why "without running any startup function stuff"? If you have a good reason, the RIP relative literal vector initialization is done with my own bytecode interpreter - so maybe you just get that bytecode block and run it from the outside?
20:25:22
bike
I don't mean making the objects in the vector - I mean initializing the vector itself - which I think is done in the (native) startup function right now, although i'm not sure.
22:01:57
drmeister_
Bike: What do we call the thing we implemented where function calls of 0,1,2,3,4,5 arguments have fixed entry points?