freenode/#clasp - IRC Chatlog
Search
16:35:31
drmeister
We need to do a little work with the static analyzer and extensions to let them work together better.
16:36:30
Bike
by the way, i tried delecting the debug_unixes and debug_macosx files outright, but it caused some nasty early build failures. looks like there's stuff threaded through the startup code dealing with registering shared objects or something
16:38:46
Bike
terminology, you mean? i don't know. i just think "shared object" since i'm used to seeing .so files
16:39:33
drmeister
I'm starting to think that the static analyzer should generate a description of objects and their layouts rather than generate C++ code like it currently does. Then have the scraper fold the description of objects from the static analyzer into the C++ code that the scraper generates.
16:40:40
Bike
some kind of declarative description does sound preferable. easier to use if we need to change gc guts like for that rust thing
16:41:16
drmeister
I think it's historical - we register shared libraries at startup because we used to setup symbol tables at startup. Then I started to use the shared libraries list for other things like stackmaps and backtraces.
16:46:25
Bike
well the new debugger code doesn't do shared object stuff at all, it just goes through the regular objects
16:47:36
drmeister
Right - I think with snapshots and faso files we completely avoid C++ DWARF information.
16:49:08
drmeister
Regarding the static analyzer - if I switch to generating a description of classes then here's what we will get.
16:49:43
drmeister
If you run the static analyzer on cando you get a description of clasp&cando classes.
16:50:27
drmeister
If you added extension FOO you would run the static analyzer with just clasp and the FOO extension and you would get a description of clasp&FOO classes assuming FOO has exposed classes.
16:51:18
drmeister
If you then assembled a clasp+cando+FOO system the scraper would merge the clasp, clasp&cando, clasp&FOO descriptions into a clasp&cando&FOO description.
16:52:57
drmeister
This way we would have to run the scraper N times for N extensions but we could assemble them into any combination as long as they are all internally consistent.
16:53:33
drmeister
There are versioning issues with this. If your clasp description of a class doesn't match the clasp&cando description - then those two versions are out of sync.
16:54:24
drmeister
We can tolerate moving fields around and adding and removing POD fields from exposed classes. But it cannot tolerate adding or removing GC managed pointer fields.
21:29:06
drmeister
My guess is you were removing code from debug_macos.cc and got that working but didn't make corresponding changes to debug_unixes.cc - correct? I'm looking to carry out similar removal of code from debug_unixes.cc and I'd like to touch base with you as to what you did.
21:35:15
Bike
like i said, i wanted to just delete both of them entirely, but it takes a little more subtlety
21:39:14
Bike
the new debugger isn't using the registered shared objects in any way but removing that code outright doesn't seem to be enough maybe
21:41:16
drmeister
startup_register_loaded_objects is defined in both debug_unixes.cc and debug_macos.cc and it is called from main.cc
21:44:11
Bike
the new debugger code doesn't use anything in either debug_ file. it just gets the stackmaps through the object files
21:44:25
Bike
which go through the, what's it called, the AllObjectsLoaded variable or whatever it is
21:45:27
drmeister
dbg_safe_backtrace() - we call that from a couple of lonely places. We don't have that anymore - right?
21:47:19
drmeister
I'm going to define a stub for it in debugger.cc that will simply print a message and abort().
21:52:35
drmeister
There might be a role for a dbg_safe_backtrace - we would use it to dump a backtrace when things are going belly up and we want some idea of what happened.
21:53:24
drmeister
But there's another way we could go with this. backtrace(...) backtrace_symbols(...) and the GDB JIT API to recover the dwarf info
21:56:43
drmeister
What I'm still missing to understand how the gdb jit interface works is to know where the text section for each object file is loaded. That isn't defined in the gdb jit interface.
22:31:42
drmeister
Bike: How much code we can remove depends on what direction we are going with fasl files.
22:32:27
drmeister
1. faso/fasp files - these are lists of object files that are added to the JIT and they carry their DWARF with them - this is what we are currently using.
22:34:01
drmeister
2. .so/.dylib files - we can link all of the object files into a shared library and load that. This won't work with our current backtraces.
22:34:53
drmeister
3. We can use LTO to link all of the bitcode for C++ and Common Lisp and then turn that into a single executable. This has problems though - it doesn't work for quicklisp code - that needs 1 or 2.
22:35:48
drmeister
Backtraces as you have developed them won't work for 3 either until we add more code to handle shared library and executable DWARF.
22:36:42
drmeister
Option 3 could improve performance if we could figure out how to inline C++ into CL and vice versa.
22:37:29
drmeister
But I think we should go a different direction - I think we should go with (1) and for performance in the future we implement beach's call site optimization.
22:39:53
drmeister
::notify bike Could you take a look at the log - I'd like to talk about REDUCING our linking options which would allow us to remove code from debug_unixes.cc/debug_macos.cc . It would also mean turning agh
22:40:44
Colleen
Bike: drmeister said 50 seconds ago: Could you take a look at the log - I'd like to talk about REDUCING our linking options which would allow us to remove code from debug_unixes.cc/debug_macos.cc . It would also mean turning agh
22:41:31
drmeister
I've been keeping shared object/library linking alive because it's very similar to LTO.
22:44:11
drmeister
I implemented faso/fasp files a year ago because I wanted to try this experiment where we load object files into the JIT and link at load time.
22:44:48
drmeister
I think the result is that this works very well. It's fast and it looks like we can do it multicore and make it almost imperceptibly fast.
22:45:10
drmeister
Previous to this we were linking JITted code into .so/.dylib files and those were our .fasl files.
22:46:47
drmeister
Ah - we will still be able to load those - we just get their return address symbols using backtrace_symbols(...)
22:48:22
drmeister
Now that I think about it - I don't know why we are calling: add_dynamic_library_using_handle(...) from core__load_binary(...)
22:50:39
drmeister
For some reason we call add_dynamic_library_using_handle(...) from core__load_binary(...) and I'm thinking of completely removing core__load_binary(...), which would remove the call to add_dynamic_library_using_handle(...), which would start tear out some code from both debug_unixes.cc and debug_macos.cc
22:51:37
drmeister
If I remove core__load_binary(...) then I can remove all the clasp and wscript code that links .fasl (.so/.dylib) files.
22:52:50
drmeister
It would remove the CLASP_BUILD_MODE="fasl", which we haven't used in about a year.
22:53:48
drmeister
That would close off the idea of using LTO for optimization - but it doesn't work properly without a lot more work.
22:54:25
drmeister
I'm thinking for optimization we go in a different direction - we implement beach's call site optimization. That works with the faso/fasp files fine.
22:56:03
drmeister
We could do call site optimization to have Common Lisp code call C++ code that takes doubles directly.
22:57:20
drmeister
If we know a C++ function takes a double and we call it from CL and we have the value in an unboxed double - then we can create a call-site that takes it from wherever it is in the CL function and put it in whatever double register the callee needs.
23:07:22
drmeister
My basic point is I think with faso files and call site optimization we can get the kind of performance that we could get with LTO and in a much more flexible/lispy way.
23:22:52
Bike
the c++ functions are static, so i would think we can do that without the call site optimization? just generate calls with the unboxed parameters