freenode/#clasp - IRC Chatlog
Search
13:07:33
drmeister
I rearranged backtraces so that now when a fasl is loaded clasp builds and stores an address sorted symbol table and stackmap table for it.
13:08:13
drmeister
So backtraces are fast and we know the start and end of every compiled Common Lisp function.
13:12:00
drmeister
The symbol tables are about 5MB of information for cclasp before I load anything for cando.
13:33:19
drmeister
The notices about trying to freeze memory are related to the backtrace generator and printer that doesn't gc.
13:35:11
drmeister
In a crash I want to be able to generate a backtrace with arguments that will work even if the gc is fubar
13:36:15
drmeister
So for that backtrace generator I had to break rule #0 - "don't put GC pointers in C++ memory".
13:38:39
drmeister
But I still freeze the gc while the backtrace is being generated - then it passes it to Common Lisp and unfreezes the gc.
13:39:21
drmeister
I have to figure out how to freeze boehm. It may not be a problem because there should be plenty of pointers to the objects in the boehm memory and roots to keep everything alive.
13:40:47
Bike
i mean, ideally, i don't think backtrace printing should involve manipulating the gc in any way. i can see having a fallback that does, though.
13:41:22
beach
Is it possible to explain in a few short sentences why GC is not allowed during backtrace?
13:41:58
Bike
because drmeister wants to be able to get a backtrace even in situations where the system is in a really bad place, e.g. if something went wrong with the gc.
13:42:21
drmeister
A std::vector of BacktraceEntry frames is generated while the gc is frozen/parked. Then a Common Lisp function is called for each frame to create a Common Lisp representation. Then the gc is unfrozen/released.
13:43:03
Bike
print-object can be arbitrary code, after all. hell, it might do something complicated like want to compile a new discriminating function
13:43:27
drmeister
I'm pretty sure I can park the mps gc and still allocate things. It just won't move objects or collect them.
13:44:31
drmeister
This is just done for the few moments while the backtrace is generated and transferred to Common Lisp.
13:45:26
Bike
it's also kind of a cross cutting concerns thing. if possible i'd like the backtrace generator and gc know nothing about one another.
13:45:44
drmeister
I'll explain when I get in - I think it's all ok - but maybe you can find a flaw in my thinking.
13:47:55
drmeister
I think you will like it - the backtrace generation doesn't know anything about the GC. It's used by the bullet proof backtrace printer (no gc) and it passes the backtrace information to Common Lisp and it uses its own printer that gc's.
13:48:51
Bike
i mean, basically, when i'm in slime doing normal things, am i going to see that boehm warning
13:49:19
drmeister
The reason I did all this is because way too many times when I called the Common Lisp backtrace printer from gdb or lldb it comes back and says it hit an error. It made me crazy.
13:51:57
stassats
Bike: print object doesn't need to be run with the gc disabled, unless you want something like call-with-each-frame
13:53:07
stassats
if you do need to stop the world, i don't think it needs to do any lisp allocation, at least theoretically
13:55:02
drmeister
fill_backtrace(backtrace,true); // <<-- This fills a std::vector<BacktraceEntry> with backtrace information from many sources (symbol table, JITed symbols, stackmaps)
13:55:58
drmeister
The rest of the function generates CL structs 'core:backtrace-frame', a list of which are returned to Common Lisp.
13:56:52
drmeister
The bullet proof backtrace printer uses fill_backtrace(backtrace,true) and then prints the frames using its special non gcing printer.
13:59:44
drmeister
The new symbol tables are available to be binary searched with addresses to look up symbols
14:03:34
drmeister
Aren’t they? The key can be a range of addresses anywhere between the start and end of the function
14:04:39
stassats
can't you just get the address, go there, and backtrack until you hit 'function FOO starts here'
14:05:54
Bike
so the gc is not frozen during the part of backtrace printing that does actual printing? i guess that might be ok
14:06:42
drmeister
Why not a sorted list of addresses and use a binary search? It seems like the best solution
14:09:40
stassats
it doesn't actually need to be any fast, how many functions are you going to have, a 1000?
14:13:50
drmeister
The symbol table is built once for each fasl on loading. It’s deleted when you reload a fasl. That’s all the maintenance there is.
14:15:23
drmeister
That is jitted - that has a separate symbol table and they just keep accumulating.
14:21:30
drmeister
Not yet at least. We have a scheme but it’s tricky. We would keep object files and generate new relocated code when we need to move code.
14:23:46
drmeister
We do use position independent code. But there is no such thing as a position independent itanium unwinding table. Those get generated alongside the code to handle unwinding. I’m still learning about this thought.
14:26:05
drmeister
I’ve been learning a lot about elf files and machO files. Maybe I’ll figure something better out soon.
14:29:41
drmeister
Since we maintain c++ compatibility - I need to work with these complex systems that I am still figuring out.
19:11:19
specbot
ensure-generic-function-using-class: http://metamodular.com/CLOS-MOP/ensure-generic-function-using-class.html
19:12:47
Bike
defmethod uses ensure-generic-function. ensure-generic-function calls ensure-generic-function-using-class. ensure-generic-function-using-class calls reinitialize-instance. so we do reinitialize-instance for any defmethod.
19:14:01
Bike
except that in clasp when a generic function is reinitialized we wipe the call history
19:41:08
Bike
I"m really not clear on what happens if, like, you reinitialize-instance a generic-function to change its lambda list but leave its methods there
19:41:30
Bike
i think i'll just avoid erasing the history if the reinitialization doesn't actually change anything
19:45:04
Bike
this also means that all that complicated code for keeping the call history consistent when a method is added has probably never been tested
21:23:08
copec
/peanut_gallery You guys should try to link clasp with Unreal Engine for some VR molecule building
21:24:07
drmeister
Ha ha - I'd love to see that - I'd even support it. Maybe we should do that as a Google summer of code project.
21:26:52
copec
I'm doing a personal project right now using wikihouse and a maslow cnc to build an office studio, and working with the current wikihouse software stack in https://www.grasshopper3d.com/ has made me interested in building all sorts of things out of plywood/osb, using some good old fashioned AI to solve problems
21:33:10
copec
I came across this the other day, very neat visuals: https://www.youtube.com/watch?v=WFCvkkDSfIU
21:49:17
drmeister
Oh great - now I'm seeing a similar crash in llvm to the one we were seeing a couple of days ago.