freenode/#clasp - IRC Chatlog
Search
5:29:20
drmeister
::notify Bike The future branch is now open for business. On bigmac you use the llvm in /opt/llvm-project-tot. bclasp builds but cclasp fails in inline.lisp. If backtraces are still not up to snuff in slime we can switch things over to use DWARF to interrogate the stackmaps and get rid of the old stackmap recording/searching mechanism.
6:37:29
drmeister
::notify Bike The next release of MMTk will be coming out in two weeks and Steve Blackburn expects we should have immix within 4 to 6 weeks. He's interested in developing conservative GC that looks like the following (I'm posting a link to one of his papers - I haven't read it - but will soon).
7:22:46
no-defun-allowed
Is RC Immix the only incremental Immix? I guess the numbers can't lie about how they optimised deferred refcounting, but you now have the cycle-breaking full collections to fear, in place of your usual major GC.
13:26:47
Colleen
Bike: drmeister said 7 hours, 57 minutes ago: The future branch is now open for business. On bigmac you use the llvm in /opt/llvm-project-tot. bclasp builds but cclasp fails in inline.lisp. If backtraces are still not up to snuff in slime we can switch things over to use DWARF to interrogate the stackmaps and get rid of the old stackmap recording/searching mechanism.
13:26:48
Colleen
Bike: drmeister said 6 hours, 49 minutes ago: The next release of MMTk will be coming out in two weeks and Steve Blackburn expects we should have immix within 4 to 6 weeks. He's interested in developing conservative GC that looks like the following (I'm posting a link to one of his papers - I haven't read it - but will soon).
14:10:48
drmeister
I don't have arguments in backtraces switched over to use the object files yet - so they may not work.
14:12:30
drmeister
The idea now is that there will be a single list of ObjectFile_O objects and each one points to a Code_O object. The ObjectFile_O object gives us DWARF and the Code_O object keeps track of the .text sections (relocated code) and .stackmaps
14:13:36
drmeister
In the future we could change the ObjectFile_O objects to leave them on disk and only mmap them when we need to scan them.
14:15:27
drmeister
We can leave faso files on disk for things like quicklisp code and image code and only keep the object files for code we ourselves JITted. I think in the image save format I'll set the object files up so we mmap them.
14:34:29
drmeister
The llvm::MemoryBuffer class supports mmap'ing from files and it supports representing blocks of memory in memory.
14:35:41
drmeister
So an ObjectFile_O object will basically preside over an llvm::MemoryBuffer that represents a unix object file and that can exist in memory or on disk.
14:36:21
drmeister
The image save/load format will put all the live object files into one image and use mmap to search them.
14:42:02
drmeister
I'll need to rearrange the filenames in src/llvmo and clean it up. What they do now barely matches the source filenames.
14:43:57
Bike
when building i get "Cannot find the external symbol FUNCTION-INFO-REFERENCE-INDEX in #<PACKAGE COMPILER>" while loading/compiling translate.lisp
14:44:34
drmeister
When building iclasp or aclasp or bclasp everything just worked. Where are you seeing this?
14:48:43
Bike
this looks like a merge failure https://github.com/clasp-developers/clasp/blob/future/src/lisp/kernel/cleavir/translate.lisp#L526 a function by that name is called here, but it doesn't seem to be defined anywhere.
14:58:01
drmeister
But this is triggering another memory - I think I need to fix something else - checking...
15:11:06
Bike
When calling MAKE-FUNCTION-INFO with the lambda-list (&KEY FUNCTION-NAME LAMBDA-LIST DECLARES SPI) the bad keyword argument :FORM was passed
15:27:52
Bike
okay, so now the problem is that a call to irc-create-invoke-wft gets NIL when it's expecting an array (a string, maybe?)
15:33:19
Bike
irc-call-or-invoke is now expecting (function-type function args), but closure-call is passing (entry-point args). where should it get the function type?
15:35:31
Bike
i have to change this in a lot of places. can irc-call-or-invoke not get the function type itself from the number of arguments?
15:37:34
Bike
i'm also not sure if LISP function type is going to work when unsafe-FOREIGN-call uses irc-call-or-invoke too
15:39:11
drmeister
jackdaniel: I hit a brick wall with Boehm's precise mode I can't get anything that should work to work without segfaulting after some time. It's been really frustrating.
15:39:38
drmeister
There is a new GC coming up that we are going to transition to as it matures. It's called MMTk.
15:42:28
drmeister
It's not suitable for us yet - but in the next couple of months it will get conservative GC.
15:46:27
drmeister
The thing is - all the work that we did to support the Memory Pool System - I can reuse for any other garbage collector.
15:47:11
drmeister
It turns out that the heavy lift is being able to describe where all the pointers are in all your objects and the size of your objects from the header.
15:47:49
drmeister
Oh - and following a coding discipline that allows MPS to work was also necessary.
15:48:37
drmeister
It also underperforms in multithreaded applications. There is a lock in the page allocator that you slam into right away.
15:50:07
drmeister
Yep. I am developing trust with the researcher/group that is leading the MMTk effort as I read his papers.
15:51:08
drmeister
Boehm and MPS were developed in the 80's and 90's - a new generation of memory manager development for modern processors will be interesting.
15:52:18
drmeister
Boehm is impressive as heck - but to GC code I need to get the precise GC to work and I can't get it to work and nobody answers my questions about it.
15:53:12
jackdaniel
that would probably require some quiet time with a debugger (and knowledge about clasp internals:)
15:53:48
jackdaniel
that said, if I were making a bet, I would think that getting bdwgc precise gc mode right would be easier
15:55:07
drmeister
The MMTk looks like it's going to be more a'la carte - where we can mix different capabilities together to tune the GC for our needs.
15:55:47
drmeister
Like precise on the heap, stackmaps on the stack for precise pointers that keep objects alive and pinned and support internal pointers at the same time.
15:56:56
drmeister
That's just my weak naming abilities run amok. I can't even remember what 'irc' was supposed to stand for anymore.
15:58:28
drmeister
It's a big collection of functions that each generate a few llvm-IR instructions.
15:59:35
drmeister
Yeah - we want to pass the function type now - it's often easy to get if you have the function.
16:03:11
drmeister
I'm going to try first to get the function-type using: (let ((function-type (llvm-sys:get-function-type entry-point)) ...)
16:06:30
drmeister
So I'm basically converting... (cmp:irc-call-or-invoke function arguments ...) to...
16:07:03
drmeister
(let ((function-type (llvm-sys:get-function-type function))) (cmp:irc-call-or-invoke function-type function arguments ...) ...)
16:08:40
drmeister
I'm remembering now the reason. llvm12 changed the IRBuilder::CreateCall and CreateInvoke
16:09:20
drmeister
And it wasn't totally trivial, I couldn't just do like I did above in every case - there were places where I had to cook up a function-type.
16:10:05
drmeister
So I added the extra argument for irc-call-or-invoke by sticking by inserting a function-type argument before all the others.