libera/#sbcl - IRC Chatlog
Search
9:36:26
semz
Maybe a silly question, but: Why can't SBCL just increase the heap size when it runs out of memory?
9:38:03
semz
I assume it's related to the GC, but I can't think of anything concrete that would cause this restriction.
9:39:17
hayley
The metadata used by the GC (page table, card maps) is of fixed size too. The card maps being the more damning, as code would have to be patched in order to grow the card mark table (or the table would have to map multiple cards to one byte, which would get funky).
9:39:17
jackdaniel
it could at least reserve some memory to signal an out-of-memory condition, so the user could run the gc or free some memory beforehand (instead of dropping straight to ldb)
9:47:11
flip214
semz: at least on Linux only really used memory is mmap()ped - so it should be safe to just use a large dynamic space, sbcl won't use all of that anyway
9:47:43
flip214
BTW, is there any work being done for hugepages? hayley, perhaps your allocator might use them?
9:48:16
hayley
Haven't heard of any use of hugepages yet, and currently I just use the normal gencgc page sized pages.
9:49:48
hayley
Is it a bad idea to rely on transparent huge pages? One could presumably disable the GC punching holes in its heap, and then things would be nice and contiguous.
9:49:52
flip214
well, for large heaps hugepages drastically reduce the amount of metadata (in the kernel), and increase the performance
9:51:01
hayley
I understand what hugepages are - makes the TLB happy. But I haven't played with them before (and I say that like I know what transparent huge pages are, and like I haven't accidentally made SBCL really slow by punching too many holes in the heap).
9:51:15
flip214
semz: well, don't use overcommit, then - and drop swap space while you're at it, too
10:31:35
luis
Our GUI seems to be spending 90% of time in SB-PCL::INSTALL-OPTIMIZED-CONSTRUCTOR when displaying some dialog for the first time. SB-PCL::PRECOMPILE-CTORS seemed promising (after tweaking it to not skip unfinalized classes) but for some reason I'm still seeing SB-PCL::INSTALL-OPTIMIZED-CONSTRUCTOR being invoked even though virtually all of
10:35:43
_death
oh yeah, I noticed this kind of runtime compilation as well.. was it always there or is it something new?
10:39:50
luis
I have no evidence that it's a new behaviour. In fact, I'm stuck on SBCL 2.0.2, sadly.
10:47:06
luis
Another weird behaviour is that I-O-C is invoked on the first and second executions of make-instance. Stops being invoked after that.
10:52:33
luis
Can't reduce that behaviour to a simple defclass + make-instance so it must be some other stateful behaviour causing that.
11:21:43
_death
maybe initargs or safe-code-p changed.. I'd set a breakpoint in ensure-ctor after precompilation to see
11:23:23
luis
ENSURE-CTOR is not being called again, but the calls to INSTALL-OPTIMIZED-CONSTRUCTOR still come from an initial constructor. hmm...
12:21:39
Shinmera
luis: while you're here, people are clamoring for a new cffi release. Could you make that happen?
13:55:18
luis
_death: at runtime, I'm seeing GF call -> initial-dfun -> make-checking-dfun -> use-dispatch-dfun-p -> dispatch-dfun-cost -> class-applicable-using-class-p -> update-ctors -> install-initial-constructor
14:13:26
luis
Krystof: it appears so, because I had to tweak SB-PCL::PRECOMPILE-CTORS to call SB-PCL:ENSURE-CLASS-FINALIZED to make it actually compile the ctors
14:14:40
Krystof
anyway, could you ensure all your classes are finalized before calling precompile-ctors instead?
14:26:51
luis
out of curiosity, replacing COMPILE with (let ((*evaluator-mode* :interpret)) (eval ...)) in INSTALL-OPTIMIZED-CONSTRUCTOR really speeds this particular benchmark up :D
14:28:51
Krystof
and actually it looks like finalizing a superclass will trigger updates to subclasses including re-initializing ctors
14:29:20
Krystof
so, maybe make sure that the root of your class graph and all of its subclasses are finalized before precompiling ctors?
14:37:55
luis
CommonQt has a finalize-inheritance :after method that finalizes all of the super classes. No idea whether that's relevant https://github.com/commonqt/commonqt5/blob/master/meta-classes.lisp#L266
15:14:20
luis
Side note: isn't it a bit much to produce different ctors for (make-instance 'foo :x 1) and (make-instance 'foo :x 2)?
15:31:32
luis
Krystof: your suggestion of pre-finalizing all classes (not just those that have ctors!) works! (Except commonqt doesn't like it and some widgets then fail to instantiate for some weird reason. But it's progress nevertheless.)
16:41:00
luis
Krystof: seems to be working, thanks! Pre-finalizing all the classes (or just the subset I care about), then calling SB-PCL::PRECOMPILE-CTORS seems to improve things, assuming I can do this during compilation before saving the image, which I haven't yet tested. And I'm sure someone will be annoyed by an extra minute of compilation time per
16:45:44
luis
_death: so, what was happening was that although I did pre-finalize all the classes mentioned in *all-ctors*, some superclasses didn't have ctors and when those were finalized, that'd invalidate all the ctors for all of their subclasses.
16:47:45
luis
This cascading effect explains why INSTALL-OPTIMIZED-CONSTRUCTOR would only go away on the third run.
19:47:26
Shinmera
luis: Strangely, sb-pcl::precompile-ctors doesn't seem present in my image, even though it is in the sources