freenode/#lisp - IRC Chatlog
Search
22:26:14
stylewarning
On SBCL, I have a function F which allocates lots of memory, but doesn't maintain the references. (loop (f)) causes a heap exhaustion, while (loop (gc :full t) (f)) does not. Any hints on why this could be?
22:31:52
didi`
stylewarning: One thing that I observed is that it consumes memory before freeing memory.
22:34:18
stylewarning
I wonder if SBCL tries to do a full GC. It looks like all of my objects are in gen1 or 2, and SBCL is just ignoring those.
23:25:45
pierpa
the hash tables that the function F creates are pretty big, so even a small percentage of them being kept because of false positives could explain the phenomenon.
23:26:34
stylewarning
pierpa: It would have to be holding on to a few of the large tables from a previous call to F
23:26:46
pierpa
this theory agrees with the forst two of your starred points, while I don't understand the third one
23:27:00
Bike
if it was conservatism, how would calling gc manually fix it? it's not any less conservative just because it's forced
23:28:27
stylewarning
I'm trying to figure out in the SBCL source where GC gets called upon a failed allocation
23:57:23
stylewarning
Looks like all the interesting stuff about GC triggering on allocation starts here: https://github.com/sbcl/sbcl/blob/master/src/runtime/gencgc.c#L3739
0:05:54
didi`
In my case, usually I have some percentage of the heap allocated, say 80%, then the GC is triggered and the heap is exhausted.
0:08:31
stylewarning
didi`: That variable seems like the wrong one to tune for a bug like this, no? IIUC, the default is 5% of the heap size, and that means when 5% of the heap is used, a GC of some sort will be attempted.
0:10:16
didi`
stylewarning: Oh, I don't know how to fix it. My workaround has been trying to use less and less programs during my work and aggressively cut my memory usage inside my program.
0:14:29
didi`
stylewarning: Another "fix" I employ is calling the GC from time to time inside the REPL.
0:14:57
stylewarning
didi`: this bug is unfortunately being trigged in the depths of a complicated compiler pass
0:42:42
stylewarning
makes me think there's something awry with collecting large objects, which I know are allocated specially
0:49:29
pierpa
as, in the previous case you needed a RANDOM integer to trigger it. In this case, uninitialized arrays. So, vectors containing random fixnums would seem to play a part in this.
0:51:16
stylewarning
pierpa: the random integer bit was because i needed different objects for EQUALP
0:51:37
stylewarning
but I changed that to EQ so all objects, even with the same contents, are basically different
1:04:36
aeth
I'm guessing that other threads (e.g. SLIME+Swank?) would trigger GC even in a non-allocating workload, but running it in a separate process entirely and seeing if the GC is triggered could be yet another test for consing.
1:05:25
aeth
Doesn't help with your problem (unless I wasn't reading it carefully), but it would be an interesting test to run on some things
1:12:10
aeth
stylewarning: Does SBCL have built in file logging or is it done as part of the *after-gc-hooks* that's mentioned in the manual?
4:59:58
mfiano
I have a stupid question perhaps, but I'm toying with compiler macros for the first time in all my years of Lisp use. Anyway, is there anyway to prevent the original function from being called in the REPL? It seems to work in a compiled file, but obviously not in the REPL.
6:10:24
jackdaniel
mfiano: or, if your implementation doesn't compie repl expressions by itself, then: (funcall (compile nil '(lambda () (foo 2))))
7:53:27
MichaelRaskin
beach: I guess the core I have can be trivially wrapped to suppotr file-agnostic code debugger; not sure what is a good way to add location support, maybe a reader that saves a hashtable of «Form object -> storage location»
7:55:46
beach
It can return a concrete syntax tree, but one of the protocol functions on that returns the underlying expression.
8:15:18
flip214
beach: mail incoming - in case you don't read the u-b.fr mails regularly. Have a nice weekend!!
8:23:24
MichaelRaskin
beach: I guess I should declare that recovery-after-macroexpand in eclector should work well enough with what I do not to think about it right now
8:29:15
beach
MichaelRaskin: I am afraid I don't understand what your plan is in sufficient detail to understand things like that.
8:31:37
MichaelRaskin
Well, as I convinced my SBCL instance that this plan works, I can show the same arguments to you. https://gitlab.common-lisp.net/mraskin/agnostic-lizard/blob/master/debugger-hooks.lisp
8:33:13
MichaelRaskin
The idea is: I have written a wrapper macro that checks a global variable before and after every form to maybe call some operation
8:35:23
MichaelRaskin
Right now to skip all the debug code in the function it just branches on the debugger-active variable in the beginning of every function; maybe the compiler is more thread-safe and won't optimise out the checks, in which case I need to improve that place a bit more.
8:36:15
MichaelRaskin
I want to give every thread a thread-local binding for the relevant global variables, and put callbacks for editing thread-local copies into a global hashtable.
8:36:59
MichaelRaskin
When debugger is activated, it sets the callback to do something, for example to talk with the debugger thread (or debugger process) and ask it if it is OK to continue
8:51:08
MichaelRaskin
I hope eclector will not be unhappy about my wrapper trick with character pushback.
8:53:06
MichaelRaskin
https://gitlab.common-lisp.net/mraskin/agnostic-lizard/blob/master/wrapping-reader.lisp
8:53:48
MichaelRaskin
I set a reader-macro on #\( that unreads that #\( reads with the previous readtable, then wraps the form in a macro
8:59:32
MichaelRaskin
Well, the bindings block is indented the same as the body, the bindings are one character to the right.
9:01:22
MichaelRaskin
Yeah, I know, in TRAG code I write for Irène I do start the bindings block on the same line as let*
9:05:28
beach
Well, it's not about personal preference. It is about conventions that exist for better communication between people. But, sure, as long as you keep your code to yourself, you can do what you want.
9:06:06
beach
But since you showed it to me, I had to tell you how distracting that style is to someone who is used to the Common Lisp conventions.
9:09:22
beach
You can use anything from SICL you like. But I don't know what an external-process autoformat tool is.
9:10:19
MichaelRaskin
Something that I can feed a file and get the newlines removed according to the conventions.
9:15:37
beach
In CST, it doesn't preserve comments, but it has hooks for doing that, and I do it in Second Climacs, though Second Climacs is still not using Eclector. I am gathering up the courage go switch.
9:21:46
beach
Such a tool would be great also for the purpose of education. People could submit their code to it and have it comment on the indentation and spacing style.
9:22:07
beach
I guess I should try to isolate that code in Second Climacs so that it could be used also as a separate tool.
9:23:15
MichaelRaskin
Can I just instantiate something like SC buffer and apply the reformatting command?
9:25:00
LdBeth
MichaelRaskin: if you specify corespondent print method for reader macros, it would work
9:25:30
MichaelRaskin
LdBeth: what I would feed the pretty printer so that the comments are even in the data?
9:28:56
LdBeth
I have some primitive ideas, first hack the reader to treat comments as docstrings attached to cons cells
9:45:57
beach
LdBeth: In Second Climacs, I preserve comments and other material for which a reader macro return no values. Eclector can do that as well and I will replace the modified reader in Second Climacs with Eclector soon. You can't arbitrarily insert comments into code, because you would then violate the syntax of many special forms.
9:46:57
beach
If the comment turns into a CONS cell, then it will be parsed as if it were the list of bindings.
9:50:41
beach
flip214: You can omit the nursery collection, but you must do something essentially equivalent, because you need to inform the global collector what objects are live. Otherwise, you have to assume that all object in the global heap that were live after the previous collection are still live, and that would kind of defeat the purpose of garbage collection in the first place.
9:53:18
fm4d
Hi, I am slightly confounded by this behaviour: (funcall #'defparameter (read-from-string "test") 123)