freenode/lisp - IRC Chatlog
Search
9:05:39
dim
sometimes I wish I had the time and skills to have a look at the SBCL GC, more often I'm just so grateful that we have CCL around
9:18:27
jackdaniel
it might be very well allocation technique which requires cleverer gc, or sbcl simply allocates more (in order to have faster code for example) etc etc
9:20:00
beach
shka: I suppose it was written a few decades ago, and then it has been patched to allow for non-moving objects and such.
9:24:57
beach
shka: I would like to turn your remark around like this: There is no reason to believe that ANY free Common Lisp implementation uses modern GC technology, simply because they were all initially created several decades ago.
9:25:08
dim
fe[nl]ix: I can always try, but well, SBCL actually uses way more memory than CCL at the same task with pgloader
9:28:27
beach
A precise collector requires help from the compiler to know what is in all registers and stack locations in every stack frame.
9:29:10
beach
One way to make sure that is trues, is to divide the registers into two sets, registers containing Common Lisp objects and registers that contain binary stuff not to be traced.
9:30:10
beach
Another way is to make sure the compiler emits information for each value of the program counter, stating what register contains what.
9:31:35
beach
But, this technique has some restrictions. You can't move objects around arbitrarily, because if was really a raw integer, you will modify it, violating the semantics of your code.
9:31:43
dim
also let's note that SBCL is *way* faster than CCL, but only until it crashes because of heap exhaustion
9:33:05
beach
dim: Generational collectors depend on the generational hypothesis: Most objects die young, and the longer an object is live, the more likely it is to stay that way.
9:33:47
dim
we have a buffer eviction policy done that way in PostgreSQL, with a kind of a generation increment from 1 to 5, and 5 means forever, basically
9:33:47
beach
dim: So objects are divided into generations. It is typically done by splitting the heap into independent generations, but there are other techniques.
9:34:38
dim
as a mere CL programmer/user, how can you hint SBCL GC into doing a better job? or as a general approach, how to track what conses and how to cons less?
9:34:51
beach
Typically, the age is measured by the fact that they survived a collection of their generation.
9:35:24
dim
sometimes I'm thinking I should get myself a personal training on CL consing avoidance techniques which would include a review and improvement of the pgloader code
9:35:33
beach
dim: That's to SBCL-specific for me to know. Maybe #sbcl would be a better place for that question.
9:37:58
beach
dim: It is not clear that avoiding consing is going to make performance better, nor is it clear that it will solve the heap-exhaustion problem.
9:40:14
beach
If you reuse CONS cells, you might create pointers from older objects to younger objects. Then the memory manager may have to record those pointers specially when a GC of a young generation is performed.
9:40:25
dim
that would explain why I can't seem to be able to fix that problem, no matter how much time and energy I've spent on trying to have pgloader cons less
9:40:30
beach
Such pointers are assumed to be rare, and of you have a lot of them, performance may suffer.
9:41:54
beach
Just so you know that you need to know the details of the GC in order to know how to optimize for it.
9:42:44
dim
the main thing I do is receive bytes from the network (typically a database driver gets that for me), then I push the data (vector of strings) into a lparallel queue (pointers, no copy), then another thread pops from the queue and transform the data into another vector of (unsigned-byte 8) this time (utf-8 representation ready for pg copy protocol), and then we send that up the network again to pg
9:43:35
dim
I also have a “batch” notion that happens after the transform and before pushing in the network to pg, where I keep around everything that is going to be sent in vectors of 25000 elements, so that in case of error I can replay the vector and skip erroneous lines
9:43:50
beach
dim: Maybe the SBCL maintainers can draw some conclusion from that use case. I certainly can't.
9:59:13
xificurC
we're trying out ABCL in a java app, planning to use it as a scripting language. My problem is with multithreading. If 2 threads are creating a new package will they clobber each other or can I ensure thread safety for them
12:43:21
xificurC
anyone knows an equivalent of `dirname $0`? I.e. finding the directory of the lisp file that is being loaded
14:15:44
klm2is
When I define a method in allegro cl with (defmethod foo ((x string)) (declare (ignore x)) (values)), I get a compiler warning for the variable x being used although it was declared ignored. Is this true of other implementations? Should it work this way?
14:17:08
Bike
"The expansion of the defmethod macro ``refers to'' each specialized parameter (see the description of ignore within the description of declare)." says clhs defmethod
14:17:10
klm2is
Makes sense. I was trying out cl-mustache and noticed that it has code like this in some places. I didn't know if that was an implementation thing.
14:19:14
klm2is
@Bike I read that, but I didn't know whether it's then required for the implementation to provide a warning.
14:21:06
Bike
it's not required to, but it's certainly allowed to, which is what you're concerned with, no?
18:13:25
pjb`
creating git repository should always be the first thing to do in a project (well just after ordering the t-shirts).