freenode/#lisp - IRC Chatlog
Search
13:34:52
|3b|
emaczen: can you load that 20MB file ok if you don't save an image? what do you load it into and is there any processing/translation during loading or just read-sequence?
16:04:09
emaczen
|3b|: with CCL it works fine from the REPL, with SBCL from the REPL it drops me into LDB and tells me "No more immobile pages"
16:09:06
|3b|
emaczen: hmm, might try newer sbcl if you don't mind compiling it from source, sounds like something that needs fixed if it isn't already
16:20:41
|3b|
can you try current code from git? (easier to try that first than to try to reduce a test case and then find out it is already fixed :)
16:37:04
|3b|
maybe a paste somewhere with code + actual and desired output? (assuming you didn't figure out out already)
17:06:34
|3b|
shrdlu68: https://github.com/quicklisp/quicklisp-projects/blob/master/projects/ironclad/source.txt
17:07:15
emaczen
|3b|: I just git cloned the latest SBCL and I get the same "No more immobile pages available" fatal error and got dropped into LDB
17:08:28
|3b|
emaczen: can you describe how you are loading it? (or point to code if it is publicly available somewhere)
17:12:08
emaczen
|3b|: I'm reading a 20MB plist into a CLOS object, from a file using something like: (with-open-file (...) (apply #'make-instance (read ...)))
17:15:11
shrdlu68
emaczen: It's nothing wrong with SBCL (except maybe for the non self-explanatory error message, means you've run out of heap space.
17:16:22
emaczen
shrdlu68: I passed 2 gigabytes to the dynamic-space-size argument to start SBCL -- would you recommend something different? It is working in CCL and I haven't had to pass any command line arguments
17:18:26
|3b|
ACTION usually uses something close to the amount of ram in the system, so it will run out before everything in system is swapped out
17:19:11
|3b|
ccl by default either allocates some huge amount of address space by default or doesn't need to reserve it to start with, don't remember which
17:19:53
|3b|
might also see how much ram CCL is using (RES or similar in top rather than VIRT) after you load the data
17:40:49
|3b|
might check and see if it is missing "" anywhere, too many symbols seems to be the most likely cause of that error i've found so far
17:46:33
emaczen
Wait, I found at the end of the file a bunch of unique symbols, the structure is: (class-name :foo ((class-name :bar "a lot of text") (class-name :bar "alot more text") ...) ... :unique-symbol1 "x:y:z" :unique-symbol2 "a:b:c" ...)
17:47:02
emaczen
let me know if you have questions about this structure. It is 22.6MB according to emacs when I open it up.
17:47:45
|3b|
as far as i can tell, sbcl doesn't like large #s of symbols ~500k in my test, not sure how much if any that depends on length of name
17:48:18
emaczen
|3b|: let me see if I can get a count -- CCL is fine with it in the REPL but not when I built an application
17:51:58
emaczen
|3b|: I just made up of this format based on my classes and their slots -- don't people use the MOP for serializing CLOS classes?
17:53:10
|3b|
format seems reasonable, but if you are interning hundreds of thousands of keywords, that sounds less reasonable
17:55:34
|3b|
if nothing else a symbol has a bunch of extra data in addition to a string, and won't be GCd once it is interned into KEYWORD (unless you explicitly unintern it)
17:57:11
|3b|
#: should be OK, though seems like it would lose most of the benefit of a symbol over a string
17:57:53
|3b|
(multiple with same name would end up as distinct symbols, and it would be hard to find them based on the name without a manual index)
17:58:57
|3b|
ACTION thinks immobile space is used by other things in addition to symbols, so my ~500k in a fresh image is an upper bound on how many is too many
18:01:45
emaczen
|3b|: how do you know that is an upper bound? Why would this work with CCL in the REPL but not when I build an executable with CCL?
18:02:56
|3b|
sbcl fails due to a particular optimization it does, under the assumption that people won't create that many symbols
18:04:40
|3b|
CCL and SBCL just save the heap and similar and tack it onto a small runtime executable
18:09:38
emaczen
|3b|: Alright, well I'm going to map over this list and turn the keywords into strings and reserialize these objects and then try it out... I'll let you know what happens
18:49:40
TruePika
meh, I don't like how every time I need to modify a project's .asd it has everything get built again
18:51:56
TruePika
yeah, I understand that, but if I want to add a file downstream, it still recompiles everything
18:53:28
TruePika
the only solutions I can think of involve using multiple .asd files, but I'm not even sure if that would work as desired unless I make multiple different systems
19:14:38
emaczen
So my executable seems to be working, it calls the toplevel function, loads the 20MB file, starts the server, but then terminates. How do I get it to not terminate?
19:16:41
|3b|
either run the server directly instead of starting a new thread, or wait for the server thread to exit
19:20:27
|3b|
sleeping indefinitely would be easiest solution, not sure how to ask hunchentoot which thread to wait on, and which function runs the normal repl is implementation specific
19:25:50
|3b|
(in case you don't think 391 billion years might not be enough, or worry about implementation bugs in edge cases like that)
19:27:44
|3b|
most probably tries to avoid having that particular problem, i assume there is some way to do it 'correctly' in hunchentoot
19:28:01
|3b|
aside from that there is usually also some external component that restarts it if it crashes, etc
19:29:49
|3b|
ACTION usually just runs a repl and starts hunchentoot manually from that, but i'm the only user so if it crashes i can just restart it next time i want to use it :p
20:43:35
dim
in start-server, I have (hunchentoot:start *acceptor*) (setf *server-is-running* t), obviously, and in stop-server setf to nil
20:45:17
dim
https://github.com/dimitri/pginstall is an hunchentoot based web application that ships as an self-contained binary image and does the trick, emaczen, if you want to have a look
20:45:39
dim
https://github.com/dimitri/pgcharts uses the same trick and might be simpler to approach
20:55:05
emaczen
I have a file which contains (asdf:load-system my-system) and I am executing "sbcl -load file" in the terminal but it doesn't load the system -- what am I doing wrong?
22:51:45
enedil
Let's say I'd like to form a function that is always 2 times smaller that a given one
23:12:54
enedil
Bike: in fact it doesn't help if I use funcall, clisp shows "EVAL: (TR (LAMBDA (X) X)) is not a function name; try using a symbol instead"