freenode/#lisp - IRC Chatlog
Search
9:49:31
flip214
kuribas: there is? really? always? and I always thought that the most applicable programming variant depends more on your algorithms and data structures...
10:00:41
flip214
"there is a performance penalty for pure functional programming." isn't always true
10:14:52
dxtr
I'm no compiler expert but I've always assumed that with an imperative compiler you can get away with being pretty naive (Don't have to copy data when it changes, etc)
10:15:34
dxtr
With a functional language you can't be that naive and have to do some acrobatics to make stuff efficient
10:18:43
kuribas
dxtr: a naive imperative compiler is easy. But doing optimizations in the presence of side-effects is hard.
11:09:00
beach
Here is an interesting conundrum. In SICL, I plan to implement a garbage collector with two generations. The younger generation is per-thread and the older generation is global. The collector of the older sets a flag in the application threads when it decides to run the GC. When a thread encounters a safe point, it runs its own young-generation GC and then indicates to the global collector that it has finished. When all threads
11:09:06
beach
But a thread can be stopped arbitrarily long, for instance waiting for I/O or some user gesture, or it can be stopped by a debugger thread. It would be unfortunate if the global collector could not run in such situations. So would it be OK if a stopped thread can still run the GC?
11:10:29
beach
It also means that there must be some restrictions on the places where a thread can be stopped. It has to be a place that allows for the GC to run.
11:11:31
beach
And that restriction seems to exclude stepping instruction by instruction in the debugger.
11:17:05
shka
beach: i don't see why preventing GC from running while stepping would be a practical problem though
11:18:11
beach
shka: Well, there can be other threads running at the same time, and those threads might allocate memory.
11:19:23
LdBeth
beach: how about invoke local GC after waiting and then the main GC just skip the waiting thread
11:20:11
beach
LdBeth: That is not possible, because the waiting thread may have references to objects in the global heap, and the global GC needs to know that they are live.
11:24:33
jmercouris
I have a list that looks like the following: (("login" ("username" . "username") ("password" . "password")))
11:24:59
jmercouris
is there something that I'm missing? or is there a smarter way to grab the elements?
11:29:56
jmercouris
the top level cons cells of this list are each a form, and then the level below that is each question
11:32:14
beach
fe[nl]ix: Yes, I think so too. I am more worried about the debugged thread being stopped. But I guess I can make those safe as well.
11:32:57
fe[nl]ix
beach: I don't think you need to care about those. it's an uncommon use case that won't matter much in practice
11:33:42
beach
fe[nl]ix: But I am a bit surprised here. What does it mean for a thread to be stopped? Is it still OK for such a thread to run the GC? Or should I fork a different thread to run the GC on behalf of a stopped thread?
11:35:53
fe[nl]ix
beach: a Lisp debugger is just a a function that is called and waits for UI commands. I don't see why it can't GC
11:37:12
beach
fe[nl]ix: Each thread is supposed to run its own GC. But if a thread is stopped, say waiting for I/O (forget about the debugger), is it still OK for it to run its own GC?
11:38:34
LdBeth
beach: I don’t think it would affect even there’s a requirement for real time application
11:38:43
Ukari
is it possible to refactor a function/macro/variable/class/structure's name in some common lisp develop tool?
11:40:20
beach
LdBeth: It would though. Imagine a REPL waiting for input for a very long time. At the same time, there is a thread working on some application that allocates a lot of memory. The system could then run out of memory just because the GC can't run due to the REPL thread.
11:41:38
flip214
beach: is the architecture so that all threads can access all memory, or can memory be thread-private?
11:41:59
flip214
a thread being stopped in the debugger shouldn't be a problem if the stack and all registers are available as GC roots --
11:43:01
flip214
beach: but the thread _asking_ for a GC could just as well take a lock from the non-running (stopped) thread and do GC for it, because the stopped threads' memory is mapped as well, right?
11:43:02
beach
flip214: It is more a philosophical question perhaps. Is it OK for a stopped thread to run some code, or would I have to fork a different thread for that?
11:43:58
beach
I could have (one of) the global GC tread(s) run the GC on behalf of stopped threads.
11:43:58
flip214
basically, I guess that having a lock that's taken as long as the thread is running and having the debugger (and/or IO) release that should work
11:44:29
flip214
might not be really cachefriendly, but then it might not matter that much for stopped threads
11:45:25
flip214
do you plan to have a thread pool for GC? actively starting new threads to do GC for some thread sounds heavy.
11:46:24
beach
The global GC might have a pool, or may create additional threads. It probably won't matter much.
11:47:12
beach
The question was more for a stopped thread. It would have to be not terribly stopped in order to run its own GC.
11:47:13
flip214
I've done quite a bit of work on some green-thread library in C over the last few years - and interrupting IO is a big problem.
11:47:59
flip214
well, "not really stopped" means async IO and signals or polling or something similar, and that's all a big hurdle.
11:48:51
flip214
it's the same as with FFI - you don't know what's happening, you can't reliably interrupt, so the only safe way seems to have a safepoint (excuse the pun ;)
11:49:58
flip214
that most likely means saving all registers in a per-thread structure to be used as GC roots (although that means some markup being done by the compiler?!), releasing the "running" lock, and doing the foreign call
11:50:10
beach
I am not sure I follow. So do you think it would be better to have one of the global collector threads run the GC on behalf of the stopped thread than to allow for the stopped thread to be unstopped to run the GC? If I do that, I must prevent the stopped thread from continuing its execution until the GC has finished.
11:51:00
flip214
that's why I'm talking about using a lock - so the not-stopped-anymore thread won't crash the right-now-running GC on its private heap.
11:51:40
flip214
let's say the thread tries to read from a pipe or a socket - how would you "stop" that?
11:53:11
beach
I don't think I see the problem. If a thread is stopped because it is waiting for I/O, why would it be bad to start it so that it can run the GC and then go back to waiting?
11:53:12
flip214
perhaps, later on, you'll want or need some "fast FFI" calls as in "they're guaranteed to complete quickly"
11:54:27
jmercouris
if it is waiting for I/O somehow you will have to switch to "GC" mode, while it is in the middle of doing something
11:54:35
flip214
jmercouris: that's another dimension -- "contexts" as in userspace/kernelspace, or lisp/FFI, ....
11:56:16
flip214
beach: do you have empirical data whether the thread-local heap is likely to include references to global data?
11:57:01
flip214
because if you could determine that since the last GC it's only doing "local" allocations that have no global impact, it could tell the global GC to not bother with this local heap
11:57:42
flip214
yeah, of course, but these are not likely to be released anyway - I'd imagine a (gc :full T :really-full T) flag or so ;)
11:57:55
jmercouris
beach: sorry to distract, but how did your language presentation go? how did the participants receive the information?
11:58:46
flip214
beach: _can_, yes, but some (for me) typical workloads actually won't... like a hunchentoot thread that just builds up an output stream buffer and reads some global state
12:00:13
flip214
beach: if all of your objects have a heap pointer (yeah, expensive, I know), then you might be able to build up a tree of dependent heaps during allocation
12:00:38
flip214
and if one private heap isn't referenced in this tree of heaps, it doesn't need GC (by definition)#
12:01:52
flip214
sorry, that wasn't clear enough. the global GC doesn't need to wait for that thread to mark global objects if the thread knows that it didn't reference any global objects since the last GC.
12:03:43
flip214
I don't know your use cases. Mine include hunchentoot frontends, which (most of the time) won't have any (changing) references to global data...
12:05:02
flip214
beach: if you allow me a bad pun, interrupting stopped threads might require you to solve the halting problem ;)
12:07:19
beach
flip214: Let me try to summarize to see whether I got this: When a thread is stopped due to I/O, it will wait on a lock before going back to its normal business. If that lock is taken, it means that there is a thread doing GC on behalf of the one that just got unstopped. Correct?
12:10:19
beach
shka: Because I don't think I need more than that, and more would just complicate things.
12:11:15
flip214
beach: uh, well, if the locks memory is in the RAM of the other socket and causes some L1, L2, and L3 cache areas to be invalidated, it might...
12:11:59
beach
flip214: Well, I don't care. Your examples (FFI and Web stuff) are not high on my list of important considerations. :)
12:13:20
beach
flip214: Besides, I'll have much better cache performance than a typical copying collector.
12:16:03
flip214
hmmm, I'm not sure about that... making a list's cons cells contiguous in memory during GC has a few nice performance implications, which keeping the cells distributed over multiple (private) heaps doesn't
12:16:39
flip214
perhaps all the CPUs in 5 to 10 years won't have that fancy stuff any more anyway, because of all the possible exploits
12:47:47
mansalss
hey I need a list of words from official Polish Scrabble dictionary. They are available in two forms: in the official Windows app (offline) and in online service. They both deliver an option to check word(s) and find words by anagrams.
12:47:59
mansalss
I want to create a tool for myself to become better Scrabble player, but I need to have a full list of words. I have no experience in reverse engineering. I think the best we can do here is to use this offline dictionary and find out how it works. All I know is that it's most likely encrypted in some way.
12:52:54
jackdaniel
first of all, coercing copyrighted material that way is illegal (in Poland you can't freely break DRM or any sort of security measures in copyrighted material), second it is offtopic for Lisp – sure you can create dictionary-managing engine in Lisp if that's your question, third – I'd recommend building your dictionary on free licenses, like the one present on sjp.pl
12:54:05
jmercouris
another idea might be, find all polish books on project gutenberg, and create your own dictionary. As far as I understand, you don't need the definitions as well, do you?
12:54:38
jackdaniel
that would be counterproductive given I've linked full dictionary which is ready to use
13:09:44
mansalss
<jackdaniel> that would be counterproductive given I've linked full dictionary which is ready to use
13:20:03
xificurC
jackdaniel already pointed out what you were (or still are) trying to do is illegal. Furthermore this is not a place to discuss reverse engineering
13:21:54
mansalss
<mansalss> How would you do it with that given dicitionary? <shka> mansalss: https://sjp.pl/slownik/growy/sjp-20180612.zip
13:22:47
minion
mansalss: please look at gentle: "Common Lisp: A Gentle Introduction to Symbolic Computation" is a smoother introduction to lisp programming. http://www.cs.cmu.edu/~dst/LispBook/
13:24:55
jackdaniel
you could also benefit from reading this essay http://catb.org/~esr/faqs/smart-questions.html (and this is offtopic on my side, sorry about that)
13:33:39
beach
flip214: I might ask for help if I can't work it out, and I might ask for remarks on what I write if I do think I can work it out.
14:24:43
jmercouris
I am reading "Gentle introduction to symbolic lisp" and one of the questions has thrown me off guard "Why can't the special symbols T or NIL be used as variables in a function definition?
14:24:55
jmercouris
My answer was because we are not allowed to evaluate the arguments used in a function definition
14:25:53
jmercouris
further, we need to, when invoking the function assign the symbols to a variable based on the value passed in, this is not possible for T or NIL
14:27:57
jmercouris
that's what I would have said, but they have not introduced that concept yet in the book
14:28:13
jmercouris
so apparently, somehow some way, given the knowledge already introduced in the book, someone should know the answer to the question at that point in time
14:49:42
Ukari
i wanna use with-macro in defmacro, but found it is hard to write the inner macro due to write (list 'fn ,arg) instead of `(fn ,arg)
15:17:16
Ukari
i write (defmacro logger (&body body) `(print (+ ,@body))) (defmacro outside (&body body) `(defmacro inside () `(logger ,,@body))), (outside 1 2), (inside)
15:55:25
fouric
Ukari: you're not sure if you believe me (don't worry, i'm not sure either) or you believe me you just don't know *why*?
15:58:16
fouric
which is to start with the innermost level of backquotes, try to understand that, and then work your way up/out
15:58:58
Ukari
beach, this is one of my condition, https://tinyurl.com/y8t27qoj. another is to make macro yield accessable locally instead of global
16:01:06
beach
Ukari: That's very different. You are defining a macro with the same name each time, namely INSIDE.
16:02:29
beach
Local macros are defined using MACROLET and not DEFMACRO if that is what you are trying to do.
16:02:53
Ukari
it provides a defmacro* which returns a generator and also has lambda list arguements like defmacro
16:03:32
beach
I don't think I am smart enough today to follow what you are trying to say. Sorry about that.
16:28:08
u92urksvz78
anyone out there care to share what kind of interesting CL projects they are currently working on?
16:29:20
u92urksvz78
anyone out there care to share any details about any interesting Common Lisp programming projects they are currently working on?
17:47:16
gendl
Hi, i'm working on some interesting CL projects based on http://gendl.org, don't have a lot of time to share details right now though.
17:47:47
gendl
I do have a question though -- if I have an error object in a variable (like from a handler-case), how can I get a stack trace of that actual error?
17:48:18
gendl
like (handler-case (some-form) (error (err) ;; now I want to get into the debugger directly for err . ...
17:51:31
gendl
nirved: well, yep, that does give a debugger, it seems pretty much the same as just calling (error err)
17:52:24
gendl
in the stack, I'm still seeing let bindings and such which are connected with the handler-bind
17:53:20
gendl
what I really want is a debug stack which would be the same as what I would get just from
17:53:46
gendl
[sorry -- replace 'div' with '/' -- 'div' is our operator which always returns double-floats)
17:54:38
Bike
lisp implementations don't usually store a backtrace in the condition object, as far as i'm aware
18:11:02
nirved
gendl: if you really must have the backtrace, it is implementation dependent, look at slime/swank
18:29:39
rpg
I'm embarrassed to ask this, but does anyone have an example of an ASDF system definition that quashes warnings to avoid failures? I have been using Allegro, whose COMPILE-FILE doesn't fail (as it properly should) on warnings, and now using SBCL, I can't get compile-system to work...
18:40:55
rpg
I tried muffle-warning, but it looks like compile-file may grab up the warning and fail before I can muffle it....
18:44:04
rpg
Xach: I believe it's that SBCL's COMPILE-FILE is refusing to write an output file, not that there are warnings. At least, I *think* so.
19:06:35
rpg
Xach: Aha! *SOMETHING* is rebinding those error interpretation flags, so that they are both valued :ERROR inside COMPILE-FILE*
19:22:59
rpg
hahaha. OK, I see part of the problem -- the AROUND-COMPILE-HOOK isn't scoped around the checking of those flag variables.
19:37:00
rpg
Yes, that's it -- you can bind the behaviours in an around-compile-hook, but that isn't scoped around check-lisp-compiler-warnings
20:49:44
dxtr
So I've got a plist (:|foo| 1 :|bar| 2) - and neither (get :bar) nor (get :|bar|) works
21:14:01
Xach
aeth: pcl mentions a use of them, iirc it stores some reader/writer information in the symbol plist. or you could use it in an interpreter to store the value bound to the symbol. there are other things!
21:14:19
Xach
i don't use them often and can't think of when i used them, but it's a way to associate a bit of your own app data with a symbol
21:19:37
pjb
aeth: (length (remove-if-not (function symbol-plist) (let (l) (dolist (p (list-all-packages) (remove-duplicates l)) (do-symbols (s p) (push s l)))))) #| --> 306 |#
21:20:58
pjb
(length (remove-duplicates (mapcan (lambda (s) (loop for k on (symbol-plist s) by 'cddr collect k)) (let (l) (dolist (p (list-all-packages) (remove-duplicates l)) (do-symbols (s p) (push s l))))))) #| --> 519 |#
21:30:05
pjb
aeth: (remove-duplicates (mapcan (lambda (s) (loop for (k nil) on (symbol-plist s) by #'cddr collect k)) (let (l) (dolist (p (list-all-packages) (remove-duplicates l)) (do-symbols (s p) (push s l)))))) #| --> (com.informatimago.common-lisp.data.constant:unit com.informatimago.common-lisp.data.constant::physical-constant swank/backend::implementation swank/backend::default optima.core:pattern-expand-function) |#