freenode/#lisp - IRC Chatlog
Search
13:31:19
VincentVega
Hi! With parachute, how do I run all known tests? `(mapcar #'parachute:test (parachute:package-tests *package*))' includes repeating tests, since some are suites and have children. Hmmm.....
14:26:20
beach
It is going great, thank you. I recently finished adapting the high-level intermediate code to the required structure according to may draft paper on call-site optimization, and I am now thinking about register allocation.
14:58:51
Josh_2
Does someone have an example of how I should structure my tests so they are seperate but I can use asdf to invoke them?
15:04:01
ex_nihilo
Yesterday I posted a question here about what seems to me to be a peculiar interaction between the Slime repl and user i/o with code run from a file. No responses yet; any takers? The question was posted over several messages, but I can repost if someone wants me to.
15:32:46
Josh_2
can I write my own metaclass that would mean all associated methods would only be evaluated conditionally, basically I want to check if I have freed some memory with an associated object before any methods (but one) are called?
15:33:08
Josh_2
theres no point calling something like (identity-keys <object>) if I had previously freed the pointer assocated with <object>
15:54:35
Bike
ex_nihilo: i think that's just slime? dunno. does the same thing happen when you run your lisp from the shell or whatnot?
15:55:17
Bike
Josh_2: I'm not sure I totally understand what you want, but method combinations can do completely arbitrary things.
15:57:08
ex_nihilo
Bike: no. If I run the code as a script from a shell it works as expected, and if I type the code directly into the Slime repl it runs as expected. It is only when I send the code from a Slime buffer to the repl, e.g., with slime-compile-and-run, that the behavior gets wonky.
15:57:33
puchacz
Bike: having read your article, the memory barrier linux kernel article and lispworks doc, I decided that the portable way between lisps of starting a thread with a new lambda or submitting a lambda to a queue to be executed by a long running thread is:
15:59:50
puchacz
(let ((arg1 "something") (arg2 3) (started-p)) (write-barrier) (submit-lambda (lambda () (loop :until started-p) (read-barrier) (use arg1 arg2))))
15:59:55
ex_nihilo
Bike: that's fair enough; what seemed most peculiar to me was that (let ((line (read-line))) (format t "~A~%" line)) works as expected when typed directly into a Slime repl, but not when sent to a Slime repl from a file.
16:01:09
puchacz
as barriers are not about timings, but partial ordering only, so in this "design pattern" when I see started-p as true, I know that other variable was already written.
16:01:34
puchacz
Bike - not really, how would I know that a lock is already available in a lexical variable?
16:02:02
puchacz
(let ((lock (bt:make-lock))) (submit-lambda () ... car I really assume lock is bound) )?
16:03:10
Bike
that's true. i suppose it would need to be a lock used by the actual long living thread
16:03:48
puchacz
yes. also lock protected section does not protect against reodering from before lock acquire to after lock release.
16:04:10
puchacz
and I want to make sure that arg1 arg2 are bound and objects they point to are fully written to memory
16:04:40
puchacz
and in the loop in my example you can add :do (bt:yield) to make the wait less busy
16:04:56
Josh_2
Bike: well I have a lot of various methods for various classes but almost all of them utilize a foreign pointer, I don't want the methods to execute their bodies if the pointer has previously been freed, but I don't want to go and write a :before / :around method for all of them
16:09:12
Bike
i would guess there's not a difference between binding it to T, and binding it to NIL and then setting it to T. i mean, you're kind of implicitly assuming that if the lambda doesn't read started-p = T, it will read started-p = NIL, no?
16:09:28
Bike
and my basic concern is that i suspect the queue just uses locks itself, in which case you don't need to do any synchronization
16:09:46
Bike
like, if submit = (with-lock (queue-lock) (enqueue lambda)) and the work thread does (with-lock (queue-lock) (dequeue))
16:12:22
puchacz
according to the memory-barriers.txt article, lock only ensures the right ordering between acquire and release, so using a lock on dequeuing only does not protect arg1 and arg2. by binding started-p to nil and then setting it to t I am (1) assuming that lisp is designed well enough that started-p will never be a garbage, but indeed it will have nil
16:13:38
puchacz
I did not figure out how to do it with fewer assumptions and not using globally available lock
16:13:41
Bike
if the enqueuing and dequeing work with locks it will be synchronized. When you do the enqueue, queue-lock is released, which gives you a write barrier, so arg1 and arg2 are written.
16:15:28
Bike
i mean if you're at this level, then for the lambda to observe started-p = nil, you are assuming some kind of write barrier anyway, right?
16:17:06
puchacz
lispworks docs is very careful about the objects that are pointed by these lexical variables. they are not guaranteed to be fully committed
16:18:18
puchacz
they say though that a variable will have any of the values it ever had, never garbage or something else
16:20:56
josrr
ex_nihilo: out of curiosity I tried it in SLY; this is what happened: https://www.rufina.link/video/vokoscreen-2021-02-20_10-12-49.mp4
16:23:24
puchacz
Bike, they say here: http://www.lispworks.com/documentation/lw71/LW/html/lw-142.htm "When one of these mutable atomic objects is modified, readers see either the old or new value (not something else), and it is guaranteed that the Lisp image is not corrupted by the modification even if multiple threads read or write the object simultaneously."
16:24:20
puchacz
so I know started-p will be nil or t as seen by the other thread, and I am assuming that other lisps are keeping this guarantee. if they don't, I don't know how I could really pass anything to a closure.
16:26:09
ex_nihilo
josrr: interesting; is SLY opening up a separate repl to take the input there? How did you compile and run; C-c C-k?
16:26:36
Bike
and i mean normally what you'd do is set up some kind of synchronization method with the long-running thread beforehand, as with the queue-lock i mentioned
16:29:00
Bike
i think i said this before, but if you're paying for lispworks, you should really be able to just ask them what the story is
16:29:22
puchacz
I also want my code to be portable, android access is not the only place I use threads.
16:30:19
josrr
ex_nihilo: no, SLY is using the minibuffer to get de input. I used C-c C-k, but C-c C-r does the same
16:30:32
Bike
i mean the only problem with the android access is that you're using this add-to-queue function that you don't control, like you can't look at it and see if it does use a lock or what
16:32:59
puchacz
so for other threading, do you agree that a portable lisp code should not rely on the lock being visible in the lexical variable?
16:35:03
ex_nihilo
josrr: for completeness, I saw the same issue when I used C-c C-r with Slime. It sounds like SLY has the issue sorted; I have never tried SLY, but I have wanted to for some time....
16:43:36
puchacz
if I was to use a lock, I would put (setf started-p t) (notify) inside a critical section, and do the same (bt:with-lock-held () loop :until started-p :do (wait)
16:44:04
puchacz
simply because the whole critical section and therefore a barrier can appear to happen after my closure is already running
16:44:31
puchacz
but I would still need to be sure about lock and condition-variable visibility in the closure
16:46:01
puchacz
so basically I would do java style pattern, but in java a lock and cv are the same object, and I could ensure its visibility if it was put in final variable
16:47:07
puchacz
so I believe as there are no final lexical variables in lisp, I am making fewer assumptions by using a binary started-p instead
16:47:22
Bike
i mean here's the fundamental problem: you're trying to ensure a closure variable is visible. to do that, you're synchronizing... through a closure variable.
16:51:26
Bike
"simply because the whole critical section and therefore a barrier can appear to happen after my closure is already running" no, because with what i'd be thinking you'd be holding the lock _while enqueing the thunk_, and there's no way a dequeue can get the thunk until after it's been enqueued
16:52:45
puchacz
my problem of critical section in the parent thread happening after dequeueing would only be an issue with my pattern, but with a lock/cv and a flag instead of a memory barrier
16:54:59
Bike
i've been reading atomic queue designs over the last few weeks, ones that mention not doing synchronization. this conversation has really put into perspective what an incredible pain in the ass that can be
16:55:33
puchacz
indeed. and I am really not doing anything low level, just trying to do normal application level Lisp correctly
16:56:52
Bike
the lispworks people are smart, i don't think they'd expose a queue if it would result in garbage appearing everywhere
16:57:45
puchacz
I am wondering now if I design a queue/dequeue mechanism (or say use lparallel), can I get "bla-bla is not a lock" on dequeuing? if the long running executor thread tries to dequeue in critical section, and encounters a lock that is not fully created? I will not get this error in lispworks, as they explicitely say that everything created before
16:57:46
puchacz
creating a thread will be fully committed, so it is enough if I create a lock first and then my long running executor. easy. but other implementations?
16:58:46
Bike
if make-thread is not synchronized, i'm not sure you could coordinate anything across threads ever
17:00:40
puchacz
I am only wondering what stassats (on #sbcl) meant by saying I need to protect everything that is shared with locks or barriers, and my scenarios are not about shared things.
17:09:59
puchacz
it does. I narrowed down my problem to android queue only. other cases are sorted by a queue/dequeue lock providing I create a long running executor after the lock in lispworks, because their manual says so, and it will work in sbcl because stassats says so, at least about similar scenarios, so I need more details.
17:13:01
puchacz
and as a side note, I heard that clojure people sorted a lot of these problems, but I never tried. and I would not want to use clojure for other reasons anyway, despite many similarities it is a different language than common lisp after all.
18:13:48
didi
What about https://paste.debian.net/hidden/9b945b27 for reading every object from a stream?
19:45:01
nij
Hello! Frequently I need to parse timestrings in several different format. I have written ad hoc solutions for them.. but I think there should be a more general way to parse.. any package that has achieved that?
19:47:02
Bike
there's local-time https://common-lisp.net/project/local-time/manual.html#Parsing-and-Formatting
19:47:15
nij
Or.. is there a general enough parser that helps me do this, with a smarter control sequence?
19:47:43
nij
Bike: local-time is actually my favorite time-related library.. iirc it doesn lemme do that. Just checked its doc. But let me double check again.
19:50:21
nij
I feel this is used many times.. but weird enough that there is no general library as I checked..
19:50:38
nij
so maybe people just use full fledged parsers.. which is something that i haven't learned
20:06:30
nij
Ah.. but the limit of natural language parsing is that there's not standard answer anyways.. so sometimes it needs to guess.
20:06:57
nij
E.g. (chronicity:parse "12/11/2020") returns November, but (chronicity:parse "12/15/2020") returns December.
20:07:47
VincentVega
how do I import/export a ```defun (setf smth)```? ```(import '(smth))``` isn't doing the trick.
20:11:08
_death
nij: I wrote this https://plaster.tymoon.eu/view/2307#2307 a long-time ago for some small-time date-time parsing
20:13:23
Bike
VincentVega: import/export is about symbols. if SMTH is imported you can refer to SMTH, (setf smth), whatever
20:15:37
_death
nij: for example (parse-using-digits-template "22/11/0000 33:44" "05/08/1962 05:25")
20:17:51
VincentVega
pfdietz: OK! By the way, is there a way to check for unused symbols which were imported with a defpackage?
20:24:01
_death
anyway.. nowadays I wouldn't make some of the choices I did back then :).. but that's what I use in some scrapers I wrote back then