freenode/#lisp - IRC Chatlog
Search
13:31:19
VincentVega
Hi! With parachute, how do I run all known tests? `(mapcar #'parachute:test (parachute:package-tests *package*))' includes repeating tests, since some are suites and have children. Hmmm.....
14:26:20
beach
It is going great, thank you. I recently finished adapting the high-level intermediate code to the required structure according to may draft paper on call-site optimization, and I am now thinking about register allocation.
14:58:51
Josh_2
Does someone have an example of how I should structure my tests so they are seperate but I can use asdf to invoke them?
15:04:01
ex_nihilo
Yesterday I posted a question here about what seems to me to be a peculiar interaction between the Slime repl and user i/o with code run from a file. No responses yet; any takers? The question was posted over several messages, but I can repost if someone wants me to.
15:32:46
Josh_2
can I write my own metaclass that would mean all associated methods would only be evaluated conditionally, basically I want to check if I have freed some memory with an associated object before any methods (but one) are called?
15:33:08
Josh_2
theres no point calling something like (identity-keys <object>) if I had previously freed the pointer assocated with <object>
15:54:35
Bike
ex_nihilo: i think that's just slime? dunno. does the same thing happen when you run your lisp from the shell or whatnot?
15:55:17
Bike
Josh_2: I'm not sure I totally understand what you want, but method combinations can do completely arbitrary things.
15:57:08
ex_nihilo
Bike: no. If I run the code as a script from a shell it works as expected, and if I type the code directly into the Slime repl it runs as expected. It is only when I send the code from a Slime buffer to the repl, e.g., with slime-compile-and-run, that the behavior gets wonky.
15:57:33
puchacz
Bike: having read your article, the memory barrier linux kernel article and lispworks doc, I decided that the portable way between lisps of starting a thread with a new lambda or submitting a lambda to a queue to be executed by a long running thread is:
15:59:50
puchacz
(let ((arg1 "something") (arg2 3) (started-p)) (write-barrier) (submit-lambda (lambda () (loop :until started-p) (read-barrier) (use arg1 arg2))))
15:59:55
ex_nihilo
Bike: that's fair enough; what seemed most peculiar to me was that (let ((line (read-line))) (format t "~A~%" line)) works as expected when typed directly into a Slime repl, but not when sent to a Slime repl from a file.
16:01:09
puchacz
as barriers are not about timings, but partial ordering only, so in this "design pattern" when I see started-p as true, I know that other variable was already written.
16:01:34
puchacz
Bike - not really, how would I know that a lock is already available in a lexical variable?
16:02:02
puchacz
(let ((lock (bt:make-lock))) (submit-lambda () ... car I really assume lock is bound) )?
16:03:10
Bike
that's true. i suppose it would need to be a lock used by the actual long living thread
16:03:48
puchacz
yes. also lock protected section does not protect against reodering from before lock acquire to after lock release.
16:04:10
puchacz
and I want to make sure that arg1 arg2 are bound and objects they point to are fully written to memory
16:04:40
puchacz
and in the loop in my example you can add :do (bt:yield) to make the wait less busy
16:04:56
Josh_2
Bike: well I have a lot of various methods for various classes but almost all of them utilize a foreign pointer, I don't want the methods to execute their bodies if the pointer has previously been freed, but I don't want to go and write a :before / :around method for all of them
16:09:12
Bike
i would guess there's not a difference between binding it to T, and binding it to NIL and then setting it to T. i mean, you're kind of implicitly assuming that if the lambda doesn't read started-p = T, it will read started-p = NIL, no?
16:09:28
Bike
and my basic concern is that i suspect the queue just uses locks itself, in which case you don't need to do any synchronization
16:09:46
Bike
like, if submit = (with-lock (queue-lock) (enqueue lambda)) and the work thread does (with-lock (queue-lock) (dequeue))
16:12:22
puchacz
according to the memory-barriers.txt article, lock only ensures the right ordering between acquire and release, so using a lock on dequeuing only does not protect arg1 and arg2. by binding started-p to nil and then setting it to t I am (1) assuming that lisp is designed well enough that started-p will never be a garbage, but indeed it will have nil
16:13:38
puchacz
I did not figure out how to do it with fewer assumptions and not using globally available lock
16:13:41
Bike
if the enqueuing and dequeing work with locks it will be synchronized. When you do the enqueue, queue-lock is released, which gives you a write barrier, so arg1 and arg2 are written.
16:15:28
Bike
i mean if you're at this level, then for the lambda to observe started-p = nil, you are assuming some kind of write barrier anyway, right?
16:17:06
puchacz
lispworks docs is very careful about the objects that are pointed by these lexical variables. they are not guaranteed to be fully committed
16:18:18
puchacz
they say though that a variable will have any of the values it ever had, never garbage or something else
16:20:56
josrr
ex_nihilo: out of curiosity I tried it in SLY; this is what happened: https://www.rufina.link/video/vokoscreen-2021-02-20_10-12-49.mp4
16:23:24
puchacz
Bike, they say here: http://www.lispworks.com/documentation/lw71/LW/html/lw-142.htm "When one of these mutable atomic objects is modified, readers see either the old or new value (not something else), and it is guaranteed that the Lisp image is not corrupted by the modification even if multiple threads read or write the object simultaneously."
16:24:20
puchacz
so I know started-p will be nil or t as seen by the other thread, and I am assuming that other lisps are keeping this guarantee. if they don't, I don't know how I could really pass anything to a closure.
16:26:09
ex_nihilo
josrr: interesting; is SLY opening up a separate repl to take the input there? How did you compile and run; C-c C-k?
16:26:36
Bike
and i mean normally what you'd do is set up some kind of synchronization method with the long-running thread beforehand, as with the queue-lock i mentioned
16:29:00
Bike
i think i said this before, but if you're paying for lispworks, you should really be able to just ask them what the story is
16:29:22
puchacz
I also want my code to be portable, android access is not the only place I use threads.
16:30:19
josrr
ex_nihilo: no, SLY is using the minibuffer to get de input. I used C-c C-k, but C-c C-r does the same
16:30:32
Bike
i mean the only problem with the android access is that you're using this add-to-queue function that you don't control, like you can't look at it and see if it does use a lock or what
16:32:59
puchacz
so for other threading, do you agree that a portable lisp code should not rely on the lock being visible in the lexical variable?
16:35:03
ex_nihilo
josrr: for completeness, I saw the same issue when I used C-c C-r with Slime. It sounds like SLY has the issue sorted; I have never tried SLY, but I have wanted to for some time....
16:43:36
puchacz
if I was to use a lock, I would put (setf started-p t) (notify) inside a critical section, and do the same (bt:with-lock-held () loop :until started-p :do (wait)
16:44:04
puchacz
simply because the whole critical section and therefore a barrier can appear to happen after my closure is already running
16:44:31
puchacz
but I would still need to be sure about lock and condition-variable visibility in the closure
16:46:01
puchacz
so basically I would do java style pattern, but in java a lock and cv are the same object, and I could ensure its visibility if it was put in final variable
16:47:07
puchacz
so I believe as there are no final lexical variables in lisp, I am making fewer assumptions by using a binary started-p instead
16:47:22
Bike
i mean here's the fundamental problem: you're trying to ensure a closure variable is visible. to do that, you're synchronizing... through a closure variable.
16:51:26
Bike
"simply because the whole critical section and therefore a barrier can appear to happen after my closure is already running" no, because with what i'd be thinking you'd be holding the lock _while enqueing the thunk_, and there's no way a dequeue can get the thunk until after it's been enqueued
16:52:45
puchacz
my problem of critical section in the parent thread happening after dequeueing would only be an issue with my pattern, but with a lock/cv and a flag instead of a memory barrier
16:54:59
Bike
i've been reading atomic queue designs over the last few weeks, ones that mention not doing synchronization. this conversation has really put into perspective what an incredible pain in the ass that can be
16:55:33
puchacz
indeed. and I am really not doing anything low level, just trying to do normal application level Lisp correctly
16:56:52
Bike
the lispworks people are smart, i don't think they'd expose a queue if it would result in garbage appearing everywhere
16:57:45
puchacz
I am wondering now if I design a queue/dequeue mechanism (or say use lparallel), can I get "bla-bla is not a lock" on dequeuing? if the long running executor thread tries to dequeue in critical section, and encounters a lock that is not fully created? I will not get this error in lispworks, as they explicitely say that everything created before
16:57:46
puchacz
creating a thread will be fully committed, so it is enough if I create a lock first and then my long running executor. easy. but other implementations?
16:58:46
Bike
if make-thread is not synchronized, i'm not sure you could coordinate anything across threads ever
17:00:40
puchacz
I am only wondering what stassats (on #sbcl) meant by saying I need to protect everything that is shared with locks or barriers, and my scenarios are not about shared things.
17:09:59
puchacz
it does. I narrowed down my problem to android queue only. other cases are sorted by a queue/dequeue lock providing I create a long running executor after the lock in lispworks, because their manual says so, and it will work in sbcl because stassats says so, at least about similar scenarios, so I need more details.
17:13:01
puchacz
and as a side note, I heard that clojure people sorted a lot of these problems, but I never tried. and I would not want to use clojure for other reasons anyway, despite many similarities it is a different language than common lisp after all.
18:13:48
didi
What about https://paste.debian.net/hidden/9b945b27 for reading every object from a stream?
19:45:01
nij
Hello! Frequently I need to parse timestrings in several different format. I have written ad hoc solutions for them.. but I think there should be a more general way to parse.. any package that has achieved that?
19:47:02
Bike
there's local-time https://common-lisp.net/project/local-time/manual.html#Parsing-and-Formatting
19:47:15
nij
Or.. is there a general enough parser that helps me do this, with a smarter control sequence?
19:47:43
nij
Bike: local-time is actually my favorite time-related library.. iirc it doesn lemme do that. Just checked its doc. But let me double check again.
19:50:21
nij
I feel this is used many times.. but weird enough that there is no general library as I checked..
19:50:38
nij
so maybe people just use full fledged parsers.. which is something that i haven't learned
20:06:30
nij
Ah.. but the limit of natural language parsing is that there's not standard answer anyways.. so sometimes it needs to guess.
20:06:57
nij
E.g. (chronicity:parse "12/11/2020") returns November, but (chronicity:parse "12/15/2020") returns December.
20:07:47
VincentVega
how do I import/export a ```defun (setf smth)```? ```(import '(smth))``` isn't doing the trick.
20:11:08
_death
nij: I wrote this https://plaster.tymoon.eu/view/2307#2307 a long-time ago for some small-time date-time parsing
20:13:23
Bike
VincentVega: import/export is about symbols. if SMTH is imported you can refer to SMTH, (setf smth), whatever
20:15:37
_death
nij: for example (parse-using-digits-template "22/11/0000 33:44" "05/08/1962 05:25")
20:17:51
VincentVega
pfdietz: OK! By the way, is there a way to check for unused symbols which were imported with a defpackage?
20:24:01
_death
anyway.. nowadays I wouldn't make some of the choices I did back then :).. but that's what I use in some scrapers I wrote back then
20:28:15
nij
Ah.. a typo sorry. Fixed: (collect "aaaa-bb-cc-b" "2020-11-20-@") => (:a "2020" :b "11@" :c
20:33:34
_death
you can use Lisp syntax for structure, instead of a string template.. like (collect '((a 2) (b 2) (c 2) (a 2) (b 2)) sequence)
20:42:29
_death
I tend not to use regular expressions.. but I guess for a task like scraping they may be ok
20:55:03
_death
nij: for format, there was a (setf format) thingy.. so you could (setf (format nil "~D-~D-~D" year month day) "1999-12-31") and it would assign the numbers to year/month/day
21:15:16
mfiano
LispWorks is nice, but it's very easy to write non-portable code with the array of impleemntation-specific niceties it has, and I can't justify the license cost over free and extremely performant SBCL.
21:17:26
nij
_death: Oh wow indeed, I thought rx is just for pattern matching.. but it's actually almost the same problem. There's a nice showcase here: http://francismurillo.github.io/2017-03-30-Exploring-Emacs-rx-Macro/
21:18:32
louis771
i'm just a few weeks into lisp and oscillate somewhat between CL and Racket. I found CL the more mature, practial Lisp however Racket provides a unified batteries-included approach incl. an IDE and a central package server.
21:19:17
louis771
What I don't like about Racket is the obsession of it being a language to program programming languages. It's just what I'm not so much interested in. I want to do more practical stuff (web app, etc.)
21:20:15
nij
louis771: A reason I heard of why Lisp is great is that it's easier for you to implement your own language for your specific problem.
21:20:21
mfiano
Well that's hard to answer. Racket being derived from Scheme has a different take on live-recompilation/incremental modifications.
21:22:36
louis771
some nice macros to make life easier is fine for me, but inventing a new DSL so nobody else but me will be able to understand my code is outside my scope :)
21:23:35
louis771
from a "getting things done" perspective, or say a more business point of view, you guys think that CL is the better alternative?
21:25:39
louis771
mfiano: I'm sorry, true. But when I do a search on quicklisp this is where I end up
21:28:17
louis771
nij: oh I have some nice conversations in the racket community in the last few days and did some coding, but somehow I "feel" that it's not the direction I wanna go. But I'm unable to express why at the moment.
21:29:32
louis771
Cthulhux: yes, Racket has a nice package management but try to get a packet in the package server.
21:29:51
louis771
You have to learn Scribble first, which is the documentation language written in Racket
21:30:34
louis771
Which was exactly the moment where I realized that maybe the price of "build a new language for every use case" is too high
21:31:29
Cthulhux
racket comes with half a dozen languages already - i'll need to find out how they can talk to each other though
21:32:00
Cthulhux
e.g. i haven't found out how to use lang/<anything else> features from lang/webserver
21:33:07
louis771
the racket devs are not interested in making Racket a day-to-day language for practical/business use cases. They are into language research, so everything in the community evolves around that.
21:33:48
VincentVega
louis771: I don't know how much of it is still relevant (e.g. racket 8.0 just came out and it's supposed to be quite faster), but here's the comparison https://gist.github.com/vindarel/c1ef5e043773921e3b11d8f4fe1ca7ac
21:33:53
Cthulhux
it's not. because there are no CL devs. the standard is there, everything else is a package
21:34:48
louis771
"No introspection compared to one with SLIME." - yes that was really disappointing
21:35:43
louis771
"most of Racket libraries are half-baked: unfinished, failing to build, and lacking documentation" - I must agree here too
21:37:15
louis771
VincentVega: a very nice article, thanks! I have to agree to most of it from what I experience the last few days of studying Racket
21:39:11
VincentVega
louis771: yeah, no problem : ) Note that Emacs is no small thing to consider either, especially with the aggressive-indent + paredit (lispyville for evil) combination, which I have grown to think of as indispensible now.
21:52:48
markasoftware
i've got a small MOP question. At http://mop.lisp.se/www.alu.org/mop/concepts.html#defmethod it says that the (make-instance) call for the method metaobject can receive a :declarations initarg
21:53:13
markasoftware
but here http://www.lispworks.com/documentation/lw71/MOP/mop/dictionary.html#Initialization there is no :declarations initarg mentioned for method metaobjects
21:56:32
markasoftware
and the SBCL definition of `standard-method` seems not to take a :declarations initarg
22:17:36
warweasle
Yay! I'm pulling data from the stock market! Ah, my ridiculously circuitous plan is one quarter complete!
22:20:06
warweasle
Honestly I'm a little more concerned with using the training data to create a stock predictor.
22:21:04
warweasle
Given the size of the data set and the number of variables... I should be done just before the heat death of the universe...if I had a quantum computer.
22:29:43
markasoftware
I'm also a confused about the example `defmethod` expansion given here http://mop.lisp.se/www.alu.org/mop/concepts.html#defmethod will work if the generic function is not defined yet
22:30:11
markasoftware
it seems to me like the (ensure-generic-function) call will initialize it with a nil lambda-list, so adding a method with a non-nil lambda list right afterwards will trigger an error
22:30:34
markasoftware
because the empty lambda list is not congruent with the lambda list of the new function
22:34:32
Bike
the MOP book has a lot of little issues like this. you might be better off looking at what implementations actually do.