libera/#commonlisp - IRC Chatlog
Search
12:13:06
jcowan
hayley: a classic hacque for Unicode char classes is to keep a sorted sequence of ranges as a vector. Contains is binary search, where the index mod 2 tells you if the char is in or out. Union and intersection are O(number of ranges), and complement is O(1)
12:14:19
hayley
Interesting. I'm not sure if O(log n) for contains would be okay, but I've set myself unreasonably high standards for scanning performance so far, honestly.
12:16:16
hayley
I already have such a sorted sequence to represent sets, but I don't want to give the branch predictor a hard time. I guess I can do a binary search of fixed length branch-free though.
12:18:55
hayley
...or perhaps my two level table idea, which would allow reusing the same #*0101... subvector for each group of characters.
12:20:08
hayley
From memory, ALPHANUMERICP had some 550 ranges or so, thus 10 iterations of binary search. Not sure if that is too many.
12:34:23
pdietz
The distribution of inputs is likely highly nonuniform in practice, which would speed things up.
13:20:23
pjb
hayley: perhaps not, when dealing with random unicode characters. it can be optimized for the iso-8859-1 case. But instead of using a binary search, perhaps one could have an indexed search. Are those ranges very non-uniformly distributed?
15:02:00
mzan
beach: about my assertion "my code is not 100% stream-oriented, because it can not process a file of 30Mb. It is ok for 16Kb"
15:02:20
mzan
it is probably wrong, because I noticed that there are ways for instructing series to optimize the code.
16:51:29
_73
In SBCL when I try to compile my file with a `defpackage` clause that has an `import-from #:alexandria ...` clause I get an error "The name ALEXANDRIA does not designate any package". If I go to the repl and `(ql:quickload "alexandria")` then recompile the file everything works fine. Should I ensure that my libraries are loaded by editing my `~/.sbclrc` file, or is their a different way to load my dependencies? I have made a `.asd`
16:51:29
_73
for my project that mentions my dependencies but I don't know how to tell SBCL about it.
16:52:58
Bike
_73: packages are orthogonal to dependency resolution. to tell asdf about dependencies you should load your system via ASDF.
17:11:43
pjb
_73: you should ensure that your dependencies are loaded, indeed, but better do it by writing an asd file rather than in you rc file.
17:11:46
beach
_73: In your ASDF system definition, put :depends-on (#:alexandria), then load your system with (asdf:load-system '#:<your-system-name>)
17:13:54
White_Flame
if you put a symlink to your project directory from ~/quicklisp/local-projects/, then you can (ql:quickload "your-system") as well
17:14:31
beach
Am I the only one who finds it strange when people who are given advice don't even acknowledge having seen it?
17:16:14
Bike
well, if they don't say anything else they could just be off to lunch for a bit or something
17:16:52
beach
White_Flame: Sure, if it takes more than a few minutes to get any reaction, but if the advice is given almost immediately, one would think they would stick around.
17:17:20
_73
I have been here the entire time struggling through docs: When I try `(asdf:load-system :my-system)` I get an error `Component :MY-SYSTEM not found`. I understand from the docs that the problem is I do not have my code in a standard location like `~/common-lisp`. I am trying to modify the `asdf:*central-registry*` but can't seem to get it right.
17:18:20
White_Flame
QL wraps ASDF's functionality and adds download and a standard path (~/quicklisp/local-projects/)
17:18:52
pjb
_73: modifying asdf:*central-registry* is one way. Another is to put it in i~/quicklisp/local-projects/ (symlinks don't work with all implementations) another is to set ql:*local-project-directories* (similar to asdf:*central-registry*)
17:19:24
pjb
_73: (push #P"/path/to/the/directory/where/you/put/your/asd/file/" #| <--note the final / |# asdf:*central-registry*)
17:19:52
pjb
_73: the asd file should have the same name as the system (it's possible to be different, but it's easier if it's the same, without the ".asd" of course.
17:26:48
Bike
_73: if your asd is in /home/seventythree/code/my-system/my-system.asd, you want (push "/home/seventythree/code/my-system/" asdf:*central-registry*)
17:29:28
Bike
_73: if you're using quicklisp, another possibility is to put a symlink to my-system/ in quicklisp/local-projects, and then make sure it's synced with (ql:register-local-projects)
17:51:04
Catie
In C, it's possible to spawn a POSIX thread with a "detached" attribute, such that when the thread's function exits its resources are freed by the operating system; there's no need to join the thread. Is anyone aware of a way to spawn a "detached" thread in Common Lisp?
17:52:40
jackdaniel
if you are interested only in the final effect, then you could join in the finalizer
17:53:25
Catie
Register a finalize on the thread, all right! I'll give that a go and see if it'll work
17:53:31
Bike
i don't think implementations have this functionality much. clasp doesn't have it, doesn't look like sbcl does
17:54:03
jackdaniel
I've written recently a short piece about a certain gotcha with finalizers: http://turtleware.eu/posts/Selective-waste-collection.html
17:55:56
Catie
Bike: the issue I run into is that you can't join a thread from within itself, and threads execute in their own dynamic environment. Originally I was hoping to make this work by signalling a condition, but none of the handlers I establish in the thread-spawning context are in effect for the thread
17:56:23
Bike
i meant within the function itself. like, (make-thread (lambda () (unwind-protect (do-stuff) (clean-up))))
17:59:37
Catie
The system-reserved resource in this case is the thread itself. There's a limit along the lines of _POSIX_THREAD_MAX after which you get no more threads, and while it's incredibly unlikely that I'll run into that limit, ignoring it entirely just feels wrong
18:01:59
Catie
Yeah, it's a real weird one. The other idea I've got in my head is to have a dedicated thread-joining thread that responds to an asynchronous Unix-type signal and joins the thread, which would let my main loop continue uninterrupted
18:03:09
Catie
jackdaniel: Wouldn't establishing a finalizer on a thread object that calls join-thread on the object itself cause a cycle?
18:04:00
Bike
as he describes in the post, on some implementations (but not all, so there's a complication) the finalizer function receives the object as an argument
18:04:14
jackdaniel
some implementations (most notably _not_ sbcl and trivial-garbage) pass the object as an argument to the finalizer -this allows rehabilitation or joining the thread
18:04:22
Catie
Which is incredibly convenient! I just don't want to have to rely on a specific implementation
18:05:07
jackdaniel
well, trivial garbage is the smallest common denominator, so it will work everywhere
18:05:37
jackdaniel
the trick is to register a finalizer that closes over the resource for the single object that wraps over it (i.e (list resource))
18:06:33
jackdaniel
(let ((thread my-thread) (wrapper (list my-thread))) (set-finalizer wrapper (lambda () (join-thread thread))))
18:07:23
jackdaniel
that said, you will probably need to take care to keep wrapper alive until the thread is finished, I'm not sure how blocking finalizers behave in implementations
18:08:20
Catie
Guest74: Under FreeBSD it's aliased to __ULONG_MAX, which is probably 2^64 (but I don't feel like looking through sys/types.h)
18:09:02
Catie
jackdaniel: If the wrapper object is referencing an alive thread, could it still be garbage collected?
18:10:26
Guest74
(os:shell "cat /proc/sys/kernel/threads-max") works. 62424 on my machine. Probably not a problem for my purposes.
18:10:31
Catie
Guest74: I think PTHREAD_THREADS_MAX is compile-time, but there may be a separate runtime tunable that's less than or equal to it
18:10:42
jackdaniel
passing the wrapper to the thread's environment should suffice for that purpose I guess (because when the thread finishes then it will be referenced no more)
18:11:36
random-nick
wouldn't the implementation of detached threads just be calling the pthread function which detaches the thread on the pthread? the thread would have to unwind all the way (and run the handlers) in order to exit anyway, right?
18:17:03
Guest74
interesting max threads is determined by real memory, max threads per process by virtual memory on linux.
18:17:20
Bike
we can just have a separate thread object that keeps whatever info we want that we garbage collect normally
18:18:16
jackdaniel
yes, I didn't check on other implementaitons, but I have never actually thought that implementation might leave some linger resources when not joined
18:19:35
Catie
The SBCL thread struct has a result slot, which indicates to me that the thread is joined by the runtime somehow and the result is collected, and that calling the sb-thread join function just returns that value
18:19:55
jackdaniel
otoh I think that I remember that sbcl doesn't care for closing streams taht are garbage collected (and such leak may end with no more descriptors)
18:22:37
jackdaniel
you could have copied the file descriptor by using some internals, closing it on gc may give strange results (but well, you've copied it by using some internals :)
18:27:26
Catie
Okay, so working backwards a bit, SBCL has a global variable *joinable-threads* and a function join-pthread-joinables. When I run a thread and let it exit, it resides in *joinable-threads*. When I trigger a garbage collection, the thread is no longer in *joinable-threads*. So maybe SBCL does the convenient thing?
18:35:44
Catie
Ah, sure enough! I missed it because pauseless-threadstart isn't in my *features* anymore, so I skipped right over it
18:36:35
frgo
Oh-kay - I just booked my flights to Porto to ELS2022. Hope to see many of you there ...
18:40:55
frgo
In my day job I work for the German Red Cross. I hope, really hope, that Omicron is covered with the current vaccines. Having all red flags on ICU bed availability in a 100 km radius around Stuttgart is no fun.
18:44:45
pdietz
From what I hear, if you have the mRNA vaccine doses and the booster they're pretty effective against omicron.
18:45:53
jackdaniel
even if you have superman immune system and there is no kryptonite around, when borders are closed then they are closed
18:46:04
pdietz
It's looking nasty in South Africa. In particular it looks much worse than previous variants for children under 2.
18:46:28
frgo
That's what we see - so far (it's too early to tell really). And that's why we don't mind to booster a few weeks before the 6 months period.
19:06:46
random-nick
am I missing something or does CFFI not have a builtin type for fixed-size arrays?
19:27:53
attila_lendvai
ACTION is annoyed by the covid discussion, especially when people echo what they say on TV, as opposed to first-hand experience/info
19:46:02
_73
from the SBCL repl, what is the way I can unqualify a library function? For example I would like to be able to use the symbol `alexandria:compose` as just `compose`. I can do this fine in a package using defpackages `:import-from`.
19:55:34
Alfr
_73, or to make your repl habits less implementation dependent, because implementations make symbols accessible in cl-user which are not in cl.
19:59:29
Catie
Guest74: My understanding is that cffi:foreign-array-to-lisp copies elements from the pointer you hand it
20:02:05
Catie
That would be really nice, I agree. I don't know of anything in CFFI that can do that, but I have no idea about any other FFI packages
20:02:07
Guest74
though WAAF looks interesting. copying isn't so bad, it's basically doublebuffering. But direct access would be nice.
20:12:23
yitzi
Well, with mem-aref you can read the individual elements directly. https://common-lisp.net/project/cffi/manual/cffi-manual.html#mem_002dref
20:12:45
Guest74
I may be wrong, but I assume writing individual bytes to 'foreign memory' would be slower than just copying.
20:23:13
Bike
that's probably not possible, because lisp arrays usually have header data that wouldn't be in foreign controlled memory.
20:30:28
random-nick
Guest74: mem-aref seems to have a compiler macro which eventually directly calls SBCL's function for accessing memory when the type is constant
20:34:34
_death
the limitations are not inherent, the interface was just sufficient to what I wanted at the time so I didn't bother removing them.. but if the idea makes sense, then one could imagine more powerful mmapped array facility.. say for IPC
20:43:33
random-nick
hmm, why does WITH-ACCESSORS not have the simple variant of slot-entry like WITH-SLOTS does?
20:46:28
random-nick
was the accessor naming style where you don't use a prefix not popular when the standard was written?
20:50:57
Catie
What's the vibe on generics nowadays, have they come around? Or are they still a bit frowned upon?
20:54:45
pdietz
The short name style is nicer with package local nicknames, as you can avoid name collision and use the PLN as the prefix.
20:59:04
Shinmera
I don't know who ever frowned upon generic functions, but whoever it was they were probably making a bad judgement call.
20:59:05
Catie
Generic functions, in particular slot accessors without the class name prefixed to them
21:03:43
hayley
Everyone who first learns CLOS apparently thinks they really need a reader macro for SLOT-VALUE. Then, hopefully they learn CLOS.
21:05:30
jackdaniel
novices don't use clos because they don't understand it, experts don't use clos because they do - or something like that
21:06:04
jackdaniel
(in truth most cl programmers use generic functions and clos protocols some way or another)
21:06:51
yitzi
How about: Novices see objects and slots, experts see generic functions and accessors?
21:16:14
rotateq
pve: I'll take that risk. :) first I shall master the state of being a sorcerers apprentice to the next step
21:17:20
jackdaniel
slot-definition-location is advertised by amop authors as a way to optimize important bits implemented with standard classes (to avoid unnecessary function calls)
21:19:06
jackdaniel
they say that to hack something interesting you either need experience, talent or patience. I'm in the last group. thanks though!
21:19:32
rotateq
pve: there's a nice quote starting "if programming languages were like religions, LISP would be like Buddhism". I think that is one of the timeless points which attract me so much.
21:29:15
pdietz
Generic functions for slot access can be nice because you can combine other methods with them. :around, :before, :after
21:33:11
rotateq
in the beginning of the week i realized a bit more how/when the other method-combinations could be effectively used or that it can be useful the when having SETFs also generic the dispatch can go to the new-value too if needed
21:44:56
NotThatRPG
Allegro offers `EXCL:RECORD-SOURCE-FILE` so that one can record where new kinds of things are defined. E.g., I am making FIVEAM do (excl:record-source-file ',name :fiveam-test) inside fiveam:def-test. Question: do CCL and SBCL have any equivalent of this?
21:53:39
Shinmera
Not sure, but that reminds me it would be great if someone added more backends to https://github.com/shinmera/defititions
22:14:18
NotThatRPG
Shinmera: If this supported Allegro, too, I'd be more excited -- Allegro, SBCL, and CCL are my target implementations....
22:26:50
NotThatRPG
Shinmera: I'm already contributing quite enough to CL open source! I'm tapped out...
22:30:00
NotThatRPG
I would have thought there would be an interface to SBCL's source recording but... apparently not?
22:32:34
Shinmera
That reminds me that I initially created https://portability.cl in the faint hopes it would drive people to try and improve the situation on either library or implementation sides, but as far as I can tell it actually changed nothing at all.
22:33:33
Shinmera
So far my impression writing several of these libraries is that unless you do the painful work of aggregating the info for 90% of the implementations out there that you don't even use, nobody'll bother contributing a backend for even one more.
22:35:56
Shinmera
Yeah Clasp has been an outlier. You folks have been doing a tremendous job increasing support coverage.
22:36:24
NotThatRPG
Shinmera: If you look into the crawling horror that is the ASDF source, you will see what it takes... I don't know that anyone besides Fare has the energy to fight through testing on all the various implementations out there.
22:36:59
Shinmera
NotThatRPG: I have, and as I said I've also written many of these libraries myself.
22:38:29
Bike
a lot of these aren't actually that bad assuming the implementation has support. trivial-extensible-sequences support on sbcl consists of #+sbcl #:sb-sequence for example
22:38:54
Bike
but in that case there's a well defined protocol people pretty much go with, which is not the case with e.g. all the compile-file extensions asdf tries to herd
22:39:24
NotThatRPG
Krystof: Does that let me set the source location myself? I will look again -- I was apropos-ing with source-file, so wouldn't have found any source-location symbols.
22:43:34
Krystof
the pattern we use is something like (defmacro deffoo (...) `(%deffoo ... (source-location))) (defun %deffoo (... source-location) <store source-location wherever you like> ...)
22:43:50
Krystof
then whatever consumes your source locations needs to know where you've stored them; I don't think we have a protocol for that
22:45:16
NotThatRPG
Ah. So one *calls* source location, and it gives you back the source location to do with what you will.
22:54:49
NotThatRPG_
I didn't mean to imply that it's more awful than it needed to be. But awful it did need to be.
22:56:29
Fare
The awfulest parts are those that deal with portability across 17 implementations, in UIOP. Especially the portable run-program facility.
22:58:43
Fare
pathnames are a mindfuck, though. There is a wide gap between that the standard specifies and what is common practice in today's world.
22:59:04
Fare
with plenty of undefined behavior that every implementation does in subtly different ways.
23:00:41
Shinmera
I have often thought about starting a thing for a secondary path mechanism that's more tailored towards modern systems, and adding support for that to implementations, but doing so is too much effort even for me.
23:02:25
Fare
but then you have to hook it up into LOAD and all other primitives that deal with paths.
23:02:33
Shinmera
did it? My idea would have been to allow using cl:open/etc with this new path type.