libera/#commonlisp - IRC Chatlog
Search
10:01:16
flip214
depending on your usecase (needs IO, or not) it might be easier to just limit number of thread == number of cores. then every thread can process a message without any wait time => minimum latency, minimum heap usage.
10:01:45
flip214
works for me with PDF generation - there's no IO involved, just CPU, so there's no reason to have more threads than can run at the same time.
10:03:20
moon-child
I think a server must of necessity do a lot of i/o. But I think the usual approach there is to just have one i/o thread which receives connections and pushes them onto a work queue; and to do writing asynchronously
10:04:11
hayley
I can't seem to understand anything that isn't "straight line" networking code with threads, such as using some callback async library or polling or whatnot. Then writing it is daunting.
10:06:13
flip214
hayley: SB-EXT:CALL-WITH-TIMING can give you a :BYTES-CONSED result, so your threads can (over time) estimate their memory use themselves ;)
10:07:03
flip214
moon-child: I meant disk-IO that would block worker threads. as soon as an HTTP request has arrived, all the work until pushing the result might be possible without any IO.
10:07:38
moon-child
ah, yes. Usual approach for this is coroutines, but I don't think we have any cl implementations that support them
10:08:22
moon-child
(of course, come closos, plain threads will be cheap enough it won't matter. But who knows then that will be)
10:08:45
hayley
Another difference is that I have connections which are expected to be persistent, whereas I can't remember if Hunchentoot supports keep-alive.
10:10:13
flip214
hayley: you can tell HT to ignore a TCP socket - and pass that on to an extra thread.
10:11:27
hayley
moon-child: FWIW my secret plan is to prototype "green threads" in a fork of SICL, which would then be somewhat reusable for a thread implementation in CLOSOS.
10:13:28
flip214
hayley: green threads - with all the features of OS threads? like scheduling priorities, blocking behaviour for OS calls, timeouts, ...??
10:14:03
flip214
I've worked on such a library in C - but it never got up to par on features with kernel threads.
10:16:21
hayley
Well, Erlang has priorities, and in the case of CLOSOS, blocking OS calls could be handled with the usual devices (mostly semaphores I suppose).
10:18:36
hayley
On a Unix system, I think the Erlang and Haskell people both either attempt to use asynchronous operations or a thread pool for synchronous operations, which still use something like mailboxes to unblock the appropriate thread and provide the result.
10:22:32
hayley
Apparently the fastest servers these days use other async stuff, which is backed by some polling technique (epoll, etc, maybe io_uring if you're fancy). Without some intermediate stuff they don't work too well with threads apparently.
10:24:43
flip214
yeah. But if your actual workload (PDFs, for me) take 30-60msec to generate, and I only have a handful of cores (and a handful of threads), the server implementation doesn't actually matter.
10:25:10
flip214
of course, the "hello world ~a" examples need to keep the server as small as possible for their benchmarks...
12:23:23
yitzi
beach: Thanks for the paper reference regarding CL and SICL debugging. Very informative.
12:24:42
yitzi
I noticed that there wasn't a section on CMUCL. From what the user manual and repo says they support stepping and breakpoints, although the stepper was broken until recently.
12:27:14
yitzi
Not sure how well maintained it is though. I can't currently test it in my environment since CMUCL can't load ironclad on my system.
12:28:25
jackdaniel
flip214: kernel threads become troublesome when there are a lot of them (the context switch of a kernel thread is more expensive)
12:29:41
yitzi
beach: If you are curious you can run the Jupyter debugger stuff here: https://mybinder.org/v2/gh/yitzchak/common-lisp-jupyter/add-debugging?urlpath=lab
12:29:42
jackdaniel
also green threads (when pooled in a single native thread) have assured a certain amount of atomicity
12:30:21
jackdaniel
a side-benefit is that if you bother to implement green threads you will probably have delimited continuations for free
12:30:41
jackdaniel
(and with that come numerous interesting programming techniques - i.e generators)
12:31:43
flip214
and with _lots_ of kernel threads you run in all the (already-solved, on the kernel-side) scheduling things - O(1), priorities, fairness, ...
12:37:18
hayley
jackdaniel: I thought of green threads in the terms of undelimited one shot continuations.
12:40:14
yitzi
beach: In addition to debugging, stepping, source and variable inspection there is also symbol completion and help/inspect. Along with a widgets and pile of other stuff.
12:41:32
yitzi
The debugging is still a work in progress. The other stuff is pretty well tested and used by drmeister's group. There is also a maxima-jupyter kernel available based on the same code.
12:44:40
yitzi
beach: Yep. I've already started tweaking some of the debug messaging based on some your ideas. Thanks again!
12:46:15
hayley
I read a delimited continuation was a continuation which was "delimited" to some frame with shift/reset.
12:48:04
jackdaniel
in non-scheme terms, a delimited continuation is a computing context that may be resumed, and after resuming it may yield control again (and be resumed again)
12:48:30
Bike
"one shot" means you can only use it once, like because the stack isn't copied so it will no longer be in place if you try to continue there later. so register saving works.
12:49:56
hayley
I /guess/ we are delimited by THREAD-YIELD (and preemptive scheduling) but it doesn't fit how they are presented in Scheme land in my opinion.
12:52:02
Bike
i don't think threads are really "delimited continuations". the way the continuations given by shift work, they add to the current call stack, so they return like normal functions (to muddle several levels of description)
12:52:05
jackdaniel
so if the stack is not copied and you allow yielding, then how does it work? all threads share the same stack?
12:53:43
hayley
Each thread has a separate stack, and yielding saves state, returns to some scheduler, then loads the new state.
12:53:53
jackdaniel
I didn't say that delimited continuations are threads, I'm saying that cooperative threads may be implemented with delimited continuations
12:55:05
hayley
But my observation was the only green thread implementation with continuations uses undelimited continuations once, as I couldn't spot the delimiting.
12:55:37
hayley
Somehow delimited continuations are harder for me to understand than undelimited even though the former is supposedly much better to use.
12:56:22
hayley
The former does some stack knotting, whereas the latter is a funny first class GOTO.
12:56:22
Bike
they have some interesting advantages over undelimited, but simplicity is not one of them
12:58:00
jackdaniel
I will read on that later, but some loose searching seems to indicate, that one-shot continuations are a certain class of delimited continuations
13:00:27
Bike
and most hagiographies of delimited continuations emphasize their composability which kind of goes with reusability
13:00:28
jackdaniel
continuations that can be called multiple times are full continuations, one-shot may be resumed only once and delimited capture the entire /remaining/ continuation; that's what I've grasped from this quick search
13:01:12
hayley
I think get/setcontext in POSIX-2001 manipulate one shot undelimited continuations, as they are just all the registers as described previously.
13:03:23
hayley
Before I was going to say setjmp/longjmp but those are mere escape continuations as I understand how you are supposed to use them.
13:04:47
Bike
yeah longjmp has the "UB if the function that used setjmp has returned" bit, so they're only an escape
13:42:03
rain3
a 2 hours long video from Atlanta Functional Programming - Common Lisp Study Group youtube channel can be condensed in 1-5 pages of good old text tutorial which in turn can be studied in 10-30 minutes instead of 2-6 hours of listening and watching and pausing and scrolling
13:42:51
rain3
interesting stuff there https://www.youtube.com/watch?v=tcmY7zstig4 but it takes too long to watch a 2h long video for only 10 minutes of essential info
13:45:48
jackdaniel
so while the former purpose is teaching, the latter purpose is socializing (in my understanding)
13:46:49
rain3
socializing around good content which can be summarized as a high quality tutorial , either before or after the meeting
13:48:38
jackdaniel
ah, that's a shame; perhaps you could write a high-quality tutorial or a paper on another topic related to lisp; I'd gladly read it
13:49:08
lotuseater
I skipped through it, is ":filter :something" a method-combination? wasn't aware they can be done that way, but I think because I didn't define ones myself yet
13:50:02
ldb
lotuseater: that's probably optional argument lists, I don't think you need method-combination for that
13:50:49
rain3
jackdaniel: on the other hand your CLIM videos are from beginning to end golden and enjoyable, good and efficient for teaching others
13:51:34
lotuseater
they have something like (defmethod stack-push :filter :number ((stack (eql 'normal))) ...)
13:55:11
jackdaniel
that's nice to hear; either way preparing a tutorial after giving a talk is an extra work - if someone does that it is fine; but expecting it from that person is a bit much
13:57:06
rain3
I am not asking for it.. I am only pointing out that the same thing can be done more efficiently . If you have the tutorial prepared before, then you don't need to waste time thinking and fighting with bugs 'sorry, it didn't work, we'll try again in the next video'
14:51:05
pjb
rain3: totally agree, videos are two slow in general. However it is not necessarily a bad thing: 1- you can listen to them in parallel to some other task. 2- sometimes you're not working at 100% of your capacities yourself, so it's not so bad they're a little slow. 3- it can give you an excuse for such a slow day.
14:51:41
pjb
(and you can always play them at 2x, even if one may sometimes regret there's not an option to play them at 10x).
14:52:59
pjb
Perhaps somebody at google will implement an AI to summarize a video, and transform it into a multimedia stream that could be absorbed by brains in high speed. Or perhaps we'll have to wait for neuralink to provide the brain downloading feature…
14:53:11
rain3
the video above actually was quite good in comparison with others. I've downloaded them so that I can play at 10x with vlc player , then I deleted all of those which only had examples from the readme of the lib
15:36:39
zos
Hello all, I am interested in learning common lisp. I have started working my way through Practical Common Lisp. Right now I am using SBCL with Doom Emacs. In some tutorials I have come across mentions of roswell and am wondering if it is worth setting up from the start.
15:37:47
rain3
it's good to set it up so that you can easily switch between different versions of SBCL when needed
15:40:20
Lycurgus
i believe in n chances, especially if somebody else does the m+1 of them, in my case m is like 2 -4
15:42:57
zos
That sounds like good advice - it is going to take some time before I am likely to need to worry about deployment. Thank you for your help
15:44:07
beach
zos: #commonlisp is not really for newbie questions, in case you want to ask such questions at some point. They are tolerated though, and if they become too trivial or too numerous, you might be asked to go to #clschool.
16:14:56
phantomics
When I eval: (defvar foo 123) (let ((set-foo (lambda (x) (setf foo x))) (foo foo)) (funcall set-foo (setq foo 55)) (* foo 3))
16:16:11
phantomics
Why doesn't the set-foo function defined in the (let) affect the top-level binding of foo? The function is defined before the value inside the (let) form is locally bound to foo
16:18:31
phantomics
Ok, so this method would work if foo was a lexical variable defined in a (let) form containing the code rather than a special variable?
16:19:15
jackdaniel
daily quiz: what's a practical value of (apply foo (car bar) (cdr bar)) compared to (apply foo bar) ;?
16:19:47
beach
phantomics: It depends on your definition of "work", but yes, you can close over lexical variables.
16:24:33
jcowan
Sure, but my claim was that there are no global lexical variables, as you cannot wrap a let around your program (becaause too many things in it insist on being top-level)
16:25:27
jackdaniel
there are file-local-variables (in the standard it is i.e *readtable*, but there is also asdf extension that gives you that)
16:27:10
beach
jackdaniel: I can imagine if foo is something like #'apply, and (car bar) is something like #'reduce, but I am not sure.
16:27:18
jackdaniel
of course one could argue that the practical application I have in mind can't be considered a good style of programming
16:28:33
jackdaniel
however you could specialize the first argument on null as a fallback, and then bar could be nil
16:39:13
pjb
zos: have a look at http://cliki.net/Getting+Started http://cliki.net/Online+Tutorial and in general, http://cliki.net
16:39:56
j-sh
What is the current best way to use the GNUstep GUI toolkit (and maybe its RAD tool Gorm) with common lisp?
16:43:27
pjb
phantomics: yes, it would work with lexical variables, but the question is why did you make a temporary binding of foo in your let?
16:44:38
pjb
phantomics: it's not wrong, that's the point of dynamic variables: they let you shadow temporarily the previous binding, during the execution of a let body. But the point is that if you want to have the previous binding mutated, you should not temporarily shadow it!!!
16:45:14
pjb
(defvar *foo* 123) (values (let ((set-foo (lambda (x) (setf *foo* x)))) (funcall set-foo (setq *foo* 55)) (* *foo* 3)) *foo*) #| --> 165 ; 55 |#
16:46:16
pjb
phantomics: note that defvar declares the variable name as special; this is a global declaration that cannot be removed or shadowed. So it is important to use the earmuff convention foo -> *foo* to avoid damaging the normal variable names.
16:48:29
pjb
j-sh: the first problem is that there are different Objective-C runtimes: Apple's runtime, GNU runtime, cocotron runtime (used on MS-Windows I believe).
16:49:49
pjb
j-sh: the next problem is the FFI from lisp to Objective-C; there are some nice things: the objc runtime is a pure C API. There are some more difficult things: depending on the runtime and the libraries or frameworks, it's not always easy to introspect the API; we still have to process the headers, and this is not easy (failproof) to do it automatically.
16:51:19
pjb
j-sh: ccl provides a FFI that works with Apple's runtime. I don't know how well it could work with GNU Objective-C runtime. There's an Objectiev-C FFI that could work with GNU Objective-C (and possibly cocotron, IIRC), but AFAIK it has bitrotten.
16:52:18
pjb
even ccl uses a ffigen that is a patch of Apple gcc version 4 that is entirely outdated and therefore is completely bit-rotten; I don't know how to generate the cdb files ccl FFI requires to deal nicely with frameworks on current systems.
16:53:42
pjb
In short, it's a mess, and we'd need to re-implement from scratch an CL -> Objective-C FFI that would work nicely for all runtimes, and all frameworks. But it's not a small project.
16:55:19
pjb
j-sh: Have a look at this objcl library (which is a reader-macro to give [recipient message:arg with:arg] syntax over ccl objc FFI: https://github.com/informatimago/lisp/tree/master/objcl
17:01:11
phantomics
pjb: I know I'm using weird conventions in this code, that's because it's for the output of my APL compiler, and APL has some odd conventions for implicitly shadowing variables but also has ways of modifying variables at the top level of their scope
17:06:02
pjb
j-sh: you can still do something. Have a look at https://cliki.net/com.informatimago.hangman
17:07:08
pjb
j-sh: well, http://git.informatimago.com/viewgit/index.php?a=tree&p=public/games/hangman-cocoa-objc&h=6a4c99a8d52e6cdaf08b28dffa67797dab8f9a89&hb=99a80c170985b9b3a753025c00870c63a94cf678 ; I don't know if I left the gnustep version on github.
17:08:06
pjb
(nope. the gnustep/ subdir only contains objective-c code… ; as I said, the problem is difficult and has not been worked on ever)
17:09:00
pjb
j-sh: basically, CL is a not used by a lot of programmers. GNUstep is not used by a lot of programmers. CL with GNUstep is used by a handful of programmers and we don't have the resources to make any progress on the combination.
17:09:21
pjb
j-sh: as beach said, it has been a long time since we talked about GNUstep in #commonlisp
17:37:52
pjb
lotuseater: j-sh: so a first step would be to define a complete FFI to the gnu objective-c runtime, to the apple objectiev-c runtime, and possibly others (cocotron, etc).
17:38:32
pjb
lotuseater: j-sh: next, define a higher level CL to objective-C interface using introspection and working with all runtimes.
17:38:52
pjb
lotuseater: j-sh: finally, use my reader macros to be able to use the nice Objective-CL syntax ;-)
17:40:09
pjb
two problems: 1- dealing with C macros in the objc runtime headers. 2- dealing with the various structure layout (packed/not packed, 32-bit/64-bit, and other #ifdef).
17:41:20
pjb
(perhaps introspecting the debugging symbols in the shared libraries of the run-time, but this adds a new complexity (gdb symbols, dwarf symbols? what to do if stripped without debugging symbols? etc).
17:52:40
j-sh
pjb Thank you for your extensive answer, I didn't know that it would be that difficult to use CL with GNUstep.
17:52:41
j-sh
I personally think that it is a shame that GNUstep didn't get the recognition that something like GTK (it is even older than GTK AFAIK) got, but I guess there aren't many people that want to touch an Objective-c codebase especially since it has the "Apple stigma" to it and even Apple themselves try as hard as possible to deprecate Objective-C
17:56:15
pjb
j-sh: well, it's not that it's difficult. It's labor intensive to do something complete.
17:56:49
pjb
j-sh: if you just want to send a few message, you really need to FFI a few functions in the runtime. You can easily do that.
17:57:35
pjb
j-sh: if there were bindings for other languages to GNUstep, that could indeed make it more popular.
17:59:10
j-sh
pjb There are apparently bindings for Java, ruby and Smalltalk, but I'm not sure how much they are maintained.
18:01:57
pjb
For example, one problem is that the code used to encode the types are defined with #define in the runtime.h header. So CL needs either to parse those headers, or to hardwire the encoding for each compiler/runtime.
18:04:58
pjb
j-sh: For example, you can cffi:defcfun objc_msgSend and sel_getUid, and then already you can send messages to objects.
18:05:32
pjb
but in CL you'd want to be able to introspect the classes and objects. So you'd want the full monty.
18:06:26
pjb
j-sh: try: (defclass foo () ((x :initarg :x) (y :initform 42))) (inspect (find-class 'foo)) (inspect (make-instance 'foo))
18:07:02
pjb
you'd want to do the same in CL for Objective-C classes. (even more, since in Objective-C the methods are attached to the classes, so you'd also want to list the methods).
18:17:11
lotuseater
a C# software developer he could also extend his language or that it couldn't be different in CL ^^ hard to make him clear why not
18:57:38
jcowan
Anyone who asks you "But why would you want to do *that*?" when talking about cross-language comparison should be resolutely ignored: nobody "wants" a feature they've never heard of, because they've learned to work around it or not to want it or are simply clueless. (However, the question does make sense within a single language, because there may well be a different way to achieve the same effect within the language.)
18:59:48
lotuseater
I also tell that's an experience no one can just tell you, you must make it yourself to really get the difference.
19:06:40
lotuseater
haha and just because something is used by more people or there are many (aweful) book about it means it's better (just better marketed)
19:09:35
jcowan
This is a highly speculative question: can anyone see a use case for a separate class hierarchy allowing you to create generic functions that aren't related to ordinary generic functions?
19:10:34
tfeb
lotuseater: 'popular does not mean good: it merely means good at being popular', as someone once said.
19:13:09
aeth
jcowan: type-based generic dispatch in CL. https://github.com/markcox80/specialization-store/
19:13:21
aeth
types allow you to dispatch on numbers/sequences/arrays that don't necessarily have distinct classes
19:13:58
aeth
since you could e.g. (simple-array octet (* *)) as a type for 2D (unsigned-byte 8) arrays (assuming octet is defined as such)
19:14:22
aeth
jcowan: I wouldn't be surprised if there are other distinct generic function systems, too, perhaps more close to what you're looking for
19:15:15
jcowan
Clojure allows uou to create independent class hierarchies, but I can find no explanation of why you'd want to.
19:16:50
aeth
and perhaps you can't do that, if it's a well-specified language in a way that's incompatible with the host language
19:17:13
aeth
but that way at least you can use built-in generics even though you can't use the built-in class hierarchy
19:24:32
pjb
jcowan: independent class hierarchies are usually an illusion, there's normally a TOP class (eg. T in CLOS).
19:26:20
pjb
jcowan: that said, in CLOS, playing with metaclasses, you can probably implement separate subhierarchies according to your wishes.
21:41:55
phantomics
Hey, another question. When a local binding is created for a dynamic variable, functions defined even outside the scope of that local binding will use the local binding, correct?
21:42:15
phantomics
(defvar foo 13) (defun set-foo (arg) (setf foo arg)) (let ((foo 3)) (set-foo 50) (+ foo 10)) foo
21:42:43
phantomics
The (set-foo) inside the (let) form doesn't actually change the top-level value of foo, even though the function is defined outside
21:47:02
pjb
lotuseater: the only advice I have for Bjarne, unfortunately, is to ask to be interned in a lunatic asylum.
21:48:01
pjb
phantomics: you don't have _local_ bindings for _dynamic_ variables. It's like asking what taste is the color blue? it's meaningless.
21:48:26
phantomics
I know that the dynamic vars have a value stack and a value gets pushed onto the stack within that (let) scope
21:49:43
pjb
phantomics: the set-foo mutates the binding of the dynamic variable foo that is active WHEN the function set-foo is called.
21:50:13
pjb
phantomics: in set-foo, there is absolutely no consideration for WHERE the variable foo is bound. Only WHEN it is bound.
21:51:28
pjb
Note: you can define global lexical variables using symbol macros. Search for a defglobal or deflex in libraries.
21:51:37
phantomics
The workaround, as far as the April compiler is concerned, is to prefix the internal representations of all lexical variables with something
21:53:56
phantomics
And then if they write a function like {foo←5 ⋄ foo+⍵}, a lexical variable called something like "𝕏foo" that shadows the value of the external "foo" is instantiated in a (let) form
21:54:08
pjb
So you're implementing lexical variables with dynamic variable when you have a language that already has lexical variable? Why? Are you masochist?
21:55:07
phantomics
It creates implicit shadowed lexical variables when you assign a value to something inside a function
21:55:47
pjb
Ok, so you have toplevel variables that are dynamic and local variables that are lexical, and the local lexical variable can shadow the the toplevel dynamic variable?
21:56:04
pjb
In that case, indeed, you need some kind of renaming or use a different namespace (package).
21:56:57
phantomics
The tricky part is that APL has a type of function that can reassign the global variable outside of a function's scope
21:57:35
phantomics
But given a lexical variable's name, I need to be able to find if it is shadowing a dynamic variable
21:58:44
pjb
(mapcar (lambda (lexical-apl-variable) (cons lexical-apl-variable (gensym))) lexical-apl-variables) to build an a-list mapping the lexical-apl-variables to lexical-lisp-variables.
21:58:48
phantomics
If I rename them to gensyms how do I find their corresponding external dynamic var symbol? Enter it all in a hash table? The prefix system is simpler, and double-struck letters are illegal in April var name so there's no chance of collision
21:59:19
pjb
You find that before, by looking up the apl-variable in a global-dynamic-apl-variables list.
22:00:18
phantomics
That seems cumbersome when I could just remove the lexical var prefix and look for the existence of a dynamic var by that name