freenode/#sicl - IRC Chatlog
Search
3:50:52
no-defun-allowed
With a green threading configuration that assigns many green threads to OS or hardware threads, would it be better to assign one nursery per green thread or per OS thread? I think the latter might be more predictable on the cache (but does it really matter on the millisecond scale?), but it would complicate tracing the nursery significantly.
3:52:27
no-defun-allowed
Oh, and the latter would also require you to copy objects to the global heap to migrate a green thread between OS threads. That is also no good.
4:09:04
beach
no-defun-allowed: I think it is impossible to say something general about the performance difference of those two cases.
5:26:06
no-defun-allowed
I ask because I was pondering what the most convenient way to do asynchronous programming in Common Lisp, and now I am thinking that it would be to use green threads, as then all the synchronous libraries, say usocket and bordeaux-threads could then use green threads.
5:27:22
no-defun-allowed
Then, in turn, green threads could be implemented conveniently if I can create and swap out processor contexts. Those contexts would have to have their own stacks, but I'm not sure about nurseries.
5:53:40
beach
I am having a hard time with the concept of green threads. If I understand things right, they exist to compensate for inadequate operating systems in terms of performance, and they put a huge burden on the application writer that, in my opinion, is the responsibility of the operating system.
6:00:21
beach
Sure, but you wouldn't have any conflict, because OS threads would be just as efficient as green threads, so there would be just one kind of thread. And I think each one would have its own nursery as I have planned for SICL.
6:04:03
beach
The reason I want a nursery per thread is so that each thread can do a local garbage collection without other threads being involved, so there is no synchronization required.
6:04:51
no-defun-allowed
Right then. It would still be nice to have better threading earlier though, and the implementation would be almost the same as if we were implementing threading without an operating system; except that you start up OS threads instead of cores, and use a system call to set up preempting timers instead of configuring a hardware timer.
6:06:57
beach
Sure, the green thread implementation in an existing implementation would very likely be similar to the OS-thread implementation in CLOSOS.
6:14:23
no-defun-allowed
I'm convinced that one nursery per thread would be less tricky than one nursery per core then. But it would be amazing if all the synchronous Lisp code could be made to use the faster asynchronous IO protocols, and SICL looks like it's headed that way in the long term (except that there's more Lisp code in the implementations of those protocols).
6:15:59
beach
Do you mean the "faster asynchronous IO protocols" as in what they are not adding to OSes like Linux to deal with faster I/O devices?
6:17:04
no-defun-allowed
There are some that are sort of in between the usual synchronous protocols, and what we could have.
6:19:07
no-defun-allowed
For example, io_uring in recent Linux kernels can apparently share buffers without copying in some situations, but there are also poll on Unix and epoll on older Linux kernels.
6:20:51
no-defun-allowed
There are some C libraries that abstract over those (such as libuv), and then Common Lisp libraries that abstract over the C libraries (such as cl-async), but I don't like the programming style, as you have to write using callbacks instead of the way you usually write sequential code.
6:22:24
beach
Yeah, callbacks make things hard. Especially in a language with no first-class closures.
6:24:00
beach
I need to start getting ready for going to buy food. I want to be there soon after they open in around 25 minutes, so as to avoid the annoying crowd that doesn't respect social distancing.
8:35:58
beach
But not *just* capabilities. I was not convinced by the EROS argument that capabilities are all we need.
8:36:46
beach
I am more convinced about the Multics argument where the object store uses ACLs and when you retrieve an object from the object store, the ACL is turned into a capability.
8:55:15
pjb
beach: in the case of EROS, there is no separate object store. All the universe is persistent. This is why they can do without ACLs. If you want to serialize objects in an object store based on vectors of bytes, then the introduction of ACL is justified.
8:57:53
pjb
beach: if you have an in-memory object store, you could just keep a reference to the capabilities. No need to convert them into ACL?
9:01:55
pjb
If you want to read back an ACL and find a pointer to a garbage collected object, this might be a problem.
9:02:22
pjb
"when you retrieve an object from the object store, the ACL is turned into a capability."
9:02:58
pjb
I mean for the reference (capability) the object in the object store has to other objects.
9:04:05
pjb
But you're saying you're converting capabilities -> ACLs -> capabilities. When theyre ACLS, there's no pointer anymore.
9:04:16
beach
Basically, the object store maps names and ACLs to maximal capabilities. When a user retrieves the object, the capability is ANDed with what the ACL says.
9:05:28
beach
The user who puts an object in the object store can decide on a maximal capability. It could have all privileges or just a subset.
9:05:55
beach
Then the ACL determines how these privileges may decrease depending on the user doing the retrieval.
9:06:57
beach
So I can say "nobody has write access to this object, but anyone in the ADMIN group has read access."
9:08:30
pjb
IIRC, in EROS, there were domain controller object who would control how the capabilities are passed from one object to another. The object store would have been such a domain controller, and would have perform this capability downsizing upon request.