libera/#commonlisp - IRC Chatlog

0:24:00 Lord_of_Life_ ** NICK Lord_of_Life

4:37:11 beach Good morning everyone!

4:37:44 Josh_2 Mornin

5:35:21 refusenick For a C-to-Lisp compiler like Zeta-C or Vacietis, how little of the POSIX API could a Lisp OS get away with implementing while still being able to reuse drivers from a Unix derivative?

5:36:46 beach Not sure what you mean. Translating C to Lisp does not need any OS drivers.

5:39:55 refusenick beach: Using the drivers in a Lisp OS would necessitate support for some aspects of Unix to call into them, no?

5:40:56 beach I am sure you are right. I just don't understand your question. It might help if you tell us what it is that you try to accomplish. Sorry, I am notorious for having a hard time understanding.

5:41:12 beach But I am very interested in this question because I have plans for a LispOS.

5:41:48 refusenick I know very little about low level development - I only took an introductory x86 assembly & architecture course.

5:42:09 refusenick Nevertheless, don't drivers provide some sort of API to use them to interface with hardware?

5:42:14 Nilby It really depends which drivers you want to use. Storage drivers are fairly easy and have a small interface. Graphics card drivers on the other hand are extremely difficult.

5:43:13 refusenick Nilby: Graphics is the big issue at hand, I suppose - if there's to be any draw towards a Lisp OS (or any other OS) over Unix, it's the graphical aspect

5:43:26 beach refusenick: Please. I am not questioning your intentions or your competence. I am trying to understand what you want to do.

5:43:35 refusenick I know

5:44:00 beach refusenick: Are you saying you want to reuse drivers in a LispOS by translating them from C using a C to Lisp compiler?

5:44:07 refusenick yes

5:44:12 beach Ah, good to know.

5:44:38 beach I don't think that is possible at all, at least not with the way such compilers currently work.

5:44:51 Nilby We have a LispOS currently, but one pain point is it doesn't use graphics hardware last time I looked, probably because of that issue.

5:45:17 refusenick Have you taken a look at https://github.com/vsedach/Vacietis ?

5:46:05 refusenick Unfortunately, Unix makes safe interop gratuitously difficult: https://arcanesentiment.blogspot.com/2012/05/pointer-arithmetic-can-be-safe.html

5:46:52 refusenick Pascal should have won. At least the type safety would've made it easier to compile drivers written in it to a better language.

5:47:14 refusenick (Well, Lisp should have won, but for most people, their choices were C or Pascal)

5:49:25 beach refusenick: A device driver needs direct access to RAM. You can't do that in Common Lisp, at least not portably, and not in C either. All such C programs exploit undefined behavior in the C standard.

5:49:31 refusenick Since Unix operates on plain text streams, for example, I assume supporting POSIX architecture for the driver would have to include UTF-8 encoding and stdout/stdin/stderr streams at the least, which is already

5:49:38 Nilby I think it is possible, but it would be quite a lot of work to get right. But probably less work than rewriting all hardware drivers.

5:50:03 Nilby Of course you can access RAM in CL.

5:50:13 refusenick which is already, in my eyes, irrelevant legacy cruft

5:50:18 Nilby Just not standardly.

5:50:36 beach Nilby: That is precisely what I said.

5:50:42 Nilby Except we have a defacto standard FFI.

5:50:57 beach In a LispOS?

5:51:53 beach I think we are talking about very different kinds of LispOSes, so I'll be quiet.

5:52:24 Nilby beach: Oh, sorry. I can barely think, type, read simultanously :|

5:53:18 pjb beach: who says a device driver needs direct access to RAM?

5:54:03 beach I am sure you are going to tell me that I was wrong, and I am listening.

5:54:35 refusenick I'm asking this here because the Lisp community is, by far, the most advanced I've seen in writing an OS from first principles without copying Unix

5:54:51 pjb beach: the principle is to be a controlled environment. I means that low level stuff is done by the lower level (eg. hardware). You can have a DEVICE that uses a (vector (unsigned-byte 8) *) as buffer, and the hardware will do the right thing with the lisp object.

5:54:53 refusenick but I actually wanted to write it in something closer to a statically-typed Forth

5:55:11 refusenick Feel free to crucify me

5:55:52 beach pjb: Fair enough.

5:55:54 no-defun-allowed (append (loop repeat 2 collect (make-nail)) (loop repeat 2 collect (make-wooden-beam)))

5:56:12 pjb beach: you may need a meeting point, but the hardware, instead of presenting devices as addresses and 8-bit registers, can as well represent them as lisp objects and fixnums.

5:56:53 beach I suppose so, sure.

5:57:10 refusenick pjb: That's how I assume any OS written from the ground up with a particular language's abstract machine in mind would do it: wrap it in the native idioms and do as the Romans do

5:57:37 pjb beach: one example was the NuBUS cards on Macintosh, that needed to be built with a ROM that gave a high level representation of their features. A precursor of EFI hardware layer (I hear you can write EFI drivers in forth).

5:57:59 refusenick After all, Unix's Achille's heel is that plain text is limited in what it can represent because it lacks structure like sexprs

5:58:14 pjb EFI hardware *abstraction* layer.

5:59:11 pjb refusenick: it's not really that. It's that the structure (of the plain text) is meta data, outside of the plain text. It's encoded in the parser and serialisers used in different programs (and therefore that can be different).

5:59:28 pjb refusenick: I say it's also an advantage, since that let you edit configuration files with a simple text editor.

5:59:40 refusenick If one were to write a microkernel for a Lisp OS to cleanly present the hardware as Lisp objects to everything about it, how would the user be prevented from mucking about with it if all objects are mutable by default in Lisp?

5:59:40 pjb refusenick: but the alternative is to store objects.

6:00:09 pjb refusenick: the only difference here is that lisp has a "standard" textual representation (printable readably sexps) for SOME lisp objects.

6:00:12 no-defun-allowed You could have access control, and only allow code in certain contexts to modify objects.

6:00:24 refusenick pjb: On that note, I'm particularly interested by Xerox's Lisp Machines, which stored everything as objects and provided first-class support for live structure editing

6:00:30 pjb refusenick: but as long as you don't use an object database, sexps, xml, or unix table files are the same.

6:01:42 pjb refusenick: but an object database is a bore, there are still problems (notably with identity, since normal databases make copies). So it's only good once you have a persitent memory and no file system, and no need for a database. Then your "files" are just lisp data structure in (persistent) memory.

6:01:44 refusenick My dream - as a math person who ends up wasting time on computers to make them do math stuff faster - my dream is a tablet with a pen and a CAS with first-class structure editing and live look-up and advice support

6:02:17 refusenick pjb: Why have files at all? I find the Smalltalk/BBN Lisp image concept intriguing

6:02:26 pjb Exactly.

6:02:37 refusenick plus, there's lots of work on capabilities for image-based systems (see: EROS, CapROS)

6:04:12 pjb However, again, the advantage of unix, the reason of its adoption, is that once you store something in a file, you can process it with any kind of tools. When you have objects, you can only send to the object, the messages that have been programmed into its class! So it can be very difficult to edit or manipulate it, when the set of methods provided is not comprehensive.

6:05:00 pjb refusenick: no, there's not a lot of work on them, unfortunately. The only guy who worked on them has been hired by Microsoft several years ago so any further development have been dropped.

6:05:23 refusenick Wasn't the idea of objects (the original one of Alan Kay, closer to the actor model) that you could send messages consisting of raw data to other objects without considering their implementation?

6:05:30 pjb There are some documented archived on the Internet, and that's all.

6:05:43 refusenick oh :()

6:05:48 refusenick :(

6:06:07 refusenick a la Erlang

6:06:10 pjb System research has disappeared 30-40 years ago.

6:06:27 refusenick That's how it appears to be

6:06:37 pjb About the time when universities started to teach MS-Word…

6:06:39 refusenick Hence my desire to reuse drivers and not reinvent the wheel

6:06:52 pjb Yep, use Linux as HAL…

6:07:01 refusenick no

6:07:28 refusenick I'd like something I can seamlessly run on a RasPi or in the browser

6:07:52 refusenick even Plan 9 would be better than what we have now, and Plan 9 is a far cry from what could be

6:08:07 pjb Only Linux has the device drivers…

6:08:24 refusenick https://github.com/snabbco/snabb

6:08:50 refusenick There's been a quiet revolution in BSD land with rump kernel and userland driver experimentation

6:10:32 refusenick As much as everything "cloud" tends to be cancerous, it has lead to a renewed interest in ideas such as exokernels and the like

6:11:32 refusenick Now is a better time to start a non-Unix OS than in the past 30 years, even if that's not saying much

6:12:06 beach refusenick: You refuse access through the use of capabilities.

6:12:30 refusenick https://nanovms.com/dev/tutorials/running-forth-unikernels

6:14:27 beach refusenick: Have you read my CLOSOS specification?

6:14:36 pjb beach: check: http://www.alfamontreal.info/NuBus.pdf eg. page 30 there's a diagram showing an example of an Apple NuBus property list. There are attributes, and block of (driver) codes.

6:15:19 beach pjb: Thanks.

6:15:39 pjb beach: the only difference is that they were designed to be used in 680x0 assembler or in C, so the data structures are pointers and uint32. But they could be formatted as an immutable lisp heap.

6:15:42 refusenick I wonder how far a unikernel with image based Lisp could be stripped down

6:15:42 refusenick https://www.informatimago.com/develop/lisp/com/informatimago/small-cl-pgms/ibcl/index.html

6:16:34 refusenick beach: I have not

6:16:38 pjb refusenick: very far. Since CL implementations have FFI, you would only need the kernel, the CL implementation and its run-time shared libraries.

6:16:54 pjb refusenick: see for example http://informatimago.com/linux/emacs-on-user-mode-linux.html

6:17:01 beach refusenick: metamodular.com/closos.pdf

6:17:08 refusenick yep, found it

6:17:25 pjb refusenick: that version of emacs didn't have module, so I had to add mount(1), but with an emacs module or with CL FFI, we can mount from the process itself.

6:17:28 refusenick I wonder what it would take to support CL atop a smaller kernel like L4

6:18:03 refusenick To be extra evil, maybe compile L4 into CL and ditch C entirely for even bootstrapping!

6:18:08 pjb refusenick: basically all CL implementation use libc (glibc, etc). They don't do direct syscalls AFAIK.

6:18:30 beach But they probably should.

6:18:34 refusenick compile Musl into Lisp?

6:18:43 refusenick I've gone drunk with power

6:18:51 pjb Indeed, there's a movement even amongst C programmers, of using direct syscalls instead of libc…

6:19:11 refusenick As if C could be more unsafe

6:19:13 pjb Since some Linux syscalls give more power than POSIX libc…

6:19:27 refusenick L4 has only 7 syscalls

6:19:40 refusenick I wonder if there's a POSIX-compliant libc which can run atop it.

6:19:48 refusenick Maybe that could compiled into Lisp

6:20:00 pjb refusenick: you have the sources, have fun!

7:02:42 dale_ ** NICK dale

7:03:46 LdBeth Hello

7:05:34 LdBeth l4 is basically a “modern” OS influenced by UNIX

7:05:50 dale_ ** NICK dale

7:14:37 dale_ ** NICK dale

7:36:19 X-Scale` ** NICK X-Scale

7:46:04 X-Scale` ** NICK X-Scale

8:53:36 ebrasca LdBeth: I think l4 is microkernel

8:55:13 d4ryus refusenick: Yes: https://l4re.org/

9:23:24 p_l LdBeth: there's nothing UNIX in L4

9:25:31 ebrasca p_l: Are you sure?

9:26:51 p_l ebrasca: I am

9:28:12 p_l you can build an UNIX system on top of it, but by itself it's closer to core layer of Spring, which was UNIXish even in philosophy, but in a mirror-verse way

9:28:43 p_l L4 supports switching processes and passing messages between them. That's about it, everything else is custom

9:29:37 ebrasca How to test this L4?

9:29:49 ebrasca Is it better than GNU/Hurd?

9:30:08 p_l L4 was for a time proposed as mach replacement in Hurd

9:30:17 p_l L4 by itself is very, very tiny kernel

9:30:22 p_l so it's not a comparison

9:31:08 p_l usually for L4 implementations there's a bunch of code that makes it more usable (and it was common to use L4/Linux to provide device drivers etc.)

9:33:30 ebrasca How to test it?

9:38:12 p_l you'd need to grab some of the more complete systems using it

9:38:30 p_l seL4 has a bunch of userland to make usable demo, so does Fiasco

10:10:26 Shinmera L4 is also off topic.

10:11:28 Cymew But it's very, very tiny. :)

10:12:11 no-defun-allowed By mass, it is still 100% off topic.

10:12:22 Cymew Indeed.

10:13:22 Cymew Incidentially, I don't think I've seen much conversation about what kind of kernel architechture to aim for among the lisp development done in kernel space.

10:14:17 no-defun-allowed The CLOSOS paper suggests that a kernel is unnecessary, and that is what I believe.

10:14:35 Cymew In theory you don't need one big flat memory space for kernel and user space, right?

10:14:37 jackdaniel Cymew: beach discussed it in his lispos paper - there is no need for a separate kernel from applications. you have no direct pointer arithmetic and memory separation is just an unnecessary baggage

10:14:58 Cymew Look at that. I had missed that paper.

10:15:01 no-defun-allowed In a language where you cannot manipulate memory, you should not need to hide memory from client code through process boundaries.

10:15:21 jackdaniel (so applications are more run as functions, not as separate processes with their own address space)

10:16:14 Cymew It sounds like it could cause some interesting bugs, though.

10:16:45 Cymew I can see some benefit by separating processes in different spaces to protect them from accidents.

10:17:18 Cymew But, I guess beach have thought about that.

10:17:40 no-defun-allowed If the compiler generates safe code, the resulting operating system should be safe.

10:17:51 beach Cymew: If those accidents are program defects, then there is no protection possible against those.

10:18:13 no-defun-allowed Is rowhammer still a problem?

10:18:25 Shinmera if you don't have ECC memory, yes

10:19:09 Cymew beach: I guess that's true to some extent. I need to read that paper I think.

10:19:32 Cymew Is there a compiled version online? My google fu only leads me to the tex sources.

10:20:03 no-defun-allowed I think only hardware bugs would be problematic for a CLOSOS with a bug-free compiler and runtime.

10:20:13 beach Cymew: metamodular.com/closos.pdf

10:20:15 no-defun-allowed http://metamodular.com/closos.pdf

10:20:44 Cymew Thanks.

10:38:33 hhdave_ ** NICK hhdave

12:05:39 pjb So, I get in the mini buffer this message: "like funciton would" from time to time. Any idea where it comes from?