freenode/#lisp - IRC Chatlog
Search
14:06:35
makomo
you've probably seen it already, but appendix c deals with some of the circularities/bootstrapping problems
14:06:53
phoe
shka: beach: you both are right, I just wanted to write it for the sake of writing it. :D
14:08:05
xificurC
makomo: yeah, not sure if I'm gonna finish the book though as I haven't done much CLOS in order to fully understand and appreciate what lies underneath it
14:08:57
makomo
also, i realized yesterday that 3-lisp was the author's phd thesis and the first formalization of the concept of reflection
14:10:11
makomo
i've heard that sonya keene's clos book is a good introduction to clos itself. what do others think of it?
14:20:23
beach
makomo: Probably because it used examples that I either could not appreciate, or that required non-standard functionality.
14:22:56
beach
makomo: If I were to write a book like that, I would probably use more mundane examples like people, companies, employees, countries, etc.
14:23:50
makomo
beach: that's something that i wanted to comment on. aren't such examples better than the washed-out examples of animals, shapes, etc.?
14:24:20
makomo
i've seen one review where someone says that the book does actually go over the canonical animal/shape example, but that there are actual "industrial" examples afterwards
14:26:14
makomo
yeah, i don't think i do either, but i also don't recall ever seeing such a thing used in an OO introduction
14:27:59
beach
I mean, as if a person who happens to be a student is always going to remain a student and has always been one.
14:28:50
makomo
sometimes it just seems to me that more elaborate examples would be much better than the usual "we'll skip this, because it would take too much time" or w/e the usual "excuse" is
14:28:57
beach
I think that's the error in most books. They make class hierarchies that would absolutely not work in a real application.
14:30:07
beach
I guess that *is* an advantage of Keene's book. The examples are real, from real situations.
14:36:17
makomo
beach: do you plan on SICL supporting stuff like threads and other usual extensions? even if you don't, how would you go about implementing these since SICL is implemented in CL?
14:38:00
makomo
it might not matter that it is implemented in CL, as you said before (and the same thing goes for the LOOP macro that you implemented in CL), but if you for example relied on SBCL's threads or FFI how would you ever get rid of that dependency once you've bootstrapped
14:39:20
beach
makomo: I would just make sure there is a bunch of functions that interface to the system calls of the operating system.
14:39:24
makomo
or would there have to be a part written in C for example to interface with the platform you're running on?
14:41:48
beach
makomo: Because SICL is the basis of a planned Lisp operating system, and that operating system will not have any libc.
14:42:45
makomo
so you would basically roll your own implementation of threads? i guess that makes sense if you're writing your own OS
14:44:23
makomo
aha, right. the lisp OS would be a separate project, but it would rely on SICL for most stuff i guess?
14:44:51
beach
SICL with first-class global environments would be the basis, since it would already be a multi-user Common Lisp system.
14:45:07
random-nick
why not have a macro for syscalls? glibc has a function that lets you call any linux syscall
14:45:15
warweasle
makomo: If someone makes a lisp os, they really should use the linux kernel for drivers. Even if they have to convert them to lisp.
14:45:58
beach
random-nick: It is not as simple as that for Common Lisp. You have to define what kind of Common Lisp data structure will be used for the call that normally accepts C structures, and then translate.
14:46:34
Bike
Ukari: "no values" is implicitly turned into NIL anywhere expecting one value,which is basically everywhere
14:47:14
Bike
Ukari: one exception is you can do (multiple-value-list (values)) => NIL versus (multiple-value-list nil) => (nil)
14:47:47
makomo
beach: and the point that warweasle makes about portability? is writing a whole kernel and all the different device drivers something that's part of the plan for the lisp os or?
14:49:33
Bike
it's really incredible how annoying it is to tell asdf where to get files. no wonder central registry is still around
14:49:38
makomo
that's what my question is. is the planned lisp os going to reuse the linux kernel or create its own
14:50:07
beach
makomo: Contrary to other people, I consider the "running on bare metal" to be less important, but a cute thing, of course. I would be happy to run it in a Linux process to start with.
14:50:09
makomo
and if the answer is the latter, isn't that a huge undertaking and something one wouldn't really want to do
14:50:16
beach
makomo: Also, people assume that I want this thing to be widely used. That would be great but I doubt it will happen.
14:50:18
beach
makomo: I am a researcher. I just want to show that we can do better than what we are currently doing, and what we have been doing in the past. But I don't have great hopes that many people will be bother.
14:51:01
random-nick
and talking to a kernel like Linux requires most of the difficult parts of a FFI
14:55:19
warweasle
I posted this to the other channel but I will here too: http://wiki.c2.com/?SynthesisOs
14:55:41
beach
I think it would be a great exercise to see whether CLOS could be used to create a better hierarchy of device drivers than what C would allow.
14:56:36
beach
Such a hierarchy *might* take some inspiration from I/O Kit, but not too much, of course.
14:56:45
makomo
beach: so in the context of SICL running as a linux process, supporting threads within such an implementation would require calling into libc right? how would that be done if SICL is written using CL?
14:56:58
beach
warweasle: Oh, I am not talking about speed. I am talking about maintainability and, yes, security.
14:57:30
makomo
warweasle: interesting. i heard something similar a while ago, something about recompiling the code of the filesystem i think?
14:58:10
beach
warweasle: I would like to see whether it could be advantageous to use CLOS to write such a collection of drivers.
14:58:52
makomo
beach: ok, but i think the question of "how would it be done within CL?" still stands. how would you connect assembler and CL?
14:59:35
Bike
the first research paper i ever read was an operating system based on having objects with jit compiled methods for syscalls. that was neat
14:59:43
beach
makomo: There is always a little machine code or "assembler" in a complex system like Common Lisp. The code generator of a compiler for instance. And there would have to be a few lines of such assembler to implement a Common Lisp function such as POSIX:OPEN.
15:00:14
warweasle
beach: The problem is OO requires the language or runtime to do things for you. In kernel space you don't have that luxury. If you take the runtime away, you just have structures.
15:01:08
beach
warweasle: Are you talking about a typical Linux-like kernel or about a new LispOS kernel?
15:02:38
on_ion
we are running countless OSs in virtual machines, virtualbox or qemu etc, why not just accept SBCL or emacs et.al for another VM? why a bare metal machine OS?
15:04:17
warweasle
It's a microkernel. That didn't seem to work out. I don't believe smaller will make it work any better. There has to be communication between pieces. And then you have a kernel again.
15:04:58
warweasle
You *could* implement lisp on top of forth and get something like you are thinking, but it's a very bare os.
15:04:59
on_ion
like they designed an implementation where the pieces cannot talk, warweasle ? =) there was just not that many people working on HURD
15:05:30
phoe
random-nick: AFAIK yes, except the user normally does not have raw access to memory like in TempleOS
15:05:48
beach
warweasle: You seem to be assuming a traditional OS with separate processes for each running program, requiring address remappings and hence communication by copying bytes between kernel and applications.
15:05:58
on_ion
people have been writing some interesting software for TempleOS. build it and they will come
15:07:14
random-nick
but user code would still be able to access all of the system's objects, right?
15:07:30
phoe
random-nick: depends if these are objects are present in that code's global environment
15:08:01
beach
warweasle: I think you need to read the design document. Otherwise, I am afraid I am going to end up re-typing it here in its entirety.
15:08:14
makomo
beach: i still don't understand how that would be done for a CL implementation written in CL. i'm ignoring the lisp OS thing for a now and focusing just on SICL being a CL implementation that i can run for example on linux. how would it implement these syscalls if it only has CL to work with? would you use some non-standard stuff from SBCL? would part of it be implemented in C or assembler? how would it work?
15:09:55
on_ion
warweasle: beach mentioned it is like Genera, but with safety and multi-user environment. if you dont know genera, how to continue the conversation ? i point this out because edging close to misunderstandings or misinterpreted idea[l]s
15:10:09
phoe
the basic means of communicating with it is executing the interrupt 0x80 CPU instruction
15:10:24
beach
makomo: SICL is (going to be) bootstrapped on a Common Lisp implementation. An executable file will be created. That file contains machine code. Some of that machine code will be the implementation of the function POSIX:OPEN. During bootstrapping, that function will be defined to execute the machine instruction for making a system call. That's all it takes.
15:10:27
phoe
which is the part of the libc syscall() instruction that actually passes control out of the process and into Linux.
15:11:00
phoe
either you link against linux's libc that contains syscall() and you call it, or you reimplement its functionality in whatever you want and then interrupt the OS manually.
15:11:38
makomo
beach: oh hmm. so the creation of this executable will be part of the bootstrapping process?
15:12:16
beach
warweasle: Unfortunately, most of the OS literature also assumes a Unix-like kernel. A better thing to look at would be Multics. And of course, Genera.
15:12:35
phoe
DOS is actually nice enough to allow you to erase it from memory and replace with anything else you'd like
15:12:53
beach
warweasle: So it is very likely that, despite your having read most of the existing OS literature, they haven't told you everything.
15:17:28
phoe
flip214: you know how the saying goes: if all you have is a DOS, everything looks like kernel space (:
15:18:41
loke
warweasle: Windows 3.1 was _not_ OK. I firmly believe you are wearing rose-coloured hindsight glasses.
15:22:29
beach
warweasle: Anyway, I am sorry that I am unable to communicate the essence of my ideas to you. Maybe in the future, I can improve my communication skills, probably in the form of more written documentation.
15:32:02
on_ion
lisp interpreted is like a genera, really; we just have more stuff on the "outside" of it, like device drivers, network management, etc.,
15:33:32
warweasle
I'm curious about vtables. Vtables in low level drivers are difficult. Especially when you can't control their lifetime and memory management directly.
15:35:23
phoe
your driver can be a Lisp function that grabs (fdefinition 'network-driver:do-some-stuff) and calls the resulting function object
15:36:42
beach
warweasle: CLOS typically does not use vtables for its implementation, in case you are referring to the kind of vtables often used to implement C++.
15:36:51
warweasle
So it would be like running lisp from DOS. It does everything including overwriting the interupt calls.
15:37:33
phoe
warweasle: again, running Lisp from DOS isn't very different from running Lisp without any underlying OS
15:38:21
phoe
the way you do low-level stuff doesn't really differ from the way you do high-level stuff in Lisp
15:38:32
warweasle
beach: How does it manually manage memory and resources in a garbage collected environment? Is it a separate interface?
15:38:40
random-nick
I still think that a lispos shares more similarities with TempleOS' architecture than with DOS'
15:39:39
warweasle
Don't get me wrong, I love lisp. I just don't see how it can be used at the embedded level.
15:40:27
phoe
warweasle: if you need some memory never to be released then you simply tell the GC that it should never free that thing
15:41:04
phoe
for example pin in it the active global environment: (defvar *very-important-device-something* ...)
15:41:48
beach
warweasle: Yes, it is called the memory manager, of which the garbage collector is an essential part.
15:43:37
warweasle
And the "class information" is not kept inside the code but rather outside it as "metadata".
15:44:24
warweasle
You don't have the interupt looking up the code to call. It just calls the compiled code directly.
15:45:14
beach
warweasle: You would very likely need a few lines of machine code to trampoline to a Common Lisp function.
15:45:31
phoe
there's no problem with creating a Lisp instance INTERRUPT-HANDLER, putting a reference to it inside some INTERRUPT-HANDLER-TABLE instance, and then call some function #'INSTALL-INTERRUPT-HANDLER-TABLE on that instance
15:46:12
warweasle
beach: About that...Even in C you don't have time to do any processing. You just post a message to act on later.
15:47:55
phoe
#'INSTALL-INTERRUPT-HANDLER-TABLE could just as well compile that instance into an actual fixed jump table, so a bunch of JMPs in memory
15:48:14
warweasle
But it's all lisp all the way down but without the typical protection lisp give you.
15:48:50
warweasle
Memory management, garbage collection, thread protection, etc. At least at the low level.
15:49:55
phoe
except Lisp gives you some more things like a sane type system that helps you actually not commit a crashing mistake
15:50:05
on_ion
i was not referring to compiler v. interpreter, not part of current context of topic, but implementation v. implmntation
15:50:35
phoe
warweasle: also "Swapping out function calls as you change the functions" is already being done in implementations
15:51:27
phoe
but redefining functions at runtime already happens well enough in multithreaded implementations
15:51:52
warweasle
OK...now I'm tracking. It's almost like having two lisps, a high level and low level.
15:53:32
phoe
the best way of not having direct acccess to memory is not having direct access to memory
15:53:44
beach
warweasle: Just as in Common Lisp, you can't access an arbitrary byte of memory, you couldn't do it in the LispOS I am planning either.
15:54:10
beach
warweasle: But it is not because there is a kernel. It is because you are simply not allowed to write code that accesses memory directly.
15:54:16
phoe
if your Lisp REPL does not give you access to any stuff like dereference-pointer or such, you simply can't do it
15:54:31
phoe
it's hard to dereference a pointer if there's no pointer and there's no dereferencing API
16:01:29
beach
nirved: Again, I seriously doubt that it would be widely adopted. I am a researcher, and I want to show that we can do better than we have been doing for the past half a century or so. If Nvidia does not buy that, then there is nothing I can do about it. As long as I can show that IF NVIDIA HAD ACCEPTED IT, then we would be better off, I am satisfied.
16:03:59
beach
nirved: I am one single person, and I can't possibly rewrite all the applications I use on a daily basis in Common Lisp so that I can ditch my GNU/Linux system. So for the foreseeable future, I would have to use both systems simultaneously. One way to do that is to run the LispOS as a Linux process. Another possibility would be to run it in some virtualizer.
16:04:54
nirved
beach: i mean, will there be a way to extend it, supporting different kind of devices
16:05:15
warweasle
beach: I might have to write embedded code in the future. So that's why I'm asking so many questions.
16:05:25
beach
Because of first-class global environments, there could very well be one environment, inaccessible to the ordinary user (would require "system" privileges) that would be able to access memory directly.
16:05:26
beach
But that functionality would not be accessible to ordinary users in "user" mode. All of that is totally unimportant for the overall safety of the system. It still has nothing to do with a kernel. Such a privileged environment might also be
16:06:26
warweasle
But parts of it might be useful. You never know what might give you an idea that simplifies a project immensely.
16:06:42
beach
... might also be used for compiler-backend stuff that should not be accessible to the application programmer.
16:07:23
nirved
beach: i looked a bit at the GC, there's Staccato, don't know if it's relevant to lisp: https://researcher.watson.ibm.com/researcher/files/us-groved/rc24504.pdf
16:08:04
warweasle
*IF* I could bootstrap an implementation of lisp onto the machine, then I might have a lot of fun poking the device.
16:08:44
beach
nirved: But GC is very complicated in terms of predicting performance, so it's too early to determine whether my plan will work out.
16:16:09
on_ion
beach: nope sorry, just wondering if this was all common lisp, which would need a lot more than bare metal core runtime
16:16:11
jackdaniel
if we insist it is CL-only channel (not lisp-in-general-related-channel), then it might be a case
16:17:16
jackdaniel
beach: ffi is part of *existing* implementations, but whatever. if people around are happy I have nothing against. just dropped my 2 cents
16:18:12
on_ion
dlowe: well, wait are you really asking? how could someone run CL without a CL implementation ?
16:18:40
beach
on_ion: You would build an executable on a different system with an existing Common Lisp implementation.
16:18:57
on_ion
yes but, you still need that CL implementation alive and kicking to have a lisp system, no?
16:20:11
jackdaniel
on_ion: same comes to any program (except ones writtenn in hand machine code) – you need kicking C compiler to build C compiler (or Linux kernel)
16:20:34
random-nick
there are already CL implementations like SBCL and CMUCL that bootstrap from an existing implementation
16:20:36
White_Flame
if you need an x86-hosted C compiler to generate an ARM program, does that mean the program has a requirement of x86? not really
16:20:38
on_ion
jackdaniel: a CL system requires the full CL system to be a CL system, no? C executables dont need C compilers
16:21:12
on_ion
a running lisp system needs a living lisp VM and all the things, no? am i missing something?
16:21:46
jackdaniel
on_ion: I said that you need C compiler to build C compiler for a new platform for that instance. I think you are a bit confused with a few different (though slightly related) topics
16:22:23
warweasle
beach: But I used them to keep change logs and to remotely access them using UUIDs.
16:23:38
warweasle
beach: Sort of a "self describing" data, but with a few basic interfaces for text, rich text, image, vector, sound, etc.
16:25:36
on_ion
even if one could gut out the things one does not want/need, to get CL running on bare, and work on one's own OS ideas from there
16:26:12
on_ion
is there a plain/vanilla bare-metal CL implementation that all the OS experiments could use ?
16:27:01
beach
on_ion: I would much rather start with a Common Lisp system NOT running on bare metal but that I can run with proper debugging support, etc.
16:27:51
on_ion
beach: true, im not knowledgeable in their abilities. i know that NASA had some lisp thing going on with a rocket which definately was bare metal
16:30:04
White_Flame
OSes generally "accept" a binary footprint of machine code & data with start address & links to system stuff
16:30:41
on_ion
White_Flame: yeah. so im just wondering why there is a llvm->CL transpiler if one cannot simply execute the llvm like on a regular OS
16:31:23
on_ion
im curious because beach mentioned security also , which i am interpreting a bit like a CL sandbox
16:33:56
Bike
I think beach's particular design doesn't really allow arbitrary binaries in the same way.
16:35:59
on_ion
fasl is what is emitted and loaded after sbcl compiles lisp, right ? same practical purpose as if it were a "CL bytecode" ?
16:36:47
White_Flame
basically, what you're asking is the equivalent of running in random patterns in a vocabulary minefield ;)
16:36:52
Bike
sbcl fasl files contain machine code, not bytecode. but yes, it's the output when you compile a file.
16:38:07
jackdaniel
for instance ECL's "FASL" files are in fact shared objects (as *.so libraries) on Linux
16:38:55
beach
on_ion: Traditional code that uses the full address space would have to be run in a separate process, presenting itself as a single Common Lisp function.
16:38:56
beach
Such code could not directly access Common Lisp objects in the system, but they would have to use something similar to file descriptors. That would be just as inconvenient as it is on Unix systems now, so it would probably be uncommon.
16:39:20
froggey
beach: no, I intend to use an in-image filesystem as the primary fs. there's a basic proof of concept already (the local filesystem), but I haven't really nailed down any details
16:42:13
beach
froggey: It would be great if you could plan to come to ELS in Genoa. We could chat about things like this.
16:44:00
froggey
my current focus is getting the mezzano backend for mcclim into an acceptable state for demo 4
16:44:09
skidd0
Hey guys. I'm attempting to build a small command line program on Linux in Common Lisp. Does anyone have experience with distributing small lisp programs? And, any favorite libraries for handling command line args? There seems to be a lot of options out there for both questions, so, I'm looking for community input
16:45:09
White_Flame
skidd0: I've deployed really small initialization utilities as standalone executables. They're obviously still laughably large in filesize for what they do, but it's nice to include all the libraries & lisp knowledge that the main application uses
16:45:47
rpg
skidd0: I have heard different things about how easy it is to move buildapp apps around. I believe that there can be issues with shared libraries when you move them, but I'm afraid I do not understand these issues myself\
16:46:01
froggey
on_ion: that's a bit of a cheat, it's telnetting to the nethack server on nethack.alt.org
16:46:28
skidd0
well so has anyone build 'apps' that are easy for other (non-lispers) to download and then execute/use?
16:46:45
jackdaniel
rpg: basically when you dump sbcl image with foreign libraries loaded, it tries to load them from the very same location when you start the application
16:46:56
jackdaniel
so in order to avoid that you need to close all shared libraries before dumping image
16:47:45
rpg
jackdaniel: I have never built anything with FFI that uses buildapp, so that shouldn't be a problem for me. I was wondering if there were any shared libraries that SBCL itself would load.
16:47:57
froggey
there's no support for running arbitrary native code though, everything must conform to the mezzano abi for the GC
16:48:28
jackdaniel
only those which you load yourself. the trick is that it assumes that libraries loaded when you dump image will be at the exactly same location
16:48:59
froggey
on_ion: yes, just basic framebuffer support right now. the video mode is set by the bootloader & not changed after that
16:49:51
jackdaniel
rpg: (of course that applies to the libraries your dependencies load). also there is a trick with cl+ssl – you need to call also "reload" function, because some structures must be initialized in the foreign library
17:10:18
White_Flame
you need to figure out what sort of file structure it wants around it, or if it's purely self contained
17:20:27
dlowe
White_Flame: it kind of is, because our lisp systems don't use libdl to automatically link all the shared library code back up
17:21:03
White_Flame
and yeah, it's a more complex space than windows or osx might be for these particular points
17:22:13
skidd0
man the more i dig into programming, the more i realize how pointless paying tuition is
17:23:50
White_Flame
programming is still a craft. there's no major thread of academics that teaches it
17:24:06
dlowe
I dropped out of college and when I compare my career with my programming peers that stuck with it, it was *much* harder for me. Those fundamentals are great to learn up front.
17:24:07
White_Flame
cs = the "science" of computation. ce = building hardware. software engineering = project management
17:25:09
White_Flame
dlowe: yeah, I'm in the same boat (though I didn't willingly drop out; I got "financial aid"ed out of school :-P)
17:27:17
White_Flame
skidd0: if you want to get out of school, then push hard on actually getting industry contacts and job offers
17:28:04
White_Flame
but when it comes to security, I think the uni papers are still a barrier to entry
17:29:50
jackdaniel
ACTION thinks this discussion steadily moves offtopic (that is in #lispcafe direction;)
17:51:22
White_Flame
it's always best to just use the default facilities first and get a hang of what's going on and what needs to be done
17:51:34
White_Flame
then you can make better decisions on what features you need if you actually do need something more complex/bundled
17:52:05
Xach
phoe: i have two hard drives and a fast computer running them but i have been away from home for a week so there has been no progress. i am optimistic i can build something soon. i really would like to get my old hard drive recovered though. that might involve shipping it somewhere.
17:53:55
Xach
I was once very disciplined about backups. I have been punished for backsliding into laxity.
18:21:27
on_ion
i sometimes wish that we had something like ChangeSets for lisp images, like squeak has for its smalltalk images
19:42:02
on_ion
phoe: er, already mentioned those earlier. intent of comparison to squeak smalltalk ChangeSets, those are portable across implementations and machines. they are essentially source code diffs.
20:02:29
pjb
Most often you cannot apply such patch, even on the same implementation and machine, when you try to apply them on a different version.
20:21:38
jmercouris
I'm reading this article here: http://joaotavora.github.io/sly/#A-SLY-tour-for-SLIME-users about SLY
20:21:49
jmercouris
and I'm wondering, any SLY users in here? can you tell me why you prefer it to Slime?
20:25:38
jmercouris
I don't see anything that can't also be done in slime, but maybe I'm just missing some things
20:27:44
Bike
this intro is starting with a bunch of stuff slime doesn't have, so maybe i'm misunderstanding you
20:29:24
Bike
i doubt sly is going for some kind of amazing transcendental experience, just better than slime
20:44:12
Xach
I remember I didn't want to try slime because i liked ILISP. But actually using it showed me how wrong I was. Even though at a high level they do the same things.
20:49:41
Xach
The slime video helped a ton in persuading me to try it. Reading static pages and screenshots, and using your imagination, is not always persuasive
21:02:34
dim
the cost of changing one's habit is very high, and slime is pretty good at what it does already
21:19:22
attila_lendvai
slime has most of the cool stuff stuffed away in contribs, e.g. the fuzzy completion
0:42:40
mfiano
The benefit of Sly is builtin mrepl, stickers, everything being a button to inspect, amoung many improvements and bug fixes.
0:44:11
mfiano
as of a few days ago, even builtin company completion with zero configuration that is much snappier than the old sly-company, or slime-company
0:45:01
mfiano
I switched from SLIME to Sly about 3 years ago and I'm still happy with that choice whenever I am reminded by using SLIME
2:00:13
kuwze
Xach: does this still work for you? http://lispblog.xach.com/post/112939066338/using-paredit-within-screen