freenode/#lisp - IRC Chatlog
Search
12:49:13
Xach
Hmm, I wonder if there's an existing easy way to "refactor" the frequent use of a complex accessor into a local variable in emacs.
13:03:24
Xach
I don't think I need a macro at all. I want to bind a value instead of computing it over and over.
13:04:16
Xach
schweers: in the near future archive integrity will be checked by signature and digest. but the composition of the archive is still a weak point.
13:05:06
Xach
there will be confidence that code comes from the expected place, but it might be bad code.
13:06:04
Xach
Anyway, I can do what I want fairly easily enough manually, with a combination of paredit, some typing, and query-replace.
13:09:35
_death
Xach: I don't use redshank, but I'd likely do something like that with multiple-cursors
14:03:08
epony
English is not my native language, now it tries to be. I always forget one word and have trouble remembering it, there is something wrong with it. Endorse is heard from marketing leaflets waving salesmen with a previous career. Thank you for reminding it to me, verisimilitude.
14:33:43
shrdlu68
beach: Your LispOS paper is thought-provoking. Something has to play the role of a kernel, right?
14:38:03
phoe
as in, the paper AFAIR does not concern itself with scheduling/init/devices, so it can be done in any suitable manner
14:39:05
beach
shrdlu68: Yes, there will be a collection of functions and such for dealing with that.
14:40:50
beach
dlowe: There is very likely going to have to be a few lines of assembly-like code here and there. Sure.
14:43:44
Shinmera
There is an environment, of course, but not a separate entity that acts as the kernel.
14:44:05
beach
I am thinking of taking inspiration from I/O Kit from Apple for device drivers, except using CLOS instead of C++ (of course).
14:44:09
phoe
shrdlu68: as far as I understand beach's reasoning here, there's *something* that does that. What exactly it is and what terminology is used to describe it is an implementation detail.
14:45:33
beach
shrdlu68: Think a Common Lisp system where some of the code is dedicated to managing devices, scheduling threads, etc.
14:47:34
beach
In Multics, since every program was contained in a segment, the "kernel" was just a collection of segments. You could replace any such segment without rebooting, and every program would instantly see the difference. A device driver could be replaced for instance, or the code for some system call.
14:47:43
shrdlu68
So the Linux kernel is a collection of OS functions and objects, plus some other thing{,s}.
14:50:20
beach
Though in the Lisp OS I am thinking of, since most applications would not be able to manipulate raw addresses, there is no particular need to make them execute in a non-privileged state.
14:50:51
beach
Applications written in C using the traditional technique would still have to execute unprivileged.
14:53:35
beach
verisimi`: SICL is different in that it uses more modern programming techniques than most existing implementations, simply because most implementations were written before CLOS existed. And the idea is to make the code simpler, so therefore easier to maintain, than existing systems. This property, I hope, will encourage improvements and optimizations that would be difficult in existing implementations.
14:53:44
TMA
beach: it is not entirely accurate. you can change any system call code at runtime, it is a matter of swapping in a different function pointer to the syscall dispatch table. It is just not pleasant and there are problems with C not having a usable notion of CHANGE-CLASS, so you are basically stuck with the data structures you already have
14:54:28
shrdlu68
beach: But how do you tell a program is capable of manipulating raw addresses or not?
14:54:58
beach
shrdlu68: Such a program would have to become a particular subclass of FUNCALLABLE-STANDARD-OBJECT.
14:55:40
beach
shrdlu68: And when such an object is called, it would remap the address space. Something that most functions would not need to do.
14:58:11
shrdlu68
The clearest difference between the kernel and the not-kernel to me at this point is that a not-kernel does not a need a reboot.
14:58:47
beach
shrdlu68: In Common Lisp you can't take some arbitrary instructions and create a callable function out of them. You have to go through COMPILE, or equivalent. In Lisp OS, to turn some C code into such a program, you would need some other similar compiler function and it would create the kind of object that remaps the address space.
15:00:09
beach
When I hear or use "kernel", I think "monolithic piece of code created by a 1960s linker and that must be replaced entirely in order to update even a single instruction".
15:01:37
shrdlu68
beach: Say I have an arbitrary binary that I want to run on your OS, what's the sequence of events from executing it to seeing a "Hello, world!" printed on the screen?
15:03:26
TMA
I use "kernel" for elevated privileges code (namely when the privilege elevation is hardware assisted)
15:05:21
beach
And it's a perfectly valid use of the word. Just not very useful for the Lisp OS I am thinking of.
15:06:18
TMA
sure, it has to go through something like COMPILE to grant the privileges of executing the unsafe parts
15:06:28
beach
It would have to be compiled with a special compiler to create a special subclass of funcallable-standard-object.
15:13:38
TMA
it becomes problematic when untrusted code can request compilation via the special compiler -- or at least that is my gut feeling
15:15:31
TMA
the whole purpose of the special compiler is that it allows marking some potentially unsafe code (code that uses raw addresses) as executable
15:16:30
beach
No, the purpose is to create a Common Lisp function that, when executed, changes the address mapping so that it can't do anything other than what a standard Unix program can do.
15:16:38
shrdlu68
The way I understand it, the system simply does not understand unsafe code. It does not know how to run unsafe code.
15:17:36
beach
And it would have to manipulate Common Lisp objects though "file descriptors", i.e. small integers.
15:18:33
TMA
oh, I have messed up the one that allows to patch the lisp-os's parts involving inline assembly/raw address with the C-programs enabling one
15:18:38
beach
I don't quite understand why the focus here is on something that I consider totally exceptional, that I would rather not see at all, but that MIGHT be required in SOME VERY unique cases.
15:19:26
beach
Patching the Lisp OS would be the same as patching a Common Lisp program, i.e. a function would be replaced, or a class would be redefined, etc.
15:19:55
beach
shrdlu68: I think I am entitled to my perspective since I am the author of the specification.
15:22:11
beach
It wouldn't be a matter of something that somehow is impossible to do the "standard" way.
15:22:30
beach
Just that someone would be lazy enough to want to incorporate C code rather than writing the equivalent Common Lisp code.
15:23:44
shrdlu68
I think I get it now. It really is a Lisp OS. Not just an OS written in Lisp, but for running Lisp.
15:25:07
beach
First sentence of the intro in fact: A Lisp Operating System (LispOS for short) is not just another operating system that happens to be written in Lisp...
15:27:39
shrdlu68
It's like writing a javascript OS, and some fella comes along with his Lisp code...
15:29:00
beach
shrdlu68: That situation is no different from running Lisp on the "C operating system" called Unix.
15:30:30
beach
shrdlu68: It is a research project. I am fortunate not to have to care about things like that. It is a project where I attempt to show how much better things could be than what they are. If people don't want better things, there is not much I can do, and it won't say anything about the validity of the project.
15:32:47
beach
Exactly, and that's the reason for most of the security problems we are faced with today.
15:33:43
beach
... which apparently don't work anyway, and only create problems for application writers.
15:36:59
TMA
the security problems are that there is so much code in existence you cannot be expected to audit it before running it compounded with that the people do not behave nicely to each others
15:37:02
loke
jmercouris: If your code can never arbitrarily access data by pointer, then it's not needed.
15:37:32
epony
"If you find that you're spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you find that you're spending almost all your time on practice, start turning some attention to theoretical things; it will improve your practice."
15:38:00
jmercouris
Yeah, I just skimmed a little bit above, if this conversation goes deeper and I missed a lot, sorry
15:38:10
loke
jmercouris: Whose code would that be? In Beach's OS, only the low level kernel code will have that capability (and even then, a very tiny bit of code since the OS itself is written in Lisp).
15:38:10
beach
jmercouris: The entire argument is that it is absurd because it fixes a problem that should have been fixed differently.
15:38:38
jmercouris
if fixing it correctly means making a new OS, I think that is a cost very few are willing to bear
15:38:45
epony
I agree with beach, the problem should have been prevented instead of sorted that way.
15:39:22
jmercouris
loke: Not only low level code would have that ability, there are ways, especially if you have physical access to the hardware
15:39:34
beach
jmercouris: You apparently also missed my saying that this is a research project, so there is no real "cost" involved.
15:40:19
jmercouris
I understand the time/place of your project, and I'm fully in agreement with you
15:40:22
beach
jmercouris: And you missed that most code in the OS I am thinking of would not have physical access to the hardware.
15:40:28
loke
jmercouris: sure. you could shut the machine down and reboot Linux or something. I'm putting words in beach's mounth here, but I'm pretty sure the project does not intend to prevent that.
15:42:23
beach
jmercouris: And you missed that most code in the OS I am thinking of would not have physical access to the hardware.
15:43:41
epony
jmercouris the machine is a machine code interpreter, how you give the machine the operands is irrelevant given the machine runs and not halts
15:43:51
loke
beach: Are you going to use some microkernel or something as foundational base, or will you be writing the lowest level code yourself?
15:44:31
jmercouris
I imagine it will have to be a microkernel, it makes a lot of sense anyway with the message passing
15:45:13
jmercouris
epony: I understand your sentence, I don't understand your meaning within the context of this discussion
15:45:44
shrdlu68
beach: Isn't there a LispOS similar to what you're proposing already in existence?
15:47:00
jmercouris
I think Lisp OS were not as beach describes, just OS' with function calls and paired with hardware optimized for common lisp operations
15:47:59
jcowan
One thing conspicuously missing from the protection scheme is any way to protect the user from themself.
15:48:04
loke
I wondering if Redox is trying to approach security in some similar way... (there are huge differences of cours,e but the fundamental idea that software should not be allowed to do the wrong thing seems similar)
15:48:25
jcowan
When everything is mutable and persistent, how are you to stop a brain fart from permanently ruining your system?
15:49:09
jmercouris
I was joking firstly, but sure, you could write conditions for everything you are afraid of
15:49:12
beach
jcowan: That's a very good question. In the worst case, the global environment would have to be ditched.
15:50:28
beach
So here we go again. I suggest something that is way better than anything we have, but it is not accepted because I can't solve problems that no other operating system can solve today.
15:51:03
jcowan
Well, one of the things it loses is the ability to throw away a local world and start again, because there is only a global world.
15:52:17
jcowan
You make global Lisp environments explicit, but that's not the same as a whole world, which is more like a VM today.
15:52:40
epony
I would not consider arguments from other environments is a decision to not accept something better, but difficulty understanding the foundation.
15:54:46
shrdlu68
jmercouris: It seems to me that would essentially the same as beach's OS, only easier to implement.
15:54:50
jcowan
To modify a heavily entrenched system, you have to have something that is not only better at certain tasks, but better as a whole. Otherwise, it will always be cheaper to reform the existing system. The way of the revolutionary is very hard.
15:55:23
beach
One idea that I have, and that I plan to implement in Second Climacs is to have an implementation of the first-class global environment protocol that allows for incrementally updated environments. Then most things can be undone by discarding selective increments.
15:55:40
jmercouris
shrdlu68: The outcome would be effectively the same, but the security considersations would be very different, you'd still have the Linux kernel running
15:56:28
beach
jcowan: I have absolutely no illusion that people would be willing to change their habits and ditch their existing systems.
15:56:48
shrdlu68
Yes. But the init (which we've replaced with a Lisp interpreter) would be the only thing interfacing with the kernel.
15:58:06
loke
shrdlu68: Using Linux would be overkill though, since the lispos wouldn't need processes for example. You'll just need threads.
15:58:12
jmercouris
shrdlu68: the kernel itself is a massive security vulnerability, anyone who cares about security is NOT running linux
15:58:34
jmercouris
loke: yes, but using the Linux kernel would give you access to a wealth of drivers
15:58:39
loke
shrdlu68: So if you want to take an existing kernel, you'll be better off with L4 or something like that.
15:58:43
jcowan
Anyone who really cares about security has abandoned the use of computers altogether.
15:59:09
jcowan
Throw them all down a big hole in the ground and fill the hole with cement. Then they will truly be secure.
15:59:17
loke
jmercouris: I care about security and I'm not using Openbsd. I'm writing this from my Qubes OS laptop.
15:59:58
shrdlu68
jmercouris: But, in the system I propose, no one else is making syscalls except the Lisp interpreter.
16:00:21
jmercouris
shrdlu68: that's not the point, there are vulnerabilities within the kernel that do not require operations in userland, things that the kernel itself handles
16:00:37
jcowan
Scheme has global environments that are *almost* first class, though most Scheme programmers never give them a thought.
16:02:23
jcowan
The current big limitation is that there is no standardized way to create a novel mutable environment.
16:02:32
shrdlu68
jmercouris: Any OS has to contend with those kinds of risks, though, even a Lisp OS. At least some of it will be implemented in assembly.
16:04:33
jmercouris
No doubt, there is no risk-free OS, I'm just pointing out that bringing in the Linux kernel brings a huge red target into a proejct
16:04:57
jcowan
https://bitbucket.org/cowan/r7rs-wg1-infra/src/default/MutableEnvironmentsCurtisCowan.md <-- proposal for mutable environments
16:04:57
jmercouris
"security through obscurity" is not an accepted practice, but it works pretty well, until it doesn't :P
16:08:22
epony
jmercouris What is your current operating system? Do all operating systems have to be one and the same method, only slightly off in their implementation?
16:12:06
jcowan
beach: I also read the Cleavir manual. I was surprised to find no primops that do allocation. How does that work?
16:12:32
jackdaniel
shrdlu68: there is nothing heretic with implementing device drivers with C - why CL would be any different?
16:13:53
beach
shrdlu68: I am seriously interested how you came to this conclusion, because you are not alone. And this "urban myth" is problematic to the entire Common Lisp community.
16:16:10
jackdaniel
no, I meant heretic - it is not heresy to implement device drivers in languages not being assembler
16:16:47
beach
little_lisper: I am not sure they understood what you meant by "not gets any input". You might need to explain that.
16:16:59
jackdaniel
I'm not sure how well such device drivers would play with GC (some have very tight time constraints) etc
16:18:01
jmercouris
I imagine something like a network driver would be fine, I can't imagine a graphical driver being that fast
16:19:50
jmercouris
jackdaniel: I see, thank you for sharing that, I am not so familiar with driver implementation
16:20:33
jackdaniel
that may be a good hint, that stating bold opinions about performance may be risky
16:21:20
jmercouris
Well, I know some, and I venture I know enough to know why drivers are usually not written in Scala or Python
16:21:53
pierpa
And even if it wasn't all done in hardware nowadays, it was already done in software 30 years ago in the lisp machines
16:22:32
jcowan
There would be no problem in writing a kernel in Ada, and in fact that is the kind of thing Ada was designed for (real-time, bare metal)
16:24:10
froggey
probably. I don't really have any set objectives, I'm just doing things that seem interesting to me
16:24:37
LdBeth
jmercouris: as long as the final product is machine code, it’s okay to use any languages that could improve development efficiency
16:24:39
jackdaniel
jmercouris: in case of Linux - ostensibly yes, in case of Windows - definetely yes, in case of minix - if you mimic C abi you may write it in sbcl why not, other unixes - yes
16:24:58
beach
froggey: If you feel like it, you can read my specification. If not, that's fine too.
16:25:14
little_lisper
thanks to bring it up beach. i have a function which takes arbitary no of args. so i made a fn 'readlist' using read and cons. i want read to return nil if it does get any input i.e. no enter key press. i am a beginner
16:27:22
jmercouris
I've been googling this throughout the course of this conversation, and the arguments are all over the map actually
16:27:46
jmercouris
some people say "because that's how it has always been", some say "it is fast", some say "because that is what the kernel is written in"
16:28:17
jmercouris
I think the lack of a clear consensus means that there must be a multitude of reasons
16:28:53
LdBeth
jmercouris: it’s actually nothing to do with run time efficiency, it’s just because Unix is popular and so does C
16:38:19
froggey
jmercouris: in my experience, drivers in CL perform well enough. the biggest sticking point I've found is interaction between the GC & audio playback. if the audio driver doesn't refill the output buffer in time it causes stuttering
16:40:00
LdBeth
So, my plan is an assembler first, and a non-tradition GC subset of Lisp aims only for high performance, and then layers of Common Lisp for user space applications.
16:41:34
verisimilitude
The C language is used for operating systems because people don't know better and believe others who don't know better.
16:42:01
Xach
seemz: froggey wrote mezzano, which is a lisp os. perhaps that's what informs his perspective.
16:45:46
jackdaniel
maybe that's because I don't know better, but I find Unix philosophy (and implementaitons) decent and very usable
16:46:29
Xach
jackdaniel: the unix-haters handbook is pretty funny and written partly from lisp-machine users' point of view.
16:46:57
verisimilitude
You may as well claim that urinating follows the, say, jackdaniel philosophy; look at how popular and successful your philosophy is, jackdaniel.
16:50:36
jeosol
Anyone here running a CL application on AWS, if so how do you package things up there? docker? After power outage, I am thinking of starting move bits of my workflow to AWS or other options
16:51:01
jeosol
I tried installing SBCL, wine, etc, on an AMI instance, it was a pain doing this one by one
16:51:31
jackdaniel
jeosol: fpm for packaging, clon for binaries delivery and hand-written systemd scripts for daemoning
16:52:09
jeosol
after managing to install sbcl, i figured there as to be a better way of doing this.
16:53:44
jackdaniel
verisimilitude: I disliked you have used eristic 'argument', and yes, I disliked you have used my name in your 'example'
16:55:28
jeosol
ok, I searched google, and a fpm github showed up, no worries, i think he just titled his .README that
16:55:29
verisimilitude
I can be quite eristic when I want, jackdaniel; anyway, I'll avoid using your name, then.
17:01:43
ecraven
what's a good way to pretty-print lisp code into an epub or a black-and-white pdf for reading on an e-reader?
17:04:24
ecraven
the main thing is somehow fontifying strings and stuff, otherwise a plain text file would suffice
17:06:03
froggey
jmercouris: for performance, and then only really for things with tight time-constraints like audio
17:09:00
froggey
interrupts are another issue because the handlers run in a constrained environment (no allocation, no touching memory that might be paged out)
17:09:32
froggey
but this isn't a big deal because interrupt handler generally don't do much. usually they just acknowledge the interrupt on the hardware and wake up a worker thread to do the real work
17:15:10
epony
(regarding device drivers, I think earlier it was mentioned that the target is Common Lisp)
18:40:29
makomo
the relevant part is: `GNU emacs used to greet Symbolics users with the message "In doing business with Symbolics, you are rewarding a wrong."` :-)
19:11:36
_death
https://github.com/emacs-mirror/emacs/search?utf8=%E2%9C%93&q=symbolics&type= doesn't find it, so maybe before apr 1985
19:12:07
_death
does have "Symbolics killed the MIT AI lab; don't do business with them." in the changelog
19:32:31
phoe
Postmodern returns table names as keywords but column names as strings. Is it possible to have it return keywords instead?
20:07:22
phoe
Which ironclad digest should I use for hashing passwords? I don't want to go for SHA1 since it smells insecure to me as of late.
20:08:44
phoe
The current bit of code I have is https://plaster.tymoon.eu/view/754#754 - please bash the hell out of it if you see anything wrong with it.
21:37:00
jcowan
beach: I have just read the chapter on data representation in the SICL manual, and I would urge you to seriously consider the possibilities of nanboxing on 64-bit systems.