freenode/#lisp - IRC Chatlog
Search
9:52:10
beach
I recently learned that I can't map page 0 because there might be bugs in the OS that dereference NULL and the C compiler does not protect against that by (say) inserting a test.
9:59:28
p_l
and it's less OS bugs and more tricks with application bugs to get them to jump to known address
9:59:43
beach
Yes, so if I distribute SICL, I have to ask every Linux user to reconfigure their systems, thereby making them vulnerable to OS bugs.
10:00:47
beach
So we keep adding kludges on top of hacks to avoid solving the fundamental problems, thereby making life harder for both developers and users.
10:02:11
beach
If I represent NIL as a the machine number 5 and store NIL at address 0, they can't use my system unless they reconfigure.
10:05:30
beach
So now I am in a situation where I must use a 32-bit immediate for NIL, because it can't be stored in the first few pages, and if i want to be able to run the implementation on a system with ASLR, then I also can't have pre-defined data structures in the executable.
10:05:56
no-defun-allowed
https://utcc.utoronto.ca/~cks/space/blog/linux/KernelPageZeroProblem describes it slightly better; the kernel is mapped into the process's address space, and so kernel code could dereference NULL and act weirdly.
10:06:18
beach
So I am basically forced to either have the system behave like a C++ application with ctors, or I am forced to do a full GC at the beginning of the execution, or possibly use relative pointers everywhere. Everything has a cost.
10:31:51
beach
LdBeth: I suspect not. I looked at how Emacs dumped memory to an executable a long time ago, but I think they changed it since. Why do you ask?
10:32:32
no-defun-allowed
"[kdeinit] executes KDE programs and kdeinit loadable modules (KLMs) starting them more efficiently." That's more terrible than I felt while waiting for at least two minutes for KDE4 to load on my old computer.
10:38:30
LdBeth
beach: https://dancol.org/pdumperpres.pdf Indeed, their new solution works on ASLR system
10:39:23
no-defun-allowed
Can you dump an image from a cross-compilation where one implementation compiles another using that method?
10:41:06
LdBeth
Emacs now does not directly mapping data into memory, at the cost of a little overhead
10:43:17
pjb
Let's note that C NULL pointers don't have to be represented by a #b0 bit pattern. At the level of the C source, casting it into a intptr_t should give you 0, and (void*)0 should give you a NULL pointer, but the binary representation of (void*)0 and (intptr_t)0 can be different.
10:43:23
no-defun-allowed
It's certainly useful, but in the "general" case dumping an image doesn't sound easy.
10:44:26
no-defun-allowed
Hm, I think Zeta C/Symbolics C had an odd representation for NULL (which was NIL?) as pointers were arrays of some kind.
10:44:55
no-defun-allowed
pjb: https://stackoverflow.com/questions/2597142/when-was-the-null-macro-not-0
10:45:21
beach
LdBeth: Thanks for the information. That is a terrible kludge that should not have been necessary in the first place.
10:45:50
pjb
beach: there are other reasons why you might not be able to use page 0 (or others pages with small addresses). Sometimes interruption vectors are stored there. Sometimes these addresses are used for fast addressing so the pages are used as temporary registers. And of course, the kernel can memory map them out of the user process reach anyways.
10:46:54
p_l
no-defun-allowed: all "plain" memoy in Symbolics C was a big array that was multiply mapped as different types
10:48:31
beach
So I guess if you take ASLR into account, there is no chance that NIL could be a fixed constant, let alone an 8-bit machine value.
10:54:54
beach
p_l: The problem is not the architecture. The problem is that we insist on programming as if we have access to the raw computer, just as we did in 60 years ago.
10:59:14
LdBeth
There’re lot of tools I can name written using specifically designed languages so they can translated to different architecture and assembled with macro
11:14:20
p_l
beach: making it explicit that you don't have access to raw machine requires either a special machine, or disregard for security. And we no longer can do the latter, niot really
11:15:11
p_l
even AS/400, despite using "special intermediary language" route, implements portion of the necessary stuff in actual physical machine
11:21:19
White_Flame
one of the big historical mistakes of computing was using C as an application langauge
11:21:58
p_l
and C is very much application language, you can't write an OS in it without compiler extensions
11:23:12
White_Flame
but that should be contained in the systems dev stuff, not exposed to the application dev
11:26:15
jackdaniel
I'm sure many people could raise a concern, that while C is indeed part of the topic, it lacks the L suffix :-)
11:29:26
no-defun-allowed
p_l: Clearly you don't use my compiler, where undefined behaviour causes unicorns to shoot out of the screen.
11:55:16
p_l
beach: AS/400 solved security issues involved in having single address space by making a much fine grained control (on hw level) on access to memory locations, with more flexibility than simplistic supervisor/usermode
11:55:57
p_l
as for specific symbol for NIL, it means that a check for NIL doesn't have to dereference memory, nor do you need any special memory location, which is problematic with ASLR and others
11:57:01
p_l
beach: by providing more advanced security measures in hw, the intermediate level "bytecode" could exploit such features for efficient *and* secure implementation
11:57:25
p_l
as for symbols - I honestly believe we have way more cases of NIL-checking than any other, and EQ can just compare the address usually
11:58:10
beach
p_l: I am suggesting making direct access to memory locations impossible. I don't see how that requires special hardware, and I don't see how it would be a security risk.
11:59:21
beach
p_l: Yes, ASLR is a problem for comparing to a constant. So, if ASLR was your argument, I agree.
12:00:14
beach
p_l: But the disadvantages of a separate tag would result in a maintenance nightmare, and also you would need two tests for each iteration in a loop over a list.
12:00:55
beach
p_l: Whereas, even with ASLR, you just have to implement EQ by comparing two lexical locations.
12:00:58
p_l
also, I kinda don't trust "direct access to memory locations is impossible", given several times people escaped from exactly that kind of sandboxing - maybe you'll find a better method to implement it though (I hope)
12:07:00
beach
Also, I would give some more thought to my idea if you could provide some more substance as to what technique would be used to access memory if no operation is provided to do so. More than your distrust, I mean.
12:07:22
beach
I guess the two tests in the loop could be avoided if you don't do it the way SBCL does.
12:12:02
p_l
beach: I'll dig out some details later, but generally there's been a plenty done in JS space - you might be able to avoid them by using a better language for implementation though, so I'm eagerly awaiting more on SICL :)
12:13:06
Shinmera
People have managed to program and launch entirely new games just by pressing the right inputs in Super Mario World
12:15:33
pjb
White_Flame: note that using C for applications is not really the fault of the creators of unix: they provided a fortran compiler!!!
12:17:35
pjb
p_l: you've got an implicit conversion from into to int*; you get a message from the compiler (and if you insist, a segfault when running it).
12:19:51
pjb
As for NIL, I don't think there would be any win today to handle it any differently than any other CL symbol.
12:20:44
pfdietz63
Archaic unix ran in a very constrained environment. It's amazing it (or its C compiler) worked at all in a 16 bit address space.
12:20:59
pjb
Shinmera: security is when you can implement a new game in Super Mario World without crashing or destroying Super Mario World. No security is when pressing buttons let you break Super Mario World, and worse, when it let you program the host computer to a different program.
13:00:29
samlamamma
There used to be a video on YouTube where some LispM guys (Symbolics?) present an emacsen C++ IDE. Any idea what that video is called?
13:08:35
Nilby
samlamamma: Maybe you mean Lucid Engergize https://www.youtube.com/watch?v=pQQTScuApWk
14:53:22
boeg
ah that works, I had been trying with normal "list" but it just wanted the "cons list" and i'm not sure I understand why, but "list*" works
14:54:29
beach
boeg: You will understand why when you read the Common Lisp HyperSpec page for LIST*.
14:55:36
decent-username
The list section from the book "ANSI Common Lisp" by Paul Graham helped me a lot.
14:58:17
boeg
yeah, I think I know the difference - the dotted list it just "the items" where as a proper list is composed of a dotted list with an element and a pointer to another dotted list, and so on, and the last dotted list will have a nil as its last element? And in this case, the function i'm calling wants a dotted list with elements all the way through, and not a nil at the end
15:01:34
boeg
right - but am I wrong or? a List say (list 1 2 3) is actually (cons 1 (cons 2 (cons 3 nil))) right whereas (list* 1 2 3) is (1 . 2 . 3) or?
15:03:48
phoe
(list* 1 2 3 4 5 ...) == (list 1 2 3 4 5 ...) as long as you only care about the first five elements
15:04:05
dlowe
boeg: lists in CL are standard singly linked lists of two-field structures called conses. all this other stuff is notation
15:04:34
boeg
dlowe: yes, I had just gotten confused - i knew lists were linked lists, but I thought for some reasons conses wasn't
15:05:58
boeg
I am trying to up my common lisp game, but now that i am trying out (next browser) I guess i'll get a lot more practical experience with common lisp since it's configured in common lisp
15:06:28
dlowe
I mean, a linked list is an arrangement - conses are just a convenient standard to build lists out of
15:10:27
boeg
boeg: no done with that for a first pass, right now I'm reading that paradigms of artificial intelligence programming
15:13:25
jdz
jmercouris: there are quite a few more keybindings with C-c C-v prefix, Press C-c C-v ?.
15:25:34
pjb
boeg: note the names: proper-list, dotted-list, circular-list. proper-list implies that the other types of lists are IMproper. This is quite negative. Why would you want to create and use IMproper lists? This will lead to all sorts of difficulties, given that most CL functions are not prepared to deal properly with IMproper lists. They will in general fail on dotted-list, and enter infinite loops on circular-lists.
15:25:53
pjb
boeg: so you have to have a strong justification to use improper lists such as dotted-lists…
15:29:12
luis
I think I'm holding ironclad wrong. https://pastebin.com/nXu6ch1e Can anyone tell me what I'm doing wrong? The encrypted message is unchanged. %-)
15:34:15
gabc
If it returns a value you're throwing it out and not saving it, which would explain what you're describing
15:40:07
_death
for padding, you can use ironclad::add-padding-bytes (for some reason it's not exported?)
15:40:51
luis
_death: seems like I can pass :padding :pkcs7 (or whatever) to the cipher and :handle-final-block t to encrypt.
15:41:03
_death
and for key, you will likely want to use key derivation.. and if you're actually plan on using it rather than learning, you shouldn't use blowfish, and shouldn't use ECB :)
15:42:27
luis
_death: I'm just porting some code, so I think I have to stick to blowfish and ECB since I have encrypted messages floating around that way
15:43:21
luis
_death: but the wikipedia article about ECB illustrates why ECB is a bad idea very eloquently :)
15:50:15
_death
I guess.. personally I like CTR mode (+ authentication) so I don't need to pad or steal from ciphertext :)
15:50:52
_death
there are other useful ironclad operators whose symbols should be exported.. e.g., ironclad::modular-inverse
15:57:22
_death
pkcs7 padding will always add at least one byte.. if the message size is an exact multiple of block size, it will add another block
16:43:39
_death
boeg: huh, we were talking about lists and cons cells, and Joe Marshall just put out a post about it (check planet lisp)
17:25:51
vivit
Is it possible to define a single method specializer across, for example, all types (SIMPLE-VECTOR N) where N is a positive integer?
17:28:57
Bike
i'm not sure what you're going for with the original question. the union of all (simple-vector n) types is just simple-vector. or do you want to use N in the body?
17:31:04
mfiano
There are libraries to emulate what you want, however. specialization-store comes to mind.
17:42:14
vivit
Bike: I was confusing classes and types; I thought that methods were specialized on types and since (simple-vector 1) is a different type from (simple-vector 2) there would be some special syntax for writing a definition across all simple-vector types. I blame Haskell.
17:46:34
_death
usually functions have a single "target audience", the callers.. with generic functions, there are now two distinct possible audiences, callers and extenders.. if you feel the need to specialize on &rest or &key, that means it may be beneficial not to conflate the two audiences, by defining a separate function for each
17:48:12
_death
(of course you still call the "extenders" function, but you can think of these particular callers as yet another audience)
18:03:01
vivit
trying to define a package-local function called +, but I'm getting the error "The variable + is unbound" when I try to compile the defun. What's up with that?
18:05:11
Bike
What does your defpackage look like? Maybe it doesn't :use :CL, so the compiler is reading your-package::defun and treating it as a function name.
18:12:20
vivit
Now I'm getting "Lock on package COMMON-LISP violated when defining + as a function while in package LRL". How do I indicate that I want to define LRL::+, not cl:+?
18:13:06
Bike
so to be clear, you did your defpackage lrl, and then (in-package #:lrl) or such, and so it's reading things in the LRL package?
18:15:00
_death
you should also know that cl:+ has more roles than a function name, which your:+ may not be able to fulfil
18:27:10
earl-ducaine
phoe: I haven't contacted the Hemlock developers re: the :qt dependancy. But as far as I can tell there's been no active development for over a decade, so I wouldn't expect anyone to take action on it.
19:25:50
jasom
I'm writing some software for an rpi; any tips or gotchas for using lisp to develop for it? Will swank and tramp-mode just work similar to how I would do between two x86oids?
19:36:10
boeg
Is it possible to have an optional argument (with or without default value) as *not* the last parameter. So say a function takes argument (a b &optional (c "") d), and the function can then be called with 3 argument (where c will get default value) or 4 arguments?
19:37:33
_death
you could implement your own arglist parsing, though that's a good sign you should rethink the interface
19:39:48
pjb
boeg: you can either 1- use &key arguments they are optional! or 2- use &rest and parse the list of arguments yourself!
19:41:38
pjb
(defun foo (a b &rest args) (let (c d cp dp) (ecase (length args) ((0)) ((1) (setf d (car args) dp t)) ((2) (setf c (car args) cp t d (cadr args) dp t))) (list a b cp c dp d))) (foo 1 2 3) #| --> (1 2 nil nil t 3) |# (foo 1 2 0 3) #| --> (1 2 t 0 t 3) |#
20:39:08
eeeeeta
jasom: you'd want to use CCL on a pi if you want threading; I was able to get swank working that way