freenode/#lisp - IRC Chatlog
Search
18:45:57
pjb
Unfortunately, with this choice, we have the problem of lisp classes vs lisp types. a uint8 is a lisp type, but an instance of the same integer system class as a int32…
18:47:22
elderK
I mean, there are some real advantages to having the ontology as Lisp classes. But there are, imo, serious costs to that, too.
18:47:47
elderK
Since the idea for my library is to make it so you deserialize /to/ Lisp values. Or serialize /from/ Lisp values to binary.
18:48:21
elderK
But of course, as you have noted, you wind up having to pass "type"s along somehow, since you cannot specialize on your more restricted primitive types.
18:48:35
elderK
That's why I was asking earlier, if we could extend the "primitive classes" that CLOS can specialize on.
18:49:16
pjb
Well, if the boxing can store directly into the binary structure, it can still be efficient. But it depends on the operations you do. If you need to perform algorithmic operations on this data, it's better to keep it as pure lisp data, and to convert only for I/O.
18:50:09
elderK
The idea is that the binary "classes" are mostly just for "transport." You deserialize to an instance of them, so you can easily "get at" the values and stuff.
18:50:43
elderK
That, at least to me, makes it more important to avoid any unnecessary boxing and things.
18:50:51
pjb
The other alternative, is to have your own class, so you can subclass an integer class, but then you will need an accessor when you want to use the value in lisp, because there are funcallable objects, but not "artimeticable" objects, or "sequencable" objects (but some implementations have extensible sequences).
18:51:58
pjb
elderK: you may have a look at com.informatimago.common-lisp.heap.heap ; there, I implemented operations on the binary types to keep them in the binary store, instead of doing the operation in lisp.
18:52:50
pjb
Also of interest: https://github.com/informatimago/lisp/tree/master/common-lisp/data-encoding
18:53:19
elderK
The idea I have in mind is that computation is done on the /deserialized/ values. So, the Lisp values. When you operating on the stuff, you are not directly messing with the "binary" stuff. There's a very clear separation: You are always working on Lisp values. But you can convert binary to LIsp, and Lisp to Binary.
18:54:18
elderK
As for the cost of boxing, it really depends on how smart implementations are. If you have a class, say, with a "primitive" slot, you'd hope that would be stored directly in the class instance, if at all possible.
18:55:44
elderK
pjb: It really depends on how storage is handled. I guess you could for instance, create a special metaclass solely for "primitives" And make it so they store their values in-line, if possible.
18:58:51
elderK
Problem is, you'd wind up having two translation steps. First, deserializing from whatever to the "object ontology" version of the thing. Then you'd have another stpe, where you "unboxed" everything you cared about.
18:59:32
pjb
ie. binary is nothing special or primitive; it's just a given format with a given ontology, and we have to convert between two different worlds, with some in common and some quite different.
18:59:35
Bike
elderK: the problem with storing values unboxed is that the compiler has to cooperate in order to avoid unnecessary boxing/unboxing operations around accessors. this means the compiler has to be aware of the types of slots and accessors when compiling calls to accessors. this makes redefinition of the slot difficult (in general requiring recompilation of all code that calls the accessor)
19:00:10
Bike
as such, i'm not sure that any implementation stores unboxed values in standard objects
19:00:48
pjb
elderK: if you have the time, try to implement several different ways. all lisp, DOM (the foreign ontology in lisp classes), all binary (possibly with CLOS/MOP).
19:01:59
Bike
if you do want unboxed values in a custom data type, defstruct is more likely to work, because redefining it is undefined.
19:02:23
elderK
It would be interesting. But I really, really want to avoid forcing people to manually box/unbox from Lisp values to the ontological ones.
19:02:55
elderK
so that you have a layer that is like (write 'type value) which really just does the boxing and stuff behind the scenes.
19:05:02
elderK
I guess you could also define a standard protocol for "boxing" and "unboxing" things. Like, unwrap <ontological-instance> and wrap <ontological-type-name> value
19:08:25
elderK
I really do wonder how like, things like Genera and stuff /did/ do efficient binary IO and stuff. I mean, for processing network formats and all kinds of things, I imagine you'd want the mapping to be as simple and fast as possible.
19:11:27
pjb
see https://github.com/informatimago/lisp/blob/master/common-lisp/data-encoding/data-encoding.lisp#L75
19:12:12
pjb
basically, yes, lisp machines had low-level tricks (kind of like C casting, but with safety, since the memory slots are still type tagged).
19:13:11
pjb
Because packages use lists of strings, not of symbols. But most package operators accept string designators, ie. string, symbols or characters, to designate strings.
19:14:27
pjb
So, we don't use normal symbols in defpackage, because they are interned in the current package, and when you use the defined package, there's a lot of collisions.
19:15:16
pjb
Instead, we may use uninterned symbols, but I find it #:ugly, or keywords, but this fills the :keyword package. Finally strings have the advantage of uninterned symbols, but they're "NICE".
19:15:33
elderK
The main problem I see from using strings, is that it's up to settings as to whether 'foo is "FOO" or "foo"
19:16:02
pjb
Now there's the consideration about readtable case. Using uninterned symbols has the advantage of using the readtable case. While using strings requires typing the absolute name case.
19:16:59
pjb
I learned programming in the 70s, there were a lot of terminals and printers with only upper case then.
19:18:00
pjb
Honestly, this makes a difference only when used in implementations with a "modern" mode implemented badly. Ie. with allegro CL (mlisp instead of alisp). clisp has also a modern mode (per package) so it doesn't break.
19:19:44
pjb
Since I do a lot of copy-paste from my repl to irc, I've this in my rc-file, so: (intern "FOO") #| --> foo ; :internal |# <--
19:57:59
pjb
elderK: just beware that sometimes there's buggy code that use format ~A to build symbol names from symbols or other, and therefore you'll get downcased symbol names! Just non-buggy libraries instead ;-)
20:06:24
elderK
Xach: How long do you think it will be before package-local nicknames are universal? :D Alternatively, how important it is these days to really "support" implementations other than say, SBCL or ECL?
20:07:23
Xach
elderK: I think there needs to be a next round of planning and implementation of package-local nicknames. i don't think the current crop are adequate.
20:07:54
Xach
I have a personal idea of how it can be done better but I have to get it out into the world for refinement (and rejection or acceptance)
20:08:35
elderK
Xach: Yeah, I'd definitely be interested in hearing shortcomings of say, SBCL's method
20:09:28
Xach
Bike: "adding" isn't a good word. i mean using sb-ext, or ext, or sys, or whatever, to make use of them.
20:09:57
Xach
I want portable use of package-local nicknames without compatibility layers, and I think it can be done.
20:10:19
Xach
Bike: I think they can be done with standard CL functions with extra arguments and return values.
20:12:37
Bike
no, they're allowed as an extension, i think, but adding return values as an extension is specifically not allowed
20:13:51
Xach
Bike: can't get any clearer than that. well, then my proposal wouldn't conform to that. bummer.
20:14:53
Bike
well, i think package local nicknames would mostly just be used in the form of :local-nicknames or whatnot in defpackage, which doesn't really require a compatibility layer
20:32:22
elderK
I have a question about the reader. Particularly with macro characters. The cylicness of the reader kind ofhurts my brain.
20:32:55
elderK
Like, let's say the reader... is reading. And the user has like, set a macro character. Depending on situation, the reader will invoke the function for that macro character, to handle the read.
20:33:13
elderK
But is the "read table" in use by that "macro character function" the same as the read table that was in effect at the time of reading?
20:33:35
elderK
Or is the read-table in effect in the macro function, the one that was in effect when the macro function was set / defined?
20:34:33
elderK
It kind of seems like, to "read", you need a functional implementation, because of reader macros...
20:44:41
Bike
elderK: it just uses what's in *readtable*, so basically what's in effect at time of reading.
20:45:29
White_Flame
a reader macro can either recursively call READ which uses the readtable & normal reader rules, or it can consume character by character and do whatever it wants locally
20:50:32
elderK
Iono. It just makes my brain kind of explode. Like, I think there must be some "basic hardwired lexer" and stuff that powers the "core" language.
20:51:57
White_Flame
pfdietz: sure, but that's not what deals with the input stream & dispatching on characters. It's Lisp code, it can do whatever it wants ;)
20:52:01
pfdietz
I am somewhat ambivalent about customizing the reader. It makes it harder to write programs that grovel over general code.
20:53:15
elderK
I just wonder how such a... reader is implemented. Like, let's say we "compile" our stuff to some form we can more easily execute. Say, byte code or something. Or maybe we just, literally remember the AST and walk it to execute.
20:53:49
elderK
So, we'd have to have a table that'd contain a pointer to the function to be invoked to do the reading. Since that can be redefined, we have to be able to decide whether to call some built-in compiled function to do it. Or, run the "potentially interpreted one"
21:01:14
White_Flame
did the Lisp reader exist before the concept of "lexer" was established as we know it today?
21:03:20
elderK
I'm just trying to... match things up. I mean, how can reading be completely independent from... stuff. It can't just be a "typical lexer / scanner" because macro characters require us to invoke some function, that is potentially set by user.
21:03:47
elderK
so, in an interpreter, that would require us to then interpret that macro function so to continue the read.
21:07:29
Bike
and funcall doesn't need to invoke the reader in any way (unless you're calling, like, READ itself)
21:08:00
no-defun-allowed
All reasonable Lisps use recursive descent which makes these things quite trivial.
21:08:58
elderK
Bike: Right. But that implies that we can somehow "funcall" a function. Which means we need to have been able to read it, and somehow store it in a form we can call.
21:09:57
elderK
Yes, but the only way it could be there in memory, the function, is if we already read it, no?
21:10:13
no-defun-allowed
It's more appropriate than lexing/parsing, and I've heard it called that before.
21:11:27
Xach
elderK: it would still be pretty normal to separate reading and evaluation in not-lisp
21:11:48
no-defun-allowed
For example, the Unix Hater's Handbook compares it to the contextual grammar of C++, calling Lisp's parser "recursive descent, meaning you can write it on a piece of paper" from memory.
21:12:09
elderK
no-defun-allowed: There's parsing, then there's lexing. to me, they are independent.
21:12:16
Xach
evaluation is what makes the system do stuff, like create function objects and maybe associate them with a global name.
21:13:00
elderK
Bike: Right. My main problem isn't with the fact that, lexing/parsing is basically "all in one." I've seen lexerless parsers.
21:14:10
Bike
being "dynamic" pretty much just means the table is an object in memory rather than an off-line thing.
21:14:55
elderK
Yeah, I get that. I'm just thinking of the necessity to have some way to actually read code in the first place, so that the user's code that sets a new reader macr-function can be understood and patched in :P
21:16:34
elderK
Bike: Which is to say, when you start the Lisp implementation - say from the standard image - everything is in place to read and understand standard Lisp. If it reads stuff, evaluates stuff, that changes that, well, okay, because it started off with the base syntax.
21:21:33
elderK
I really want to learn how to make a lisp implementation. Toy to start with, but increasing complex and non-toy-ish.
21:21:47
elderK
But I'm not really sure how to best start. Finishing LiSP and studying RABBIT, maybe,
21:23:21
Xach
elderK: Lisp in Small Pieces is a good book on the topic. i like the progression it follows from simple to complicated
21:23:45
Bike
i mean a basic metacircular interpreter is like a page long, if you just have scheme special oeprators
21:24:38
elderK
:) I wonder if I can do like, all the stuff it covers, but in CL rather than Scheme.
21:25:51
elderK
I also need to learn how to do ... like, usual lexing in Lisp. I'm used to implementing lexers in C using transition tables and stuff. Basically state[current_state][input_category] is the next state.
22:37:25
pjb
elderK: about *readtable* and reader macros, note that *readtable* is a special variable, therefore it has dynamic scope. lexical = WHERE, dynamic = WHEN. So the question is WHEN the reader macro is executed, what binding has the *readtable*. Of course, if the reader macro function is called thru the *readtable*, by reading the macro character, at that time, the *readtable* is bound to the readtable that contains the mapping of
22:38:08
pjb
elderK: and once the reader macro function is executing, nothing prevents you to bind *readtable* to another readtable!
22:38:49
pjb
elderK: for an example of this: https://github.com/informatimago/lisp/blob/master/objcl/objcl.lisp#L88
22:39:36
pjb
Notice how the reader macro functionr ead-objcl-expression binds the *readtable* to *objc-readtable*.
22:40:43
pjb
elderK: the syntax is [object messageWith: (foo) andWith: bar] where messageWith:andWith: must be read case sensitively amongst other things.
22:42:28
pjb
elderK: but if object is not a special identifier super, then it must be read as a lisp expression: [(if foo obj1 obj2) getIt] hence the other binding to *lisp-readtable*.
22:43:56
elderK
pjb: Of course, when the macro function is compiled, the syntax that is in effect... then, is set. Like, just beacuse you change the readtable, doens't mean the macro function suddenly gets recompiled, right?
22:45:20
pjb
White_Flame: it was developed about at the same time. Remember that Fortran was defined earlier tha LISP. Backus and Naur formalized the BNF for ALGOL 58 in 1959.
22:47:32
pjb
elderK: And yes, you can also mutate the readtable bound to *readtable* itself while reading. Ie. you can write a reader macro function that will set macro-chracters while reading.
22:48:58
pjb
White_Flame: However, the syntax of LISP (1959) was very simple. Basically, a lexer to read symbols, strings and numbers, and a parser for the sexp syntax.
22:49:58
pjb
White_Flame: elderK: yep, when the source code is compiler, it's the readtable that is bound to *readtable* in the compilation environment that is used. It can be the standard readtable, or something completely different. For example, if you write your reader macro in vacietis!
22:52:58
pjb
White_Flame: and in CL, the lisp reader is clearly defined in two parts: the basic lisp reader algorithm specifies the lexer (which is a tad more complex than what you'd do in general with lex (but you could do it in lex, there are states)), and then it defines the standard reader macros of which there are two kinds: macros such as #\" scan tokens such as strings, and macros such as #\( parse s-exps. Again, not much syntax here (
23:15:38
buhman
if I had a list of flags, and a flag I wanted to "toggle" by adding it or removing it from the list, is there a fancy way to do that?
23:21:10
pjb
(if (zerop i) (setf (car list) (not (car list))) (let ((cell (nthcdr (1- i) list))) (setf (car cell) (not (car cell))))) is slightly faster.
23:23:58
pjb
(defun neg (x) (if (zerop x) 1 0)) (define-modify-macro negf () neg) (let ((flags (make-array 50 :element-type 'bit :initial-element 0))) (negf (aref flags (random 50))) flags) #| --> #*00000000000000000000000000000000010000000000000000 |#
23:24:51
pjb
Then you could just mutate the list (unless you do it a lot in an inner loop and need the speed).
23:28:17
pjb
(let ((flags (list :carry :zero))) (deletef flags :carry) (pushnew :minus flags) flags) #| --> (:minus :zero) |#
23:29:20
pjb
and you can use find to test: (let ((flags (list :carry :zero))) (deletef flags :carry) (pushnew :minus flags) (values flags (find :carry flags))) #| --> (:minus :zero) ; nil |#
23:30:36
pjb
On the other hand, with bitvectors, you can mask off flags more easily and efficiently than with sets (lists). (intersection flags mask) vs. (bit-and flags mask)
2:01:28
PuercoPop
Does (nth-value 0 ...) has any benefit? Does it make it easier for the compiler to avoid unnecessary allocations?
2:02:56
no-defun-allowed
with (SPEED 3) on sbcl, FLOOR and whatever other int maths you have don't play nice without a VALUES or NTH-VALUE in the middle
2:03:36
no-defun-allowed
see https://gitlab.com/Theemacsshibe/cl-vep/blob/master/outside-world/image.lisp#L9 for example
2:05:30
PuercoPop
no-defun-allowed: I don't see (nth-value 0 ), but I'm guessing the extra (values ) is also a way to 'convince' the compiler that there is only one value
2:07:25
Bike
they are semantically identical, since VALUES is just a normal function and its argument forms have values past the primary discarded.
3:54:22
beach
I know what you mean. But in two weeks it turns again, provided you are on the northern hemisphere, of course.
4:10:42
beach
If anyone is good with LaTeX, I would like to see the dpANS document(s) converted to a single LaTeX file. Any takers?
4:12:40
beach
esper0s: It is just that there is so much to do in the Common Lisp world that I am trying to direct attention to it, in the hopes that some of it will get done by someone other than me.
4:15:26
beach
Now, if we can turn the dpANS into a LaTeX document, we could slowly turn it into a specification of WSCL (which means Well Specified Common Lisp, and is pronounced like "whistle").
4:26:59
beach
I am not complaining though. Lots of good work is getting done on McCLIM on a daily basis by jackdaniel, slyrus, loke, and several others. And I am convinced that McCLIM is going to be an essential part of any end-user application written in only Common Lisp.
5:08:37
no-defun-allowed
erm, i changed phone and my 2fa codes aren't working for gitlab.c-l.net, who should i contact about that?
6:05:17
slyrus
Hrm... That's an awfully big file. Seems to me the source should be lisp s-expressions (in one or many files) from which the LaTeX is generated.
6:09:41
beach
The source code for the dpANS is already TeX (but not LaTeX). Converting it to something else would be an even harder task.
6:11:38
no-defun-allowed
but since there isn't anything too modern like TikZ, i thought it'd be like collecting commands and bracketed things
6:11:52
beach
slyrus: Oh, I am sorry. I didn't mean a single LaTeX source file. I meant a single LaTeX document.
6:12:32
beach
slyrus: Currently, each chapter is a separate document, which makes things like cross referencing and citations hard.