freenode/#lisp - IRC Chatlog
Search
10:15:02
schjetne
But that sort of defeats the purpose of exceptions, when you have to do all kinds of in-band signalling, you're not far from manual error checking like old-fashioned C, or the pure functional languages.
10:16:33
schjetne
So ABCL is an option (but I'm not sure I want to deal with a lot of foreign function calling)
10:17:27
ogamita
schjetne: Ken Pitman has a metalinguistic implementation of the CL condition system including restarts; it could be ported to any language… (theorically).
10:18:30
schjetne
Another option could be to pass the responsibility on the error logging system to know in which file and line the problem happened (and ideally which line in the DSL files too, I wonder if that could be somehow extracted)
10:19:28
schjetne
But that probably means still passing something as explicit parameters, Kotlin lacking dynamic scope.
10:20:21
splittist
If the logger wraps the parser and provides the stream(s) to the parser, then it can track stream progress, perhaps.
10:25:58
schjetne
Maybe I can throw a list of all the exceptions thrown when passing the fields, and then throw a list of all the exceptions thown when parsing the lines, and catch all of them in order at the file level.
10:26:09
no-defun-allowed
David Schmidt was selling a macivory machine with 4 megawords which is about 16 to 20MB of memory if you wanted something modern-ish and not embedded.
10:26:17
v0|d
beach: i tried to compile small ECL binaries back in the day yet faild to run on routers
10:27:16
schjetne
Java exceptions have a notion of a cause, which can be another exception. In this case there would be multiple causes. There would be no way to control the restart behaviour, but it should do the trick.
10:52:30
jackdaniel
v0|d: you may be interested in fact that green threads (coroutines) and delimited continuations will be in next ECL release
10:54:02
jackdaniel
but yes, it will be possible to use both at the same time (i.e a few native threads on which multiple green threads run)
10:56:51
loke`
jackdaniel: That's very intyeresting. What does the API look like? Similar to Scheme?
11:03:11
jackdaniel
loke`: http://hellsgate.pl/files/6d156f6b-user.pdf (navigate to multithread) - this is old ecl documentation from before introduction of native threads (coroutines evaporated in a meantime)
11:04:03
jmercouris
I'm trying to completely eliminate cffi from my application, and if I can't find a suitable replacement for a sqlite wrapper, then I'll have to implement my own pseudo database with files in Lisp :D
11:05:06
Odin-
jmercouris: So, you're basically asking if someone has reimplemented SQLite in Common Lisp?
11:07:10
jackdaniel
if you refuse to use ffi you don't have many options left. you either need a database which you may access via some wire protocol and set up on the machine (i.e postgres), or you serialize/deserialize data
11:07:40
jmercouris
jackdaniel: what are the advantages of cl-store vs just printing and then reading the data structures from a file?
11:07:59
jackdaniel
as of sqlite: it is not a standard but rather sql database engine, so "cl library for sqlite" doesn't make much sense, "clone of sqlite" would be a better phrasing
11:08:08
jmercouris
e.g. let's say I have a list of alists, why not just print them to a file? why might I use cl-store in this case? not trying to argue, trying to understand
11:09:19
jackdaniel
cl-store may serialize for instance hashtables (you'd have to effectively reinvent serialization by yourself if you insist on printing objects to file)
11:09:56
jmercouris
so if I end up serializing things that aren't simple lists of strings or something, then it would be quite useful
11:09:57
jackdaniel
also serialization format may be (not necessarily is!) more compact and faster to read/write to disk / over the network
11:10:14
Odin-
Particularly because there's a bunch of types with no readable form mandated by the standard...
11:10:35
jmercouris
let's say you are developing a system, and you want it to depend on another system
11:10:52
jmercouris
that you are also developing, would you put those in the same repository, or different reposiitories?
11:11:46
Odin-
jmercouris: The thing about SQLite is that it's a full-featured relational database. It can do a _lot_ of neat tricks that aren't obvious if you're just thinking about storing values by key. :p
11:12:09
jackdaniel
as of question what I think you should do: I think you should access sqlite via ffi
11:13:24
jackdaniel
and regarding the second question: keep dependencies in the same repository and update them after testing they work (for builds), but also mainain a separate repository if you want to publish your library i.e on quicklisp
11:14:32
jackdaniel
it depends what you plan to do. if you are interested in providing final builds, then versioning all dependencies makes a lot of sense
11:15:05
jmercouris
Odin-: that's not my issue per se, for example, when I distribute a binary for macOS, I think some users are having issues because the image contains paths to locations of sqlite on my disk (because I installed via macports), things like that
11:16:36
jmercouris
ok, I will see how this part of the codebase evolves, and maybe I will break it out into a library depending on its usefulness/whether I decide to keep sqlite or not
11:17:13
jmercouris
I will look at what kind of data structures I plan on persisting/traversing and decide if it is simple enough to do in just lisp
11:17:26
jmercouris
maybe my needs aren't as great as something a powerful facutly like sqlite provides
11:17:43
Odin-
Hrm. I haven't really dealt with binary images much, but I'd presume there's a way to have the FFI object loading (which shouldn't depend on the exact location, but use system configuration) to happen when the image is loaded?
11:19:02
Odin-
jackdaniel: I'm just leaving space for the possible existence of completely insane implementations. :p
11:19:09
jmercouris
the issue with macOS, is that a user using brew, will not have things in standard locations
11:20:08
jmercouris
I have another question, where can I learn more about how struct comparison is done for a hash table with a :test of equalp?
11:21:00
elderK
Guys, is there a way to get ASDF to spit out an ordered list of files, like, that it loads for some system?
11:23:15
jackdaniel
another super-userful feature I'd love to see is: dump plan into a file which may be loaded without asdf involved to (say) compile the system
11:23:28
jmercouris
jackdaniel: so you are saying that a hash-table in CL will literally traverse a whole list and use equalp as a comparator to find an object?
11:24:35
jmercouris
if the above is true, it is erroneously called a hash table, and should be instead called a dictionary
11:25:50
jackdaniel
equalp was not initially planned to be an option for hash-tables, but a demand has been envisioned
12:07:27
elderK
Guys, are "structures that are literals", like, via make-load-form and stuff, guaranteed to be "coalesced"?
12:08:03
elderK
Are such literals even guaranteed to be like, "interned"? Or could they be reconstructed over and over and over and over.
12:09:13
ogamita
(list #S(point :x 2) #S(point :x 3)) #| --> (#S(point :x 2) #S(point :x 3)) |# mutable structures.
12:15:46
ogamita
elderK: IMO, by default structures are not similar one to the other. This section says that make-load-form defines a similarity between its argument and the object produced by its result, which is necessary for logical consistency. But it says nothing of the similarity between two different structures that happen to have the same slots.
12:16:32
ogamita
elderK: Therefore, AFAICS, CL will do no coalescing of the structures, but make-load-form could return a form that would internalize the structures it would create, and it could return several times the same one.
12:17:13
elderK
ogamita: Well, I use make-load-form already. But, like, you know, it expands to a form that recreates some instance, say.
12:17:28
ogamita
But notice what I said above: unquoted -> no expectation of coalesing or immutability. make-load-form couldn't known if the instance was quoted or not!
12:17:45
elderK
Or ,structure or whatever, I'd prefer it didn't keep calling (make-instance ...) every time that form was executed.
12:18:28
ogamita
So if you implement such a internalizing make-load-form, you should document it clearly that all your structure literals should be considered immutable, quoted or not.
12:22:03
elderK
There's a "top-level" function, read-from. It expects a symbol denoting the type to read, etc.
12:22:16
elderK
Then there are other functions, that don't do type checking and are internalish. They are specialized on type metaobject.
12:23:05
elderK
My macro expands into a call to read-from, rather than %read-from. Because, at that point in the expansion, there is no "visible" metaobject. The macro expander knows the metaobject (and that metaobject /will/ be avilable at runtime)
12:23:24
elderK
So I was thinking I could side-step that read-from, and use %read-from with a 'class literal', patched into the expansion
12:24:06
elderK
All the current way means is that there is some extra type-checking done, and a conversion from the "symbol" to teh "type metaobject"
12:24:26
elderK
(Which should be reasonably cheap: The metaobject is attached to the symbol's plist)
12:25:34
ogamita
jmercouris: putting everything in the same repository is easier to manage, and to keep the versions in sync.
12:27:04
jmercouris
ogamita: that was the idea I had in mind, I was just wondering if it was considered a sin or anything like that
12:35:55
Odin-
(Says the man with documentary evidence Google paid a lawyer to get a seal lifted on a warrant for his data.)
12:38:01
elderK
Guys, is it bad taste to like, generate the exports for a "superpackage" dynamically?
12:38:15
ogamita
what is the result of: (format nil "~8,'0X" (reduce (lambda (n b) (+ (* 256 n) b)) '(1 2 3 4) :initial-value 0))
12:38:15
ogamita
what is the result of: (format nil "~8,'0X" (reduce (lambda (n b) (+ (* 256 n) b)) '(1 2 3 4) :initial-value 0 :from-end t))
12:39:43
ogamita
(defparameter *packlist* '("CL" "FOO" "BAR")) (defpackage "SP" (:use . #.*packlist*) (:export . #.(let (n) (dolist (pack *packlist*) (do-external-symbols (s pack) (push (symbol-name s) n))) n)))
12:48:47
ogamita
Yep. With :from-end t you need to swap the parameters of the function. Ie. the accumulated value changes place.
13:03:52
_death
jmercouris: boolean is (member nil t) .. since type-of will return NULL for NIL, the answer is no
13:07:36
jmercouris
so you would have to do class-of instead of type-of to get what I was originally believing
13:08:00
jmercouris
sorry if its unclear on paper, I know you guys aren't in my head, but I understand now
13:08:55
jackdaniel
http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_g.htm#generalized_boolean
13:09:05
Odin-
jmercouris: Common Lisp's type system is by no means a one-to-one mapping, which can also confuse.
13:09:11
jmercouris
I'm trying to fix the s-xml-rpc library which casts "t" to a string and "nil" to a string when returned by a function instead of a boolean
13:11:33
jmercouris
It seems we are paying tech debt since the dawn of time, here we are running x86 processors...
13:12:01
jackdaniel
sure, I get that (and I'm not complaining), I'm just pointing out that it is a little unfortunate at various occasions
13:33:04
ogamita
and by the way in scheme, else can be redefined, so (cond #f (else 1)) may return #void.
14:04:39
jackdaniel
they usually do, however hash-table class is not guaranteed to be a structure-class (like it is on SBCL) so it may fail on conforming common lisp implementation
14:16:02
jackdaniel
Bike: do I get this part wrong (from make-load-form): "Conforming programs may call make-load-form directly, providing object is a generalized instance of standard-object, structure-object, or condition."?
14:17:07
Bike
i mean, stuff like vectors are dumpable too, but not through make-load-form necessarily
14:19:22
jackdaniel
elderK: it is - until it is insufficient (or too complicated); it is not extensible and hitting undefined behavior with it is not that very hard given how it is specified
14:19:58
elderK
jackdaniel: A lot of the time I find that I can use a let, and a dolist or something inside that, but more often than not, I make use of loop.
14:20:17
elderK
Of course, it proably loops kind of horrible, since I use with = ... for hte same reason I'd use the surrouding let otherwise.
14:21:02
jackdaniel
most of the time I use mapc combined with collect macro, and if I'm in need for something more sophisticated I use DO (and in rare occasions I write a macro which is built on top of DO)
14:26:19
elderK
jackdaniel: I feel this would probably be more complex if I had done it using let and other stuff: https://pastebin.com/BFXHLazL
14:42:53
elderK
_death: Partly - I'm not sure it matters that much. I have subtypes of integer, defined based on binary-integer types.
14:43:22
elderK
I'm using check-type integer in a few places, and I'd rather use check-type <the-actual-binary-integer-type>
14:47:49
_death
elderK: lisp supports wish-based programming.. you can define your own check-type-like macro..
14:54:08
elderK
If I have a make-load-form, and it calls some function in my package, does that function need to be exported from the package?
14:56:50
elderK
The symbol would need to be accessible, since the function I want to call is bound to that.
14:58:40
jmercouris
terminology can be important because it changes how you are thinking about the concepts
15:01:26
elderK
Maybe this is a time where :: would be useful - since the function is only ever going to be called from a macro's expansion.
15:03:56
ogamita
Then indeed, don't export it. As I said, write a public function that will call the internal function.
15:03:57
elderK
It's not meant to be exposed. The only reason I *want* to use it, as part of the form make-load-form returns.
15:04:22
_death
if you have a load form, there are no particular requirements of the symbols in it.. at load time, you need to make sure the package is there and when it is evaluated that they are fbound..
15:05:25
Bike
elderK: the return value of make-load-form is not READ again, so the export status of symbols in it is largely irrelevant
15:07:01
elderK
Bike: Interesting. I thought it would be, so like, you know, it could reconstruct the whatever?
15:08:57
Bike
the form will be evaluated, in some sense. but there's no reason to READ the code in again, it's right there.
15:12:46
elderK
I'm still not 100% on like... the details of make-load-form, and when the form it generates, is evaluated. How many times it is evaluated, etc.
17:02:19
elderK
I post it here so you can take a look, if you find yourself bored and with nothing else interesting to do :)
17:14:47
elderK
beach: :) I will try to avoid using such sigils in the next iteration, and in anything further I create.
17:15:31
pfdietz
My convention is to use % as the leading char of the names of flet or labels functions.
17:16:05
elderK
I need to learn more about such conventions. The Google Style guide doesn't really go into this stuff.
17:16:45
elderK
Still, I am mostly happy with what I have created so far. I am yet to heavily dog-food it. But, I feel like I'm made some progress.
17:23:40
elderK
I guess the only way to keep improving, is to keep studying code written by others, continue posting here for feedback, and simply continue trying things.
17:37:15
pjb
limiting yourself to 64-bit is not sane, it's insane and restrictive. IPv6 addresses are 128-bit, and in crypto, we often need even bigger numbers…
17:37:32
elderK
pjb: nil signals that the byte order is set dynamically, rather than hardcoded into the integer type itself. If byte-order is nil, the effective byte-order is determined by the current value of hte *default-byte-order* variable.
17:40:22
pjb
so, yes, % is used for "dangerous" functions, not just non-exported functions. Normal functions such as convert-from, convert-to, range, etc don't need it.
17:43:02
elderK
What is the convention for things that say, are not meant to be used by "general users", but can be used for "developers." Like, someone who's just using the library vs. those extending it
17:44:33
pjb
Well, you use :: when you cannot do otherwise. Eg. when the developer of the library forgot to export a symbol that you really need to use. (and you can't patch the library).
17:45:07
elderK
So basically, avoid using :: at all costs. The current way I feel about it is "Beware"
17:45:15
pjb
But even in that case, it would be better to define a package exporting those symbols, since that would define clearl the API you are using.
17:45:34
Colleen
Unknown command. Possible matches: 8, time, set, say, mop, get, tell, roll, help, deny,
17:48:17
pjb
elderK: your parameter lists are strange: (offset type value vector). I would order them (type value #| -> |# vector offset)
17:49:38
pjb
I would group the parameter by category: input parameters first, output parameters next.
17:50:09
pjb
You may choose the other ordering if you want. But not mixing input and output parameters.
17:51:33
pjb
write-into has an implicit "parameter" which is the size of the integer written. It's specified by the type parameter. But since it takes an offset parameter, it would be better to return the size (or the new offset), rather than the value that has been written.
17:53:10
elderK
I will apply these suggestions :) And I will also work on getting rid of the %s and stuff.
17:54:49
sukaeto
re: %, I always thought it was for implementation specific, or as pjb says "dangerous" functions
17:55:32
elderK
Another issue, of course, is that I want to use short names like type-of and stuff, but CL already uses them :P
17:56:37
sukaeto
I sometimes put a . in front of symbols that name things that would otherwise be exported (e.g. slots whose accessor is named by the same symbol as the slot would be, if I didn't stick the . in the slot name)
17:57:05
pjb
Let's take an example. You implement a hash-table. There's a vector of buckets, and a function that finds the index to the right bucket. Not all buckects are filled with valid data. So if you use aref on the vector of buckets randomly, you could find invalid data and get bad results or behavior. Then you could have an accessor (%bucket ht index) that would let you use all the slots, but normal code should use (bucket-for-key ht
17:58:27
pjb
The only downside is that if you need to use a shadowed symbol, you need to qualify it: (cl:type-of 42)
18:00:27
pjb
For internal stuff, don't hesitate. If your shadowing symbol is exported, consider how your package will be used. If it's used with CL (or the package owning the symbol you shadowed), then you may consider defining a version of the CL package with your symbols instead, otherwise the user will have to use (:shadowing-import-from :your-package :s1 :s2 :s3 …)
18:00:43
sukaeto
elderK: well, on the one hand, I don't want to use something that has an understood meaning for a different meaning. On the other hand, I've never seen anyone else use a . prefix in Lisp - I kind of stole it from UNIX's hidden file model. So people may read that code and be like "wtf was this guy thinking?"
18:01:25
pjb
sukaeto: . is often used in examples showing how to implement standard operators, to avoid shadowing (or in scheme where no such shadowing exists).
18:06:27
pjb
So, for structure, you are basically re-implementing a simple MOP. Not bad. But eventually, perhaps you will learn about CLOS and the MOP, and then you will be able to implement your binary structures directly as a CLOS metaclass. This would give a better integration with CL.
18:07:36
elderK
pjb: I spent several days studying MOP and I have completed reading the book Jachy suggested. The main reason I avoided using the MOP here, was simply because it seemed that it would be simpler to do so.
18:08:28
elderK
:) shka_ suggested MOP to me a few days ago. It's definitely something I would like to move to. But only if necessary.
18:10:13
pjb
structures have single inheritance (the :include option). A given clos metaclass could restrict the number of superclasses.
18:11:47
elderK
pjb: I was more meaning like... CL defines classes for say, integer and cons and stuff. I was wondering if I could do the same, so that people could specialize on "primitive" binary types like unsigned integers and stuff.
18:13:14
sukaeto
elderK: and you may be right about that! The only reason I spoke up is because I'm pretty sure I'm doing it wrong (or at least, what I'm doing is less than ideal), and I wanted to see what other people in the channel would say.
18:13:18
elderK
Jachy: I found it a very interesting read. I especially enjoyed the contrasting of various OO approaches.
18:14:16
elderK
sukaeto: I was curious if there was a standard convention for "I wanna call this variable SOMETHING but I can't, because it's shadow something else I need. So, I need to name it differnetly..."
18:14:19
pjb
Well, you use defclass to define some classes. So there are type designators of same name to designate their type already. The question would be if your classes weren't CLOS classes (or CL structures). How would you write (deftype foo () ?) You can always use satisfies I guess.
18:14:31
sukaeto
how's the old joke go? There are two hard problems in programming: Cache invalidation, naming, and off by one errors
18:15:11
pjb
elderK: you can play tricks: (let ((|my variable| 42)) (/ |my variable| 2)) #| --> 21 |#
18:15:37
elderK
pjb: I was more asking about how I could integrate my own "primitive types" just as things like "integer" and "list" are with CLOSs. Like, so you can specialize on something that is not a class instance, but a primitive.
18:17:32
elderK
I tend to the use the same conventions I use elsewhere: Shadowing is okay, if it's extremely obvious that they /are/ shadowed and stuff. Otherwise, use different names.
18:17:46
pjb
elderK: well the thing is that you cannot specialize built-in classes, only standard-classes and structure-classes.
18:18:49
elderK
It would be nice if I could do the binary-parsing stuff without parsing integers and things byte by byte. But even so, doing it byte by byte avoids potential alignment issues if I were to use, say, CFFI to read integers...
18:29:21
pjb
elderK: don't expect they know what they're doing. Provide a simple API that let them do what they want without knowing anything about it.
18:29:55
elderK
pjb: Right. But even if you do provide that simple API, people can still directly use defclass with your :metaclass and stuff.
18:31:16
elderK
I'd also need to think about how to make like, the read/write interface the same for "primitives" and "classes"
18:36:09
pjb
elderK: so basically, you need to define a type system to describe the binary types. This type system is not the same as the lisp type system.
18:36:34
pjb
elderK: you may want to have some gateway, some automatic correspondance between them, but it's not necessary.
18:37:09
pjb
elderK: the lisp values used to represent the binary values can be of different types/classes too.
18:38:23
elderK
Another detail I'd have to think seriously about is redefinition of classes and stuff with the MOP.
18:38:55
pjb
For example, you read a uint16/be as a lisp integer, character, bit-vector, etc. and vice-versa, to write a binary value, you can take different types of lisp values.
18:39:23
elderK
Like, if you redefined the class so that a slot's type were different. So, that the value in the instance is no longer conformant with the new type.
18:39:26
pjb
Including, you can define in lisp classes the ontology of binary types. Ie. a uint16/be could be an instance of a clos class uint16/be.
18:40:22
pjb
elderK: Well, it wouldn't be the binary integer, it would be the CLOS representation of a binary integer.
18:40:57
elderK
Well, yes. But that would mean for instance, instead of (+ int int) you'd have something like (+ (uint16/be-value whatever) ...)
18:41:05
pjb
Now, the advantage of that is that you can create a representation of binary structure as lisp objects.
18:42:16
pjb
Now there's also the possibility to keep the binary values, and to have in lisp descriptors that will let you access the binary values directly.
18:43:03
pjb
The question is when you want to do the conversion between lisp objects and binary objects.
18:44:02
elderK
Well, the idea I was going on was that "binary types" were just... restricted or otherwise limited "normal Lisp types." The actual serialization of values is done as late at possible.
18:44:36
pjb
In lisp you can manipulate the descriptor, and convert the value on-demand when needed. Or you can convert into a DOM (to keep the ontology of the binary type in lisp). Or you can convert into lisp objects (but then it's more complicated to make the correspondance between the two type ontologies, you may have to add descriptors, or you may have some ambiguities.
18:45:57
pjb
Unfortunately, with this choice, we have the problem of lisp classes vs lisp types. a uint8 is a lisp type, but an instance of the same integer system class as a int32…
18:47:22
elderK
I mean, there are some real advantages to having the ontology as Lisp classes. But there are, imo, serious costs to that, too.
18:47:47
elderK
Since the idea for my library is to make it so you deserialize /to/ Lisp values. Or serialize /from/ Lisp values to binary.
18:48:21
elderK
But of course, as you have noted, you wind up having to pass "type"s along somehow, since you cannot specialize on your more restricted primitive types.
18:48:35
elderK
That's why I was asking earlier, if we could extend the "primitive classes" that CLOS can specialize on.
18:49:16
pjb
Well, if the boxing can store directly into the binary structure, it can still be efficient. But it depends on the operations you do. If you need to perform algorithmic operations on this data, it's better to keep it as pure lisp data, and to convert only for I/O.
18:50:09
elderK
The idea is that the binary "classes" are mostly just for "transport." You deserialize to an instance of them, so you can easily "get at" the values and stuff.
18:50:43
elderK
That, at least to me, makes it more important to avoid any unnecessary boxing and things.
18:50:51
pjb
The other alternative, is to have your own class, so you can subclass an integer class, but then you will need an accessor when you want to use the value in lisp, because there are funcallable objects, but not "artimeticable" objects, or "sequencable" objects (but some implementations have extensible sequences).
18:51:58
pjb
elderK: you may have a look at com.informatimago.common-lisp.heap.heap ; there, I implemented operations on the binary types to keep them in the binary store, instead of doing the operation in lisp.
18:52:50
pjb
Also of interest: https://github.com/informatimago/lisp/tree/master/common-lisp/data-encoding
18:53:19
elderK
The idea I have in mind is that computation is done on the /deserialized/ values. So, the Lisp values. When you operating on the stuff, you are not directly messing with the "binary" stuff. There's a very clear separation: You are always working on Lisp values. But you can convert binary to LIsp, and Lisp to Binary.
18:54:18
elderK
As for the cost of boxing, it really depends on how smart implementations are. If you have a class, say, with a "primitive" slot, you'd hope that would be stored directly in the class instance, if at all possible.
18:55:44
elderK
pjb: It really depends on how storage is handled. I guess you could for instance, create a special metaclass solely for "primitives" And make it so they store their values in-line, if possible.
18:58:51
elderK
Problem is, you'd wind up having two translation steps. First, deserializing from whatever to the "object ontology" version of the thing. Then you'd have another stpe, where you "unboxed" everything you cared about.
18:59:32
pjb
ie. binary is nothing special or primitive; it's just a given format with a given ontology, and we have to convert between two different worlds, with some in common and some quite different.
18:59:35
Bike
elderK: the problem with storing values unboxed is that the compiler has to cooperate in order to avoid unnecessary boxing/unboxing operations around accessors. this means the compiler has to be aware of the types of slots and accessors when compiling calls to accessors. this makes redefinition of the slot difficult (in general requiring recompilation of all code that calls the accessor)
19:00:10
Bike
as such, i'm not sure that any implementation stores unboxed values in standard objects
19:00:48
pjb
elderK: if you have the time, try to implement several different ways. all lisp, DOM (the foreign ontology in lisp classes), all binary (possibly with CLOS/MOP).
19:01:59
Bike
if you do want unboxed values in a custom data type, defstruct is more likely to work, because redefining it is undefined.
19:02:23
elderK
It would be interesting. But I really, really want to avoid forcing people to manually box/unbox from Lisp values to the ontological ones.
19:02:55
elderK
so that you have a layer that is like (write 'type value) which really just does the boxing and stuff behind the scenes.
19:05:02
elderK
I guess you could also define a standard protocol for "boxing" and "unboxing" things. Like, unwrap <ontological-instance> and wrap <ontological-type-name> value
19:08:25
elderK
I really do wonder how like, things like Genera and stuff /did/ do efficient binary IO and stuff. I mean, for processing network formats and all kinds of things, I imagine you'd want the mapping to be as simple and fast as possible.
19:11:27
pjb
see https://github.com/informatimago/lisp/blob/master/common-lisp/data-encoding/data-encoding.lisp#L75
19:12:12
pjb
basically, yes, lisp machines had low-level tricks (kind of like C casting, but with safety, since the memory slots are still type tagged).
19:13:11
pjb
Because packages use lists of strings, not of symbols. But most package operators accept string designators, ie. string, symbols or characters, to designate strings.
19:14:27
pjb
So, we don't use normal symbols in defpackage, because they are interned in the current package, and when you use the defined package, there's a lot of collisions.
19:15:16
pjb
Instead, we may use uninterned symbols, but I find it #:ugly, or keywords, but this fills the :keyword package. Finally strings have the advantage of uninterned symbols, but they're "NICE".
19:15:33
elderK
The main problem I see from using strings, is that it's up to settings as to whether 'foo is "FOO" or "foo"
19:16:02
pjb
Now there's the consideration about readtable case. Using uninterned symbols has the advantage of using the readtable case. While using strings requires typing the absolute name case.
19:16:59
pjb
I learned programming in the 70s, there were a lot of terminals and printers with only upper case then.
19:18:00
pjb
Honestly, this makes a difference only when used in implementations with a "modern" mode implemented badly. Ie. with allegro CL (mlisp instead of alisp). clisp has also a modern mode (per package) so it doesn't break.
19:19:44
pjb
Since I do a lot of copy-paste from my repl to irc, I've this in my rc-file, so: (intern "FOO") #| --> foo ; :internal |# <--
19:57:59
pjb
elderK: just beware that sometimes there's buggy code that use format ~A to build symbol names from symbols or other, and therefore you'll get downcased symbol names! Just non-buggy libraries instead ;-)
20:06:24
elderK
Xach: How long do you think it will be before package-local nicknames are universal? :D Alternatively, how important it is these days to really "support" implementations other than say, SBCL or ECL?
20:07:23
Xach
elderK: I think there needs to be a next round of planning and implementation of package-local nicknames. i don't think the current crop are adequate.
20:07:54
Xach
I have a personal idea of how it can be done better but I have to get it out into the world for refinement (and rejection or acceptance)
20:08:35
elderK
Xach: Yeah, I'd definitely be interested in hearing shortcomings of say, SBCL's method
20:09:28
Xach
Bike: "adding" isn't a good word. i mean using sb-ext, or ext, or sys, or whatever, to make use of them.
20:09:57
Xach
I want portable use of package-local nicknames without compatibility layers, and I think it can be done.
20:10:19
Xach
Bike: I think they can be done with standard CL functions with extra arguments and return values.
20:12:37
Bike
no, they're allowed as an extension, i think, but adding return values as an extension is specifically not allowed
20:13:51
Xach
Bike: can't get any clearer than that. well, then my proposal wouldn't conform to that. bummer.
20:14:53
Bike
well, i think package local nicknames would mostly just be used in the form of :local-nicknames or whatnot in defpackage, which doesn't really require a compatibility layer
20:32:22
elderK
I have a question about the reader. Particularly with macro characters. The cylicness of the reader kind ofhurts my brain.
20:32:55
elderK
Like, let's say the reader... is reading. And the user has like, set a macro character. Depending on situation, the reader will invoke the function for that macro character, to handle the read.
20:33:13
elderK
But is the "read table" in use by that "macro character function" the same as the read table that was in effect at the time of reading?
20:33:35
elderK
Or is the read-table in effect in the macro function, the one that was in effect when the macro function was set / defined?
20:34:33
elderK
It kind of seems like, to "read", you need a functional implementation, because of reader macros...
20:44:41
Bike
elderK: it just uses what's in *readtable*, so basically what's in effect at time of reading.
20:45:29
White_Flame
a reader macro can either recursively call READ which uses the readtable & normal reader rules, or it can consume character by character and do whatever it wants locally
20:50:32
elderK
Iono. It just makes my brain kind of explode. Like, I think there must be some "basic hardwired lexer" and stuff that powers the "core" language.
20:51:57
White_Flame
pfdietz: sure, but that's not what deals with the input stream & dispatching on characters. It's Lisp code, it can do whatever it wants ;)
20:52:01
pfdietz
I am somewhat ambivalent about customizing the reader. It makes it harder to write programs that grovel over general code.
20:53:15
elderK
I just wonder how such a... reader is implemented. Like, let's say we "compile" our stuff to some form we can more easily execute. Say, byte code or something. Or maybe we just, literally remember the AST and walk it to execute.
20:53:49
elderK
So, we'd have to have a table that'd contain a pointer to the function to be invoked to do the reading. Since that can be redefined, we have to be able to decide whether to call some built-in compiled function to do it. Or, run the "potentially interpreted one"
21:01:14
White_Flame
did the Lisp reader exist before the concept of "lexer" was established as we know it today?
21:03:20
elderK
I'm just trying to... match things up. I mean, how can reading be completely independent from... stuff. It can't just be a "typical lexer / scanner" because macro characters require us to invoke some function, that is potentially set by user.
21:03:47
elderK
so, in an interpreter, that would require us to then interpret that macro function so to continue the read.
21:07:29
Bike
and funcall doesn't need to invoke the reader in any way (unless you're calling, like, READ itself)
21:08:00
no-defun-allowed
All reasonable Lisps use recursive descent which makes these things quite trivial.
21:08:58
elderK
Bike: Right. But that implies that we can somehow "funcall" a function. Which means we need to have been able to read it, and somehow store it in a form we can call.
21:09:57
elderK
Yes, but the only way it could be there in memory, the function, is if we already read it, no?
21:10:13
no-defun-allowed
It's more appropriate than lexing/parsing, and I've heard it called that before.
21:11:27
Xach
elderK: it would still be pretty normal to separate reading and evaluation in not-lisp
21:11:48
no-defun-allowed
For example, the Unix Hater's Handbook compares it to the contextual grammar of C++, calling Lisp's parser "recursive descent, meaning you can write it on a piece of paper" from memory.
21:12:09
elderK
no-defun-allowed: There's parsing, then there's lexing. to me, they are independent.
21:12:16
Xach
evaluation is what makes the system do stuff, like create function objects and maybe associate them with a global name.
21:13:00
elderK
Bike: Right. My main problem isn't with the fact that, lexing/parsing is basically "all in one." I've seen lexerless parsers.
21:14:10
Bike
being "dynamic" pretty much just means the table is an object in memory rather than an off-line thing.
21:14:55
elderK
Yeah, I get that. I'm just thinking of the necessity to have some way to actually read code in the first place, so that the user's code that sets a new reader macr-function can be understood and patched in :P
21:16:34
elderK
Bike: Which is to say, when you start the Lisp implementation - say from the standard image - everything is in place to read and understand standard Lisp. If it reads stuff, evaluates stuff, that changes that, well, okay, because it started off with the base syntax.
21:21:33
elderK
I really want to learn how to make a lisp implementation. Toy to start with, but increasing complex and non-toy-ish.
21:21:47
elderK
But I'm not really sure how to best start. Finishing LiSP and studying RABBIT, maybe,
21:23:21
Xach
elderK: Lisp in Small Pieces is a good book on the topic. i like the progression it follows from simple to complicated
21:23:45
Bike
i mean a basic metacircular interpreter is like a page long, if you just have scheme special oeprators
21:24:38
elderK
:) I wonder if I can do like, all the stuff it covers, but in CL rather than Scheme.
21:25:51
elderK
I also need to learn how to do ... like, usual lexing in Lisp. I'm used to implementing lexers in C using transition tables and stuff. Basically state[current_state][input_category] is the next state.