libera/#commonlisp - IRC Chatlog
Search
9:42:16
pjb
pve: alternatively, you could store the hash-table in the symbol-plist of the name of the namespace.
11:52:55
jcowan
beach: The term "S-expression" seems to be incurably ambiguous: it can mean an internal representation or an external one. (The question of what these representations represent I leave to Kantians.)
12:06:04
hexology
maybe it's good to come up with an alternative term for the "internal" variety (hy calls it a "model"), then define terms unambiguously and adhere strictly to the distinction
12:06:41
hexology
i can't speak for lisp specifically, but i find that this is a useful communication technique in fields with confusing terminology. it's a local lexical binding but for language :)
12:09:37
random-nick
why would s-expression mean an internal representation? as far as I know those are called lists
12:11:05
_death
lisp terminology has been around for a long time, so a person who has been sufficiently exposed to it has a good sense of the channel transition matrix for practical block lengths
12:13:48
hayley
Using the definition "an S-expression is either an atom or a CONS cell ..." how is an "S-expression" different to, say, an "object"?
12:13:49
beach
I find it interesting that some people have opinions about pedagogy, yet instead of using those opinions to teach newbies about Common Lisp, they seem to wait until someone else does, only to complain about the pedagogical techniques used by that someone else.
12:16:52
hayley
Just asking, since a few days ago I got into a "discussion" which involved confusing internal and external representations, and the two "layers" of Lisp syntax, and would like to avoid having that experience again.
12:22:32
hayley
In particular, someone was trying to tell me that one could encode any data types using S-expressions (with just lists, symbols and numbers and such). They are correct, but they wrongly claimed it was specific to S-expressions; I don't think it is, because one would need their own "evaluator" of sorts to produce a graph of objects from the list structure read. One could do just the same with any other syntax.
12:25:29
hexology
beach: i hope i didn't give you the impression that i was complaining. that was not my intention.
12:26:40
beach
I can't say I much care anymore. I am getting used to this state of things. Or rather, I now know what to expect.
12:30:38
hayley
On the other hand, the S-expression syntax (or first layer of syntax) is quite permissive, and one can read forms that don't make any sense as Lisp code. But I wouldn't be so bothered if I had to encode everything as a function call, if I had to use some other syntax. Say, writing (person "Hayley" (in "#lisp")) isn't profoundly better than person("Hayley", in("#lisp")) or something like that.
12:31:06
hayley
(We could indeed use reader macros, as is done in Cleavir, to introduce syntax that denotes objects, but I had the sense that I was talking to (or, "past") someone who had an idealised notion of "Lisp" rather than any particular language, because they couldn't distinguish between internal and external representations.)
12:35:01
hexology
hayley: i think the thing about s-expressions is that a(b, c, d) is a very unnatural way to represent a list in writing, while (a b c d) more or less looks like "a list of things"
12:36:15
hayley
In this example, person(...) does not denote a list, though, it denotes some information about a person. A list is one of many kinds of things one might want to represent.
12:38:48
hexology
i don't hate the clojure mindset here, of breaking the equivalence between s-expr "lists" and null-terminated-singly-linked-lists
12:39:11
hexology
re: person(...) vs. (person ...), it was a pretty eye-opening moment when i realized that you couldn't effectively write a static analyzer (e.g. a tree-sitter grammar) for an s-expr language and expect to be able to highlight function calls, provide sensible indenting, etc. without encoding lots of special cases
12:39:43
hexology
static analysis seems to be the biggest deficiency of highly-uniform syntax. you need some kind of runtime information and/or a big list of special cases.
12:40:03
hayley
Still, though, (person "Hayley") is a list, which might denote information about myself under some interpretation. In a way the first interpretation is just "syntax", and the latter is "semantics".
12:42:01
hayley
With regards to analysis: you "just" need to reason about the second (or now "semantic") layer of syntax. I don't know how easy that would be with libraries that seem designed for a single-layer syntax. But, in a way, an evaluator needs to "parse" the list structure to figure out what action it should take; a syntax highlighter should do the same.
12:45:19
hexology
vim has a pretty clever "lispwords" system for indenting lisp-like languages, idk if emacs has an equivalent
12:46:41
hayley
A friend wanted to write an implementation of a language using S-expressions for some assignment work, but had to produce a single-layer syntax. They sent me the parser code and it was quite nasty. Imagine having separate rules for DEFUN forms, LET forms, and such. Though I've seen similar in published code, unfortunately.
12:48:40
hayley
(And when that parser is written by hand in an unsafe language, fuzzing will find somewhat amusing crashes most of the time, because of the sheer number of cases that one has to implement without making mistakes.)
12:53:31
morganw
I think emacs is the opposite, the built-in parser is originally for lisp and things get more difficult the further removed you get from lisp.
12:53:37
jcowan
beach: I intended no offense, but in fact one does not need to be a carpenter to point out that a table wobbles.
12:55:47
hayley
morganw: Suppose I want to model decay, and I write something like (let ((lambda 0.5)) (expt lambda time)). The "lambda" just before "0.5" is highlighted like it starts a LAMBDA form, when it doesn't.
12:56:49
hayley
(The highlighting seems correct in emacs-lisp-mode, but I still get information on the LAMBDA special form in the mini-buffer, which is wrong.)
13:00:16
hayley
(Or perhaps I should have written (let ((lambda 0.5)) (exp (* -1 lambda time))) but that doesn't really matter.)
13:02:27
morganw
The documentation mode is a separate part, I'd guess it isn't using all of the parser information.
13:47:59
White_Flame
rendar: for playing around with the reader to see what it does, (read-from-string "'(foo #.(+ 1 1))") might be an easier interface, which streams from the string you give. As well as (setf *print-pretty* nil) so you can see the raw gory details printed
19:08:48
pjb
rendar: #. reads an expression, evaluates it, and returns its result as the object read by #.
20:10:19
nij-
Can we hack the evaluating scheme of CL? For example, when evaluate a list that starts with a specific type of data (e.g. a keyword), can I make it do whatever I want (e.g. test if the first parameter is a plist, and get the corresponding field if so)?
20:28:29
White_Flame
you can write your own evaluator, or you can edit the existing implementation-specific editor
20:29:05
White_Flame
you can shadow CL:DEFUN or CL:LAMBDA or use alternatives, and have it walk and transform the code for you
20:29:32
White_Flame
the "evaluator" is generally machine code generated by the lisp compiler written in lisp
20:30:08
White_Flame
some runtimes have the garbage collector, early bootstrap, and OS interfacing written in C
20:30:36
White_Flame
it's not standard common lisp, it's the implementations' own specific implementation
20:31:12
White_Flame
it's much easier to do so in a simple interpreted lisp; the compiler adds a lot of complexity
20:32:00
nij-
I mean.. let's assume we have a CL implementation that's written in C (or any other lang)..
20:32:03
White_Flame
it would be early passes of it where you would choose to transform things that are illegal by the standard, into legal expansions
20:32:35
White_Flame
but you can do the exact same thing with your own custom DEFUN/LAMBDA/etc and a code-walking transformer
20:33:41
aeth
if (:foo then do your regular logic, otherwise carry on. Of course, this is fragile, e.g. keyword:foo, ::foo, etc
20:33:44
White_Flame
how would you do it at the reader level? only evaluated s-expressions would want the transformations, and the reader can't distinguish code from data yet
20:35:00
White_Flame
then get into the compiler or code walker of your implementation, and see where you can have it perform that transformation on all code
20:35:46
White_Flame
there's nothing stopping you from redefining any lisp code in the lisp implementation
20:37:35
nij-
But I mean, a reason that makes lisp "hackable" is that much hacking can happen within the target lang.
20:38:02
White_Flame
but changing the definition of what's legal to evaluate means getting into the guts of it
20:38:27
White_Flame
luckily, this is all in CL, as M-. traversable as any user code, and as re-definable as any user code. It's just lisp
20:39:40
nij-
The pythoners would also fringe upon the fact that we can change the reader in the target lang.
20:39:42
White_Flame
I had to remove a package lock in SBCL, but that is just a function replacement from my user code early in loading my project
20:40:23
White_Flame
so it's not even a permanent change to my on-disk lisp image; it only affects the language when my project is loaded
20:40:58
White_Flame
it's 100% possible, 100% not legal, and somewhat% portable among the same implementation
20:41:35
White_Flame
dont' be afraid to hack it, that's how you learn the internals of the implementation, which gives you a broader understanding of what goes on in lisps
20:41:49
Bike
as for why, i would sum that up by saying that having that capability would make compilation hard to impossible
20:42:40
White_Flame
there's no separate language, or separate running instance, that has the compiler code; it's all in the same image
20:42:53
White_Flame
mutate it to your heart's content. you can always restart from the unchanged reference code
20:43:40
White_Flame
embedded cl I think might have more C, but no reason it can't be as much CL in the compiler as any other lisp. THe target is just C instead of machine code
20:44:12
White_Flame
I think there's some limitation that you're imagning that you're assuming, but simply isnt' there
20:45:18
White_Flame
if you cannot change the implementation (eg, if it's not written in CL), then you'll have to write things on top of it
20:45:44
White_Flame
again, providing your own wrapper macros, shadowing lambda/defun/etc, and generating that which the foreign unchangeable compiler can grok
20:46:30
White_Flame
and taht's what's so confusing, is that you're proposing a mostly non-existent scenario that's much harder to work with than just a standard CL written in CL
20:47:09
White_Flame
but it's generally easiest to implement in CL, and it would be difficult to find a CL where the compiler is not written in CL
20:48:04
White_Flame
and then I believe the same answer holds: if for some reason you cannot change the implementation, you must work around it
20:49:06
White_Flame
you're not "in" the implementation if it's foreign, non-lisp code, in terms of what your code affects
20:50:12
White_Flame
if CL:DEFUN, for instance, is not a lisp function, but a direct call to some C code at some fixed location, then you're not going to be redefining CL:DEFUN from within lisp. You can shadow it with your own package:DEFUN, though
20:50:36
White_Flame
there is no standard interface for redefining the evaluator, it's an implementation detail
20:50:52
White_Flame
because it might be a compiler, it might be an interpreter, it might be a bytecode interpreter, and these have implications all over the place
20:51:39
White_Flame
but now that you've defined your terms better, yes if you cannot have access to the compiler's source code, you cannot change the compiler
20:52:21
White_Flame
the answer is yes, for the vast majority of CLs. it is not standard, as I said up front
20:52:55
White_Flame
but "theoretically if the evaluator is not written in CL, and thus CL can't access it", then no, it cannot change it as it cannot access it
20:54:37
White_Flame
you're talking about hacking a _foreign_ evaluator, in some unknown language, with unknown mechanism, from another language
20:55:22
White_Flame
it's like saying "it'd be cool if CL could hack whatever microcode is running on whatever CPU it's running on"
20:56:05
White_Flame
for every major instance of CL, the evaluator is written in CL and is as accessbile & hackable as any other user code
20:57:02
nij-
I don't mind the major instance of CL. I mind CL as a language. And I apologize if I didn't express well enough.. I don't mean to undo the conversation.
20:57:41
White_Flame
if it specified ways to change it, then implementations would be locked to the low-level evaluation assumptions of the 1980s
20:59:16
White_Flame
it's completely unspecified by the standard, and provided by the implementation
20:59:39
White_Flame
as such, the standard does not specify _anything_ about what's in the environment object, and you cannot change it without being non-portable
21:00:28
White_Flame
a singular evaluator object would be more comparable to an interpreter, because it would have to take s-expressions
21:00:49
White_Flame
a compiler would have to do weird and expensive dispatching to cache compiled forms if it obeyed such an evaluator API
21:01:02
nij-
Hmm.. are you saying what I been using is not a repl, but a rcpl (read-compile-print-lop)?
21:01:25
White_Flame
which of course is expensive cmpared to compile once and execute as many times as you want
21:02:02
White_Flame
so if you have an "evaluator" object, which by the vocabulary lisp would take s-expressions, then jacking in a compiler would have impedance mismatch with the API's assumptions
21:02:15
White_Flame
and this is a "lock-in" of overspecifying the implementation, as I was talking about above
21:02:58
Bike
there's a balance between exposing things to the user and overly constraining the implementation
21:03:08
Bike
there used to be an *evalhook*, which you could bind to some function that would do evaluation
21:03:12
White_Flame
and how much nonsense would it add to your workload if you were writing a simple interpreted CL?
21:03:25
Bike
but that pretty much required lisp to do an interpreter in a way that's very constraining and slow
21:03:57
White_Flame
nij-: interface = the set of functions, datastructure, & call ordering you need to make it owkr
21:04:53
Bike
there are some existing hooks into the compilers, like macros. that has an interface via defmacro, macroexpand, bla bla
21:05:18
White_Flame
the CL standard was written to allow implementations to make smart compilers, dumb compilers, interpreters, and support a whole host of variety of hardware as existed back then
21:06:09
White_Flame
making an assumption that a compiler has to work like X and must have Y knobs to help implement your feature constrains the design of the implementation
21:06:51
White_Flame
"the compiler has this hook/knob/etc" is assumptions about the design of the compiler
21:07:02
nij-
But I'm thinking of a possibility for the user to hack it without going into the source.
21:07:17
White_Flame
what's nice about (declare (optimize (speed..) (safety..) ..)) etc is that it doesn't mandate any specifics
21:07:29
Bike
nij-: but the compiler would have to support some mechanism to let the user do that, is the point
21:07:43
White_Flame
ahd thus compilers can be wildly different, they could JIT, they could do whatever, and the same options still apply
21:08:04
White_Flame
nij-: hacking _is_ getting into the source. what you're talking about is configuration
21:08:35
White_Flame
basically turning the spec into a dynamically editable language specification. Which is fine for research languages, and many do, but a spec must draw lines somewhere
21:09:55
White_Flame
oh asking me? yeah, when creating a new language, many things are dynamically configured, until you decide what features you want to standardize on
21:10:27
White_Flame
and then you can get a lot of holistic interactions between solidly designed components instead of everything having to interact with moving targets
21:11:22
White_Flame
and that's not something languages include in their core, because then what do you design to?
21:12:21
Bike
more flexible lisps in this respect include maru, where eval is a generic function, and kernel, where you can define your own special operators
21:12:36
White_Flame
but still, if you have a CL running in front of you, you can code in it, or hack the CL implementation itself to bend to your will & features ;)
21:12:43
Bike
so you could look into those if you're interested, but they are, as white flame said, pretty much research languages rather than something usable for your program
21:13:25
White_Flame
the slightest change that you make to the CL evaluator would semantically make a whole new language
21:13:29
nij-
I wish to be able to switch to another CL-package, and voila, the evaluator has now changed.
21:14:43
Bike
because you clearly have something you want to specify, it's not like it would be done for you
21:14:59
White_Flame
and again, you can shadow DEFUN/LAMBDA/etc in your own package and run through your own transforms, and keep it 100% portable CL
21:16:36
Bike
say there was a generic function FUNCTION-LOOKUP, that took a function name and an environment, and looked up the name to return the function
21:17:10
White_Flame
we'll conveniently ignore the case of looking up FUNCTION-LOOKUP itself, right? ;)
21:17:33
Bike
then you defined a method (defmethod function-lookup ((name keyword) env) (lambda (plist) (getf plist name)))
21:17:49
Bike
so then (:foo a) evaluates to ((lambda (plist) (getf plist :foo)) a), which is i think what you had in mind
21:18:29
Bike
the user could define whatever method on function-lookup at any time, so the compiler can't actually look up anything up ahead of time
21:19:14
Bike
this isn't the only possible interface, obviously, just the first one that popped into my head
21:21:05
White_Flame
declarations, macroexpansions, etc, are all used by the compiler at the current state that they're set when the compiler is invoked
21:21:24
White_Flame
if you change those things in the environment, the compiled code does not change to reflect them
21:21:36
Bike
the metaobject protocol is a good example of the effort that goes into making some part of the language more flexible
21:22:26
Bike
it lets you warp CLOS into a whole lot of different things, if you want. i've implemented something like python decorators using it, cl-json has self/javascript-style prototype programming
21:22:28
White_Flame
and what is set at compile-time, and what is undefined consequence to change after it's been used, etc, is tackled a lot by the CL specification
21:22:50
Bike
but they had to put a lot of thought into how the MOP interface would be designed, so that it isn't so free that the implementation has to do everything slowly
21:23:18
Bike
and also so that users could use objects without having to fear that some extension would pull the carpet out from under them
21:25:13
nij-
What I want is a way for the target lang to send a message to the host lang that changes stuff there.
21:26:26
White_Flame
it's good for flexibility, it's bad for ecosystem & portability, it's bad for practical implications of speed & scale
21:27:18
White_Flame
and again I think the optimization declarations in CL are a great example of sending a level of information to the compiler that is applicable for any architecture the implementation happens to use