libera/#commonlisp - IRC Chatlog
Search
20:10:19
nij-
Can we hack the evaluating scheme of CL? For example, when evaluate a list that starts with a specific type of data (e.g. a keyword), can I make it do whatever I want (e.g. test if the first parameter is a plist, and get the corresponding field if so)?
20:28:29
White_Flame
you can write your own evaluator, or you can edit the existing implementation-specific editor
20:29:05
White_Flame
you can shadow CL:DEFUN or CL:LAMBDA or use alternatives, and have it walk and transform the code for you
20:29:32
White_Flame
the "evaluator" is generally machine code generated by the lisp compiler written in lisp
20:30:08
White_Flame
some runtimes have the garbage collector, early bootstrap, and OS interfacing written in C
20:30:36
White_Flame
it's not standard common lisp, it's the implementations' own specific implementation
20:31:12
White_Flame
it's much easier to do so in a simple interpreted lisp; the compiler adds a lot of complexity
20:32:00
nij-
I mean.. let's assume we have a CL implementation that's written in C (or any other lang)..
20:32:03
White_Flame
it would be early passes of it where you would choose to transform things that are illegal by the standard, into legal expansions
20:32:35
White_Flame
but you can do the exact same thing with your own custom DEFUN/LAMBDA/etc and a code-walking transformer
20:33:41
aeth
if (:foo then do your regular logic, otherwise carry on. Of course, this is fragile, e.g. keyword:foo, ::foo, etc
20:33:44
White_Flame
how would you do it at the reader level? only evaluated s-expressions would want the transformations, and the reader can't distinguish code from data yet
20:35:00
White_Flame
then get into the compiler or code walker of your implementation, and see where you can have it perform that transformation on all code
20:35:46
White_Flame
there's nothing stopping you from redefining any lisp code in the lisp implementation
20:37:35
nij-
But I mean, a reason that makes lisp "hackable" is that much hacking can happen within the target lang.
20:38:02
White_Flame
but changing the definition of what's legal to evaluate means getting into the guts of it
20:38:27
White_Flame
luckily, this is all in CL, as M-. traversable as any user code, and as re-definable as any user code. It's just lisp
20:39:40
nij-
The pythoners would also fringe upon the fact that we can change the reader in the target lang.
20:39:42
White_Flame
I had to remove a package lock in SBCL, but that is just a function replacement from my user code early in loading my project
20:40:23
White_Flame
so it's not even a permanent change to my on-disk lisp image; it only affects the language when my project is loaded
20:40:58
White_Flame
it's 100% possible, 100% not legal, and somewhat% portable among the same implementation
20:41:35
White_Flame
dont' be afraid to hack it, that's how you learn the internals of the implementation, which gives you a broader understanding of what goes on in lisps
20:41:49
Bike
as for why, i would sum that up by saying that having that capability would make compilation hard to impossible
20:42:40
White_Flame
there's no separate language, or separate running instance, that has the compiler code; it's all in the same image
20:42:53
White_Flame
mutate it to your heart's content. you can always restart from the unchanged reference code
20:43:40
White_Flame
embedded cl I think might have more C, but no reason it can't be as much CL in the compiler as any other lisp. THe target is just C instead of machine code
20:44:12
White_Flame
I think there's some limitation that you're imagning that you're assuming, but simply isnt' there
20:45:18
White_Flame
if you cannot change the implementation (eg, if it's not written in CL), then you'll have to write things on top of it
20:45:44
White_Flame
again, providing your own wrapper macros, shadowing lambda/defun/etc, and generating that which the foreign unchangeable compiler can grok
20:46:30
White_Flame
and taht's what's so confusing, is that you're proposing a mostly non-existent scenario that's much harder to work with than just a standard CL written in CL
20:47:09
White_Flame
but it's generally easiest to implement in CL, and it would be difficult to find a CL where the compiler is not written in CL
20:48:04
White_Flame
and then I believe the same answer holds: if for some reason you cannot change the implementation, you must work around it
20:49:06
White_Flame
you're not "in" the implementation if it's foreign, non-lisp code, in terms of what your code affects
20:50:12
White_Flame
if CL:DEFUN, for instance, is not a lisp function, but a direct call to some C code at some fixed location, then you're not going to be redefining CL:DEFUN from within lisp. You can shadow it with your own package:DEFUN, though
20:50:36
White_Flame
there is no standard interface for redefining the evaluator, it's an implementation detail
20:50:52
White_Flame
because it might be a compiler, it might be an interpreter, it might be a bytecode interpreter, and these have implications all over the place
20:51:39
White_Flame
but now that you've defined your terms better, yes if you cannot have access to the compiler's source code, you cannot change the compiler
20:52:21
White_Flame
the answer is yes, for the vast majority of CLs. it is not standard, as I said up front
20:52:55
White_Flame
but "theoretically if the evaluator is not written in CL, and thus CL can't access it", then no, it cannot change it as it cannot access it
20:54:37
White_Flame
you're talking about hacking a _foreign_ evaluator, in some unknown language, with unknown mechanism, from another language
20:55:22
White_Flame
it's like saying "it'd be cool if CL could hack whatever microcode is running on whatever CPU it's running on"
20:56:05
White_Flame
for every major instance of CL, the evaluator is written in CL and is as accessbile & hackable as any other user code
20:57:02
nij-
I don't mind the major instance of CL. I mind CL as a language. And I apologize if I didn't express well enough.. I don't mean to undo the conversation.
20:57:41
White_Flame
if it specified ways to change it, then implementations would be locked to the low-level evaluation assumptions of the 1980s
20:59:16
White_Flame
it's completely unspecified by the standard, and provided by the implementation
20:59:39
White_Flame
as such, the standard does not specify _anything_ about what's in the environment object, and you cannot change it without being non-portable
21:00:28
White_Flame
a singular evaluator object would be more comparable to an interpreter, because it would have to take s-expressions
21:00:49
White_Flame
a compiler would have to do weird and expensive dispatching to cache compiled forms if it obeyed such an evaluator API
21:01:02
nij-
Hmm.. are you saying what I been using is not a repl, but a rcpl (read-compile-print-lop)?
21:01:25
White_Flame
which of course is expensive cmpared to compile once and execute as many times as you want
21:02:02
White_Flame
so if you have an "evaluator" object, which by the vocabulary lisp would take s-expressions, then jacking in a compiler would have impedance mismatch with the API's assumptions
21:02:15
White_Flame
and this is a "lock-in" of overspecifying the implementation, as I was talking about above
21:02:58
Bike
there's a balance between exposing things to the user and overly constraining the implementation
21:03:08
Bike
there used to be an *evalhook*, which you could bind to some function that would do evaluation
21:03:12
White_Flame
and how much nonsense would it add to your workload if you were writing a simple interpreted CL?
21:03:25
Bike
but that pretty much required lisp to do an interpreter in a way that's very constraining and slow
21:03:57
White_Flame
nij-: interface = the set of functions, datastructure, & call ordering you need to make it owkr
21:04:53
Bike
there are some existing hooks into the compilers, like macros. that has an interface via defmacro, macroexpand, bla bla
21:05:18
White_Flame
the CL standard was written to allow implementations to make smart compilers, dumb compilers, interpreters, and support a whole host of variety of hardware as existed back then
21:06:09
White_Flame
making an assumption that a compiler has to work like X and must have Y knobs to help implement your feature constrains the design of the implementation
21:06:51
White_Flame
"the compiler has this hook/knob/etc" is assumptions about the design of the compiler
21:07:02
nij-
But I'm thinking of a possibility for the user to hack it without going into the source.
21:07:17
White_Flame
what's nice about (declare (optimize (speed..) (safety..) ..)) etc is that it doesn't mandate any specifics
21:07:29
Bike
nij-: but the compiler would have to support some mechanism to let the user do that, is the point
21:07:43
White_Flame
ahd thus compilers can be wildly different, they could JIT, they could do whatever, and the same options still apply
21:08:04
White_Flame
nij-: hacking _is_ getting into the source. what you're talking about is configuration
21:08:35
White_Flame
basically turning the spec into a dynamically editable language specification. Which is fine for research languages, and many do, but a spec must draw lines somewhere
21:09:55
White_Flame
oh asking me? yeah, when creating a new language, many things are dynamically configured, until you decide what features you want to standardize on
21:10:27
White_Flame
and then you can get a lot of holistic interactions between solidly designed components instead of everything having to interact with moving targets
21:11:22
White_Flame
and that's not something languages include in their core, because then what do you design to?
21:12:21
Bike
more flexible lisps in this respect include maru, where eval is a generic function, and kernel, where you can define your own special operators
21:12:36
White_Flame
but still, if you have a CL running in front of you, you can code in it, or hack the CL implementation itself to bend to your will & features ;)
21:12:43
Bike
so you could look into those if you're interested, but they are, as white flame said, pretty much research languages rather than something usable for your program
21:13:25
White_Flame
the slightest change that you make to the CL evaluator would semantically make a whole new language
21:13:29
nij-
I wish to be able to switch to another CL-package, and voila, the evaluator has now changed.
21:14:43
Bike
because you clearly have something you want to specify, it's not like it would be done for you
21:14:59
White_Flame
and again, you can shadow DEFUN/LAMBDA/etc in your own package and run through your own transforms, and keep it 100% portable CL
21:16:36
Bike
say there was a generic function FUNCTION-LOOKUP, that took a function name and an environment, and looked up the name to return the function
21:17:10
White_Flame
we'll conveniently ignore the case of looking up FUNCTION-LOOKUP itself, right? ;)
21:17:33
Bike
then you defined a method (defmethod function-lookup ((name keyword) env) (lambda (plist) (getf plist name)))
21:17:49
Bike
so then (:foo a) evaluates to ((lambda (plist) (getf plist :foo)) a), which is i think what you had in mind
21:18:29
Bike
the user could define whatever method on function-lookup at any time, so the compiler can't actually look up anything up ahead of time
21:19:14
Bike
this isn't the only possible interface, obviously, just the first one that popped into my head
21:21:05
White_Flame
declarations, macroexpansions, etc, are all used by the compiler at the current state that they're set when the compiler is invoked
21:21:24
White_Flame
if you change those things in the environment, the compiled code does not change to reflect them
21:21:36
Bike
the metaobject protocol is a good example of the effort that goes into making some part of the language more flexible
21:22:26
Bike
it lets you warp CLOS into a whole lot of different things, if you want. i've implemented something like python decorators using it, cl-json has self/javascript-style prototype programming
21:22:28
White_Flame
and what is set at compile-time, and what is undefined consequence to change after it's been used, etc, is tackled a lot by the CL specification
21:22:50
Bike
but they had to put a lot of thought into how the MOP interface would be designed, so that it isn't so free that the implementation has to do everything slowly
21:23:18
Bike
and also so that users could use objects without having to fear that some extension would pull the carpet out from under them
21:25:13
nij-
What I want is a way for the target lang to send a message to the host lang that changes stuff there.
21:26:26
White_Flame
it's good for flexibility, it's bad for ecosystem & portability, it's bad for practical implications of speed & scale
21:27:18
White_Flame
and again I think the optimization declarations in CL are a great example of sending a level of information to the compiler that is applicable for any architecture the implementation happens to use
22:12:50
jcowan
I think this is a complete list of the types that have representations in standard syntax (disregarding sharpsign-dot expressions): numbers, symbols, conses, strings, characters, simple vectors, and bit vectors. Am I missing anything?
22:35:31
aeth
keywords are kind of sort of different from symbols in that they're :foo instead of keyword:foo so you could include them on the list
22:36:10
aeth
since you could view them as being of a subtype with its own representation in standard syntax
22:37:11
aeth
along those lines, you may want to consider complex numbers with the #C(...) syntax separate from the other numbers
22:39:36
aeth
(which is a difference from Scheme, where they have a very number-y representation, not unlike the rationals)
23:05:12
White_Flame
you'd also split up float vs int with that syntax-level distinction, but I don't think that's as meaningful to the list
23:33:29
seok
Anyone know whether zpb-ttf can return all character code points available in the ttf or I have to implement a loop looping through all code points with glyph-exists-p?
23:57:15
jcowan
Okay, thanks, so pathnames are included. Random-states are readable/printable, but have no specific syntax.