freenode/#lisp - IRC Chatlog
Search
2:58:51
beach
pjb: A hard question to answer. During bootstrapping, I compile (using the SICL compiler) files containing definitions of macros, normal functions, generic functions, classes, etc. Then I load the compiled files into first-class global environments and the resulting code is executable in the host Common Lisp system (using an interpreter for intermediate code). And I generate native code, but there is no system for that code to
3:01:17
beach
You can say that, yes. I tried turning the intermediate code into Common Lisp code for execution in the host, but the SBCL compiler was unable to handle the size of the resulting code.
3:01:53
beach
So it means I can test a lot of the SICL infrastructure, including the generation of discriminating functions and such, but I can't test the native-code generator yet.
3:04:45
beach
Plus, Bike and karlosz are working on Cleavir compiler optimizations that I have not integrated into SICL yet, but they are part of Clasp now.
3:05:19
beach
And heisig is working on a very nice implementation of the sequence functions, using sealed classes and generic functions.
3:05:59
beach
His functions are portable and they often perform better than the native ones in SBCL.
3:07:23
beach
And scymtym continues to improve Eclector. It is an extremely useful library module now.
3:08:21
beach
Trucler (compile-time lexical environments) is not moving, but appears to be working as advertised. That was also heisig's work.
3:09:37
beach
Coming soon: Extraction of first-class global environments into a separate repository, and a version of PRINT-OBJECT that can be customized by the client.
3:11:56
beach
Coming a bit later: Separate repository of the Cleavir compiler framework, including the representation of abstract syntax trees and intermediate representation.
3:14:57
beach
All those modules are possible to use in order to create a complete Common Lisp implementation. But, most existing implementations can't use them because of the way these implementations are built.
3:14:58
beach
The modules often require the full Common Lisp language, and that is incompatible with the way most Common Lisp implementations are gradually bootstrapped from a subset of the language.
3:15:48
beach
So at some point, I need to turn the SICL bootstrapping technique into something that can be used by other Common Lisp implementations as well.
3:26:37
LdBeth
I’d imagine that if SICL can access host lisp’s functions and data it is already ideal to be used for all sorts of code analysis
3:34:31
beach
But yeah, the Cleavir-based compiler can be customized, so it can be used to turn source code into an abstract syntax tree, using the macro definitions of your choice, including the host ones, if that is what you prefer.
3:35:27
beach
Those macro definitions are just part of a first-class global environment that the compiler works against, so it is even possible to stick (say) ECL macros in such an environment and compile ECL code in SBCL.
3:45:56
LdBeth
I don’t have exact idea, but it seems possible to be used to develop a distributed computing/compiling environment that multiple instances of lisp can sharing compiling environment via network
3:47:32
beach
That sounds possible. But I am not sure what it would be used for. It is also tricky because of compile-time execution, so a lot of sandboxing must be put in place.
3:52:07
beach
Aside from creating a new Common Lisp implementation (i.e., SICL) using these new techniques, another (possibly futile) objective is to convince maintainers of other Common Lisp implementations to use some of these modules, so as to cut down on the collective maintenance burden.
3:53:04
beach
I mean, imagine the possibility of writing compiler optimization passes that will automatically benefit several different Common Lisp implementations.
4:01:43
LdBeth
Yes, the probability to have more aggressive opts with configurability to disable or enable by need
4:27:06
Fare
ACTION writes a detailed explanation of why the definition of uiop/package cannot be modified, ever.
4:40:08
solrize
interesting, i hadn't heard of sicl or cleavir before. i didn't realize much new CL stuff was going on
4:40:39
solrize
i'd like to try clojure but hate the JVM world so i figured maybe a java-less port would happen sooner or later
4:45:46
phantomics
For instance, it cannot create and use hash tables at compile time (i.e. during macroexpansion)
4:54:01
beach
solrize: Check the ELS proceedings to get a better idea of what is going on in the Lisp world in general, and in the Common Lisp world in particular.
4:57:17
beach
We have had 15 SICL-related papers published since 2014, 12 of which in ELS and 3 in ILC.
4:59:40
beach
Actually, some of the papers by heisig are also SICL related, so 15 is a conservative estimate.
5:06:50
no-defun-allowed
Do you mean that macros can't use hash tables, or that they can't expand to hash tables?
5:13:28
Fare
ACTION notices that mkcl only ships a modified asdf 3.1.7, and can't upgrade from that to 3.3.4
5:25:12
p_l
Just managed to check it, I was previously somehow sent to 2012 when I was looking for last one
5:27:34
p_l
The last one I remember well was in Japan, somehow I recalled the "Lisp on the Move" but for 2014 but not that it was ILC
5:30:53
beach
I was also at the one in San Francisco in 2002. I didn't have any papers to present, but moore33 (Tim Moore) presented McCLIM there. And that's when I invited him to come spend a year in Bordeaux. He is still living here. :)
6:07:02
p_l
and we first need to get rid of the pandemic, cause if I were organizing, I'd prefer an on-site one
6:12:34
beach
Fare: I have a small(!) improvement to ASDF to suggest. I need for it to use first-class global environments, so that I can tell it in which environment to compile and load things. Could you do that for me, please? :)
6:13:41
beach
I guess that won't work anyway will it. The host system would have to have a compiler that takes first-class global environments as well.
6:20:43
solrize
i've never been to ELS but pictures i've seen of Lisp events tend to look like Forth Day. ;)
6:32:37
Fare
beach, of course it's even worse, because the reader has to know about the environment to properly intern the symbols in the sub-forms.
6:34:51
Fare
how does the reader know it's for immediate evaluation, vs just a constant list '(with-environment ...) ?
6:37:53
phoe
this allows one to set up dynamic environment around the process of compiling an ASDF component
6:39:18
beach
But I don't know whether ASDF calls COMPILE-FILE and LOAD directly. I guess if ASDF can be programmed to use a different compiler and loader I would be fine.
6:40:17
Fare
beach, ASDF always calls them via the LOAD* and COMPILE-FILE* helpers in UIOP/LISP-BUILD.
6:46:02
Fare
also, you are not authorized to change the signature of any function in UIOP, except in backwards-compatible ways (e.g. adding a keyword argument). But you can create a new function with a new name, and write a shim to preserve the old interface.
6:50:16
Fare
One good thing about UIOP and ASDF: all functions are NOTINLINE, so intercepting them should always work.
6:50:24
beach
Fare: Actually what we are doing in several libraries now, like Eclector and Trucler, is that the protocol functions trampoline to a generic function, passing it the value of the library-defined *CLIENT* variable. Pre-defined methods of those generic functions do not specialize on the client parameter.
6:51:24
beach
And clients can then create its own primary or auxiliary methods on the generic functions.
6:52:49
beach
phoe: Assume you have a protocol function like COMPILE-FILE with the lambda list (FILE).
6:53:18
beach
phoe: It would trampoline like this: (defun compile-file (file) (generic-compile-file *client* file))
6:53:52
beach
So client code can do (defmethod generic-compile-file ((client my-client-class) file) ...)
7:06:10
solrize
beach, here's my new functionally-inspired version of that bellied number problem https://kate.hackyon.org/~phr/bellied2.lisp
7:09:14
beach
One thing though, in Common Lisp, the convention is for END to mean a position one beyond the last element of stuff.
7:15:33
solrize
well SICL is another CL implementation... i meant applications. something similar happens in forth, where everyone who gets interested in it writes a forth interpreter ;)
7:16:18
no-defun-allowed
beach: "Having asked about the possibility that there are real people out there who use LISP (as opposed to AI People who are known to be non-real) and having received no answers, I can only conclude that LISP is not being used and that it is not, therefore, a real language." ~ the closing announcement for a section of Lisp Pointers
7:16:59
solrize
no-defun-allowed, AI counted as an answer in the 1980s-90s but i think CL is not being used in AI very much any more. is that wrong?
7:17:33
no-defun-allowed
solrize: My university will (poorly) teach me Lisp in an AI context next year, so does that count?
7:18:45
solrize
i guess that counts ;). anything that isn't itself a CL implementation or CL dev tool counts
7:19:08
beach
solrize: In the past, I worked on an interactive editor for music scores. Version 1 shows that my ideas work, but it doesn't look great, and I made some incorrect design decisions. I don't have time to work on version 2, but jackdaniel is planning to work on the GUI for it, using McCLIM of course.
7:19:31
no-defun-allowed
I think RavenPack uses Lisp in an AI context as well for analysing "unstructured data" like news reports as well.
7:21:12
solrize
no-defun-allowed, that is cool, it sounds like old school lisp doing natural language stuff
7:22:38
no-defun-allowed
Myself, I work on a kind of distributed object system that I think is the most distributed of them all (though I might be biased).
7:23:18
no-defun-allowed
solrize: I was going to say "Oh, so Python and a C library", but nltk is actually written in Python. I have a hunch it would be quite slow and wouldn't scale to however much data that company processes.
7:24:33
solrize
that's possible, i dunno. i thought maybe they use nltk to extract statistics from the data, then dump it all into a GPU neural net or something. i haven't looked at it though. i should, since NL is an interest of mine
7:25:56
no-defun-allowed
(One nice thing is that code written for the average Common Lisp library has a shelf life of longer than a few months, which I can't say for any programs that use TensorFlow.)
7:26:41
solrize
you see they weren't satisfied with breaking the python language in the python 2 to python 3 downgrade, so now they're trying to break the standard library as well?
7:29:26
no-defun-allowed
I guess that's something that takes a while when some libraries are effectively part of the language, sure.
7:31:06
beach
Not really, since you can program the compiler by writing macros. That's not possible in other languages. There you have to wait for a new standard.
7:31:59
no-defun-allowed
All the stuff is there in CL that you would need to write performant libraries, and S-expressions don't "penalise" you for using your own operators.
7:33:09
no-defun-allowed
In this case, we are talking about the "corroded batteries" that come with Python has for some extinct formats, which are all modules of functions.
7:36:39
no-defun-allowed
(Side note on corroded batteries: I wonder why the reaction to having :host and :version among other "extinct" parameters to MAKE-PATHNAME is "Why the hell do we have those in Common Lisp?", not "Why the hell don't I have those on my computer?" Having those in the filesystem sounds quite handy.)
7:40:07
phoe
where the file compiler right now is the hardcoded cl:compile-file, but it might as well be foo:compile-file
7:48:02
beach
phoe: So, I assume that in your library you use special variables for things like adding handlers and restarts. What I would like to see is the use of this CLIENT mechanism so that I can change that, since I would prefer to use specific entry types in the dynamic environment, rather than special variables.
7:50:07
beach
phoe: For instance, HANDLER-BIND could call INVOKE-WITH-HANDLER with the CLIENT object and the default mechanism would be to use a special variable, but it could be customized then.
7:52:15
phoe
beach: sure, I can do that - tell me how to access and augment your dynamic environment object and we can do that
7:52:46
beach
NO, NO. The point is that you don't have to do anything other than using this mechanism.
7:53:11
beach
Client code would stick a method on INVOKE-WITH-HANDLER, specialized to their client class.
7:53:57
beach
But you need to publish and document the generic protocol, of course, so that clients can adapt it properly.
7:57:04
phoe
beach: correct, I can imagine that. I'll do that when my book is no longer something I need to pay attention to.
8:00:49
Fare
phoe, at first I didn't see why your talk even mattered, and then I understood that it has something deep to say about designing language features for extensibility.
8:01:17
Fare
and I wondered -- what is (if any) the relationship between your condition-system-on-CL and SICL's condition system?
8:02:16
no-defun-allowed
I think phoe's portable-condition-system is going the SICL condition system, last I heard.
8:02:18
phoe
Fare: the condition code I wrote is planned to be integrated as SICL's condition system someday
8:03:15
phoe
I think that throughout my voyage I found the major pain points where the standard CL condition system is *not* extensible or programmable
8:03:52
phoe
does it matter for end users? I guess no, not really, they usually don't need to dabble with such matters
8:05:44
phoe
likely they'll do that only if there's a pretty good use case for that that outweighs the costs of decoupling the old condition system, integrating mine, and paying the maintenance and performance costs
8:06:30
Fare
some day, when I have time, I'd like to somehow port SICL to Gerbil. Maybe with a modular way to embrace or not embrace some CL features, if I figure out how to do that.
8:06:58
phoe
I think you have a lot more wiggle room when you design an implementation from scratch, which is the case for SICL, but this isn't the case for implementations that have grown organically over the time and have a lot of history of that organic growth - which is, like, all fully functional CL implementations nowadays.
8:08:53
beach
phoe: So if SICL ever becomes as high performance as existing implementations, that means that SICL will have a much lower maintenance cost for the same performance.
8:09:12
Fare
(so with minimal amount of wrapping, you could access CL from Scheme, and Scheme from CL)
8:14:33
SAL9000
e.g. if you call an external program via system() and use the return value -- the exit status -- as a boolean, 0 will be "true" because that's "success"
8:19:12
phoe
Fare: but then again, the *original* idea for my talk was, to quote a famous non-existent researcher, "Science isn't about «why», it's about «why not»?"
8:20:43
phoe
so I just explored the idea and talked about what I found. I literally created a solution that is looking for problems - and it's good that SICL happened to be one such problem
8:21:27
phoe
plus, it was a very good exercise in pushing the condition system to its portable limits
8:26:12
phoe
so, summing all the reasons for the talk, we have 1) because-we-can-ism, 2) spreading knowledge about the functioning of the condition system, 3) possibility to integrate condition/exception systems of CL + other languages (possibly also CL) that have a common compilation target (possibly also CL), 4) SICL adopting the portable condition system, 5) checking how far we can push the CL standard in regard to the
8:28:28
phoe
;; also, I really expect (1) to be a meme by now, especially since in my case (1) came from a quote from the late Cave Johnson of Aperture Laboratories
8:32:41
phoe
Fare: I think we can all bear the consequences of my research existing, contrary to the research of the Jurassic Park
8:32:43
SAL9000
phoe: "Science isn't about WHY. It's about WHY NOT. Why is so much of our science dangerous? Why not marry safe science if you love it so much. In fact, why not invent a special safety door that won't hit you on the butt on the way out, because you are fired."
8:33:32
SAL9000
the CL condition system is more of a bird than a dinosaur... it's still alive and useful :-)
8:47:52
no-defun-allowed
Sometimes people put : before LOOP keywords, as their editors will render those in a different colour.
8:48:47
no-defun-allowed
I don't do that myself, because I don't really have a problem with reading LOOP.
8:49:04
solrize
i don't see different coloring in slime after adding the colons, but they make the keywords stand out more, which is good
8:49:18
solrize
i had originally called the parameters of that function "from" and "to" but that looked really confusing
8:49:41
no-defun-allowed
(Putting : before a LOOP keyword causes it to read as a keyword, which is a different concept, and LOOP will accept LOOP keywords from any package.)
8:52:48
beach
Sometimes I say: I recommend you do M-x define-global-abbrev<RET>thx<RET>Oh, thank you very much<RET>
8:57:26
no-defun-allowed
(From memory, I came in #lisp one day asking for someone to eye over cl-decentralise1, and explained it was an attempt at writing a distributed networking system, and then I went off about how it was written because I was annoyed everyone wrote programs with centralised semantics, then "glued" distribution atop it, then ending up with some technical debt and having to rewrite a great portion of the program. You said
8:57:26
no-defun-allowed
that was an interesting observation, I said "ty beach", and you said "ywlcm"...maybe.)
9:01:48
no-defun-allowed
I used to rant a bit more; but then I had a conversation with another #lisp participant whom taught me a lot about distributed programming, so I like to think it wasn't very embarrassing.
9:02:34
beach
Embarrassment is an integral part of the learning process. Think about learning a foreign language, for instance.
9:03:14
beach
People who do everything they can to avoid being embarrassed don't learn very much. Those are dangerous people.
9:10:32
jackdaniel
I've been told that I'm not very gentle with maintaining the channel on-topic, so I've added a word "gentle" to avoid this accusation!
9:17:16
no-defun-allowed
jackdaniel: Maybe you should write the book "A gentle introduction to the #lisp topic"
14:04:00
phoe
Is there something like a visual S-expression editor? Something that will allow me to explore S-expressions by double-clicking them and edit slots of the browsed objects.
14:04:30
phoe
I'm thinking a Lisp inspector except with more visuals, possibly showing tree structure.
14:08:04
scymtym
McCLIM's rewritten Clouseau is kind-of like that. it can display certain pieces of information as graphs or trees and does so by default in a few cases. it is also possible to add custom ways of displaying objects. that said, clouseau is not suitable for editing source code
14:09:13
phoe
I don't really want to edit source code; I want more of a simple spreadsheet sort of thing.
14:09:47
scymtym
for example, here is the list of superclasses as a graph: https://techfak.de/~jmoringe/new-inspector-5.png
14:10:10
phoe
An interactive plist editor where I can predefine some keys and create/retrieve/update/delete objects of such structure.
14:11:17
phoe
...Also, should (ql:quickload :mcclim) result in "Invalid index 2974 for (SIMPLE-VECTOR 2974), should be a non-negative integer below 2974."?