freenode/#clasp - IRC Chatlog
Search
4:22:41
drmeister
General: I think I have the whole CI thing sorted out now. I have a free tier machine on AWS running buildbot and I can set up latent EC2 instances that get spun up as they are needed to build different things.
7:49:57
heisig
beach: I just read Pascal Costanza's paper. I think you are right about MAKE-METHOD-LAMBDA - using a compile-time instance of the generic function solves the issue of macroexpansion time dispatch.
7:52:21
beach
Yes, I can't see what information MAKE-METHOD-LAMBDA could extract from the generic function, so it might as well just be a class prototype.
7:52:40
heisig
What I am still not sure is how a method function can access its associated generic-function and method metaobjects when calling no-next-method.
7:55:05
heisig
It is not very important at the moment. The same goes for CLOS optimizations. Might be a paper for 2020 :)
8:08:25
beach
Hmm, my plan was to create an initial SICL image without the compiler in it, but if compute-effective method requires the compiler, that won't work.
8:09:48
beach
I guess I'll have to include some temporary non-conforming mechanism for doing it and just replace it later by loading FASLs.
8:13:58
beach
Well, during bootstrapping I have no problem. I can include the Cleavir-based compiler in the first-class global environment.
8:14:54
beach
And if I need to load the compiler (which contains tons of generic functions) I need COMPUTE-EFFECTIVE-METHOD to do something reasonable.
8:16:40
heisig
I see. Why do you not just load the compiler during bootstrapping and have it placed directly in the initial image?
8:20:28
heisig
So the alternative is to have the initial image dispatch its calls to compile to another process (e.g. Cleavir in SBCL)?
8:52:21
makomo
morning! i was reading costanza's paper as well but i don't see how MAKE-METHOD-LAMBDA is called during macroexpansion, given the code on page 3?
8:53:22
makomo
it says "here's the expansion of the above DEFMETHOD" and this expansion includes a call to MAKE-METHOD-LAMBDA. doesn't this mean it will be called only when the expansion is finally evaluated?
8:58:59
makomo
beach: hm, that's what i understood from what Bike said yesterday. i also thought costanza was getting at the same problem a page or so later.
8:59:45
makomo
costanza says: "This implies two problems: The generic function metaobjects have to be present at macroexpansion time, ..."
9:00:02
makomo
and i understood the reason as "because MAKE-METHOD-LAMBDA is called at macroexpansion time"
9:00:35
beach
The proble is that if the defgeneric followed by the defmethod are in the same file, and the file is processed by the file compiler,..
9:01:28
makomo
and Bike said yesterday: "make-method-lambda is called during the macroexpansion, which is most of why it's a weird part of the protocol"
9:02:31
makomo
beach: right, but that would only be a problem if some "trickery" was going on at macroexpansion-time, right? such as calling MAKE-METHOD-LAMBDA
9:03:51
makomo
that's why i'm confused. page 3 only shows the final expansion, but it includes MAKE-METHOD-LAMBDA -- shouldn't it already have been called?
9:04:41
makomo
beach: btw, i think there's a typo in http://metamodular.com/CLOS-MOP/make-method-lambda.html "convert the lambda expression a method function"
9:10:11
beach
If the code on page 3 is correct, then the generic function *will* be avalable when make-method-lambda is called.
9:11:17
beach
But if that code is correct, it means that the compiler is called when the code is loaded and that's no good.
9:11:42
makomo
yup! i was eagerly looking to see if he misplaced a backquote somewhere (in which case the code wouldn't show the expansion, but the "expander"), but nope.
9:15:27
beach
... it won't compile the method body in the lexical environment of the DEFMETHOD form.
9:17:38
beach
If make-method-lambda is not called at compile-time, that environment is going to be gone.
9:26:52
makomo
beach: so if capturing the lexical environment is ensured by placing the modified method lambda into the expansion (wrapped in a FUNCTION call), what purpose does the ENVIRONMENT parameter serve in MAKE-METHOD-LAMBDA?
9:27:47
beach
I am not sure. I guess it could be used to look up definitions at compile time. But I haven't found a use for that.
10:05:31
makomo
beach: costanza also says on page 6 "Note that the method body is not processed anymore but can actually be a ’real’ closure, whereas in the approach proposed by the CLOS MOP, method bodies must be s-expressions that are processed by make-method-lambda and then explicitly compiled into method functions. An implication is that such method functions cannot close over local lexical environments, which can be
10:05:57
makomo
but this is in contradiction with http://metamodular.com/CLOS-MOP/processing-method-bodies.html that you linked
10:06:22
makomo
because your link describes exactly how the method function captures the lexical environment
10:36:44
beach
Anyway, my favorite coauthor is going to show up for lunch in a few minutes. So I need to go.
11:51:34
Shinmera
drmeister: If you need specific help with custom staple projects, let me know, though I suggest you first start by trying out projects that have an asdf system to back them
12:22:44
drmeister
The feeling is that the reason llvm is slow is because they want a clean separation of stages.
12:33:12
Bike
beach: i don't think you should have compute-effective-method require the compiler. my reading is that compute-effective-method returns a form, and then we can compile it later. this has turned out well for us.
12:33:46
Shinmera
They might be used to the historically slow compile times of C++ and thus see current LLVM speed as acceptable
12:34:45
beach
Bike: My problem is the following: I want an initial executable without any evaluator in it.
12:35:08
beach
Bike: But then if I load a FASL that defines methods on some generic function, I must compute effective methods for those functions.
12:35:31
drmeister
Shinmera: I think they compare it to GCC and since they do well enough there - there isn't a strong driver to go faster - maybe?
12:35:32
Bike
Oh, how coincidental that I've been looking at how to compile generic function things ahead of time.
12:36:52
Bike
I've worked out a macro that essentially expands into (setf (call-history gf) (list (cons [some specializers] (lambda ...)) ...)), where the lambdas are effective methods
12:40:48
beach
And I guess it doesn't really matter how you obtain that lambda expression, as long as it does what the effective method does.
12:40:58
Bike
though most of it has to do with special cases. if you're okay with every effective method being a function it would be fine.
12:42:23
beach
If you load a fasl that defines a method on an existing generic function, like initialize-instance, you still need the compiler to fix that one up.
12:46:21
makomo
beach: why exactly do you want to avoid having a compiler in the initial image? you mentioned that compiling SICL code within the host is slow or something like that, but i didn't fully understand it
12:46:26
beach
My thinking was to use a different mechanism that does not construct an effective method at all (in the initial image), but just calls the different method functions in some specific order.
12:49:45
beach
makomo: I execute SICL code in SBCL by translating source to HIR and then HIR to Common Lisp. The resulting code is very slow.
12:51:35
beach
Already, I bump the SBCL heap to 10GB. If I add another copy of Cleavir (in the first-class global environment in addition to the host environment), I will need even more.
12:53:07
heisig
It could be worth a try, all the other solutions proposed so far are monumental hacks.
12:56:07
beach
If I gen get the host to compile the compiler, then time might not be such a problem.
12:56:23
makomo
beach: so SICL currently compiles and executes Common Lisp code by... compiling it to Common Lisp for the host to execute? :D
12:56:55
beach
But it is very strange Common Lisp code. It makes no references to the global host environment, only to the first-class global environment it was "tied" to.
12:57:40
drmeister
makomo: Cleavir also runs within Clasp. Within Clasp - Cleavir is used to generate llvm-ir and that is lowered to native code.
12:58:23
makomo
mhm. so this is just one particular language Cleavir is compiling to (i.e. the fact that's it's again CL doesn't matter?)
13:00:12
beach
makomo: Essentially, a source file turns into a single function. Loading a fasl means calling that function with the environment as an argument.
13:01:07
beach
makomo: when that function runs, it installs the definitions of functions etc in the environment, and it loads function cells from that environment so that references to global functions in the source refer to the definitions in that environment.
13:02:08
beach
makomo: So (f ...) turns into (funcall (load-time-value (car (sicl-genv:find-function-cell 'f))) ...)
13:06:21
beach
So the top-level looks sort of like this: (lambda (env) (let ((f-cell (sicl-genv:find-function-cell 'f env))) ... (funcall (car f-cell) ...)))
13:10:25
makomo
so, (f ...) would be SICL code, while (funcall (load-time-value ...)) would be the generated CL code?
13:16:08
makomo
beach: i reread the above about tying a couple of times -- i think i understand now.
13:20:03
makomo
beach: so if you didn't have LOAD-TIME-VALUE, i guess it would still be doable, but you would have to come up how to encode such a thing within the fasl yourself?
13:22:28
scymtym_
context-dependent completion experiment: https://techfak.de/~jmoringe/eclector-context-completion.ogv
13:29:15
scymtym
the processing goes something like emacs+company <-> lsp-mode <-> cl lsp <-> more custom stuff <- concrete-syntax-tree <- eclector
13:36:51
scymtym
Bike: yeah. i made a dedicated emacs mode so i don't mess up lisp-mode and can work on the code in the same emacs
14:19:39
drmeister
Bike: I have CI and buildbot totally figured out. We can automate everything, building branches, running the static analyzer, building docker images.
14:20:26
drmeister
We can use our machines and AWS spot instances. I still need to demonstrate that I can spin up a spot instance to do a particular task and then shut it down again.