freenode/#clasp - IRC Chatlog
Search
6:57:16
Shinmera
drmeister: no problem, just need to tell staple about which packages belong to your system explicictly
7:36:55
Colleen
scymtym: beach said 4 hours, 53 minutes ago: Forget what I said. The problem was due to my first compiling SICL, then pulling a new version of Eclector. There was a remaining reference to the variable *start* in a different package of Eclector than where it is now.
7:40:33
beach
The symbol mentioned in the error message was in a package that did not contain that symbol.
7:41:22
beach
And when I tried `v' on the stack frame, it couldn't find it, giving something like "end of file on string input stream".
7:51:45
scymtym
that sounds strange, but since we do not advertise an in-image upgrade capability for eclector, it should be fine
7:55:46
beach
In a few minutes, I have a phone meeting with Didier Verna to talk about method combinations and his extension: method combinators.
7:57:14
beach
It is a subject that I had swept under the rug, and it came back to bite me when I was going to implement phase 3 of the bootstrapping procedure.
7:57:24
scymtym
have you seen http://christophe.rhodes.io/notes/blog/posts/2018/sbcl_method-combination_fixes/ ?
7:58:01
beach
I haven't read it, but I know he was working on it as a result of Didiers presentation at ELS.
12:59:32
Bike
beach: clasp doesn't use any of those modules. right now it's just cleavir and sicl-additional-conditions
13:15:20
Bike
sanity check here - if a class is forward referenced, it shouldn't be class-finalized-p T, right?
13:16:24
Bike
if a class is finalized we know all its effective slots and its parents, which is obviously not true of a forward referenced class
13:20:55
Bike
we have a finalize-unless-forward that finalizes classes from ensure-class-using-class unless they have a superclass that forward references something
13:40:13
drmeister
Shinmera: Is this normal behavior because I'm not setting things up properly - or is it exceptional behavior?
13:44:22
pfdietz
beach: hurricane season in the western Atlantic started June 1. We're near the peak now.
13:45:34
pfdietz
The weird part this year is the central Pacific, with how many storms have been coming close to Hawaii.
13:49:47
drmeister
You would think with the millions of dollars that are invested in it that it would have it.
13:49:57
pfdietz
Do you have a lot of llvm optimizations turned on? Or is this just llvm overhead without those?
13:50:35
beach
It is entirely possible that you are submitting totally huge programs to LLVM and if it contains some quadratic algorithms, then that would explain the performance.
13:51:25
pfdietz
New styles of input to llvm could be exposing new issues not normally exercised by other users.
13:52:17
beach
If we have nested functions in HIR, do you submit them separately to LLVM, or as a single block of code?
13:53:51
jackdaniel
ftr: 80% of the compilation time in ECL is spent in GCC, and >50% of that is call overhead (i.e sending one file with all code concatenated is much faster than compiling files one by one)
13:54:28
Bike
Well now I'm wondering how sbcl is so fast, which is not a question that would have occurred to me a few years ago
13:55:18
pfdietz
with enough yak shaving, software can mature like fine wine. Or is that a too disturbingly mixed metaphor?
13:55:26
Bike
I think that would be possible. They don't refer to one another on the LLVM level except in the way global functions do
13:59:12
drmeister
pfdietz: In the past five years, five minutes haven't gone by that I haven't thought that.
13:59:35
pfdietz
I wonder if LLVM is doing something like alias analysis, even without optimizations.
14:04:23
Bike
more mop questions: If a class has a forward referenced superclass with no definition, should it be finalize-inheritance or compute-class-precedence-list that signals an error (or should something else happen)
14:04:43
Bike
i guess a user could hypothetically call compute-class-precedence-list themselves, and we'd want a nice error behavior there
14:07:43
pfdietz
sbcl also has some scalability issues, with compile time nonlinear in function size. And I can write a seven line function that causes the compiler to never return, due to extremely large integer types in type propagation.
14:08:57
pfdietz
Not that huge a deal unless you have macro-generated enormous functions. There, the solution can be to wrap some code in flets/labels and declare them NOTINLINE, I think.
14:12:18
beach
Bike: I think your analysis is correct. Let compute-class-precedence-list signal the error.
14:25:44
drmeister
Every time I run something in sbcl I wonder if I've wasted the last 5 years of my life.
14:27:12
drmeister
These are two different compilers - compiling the exact same code - to essentially the same result - a native code representation of the code.
14:28:16
drmeister
About 2/3 of that time is optimization and 1/3 of that time is just instruction selection.
14:30:15
drmeister
I have set up a global variable cmp::*optimization-level* - set it to zero and it compiles everything at -O0 and disables some llvm-style inlining that clasp does explicitly.
14:30:44
drmeister
I think it's pretty much (setf cmp::*optimization-level* 0) and there is essentially no optimization.
14:31:18
pfdietz
My question was: was the profiling done with that in place, or does the profile then show no time in optimization when you do that?
14:32:42
drmeister
Profiling was done with cmp::*optimization-level* set to 0 and set to 3. I also now carefully time time spent in llvm so I can compare it to the overall compilation time.
14:35:44
pfdietz
sbcl doesn't even use ssa. The first ssa paper was 1988; CMUCL was derived from Spice Lisp around 1980 and the compiler rewritten around 1985.
14:35:53
drmeister
We have set things up so that we can use dtrace to profile clasp - all functions are reduced to llvm-ir - so they are all captured by the profiling.
14:37:16
pfdietz
Being originally designed to be used on extremely slow hardware (by modern standards) means the software runs like a bat out of hell now.
14:40:08
drmeister
Now, for some reason loading stuff with quicklisp suppresses the reports that clasp generates that describe how much time is spent in llvm vs the cleavir side of the compiler.
14:41:30
pfdietz
Some idea of hardware speed at that time: in 1983, the Sun-2 workstations used 10MHz 68010 processors.
14:43:05
pfdietz
This is telling me there should be huge opportunities for improvement, but you may have to get down into llvm. That would help a lot more than clasp, though.
14:44:00
drmeister
Now, on October 17th - I'm giving the opening talk at the llvm developers meeting in California. This is one of the things I want to bring up.
14:44:54
drmeister
I'd like to get help from that community to improve the situation. Is it something stupid that I am doing? Or is it fundamentally the way that llvm is structured?
14:45:23
Bike
it seems pretty likely to me that we're throwing things at llvm that it hasn't been tuned for
14:45:56
drmeister
Bike: I haven't run the static analyzer for MPS for several weeks - we've just been running with boehm.
14:46:49
drmeister
I started looking into another CI system called "Codeship" - it looks like it could do everything we want - but I can't find anything about time limitations.
14:48:06
drmeister
Well, how do we find those things that it hasn't been tuned for? We build modules full of functions full of valid llvm-ir and pass it to llvm and away it goes.
14:49:51
Bike
the set of llvm-ir that's normally passed to llvm is a lot smaller than the set of valid llvm-ir
14:49:53
drmeister
We can profile clasp+llvm and generate flame graphs - I can see where it's spending its time. I haven't done this a lot because I don't know what to do with that information.
14:50:36
drmeister
I'll need to do some of this when I get back from Bordeaux - I'll have a couple of weeks to get some numbers to present in my talk.
14:53:41
drmeister
All of these other llvm based languages, Rust, Julia don't have other compilers to compare themselves to. Clang only has GCC.
14:55:08
drmeister
It's been said before that the more we make our llvm-ir look like the stuff that clang generates - the better off we will be.
14:56:22
drmeister
Maybe there is a test for this. If we compare the llvm-ir compilation time for functions generated by clang vs those generated by clasp+cleavir. How does compilation time scale with something simple like the number of instructions in the functions.
16:25:09
beach
You can have it expand to (make-instance 'standard-method :lambda-list ... :function <arg>)
16:25:39
beach
The lambda list values and the rest would be taken from the generic function that you are processing.
16:26:25
Bike
the form is specified to be evaluated in an environment that's null plus a binding for call-method
16:26:58
Bike
(and as a separate question, if there are custom method classes involved i'm not sure standard-method is always appropriate, but i'm not concerned with that at the moment)
16:28:05
beach
But isn't it the case that the evaluation of the form resulting from compute-effective-method is always evaluated in the null lexical environment?
16:28:42
Bike
sure, but what if the effective method has (let ((cl-user:whatever ...)) (make-method ...))?
16:30:21
beach
Where does it say that the calls to make-method are evaluated in the null lexical environment?
16:30:44
Bike
"The form used with make-method is evaluated in the null lexical environment augmented with a local macro definition for call-method and with bindings named by symbols not accessible from the COMMON-LISP-USER package."
16:36:22
Bike
The thing is that I'm seeing whether I can serialize (dump-to-FASL) effective methods. What we have right now is very roughly equivalent to (defmacro make-method (form) (compile nil `(lambda ... ,form))). so the macroexpanded form has functions in it, and functions can't be serialized.
16:38:14
Bike
I know that, but it's going to be annoying if I'm only hamstrung by this obscure requirement.
16:42:58
Bike
it doesn't seem very odious for the user if it's not a null lexical environment... i mean, they can just shadow whatever they want
16:51:58
Bike
sure. if i could have make-method macroexpand into a lambda expression that would be fine.
16:54:21
Bike
maybe i could just have (load-time-value (lambda ...)), since load-time-value ensures a null lexical environment
16:57:57
beach
In SICL, I have the concept of a "code object" that is the result of compiling a file. It contains the machine code of the entire file, debugging information, debugger support, source information, etc. A function is just an environment and an entry point into the machine code vector of the code object. So I could serialize the code object and just create the function at load time. At least I think so.
16:59:20
beach
jackdaniel: Well, that's how I intend to load the fasl, but once the code in it has been executed, I will close it.
16:59:28
Bike
beach: I know I could do something like that,and it's an interesting idea, but requiring new and interesting semantics for the sake of make-method mostly makes me think i'm misunderstanding make-method.
17:00:09
Kevslinger
::notify drmeister As a housekeeping, don't forget about your CISE meeting tomorrow.
17:54:40
Colleen
karlosz: Bike said 15 hours, 47 minutes ago: yeah, i basically just redid your incremental ownership changes myself, since i'd changed a bunch of other stuff. it's within 2x of generate-ast now.
18:01:37
pfdietz
The next entry might also be of interest here, as it goes into the guts of the things LLVM is doing.
18:02:55
pfdietz
In fairness, McKeeman started the random testing thing on C compilers. Csmith took it to another level though.