freenode/#clasp - IRC Chatlog
Search
14:50:55
Bike
for asts it occurs to me that the parallel build is probably going to interact badly with our side effectual setup about inlining
16:06:38
Bike
side effects during load other than definitions are like, a few things in the compiler, and clos
16:08:04
Bike
i'd like to isolate the standard library (i.e. lsp) from the compiler. right now there are a few files that should basically be with the compiler (fli, direct-calls, backtrace) and then only some tiny things
16:15:21
beach
Did you find an explanation for the big performance difference between Clasp and CLISP?
16:16:18
Bike
no, but i figure a lot of it is that clisp is compiling to a cl-friendly bytecode, whereas clasp compiles to machine code.
16:16:55
Bike
and both of them are much slower than say sbcl, so cleavir generating poor code for itself is probably still a central issue
16:19:50
karlosz
i dont think generating poor code makes the compiler run slow. it seems much more like a microoptimization compared to genuine algorithmic slowness in the algorithms cleavir uses
16:21:11
Bike
the result you got with basic blocks is pretty interesting, so i'll probably give that a shot
16:21:51
Bike
the thing about bad code is that it's hard to see how cleavir would be generating code sufficiently horrible to be that slow, compared to compilers that don't optimize a ton anyway like ccl or clisp
16:23:36
karlosz
or when you can use basic blocks instead of instructions, since you typically have way less basic blocks than instructions
16:24:12
Bike
mapping instructions could inherently use basic blocks, since as is it records every instruction it's hit in a hash set to avoid doing things more than once
16:24:37
beach
karlosz: I don't follow your argument that the generation of poor code does not slow down the compiler. After all, the compiled compiler would then run slowly, no?
16:25:43
Bike
so it's hard to imagine how adding type inference would improve the speed of the compiler
16:26:50
karlosz
i may add new bytecode instructions that dont do type checking so i can leverge type inference tho
16:27:36
Bike
i mostly would just like cleavir to be like, at most half as slow as sbcl, so this would stop being an overriding concern
16:31:43
beach
I guess I could use the SBCL statistical profiler when I create an extrinsic environment for SICL. That way I could find out whether there are any bottlenecks in AST or HIR creation. But I am not running the type inferencer I think, so that one should then be added.
16:32:19
beach
Isn't there some library that helps with the interpretation of the output of the profiler? Maybe even producing flame graphs?
16:33:27
Bike
since ast generation and ast-to-hir are very recursive, the flames have a bunch of spires
16:34:01
Bike
additionally, ast generation calls the entire compiler recursively (for eval-when etc), so, it's just tricky overall.
16:34:32
Bike
a few times i've done profiling and ended up concluding that the compiler is slow because the compiler is slow, which was frustrating
16:41:51
Bike
what's the procedure now? i can try it locally and fix the problems that are probably my fault
16:42:41
beach
It looks like I have an instruction where I used to have a list when HIR is translated, presumably to Common Lisp.
16:45:24
Bike
basically there's a few (destructuring-bind (first last owner) basic-block ...) that need to be changed into with-accessors. i can do it if you'd like.
16:49:17
beach
I don't follow what you are saying. But there is a call to REMOVE that expects a list of lists where the inner lists have three elements. But now, instead, it is called with a list of basic blocks.
16:50:12
Bike
Right, those too... in that case there's like ":key #'third". that needs to be :key #'cleavir-basic-blocks:owner now
19:42:23
attila_lendvai
drmeister: what's the deal with the dsymutil task in the wscript? why is it commented out? it's only needed occasionally, when debugging something? or is that obsolete by now?
19:53:27
attila_lendvai
err, scratch that, the dsymutil task is called in later stages, but not for iclasp
19:55:28
drmeister
attila_lendvai: My understanding is that on macOS dsymutil is needed to prepare the DWARF information for debugging.
19:55:29
attila_lendvai
copy-paste operations larger than a few words should cost $10 for programmers...
19:57:05
attila_lendvai
ACTION already starts practicing that cross-stage cherry-picking by factoring out something in dev
19:58:45
drmeister
Which is worse - talking to a bot like it's a human or talking to a human thinking they were a 'bot
20:17:06
Inline
it got so intensive my whole system went unresponsive and then i used the nice 19 ionice stuff.....
20:23:15
attila_lendvai
Inline: you need to watch out for the memory usage. with 8G ram i pretty much need to empty my linux, even though it has +1G of swap
20:34:30
scymtym
stassats: i implemented a decent chunk of it for playing with parser and compiler things in emacs if that's the question you wanted answered
20:35:53
stassats
probably not for myself, but it'd be a good alternatively for the allergic to emacs
20:38:56
scymtym
lsp-mode for emacs combined with my language server implementation and eclector and some compilery stuff
20:53:47
kpoeck
You cannot allocate a SHORT-FLOAT with no arguments because it is missing a default constructor
20:56:44
Bike
code merely existing doesn't mean we "have" it in a meaningful sense, since the codebase is full of detritus
20:56:47
kpoeck
Well it was forgotten in interpret_token_or_throw_reader_error, so I just made a change to accept short-floats and long-floats
20:59:47
attila_lendvai
drmeister: so, as I understand, as aclasp is getting loaded it redefines a bunch of stuff that is implemented in c++ in iclasp, right? how intricate is the interdependency/touching-surface between the iclasp C++ internals and the overloaded aclasp lisp code? is it like hard to even enumerate, or is it of a manageable size? or is to mostly just additive with little touching surface?
21:02:12
attila_lendvai
drmeister: roughly I mean code lines in the C++ files that must match code lines written in lisp files
21:03:21
attila_lendvai
but whatever is loaded has assumed knowledge of the iclasp internals. what I'm wondering is how much and how intricate that knowledge is. I guess I should get my hands dirty to really get a feel of it
21:03:46
kpoeck
(let ((*read-default-float-format* 'single-float)) (read-from-string "1.7976931348623157d306"))
21:05:31
Bike
attila_lendvai: i'm confused, i mean, there's redefinitions, and there's stuff that knows iclasp internals, but those two things are kind of orthogonal
21:07:00
attila_lendvai
basically this "touching surface" is the "interface" between stage-n and stage-n+1. if it's large, then modifying stage-n+1 is a minefield regarding bootstrapping. the smaller it is the fewer bootstrap surprises. and ideally, it should be formal, or at least reified in the code.
21:08:01
attila_lendvai
Bike: forget the redefinition part. I assumed that much of the C++ code is of just-good-enough-to-get-walking quality, which is then replaced by lisp code.
21:08:34
Bike
well the main thing is just the structure of it. iclasp isn't a full cl, it's just got a few things
21:16:41
kpoeck
(let ((*read-default-float-format* 'single-float)) (read-from-string "179769313486231570000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.00") )
21:18:01
kpoeck
When calling FILL-POINTER the argument "#.ext::single-float-positive-infinity" is not an array with a fill pointer.
21:44:16
karlosz
a proper fix for all those warnings involves stack shuffling when implementing tose phi functions
21:45:05
karlosz
theres almost no literature on it and it's pretty nasty with respect to merge points and keeping track of stack indices
21:45:38
attila_lendvai
is this so? when cclasp.exe is linked by ld, then it gets linked together with all of the compiled C++ definitions that basically make up iclasp.exe, but some of those definitions are shadowed by the stuff that is compiled from lisp sources by cleavir and written into .o files that ld understands, right? and this happens because/according to the careful input file ordering given to ld?
21:46:11
Bike
attila_lendvai: i think the iclasp and lisp definitions have different names in the C sense.
21:49:52
attila_lendvai
so, then iclasp's '+ is replaced by cclasp's '+ by (setf (fdefinition ..)..) (as issued by the init code of some foo.o at exe startup?), and by cleavir picking the right definition when inlining, right? because as far as I understand all of iclasp gets linked not only to cclasp.exe, but even to cando.exe
21:51:14
attila_lendvai
IOW, all of iclasp is around in the cclasp.exe, but parts of it are shadowed in the CL universe?
21:53:46
Bike
inlining is kind of a separate matter, but yeah, there's a setf fdefinition side effect.
22:10:37
attila_lendvai
so, basically once iclasp (aka stage-1) has loaded aclasp (aka stage-2), and once aclasp compiled itself into artifacts that ld understands, then we could link an aclasp.exe that is potentially completely independent of the codebase of iclasp (i.e. it could even have a different memory layout). and the stage-2 C++ codebase would need to implement less stuff in C++ (because aclasp implements more stuff in lisp)
22:11:10
attila_lendvai
ACTION needs to much better understand how the lisp universe is bound to the c universe in clasp, and thus starts to look for docs
22:14:02
attila_lendvai
I don't necessarily mind when there are no docs, but when the same codebase is also full of random files (i.e. complexity, noise)... that doesn't help
22:15:50
attila_lendvai
can you think of a definition that has a relatively unique name that I could grep for to see how its C++ implementation is replaced by a lisp reimplementation?
22:17:12
attila_lendvai
Bike: a short bird's eye view would definitely help me here, but detailed docs cost programmer time that can be put to rather bringing the codebase closer to a readable documentation
22:19:27
karlosz
a lot of useless computation coud be saved while compiling by doing some math to figure out how to keep accessory information correct and not just recomputing it every time in a loop
22:23:15
karlosz
maybe a declarative macro interface declaring what information gets changed/needs to be recomputed for each individual pass and some manager that can figure out how to recompute things from that info
22:25:22
attila_lendvai
looks like I'll have to take a deep breath and look at the code itself... I'm not overly excited about the details of ld and C++ internals, but I learned programming in m68k asm, so I should have the means if I can muster the motivation...
22:26:24
attila_lendvai
but the clasp codebase could be improved plenty to be more welcoming to newcomers
22:30:41
Bike
attila_lendvai: i mean, it's pretty much that there's an iclasp executable, and when you run cclasp you have it load a bunch of fasls that do redefinitions
22:38:04
attila_lendvai
this is what I'm aiming to address with a more structured bootstrap: clasp-builder.lisp is full of weird tagbody-go code because iclasp is very limited, but it's the same code that is run to build cclasp, i.e. when full CL is available. this is an unnecessary waste of programmer brain cycles.
22:55:46
attila_lendvai
the wscript gives a list of files to compile/load-a/b/cclasp, that then arbitrarily filter out ranges of that list using the filenames at the expected start/end indexes. and I wonder why I don't understand what's going on...