freenode/#clasp - IRC Chatlog
Search
14:47:14
drmeister
I think the impact on build time will be small because the big problem is still there - lexical variables being stored in activation frames. That's a structural problem of the bclasp compiler that will require a fair amount of rewriting.
14:47:37
drmeister
It should impact cclasp compilation time though - but I don't have evidence for that yet because I haven't gotten to that yet.
14:49:13
drmeister
And when I squeeze out more glue in effective method calls that I am much more aware of and capable of handling now with the experiences of the last months - it should speed up even more.
15:39:51
drmeister
The frames with ??? are JITted functions and debug information is ignored - so there is no information on them.
15:40:39
drmeister
There used to be C++ functions between every pair. Now functions call each other directly.
15:42:14
drmeister
Instance_O::entry_point (that's the start of a generic function call using the old dispatch method)
15:43:10
drmeister
Instance_O::entry_point -> standard_dispatch -> apply_method -> funcall_consume_valist_ -> COMBINE-METHOD-FUNCTIONS2.LAMBDA -> apply_method -> funcall_consume_valist_ -> ???
15:56:50
drmeister
Now that functions are calling each other directly - it is easier to incorporate, test and verify inlining.
15:57:28
drmeister
:notify frgo The new way functions are calling each other may require some adjustments to defcallback.
15:57:33
drmeister
::notify frgo The new way functions are calling each other may require some adjustments to defcallback.
17:04:54
drmeister
Bike: I increased the funcall limit to 140 arguments and implemented that change in dictionary.lisp that stassats suggested. The 140 argument limit should work with the current sicl - so these changes should build on other systems.
17:14:37
drmeister
I would reset the dictionary.lisp code, reduce the funcall limit back to 64 arguments, rebuild clasp and then look into what it is compiling when it isn't crashing.
17:15:36
drmeister
I don't know yet on speed - it is building on Shinmera's machine and once it's done I'll have more info.
17:18:18
drmeister
That means that absolutely no bclasp functions are on the stack - they generate backtrace records.
17:19:13
drmeister
It will take a bit more wrangling to get backtraces in cclasp compiled code - the machinery is all there - it just needs to be tied in.
17:59:07
drmeister
stassats: How would an assembly language APPLY work? It would move the stack pointer down to store the overflow arguments, copy those arguments into that space and then load the register passed arguments and then call the target function?
18:00:03
drmeister
On return it would reset the stack pointer, ensure that the results are in the result registers and return to the caller.
18:53:00
stassats
if it has to do something, then it spells bad things for your tail call optimizations (in general, not just for apply)
18:53:16
drmeister
This assembly APPLY is going to take N+1 arguments where the last argument is a CL list or one of Clasp's special tagged VaList_S* pointers that point to a list of arguments on the stack.
18:54:56
drmeister
I guess I'll have to figure out how inlining of assembly works? I don't know what registers are available - or I push/pop the ones I use?
18:58:11
stassats
probably would make it easier if it's not inlined, so it would just follow the ABI
19:59:31
drmeister
This is the C++ code that currently implements APPLY: https://github.com/drmeister/clasp/blob/dev/src/core/evaluator.cc#L95
20:00:22
drmeister
It is complicated because it tries to optimize a couple of cases and it handles two kinds of lists as the last argument (either a CL list or a VaList_S* (va_list with length)).
20:01:50
drmeister
The only thing it can't do is set up the final call with the stack and base pointer and register arguments properly set - that's frustrating.
20:08:37
drmeister
Or I change Common Lisp functions to accept (closure, nargs, arg0, arg1, arg2, valist)
20:08:59
drmeister
Then if there are more than 3 arguments the remaining arguments are pulled from the valist.
20:09:27
drmeister
Then I can implement apply in C++ and I don't need to call funcall_consume_valist_
20:24:49
drmeister
Static analyzer ran - I pushed the changes to testing. Once it's built I'll have new timing.
21:31:39
drmeister
I'd say maybe 5 min on the cclasp compilation time. This is bclasp though - so I'm not sure.
21:36:27
drmeister
akkad: I do - it's a decent example of Clasp - but it's a couple of weeks old and has some things you aren't expecting (1) chemistry code (2) runs a Jupyter notebook server.
21:56:30
drmeister
Ok, this is from digging into the output and extracting the cclasp compilation times - it doesn't include the link times and time to build ASDF
21:56:55
akkad
yeah this was it. [C 21:56:42.695 NotebookApp] No such file or directory: /home/app/bash
21:57:33
Shinmera
drmeister: It would be even nicer if it also had memory peak, total allocations, disk i/o, etc.
21:58:49
drmeister
akkad - it's pretty stripped down to keep it as small as possible. But there is a bash shell.
21:59:23
drmeister
I'm not sure why they have --entrypoint - but my friend who helped me set these docker scrips put that together.
21:59:59
drmeister
I have gotten into the docker container and mucked about in the shell to verify things.
22:00:51
drmeister
But that docker image is designed to start a Jupyter notebook server for non programmer chemists. You can start a slime session from the Jupyter notebook and talk to the Clasp environment through slime.
22:01:20
drmeister
I love docker - it opens up all sorts of possibilities and solves lots of problems.
22:02:26
drmeister
This is why I'm putting time into speeding up the cclasp compiler - because it takes a long time to compile all the Common Lisp dependencies to prepare that docker image.
22:05:36
drmeister
It's a little different with LLVM. It's more compile-C++/compile all Common-Lisp/link everything together into one executable.
22:06:26
drmeister
And yes - it can do that. You could do things like compile quicklisp and all dependencies and your code and link everything with the compiled C++ code into one executable.
22:13:27
drmeister
I'll combine this with fastgf and I need to figure out how to get more glue out of the generic function calls. It's copying arguments at least twice for every GF call.
23:03:04
drmeister
Bike: This is probably more of a beach question: I do this in Cleavir: (setf (fdefinition 'cleavir-primop:call-with-variable-bound) (fdefinition 'core:call-with-variable-bound))
23:03:26
drmeister
I'd like to replace the core:call-with-variable-bound call with something special.
23:04:08
Bike
but you could define call-with-variable-bound as a special operator instead. pretty sure.
23:36:58
drmeister
Bike: With that (setf (fdefinition 'cleavir-primop:call-with-variable-bound) (fdefinition 'core:call-with-variable-bound))
23:37:33
drmeister
Calls to (core:call-with-variable-bound symbol value thunk) are being generated every time a special variable is bound
23:38:09
drmeister
That generates a call to a useless wrapper that then calls core__call_with_variable_bound
23:38:47
drmeister
I can convert that to use a special operator (core:multiple-value-foreign-funcall "call_with_variable_bound" symbol value thunk) and cut out the useless wrapper.
23:40:50
Bike
currently cleavir generates a funcall ast and such directly. i could maybe rewrite it to fully convert a form instead, in which case you could use a macro
23:49:50
drmeister
Is there any other way? So rather than (funcall cleavir-primop:call-with-variable-bound symbol value thunk) I'd like (core:multiple-value-foreign-funcall symbol "call_with_variable_bound" symbol value thunk)
23:54:17
drmeister
I think there are some other things like this as well - but I think I have more control. Like how unwind-protect is implemented and catch and throw.
0:39:05
drmeister
Bike: Thank you - if/when you make that change, can you give me a heads up? At that point I'll define a macro.
0:58:19
Bike
https://github.com/Bike/SICL/commit/bab9c10a1df4bf068c45965682bfa7066e21a4cf don't seem to be any problems
1:01:21
drmeister
I mean - how much trouble is it to update the repository and maintain the &va-rest stuff.
1:01:37
Bike
i don't anticipate having to mess with lambda list parsing at all. i didn't put this in the clasp repository yet though.
1:34:17
Bike
shouldn't you have to worry about invoke for any use of multiple-value-foreign-call, then?
1:38:36
drmeister
This is what it was about. Cavalier use of 'call' when I need to be using 'invoke' when there is a landing-pad set up in the function.
1:39:13
Bike
i'm still not sure i get it. A caller needs to use 'invoke' when the caller is prepared to except abnormal exits from the called function, right?
1:39:33
drmeister
I just noticed that multiple-value-foreign-call was not completely fixed up. I'm not using it much and the things I'm using it for may not unwind the stack - or a nasty bug is waiting to bite me in the butt.