freenode/#clasp - IRC Chatlog
Search
13:05:01
drmeister
I almost have cclasp building but I encountered an exception handling problem when EXIT is invoked when building cclasp.
13:17:19
Shinmera
In http://www.lispworks.com/documentation/HyperSpec/Body/f_shared.htm it mentions this for the initialisation of slots with initforms:
13:17:21
Shinmera
"Any slots indicated by slot-names that are still unbound at this point are initialized according to their :initform forms. For any such slot that has an :initform form, that form is evaluated in the lexical environment of its defining defclass form and the result is stored into the slot. For example, if a before method stores a value in the slot, the :initform form will not be used to supply a value for the
13:17:46
Shinmera
How would I hope to "evaluate the form in the lexical environment of the defclass"?
13:18:45
beach
Shinmera: What Bike said. It is very explicit in the AMOP, but not so much in the Common Lisp HyperSpec.
13:19:25
beach
Shinmera: the macro DEFCLASS turns the INTIFORM into an INITFUNCTION. You need to do nothing.
13:21:25
Shinmera
It would be great if that was linked or hinted at in http://metamodular.com/CLOS-MOP/slot-definition-initfunction.html too
13:36:07
beach
Shinmera: Like that: http://metamodular.com/CLOS-MOP/slot-definition-initfunction.html
14:47:14
drmeister
I think the impact on build time will be small because the big problem is still there - lexical variables being stored in activation frames. That's a structural problem of the bclasp compiler that will require a fair amount of rewriting.
14:47:37
drmeister
It should impact cclasp compilation time though - but I don't have evidence for that yet because I haven't gotten to that yet.
14:49:13
drmeister
And when I squeeze out more glue in effective method calls that I am much more aware of and capable of handling now with the experiences of the last months - it should speed up even more.
15:39:51
drmeister
The frames with ??? are JITted functions and debug information is ignored - so there is no information on them.
15:40:39
drmeister
There used to be C++ functions between every pair. Now functions call each other directly.
15:42:14
drmeister
Instance_O::entry_point (that's the start of a generic function call using the old dispatch method)
15:43:10
drmeister
Instance_O::entry_point -> standard_dispatch -> apply_method -> funcall_consume_valist_ -> COMBINE-METHOD-FUNCTIONS2.LAMBDA -> apply_method -> funcall_consume_valist_ -> ???
15:56:50
drmeister
Now that functions are calling each other directly - it is easier to incorporate, test and verify inlining.
15:57:28
drmeister
:notify frgo The new way functions are calling each other may require some adjustments to defcallback.
15:57:33
drmeister
::notify frgo The new way functions are calling each other may require some adjustments to defcallback.
17:04:54
drmeister
Bike: I increased the funcall limit to 140 arguments and implemented that change in dictionary.lisp that stassats suggested. The 140 argument limit should work with the current sicl - so these changes should build on other systems.
17:14:37
drmeister
I would reset the dictionary.lisp code, reduce the funcall limit back to 64 arguments, rebuild clasp and then look into what it is compiling when it isn't crashing.
17:15:36
drmeister
I don't know yet on speed - it is building on Shinmera's machine and once it's done I'll have more info.
17:18:18
drmeister
That means that absolutely no bclasp functions are on the stack - they generate backtrace records.
17:19:13
drmeister
It will take a bit more wrangling to get backtraces in cclasp compiled code - the machinery is all there - it just needs to be tied in.
17:59:07
drmeister
stassats: How would an assembly language APPLY work? It would move the stack pointer down to store the overflow arguments, copy those arguments into that space and then load the register passed arguments and then call the target function?
18:00:03
drmeister
On return it would reset the stack pointer, ensure that the results are in the result registers and return to the caller.
18:53:00
stassats
if it has to do something, then it spells bad things for your tail call optimizations (in general, not just for apply)
18:53:16
drmeister
This assembly APPLY is going to take N+1 arguments where the last argument is a CL list or one of Clasp's special tagged VaList_S* pointers that point to a list of arguments on the stack.
18:54:56
drmeister
I guess I'll have to figure out how inlining of assembly works? I don't know what registers are available - or I push/pop the ones I use?
18:58:11
stassats
probably would make it easier if it's not inlined, so it would just follow the ABI
19:59:31
drmeister
This is the C++ code that currently implements APPLY: https://github.com/drmeister/clasp/blob/dev/src/core/evaluator.cc#L95
20:00:22
drmeister
It is complicated because it tries to optimize a couple of cases and it handles two kinds of lists as the last argument (either a CL list or a VaList_S* (va_list with length)).
20:01:50
drmeister
The only thing it can't do is set up the final call with the stack and base pointer and register arguments properly set - that's frustrating.
20:08:37
drmeister
Or I change Common Lisp functions to accept (closure, nargs, arg0, arg1, arg2, valist)
20:08:59
drmeister
Then if there are more than 3 arguments the remaining arguments are pulled from the valist.
20:09:27
drmeister
Then I can implement apply in C++ and I don't need to call funcall_consume_valist_
20:24:49
drmeister
Static analyzer ran - I pushed the changes to testing. Once it's built I'll have new timing.
21:31:39
drmeister
I'd say maybe 5 min on the cclasp compilation time. This is bclasp though - so I'm not sure.
21:36:27
drmeister
akkad: I do - it's a decent example of Clasp - but it's a couple of weeks old and has some things you aren't expecting (1) chemistry code (2) runs a Jupyter notebook server.
21:56:30
drmeister
Ok, this is from digging into the output and extracting the cclasp compilation times - it doesn't include the link times and time to build ASDF
21:56:55
akkad
yeah this was it. [C 21:56:42.695 NotebookApp] No such file or directory: /home/app/bash
21:57:33
Shinmera
drmeister: It would be even nicer if it also had memory peak, total allocations, disk i/o, etc.
21:58:49
drmeister
akkad - it's pretty stripped down to keep it as small as possible. But there is a bash shell.
21:59:23
drmeister
I'm not sure why they have --entrypoint - but my friend who helped me set these docker scrips put that together.
21:59:59
drmeister
I have gotten into the docker container and mucked about in the shell to verify things.
22:00:51
drmeister
But that docker image is designed to start a Jupyter notebook server for non programmer chemists. You can start a slime session from the Jupyter notebook and talk to the Clasp environment through slime.
22:01:20
drmeister
I love docker - it opens up all sorts of possibilities and solves lots of problems.
22:02:26
drmeister
This is why I'm putting time into speeding up the cclasp compiler - because it takes a long time to compile all the Common Lisp dependencies to prepare that docker image.
22:05:36
drmeister
It's a little different with LLVM. It's more compile-C++/compile all Common-Lisp/link everything together into one executable.
22:06:26
drmeister
And yes - it can do that. You could do things like compile quicklisp and all dependencies and your code and link everything with the compiled C++ code into one executable.
22:13:27
drmeister
I'll combine this with fastgf and I need to figure out how to get more glue out of the generic function calls. It's copying arguments at least twice for every GF call.
23:03:04
drmeister
Bike: This is probably more of a beach question: I do this in Cleavir: (setf (fdefinition 'cleavir-primop:call-with-variable-bound) (fdefinition 'core:call-with-variable-bound))
23:03:26
drmeister
I'd like to replace the core:call-with-variable-bound call with something special.
23:04:08
Bike
but you could define call-with-variable-bound as a special operator instead. pretty sure.
23:36:58
drmeister
Bike: With that (setf (fdefinition 'cleavir-primop:call-with-variable-bound) (fdefinition 'core:call-with-variable-bound))
23:37:33
drmeister
Calls to (core:call-with-variable-bound symbol value thunk) are being generated every time a special variable is bound
23:38:09
drmeister
That generates a call to a useless wrapper that then calls core__call_with_variable_bound
23:38:47
drmeister
I can convert that to use a special operator (core:multiple-value-foreign-funcall "call_with_variable_bound" symbol value thunk) and cut out the useless wrapper.
23:40:50
Bike
currently cleavir generates a funcall ast and such directly. i could maybe rewrite it to fully convert a form instead, in which case you could use a macro
23:49:50
drmeister
Is there any other way? So rather than (funcall cleavir-primop:call-with-variable-bound symbol value thunk) I'd like (core:multiple-value-foreign-funcall symbol "call_with_variable_bound" symbol value thunk)
23:54:17
drmeister
I think there are some other things like this as well - but I think I have more control. Like how unwind-protect is implemented and catch and throw.
0:39:05
drmeister
Bike: Thank you - if/when you make that change, can you give me a heads up? At that point I'll define a macro.
0:58:19
Bike
https://github.com/Bike/SICL/commit/bab9c10a1df4bf068c45965682bfa7066e21a4cf don't seem to be any problems
1:01:21
drmeister
I mean - how much trouble is it to update the repository and maintain the &va-rest stuff.
1:01:37
Bike
i don't anticipate having to mess with lambda list parsing at all. i didn't put this in the clasp repository yet though.