freenode/#clasp - IRC Chatlog
Search
14:36:09
drmeister
The problem I fixed last night I found by listening to the compiliation warnings.
14:39:53
drmeister
I'm noticing something weird though. It's not clear yet, but when I run clasp under the debugger it's like linking doesn't work properly. I get .fasl files with zero length and that's a show stopper.
14:41:14
drmeister
I can't even fathom why that would make a difference. If you do try to run clasp when it is connected to the debugger and you see that - can you confirm it?
14:42:55
drmeister
Let me put it this way - if you are tempted to run under the debugger because clasp backtraces are nonexistant and you start seeing fasl files with zero length - don't panic - I've seen the same thing. When I don't run under the debugger - I haven't seen it.
15:58:57
drmeister
Clasp is working pretty well - cl-jupyter builds and runs - that means Clasp compiles and compiles a whole mess quicklisp systems.
15:59:39
drmeister
I want to get up to the latest sicl - this means figuring out what is going on with that cc_writeCell call that is failing.
16:01:05
drmeister
If we could get a framework for adding inlined code for AREF then I can keep adding to it.
16:03:22
drmeister
What I'm thinking is if you could help me get started on the inlining and code generation stuff then I can keep working on that alongside other things that I need to implement.
16:03:57
Bike
like, mapcar and friends can be inlined and that's not even anything to do with cleavir, for one
16:04:32
drmeister
Meanwhile you could continue working on kildall and other compiler optimization while taking a look at the cl-jupyter notebooks and jupyter widgets stuff.
16:05:17
drmeister
Kevin and Hana will be working on that full time - but they don't know much Common Lisp. So if you could help them there that would be great.
16:05:33
drmeister
I think we can get some starter code put together for them and let them expand on it.
16:06:29
drmeister
I have a rough idea of what is involved to implement widgets in Common Lisp. I know where the Python whos functionality we are copying is and I have hacked it to dump logging information of the communication that we can work to replicate.
16:07:27
drmeister
mapcar can be inlined? Neat. I recently fixed them so that they wouldn't CONS - inlining would be even better.
16:08:17
drmeister
Right. My old implementation was before I figured out how to do more dynamic stuff on the C++ stack.
16:08:20
Bike
yeah, that sounds fine. i like teaching. and rapidly learning jupyter while having to explain it sounds like a good way to make it solid
16:09:58
Bike
just basically (mapcar #'foo bar baz) = (loop for x in bar for y in baz collect (funcall #'foo x y)).
16:12:23
Bike
and if loop doesn't work you can just write it out with dotimes or tagbody or whatever. loop's just the intuitive way to do it.
16:13:15
drmeister
It should be fine - mapcar is only used three times in loop2.lisp and not in any way that would cause problems.
16:21:58
drmeister
I think of it like auto-compile.lisp turns on Cleavir and that compiles inline.lisp and generates lots of inline definitions and compiler-macros and then it recompiles EVERYTHING with those definitions.
16:23:52
drmeister
General announcement: Folks - it's no big secret but now that it's pretty much finalized... Bike is coming to Philadelphia to work over the summer on Clasp/Cleavir. I have two other undergraduates starting this week and next who will start by incorporating widgets into cl-jupyter.
16:26:46
Bike
i was looking at ccl's macros too. but a lot of them use implementation guts cleavir doesn't have yet. or won't, like all that bit bashing
16:27:53
Bike
funcall might be better as (cond ((typeq ,fsym function) ,fsym) ((typeq ,fsym symbol) (symbol-function ,fsym)) (t (type-error ...)))
16:28:44
Bike
there are about a dozen functions that should expand to typeq, but it won't do any good until the thing with booleans is rsolved, so i'll skip those
16:29:14
drmeister
That funcall - you mean instead of the (if (cleavir-primop:typeq ,fsym function) ...
16:31:22
Bike
well, you'd have e.g. (defun vectorp (o) (if (typeq o vector) t nil)), and then if you had (if (vectorp x) ...) it would expand to (if (if (typeq x vector) t nil) ...), but cleavir doesn't know how to collapse that into (if (typeq x vector) ...) like it should be
16:41:12
drmeister
I have a feeling that there is a lot of array manipulation going on. Cando needs a lot of array manipulation.
16:41:36
drmeister
Cando now has lots of specialized arrays. You've implemented a better cleavir-primop:aref.
16:41:51
drmeister
I'd like to put it all together and start generating inlined code for array operations.
16:44:57
drmeister
https://github.com/drmeister/clasp/blob/dev/src/lisp/kernel/cleavir/translate.lisp#L530
16:46:50
drmeister
That is this: https://github.com/drmeister/clasp/blob/dev/src/llvmo/intrinsics.cc#L269
16:49:25
Bike
there are some others that should work in implementation ways. e.g., apply should work like funcall in that the coerce-fdesignator bit is inlined, but then it calls some kind of "%apply" instead of a cleavir primop.
16:50:50
drmeister
I turned backtraces on for bclasp - but this is cclasp compiled by bclasp compiling cclasp.
16:52:32
drmeister
But the very first order of business for me is to get backtrace frames back into cclasp.
16:55:38
Bike
What is (funcall (compiler-macro-function 'mapcar) '(mapcar (lambda (x) x) '(1 2 3 4)) nil)?
16:56:09
drmeister
(LOOP CORE::NAMED #:UNUSED-BLOCK14396 CORE::WITH #:MAPCAR-FUNCTION14394 = (LAMBDA (X) X) CORE::FOR #:MAPCAR-ARGUMENT14395 CORE::IN '(1 2 3 4) CORE::COLLECT (FUNCALL #:MAPCAR-FUNCTION14394 #:MAPCAR-ARGUMENT14395))
16:56:42
Bike
does (loop with f = (lambda (x) x) for arg in '(1 2 3 4) collect (funcall f arg)) work?
17:00:59
drmeister
I changed (define-compiler-macro funcall ... to (define-compiler-macro funcall-foo ...
17:05:40
Bike
Uh... well I don't know exactly where in the depths it is. But try macroexpanding (define-compiler-macro identity (form) form)
17:06:10
Bike
there's something like (if (and (eq (car #:g) 'funcall) (eq (caadr #:g) 'function)) ...)
17:08:53
Bike
compiler macro functions are supposed to deal specially with (funcall ...) forms so defining a compiler macro for defun is a bit harrowing, i guess
17:10:21
Bike
i'm not actually sure how to fix it since the name the macro is for doesn't seem to be available
17:12:43
drmeister
Rather - I've spent a lot of time debugging the compiler when it hits 'destructure'
17:13:13
Bike
basically you want that to be (if (and (eq (car ,whole) 'cl:funcall) (consp (cadr ,whole)) (eq (caadr ,whole) 'cl:function) (consp (cdadr ,whole)) (eq (cadadr ,whole) whatever-function-this-is-for))
17:22:50
drmeister
So... a third, maybe &optional, argument to sys::destructure and fix all of the callers up to define-compiler-macro - if indeed it calls this.
17:23:47
drmeister
define-compiler-macro is here - it calls sys::expand-defmacro... https://github.com/drmeister/clasp/blob/dev/src/lisp/kernel/lsp/evalmacros.lsp#L172
17:27:47
drmeister
So I could add a second optional argument to expand-defmacro to pass the macro name and pass that to destructure.
17:42:59
drmeister
I know a guy who wrote a compiler - he has more than 100 employees. He's been looking for years for someone who can take over writing compiler code.
17:43:46
drmeister
Ok, this is going to build for a while - I'm going to go back to working on my proposal.
17:45:08
drmeister
I had commented out the (define-compiler-macro funcall ... ) and tried to rebuild - it choked on something to do with returning multiple values - so there may be more rough edges in there.
17:46:12
drmeister
The way I see it, this should work smoothly to incorporate compiler macros and inlined functions. I've been working towards this for a long time.
17:47:09
drmeister
This code gets compiled/tested at the end of a long build cycle - so it's slow work to fix these bugs. Thats why I keep vacillating back and forth between incorporating these things and trying to speed up cclasp compilation.
17:48:00
drmeister
I think we've barely scratched the surface in terms of speeding up clasp using these features.
17:48:52
drmeister
I want to use inlining and custom code generation a lot in Cando. I think there are lots of places where the processor is underutilized in scientific computing because of code that doesn't take advantage of registers/cache the way it could.