libera/#clasp - IRC Chatlog
Search
17:35:46
Bike
the other easyish thing i can do is change how the contagion transforms work in such a way that the code shouldn't box in the loop even without the "of-type double-float"s, though it would do more type checks
17:43:14
drmeister
It generates a list of centroid/normal vectors for a collection of rings repeatedly
17:43:45
drmeister
It takes 2.8 seconds with the old compiler and 1.14 seconds with the new compiler.
18:03:33
Bike
i started on unboxed array access. seems like it should work. i hit a dumb roadblock for writing but ic an work around it
18:53:12
Bike
define an intrinsic or codegen sequence that takes a geom and outputs an unboxed double/float whatever, then hook it up with the transformers
18:55:36
Bike
say unboxed double exp. we have an intrinsic for it https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cmp/primitives.lsp#L234 define a bir primop https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cleavir/bir.lisp#L383 and a translation for it
18:55:36
Bike
https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cleavir/translate.lisp#L1065
18:56:04
Bike
and then finally define a transformer that turns (exp double-float) calls into a use of the primop https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cleavir/transform.lisp#L559
19:00:21
drmeister
What if you defined an intrinsic that reads a double from an integer offset from the start of an object?
19:01:07
Bike
if you gave a function like that the wrong kind of object you'd get some garbage double and it would probably be pretty confusing
19:01:51
drmeister
If we want to add these from extensions we need to hook them deep into the compiler - don't we?
19:01:56
Bike
also, for this case i don't think you'd actually need a transformer - you could just have a primop that inputs a geom object, checks the type, and outputs a double, and use that directly
19:02:24
Bike
like in this case a "here's an arbitrary C++ object, get this thing out of it" kind of operation
19:03:27
drmeister
I have a way for extensions to register initialization code. I can extend it to initialize stuff the compiler needs.
19:04:27
Bike
for this reading-from-an-object case you don't actually need any of this fancy type-directed translation stuff, anyway. you just need a primop that the compiler knows outputs an unboxed float
19:06:58
drmeister
Or what do you need from the extension support code that would make it not a problem.
19:07:09
Bike
the clasp-cleavir parts shouldn't mind a new primop being added after the fact, i don't think
19:08:08
drmeister
Intrinsics are defined in cmp/primitives.lsp - do primops need to use intrinsics?
19:08:53
Bike
no, they just need a translate-primop method that generates whatever code. e.g. this one https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cleavir/translate.lisp#L1091-L1095
19:15:32
drmeister
Sure, but that's an llvm instruction. If I want to generate a call instruction I thought it needs to be defined in primitives.lsp
19:16:36
drmeister
It's not a huge problem. I can add a mechanism where extensions register primitives when primitives are defined. I just want to make sure it's needed or otoh I'm misremembering things.
19:19:55
Bike
i think if you want a call to a named function it needs to be defined as an intrinsic, yes
19:21:57
Bike
for the particular case of getting things out of structures it might be good to define a general definition mechanism
19:22:57
Bike
like so we could write (define-structure geom:vec double double double), and then that lets you do (cmp:structure-read geom:vec 0) in code to get the x coordinate out
19:23:46
drmeister
I think the *primitives* dynamic variable is lazily set for every thread that is created.
19:28:22
Bike
oh, and (/ 1.0 (length points)) in your code is also going to box a float, but i can fix that one fairly easily
20:16:04
drmeister
The reason the primitives need to be defined in each thread is because the LLVMContext's are all thread local. The types that the primitives use as arguments are all defined relative to the thread local LLVMContext.
20:45:39
drmeister
And then have a function like (cmp:lookup-type :double) -> result of evaluating %double%
21:14:30
Bike
can we not also call c functions through ffi? does not that end up as an llvm named call?
21:16:18
drmeister
Hmm, maybe we could. I think I understand what we are doing here with primitives. I'm not so sure about ffi.
21:22:36
drmeister
ffi would go through dlsym - or we have to use load-time-value to lookup the dlsym.
21:35:25
drmeister
It's really straightforward to switch to type names like :double. I've switched over already.
21:36:19
drmeister
We can turn cmp:*primitives* into an atomic list and push new primitives into it at any time now I think.
21:46:35
Bike
i just noticed the wrong intrinsics were being used for exp and some others on doubles, so if you compile code with that an llvm assertion will fail and kill clasp
21:48:59
drmeister
I've changed all of the primitive definitions so they use keywords to represent the types like :double :t* etc.
21:49:40
drmeister
I've written a function (defun primitive-return-type (info) ...) and (defun primitive-argument-types (info)...) and they use the names to lookup the types at runtime and return them.
21:50:34
drmeister
Now I'll change the cmp::*primitives* dynamic variable so we don't have this complicated rigamarole where we redefine it for each thread.
22:10:23
Bike
(setf (row-major-aref a n) (exp (row-major-aref a n))) becomes "%2 = call float @cc_simpleFloatVectorAref(i8* %0, i8* 1); %3 = call float @llvm.exp.f32(float %2); call void @cc_simpleFloatVectorAset(float %3, i8* %0, i8* %1)". nice nice. apparently operator[] on arrays doesn't bounds check, though
22:18:25
drmeister
https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cmp/primitives.lsp#L45
23:32:15
drmeister
Bike: You are using cmp:%double% when you call (define-unary-primop core::df-exp "..." cmp:%double%) - can we switch this to use :double and lookup the type at runtime using (cmp:primitive-lookup-type :double)?
23:35:25
drmeister
Why do you have separate functions/macros for unary and binary operations? What about ternary and so on?
23:36:00
Bike
actually, %double% and so on are symbol macros, right? you could just change what they expand into without altering the code
23:36:14
Bike
no special reason for the different ones, and you can see a couple that don't quite fit
2:39:14
Bike
https://github.com/clasp-developers/clasp/blob/main/src/lisp/kernel/cmp/cmpintrinsics.lsp#L147 well it looks like the existing definitions should be fine? maybe i'm misunderstanding the goal here
2:40:02
Bike
with primitives you want a delay but the translators should just be able to do what theyre already doing
2:40:02
drmeister
Currently (macroexpand 'cmp::%t%) -> (LLVM-SYS:TYPE-GET-INT8-TY (COMPILER:THREAD-LOCAL-LLVM-CONTEXT))
2:42:51
drmeister
I'm afraid of the situation where it gets resolved to a thread local type at compile time.
2:46:33
drmeister
I've switched cmp:*primitives* so it doesn't contain the thread local types anymore. I pushed the changes to unboxed-floats
2:56:05
drmeister
I don't want to muck around with your code too much. Can you give me a primop defining macro?
2:57:15
drmeister
Bike: You know what else is interesting about this? We could get llvm to do inlining of these intrinsics.
3:12:21
drmeister
I'm going to use :double-float rather than :double and :single-float rather than :float - that will make it match the keywords you are using.
3:51:17
drmeister
If we change the cmp:%double% to :double-float then we could simplify the definition.
3:53:28
Bike
i can do a macro tomorrow. i was going to do one that includes the translate-primop body so everything's in one place
3:56:25
Bike
i don't think it makes sense to incorporate the intrinsic into the defprimop since some primops don't have corresponding intrinsics
4:00:17
drmeister
(disassemble '(lambda (x) (geom::ovector3-getx x)) :type :ir) says "Undefined function OVECTOR3-GETX"
4:03:27
Bike
https://github.com/clasp-developers/clasp/blob/main/src/lisp/kernel/cleavir/setup.lisp#L176-L192 add it in here, i think
4:03:43
Bike
yeah, this is all pretty unwieldly. i'll put together something more convenient tomorrow
4:15:42
Bike
i think the easiest solution would be if i define a cleavir-primop:primop special operator that takes the name as its first pseudo parameter
4:23:44
Bike
just to be clear, you probably don't want to use primops directly in the application code. you could do like (declaim (inline ovector3-getx)) (defun ovector3-getx (ovector3) (core:primop %ovector3-getx ovector3)) and then you have an actual function but the compiler can still work with obvious calls
4:25:50
Bike
and i might want to port the representation selection stuff from clasp to cleavir, it's pretty general
4:28:51
Bike
the short version is i got type inference going well enough that long stretches of floating point operations can be compiled without boxing and without making the compiler unusably slow
4:29:09
Bike
and it's generalizable to other type directed optimizations, though doing them while preserving safety is a little less obvious