libera/#clasp - IRC Chatlog
Search
11:51:14
drmeister
yitzi: For my bond through ring detection problem. The winding algorithm works on 2d planes. Molecular rings aren't flat. So I'm calculating a plane (centroid, normal) for each ring and projecting the ring into the plane and then calculating the intersection of each bond that cuts across the plane with the plane and then calculating the winding number.
11:54:04
yitzi
drmeister: Understand. I think there might be some other methods, but I haven't worked it out quite yet.
12:05:56
yitzi
drmeister: I think that there might be small errors possible if the bond is close to the edge of the ring due to projecting ring atoms onto plane that is not perpendicular to the bond in question.
12:08:10
yitzi
Basically. instead of find the ring plane, instead one would make a plane perpendicular to the bond by using a vector that is parallel to bond as the normal and the point along the bond that is closest to all the ring atoms as the center.
12:09:08
yitzi
Then project each ring atom onto the plane by using a vector that runs from one one the bond atoms to the ring atom in question. Finally, do the winding test.
12:15:10
yitzi
drmeister: I'll to work it a bit more, it just occurred to me that you could use the other bond atom as the center of the plane and skip finding the point closest to the ring. Its early so I could be completely off base here.
12:15:55
drmeister
One thing is the bonds will never be really close to the ring edges because of the geometry optimization.
12:16:09
drmeister
They should cut pretty much straight through the center of the ring or not at all.
12:21:05
drmeister
I'd like to declare geom:v-, geom:vdot as primops and specify that their arguments don't escape.
12:21:35
drmeister
Then we can stack allocate l and p0-l0 and avoid allocating memory for them on the heap.
12:22:19
Bike
ideally they wouldn't need to be primops for that information to be available, but to actually stack allocate i suppose they would need primopness
12:22:48
Bike
it might be simpler to define an allocate-v primop, and have v- be an allocate-v followed by an side effecting v-!
12:23:27
drmeister
Primops would be fine for things like this - vector math has just a few operations that are done over and over and over again.
12:24:25
Bike
sure, but primops are magic and it's good to keep sorcery to a minimum. it would be nice if normally defined functions could do things like return unboxed values when possible, without the programmer needing to know compiler internals
12:24:47
drmeister
Then we have the compiler warn us if we do something illegal with a dynamic-extent allocated variable - illegal like pass it to a non-primop function where it could escape.
12:25:52
Bike
we already have a bit of tracking of function dx behavior, for the sake of calls. it shouldn't be that hard to make it work for non functions, i think
12:31:50
drmeister
Returning and passing unboxed values to functions - we'd have to overhaul the calling convention.
12:38:34
Bike
instead of a primop, we define a compiler transform on v- so when the compiler sees a call to it, and the result of the call is DX, it replaces the call by an allocate v-! sequence
12:39:31
Bike
of course cleavir doesn't actually track DX like that right now, but it shouldn't be too hard to make it do so
12:39:55
Bike
and if it's told that vdot doesn't extend the extent of its arguments, even the dynamic-extent declaration shouldn't be necessary
12:40:35
drmeister
I have put off writing a lot of this kind of code because I knew it would run like shit.
12:41:49
drmeister
If I knew I could write efficient code that won't break in the future I would be off to the races.
12:42:29
drmeister
Add the define-primop thing first and then deal with the DX stuff if that is a logical path.
12:44:30
Bike
i'm gonna do the define-primop thing, and then another compiler thing that should make unboxed array access possible, and then i think i can get doing the dx stuff
12:46:16
Bike
the other thing with DX is that there's kind of an inherent safety problem, in that if you redefine vdot later so that its arguments escape, problems will happen and there's not really a way to defend against it
12:47:44
Bike
tangential but i could also look at using llvm vector arithmetic for vec3s, if you're not doing that already
12:55:48
drmeister
Use :double-float rather than :double ... And :single-float rather than :float for specifying types in primitives.
12:57:21
drmeister
Look at what I've done to primitives.lsp. We don't store thread local types in the PRIMITIVE struct anymore - it's all symbols.
12:57:48
Bike
i'm not sure about some of the ffi stuff. i reused the to_object_float/double we have for those, but i defined new intrinsics for unboxing, since ffi's does coercions we don't need or want in the compiler
12:57:56
drmeister
cmp:*PRIMITIVES* is a thread-safe EQUAL hash table that maps strings to PRIMITIVE structs that contain everything you need to generate a primitive.
12:59:35
drmeister
Yeah. I have a really, really good feeling about all of this. I think we are going to see some big performance enhancement.
13:01:34
drmeister
I've been building cando and running within the jupyterlab environment. Everything is working.
13:52:58
drmeister
https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/llvmo/llvmoExpose.cc#L4438
13:53:23
drmeister
This is how I fixed the problem: else if (S.getProtectionFlags() & llvm::sys::Memory::MF_EXEC)
13:54:36
drmeister
llvm13 has gone off the rails with object file section names and now it's brittle to use the names to recognized "text" sections. Lang said use page protection flags and so that's what I'm doing now.
14:50:44
drmeister
I may have built and then made what I thought were trivial last minute changes and then pushed.
14:54:23
Bike
https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cmp/primitives.lsp#L456
14:55:09
Bike
while it's trying to construct an intrinsic's function type, i think, but the backtrace is kind of bad because of all the evaluator frames
14:56:36
Bike
if you're not seeing this i might have just messed up a merge, but it looked like it was fine so i dunono
14:59:05
drmeister
I think it's a merge issue on your end. But I'll stop messing around with this code now and work on other things. I don't want to mess you up.
14:59:49
drmeister
Feel free to take out the (format t "Defining primitive: ~a~%" name) from the define-primitive function.
15:11:09
drmeister
For example, I'm adding Atom_O::getPosition_BANG_(Vector3& pos) { pos = this->position; };
15:12:09
drmeister
Then in Cando code I'll use (let ((pos (geom:vec 0 0 0))) (declare (dynamic-extent pos)) (chem:do-atoms (atm molecule) (chem:get-position! pos atm) ...))
15:41:05
yitzi
drmeister: I worked out the logic on my ring test idea. Sent via gchat. I haven't tried to implement it yet.
15:46:47
yitzi
You could find the center of the ring and use the normal that mention and see if it intersects the bond.
15:51:08
drmeister
I'll give it a try. I'm working on this while writing code that doesn't allocate memory and doing other things.
17:02:36
yitzi
drmeister: That diagram I sent appears to mix up the crossing number test versus the winding number test. I'm going to tweak my method a bit.
17:19:32
Bike
just to be clear on this, any DX analysis is going to be work. i won't be able to do it today and probably not this week. i'll have to rewrite the entire meta evaluate pass (which is what's powering all these optimizations) to totally change its algorithm to do backwards dataflow
17:19:48
Bike
for this particular application it might actually be faster for me to throw in some simd intrinsics
17:21:55
Bike
does that sound okay? it's not a general solution but would probably work better for this than the general solution would
17:22:15
Bike
assuming llvm's intrinsics work as advertised, which i expect they do, given who powers llvm development
17:24:08
drmeister
It's more of a question - is what I write appropriate for what you are going to do with DX analysis?
17:25:39
Bike
i'll have to figure out how to do dynamic-extent declarations well, but yeah, that's probably fine. i /do/ already have a plan, i think, for just marking functions as dx-safe, which would mean the declarations aren't necessary
17:25:58
Bike
like in your last example here, telling the compiler that chem:get-position! doesn't let its arguments escape (or at least not the second one)
17:26:20
Bike
then the dataflow analysis would follow that back, see that p1/p2 are only used as arguments to this safe function, and so decide that it can dx-allocate them
17:27:24
drmeister
yitzi: It might be a premature optimization but it might be better if the outer loop is on the rings and generate a plane for each and then quickly eliminate all bonds that cannot intersect the plane.
17:29:35
drmeister
Bike: It would be very, very nice if the compiler can figure out what can be DX allocated.
17:31:06
Bike
currently we note some functions as calling their arguments in unchanged dynamic environments and not doing anything else with them, which (a) lets closures be dx-allocated and (b) means if they unwind they can just use longjmp instead of throw
17:31:41
Bike
https://github.com/clasp-developers/clasp/blob/main/src/lisp/kernel/cleavir/setup.lisp#L64-L174 it's done in this one place in the compiler
17:31:57
Bike
ideally that would be made into a sort of proclamation form so it could go in application code instead
17:50:22
yitzi
drmeister: Yes, I don't know if my method is fast. I am basically treating one of bond atoms as a camera and doing a projection of the ring onto a plane that goes through the second bond atom.
18:38:35
drmeister
What you are telling me is that you are going to do an analysis where you are going to make the call if a variable can be DX allocated if every function that it is passed to guarantees that it will not escape?
18:39:13
Bike
ideally yeah. it also depends on the compiler understanding the function that's called to allocate
18:39:29
Bike
like, it's not going to be able to dx allocate the list returned by (map 'list ...) probably
18:40:12
drmeister
You would have to know how long the list will be - and that cannot be determined until you run it.
18:41:16
drmeister
Will I need to make these kind of declarations? (let ((det-max (max det-x det-y det-z))) (declare (double-float det-max)) ...)
18:43:15
Bike
no, it should be able to infer that the result of max on doubles is a double, though i haven't put that in yet
18:43:44
drmeister
Your idea for DX analysis sounds very, very interesting. That could impact a lot of existing code.
18:44:32
Bike
actually, looking at the compiler macro on max it might know it's a double already, but i can check
18:44:55
drmeister
cracauer told me this morning that his recollection is that you have to declare dynamic extent on anything dynamic extent.
18:45:25
Bike
"SBCL has fairly extensive support for performing allocation on the stack when a variable is declared dynamic-extent. The dynamic-extent declarations are not verified, but are simply trusted as long as sb-ext:*stack-allocate-dynamic-extent* is true. " says the manual
18:45:52
Bike
a dx analysis is something beach has wanted for a while, since it means being able to stack allocate without relying on potentially inaccurate declarations (or at least different potentially inaccurate declarations)
18:47:02
Bike
also, with map, how it would have to work is it computes the length of the list at runtime based on the input lists, and stack allocates that
18:47:48
drmeister
CL_DEFMETHOD void OVector3_O::crossProduct_BANG_(const Vector3& other, Vector3& result )
18:47:57
drmeister
CL_DEFUN void geom__v_PLUS__BANG_(const Vector3& x, const Vector3& y, Vector3& result )
18:49:25
drmeister
Have I mentioned in the last five minutes how excited I am by these developments? Tremendously.
18:51:46
Bike
the way karlosz wrote it it just charges forward, which is naive, but reasonably quick and works fine for this arithmetic stuff where inputs depend on previous results
19:23:49
Bike
ok it does look like the compiler understands max well enough already, at least for simple cases
20:04:31
Bike
ok, now i get "error: use of undeclared identifier 'CLASP_GIT_FULL_COMMIT'" from unboxed-floats with my stuff merged in
20:14:11
Bike
we do do some weird crap with calling conventions, but i don't think we have any assembly, no
20:27:16
Bike
drmeister: i defined some defprimop macros. the simplest one, when it just calls an intrinsic, looks like this https://github.com/clasp-developers/clasp/blob/unboxed-floats/src/lisp/kernel/cleavir/primop.lisp#L166-L186