freenode/#clasp - IRC Chatlog
Search
12:57:47
beach
Bike: So if I understand you correctly, just have a single AREF instruction with not type annotation, and use TYPEQ to determine the type of the array input and THE to for the type of the output. I think that's a simplification, except perhaps for the parson writing the HIR->MIR translator who must then also know the type(s) in order to translate the AREF instruction to a memory reference.
13:00:29
beach
I need to think more about having UNBOX and BOX ASTs. That would mean that ASTs can manipulate unboxed Lisp objects, and I haven't thought of the consequences of such a thing, but I guess it is no worse than having HIR manipulate such objects.
13:44:44
Bike
morning. my idea was that if the hir->mir translator writer wants to do different memory references based on the type of the array, they'd subclass the array instruction and put the element type or whatever in it.
13:46:12
drmeister
Hello. I ran the static analyzer on clasp last night. I just pushed clasp_gc.cc. It should be good to go with shared-mutex.
13:46:52
beach
Bike: Since that subclass of the AREF instruction is something that every implementation likely wants, we might as well put it in Cleavir.
13:48:09
beach
There is no point in removing some feature from Cleavir that then has to be implemented in every client.
13:53:33
Bike
you could have an implementation where array upgrading only controls what's stored in array-element-type, and all arrays are accessed identically. or, say there's an implementation that has characters as immediate UTF32, and also has immediate 32-bit integers. both have an upgraded array type. at MIR level they could be identical, and the implementation doesn't need to distinguish them in hir.
13:54:17
Bike
maybe you're right though. and in clasp i could just subclass array to keep track of rank.
13:55:54
beach
For such implementations, there is very little harm done to have a type field in the AREF instruction.
13:56:08
Bike
actually that reminds me, what about complex arrays? it seems doubtful to me that an implementation could compile much down for a non-simple-array.
13:56:22
Bike
maybe it could if there were like, arrays that aren't adjustable but do have fill pointers.
14:00:16
Bike
in clasp i think an array access would ideally be (cond ((typeq array (not simple-array)) (full-call)) (t (let ((temp (if (typep array vector) array (md-array-underlying array)))) ...access into vector based on element type...)
14:02:20
beach
It depends on how you represent those arrays. SICL, for instance has a header and a "rack" for every object. The rack of a non-simple array contains the rank, so in a loop, for instance it is possible to have consistent access even if some other thread adjusts the array.
14:03:03
beach
Yes, some implementations may not be able to do much with non-simple arrays, but that's their choice.
14:04:27
beach
My plan for SICL is to implement adjust-array by allocating a new rack and then use CAS to replace the old rack.
14:04:46
Bike
does sicl have simple arrays (in the sense of arrays that are actually not adjustable, etc)
14:05:59
Bike
so right now cleavir aref tracks simple-p, which is inadequate for clasp and irrelevant to sicl.
14:06:06
beach
The other advantage of the header/rack thing is that no locking is required between the time the size is checked and the time the access is made.
14:07:51
beach
We used to have a host of aref instructions, and I believe you simplified that by putting the type into a slot.
14:08:41
Bike
currently aref instructions have three slots: the upgraded element type, simple-p, and whether the elements are boxed.
14:09:57
beach
All those three slots could potentially be part of a single extended type descriptor.
14:11:28
Bike
SBCL has "specialized array element type properties" structures. each one has an element-type, along with low level stuff like padding and the stride for memory reference, and probably stuff about boxing
14:13:31
Bike
let me check. the specifier, the sbcl internal type specifier, "fixnum-p", the default initial-element, n-bits, "typecode", "complex typecode", "primitive type name", and pad
14:15:38
Bike
anyway, what i can do for now is have clasp subclass the array asts/instructions/etc to store the rank, and cleavir doesn't have to be rewritten at all
14:18:45
beach
But yeah, CST is going well. Working on the documentation for lambda-list parsing at the moment.
14:19:18
beach
About the rank in Clasp. Like I said before, in general, my hunch is that it is not worth making special cases for a single access in a function, so it is worth thinking about how to simplify the code. I think, for instance, that making NIL both a CONS and a symbol (so to speak) is not worth the trouble.
14:19:19
beach
In a loop over the elements of a list, CONSP could be checked for first, making the SBCL optimization valid only for the last element in a list. Similarly, making a special case for a vector in order to make access to the size faster is probably not worth it. But, I advice against changing anything until we have loop-invariant code motion in Cleavir.
14:21:02
Bike
oh, it's not to get to the size faster, it's to get to the data. which i suppose is also loop invariant, yes
14:21:18
beach
And in fact, what SBCL does makes it necessary to do two tests in each iteration of a loop over a list. But we have been through this. stassats measured the performance and the additional test is not costing much. But, I am also concerned about simplifications of code maintenance.
14:21:59
beach
Yes, well, in general, I would think twice about optimizing cases for a single access.
14:23:01
Bike
that makes sense, but right now it just calls aref, so there's no chance of cleavir doing any code motion even if it did code motion.
14:24:11
Bike
and i don't think anyone was planning on nil = cons, as i remember that implies a lot of weird stuff in sbcl anyway
14:27:54
beach
Sure, but I often hear drmeister "justifying" a choice in Clasp by "that's how SBCL does it" as if that automatically meant it's the right choice.
14:27:57
Bike
a few days ago i fixed it so that calling nil didn't cause segfaults, so i don't want to add any more maintenance like that :p
14:28:26
Bike
drmeister: when you're not busy - why again are there separate C++ types for not simple vectors (as in simple array and vector, not simple-vector)? I mean, there could just be complex-mdarray, simple-mdarray, and then a constellation of simple vectors with element types.
14:30:31
Bike
sbcl is, you know, good, but i'd rather emulate the compiler than the runtime. the runtime is kind of scary.
14:30:46
Bike
i mean, so's the compiler, but that's mostly in terms of how it's written, not what it's doing.
14:31:12
beach
The result, seen by the application programmer, is very good indeed. I am not sure that it is as good for the maintainers.
14:32:20
beach
I do not want to plan to put myself (or anyone else for that matter) in a situation of having to maintain way too many special cases, especially if they can reasonably be avoided.
14:33:06
beach
Bike: Somewhat related: I think vtomole needs some additional training and experience before we can put him on the type stuff. For now it is probably better to stick to simpler tasks.
14:35:39
beach
About the maintainability issue, I think it's an age thing. I feel my time is running out. drmeister has considerably more time left and you even more, so it is natural that the issue is not as important to you as it is to me.
14:40:02
beach
About CST, I realized the other day that the work is not done when I finish the lambda-list parsers. I also need to implement PARSE-MACRO in terms of CST.
14:40:49
beach
Anyway, it's shaping up. Once I have the documentation in place, I'll ask you to have a look.
14:40:52
Bike
You did the macroexpansion guesser thing already, I think you said? and parse macro is closely tied to lambda list parsing anyway.
14:47:48
drmeister
Bike: There are separate C++ types because I wanted template function accessors in C++.
14:53:09
drmeister
This is a double helix that has been nicked twice to form segments: long-short-shortest
15:07:37
drmeister
Actually, one that returned references to values of that specialized array - so that I can use (*array)[i] = <value>;
15:09:14
drmeister
Then for each specialized array I subclass template_Array and template_SimpleArray and I have a non-simple and a simple specialized array.
15:10:04
drmeister
Also there is template_SimpleVector that provides the behavior for the simple vector for that type.
15:12:00
Bike
I'm thinking in terms of what should be done in llvm. say you have a simple mdarray or whatever it's called. then a read is just (mdarray->underlying)[row_major_index] regardless of the element type (because underlying has that). right?
15:12:02
drmeister
So really, there are just three C++ classes that need to be dealt with: template_SimpleVector, template_SimpleArray, template_Array - it is only those that layout structures in memory.
15:14:24
drmeister
But from LLVM - for the arrays that work with everything else it's not too bad. The layout is defined by the template_SimpleVector, template_SimpleArray, template_Array template classes.
15:16:32
drmeister
It's even simpler. There are only two memory layouts - for multi-dimensional/simple/non-simple arrays it is defined in MDArray_O and...
15:32:59
Bike
well, right, i guess i was just confused why there are so many classes for teh two layouts, since the XXXX is only in the latter.
16:20:31
drmeister
The only difference between a simple-array and a non-simple array is that the simple-array guarantees that the array it contains is not a displaced array