freenode/#clasp - IRC Chatlog
Search
21:01:53
drmeister
Bike: Thank you. I've merged your new master code into badge and I'm building. What about realloc do you need? The ability to resize the vector - right?
21:05:32
Bike
basically, with bignums it's frequent to want to allocate the worst-case size, and then shrink
21:12:37
drmeister
Yeah - resize. You are using this class - right ? https://github.com/clasp-developers/clasp/blob/master/include/clasp/core/bignum.h#L326
21:13:09
drmeister
So if you want to allocate for the worst-case size and then shrink - that's a very helpful thing to say.
21:13:53
drmeister
GCVector as a 'capacity' and a 'length' and we can make the length from zero to 'capacity' - it's a stretchy vector.
21:17:26
drmeister
You want the data in the object, we don't want the data in a separate object through an indirection.
21:19:48
drmeister
Basically though there isn't a realloc-like function - you allocate a new TheNextBignum_O. Then you can adjust the size by changing the _End slot.
21:20:59
drmeister
This is on the heap. Although you could allocate one on the stack I suppose - yes.
21:21:03
Bike
i have it allocating the worst case array on the stack, and then at the end allocating the actual bignum with the shrunken size.
21:21:54
drmeister
Yes - that will work. I can give you a function to calculate the size of TheNextBignum_O instance you want based on how many limbs you want.
21:22:27
Bike
alright now i'm staring at another fucking const qualification complaint from the compiler. how can i get the address of the array data without it being const qualified because that screws everything up for whatever reason
21:22:54
Bike
i don't understand. when i make a bignum the create function has a parameter that should just be the number of limbs. i don't care how much actual space it takes up
21:23:13
drmeister
You can use const_cast if it's really bad - I usually knuckle down and start declaring things const until the compiler stops complaining.
21:23:34
Bike
that's what i did, but then i hit a library function that i can't alter the declaration of
21:24:08
Bike
i have "mp_limb_t* limbs() const { return &(_limbs._Data[0]);}" but that ends up needing to return a const mp_limb_t* for some reason
21:24:12
drmeister
Yeah but if you alloca it you need the actual size of the entire object with header and tail if you want to use it from Common Lisp.
21:25:22
drmeister
So you don't want to manipulate this from Common Lisp? It's just temporary to get you to something you can allocate on the heap?
21:25:42
Bike
yes. scratch space, like i said. it's only used in the course of computing whatever arithmetic operation.
21:27:59
Bike
literally the only point of const is to help the compiler point out problems. no runtime effect. but instead it's just this ridiculousness
21:39:31
drmeister
So - do you need a 'realloc' - or is allocating an instance of TheNextBignum_O and then dialing down the length by adjusting _End good for what you need?
21:41:53
drmeister
That is a GCArray - that has one word for the length (positive or negative) followed by that many entries for data.
21:43:16
drmeister
The length will tell it how many limbs there are. Do you need to be able to specify a shorter length at this point? Or when you allocate this on the heap will you know precisely the number of limbs you need?
21:44:13
drmeister
Because I can implement a GCVectorSignedLength_moveable<limb_type> and that would have an extra word to store how many entries that follow are really limbs.
21:45:08
kpoeck
Bike drmeister with latest master I can again build asdf and serve-event. Regression Test run fine
21:45:51
Bike
let me explain the problem in more detail. say we subtract a 2-limb number from a 4-limb number. the result could have anywhere from 2 to 4 limbs. so first, we allocate space for four limbs, and write the result of the subtraction into that.
21:46:12
drmeister
Right? GCArray_SignedLength_moveable is basically this: template <class T> struct { int64_t _Length; T _Data[0]; }
21:46:15
Bike
then, we can check the most significant limbs, and if any are zero, shrink - we don't need the space a zero limb takes up.
21:46:50
drmeister
GCVector_SignedLength_moveable would be: template <class T> struct { int64_t _End; int64_t _Capacity; T _Data[0]; }
21:47:33
Bike
i don't understand why there would need to be a second count. i'm talking about actually freeing the memory in question.
21:47:36
drmeister
In the latter case you would allocate enough memory for _Capacity entries _Data[0..._Capacity-1]
21:48:57
drmeister
But adding an extra word for the _End would make dialing down the number of real limbs very cheap at the expense of having a few entries that follow wasting a bit of space.
21:49:54
Bike
the ONLY purpose this would have is to save some memory, which adding an extra word to every single bignum would obviously ruin
21:50:25
Bike
and in fact it would be worse, since the extra memory being allocated now is only temporary, unlike this extra word in every bignum
21:50:32
drmeister
Ah - I thought the purpose was because GMP needs to know the real number of limbs that represent the number.
21:51:17
Bike
so, i would allocate a number with eight digits, then notice that the top four digits are all 0, and so shrink the number to four digits
21:52:18
Bike
i cannot even begin to comprehend how this change i made adding functions to the-next-bignum that aren't actually used by anything anywhere could cause this boot problem
21:52:40
drmeister
It depends on what we want to optimize. If you are talking about say you have a number 0123123841 - is it worth GC allocating a whole 'nother bignum and copying 123123841 into it to save the space for that one '0'?
21:53:42
Bike
i can already do the extra long allocation. that is what i'm doing now. it's what i already described. i don't need any extra machinery to do that, it's easy.
21:54:16
drmeister
The only way to allocate one bignum and then shrink it means you need an extra word to store the length, the number of valid limbs. We need a word to store the ACTUAL, maximum number of limbs because MPS needs to know how to calculate the size of the bignum object.
21:54:38
Bike
i'm talking about actually shrinking it. the maximum number of limbs is the shrunken number.
21:55:14
drmeister
Actually shrinking it means allocating a smaller one, copying the values into it and allowing the GC to collect the larger one. That's like a realloc - that's expensive from the point of GC.
21:55:17
Bike
i mean, that's all it can do, since obviously in C there's no notion of array objects having an accessible length
21:56:22
drmeister
they allocate by bumping a pointer. I can't shrink an object after it's allocated with the MPS.
21:56:49
drmeister
I can allocate another, smaller object and copy into it and abandon the old one. But I can't shrink an existing object.
21:57:55
drmeister
I CAN created a special object that stores it's real capacity and then a length that is less than or equal to the real capacity and then dial down the length and just ignore the memory between the length and the capacity.
21:59:07
drmeister
Because I also have to make it handle the signed length that GMP uses for the sign of the bignum.
22:01:14
drmeister
Can you put in a sanity check that will test whatever is causing it to think whatever thing it thinks isn't setup yet at load time?
22:17:24
Bike
when clasp is fully up, if you compile a (make-instance 'class ...) form, a compiler macro rewrites that into (funcall (ensure-constructor-cell 'class ...) ...) or so
22:17:42
Bike
ensure-constructor-cell is a function that finds or makes a funcallable-object that implements the behavior of make-instance
22:18:28
Bike
now, the code involved here itself uses make-instance. but if those make-instance calls were rewritten in this way, you'd end up with the system trying to call ensure-constructor-cell before all the constructor cell machinery is ready
22:19:06
Bike
since a while back i fixed bclasp to respect notinline, all you have to do is declare make-instance notinline there, which is what my change did.
22:20:51
Bike
https://github.com/clasp-developers/clasp/commit/76baa9fe8c867a5a79d10993b2dbca2f41de7dc2#diff-6e225f117d3c2c8b7e67e6186fe0e42d as you can see here
22:21:25
Bike
the "apply #'make-instance" was an earlier workaround to accomplish the same goal of stopping the rewrite, but i busted that up a while back. the notinline thing should be more robust.
22:23:07
drmeister
Or there is still a problem and we will have to wait until it surfaces and then figure out why it's happening.
22:23:47
drmeister
Or wrap this thing in a (let ((*absolutely-dont-invoke-bclasp-compile-at-this-point* t)) ... and test in bclasp-compile if that dynamic variable is set?
22:44:07
drmeister
Here as well. I'm building the MPS version of cando now to see if this problem is the one that I was encountering.
0:53:31
Bike
the only thing in that file is like (defvar *english* (make-instance 'english)) or something.
1:03:08
drmeister
It really is helpful to use a debugger with some kind of a GUI - I'm using the -tui option with udb - which is like gdb.
1:06:43
drmeister
And codegen-startup-shutdown is a decent sized, regular looking function. What's up with that?
6:42:26
karlosz
so we've determined none of the recent type errors popping up is related to potential fall out from type inferenced being put into master?