freenode/#clasp - IRC Chatlog
Search
20:19:30
kpoeck
with commit c802f5f75c510ac227e863334953d0517f2c7a42 Tue Jul 21 21:12:08 2020 -0400 Tue Jul 21 21:12:08 2020 -0400 i can build fine
20:34:45
drmeister
Hey - we are hitting a problem that looks like the problem you hit. Or it might be a different problem. Bike is currently working on it.
20:35:35
drmeister
You keep coming into irc with a different ID. I worry sometimes that notifications won't get through to you.
20:36:20
drmeister
kpoeck67: Ok. If you are getting the problem in codegen-startup-shutdown then that really looks like the problem we are all hitting.
20:40:10
drmeister
I was too - it was on Bike's desk at Temple University. Now it's at ThirdLaw - but I'm thinking of moving it back to my house now that my fiber optic line is back up and not in danger of being ripped out by falling trees.
20:40:41
kpoeck67
I now share my office with a guy from Brazil, so trying to learn Portuguese now to mix up my languages even more
21:01:53
drmeister
Bike: Thank you. I've merged your new master code into badge and I'm building. What about realloc do you need? The ability to resize the vector - right?
21:05:32
Bike
basically, with bignums it's frequent to want to allocate the worst-case size, and then shrink
21:12:37
drmeister
Yeah - resize. You are using this class - right ? https://github.com/clasp-developers/clasp/blob/master/include/clasp/core/bignum.h#L326
21:13:09
drmeister
So if you want to allocate for the worst-case size and then shrink - that's a very helpful thing to say.
21:13:53
drmeister
GCVector as a 'capacity' and a 'length' and we can make the length from zero to 'capacity' - it's a stretchy vector.
21:17:26
drmeister
You want the data in the object, we don't want the data in a separate object through an indirection.
21:19:48
drmeister
Basically though there isn't a realloc-like function - you allocate a new TheNextBignum_O. Then you can adjust the size by changing the _End slot.
21:20:59
drmeister
This is on the heap. Although you could allocate one on the stack I suppose - yes.
21:21:03
Bike
i have it allocating the worst case array on the stack, and then at the end allocating the actual bignum with the shrunken size.
21:21:54
drmeister
Yes - that will work. I can give you a function to calculate the size of TheNextBignum_O instance you want based on how many limbs you want.
21:22:27
Bike
alright now i'm staring at another fucking const qualification complaint from the compiler. how can i get the address of the array data without it being const qualified because that screws everything up for whatever reason
21:22:54
Bike
i don't understand. when i make a bignum the create function has a parameter that should just be the number of limbs. i don't care how much actual space it takes up
21:23:13
drmeister
You can use const_cast if it's really bad - I usually knuckle down and start declaring things const until the compiler stops complaining.
21:23:34
Bike
that's what i did, but then i hit a library function that i can't alter the declaration of
21:24:08
Bike
i have "mp_limb_t* limbs() const { return &(_limbs._Data[0]);}" but that ends up needing to return a const mp_limb_t* for some reason
21:24:12
drmeister
Yeah but if you alloca it you need the actual size of the entire object with header and tail if you want to use it from Common Lisp.
21:25:22
drmeister
So you don't want to manipulate this from Common Lisp? It's just temporary to get you to something you can allocate on the heap?
21:25:42
Bike
yes. scratch space, like i said. it's only used in the course of computing whatever arithmetic operation.
21:27:59
Bike
literally the only point of const is to help the compiler point out problems. no runtime effect. but instead it's just this ridiculousness
21:39:31
drmeister
So - do you need a 'realloc' - or is allocating an instance of TheNextBignum_O and then dialing down the length by adjusting _End good for what you need?
21:41:53
drmeister
That is a GCArray - that has one word for the length (positive or negative) followed by that many entries for data.
21:43:16
drmeister
The length will tell it how many limbs there are. Do you need to be able to specify a shorter length at this point? Or when you allocate this on the heap will you know precisely the number of limbs you need?
21:44:13
drmeister
Because I can implement a GCVectorSignedLength_moveable<limb_type> and that would have an extra word to store how many entries that follow are really limbs.
21:45:08
kpoeck
Bike drmeister with latest master I can again build asdf and serve-event. Regression Test run fine
21:45:51
Bike
let me explain the problem in more detail. say we subtract a 2-limb number from a 4-limb number. the result could have anywhere from 2 to 4 limbs. so first, we allocate space for four limbs, and write the result of the subtraction into that.
21:46:12
drmeister
Right? GCArray_SignedLength_moveable is basically this: template <class T> struct { int64_t _Length; T _Data[0]; }
21:46:15
Bike
then, we can check the most significant limbs, and if any are zero, shrink - we don't need the space a zero limb takes up.
21:46:50
drmeister
GCVector_SignedLength_moveable would be: template <class T> struct { int64_t _End; int64_t _Capacity; T _Data[0]; }
21:47:33
Bike
i don't understand why there would need to be a second count. i'm talking about actually freeing the memory in question.
21:47:36
drmeister
In the latter case you would allocate enough memory for _Capacity entries _Data[0..._Capacity-1]
21:48:57
drmeister
But adding an extra word for the _End would make dialing down the number of real limbs very cheap at the expense of having a few entries that follow wasting a bit of space.
21:49:54
Bike
the ONLY purpose this would have is to save some memory, which adding an extra word to every single bignum would obviously ruin
21:50:25
Bike
and in fact it would be worse, since the extra memory being allocated now is only temporary, unlike this extra word in every bignum
21:50:32
drmeister
Ah - I thought the purpose was because GMP needs to know the real number of limbs that represent the number.
21:51:17
Bike
so, i would allocate a number with eight digits, then notice that the top four digits are all 0, and so shrink the number to four digits
21:52:18
Bike
i cannot even begin to comprehend how this change i made adding functions to the-next-bignum that aren't actually used by anything anywhere could cause this boot problem
21:52:40
drmeister
It depends on what we want to optimize. If you are talking about say you have a number 0123123841 - is it worth GC allocating a whole 'nother bignum and copying 123123841 into it to save the space for that one '0'?
21:53:42
Bike
i can already do the extra long allocation. that is what i'm doing now. it's what i already described. i don't need any extra machinery to do that, it's easy.
21:54:16
drmeister
The only way to allocate one bignum and then shrink it means you need an extra word to store the length, the number of valid limbs. We need a word to store the ACTUAL, maximum number of limbs because MPS needs to know how to calculate the size of the bignum object.
21:54:38
Bike
i'm talking about actually shrinking it. the maximum number of limbs is the shrunken number.
21:55:14
drmeister
Actually shrinking it means allocating a smaller one, copying the values into it and allowing the GC to collect the larger one. That's like a realloc - that's expensive from the point of GC.
21:55:17
Bike
i mean, that's all it can do, since obviously in C there's no notion of array objects having an accessible length
21:56:22
drmeister
they allocate by bumping a pointer. I can't shrink an object after it's allocated with the MPS.
21:56:49
drmeister
I can allocate another, smaller object and copy into it and abandon the old one. But I can't shrink an existing object.
21:57:55
drmeister
I CAN created a special object that stores it's real capacity and then a length that is less than or equal to the real capacity and then dial down the length and just ignore the memory between the length and the capacity.
21:59:07
drmeister
Because I also have to make it handle the signed length that GMP uses for the sign of the bignum.
22:01:14
drmeister
Can you put in a sanity check that will test whatever is causing it to think whatever thing it thinks isn't setup yet at load time?
22:17:24
Bike
when clasp is fully up, if you compile a (make-instance 'class ...) form, a compiler macro rewrites that into (funcall (ensure-constructor-cell 'class ...) ...) or so
22:17:42
Bike
ensure-constructor-cell is a function that finds or makes a funcallable-object that implements the behavior of make-instance
22:18:28
Bike
now, the code involved here itself uses make-instance. but if those make-instance calls were rewritten in this way, you'd end up with the system trying to call ensure-constructor-cell before all the constructor cell machinery is ready
22:19:06
Bike
since a while back i fixed bclasp to respect notinline, all you have to do is declare make-instance notinline there, which is what my change did.
22:20:51
Bike
https://github.com/clasp-developers/clasp/commit/76baa9fe8c867a5a79d10993b2dbca2f41de7dc2#diff-6e225f117d3c2c8b7e67e6186fe0e42d as you can see here
22:21:25
Bike
the "apply #'make-instance" was an earlier workaround to accomplish the same goal of stopping the rewrite, but i busted that up a while back. the notinline thing should be more robust.
22:23:07
drmeister
Or there is still a problem and we will have to wait until it surfaces and then figure out why it's happening.
22:23:47
drmeister
Or wrap this thing in a (let ((*absolutely-dont-invoke-bclasp-compile-at-this-point* t)) ... and test in bclasp-compile if that dynamic variable is set?
22:44:07
drmeister
Here as well. I'm building the MPS version of cando now to see if this problem is the one that I was encountering.
0:53:31
Bike
the only thing in that file is like (defvar *english* (make-instance 'english)) or something.
1:03:08
drmeister
It really is helpful to use a debugger with some kind of a GUI - I'm using the -tui option with udb - which is like gdb.
1:06:43
drmeister
And codegen-startup-shutdown is a decent sized, regular looking function. What's up with that?
6:42:26
karlosz
so we've determined none of the recent type errors popping up is related to potential fall out from type inferenced being put into master?