freenode/#clasp - IRC Chatlog
Search
20:40:07
nivpgir
my initial though was that I didn't install the correct dependencies, that I didn't convert package names correctly somehow
20:41:10
drmeister
What we are seeing looks exactly like what is described here: https://tecnocode.co.uk/2014/10/01/dynamic-relocs-runtime-overflows-and-fpic/
20:43:26
drmeister
We followed the prescription "Another (highly related) solution is to link against a shared version of the static object."
20:43:50
drmeister
So we had a static object, we switched to the dynamic object and the problem persists?
21:28:09
drmeister
Once compile-file starts MPS slows down because mps_ap_fill takes 67.4% of the time.
21:28:32
drmeister
That's the function that is called when an allocation point runs out of space and has to allocate a new page to start filling.
21:59:43
drmeister
Cleavir uses hash-tables to traverse trees of instructions. It has to rapidly build up a hash table and then it throws it away.
22:30:48
drmeister
Bike: beach suggested that we could convert the map-instructions-xxx instructions into generic functions so that we can redefine them in clasp.
22:31:52
Bike
would take a lot of editing, and i don't know if it would be hugely better if we did the thread local thing
22:31:56
drmeister
That means adding a required parameter to each of them and defining a dynamic variable cleavir-ir:*instruction-mapper* that is by default NIL and calls the current functions.
22:34:22
drmeister
But is this how we would do it? Right - can't we define a special variable (defvar cleavir-ir:*instruction-mapper* nil) and then pass that everywhere in Cleavir? Or do we need to add a system argument to their callers and to their callers and so on?
22:35:06
Bike
by thread local thing i meant the "putting an extra slot in instructions that indicates whether they've been mapped over" dealie
22:36:39
drmeister
It wouldn't be hugely different but for the fact that allocations due to map-instructions-xxx are hammering MPS.
22:37:32
drmeister
There are there options - but it's hard to figure out if they will improve things wrt MPS performance until we try them.
22:38:02
drmeister
(1) Move to open hashed hash-tables - eliminate cons cells from hash tables. There are other good reasons to do this.
22:38:49
drmeister
(2) use stealth mixins and rewrite map-instructions-xxx to add a 'touched' slot to instructions.
22:43:21
drmeister
Well, I would argue that it's a bit perverse to be allocating so much memory just to walk trees over and over again.
22:45:20
drmeister
I probably won't hear from Ravenbrook until next week to get any ideas of how to improve this from the MPS size.
22:46:35
drmeister
Do we need to pass a system parameter into every function that calls map-instructions-xxx and so on?
22:47:01
Bike
either that or have a dynamic variable for the system, or something, but that would be unusual yeah
0:31:20
Bike
why would mps_ap_fill be called more by one kind of allocation in particular? unless that allocation is really common i guess...
0:33:28
drmeister
It is pretty common - every time you enter a function. They are supposed to be pruned if there is no return-from that can target them
0:49:01
Bike
i mean, if i do like (concatenate 'string "foo" "bar") i thought it'd give back a character strin
0:51:34
drmeister
foo allocate bar allocate baz allocate biz allocate tang allocate wang allocate cc_gatherRestArguments allocate
0:52:54
drmeister
How difficult would it be to write an optimization that checks if the rest variable escapes and if it doesn't use a &va-rest?
0:55:34
Bike
i was going to ask if it's called by the caller or by the &rest function itself, but i guess that's obvious
0:57:52
Bike
even without my fancy optimization, i could put in something to eliminate almost all calls to the symbol method. might as well give it a shot i guess.
0:59:16
Bike
though... how are there 129 calls to make-instance/symbol and only 3 to make-instance/class
1:02:19
drmeister
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/clos/builtin.lsp#L40
1:03:37
Bike
like i said, that one specifically i can deal with by other means. i'm wondering about some other ones, though
1:04:56
Bike
the standard allocate-instance methods ignore the initargs, but i guess the compiler doesn't know that so it gathers them anyway
1:05:57
drmeister
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/clos/standard.lsp#L38
1:06:34
drmeister
That one only iterates over the list or passes it to an error. I can make a copy and iterate over the copy.
1:07:26
Bike
i don't like having implementation stuff in these general methods... but the compiler isn't good enough right now, so i'll suck it up
1:12:48
drmeister
No, I can't copy a vaslist - I'd need to be able to allocate a 24byte structure in the stack frame and use that.
4:25:00
drmeister
Bike: I get one pass through a &va-rest list - even if I copy it into another variable.
4:26:20
drmeister
I would need a special operator to copy the &va-rest parameter to into another variable so that I can convert it to a list if there is an error.
4:27:52
drmeister
But we would have to know how large they need to be at compile time - we don't have that.
4:29:01
Bike
we would? you can have variable alloca... the "function" anyway, dunno the llvm semantics
4:45:19
drmeister
The only thing we lose is the ability to provide the arguments in the error message.
4:46:42
drmeister
If I could go (let ((saved-initargs (core:vaslist-copy initargs))) (declare (dynamic-extent saved-initargs)) ... )
4:52:08
drmeister
I could allocate it on the heap - but that kind of defeats the purpose of avoiding consing.
5:11:21
drmeister
Hoookay - that was a terrible idea. shared-initialize repeatedly loops over the initargs
6:00:38
drmeister
I changed less than 10 lines of code and now vaslist's are 64bytes rather than 32 and they keep a copy of their original value.