freenode/#clasp - IRC Chatlog
Search
15:07:29
beach
So, take SORT for instance. The function that calls SORT (call it F) has a single environment active while sort is executing, so the environment created when F is called is destroyed when F exists.
15:09:25
beach
But, again, if F is only called once during the invocation of ITS caller, then we are again OK.
15:09:30
Bike
if F calls some unknown function and one of the arguments in the call is a closure, just to be clear?
15:12:39
beach
We are also OK in the (unusual) situation where the closure created by F when it calls ARBITRARY does not refer to any of the variables in F's environment.
15:16:13
beach
So it boils down to determining whether it is fine for the callee environment to be part of the caller environment, and this is fine if and only if no more than one instance of the callee environment is active at any point in time.
15:18:36
beach
Interestingly, whether the callee accesses or modifies variables in its caller, does not influence the inlinability.
15:23:42
Bike
If you have (lambda (x) (let () (lambda () x))), the implicit LET-lambda can still be inlined, even though it has a closure referring to the caller's environment, right? is returning a closure okay, just as long as it's not passed to an arbitrary function?
15:25:00
beach
Yes, like I said, references to the caller's environment don't influence the decision. In this case the (let () ...) has an empty environment, so it can trivially always be shared with that of its caller.
15:25:30
Bike
drmeister: clos::set-funcallable-instance-function checks whether the new value is a funcallable instance (with asOrNull<FuncallableInstance_O>) and then passes to another function. but we pass it symbols all the time, what's going on?
15:26:53
Bike
so in (lambda (x) (let ((y x)) (lambda () y))) we couldn't inline the let-lambda, at least not without further analysis, because every time the outer lambda is called the let-lambda should make a new environment.
15:30:37
beach
The good news is that the vast majority of cases are very simple, so we can get away with a simple analysis initially and work on more sophisticated ones later.
15:32:11
beach
For value numbering and type inference, we must first determine the set of variables that can be counted on to be modified ONLY as a result of the usual control flow. Clearly, variables that are not accessed in nested functions qualify. Whether shared variables qualify depends on considerations similar to the ones about inlining.
15:34:45
beach
In fact, if a nested function (say F) that modifies some variable of its caller can be inlined, then by inlining it, we have normal control flow again.
18:05:30
drmeister
I tried that new profiling code and by scanning the results I identified that (llvm-sys:cxx-data-structures-info) was in 7% of the backtraces because I used that function in several symbol-macros.
19:18:50
drmeister
I think the instance sig is a pointer to the list of effective-slot-definitions for the objects. It's used to rewrite the object when it changes class or is updated. instance-sig-set sets that.
19:22:50
drmeister
Here's my guess. In ECL there is only one kind of instance and it's funcallable. Generic functions are instances and so they need a signature.
19:23:12
drmeister
But would you ever change the class of a generic function or redefine the generic-function class?
19:24:10
drmeister
I'm trying to find the nice comment in the ECL source that describes the 'sig' of an instance.
19:26:32
drmeister
Here: https://github.com/drmeister/clasp/blob/dev/src/lisp/kernel/clos/change.lsp#L113
19:28:43
drmeister
It uses class-slots to figure out if the object is invalid - we don't need to do that.
19:29:21
drmeister
(eq (instance-sig x) (class-slots (class-of x))) is the test that ECL uses to determine if an instance is obsolete.
19:29:47
Bike
so the obvious thing to do to satiate compute-effective-method is to use an anonymous generic function
19:30:52
Bike
yeah, and it uses a semi-anonymous generic function, but that doesn't work directly for us i don'tt hink
19:32:33
Bike
the specializer profile probably says to ignore the last two parameters, and standard-generic-function doesn't have any subclasses we care about i don't think
19:34:15
drmeister
Then we can put it together as a bogus call history and call the compiler for the discriminator.
19:35:23
Bike
i'm not sure what generic functions are needed for satiation... there's that one, and maybe some accessors? i don't know when accessors ar edefined properly.
19:35:53
drmeister
Right - so we can compile discriminator functions all the way down in aclasp. All you need is a specializer vector, a specializer profile (T NIL NIL) in this case and an effective method function.
20:25:17
Bike
compute-effective-method, the final one, is declared with optimize speed (safety 0). that's maybe the weirdest optimization target i've seen
20:25:58
Bike
it also means that if you call it with something other than a method combination, clasp hits an implement_me and dies
21:38:04
Bike
the method uses early accessors, so it calls instance-ref on whatever you give it, which in my case was 4
21:38:32
Bike
probably would be more appropriate if it was some kind of type error rather than implement_me, since we don't need instance_ref on random things
21:39:07
Bike
the particular fix would be to have compute-effective-method check that it's getting a method combination, though. minor stuff
21:39:37
Bike
anyway, i rewrote combin.lsp to compile later, so it should be possible to compute a proper effective method that we can use in fastgf
21:39:54
Bike
i hit some weird error where it couldn't find the fdefinition of a function i deleted, so i'll just let it build and go
21:40:02
drmeister
instanceRef and instanceSet are virtual functions - from way back where classes were separate things from instances
21:43:12
specbot
standard-instance-access: http://metamodular.com/CLOS-MOP/standard-instance-access.html
21:44:33
Bike
yeah, it says "function" but it's actually an accessor. so (setf standard-instance-access)
21:46:02
drmeister
OTOH I can change core:instance-ref and core:instance-set to use the new accessor.
21:47:08
drmeister
Do these functions do error checking to see if the object they are manipulating is an instance or funcallable instance?
22:13:00
drmeister
The FOO-DEBUG function is at (debug 3) and the little blip to the right of it is FOO-RELEASE (debug 0)
22:59:19
drmeister
Lets say I want generic function dispatch to be as fast as possible - (1) I want to dispatch on registers as much as possible but if there are more required arguments than registers I need to transition to a va_list once I run out of registers.
23:01:30
drmeister
(2) I need to save and rewind the va_list on entry to the generic function discriminator because the methods may need the argument list.
23:02:20
drmeister
(3) If we have fast-methods then in those cases I can dispense with #2 and just pass registers to the fast-methods.
23:28:01
drmeister
Here's a thought. The call-history is an alist of (specializers . effective-method-function/optimized-slot-(reader|writer) )
23:28:44
drmeister
If we wrap the effective-method-function in a struct that stores some additional information with it...
23:29:43
drmeister
Mostly I want to know in the dispatcher if I need to generate a vaslist and rewind it to handle all of the arguments.
23:30:13
drmeister
So if I know that none of the effective-methods need a vaslist - I won't bother allocating space or initializing it.
23:30:47
drmeister
Does the generic-function lambda-list reflect what all of the methods are looking for?
23:31:37
Bike
but all the methods have to have the same number of required arguments, optional arguments, and whether to accept rest arguments
23:31:43
drmeister
But if any of them take keyword arguments - then the generic function lambda list has to reflect that - correct? With &key and &allow-other-keys?
23:32:26
specbot
Congruent Lambda-lists for all Methods of a Generic Function: http://www.lispworks.com/reference/HyperSpec/Body/07_fd.htm
23:33:03
Bike
okay, so i guess actually no, you could have a method that takes &key while the generic function only has &rest.
0:26:25
drmeister
Basically, if we wrap the effective-method-functions in the call history with a struct that stores optimization information about them - specifically if they can use registers or they need a list of arguments...
1:53:56
drmeister
How could we manage that other than having llvm do the inlining? The discriminator compiler is a special purpose compiler that doesn't follow bclasp or cclasp conventions.
1:56:56
drmeister
I needed a compiler that wouldn't use any generic functions and bclasp generates shitty code - so I wrote a custom compiler for it.
1:57:50
drmeister
What goes on in GF dispatch is really simple compared to general code - (1) read stamp, compare to integer ranges - repeat.
1:58:17
drmeister
Also, there are no issues with closures or anything complicated - so inlining could be done perfectly well using llvm inlining.
2:02:07
Bike
i want to have it so that the discriminating function computer or whatever calls compute-effective-method, gets the effective method, and processes it in a lexical environment with helpful macros
2:02:34
Bike
So like, each outcome could have a call-method definition that looks at the method, and if it's a standard accessor, just expands into standard-instance-access for the particular class slot location
2:03:32
Bike
and we could expand it, so, for example, put a slot in methods that indicates whether it calls next methods
2:03:45
Bike
the call-method macro can look at that slot, and if the method doesn't do anything like that be more direct about it
2:04:09
drmeister
Can we set it up so that all of this processing happens ahead of time and the results are stored in the outcome of the call history. And do it in closfastgf.lsp or some other clos file.
2:05:08
drmeister
You could put all of that optimization information in a 'outcome' struct and the cmpfastgf.lsp compiler will act on it.
2:06:41
drmeister
Ahead of the time the discriminating function is compiled as a result of an invalidated-dispatch-function call.
2:07:21
Bike
well, it can be done whenever the compute-effective-method-maybe-optimize thing is called
2:08:09
drmeister
One stage happens during a dispatch miss. It does compute-applicable-methods-xxx and analyzes the methods that would go into the effective method, builds the effective method and stores the optimization analysis with it in the 'outcome' of a call-history entry.
2:09:21
drmeister
Yes, the slot location is calculated in the first stage and stored in an optimized-slot-reader or optimized-slot-writer structure.
2:09:47
Bike
well, t hat's fine then. we just reduce the "outcomes" back to being functions, but inline the functions
2:11:03
drmeister
Yes - that will work - we just need to store the llvm-ir in the outcome and then add it to the llvm Module of the discriminating function.
2:19:34
Bike
right now we have every outcome be something like (funcall emf args nil) as far as i can tell
2:20:02
drmeister
We are coming from different worlds - I don't see how inlining makes it simple - an llvm function still needs to accept arguments.
2:20:28
drmeister
Accessors are easy to inline - I'm essentially doing it already except the for some reason they aren't actually inlined.
2:20:59
drmeister
I'm thinking more general effective methods that take 0-4 required+optional arguments.
2:21:34
Bike
okay, so say we have (defgeneric foo (x y)), then our discriminator can be (once you fix things) (lambda (x y) (tagbody ... ...effective method...)), right? but with llvm weirdness.
2:22:02
Bike
and something in there puts x and y in a va-list, and effective methods are called with that va-list.
2:23:16
drmeister
But if none of the effective methods need a va-list and they all take two arguments - then you want (lambda (x y) (tagbody ... (funcall fast-effective-method x y)))
2:23:49
Bike
if we could compute the effective method form and just put that there instead, it could refer to variables directly
2:25:20
drmeister
Yes, in Common Lisp land - but we aren't quite in Common Lisp land. The discriminator is a special purpose, pure llvm-ir function that calls other llvm-ir functions.
2:26:25
drmeister
So you could compile fast-effective-method to an llvm-ir function that accepts a closure, number of arguments and two arguments and then in the discriminator you call foo(closure,2,x,y) and then inline the first function at that call site.
2:28:03
Bike
if there are effective methods, they just get the arguments to the gf and maybe the gf as arguments
2:29:45
drmeister
The general Clasp calling convention is foo(closure,num-args,farg0,farg1,farg2,farg3,...)
2:30:35
Bike
the effective method function is pretty special purpose and we basically want it to be inlined, so i don't see any need to conform to practically anything if it doesn't have to
2:32:35
Bike
an effective method is certainly not a closure, the clhs is even weirdly explicit about what's in its environment
2:33:30
drmeister
A closure is an environment and a function. When the function is called it is passed its environment.
2:35:54
Bike
and potentially whatever the user specifies but that's exotic and underspecified so i'm not worried about it
2:37:08
Bike
"When a method is actually called by an effective method, its first argument will be a list of the arguments to the generic function. Its remaining arguments will be all but the first argument passed to call-method."
2:37:48
Bike
but i'm not too worried about it because the closer mop guy wrote a paper about how much this sucks
2:40:01
Bike
i mean, there are a few other problematic things, like here call-next-method gets actual methods in the next-methods, but both clasp and sicl pass the method functions
2:59:36
Bike
anyway, if we're smarter with the result of compute-effective-method, we can use the fact that call-method is a macro but gets actual methods as argument to flexibly change the method invocation protocol