freenode/#clasp - IRC Chatlog
Search
16:14:11
drmeister
Damn - I can't connect to the process with lldb because Apple has nerfed their operating system.
16:17:30
drmeister
Here is the relevant part of the backtrace generated using lldb and then symbolicating it with jitted symbols.
16:17:57
Bike
i also tried tracing interpolate-function, which is what does inlining, and it's not being called
16:22:25
drmeister
Ok, I'll generate the llvm-ir and look at it. It's not being generated at the moment.
16:24:29
karlosz
i forgot to mention, but the new changes now make it so we use sjlj for the lexical non local exits in Eclector, so there is no more unwinding for terminate-token in particular
16:27:18
Bike
i had to merge your changes with mine, so i suppose i could have screwed it up somehow, but it seemed pretty straightforward
16:27:37
karlosz
the l_... versions of the functions are just the "inner" functions that get called by the external entry point
16:27:54
karlosz
i was trying to make it so that the l_... versions of the function are fastcc but couldnt figure how
16:28:15
karlosz
as you can see a lot of the times both the l_ version and the non-l version exist of those exist in the backtrace
16:30:19
karlosz
because i marked the inner functions private i think that gives llvm liberty to automatically make the functions i was trying to make fastcc
16:32:10
Bike
karlosz, different functions in the same file should be in different bir modules, right? they're not local calls
16:33:21
drmeister
I was expecting a call chain line REPL-[regular-call]>AAA(ccc)-[tail-call]>AAA(fastcc)-[regular-call]>BBB(ccc)-[fast-call]>BBB(fastcc)...
16:35:13
drmeister
How can this even work? I thought this was impossible - a tail call to a varargs function???
16:35:35
Bike
ok well iirc llvm can ignore the tail annotation, so they wouldn't be actual tail calls
16:39:56
drmeister
Can you guys figure out what is going on? Is this not an abomination? %3 = tail call { i8*, i64 } (i8*, i64, i8*, i8*, i8*, i8*, ...) %entry-point-gep-gep(i8* %2, i64 1, i8* %0, i8* undef, i8* undef, i8* undef), !dbg !33
16:41:38
Bike
https://github.com/clasp-developers/clasp/commit/525bc12f9e5607ca9d2ea3d9a33ad80dc60568e4#diff-6a3dee58068bbd521a10f7eb383dea195b100bebe4256fbbacf681dcacd5a1ecR801-R810 or tail, even? what the heck
16:44:50
drmeister
You mean frame-pointer-elimination? We take trouble to NOT do frame-pointer-elimination because that damages the backtraces.
16:45:14
drmeister
Not like this though - we see the backtrace - it's not truncated (which is what fp elimination does).
16:46:14
Bike
i mean i thought sbcl mgiht do frame pointer elimination on things marked as tail call even if it doesn't do an actual tail call
16:47:19
karlosz
i just marked the inner function as "private" linkage, and the external entry points are still internal linkage
16:48:42
drmeister
karlosz: I've used internal or private linkage before - indeed - I make sure nothing is external linkage other than one function to initialize the module.
16:50:16
karlosz
even when i was making my local call change, i never saw fastcc show up when doing (disassemble ... :type :ir)
16:52:32
drmeister
I wouldn't trust disassemble - or rather - I would absolutely trust the .ll file you get from (cmp:compile-file-serial ...) if you turn on USE_HUMAN_READABLE_BITCODE in wscript.config
16:53:14
drmeister
The .ll file that you get from compile-file-serial contains absolutely everything that the compiler generates and that the runtime will do when it runs the code.
16:53:33
karlosz
in the ll file you can see there is still a distinction between tailcall fastcc and tail call
16:53:56
karlosz
interestingly enough the tailcall fastcc's are keeping the backtrace info but not plain tail calls
16:55:42
karlosz
not sure where in the BIR process did we start telling llvm its okay to always use tail calls
17:02:46
karlosz
i think the backtrace issue where the frames for tail calls are missing is indeed one of the issues with the debug info we were seeing but Bike should confirm
17:06:29
Bike
my comlpaint was frames looking like "_DDD^COOMON-LISP-USER^FN^^.11" with no arguments
17:07:42
drmeister
Ok, the no arguments is probably because you aren't setting up the argument save area and you aren't spilling the arguments into it.
17:08:10
drmeister
None of the functions have that big annoying block of argument spilling code in them. That's why there are no arguments.
17:09:01
drmeister
This needs to be called... https://github.com/clasp-developers/clasp/blob/bir/src/lisp/kernel/cmp/cmpintrinsics.lsp#L680
17:10:00
drmeister
I think the function names are weird because they aren't being demangled somewhere in the backtrace processing code for lisp frames.
17:10:22
karlosz
right, it should be eeasy to generalize the code that makes the 2 entry points into making multiple
17:10:29
drmeister
I think it uses that mangled name to come up with the name that it prints in the backtrace for lisp frames.
17:11:33
drmeister
Also, we will want to conditionally spill registers - so we can turn it off for high optimization levels. But I like seeing arguments in backtraces and I'll happily pay a bit of performance to get them.
17:12:15
drmeister
Ok, can you guys figure out what is going on from here and decide how we want things to work?
17:13:17
karlosz
Bike: given what drmeister said, i think you just need to add the call to maybe-spill-to-register-save-area in layout-xep-function like where it was in HIR
17:13:31
drmeister
I see: 1. Not spilling register arguments; 2. Frame names need to be demangled and the demangling code may not recognize some new mangling that is going on - find the demangling code and figure out what's up with it; 3. WTF is going on with fastcc and tail calls?
17:14:04
drmeister
3. (cont) and once we figure out what is going on with fastcc and tail calls - can we use it constructively?
17:14:39
karlosz
I think fastcc is being used correctly, will have to figure out whats going on with the plain tail calls and how to fix the backtrace
17:16:43
drmeister
Are the tail calls with the C calling conventions legal? Or is this some kind of bizarro undefined behavior?
17:21:14
Bike
it doesn't seem like a tail marker with the c convention is illegal so much as that it won't be optimized as a tail call
17:21:39
karlosz
it seems like it is a tail call in this case because we are losing the stack frame precisely when we have non fastcc call
17:24:32
Bike
and i don't think there's any new mangling. i mean, i can see how l_ would screw it up, but it's not unmangling the regular ol function names either
17:29:17
karlosz
yes i think the l_ functions are purely just from the main function entry point i added which is fine
17:33:55
Bike
i'll just make it always t and see what happens and then worry about policy communication
17:41:51
karlosz
so i was thinking about how to get rid of the continuation variable in unwind and i think this approach might be cleaner than what we have right now: 1. Define a new class LEXICAL which lexical-variable and catch both subclass. LEXICALs are things that can go into environments. Instead of closing over the continuation variable, we just close over the catch instruction directly instead.
17:42:09
karlosz
This has a few benefits: No strange non lisp object rtype needed for the continuation object
17:42:28
karlosz
No need to maintain the readvar stuff in unwind - it already has a slot for its catch
17:42:34
Bike
the rtype is just a reflection of what's actually happening on the lower level. it's not a tagged pointer
17:43:13
karlosz
Since the contvar is already mutable, we can simply identify it with the catch instruction anyway
18:59:01
karlosz
it would be good to figure out why cst->ast is just such a slow phase of compilation especially when bir and and eclector now hardly make a dent
18:59:37
karlosz
we have a pretty broad sjlj optimization so hopefully we run into unwind less on stupid stuff like lexical non local exits
19:05:20
drmeister
I just tried to build BIR and it dropped into the repl right after loading inline.lisp
19:06:54
Bike
the change to improve the backtraces has a change in cleavir too, so you have to get that
21:21:10
drmeister
With the sea-of-nodes (SON) compiler about half the time was spent in llvm and the other half in cleavir
21:22:14
drmeister
In llvm much of that time was spent in instruction combining. I talked with Ian Rogers (a programmer at Google) and he thought that was strange. He sees lots of time spent in register selection.
21:25:42
karlosz
drmeister: bir is definitely faster than hir, compile time wise. most of the time is now spent in cst->ast especially since eclector no longer spends so much time doing terminate-token
21:32:59
Bike
the initial bir seems to involve writing variables to themselves... not sure what's going on there
21:43:56
Bike
i put in something so that different datums with the same names should have different names in the disassembly
22:11:44
Bike
well, local call says "for" this function, not "of" this function, maybe it's calls in the body?
22:35:43
Bike
here's an example: say an enclose instruction is deleted as unreachable. it looks like the remove-use method will try to remove the enclose from the local-calls
22:37:19
Bike
would it make sense to just maintain the USES set and compute the local-calls and encloses set from it, or would that be inefficient?
22:39:59
Bike
oh, hey, there's something in find-module-local-calls saying the refresh-iblocks call can be deleted once tagbody ast-to-bir is fixed
22:40:39
karlosz
Bike: also since code is not an input of enclose, i don't think the situation you described with remove-use can happen
22:42:01
karlosz
it would be good to figure out how to make the inputs/additional slots situation clearer or more uniform
22:42:44
Bike
readvar and writevar originally didn't have the variable as an input or output and it was a little uglier that way in my opinion
22:43:12
Bike
the downside is that now you don't know if an input is linear or not immediately, but i think that's alrigth
22:44:00
karlosz
i guess right now we're just simplifying bir now that it seems to more or less work
22:44:26
Bike
i'm also writing documentation again, in the hope that i won't have to rewrite it much when we change stuff in a week
22:54:39
karlosz
The symbol DISASSEMBLE-INSTRUCTION is bound to an ordinary function and is not a valid name for a generic function
23:26:09
karlosz
okay, so optimizations in BIR we have over HIR: sjlj for local functions, local calls, zero heap allocation closures
23:29:40
karlosz
bir will have much better type inference, contification is straight forward to implement in bir
23:40:46
Bike
like if you only care about terminators, like in reduce-typeqs, mapping over the rest is dumb
23:42:03
karlosz
i ran into a nasty bug when i was doing local call stuff because i forgot map-instructions looks recursively for enclose
23:42:39
karlosz
eclector is still unwinding but thats because the bir one isn't using the no-signals branch i think
23:43:49
karlosz
verifier takes about the same amount of time as the rest of the bir passes combined
23:45:52
karlosz
Cannot add the method #<STANDARD-METHOD UNNAMED (CLIM-INTERNALS::|(presentation-type COMMON-LISP::NULL)| T T CLIM:TEXTUAL-VIEW)> to the generic function #<PRESENTATION-GENERIC-FUNCTION CLIM-INTERNALS::%ACCEPT> because their lambda lists (TYPE-KEY TYPE STREAM VIEW &KEY) and (TYPE-KEY TYPE STREAM VIEW &KEY DEFAULT DEFAULT-TYPE &ALLOW-OTHER-KEYS) are not congruent.
23:48:01
Bike
https://github.com/clasp-developers/clasp/commit/8a33acbb09357531ff645576d883542e7870d47f
23:51:16
karlosz
Bike: what ended up happening to the type inference stuff? did the transforms pan out?
23:52:48
Bike
i think the approach i want to take is having the transform work by just returning an ir function, which is then inlined
23:53:02
Bike
so all the actual rewriting code is just part of inlining and the transform doesn't have to deal with it
23:54:41
Bike
i mean, what i'm thinking is that the transform mechanism, which looks at types and stuff, doesn't really care how the transform is implemented
23:54:59
Bike
the transform could do that kind of macroexpansion approach, or it could use a static function (like it would for the simple CAR rewrite), or whatever
23:58:30
Bike
https://gist.github.com/Bike/678179dd7eadb55ff50114e85360ec35 as you can see it's very basic
0:01:07
Bike
i put the derived types in a hash table mostly because i didn't want to go through the process of adding a slot to data
0:03:48
Bike
it'll work with variables, but it's only the "local" propagation, it doesn't know about typew or control flow generally
0:05:13
karlosz
when (- (+ (the small-fixnum x) (the small-fixnum y)) 2) derives the output type of + to be fixnum, we can just worklist the rest so it keeps firing for -
0:07:04
Bike
i know you've been thinking of things in terms of known calls, but i think if possible i'd like to attach that derivation information to function types and propagate like anything else
0:07:20
karlosz
well, if you had a slot on the linear-datum directly, you can just use the same "rewriting" mechanism
0:07:24
Bike
would be cool if we could determine (funcall (if ... #'+ #'-) fixnum fixnum) an integer, that kind of thing
0:09:02
Bike
i mean i was imagining that the function type would have the whole type deriver function shebang attached
0:16:50
karlosz
if we can get the set of primops in HIR to get compiled in bir but without the inlining and delete machinery id say thats a win
0:22:31
Bike
with the local call changes it might just be printing the external entry point, or something
0:23:05
Bike
https://github.com/clasp-developers/clasp/blob/master/src/lisp/kernel/cmp/disassemble.lsp#L34-L43
0:44:08
Bike
the local call stuff might really require some rethinking there, though, since there are multiple llvm functions to disassemble
1:00:29
karlosz
im thinking of how to do contification more elegantly with hir because we have a built in way to merge with phis
1:01:01
karlosz
so seeing if theres a good abstraction to be made out of making blocks with phis and dealing with that kind of thing
1:02:31
Bike
llvm has the concept of a module, but by the time you're looking at a function object it's gone
1:04:05
Bike
i mean, it's a compiler thing. it's gone for the same reason the bir module is gone. you just have the binary left
1:04:36
Bike
obviously the function has a pointer to the start of the code, but that's about it. i don't remember how we even find the end
1:15:11
karlosz
yeah im disappointed that our runtime function objects don't have more stuff attached to them
1:16:39
karlosz
and the code component object is like the runtime representation of a module and has the metadata for that group of compiled together functions/entry points
1:17:09
karlosz
and plus the GC needs to know about it so the thing can get collected when its no longer referenced