freenode/#clasp - IRC Chatlog
Search
12:09:30
Colleen
Bike: drmeister said 7 hours, 5 minutes ago: I rebased your changes into mine and I'm rebuilding everything on this end. The new backtraces and fastgf changes appeared to work before the rebase and will hopefully continue to work.
12:13:09
drmeister
Clasp built on my system - but I have a tiny patch to LLVM 5.0 installed that allows it to build on OS X.
12:13:55
drmeister
Clasp failed to build on linux (Shinmera's linut system) because his llvm 5.0 doesn't include that patch - and no one elses will either.
12:14:30
drmeister
The patch is just for disassemble, but it fixes a bug in llvm5.0 that at the moment, crashes the build. :-(
12:15:09
drmeister
So I need to put a #if /#endif around that code so that it builds ok with non-patched llvm5.0
12:15:11
frgo
Do we speak of stock LLVM 5.0 as installed by e.g. homebrew or do we talk about external-clasp's LLVM 5.0?
12:16:15
drmeister
I'm guessing both llvm 5.0 that you are describing were shipped with the bug. It's an easy fix on my end - I forgot to add it with all the other stuff I was doing.
12:19:00
drmeister
Check your llvm5.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h file - maybe it already has the patch - or you didn't get the latest version that I pushed about 7 hours ago.
12:19:48
drmeister
Check your hash as well. If you don't have the latest - you can wait until I wrap and #if/#endif around the effected code (a few hours).
12:24:18
drmeister
This patch allows clasp to generate meaningful backtrace information for JITted code at zero runtime cost.
12:25:39
drmeister
Great - the llvm web site says the release date for llvm 5.0.1 is "To be decided".
12:27:13
drmeister
In that case I'll add a conditional flag to wscript.conf that lets you use the patch if it's been applied and get better backtraces. Or disable the code that requires the patch and loose the names of JITted functions from the backtrace.
12:28:21
frgo
Don't bother what I am doing - It was menat as a pure comment. The less work for you the better
12:30:06
drmeister
https://github.com/llvm-mirror/llvm/blob/master/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h#L103
12:31:42
drmeister
I couldn't figure out the error message the compiler was giving me before I knew about the fix. I thought they had done some weird C++ std::function vs lambda switch.
12:32:06
drmeister
I'm getting pretty good at reading C++ template programming compiler error messages - I lost it on this one.
12:39:37
frgo
Ok - but I figure the fix should be a "using" statement rather than doing the namespace annotation as pointed out by the diff, no?
12:40:47
drmeister
I haven't thought about it. Lang Hames - the guy who is developing ORC can deal with it.
13:02:11
Bike
it occured to me that SSA would also fix the redundant assignments problem, because once in SSA form any assignment is redundant (phi instructions not being assignments)
13:06:35
drmeister
Yes - llvm does remove a lot of redundant assignments - that's one reason I haven't been too worried about them.
13:07:22
Bike
having a bunch of temporaries around makes it harder to do a lot of analysis. it's why i couldn't do the un/box elimination
13:07:37
drmeister
But I know that we really have two compilers - and the second compiler (llvm) is picking up some of the slack.
13:09:15
Bike
and i think if we do SSA in HIR, we could skip mem2reg, and avoid redundant work that way. but mainly i want to be able to do analysis
13:12:04
beach
Tell me again the stereotypical case that causes problem in the type inferences; the one that would require value numbering to fix?
13:14:04
Bike
when the typeq is compiled, it makes a temporary, which x is stored in, so the typeq tests the temporary. then when you get to float-add, it uses x, not the temporary, so there's no type information from the typeq
13:16:09
beach
What is eliminated by SSA is assignments to variables that already have a value from previous assignments.
13:16:43
beach
The temporary is created once and not assigned to again. The variable x is not assigned to.
13:17:20
beach
HIR assignment instructions create variables. Including temporaries. Those assignments are still needed.
13:19:38
Bike
SSA itself won't, but without SSA i can't do it because it's possible the assignment is actually needed, because after the assignment X could be reassigned and then the temp used.
13:22:15
beach
SSA itself eliminates assignments to variables that have already been previously created and initialized.
13:25:08
Bike
if we did i could eliminate them pretty easily. so have an "actual assignment" instruction for setqs and a "create and initialize" otherwise?
13:26:32
beach
That is one possibility, yes. But notice that plenty of other instructions create and initialize temporaries.
13:27:52
beach
And I don't see how the task of eliminating "them" would be easier if we reserved the assignment instruction for the non-initial case.
13:29:49
beach
By the way, notice that we have been saying the entire time that SSA has this property, but so does SFA. We are still not using the fact that there is (statically) a single assignment to each variable.
13:30:44
beach
And I maintain that SSA ruins everything, because now you can't eliminate a predecessor to some instruction, even if you know that it can never be reached.
13:32:00
beach
That's what I mean. Before SSA, it doesn't matter how many predecessors an instruction has, and no order between them is defined.
13:32:33
beach
SFA just replaces a φ with an assignment (creation+initialization really) to the fresh variable in each of the predecessor arcs.
13:32:59
beach
... which has the same property as everyone wants, but not the particular problem that SSA has.
13:42:47
beach
I know of only one algorithm that uses the SSA property of SSA (as opposed to the other property that preserves old values as much as possible without changing the control flow), and that algorithm treats the φ instruction as an operation. As a consequence, there is a large class of situations that this algorithm can not handle.
13:43:11
beach
I would tell you more, but it was written by people who write the most incomprehensible papers that I have read.
13:45:39
beach
Those authors could use a big dose of Steven Pinker's book (A Sense of Style) and/or the associated talks by Pinker about the subject.
13:47:40
beach
I just get upset when the authors apparently deliberately obfuscate the message in order to look like they know more than the reader.
14:21:18
Bike
the SSA paper cleavir cites mentions the transformationc an be used to eliminate redundancies. cool, then
14:24:25
Bike
it cites some other stuff on SSA, so i didn't think that was the case. maybe "the original SSA paper that made compiler writers pay attention" judging by the efficiencies
14:29:40
Bike
i'm looking at the SSA stuff in cleavir. i don't know if it works, but i assumed it would be relevant to SFA (the SFA file seems to be blank)
14:32:10
beach
The SSA code is in there, not because I think it is a good idea, but because there might be some Cleavir client that wants it.
14:36:51
beach
OK, I now have a test for LOCALLY [this is CST-to-AST]: (locally (declare (special x)) x) seems to work.
14:37:51
beach
Even when this stuff works, there are a lot of syntax checks missing, resulting in mysterious errors when incorrect code is submitted for conversion.
15:07:06
beach
OK, enough work for today. Time to do something else; something that requires less concentration.
15:15:44
beach
This is very typical for compiler technology. All work is done as if the only valid language is C.
15:17:45
beach
For example, when the nested function that shares a variable in only called, and not captured in any other way, control flow is predictable, so things should be easier then.
15:18:40
beach
Notice though that, as I hinted, I am to tired to do real work at this point, so also too tired to figure out exactly what cases would work.
15:19:24
Bike
SSA. Kildall. Lots of stuff. If you're too tired to think about it that's fine though.
15:20:12
beach
Yes, we have had this discussion before. I will figure it out eventually, but it is hard for me to jump between different tasks too frequently, so I am putting it off.
15:20:37
beach
It all depends on this famous extended control-flow calculation that I have mentioned several times I think.
15:21:17
Bike
You have, yes. But it's proven to be kind of a bottleneck for me so I'd like to help if I can.
15:22:28
beach
OK, let me put it on the back burner then and see whether I can come up with something that can make it easier for you to help.
15:56:31
beach
Let us say that it is extremely rare to have an assignment (other than the defining assignment) to a variable that contains a function, as in (let ((f (lambda ...))) ... (setf f ...))
15:57:38
beach
Then we can get a conservative idea of what happens to the value created by (lambda ....).
15:58:33
beach
It is conservative, in that, if the variable is assigned to, then we might incorrectly assume that the function is propagated in ways that it really isn't.
15:59:29
beach
Here, we may incorrectly conclude that it is possible for the function to become an argument to the call to G.
16:00:27
beach
We get a conservative idea of what happens to the function by looking at instructions that take F as an input.
16:00:59
beach
If the instruction is an assignment, we recursively consider the output of that instruction.
16:02:49
beach
It can be a function that we know will not hold on to the value, such as CONSP, FUNCTIONP, etc. For such cases, we can stop tracking.
16:04:37
beach
This analysis should cover the vast majority of cases, i.e., in most cases, we may have a nested function that we know is only going to be called.
16:08:09
beach
Anyway, just a thought as to how to break the interdependence between several phases.
16:09:42
beach
Because, to be more precise about what can happen to a function, we need value numbering, and value numbering requires knowledge about control flow, so we must break this cycle of dependence somehow.
16:12:34
beach
And of course, whether there really *is* capture is of course undecidable, so we need approximations anyway.
16:23:53
beach
Yes, we can't do a good control-flow analysis when we don't know which functions may be captured.
16:24:04
Bike
anyway, i already wrote a quick version of that when i was trying to do escape analysis, so that's heartening
16:24:07
beach
And we don't know what functions may be captured until we have a control-flow analysis.
16:26:04
beach
I'll continue thinking about it. Just wanted to think out loud. Especially given my state of fatigue.
16:32:28
beach
In general, I think we should think in terms of phases to successively increase our knowledge about the control flow. Then, we can start with simple implementations of each phase, and later, we might think of more precise analyses that we can do.
16:39:49
beach
For example, I can think of cases where this simple analysis gives a false positive in that it can not determine for sure that some function F is only called. But, once the result of this analysis is available and it has been determined that some other function, say G, is only called, this information will allow us to conclude that F is also only called.
16:40:11
beach
Not that I can come up with an example in real-time right now. I would have to play with it on paper.
16:57:47
Bike
a function that's only called could still be captured, right? or is this just to rule out functions being called at any time due to threading etc.
17:13:51
scymtym
i think "compiling with continuations" has something about iterative escape analysis. will have a look when home. that book can be a bit vague in places, though
17:43:11
shiho
drmeister I think I need index data to get atom name etc from energyAtomTable. How can I get all atom's index?
18:32:42
shiho
drmeister Sorry, I misstyped your name... You don't need to care about my 1:43pm question.
19:15:37
drmeister
I'm pretty sure the problem with slime is a race condition when fastgf is used with multiple threads
19:58:06
frgo
drmeister: Have you upgraded to Xcodse 9 already? (I am asking because I see strange compile errors)
20:00:56
drmeister
I did get a weird license approval request this morning and by slamming my hands on the keyboard for a few min while yelling loudly I got it to go away.
20:06:13
drmeister
Because here's the poop ... I mentioned some of this this morning - but I'll summarize.
20:07:27
drmeister
To get zero-cost backtraces I implemented code that gets the llvm JIT engine to tell us where JITted code is put in memory. It worked for about 15 min and then I upgraded to llvm 5.0 and it broke because of a bug in llvm 5.0 :-(
20:08:45
drmeister
Because I haven't tested that. I applied the patch and built llvm in a way that would not undo the patch.
20:09:12
drmeister
Perhaps you didn't build externals clasp again - or you used 'make' which might wipe out the patch.
20:09:39
drmeister
I don't know - I haven't gamed out the whole patch/no-patch/undone patch thing. The whole situation is annoying as F.
20:15:22
drmeister
If you pull the latest externals-clasp and use 'make really-clean' that will wipe out llvm 5.0 on your system, pull it down again, apply the patch and rebuild.
20:16:55
drmeister
Or... I just pushed a change to clasp 'dev' that ignores the broken code - but it doesn't give you backtrace info for JITted code - which isn't a huge deal unless you are developing code in Clasp.
20:17:21
drmeister
It ignores the broken code conditionally - you can tell Clasp that the patch is installed and then it will use it.
20:19:04
drmeister
If it's False then it won't use the broken llvm code. If it is True then it will assume the patch is installed and try to use it.
20:19:23
drmeister
If the patch isn't installed when you set LLVM5_ORC_NOTIFIER_PATCH = True then you will get the error that you just saw.