freenode/#clasp - IRC Chatlog
Search
19:30:56
drmeister
karlosz: The current build problem looks like a race condition in compile-file-parallel.
19:37:58
drmeister
Also - we have the undo debugger now - https://undo.io/resources/debugging-race-conditions-cpp/
19:46:28
drmeister
The problem I saw with ThreadSanitizer is that it looks like everything has to be built with it. Every library.
19:56:27
karlosz
i wonder if we're modifying any global state in the bir compiler that we didn't with the hir compiler
19:57:42
karlosz
i think a race condition signifies that our problem isn't a problem with bir per se at least
20:05:13
drmeister
Yes - I'm quite convinced that it's a race condition in the parallel compiler - beyond that I have no idea.
20:06:03
drmeister
I'm building a cando version that uses compile-file-parallel on linux and I'll run it under the undo debugger until I get a crash. Then I can try to backup into it.
20:08:18
Bike
anyone with a running clasp master, can you try (compile nil '(lambda (x) (declare (ignore x)))) in the cl-user package, and then in the cleavir-bir package, and tell me what happens?
20:42:34
karlosz
i just didn't know what to call the accessor besides IGNORE, but that was probably a bad idea in hindset
21:09:56
karlosz
(if <constant-value> ... ...) => ... optimization applies over 20,000 times in self build
21:19:57
karlosz
so what i have in meta evaluation is getting rid of unreachable branches and also flushing dead references
21:20:43
karlosz
constant folding of known functions will be difficult because we have primops in bir
21:36:05
karlosz
ultimately the idea is when we annotate types on all computations i'd like to push the types in the forward meta evaluation pass
21:36:41
karlosz
deleting unreachable branches should also trigger better type prop when merges are gone as well
21:37:04
karlosz
probably the next thing to do is get if-if working by changing the representation of branches in bir
23:27:03
drmeister
I think the crashes I've been experiencing when I run multiple quicklisp compilations may be due some sort of collision in the cache.
23:27:54
drmeister
To a separate directory and stuff is being written in there. But when I clear that cache and run the build again the quicklisp code doesn't build after the first time.
23:28:26
drmeister
More important - I can't get cando running the parallel compiler to crash when I run it again and again and again.
1:51:43
drmeister
So the current state of things is 1. With compile-file-parallel the buildbot fails about 1/3 to 1/2 of the time.
1:52:20
drmeister
2. When I try to set up 10 parallel builds on hermes by changing XDG_CACHE_HOME I get build failures
1:53:35
drmeister
However there is something weird. The XDG_CACHE_HOME directories I set are populated but I flushed them between builds and after the first build the quicklisp code is just loaded from somewhere - I'm doing something wrong but I don't know what. My test is screwing up and I don't trust it.
1:54:08
drmeister
4. When I build on hermes one at a time with compile-file-parallel - I am unable to get the quicklisp compilation to fail. I've done it about 8 times now.
1:59:24
drmeister
Thank goodness that with bir and compile-file-parallel - a full build of the cando quicklisp code takes 2 min.
2:00:18
drmeister
And as I type that I get a crash while repeatedly building the quicklisp - finally.
2:28:29
karlosz
Bike: do you have any advice on the primops issue? i just defined a flushable attribute and i want to put it on things like car and cdr and the arithmetic operators
2:29:00
karlosz
i have it as part of the dead code eliminator in metaevaluation, but it seems like i don't actually get an abstract call?
2:30:11
karlosz
i mean the issue seems to be that clasp doesn't even make an abstract call for car or cdr
2:31:12
Bike
not sure what you mean about not getting an abstract call. i would think that it makes a call to the car function, which is then inlined.
2:32:43
Bike
well what i mean is we could mark primop uses with attributes the same way we mark calls.
2:41:13
Bike
i can do that. longer term i'd like to be able to use the flow of the callee like i've been saying but we work with what we have
2:42:41
karlosz
not sure how we can do the flow of the callee with the constant foldable ness though
2:45:56
Bike
reocgnizing mv-bind lambdas would be tricky, i don't think we have information about the defaults available in bir, and we need it
2:55:57
karlosz
OK, so in order to have (lambda () (- 1 (+ 1 2)) t) flush the arithemtic expression i needed:
2:59:18
karlosz
i'm going to see how many of these arithmetic calls actually get flushed during self build now
2:59:52
karlosz
these are things llvm can't flush for us. i think llvm could do readvars and constants
3:07:14
Bike
ok, well going back to inlining for a second i've been thinking we could make inlining just another kind of transformation
3:09:33
karlosz
also there are three primop calls to SYMBOL VALUE that get flushed during self build, for whatever reason
3:11:41
karlosz
nothing to do with (lambda (x) (car x) t)... i was just running the code flusher over self build