freenode/#clasp - IRC Chatlog
Search
15:16:14
drmeister
I built the quicklisp code one thousand times with the serial compiler - no problems.
15:29:01
drmeister
Are there any new potential race conditions that are opened up by the BIR compiler?
15:38:14
kpoeck
Regarding this indeterministic problem that we have. I have a script that compiles like 40 main quicklisp systems, Is it of any help if I run this with latest everything? What should I do if I get an error like drmeister got for ;iterate? Can run the sequential or parallel compiler, just would ned to know what helps most
15:41:00
drmeister
I'm building all of the cando quicklisp systems 100x in parallel threads that each use a different value for XDG_CACHE_HOME - the asdf cache.
17:41:59
drmeister
I'm running the multithreaded quicklisp code compilation after making compile-file-serial the default and it's not failing yet.
18:30:41
karlosz
Bike: could you merge the clasp PR? i realize unwinding regressed because i accidentally overrode the dynenv condition
18:35:35
karlosz
on the topic of constant folding, i realize that primops will throw a wrench into the scheme
18:36:03
karlosz
by the time we get to BIR we have primops to worry about too, so we can't just fold functions
19:03:09
Bike
while ast-to-hir-ing a bind-ast it ends up passing a function as the ast argument of compile-ast. something to do with the cst-to-ast thunk changes...? I don't know
19:09:07
karlosz
i was just thinking that sometimes by the time you merge my PR the hash we want may have changed
19:30:56
drmeister
karlosz: The current build problem looks like a race condition in compile-file-parallel.
19:37:58
drmeister
Also - we have the undo debugger now - https://undo.io/resources/debugging-race-conditions-cpp/
19:46:28
drmeister
The problem I saw with ThreadSanitizer is that it looks like everything has to be built with it. Every library.
19:56:27
karlosz
i wonder if we're modifying any global state in the bir compiler that we didn't with the hir compiler
19:57:42
karlosz
i think a race condition signifies that our problem isn't a problem with bir per se at least
20:05:13
drmeister
Yes - I'm quite convinced that it's a race condition in the parallel compiler - beyond that I have no idea.
20:06:03
drmeister
I'm building a cando version that uses compile-file-parallel on linux and I'll run it under the undo debugger until I get a crash. Then I can try to backup into it.
20:08:18
Bike
anyone with a running clasp master, can you try (compile nil '(lambda (x) (declare (ignore x)))) in the cl-user package, and then in the cleavir-bir package, and tell me what happens?
20:42:34
karlosz
i just didn't know what to call the accessor besides IGNORE, but that was probably a bad idea in hindset
21:09:56
karlosz
(if <constant-value> ... ...) => ... optimization applies over 20,000 times in self build
21:19:57
karlosz
so what i have in meta evaluation is getting rid of unreachable branches and also flushing dead references
21:20:43
karlosz
constant folding of known functions will be difficult because we have primops in bir
21:36:05
karlosz
ultimately the idea is when we annotate types on all computations i'd like to push the types in the forward meta evaluation pass
21:36:41
karlosz
deleting unreachable branches should also trigger better type prop when merges are gone as well
21:37:04
karlosz
probably the next thing to do is get if-if working by changing the representation of branches in bir
23:27:03
drmeister
I think the crashes I've been experiencing when I run multiple quicklisp compilations may be due some sort of collision in the cache.
23:27:54
drmeister
To a separate directory and stuff is being written in there. But when I clear that cache and run the build again the quicklisp code doesn't build after the first time.
23:28:26
drmeister
More important - I can't get cando running the parallel compiler to crash when I run it again and again and again.
1:51:43
drmeister
So the current state of things is 1. With compile-file-parallel the buildbot fails about 1/3 to 1/2 of the time.
1:52:20
drmeister
2. When I try to set up 10 parallel builds on hermes by changing XDG_CACHE_HOME I get build failures
1:53:35
drmeister
However there is something weird. The XDG_CACHE_HOME directories I set are populated but I flushed them between builds and after the first build the quicklisp code is just loaded from somewhere - I'm doing something wrong but I don't know what. My test is screwing up and I don't trust it.
1:54:08
drmeister
4. When I build on hermes one at a time with compile-file-parallel - I am unable to get the quicklisp compilation to fail. I've done it about 8 times now.
1:59:24
drmeister
Thank goodness that with bir and compile-file-parallel - a full build of the cando quicklisp code takes 2 min.
2:00:18
drmeister
And as I type that I get a crash while repeatedly building the quicklisp - finally.
2:28:29
karlosz
Bike: do you have any advice on the primops issue? i just defined a flushable attribute and i want to put it on things like car and cdr and the arithmetic operators
2:29:00
karlosz
i have it as part of the dead code eliminator in metaevaluation, but it seems like i don't actually get an abstract call?
2:30:11
karlosz
i mean the issue seems to be that clasp doesn't even make an abstract call for car or cdr
2:31:12
Bike
not sure what you mean about not getting an abstract call. i would think that it makes a call to the car function, which is then inlined.
2:32:43
Bike
well what i mean is we could mark primop uses with attributes the same way we mark calls.
2:41:13
Bike
i can do that. longer term i'd like to be able to use the flow of the callee like i've been saying but we work with what we have
2:42:41
karlosz
not sure how we can do the flow of the callee with the constant foldable ness though