freenode/#clasp - IRC Chatlog
Search
13:12:38
Colleen
Bike: drmeister said 5 hours, 41 minutes ago: I added a core:set-badge and core:get-badge pair of accessors. They set and get the badge of heap allocated objects. If you set the badge of an object right after it is created the badge will be used to hash the object in any hash-tables you add it to. This will let us make hash-tables deterministic.
13:12:38
Colleen
Bike: drmeister said 5 hours, 32 minutes ago: I'm setting up a test where the quicklisp code is compiled in 10 threads 100x each with compile-file-serial to see if I can reproduce the problem under these conditions. If it does - with core:set-badge we should be able to make the sets deterministic and controlled by a seed and then find a seed that causes the build to crash reproducibly.
13:12:38
Colleen
Bike: drmeister said 5 hours, 21 minutes ago: Minecraft does build the same world - no matter what order chunks are created. I tested it again more carefully tonight. Also - I found a cool place to build a base! Bonus!
15:16:14
drmeister
I built the quicklisp code one thousand times with the serial compiler - no problems.
15:29:01
drmeister
Are there any new potential race conditions that are opened up by the BIR compiler?
15:38:14
kpoeck
Regarding this indeterministic problem that we have. I have a script that compiles like 40 main quicklisp systems, Is it of any help if I run this with latest everything? What should I do if I get an error like drmeister got for ;iterate? Can run the sequential or parallel compiler, just would ned to know what helps most
15:41:00
drmeister
I'm building all of the cando quicklisp systems 100x in parallel threads that each use a different value for XDG_CACHE_HOME - the asdf cache.
17:41:59
drmeister
I'm running the multithreaded quicklisp code compilation after making compile-file-serial the default and it's not failing yet.
18:30:41
karlosz
Bike: could you merge the clasp PR? i realize unwinding regressed because i accidentally overrode the dynenv condition
18:35:35
karlosz
on the topic of constant folding, i realize that primops will throw a wrench into the scheme
18:36:03
karlosz
by the time we get to BIR we have primops to worry about too, so we can't just fold functions
19:03:09
Bike
while ast-to-hir-ing a bind-ast it ends up passing a function as the ast argument of compile-ast. something to do with the cst-to-ast thunk changes...? I don't know
19:09:07
karlosz
i was just thinking that sometimes by the time you merge my PR the hash we want may have changed
19:30:56
drmeister
karlosz: The current build problem looks like a race condition in compile-file-parallel.
19:37:58
drmeister
Also - we have the undo debugger now - https://undo.io/resources/debugging-race-conditions-cpp/
19:46:28
drmeister
The problem I saw with ThreadSanitizer is that it looks like everything has to be built with it. Every library.
19:56:27
karlosz
i wonder if we're modifying any global state in the bir compiler that we didn't with the hir compiler
19:57:42
karlosz
i think a race condition signifies that our problem isn't a problem with bir per se at least
20:05:13
drmeister
Yes - I'm quite convinced that it's a race condition in the parallel compiler - beyond that I have no idea.
20:06:03
drmeister
I'm building a cando version that uses compile-file-parallel on linux and I'll run it under the undo debugger until I get a crash. Then I can try to backup into it.
20:08:18
Bike
anyone with a running clasp master, can you try (compile nil '(lambda (x) (declare (ignore x)))) in the cl-user package, and then in the cleavir-bir package, and tell me what happens?
20:42:34
karlosz
i just didn't know what to call the accessor besides IGNORE, but that was probably a bad idea in hindset
21:09:56
karlosz
(if <constant-value> ... ...) => ... optimization applies over 20,000 times in self build
21:19:57
karlosz
so what i have in meta evaluation is getting rid of unreachable branches and also flushing dead references
21:20:43
karlosz
constant folding of known functions will be difficult because we have primops in bir
21:36:05
karlosz
ultimately the idea is when we annotate types on all computations i'd like to push the types in the forward meta evaluation pass
21:36:41
karlosz
deleting unreachable branches should also trigger better type prop when merges are gone as well
21:37:04
karlosz
probably the next thing to do is get if-if working by changing the representation of branches in bir
23:27:03
drmeister
I think the crashes I've been experiencing when I run multiple quicklisp compilations may be due some sort of collision in the cache.