freenode/#clasp - IRC Chatlog
Search
6:09:23
drmeister
I was doing an experiment. Minecraft does generate the same world no matter what direction you go when you start from the same seed.
6:14:01
drmeister
What I'd like to do is get the set hash table to hash contents using a separate pseudo random number generator.
6:15:25
drmeister
One that is dedicated only to the objects that we put in the set. It should be straightforward because we use the 'badge' slot in every object. We can set the badge for objects that are put into the sets using a custom pseudo-random number generator and the hash table will use that in the hash function.
6:16:07
drmeister
Then we set up bigmac to build all of the quicklisp using the serial compiler repeatedly with different seeds.
6:17:28
drmeister
I can have quicklisp build into different cache directories by setting XDG_CACHE_HOME
6:18:58
drmeister
All I need is to expose a function to write into the badge and we can put whatever we want in there - it will be used for hashing.
6:39:20
karlosz
drmeister: if we can ensure the serial compiler can behave deterministically with a given seed it should work okay
6:41:04
drmeister
Are by any chance, objects only added to each set once through make-set and adjoin? If not we would have to set the badge when the objects are created.
6:44:00
karlosz
what happens is that (setf (gethash ...)) can get called more than once with the same key
6:45:25
drmeister
That's not a problem. We just need to ensure that every objects badge is set only once. It would be best if we did that when objects that go into the sets are created.
6:47:32
drmeister
We can ensure that badges are initialized properly by using a few bits of the badge as a unique tag.
7:21:28
drmeister
So - we can set the badge of objects now. This is used by the hash function so we can make hash-tables deterministic now.
7:22:08
drmeister
If you set the badge to a constant - like 0 - then it turns into a complicated list-like object.
7:30:59
drmeister
::notify Bike I added a core:set-badge and core:get-badge pair of accessors. They set and get the badge of heap allocated objects. If you set the badge of an object right after it is created the badge will be used to hash the object in any hash-tables you add it to. This will let us make hash-tables deterministic.
7:40:23
drmeister
::notify Bike I'm setting up a test where the quicklisp code is compiled in 10 threads 100x each with compile-file-serial to see if I can reproduce the problem under these conditions. If it does - with core:set-badge we should be able to make the sets deterministic and controlled by a seed and then find a seed that causes the build to crash reproducibly.
7:41:37
drmeister
I have ideas how we can make it deterministic even if it's a compile-file-parallel problem.
7:51:21
drmeister
::notify Bike Minecraft does build the same world - no matter what order chunks are created. I tested it again more carefully tonight. Also - I found a cool place to build a base! Bonus!
8:16:57
no-defun-allowed
Fun fact: Minecraft generates the same bedrock pattern regardless of world seed.
13:12:38
Colleen
Bike: drmeister said 5 hours, 41 minutes ago: I added a core:set-badge and core:get-badge pair of accessors. They set and get the badge of heap allocated objects. If you set the badge of an object right after it is created the badge will be used to hash the object in any hash-tables you add it to. This will let us make hash-tables deterministic.
13:12:38
Colleen
Bike: drmeister said 5 hours, 32 minutes ago: I'm setting up a test where the quicklisp code is compiled in 10 threads 100x each with compile-file-serial to see if I can reproduce the problem under these conditions. If it does - with core:set-badge we should be able to make the sets deterministic and controlled by a seed and then find a seed that causes the build to crash reproducibly.
13:12:38
Colleen
Bike: drmeister said 5 hours, 21 minutes ago: Minecraft does build the same world - no matter what order chunks are created. I tested it again more carefully tonight. Also - I found a cool place to build a base! Bonus!
15:16:14
drmeister
I built the quicklisp code one thousand times with the serial compiler - no problems.
15:29:01
drmeister
Are there any new potential race conditions that are opened up by the BIR compiler?
15:38:14
kpoeck
Regarding this indeterministic problem that we have. I have a script that compiles like 40 main quicklisp systems, Is it of any help if I run this with latest everything? What should I do if I get an error like drmeister got for ;iterate? Can run the sequential or parallel compiler, just would ned to know what helps most
15:41:00
drmeister
I'm building all of the cando quicklisp systems 100x in parallel threads that each use a different value for XDG_CACHE_HOME - the asdf cache.
17:41:59
drmeister
I'm running the multithreaded quicklisp code compilation after making compile-file-serial the default and it's not failing yet.