libera/#clasp - IRC Chatlog
Search
1:57:14
Colleen
Bike: drmeister said 8 hours, 50 minutes ago: - I just realized my workers can update everything they need to using mp:atomic-incf.
1:58:34
Bike
"Does (mp:atomic-incf (gethash x y 0)) make sense?" so part of the trick here is what "atomic" means. doing (with-lock (lock) (incf (gethash ...))) is atomic in the sense that if other code is using locks correctly, it will never see a partial state
1:59:06
Bike
in C++, as i understand (and it's very possible i don't, because this is an especially messy part of the standard), std::atomic doesn't actually have to be lock-free
1:59:24
Bike
and in actual implementations they fall back to locks when the machine can't actually do things atomically
2:03:09
Bike
"atomic", not lock free incf of gethash could be done by providing an additional internal operation for synchronized (locked) hash tables to do read-modify-write, if we wanted, i think
13:45:37
drmeister
That's the estimated time to complete processing 48GB of sequencing data on hermes (my linux box) and on bigmac (a 36 core iMacPro).
13:46:39
drmeister
I think this reflects "false sharing" that occurs when lots of threads access the queue lock or the lock I placed on the hash table that accumulates results.
13:48:32
drmeister
What's frustrating about this is profiling and flame charts don't show this problem.