libera/#sicl - IRC Chatlog
Search
22:51:58
Bike
https://github.com/s-expressionists/Cleavir/How-Cleavir-optimizes started writing up what I'm doing in Cleavir
3:00:05
beach
moon-child: The CLOSOS specification has a chapter on crash-proof-ness. It has incremental snapshots, so those are transactions in some ways.
3:02:08
hayley
One interesting thing I had "discovered" is that if we snapshot the state of threads, and there are no intervening I/O operations, the system can be made to complete a "transaction", that it was in the middle of performing when the system crashed.
3:04:26
hayley
While the result of the transaction is not written to disk, it is somewhat "durable" in the sense that we will eventually observe the results. But subsequent crashes can cause the transaction to be undone temporarily, so it isn't really durable.
3:04:42
moon-child
yes, and you can't be transactional wrt i/o anyway, but like I said, I am not sure that taking very frequent, coherent snapshots is scalable
3:06:38
beach
My best attempt is the one based on log-structured file systems, and I see no reason why it would not be scalable.
3:07:10
beach
The one based on EROS may not be. At least, that's what Shapiro told me when I asked him about it.
3:09:19
hayley
I think we'd need to stop the world to snapshot the state of threads, but I had proposed snapshotting threads rather late into the existence of CLOSOS. Stopping the world might become an issue with many cores; though it was good enough for the GC Azul did on their hardware with hundreds of cores.
3:13:04
moon-child
looks like it marks everything as read-only (presumably using the mmu) and then starts spurting
3:31:05
moon-child
full checkpoints may be rare, but there should be some way for certain applications--such as text editors or image editors--to persist state in close to real time
3:36:37
beach
Why do you keep using language such as "full checkpoints", which suggests that there is some checkpoint that works on the entire system at once?
3:38:12
hayley
I would guess "full" as in the checkpoint represents changes to the entire system, rather than changes in a particular application, or some other portion of the system.
3:39:05
moon-child
call it whatever you like; I am suspicious that this operation is performant enough to be sensibly performed at, say, five minute intervals
3:39:58
moon-child
for that matter, I might have some background process doing intensive data processing, and I don't care whether that makes it into the checkpoint. Adding controls for this then seems valuable
3:40:27
hayley
I might have mentioned it before, but once the dirty pages are copied in memory, the pages can be written to disk concurrently with the system running.
3:43:00
hayley
(I say this, having gotten bitten by C++ copying things because I forgot an & in a function definition, making a parallel program take twice as long. Hopefully there are fewer dirty pages.)
3:45:02
moon-child
https://yourdatafitsinram.net/ a fun game is finding somebody to pay for one of these