libera/#commonlisp - IRC Chatlog
Search
11:29:38
mfiano
jackdaniel: I'm writing a snapshot management system for ZFS filesystems, with retention policies and replication to other hosts.
11:30:28
mfiano
I'm trying to design a retention policy. I could use some feedback on an algorithm (in prose/ascii-art) to check for any holes.
11:33:18
Shinmera
jackdaniel: Workin' on Kandria. Currently doing level design, but earlier I was implementing a dust effect in the hopes that it'll make some environments a bit more interesting to look at.
11:35:01
Shinmera
I got a rough cut of the new trailer just yesterday. It'll be out when the Steam Next Fest launches on the 13th.
11:41:15
flip214
I look at the existing snapshots and remove the ones that are required for the specifications -- so the outcome is a list of snapshots to remove
11:41:19
mfiano
I've used btrfs for a couple years, but I wouldn't trade it for the power of ZFS these days.
11:43:12
flip214
though "newer snapshots are pruned first" means that you'll delete your newest ones as long as they're in the same bucket as the next ones
11:43:43
flip214
ie. do snapshots every 5 min, but keep only one for the last hour => you discard the 11 newest ones before retaining one
11:46:01
mfiano
The thing is both local host and each replication host can have different retention policies
11:47:40
flip214
I've got two independent retentions and independent snapshots for one data set (with one off-site copy); synchronizing them was too much effort and too fragile.
11:48:28
flip214
just have data and snapshots at the source; copy the current stuff via cron to remote; and remote, triggered automatically by the rsync, does independent snapshots after a successful sync
11:50:58
mfiano
ah zfs has `zfs send`/`zfs recv` with incremental delta support. so you can do zfs send latest-snapshot | zfs recv foo (or piped through ssh for remote)
11:51:28
flip214
independency was a good idea when I broke a retention script and killed all snapshots on one site ;/
11:52:23
flip214
yeah, btrfs can do that as well. Still, using rsync protects me against a possible filesystem corruption (which might get sent on via these internal mechanisms)
11:53:37
mfiano
I quadratically backup every dataset locally and to every other host, and some are running mirrored arrays so i'm not worried
11:54:35
flip214
well, how does mirroring help against filesystem corruption? would just be synced on as well?
11:56:40
mfiano
zfs has filesystem corruption protection built in. it's the only filesystem that can survive and repair a power loss during a buffered write. plus a filesystem is a pool, and if you're snapshotting/replicating a pool like i am you have snapshots everywhere to restore it.
12:03:12
flip214
I still wouldn't rule out software bug, gamma rays, etc.... using different tools makes my backup more resilient, I believe
12:12:30
flip214
mfiano: well, salt mines have much higher background radiation, so they should be avoided too
13:47:40
Guest74
drmeister: while the error is annoying, it seems to make logical sense, given the CL abstraction for pathnames, as dot files are singular.
13:49:31
drmeister
It sounds like there is a thought process behind what you said and I'd like to understand that.
13:49:48
Guest74
there can be many txt files but there is only one .sbclrc. So it make sense that .sbclrc is a name and not a type.
13:53:25
Guest74
actually, I really wish all implementations stored device information in pathnames.
13:57:11
drmeister
Guest74: Your point is that with pathnames `:type` is supposed to indicate a type of pathname and that doesn't quite fit with dot files being singular.
13:59:55
drmeister
In ecl `(namestring (make-pathname :type "clasprc"))` returns NIL - clasp copied that behavior until yesterday. I changed clasp yesterday to return ".clasprc" because I interpret the pathname-type as more of a pathname-"extension". I see your point however.
14:00:45
drmeister
It appears that I am not violating the Common Lisp specification - this is implementation dependent behavior.
14:02:21
drmeister
This came up because we tried to define the RC file using the logical pathname #P"home:.rc" and it didn't work.
14:04:42
Guest74
This is definitely one of those wscl things. It's hard to work with an abstraction that nobody can agree upon.
14:05:44
semz
Agree. I like the idea behind pathnames a lot but they end up rather awkward in practice because of this.
14:40:58
mfiano
Ugh, yes. Pathnames are awkward to work with at times. I usually use UIOP's abstractions just to have some common ground to work with. Too much implementation-dependent behavior and other gotchas that are better, though far from ideal, using UIOP.
15:36:51
minion
danisanti: please look at PCL: pcl-book: "Practical Common Lisp", an introduction to Common Lisp by Peter Seibel, available at http://www.gigamonkeys.com/book/ and in dead-tree form from Apress (as of 11 April 2005).
15:51:19
beach
danisanti: You can ask questions here too, but if they become too basic and too numerous, you will be directed to #clschool. But some basic questions are tolerated here.