freenode/#shirakumo - IRC Chatlog
Search
9:44:11
Shinmera
And it seems like it would be worthwhile to turn it into a more generalised file system API
13:28:54
Shinmera
I can't come up with anything other than just "file-system" or "file-systems", but that's not a great name for obvious reasons
13:37:09
SAL9000
Shinmera: when you say "file system API", are you thinking of something like UIOP, or...?
13:38:10
Shinmera
I'm thinking of a general protocol that describes systems that keep a set of "files", objects with several attributes and a central payload.
13:39:14
Shinmera
The primary idea is to provide a protocol that allows you to address stuff stored on the OS FS or within a "file FS" the same way.
13:42:07
Shinmera
I have a progenitor of this in Kandria to allow compressing parts of it when deploying, but not needing separate addressing.
13:42:36
Shinmera
But, while I'm ratifying it, might as well turn it into something more worthwhile.
13:44:19
Shinmera
For instance one thing I'm aiming for is to allow stuff like non-hierarchical file-systems, where hierarchical file-systems are actually described as a set of nested file-systems -- each directory is both a "file" and a "file system".
13:47:11
Shinmera
another important part is that it can be extended to in-memory stuff that's just exposed through the same protocol so you could also load stuff in and then address it the same way, still.
13:48:00
Shinmera
anyway, it just boils down to a generic protocol in the end, one implementation of which is the os file system.
15:11:16
SAL9000
Shinmera: no clue... tbh I've long given up on that kind of stuff in favour of a fileserver box running ZFS + CIFS
17:45:37
Shinmera
SAL9000: I have traumatic memories of SMB and CIFS and all those network file systems
17:46:14
Shinmera
Anyway, converting ext4 to btrfs was painless after installing a new version of btrfs-tools
17:46:21
SAL9000
Yeah, I'm aware. In my experience they suck *less* than Win native drivers for things
17:48:30
Shinmera
And I'd like to have as few partitions as possible so I don't have to arbitrarily split my shit or have dupes
17:49:22
Shinmera
not because of the driver, that's fine these days, but the user/permissions mapping and speed.
17:49:55
Shinmera
I'm hoping for WinBTRFS now. Installation was painless and it seems pretty robust except for advanced features I'm not intending on using anyway (compression/encryption/snapshotting)
17:51:32
SAL9000
let me know how that goes? it seems a cool idea but a bit too alpha for my taste despite the age of the project
17:52:38
SAL9000
point being, remember to take into account the technical competence (or lack thereof) of your target audience
17:53:10
Shinmera
My mom is mostly just living on her android phone. If I can set it up to sync a folder automatically it'll be just fine.
17:53:40
SAL9000
the impression I have is that it's designed to piggyback onto existing cloudy things and back them up in One Place
17:54:17
SAL9000
if you want turnkey-ish sync, consider Syncthing -- admittedly, I haven't had much success with running the phone-app in background sync mode.
17:54:45
SAL9000
on phone I've fallen back to explicitly opening it when I want to sync, and closing it afterward
17:57:32
SAL9000
I'd welcome a sync solution built with your usual level of simplicity & thoughtfulness, but that involves diving into the Void of phone apps and I'd like to keep you sane :-)
17:57:56
Shinmera
too late Seqread: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
17:57:58
Shinmera
Seqwrite: (g=1): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
17:58:00
Shinmera
512Kread: (g=2): rw=randread, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=libaio, iodepth=1
17:58:02
Shinmera
512Kwrite: (g=3): rw=randwrite, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=libaio, iodepth=1
17:58:04
Shinmera
4kQD32read: (g=4): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
17:58:06
Shinmera
4kQD32write: (g=5): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
17:58:38
Shinmera
bw ( KiB/s): min=329045, max=365909, per=100.00%, avg=334263.00, stdev=10622.51, samples=21
17:59:18
Shinmera
bw ( KiB/s): min=357717, max=407552, per=99.90%, avg=370774.75, stdev=12170.05, samples=20
17:59:58
Shinmera
bw ( KiB/s): min=271018, max=325632, per=100.00%, avg=298973.21, stdev=11833.20, samples=24
18:00:40
Shinmera
bw ( KiB/s): min=329728, max=358400, per=100.00%, avg=341140.43, stdev=6902.89, samples=21
18:01:22
Shinmera
bw ( KiB/s): min=391496, max=396058, per=100.00%, avg=394567.67, stdev=1308.75, samples=18
18:02:04
Shinmera
bw ( KiB/s): min=219904, max=309520, per=100.00%, avg=280498.67, stdev=13917.66, samples=36
18:02:28
Shinmera
READ: bw=326MiB/s (342MB/s), 326MiB/s-326MiB/s (342MB/s-342MB/s), io=5000MiB (5243MB), run=15317-15317msec
18:02:32
Shinmera
WRITE: bw=362MiB/s (380MB/s), 362MiB/s-362MiB/s (380MB/s-380MB/s), io=5000MiB (5243MB), run=13795-13795msec
18:02:36
Shinmera
READ: bw=292MiB/s (306MB/s), 292MiB/s-292MiB/s (306MB/s-306MB/s), io=5000MiB (5243MB), run=17139-17139msec
18:02:40
Shinmera
WRITE: bw=333MiB/s (349MB/s), 333MiB/s-333MiB/s (349MB/s-349MB/s), io=5000MiB (5243MB), run=15021-15021msec
18:02:44
Shinmera
READ: bw=385MiB/s (404MB/s), 385MiB/s-385MiB/s (404MB/s-404MB/s), io=5000MiB (5243MB), run=12989-12989msec
18:02:48
Shinmera
WRITE: bw=273MiB/s (286MB/s), 273MiB/s-273MiB/s (286MB/s-286MB/s), io=5000MiB (5243MB), run=18301-18301msec
18:02:50
Shinmera
linus@linuzz ~ fio --loops=5 --size=1000m --filename=/run/media/fastchive/data.tmp --stonewall --ioengine=libaio --direct=1 \
18:03:04
Shinmera
Seqread: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
18:03:06
Shinmera
Seqwrite: (g=1): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
18:03:08
Shinmera
512Kread: (g=2): rw=randread, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=libaio, iodepth=1
18:03:10
Shinmera
512Kwrite: (g=3): rw=randwrite, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=libaio, iodepth=1
18:03:12
Shinmera
4kQD32read: (g=4): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
18:03:14
Shinmera
4kQD32write: (g=5): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
18:03:46
Shinmera
bw ( KiB/s): min=629166, max=925696, per=100.00%, avg=711380.22, stdev=106519.95, samples=9
18:04:28
Shinmera
bw ( KiB/s): min=741376, max=1028101, per=100.00%, avg=853379.62, stdev=104903.38, samples=8
18:05:10
Shinmera
bw ( KiB/s): min=451711, max=788165, per=100.00%, avg=562305.08, stdev=132057.22, samples=12
18:05:19
gingerale
SAL9000: Shinmera says he's tried to leave and return but it's throttled by the server.
18:05:52
Shinmera
bw ( KiB/s): min=651264, max=937869, per=99.46%, avg=722899.10, stdev=84908.16, samples=10
18:08:12
SAL9000
okay so bracketed paste is an extra mode, which can be enabled by the application (i.e. weechat)
18:08:13
|3b|
ACTION thinks the "irc will kick you if you paste a bunch, so lets paste it really slowly" feature is somewhat a bad idea :p
18:13:51
Shinmera
Anyway, back to android: I've already been tainted by the pain of android development https://github.com/shirakumo/ocelot
18:13:51
Colleen
github.com/shirakumo/ocelot Website (HTML), Title: GitHub - Shirakumo/Ocelot: An Android client for Lichat
18:15:13
SAL9000
I recommend trying the following: start a new ssh session (w/o tmux), enable bracketed paste (see above), cat --show-all, then try to paste and see what your terminal is sending.
18:15:28
SAL9000
that will help you confirm whether (your version of) xfce4-terminal supports bracketed paste
18:16:53
SAL9000
Shinmera: there's also the ersatz solution -- install the 'multiline.pl' plugin for weechat, then your paste should in theory become a mulitline input which you can cancel.
18:19:33
SAL9000
as for Syncthing and mobile sync/background stuff in general... basically, you're stuck between a rock and a hard place, from what I can tell
18:19:57
SAL9000
on the one side, you have the battery management police trying it's best to find an excuse to kill your background process
18:20:36
SAL9000
from what I can tell anything that doesn't use Google's push-notification API (or alarms/timers, I guess?) is basically screwed over by those two aspects :(
18:28:24
SAL9000
Shinmera: you can either use Ctrl-U line by line, or unset plugins.var.perl.multiline.modify_keys so then weechat treats the whole multiline as one line, and Ctrl-U will delete the whole lot...
18:29:04
SAL9000
alternatively, you can use Ctrl-Down to return to an empty prompt immediately (weechat command history trick)
18:29:26
SAL9000
there's some settings for paste detection, but I can't test those myself because I'm running bracketed
19:10:11
SAL9000
Shinmera: something with "cabinet", maybe? although that might evoke windows' .cab file horrors...
19:10:50
Shinmera
but both packets and cabinets have the issue of implying it's about stuff within a logical file.
19:19:33
SAL9000
Perforce is (apparently) widely used in the games industry due to being relatively good at handling large binary blobs
19:20:37
SAL9000
it's very much a BigCorp thing, not something indie devs would want to use, if only because non-free clients
19:20:59
Shinmera
right, well, I'm not concerned about the name clash from a lisp library that implements a different concept.
19:24:21
SAL9000
so long as we don't end up, later, with a trivial-version-control compat library trying to abstract over legit/svn/perforce /s
19:30:39
Shinmera
Well, here's an initial draft. https://github.com/Shinmera/depot/blob/master/protocol.lisp
19:30:39
Colleen
github.com/Shinmera/depot/b... Website (HTML), Title: depot/protocol.lisp at master · Shinmera/depot · GitHub
19:43:05
Shinmera
Not sure how best to facilitate switching protocols when encountering an entry that could be handled more specifically
19:43:32
Shinmera
Meaning if you access an entry that's a zip file, it should be transformed into something that's also a zip-file-depot.
19:47:28
SAL9000
Shinmera: not sure where you're going with the transaction stuff... not all fs have transactional semantics?
19:48:37
SAL9000
as for zip files -- does depot itself know that entry is a zip-file? seems to me like that is a job for a magic(5) library
19:49:58
Shinmera
they don't, but it can be emulated and or it'll just write anyway and "succeed" every commit.
19:50:43
SAL9000
might be cleaner to emulate transactions as the depot library deferring everything until commit occurs?
19:50:59
SAL9000
so long as it's clear to the API user that it doesn't guarantee atomicity, of course
19:51:15
Shinmera
sure, my point is just that it's more general to do it like this than to present a raw streams interface.
19:51:48
SAL9000
point being that you're setting yourself up for feature-flags or real-transactions-p or SOMETHING
19:51:48
Shinmera
as for your other question, detecting whether an entry is denoting a certain type is a separate problem
19:53:49
SAL9000
maybe depot can have backends or hooks -- e.g. magic v.s. file extension -- which allows CHANGE-CLASSing the depot:entry into any of its subclasses?
19:54:48
Shinmera
that's what I'm asking. I can see three avenues: 1) a function that takes a "type" and returns a class to use. 2) a function that encapsulates the entry in another instance 3) a set of functions that can decide to "accept" the entry and change-class it
19:55:41
SAL9000
#1 seems quite limited. Does it matter for depot whether the caller does change-class or encapsulation or mixins...?
19:55:44
Shinmera
My issue with 1 and 3 is that you might need to multiplex the type depending on the "origin depot"
19:56:33
Shinmera
My issue with 2 is that you need a separate call to convert it to the depot that is denoted by the entry.
19:57:08
SAL9000
caller could theoretically do encapsulation + delegation so you don't need a separate call
19:58:09
SAL9000
(i.e. the wrapper object quacks like a depot:entry for all intents and purposes, without necessarily satisfying DEPOT:ENTRY-P)
19:58:18
Shinmera
Wait, no, 2) is bad because you could not transparently traverse through depots, since now you need to request depot conversion.
19:59:07
Shinmera
One idea of the system is that you can go like (entry* os-depot "home" "linus" "some.zip" "my-file.dat")
19:59:31
Shinmera
or, more transparently, "some" instead of "some.zip" where "some" could be a zip or a directory, or whatever.
20:02:38
Shinmera
though it would be more like (entry* http-depot '("org" "wikipedia" "en") "wiki" "common lisp')