libera/#shirakumo - IRC Chatlog
Search
9:26:03
Colleen
<SAL9000-> shinmera: I pushed a basic version of backfill deduplication last night; it doesn't serialize to disk so only applies if the buffer is still open (i.e. across part/join or reconnect)
9:27:40
Colleen
<SAL9000-> not sure how we want to handle serialization tbh... a folder under .weechat containing pickle files (or json), one per channel?
9:29:19
Colleen
<SAL9000-> in a fresh buffer, then, we either read in weechat's normal log file as the irc plugin does, or do backfill as usual but flag the messages as no_log and no_highlight
9:29:34
Colleen
<shinmera> Is there a way to emit into the folders where weechat already dumps its own logs?
9:30:26
Colleen
<SAL9000-> I'm kinda wary of mixing human-readable logs with "internal" state though
9:35:38
Colleen
<shinmera> something like injecting a space at the beginning of a continuing line would make it still easy to parse and easy to read.
9:36:55
Colleen
<shinmera> We're going to need to implement logic to avoid reading in the entire log anyway
9:37:29
Colleen
<SAL9000-> I was thinking that we rely on the standard weechat log for the years of data and never read *that* back in
9:37:47
Colleen
<SAL9000-> this would be a short slice of the most recent updates -- current settings are n < 256 and t < 30s
9:38:42
Colleen
<SAL9000-> backfill updates land in one of 3 categories -- obviously too old (older than oldest recent update), in the uncertainty window (between oldest and newest recent update) and new (newer than newest recent update)
9:39:22
Colleen
<SAL9000-> too old get ignored, too new gets emitted, uncertainty window means they get compared against all recent updates and ignored if there's a match
9:40:14
Colleen
<shinmera> but there's no real reason to not make it be anything other than an integer subtype
9:40:40
Colleen
<SAL9000-> maybe if we bridge another protocol later that has their own more flexible IDs?
9:41:57
Colleen
<shinmera> Anyway, if we do bridge something like that I think the bridge will have to invariable do a lot of transforming
9:42:48
Colleen
<SAL9000-> right - I was thinking that there may be some value in translating external IDs in a reversible manner
9:51:23
Colleen
<SAL9000-> at which point we send a patch to glowing-bear involving emscripten to pull in the libsixel decoder?
15:12:42
SAL9000
Shinmera: lichat:create had an inheritance set of () rather than (lichat:update); I've fixed it in the lichat repo, haven't pulled the change into py-lichat yet
15:33:35
Colleen
<SAL9000> Subtree'd fixes in question into py-lichat. Also, looks like ex-lichat holds onto connections/users for way too long?
19:40:42
Colleen
<shinmera> the code that does the autogen is gross, but whatever. https://github.com/Shirakumo/jLichat/blob/master/src/main/java/org/shirakumo/lichat/SpecGenerator.java