Search
Monday, 19th of September 2022, 12:49:21 UTC
15:01:16
Josh_2
I am not looking forward to mapping my lisp system to postgres
15:01:47
Josh_2
Does elephant even work?
15:02:09
ecraven
no idea ;) it used to work many years ago when I last tried it
15:02:18
Josh_2
Last commit on github was in 2018
15:02:45
Josh_2
I found a bug in BKNR so now I dont want to use that
15:02:59
Josh_2
I always get a very uneasy feeling using BKNR
15:03:34
Ober
full clos support, and it's fast
15:04:15
Ober
20k inserts/sec on sbcl. plus 7 concurrent threads all writing to it.
15:04:39
Josh_2
Well thats faster than Hunchentoot can respond to requests
15:07:37
Josh_2
Ober: do you use Manardb in any production code?
15:08:02
Ober
yeah. for parsing cloudtrail logs and up to like 2tb of stored data.
15:08:24
Ober
but I rotate the directories, so I never delete.
15:09:02
Ober
https://github.com/ober/metis is the project I wrote. all based on manardb.
15:09:18
Ober
it's fairly well documented as well.
15:10:13
Josh_2
I see you have multiple dbs, how does manardb compare to postgres?
15:10:47
Ober
manardb is fast, and it's all mop/clos
15:11:31
Ober
I can get 20k on a single thread to manard, most I get to postgresql slightly tuned is a fraction of that. leveldb/lmdb might be better comparisons
15:12:10
Ober
metis code is poo, but manardb has been solid for me.
15:13:14
Ober
you can open manardb multiuple times too. so one process writes to it with multiple concurrent threads, and then a cli can query it directly with no major slowdowns.
15:13:53
Ober
once I have my mito conversion done, I can compare real rdbms to it.
15:26:49
Josh_2
I might just go with both, start with Manardb but keep it generic so that I can easily change to Postgres
15:27:21
Josh_2
I need to consider that at some point we will want to access the DB from another service
15:51:17
Ober
you using mito? or postmodern?
15:53:16
Josh_2
I do not want to use an ORM
16:24:59
Ober
(thnappy:compress-string "1") works, but 10000000 -> thnappy:buffer-too-small
16:45:34
Josh_2
Good snappy compression huh
16:46:04
Ober
(flexi-streams:octets-to-string (salza2:compress-data (flexi-streams:string-to-octets "hello hello hello hello") 'salza2:zlib-compressor)) ;; it is
17:29:41
Josh_2
Ober: That string compressor turns my 128 string into 137
17:44:01
jcowan
Is that McCarthy's Elephant?
18:09:43
Ober
so progress! only using it on very large structures
18:44:52
Ober
is cl-store the best option to serialize structures/hashes, etc?
18:47:35
aeth
I think what's more common is to serialize into plists
18:48:17
aeth
as .sxp (more commonly .sexp, but that fails the Scunthorpe problem and will probably get filtered)
18:48:45
aeth
maybe not the *best* option, though
18:49:21
aeth
also requires a readable representation for everything involved
18:50:06
aeth
(structs have that, hash tables would need an extra step)
18:53:26
Lord_of_Life_
** NICK Lord_of_Life
19:01:17
Ober
oh serialize to a string for storage in a db.
Tuesday, 20th of September 2022, 0:49:21 UTC