freenode/#lisp - IRC Chatlog
Search
6:25:45
bhartrihari
Hello, how do we address the problem of somebody pushing malware in an update to a package published on quicklisp (mainly Xach's dist)? It seems to me that currently the onus is on the user to check the various systems for potential malwares. Which could turn into a nightmare if the number of packages scale (and so could any code review to verify them). It seems that being able to depend strictly upon a
6:25:45
bhartrihari
particular version of a library, (which may be marked as reviewed or verified) might be of slight help in this regard.
6:32:11
heisig
bhartrihari: Do other software ecosystems have a solution for that problem? Thorough code reviews are hard, so I'd consider that impractical.
6:33:27
heisig
My approach is that I mostly use software from people that I trust. (Where trust means 'I know where they live' :D)
6:38:09
heisig
The more lightweight approach would be to place the project in question in ~/quicklisp/local-projects.
6:52:28
bhartrihari
heisig: I understand that you are trying to give solutions for individual programmers, but I think we need some support for this in quicklisp. I can put certain versions in local-projects, but I need a way to share that information with the users of my libraries, having to setup a dist for that (I don't want to push this too far but there's very less documentation on how to do that) is too burdensome.
6:58:44
fe[nl]ix
the model of npm, maven, etc... is that 1) each package has its own channel, and 2) the package author can push independently
7:00:18
fe[nl]ix
in this model, the client that fetches packages has to resolve dependencies, and it's inevitable to end up with a 3-SAT problem a.k.a. dependency hell
7:08:29
bhartrihari
I see. I should've been more precise about "pushing". I meant somebody pushing malicious code in their repo, and Xach pulling it in his dist, or ultralisp for that matter. So one is constrained to use the version of a library that was included in a certain distribution of Xach's dist? Can one choose which distribution to use?
7:10:28
bhartrihari
I would like it if there was some option to pin the version of a library. Then the continuous stream would make new versions available quickly.
7:10:46
fe[nl]ix
if you don't trust Xach, I suggest you simply fetch the sources, bundle them in your repository and review the code
7:14:49
bhartrihari
It's not my personal problem fe[nl]ix. Everybody who uses any quicklisp dist is affected by it. I was merely trying to start a discussion on evaluating what can be done on the quicklisp side of things.
7:25:44
fe[nl]ix
I first switched to pinning and flat dependencies, and just checked-in the pin file
7:26:57
fe[nl]ix
then people kept updating the pinned version every other day and causing more breakage, so I just checked-in the sources
7:34:26
bhartrihari
Maybe it works for bugs which break code in a more obvious manner. There's only so much one person can do.
7:34:54
Shinmera
bhartrihari: you can run your own dist that only publishes audited updates. That's the best anyone can do to secure against malicious injection.
7:39:24
bhartrihari
We can do that. I was wondering if being able to pin audited versions of libraries from the same dist could work better, in that it is less burdensome and doesn't fragment the efforts of the community.
7:41:14
Shinmera
the point of the dist is to provide a snapshot of the world that has some guarantees about stability. as soon as you pin only some libraries, that model breaks.
7:41:43
Shinmera
you can do that yourself, by cloning the library into local-projects at whatever you want, but you also bear the consequences of doing so.
7:43:35
phoe
bhartrihari: you can e.g. pin NAMED-READTABLES to the current, reviewed version and enjoy it working
7:44:23
bhartrihari
Okay, how about using a package from a previous publication of a world? Can one do that currently?
7:47:48
phoe
scymtym: anyway, has there been any issue created about this SBCL-related breakage you mentioned?
13:07:54
Harag
is it just me or is hash tables in sbcl 2.0.5 a lot faster than 2.0.0!!! My db test is loading 1mil records with hash-table indexes in 17 seconds where it was taking 100 seconds previously!!!
13:20:58
phoe
Harag: http://www.sbcl.org/all-news.html mentions a few hash table modifications, but mostly for EQUALP
13:46:44
Harag
if I run the tests over and over sbcl eventually gets its nickers in a not and performance goes out the window again. (event with restarts)...trying a reboot now to see if that helps
14:20:12
Harag
I tried a naive avl-tree instead of the hash-tables but it was horrible... at least the populating it was
14:36:13
Harag
pffft sbcl 2.0.5 went backward in gc ... running my test twice in a row now crashes sbcl... last night on sbcl 2.0.0 I ran the tests over and over for hours without one crash while I was trying to tweak the code
14:37:13
Harag
An mprotect call failed with ENOMEM. This probably means that the maximum amount of separate memory mappings was exceeded
14:41:11
Harag
To fix the problem, either increase the maximum with e.g. 'echo 262144 > /proc/sys/vm/max_map_count' or recompile SBCL with a larger value for GENCGC-CARD-BYTES in
14:49:10
Harag
cant find backend-parms.lisp ..grepped for GENCGC-CARD-BYTES and can only find notes about it nothing to set it
15:12:37
Harag
according to https://docs.actian.com/vector/5.0/index.html#page/User/Increase_max_map_count_Kernel_Parameter_(Linux).htm it should be 65536 if my math is right (/ 2097152 32)
16:24:53
jmercouris
there is some strange bug I can’t figure out that keeps causing our sever to crash, leading to downtime if i am not paying attention
16:25:15
jmercouris
wondering if I can just wrap everything in a condition handler and restart or something
16:33:05
phoe
jmercouris: run with --disable-debugger or an equivalent, use your BSD's init system to restart it on crash
16:36:35
phoe
it's not really a Lisp question at this point, it's a question about BSD services and their restart strategies