libera/commonlisp - IRC Chatlog
Search
7:58:57
lukego
Hey anyone have an idea why static-vectors version 20210807-git - same version as in latest quicklisp - would be failing to build in a certain env with this error? The function (COMMON-LISP:SETF STATIC-DISPATCH::COMBINATION-FUNCTION) is undefined.
8:00:02
lukego
seems weird especially since the error is preceded by this message: The function (COMMON-LISP:SETF STATIC-DISPATCH::COMBINATION-FUNCTION) is undefined.
8:08:13
jackdaniel
the word static seems to indicate, that the function may be expected at compilation time perhaps?
8:08:37
jackdaniel
i.e something tries to call it from the macro and the functio ndefinition is not wrapped in eval-when
8:10:23
scymtym
lukego: is it really the combination of static-vectors and static-dispatch? i had those as completely separate in my mental model
8:34:29
lukego
jackdaniel: yeah that's how it reads to me too, but it seems weird because it builds fine in other similar environments, same code on another SBCL and Linux userspace of a similar vintage...
8:35:02
lukego
scymtym: oh, sorry, I see what you mean, no it's just static-dispatch and that's a typo/thinko
8:35:47
lukego
maybe it's a different in the way compile-time and load-time are staged in this environment...
8:37:06
jackdaniel
and on both environments you've tried with same initial conditions? i.e an empty cache
8:37:58
lukego
not really, I have a pretty loose understanding of how the environments are initialized in many ways, more moving parts than I can really account for. I'll try doing separate COMPILE-OP and LOAD-OP etc.
8:39:59
lukego
the new failing case is when I try to add my dependencies to the upstream packaging of Quicklisp in NixOS/nixpkgs. I used a similar setup before and there I did some specific tweaks e.g. forcing packages to be loaded during packaging to realize various initialization steps...
8:40:26
lukego
but that wasn't due to Lisp weirdness so much as Nix weirdness i.e. package directories being migrated to read-only storage after build and not being able to generate artifacts there
8:44:29
lukego
The good thing is that in this framework I probably have an easy way to apply local patches to projects and can start fixing things without quasi-permanently forking them and having them go stale on me
9:06:37
lukego
at least accumulating individual bits of duct-tape is at this point preferable to ~/quicklisp/local-projects/ being a completely ad-hoc and outdated mirror of half my dependencies :)
9:07:29
lukego
also this would be sharable because it turns out that other people are actively doing stuff with the Lisp packages in Nix land.
9:26:01
jackdaniel
yes and no :) you may create a bundle with quicklisp (then dependencies will be also downloaded) into the destination folder
9:26:32
jackdaniel
that said "only compile" may be a bit misleading - in order to compile foo that depends on bar, you need to first load bar
9:26:56
jackdaniel
(i.e cl-jinx depends on alexandria and uses its function in macro expansions - i.e alexandria:parse-lambda-list)
9:30:13
lukego
jackdaniel: thanks yeah that seems good enough for having a simple env to make bug reports from
9:39:01
jackdaniel
for source code distribution. instead of depending on quicklisp in CI you build a bundle artifact you may load with only asdf
9:44:15
lukego
Sounds nice. Generally I'm always looking for moar import/export of well-defined passive data. Often this seems to be hiding behind operational APIs. For example Quicklisp releases are defined by text files listing the source for each package - and CLPM too as I understand it - but these seem to be treated mostly as internal implementation details while for me it's the primary artifact of interest
9:46:21
lukego
i.e. for me the really phenomenal thing about Quicklisp is that it defines a set of relatively up-to-date software versions that are known to actually work together. that's huge -- especially in that often the reason they work is that Xach has given authors a heads-up when they break. The fact that it also has a client to automatically install stuff is just a cherry on the top.
9:50:11
lukego
I suppose that in Nix terms what's needed - and is cobbled together in various out-of-tree packages - is something similar to Bundles but exporting only metadata e.g. the list of packages, the list of dependencies between packages, the list of systems provided by each package, etc. Maybe exporting that could be a Quicklisp (or CLPM or ...) feature one day to cut down on ad-hoc wheel reinventing.
9:58:41
lukego
Sorry to spam, but here is my "duct-tape bliss" workaround to the static-vectors problem, quite satisfactory for the moment:
10:00:35
lukego
which is considerably less nasty when I add a comment linking to a proper bug report on github
10:01:53
lukego
I guess you always have to find your own tribe. This packaging setup already has a few very similar hacks to this so likely it represents my people :)
10:02:26
lukego
in a perfect world I'd prefer something managed by Git but that seems like too much of a headache when you have hundreds of projects that each need to be forked separately
10:03:56
lukego
although now we come to the real yuck for me which is that it will take about 45 minutes to see if this works due to the serial nature of the package updating code
10:03:57
jackdaniel
- does it work? ; - well, kind of. first you need to apply these 2 patches to compile it, then this binary patch to fix a miscompiled magic number - easy peasy
10:06:19
lukego
for me it's comforting to have one central file, in this case https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lisp-modules/quicklisp-to-nix-overrides.nix, which declares all the obscure rules for installing a consistent set of working packages, including all their FFI dependencies and required patches due to bugs that need to be worked around and so on.
10:06:51
lukego
so yes, you do have all those obscure rules as you describe, but you don't have to remember them because they are collected and automatically applied.
10:07:38
jackdaniel
until you, what, change the package manager? ;) either way this seems to be only remotely related to lisp, sorry for interrupting
10:07:49
lukego
and my BATNA here is to make a fork of static-dispatch, somehow redirect my builds to that instead of the Quicklisp default, and then feel guilty about not having any strategy for keeping that in sync with upstream
10:08:56
Nilby
At least 3 OSs that I use seem to be held together by a lot of such duct tape "bliss" when you look under the hood.
10:10:12
lukego
jackdaniel: I think it's a practical issue too, e.g. I don't get automatic updates of McCLIM on my machine, because I have an older version forked in ~/quicklisp/local-projects/, and I can't easily get rid of that because it contains a dirty hack that's needed to locate fonts on my machine that is too nasty to upstream
10:10:42
lukego
so it would be an upgrade for me if my "distro" would automatically pull the latest mcclim and just automatically remember to apply my patch to that, or complain if it doesn't apply cleanly anymore.
10:13:36
Nilby
Not saying it's good, but even old Lisps you'd have to all apply-all-patches before getting a sane system.
10:13:44
jackdaniel
speaking of fonts, cl-dejavu is available on quicklisp, so if anyone has some free time, then making McCLIM depend on that would be great
10:15:09
scymtym
lukego: do you know whether this https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lisp-modules/quicklisp-to-nix-overrides.nix#L226 and the next one are nix-specific? i can't immediately see why they would be
10:15:25
lukego
jackdaniel: Sure, but fixing stuff often has to be asynchronous. Particularly on NixOS there is a genre of problem where a package has done things in a way that's not correct but that you can get away with on most distros - e.g. hard-coding a path into /usr/share/xxx and hoping that's where the data is - but that stuff all breaks once you get to NixOS. So it creates a burden on NixOS users to try and figure out the "right" way
10:15:55
jackdaniel
lukego: I'm not preaching you to change your ways, I'm just saying that this looks damn ugly from sidelines
10:17:19
lukego
I spent a whole bunch of energy trying to "fix" the expedient workarounds that Pharo Smalltalk does on NixOS and pretty much burned out on that to the point I don't want to even look at the environment again. I'm trying not to get into that situation with Lisp :)
10:19:24
lukego
I suppose there are different kinds of ugly. This is my preferred one. Central collection of nasty hacks that are applied automatically and systematically and can be shared with other users on that same basis.
10:20:38
lukego
Here's the mcclim code btw: https://github.com/McCLIM/McCLIM/blob/master/Extensions/fonts/fontconfig.lisp#L18-L34. I just did what everyone else apparently does, added my own directory to the list, but my directory is "/nix/store/70rrm133w22d68lmx7gy3pzdp2xmjfar-dejavu-fonts-2.37" and that doesn't seem like something that should go upstream.
10:21:55
lukego
So anyway if I successfully migrate from "random hacked forks in ~/quicklisp/local-projects/" to "random hacked patches in quicklisp-overrides.nix" then I'll at least be getting automatic updates again, both of the actual lisp packages and also crowdsourced expedient hacks from other nix users...
10:23:16
lukego
and on the plus side, I'm not spending at least a whole day on this hacks-curating, and it seems like at the end I'll be able to make a pull request to nixpkgs to share the fruits of it with some small number of other people. I do like the idea of collaborative package-wrangling instead of doing everything in $HOME/
10:26:02
lukego
scymtym: Good question. No idea, it would be nice to have comments on these. I guess at least 'git blame' should work.
10:27:19
scymtym
lukego: i see, thanks. does nix require build steps to produce bit-level deterministic results?
10:27:35
lukego
scymtym: This is the upstream Lisp tooling for Nix and I never really looked at it before this week, having used a downstream one called ql2nix for creating "bundles." I believe there is some asdf-level instrumentation going into the builds, something about caching, and possibly this causes problems that are specific to this environment and need such workarounds
10:29:00
lukego
Generally speaking Nix doesn't do bit-level determinism, though maybe that is a thing now as a more recent advancement. the Nix brand of determinism is just that it was built with the same commands from the same sources, applied recursively throughout all dependnecies including e.g. compiler and libc and so on. If the actual program is non-deterministic, e.g. writes a random number to a file, that's mostly ignored.
10:29:23
_death
the nix introduction says "An important consequence is that operations like upgrading or uninstalling an application cannot break other applications, since these operations never “destructively” update or delete files that are used by other packages." but this can only be true if programs don't interact (anyway)
10:29:32
lukego
I think that the CI might try building packages multiple times and checking for differences, but that would be more of a style-warning
10:31:20
lukego
_death: I'd rate that as "mostly true." If program A depends on program B then it will be pinned to one specific version of B, identified by sha256 hash. If another program depends on B, or B is installed as stand-alone, then those will be separate copies (potentially shared if they are identical.)
10:32:54
_death
lukego: when using a program, a user can feed it output from another program, for example
10:33:21
lukego
_death: I mean in the sense that it uses it to compile, or wants it in the $PATH at runtime, or in $LD_LIBRARY_PATH, etc. But not in the sense of e.g. both attached to the same X11 server and being copy-pasted between
10:34:13
jackdaniel
OK, I'm as guilty as anyone, but let's cease offtopic. unless it is lisp-related please move i.e to #lispcafe
10:35:49
lukego
This is actually Lisp related. This is the way I'm managing my Lisp dependencies, including "shell out" executables like openssl and z3. I declare them as dependencies and Nix generates a line like "export PATH=/nix/store/abcd1234-z3/bin:..." so that the Lisp process always shells out to precisely the command that was declared as a dependency, including its patches and which compiler was used and what compiler arguments and so on.
10:36:52
jackdaniel
discussing inner workings of nix is not though; I sometimes eat when I code in common lisp but this doesn't render sandwitches ontopic
10:37:05
lukego
I have a vague fantasy of integrating this a bit more with Lisp, e.g. for Nix to be able to build a set of stand-alone executables and shared libraries that correspond with Quicklisp, so that the Quicklisp client could automatically fetch known-good working versions of all of those things. Julia ecosystem seems to have this and it works surprisingly well.
15:48:28
minion
lisp123, memo from beach: What I call "concrete data type" is anything data type where there is essentially only one reasonable implementation. An abstract data type is defined entirely by the operations possible on its instance, without reference to its implementation.
17:39:57
jcowan
beach: I don't follow that definition of CDTs. It looks to me like fixnum is a CDT on your account and so is float, but bignum is an ADT and so is string.