freenode/lisp - IRC Chatlog
Search
10:25:00
lukego
Is anybody else using ql2nix (https://github.com/bradleyjensen/ql2nix) to snapshot quicklisp distributions with nix or is it just me? I'm noticing that some packages aren't happy living on read-only filesystems and trying to figure out a fix/workaround.
10:26:39
lukego
seems like it's mostly packages that want to do some code generation and store the results on disk. I'm hoping it will be enough to ASDF:LOAD-OP them on a read/write filesystem and then transport the files onto read-only later. Or will they want to always be able to read/write their source dirs?
10:27:36
Shinmera
For the packages I control they either do their caching on first load, or whenever the user requests it.
10:29:44
Shinmera
From the top of my head, those would be qt-libs, uax-9, uax-14. There might be more, I don't quite remember.
10:33:00
phoe
lukego: I remember Xach mentioning this issue some time ago for packages that require write access to their own directory. Most notably it was CL-UNICODE I think.
10:33:38
Shinmera
Question: there is the DECLARATION declaration, which allows you to declare custom declarations symbols. My question is this: how would I now process these declarations if they appear in a top-level context?
10:33:45
phoe
lukego: in general, you cannot do that, since Lisp code can write stuff into asdf:system-relative-pathname at any time.
10:34:48
phoe
lukego: in practice, most systems *should* be happy with read-only access after they are compiled and built. At this point, if RW access is required, I'd try to file a bug so the system uses some temporary directories or configuration paths instead.
10:35:09
phoe
Shinmera: process? As in, do you want to be able to (DECLAIM (FROBNICATE FOO BAR BAZ))?
10:35:55
phoe
Impossible in pure ANSI CL. You'd need some mechanism to hook into the declaration system of each implementation.
10:35:57
Shinmera
I can do it if I have a macro that controls a body wherein this declaration may occur
10:36:20
phoe
Yes, but you can't do it at toplevel. The only thing standard ANSI CL can do is ignore these declarations.
10:36:27
Shinmera
just weird that DECLARATION exists at all given it's pretty useless if you can't react to it.
10:37:08
phoe
Shinmera: every Lisp implementation is allowed ignore *ALL* declarations aside from notinline, optimize safety and two more I think.
10:37:45
phoe
So, in pure theory, e.g. DECLARE CONNECTED from qtools should be freely ignorable in standard CL.
10:48:12
lukego
phoe, Shinmera: Seems like in this case the package gets a (QL:QUICKLOAD ...) before being transported onto read-only. Seems like that should trigger an ASDF:LOAD-OP right? I'm surprised there is more generated stuff being done after this.
10:53:55
jackdaniel
in principle compile-op compiles everything, load-op may use pre-existing artifacts
10:57:23
phoe
OK, to be correct - LOAD-OP may invoke COMPILE-OP if something is not compiled, but after LOAD-OP the system is sure to have been compiled and loaded
11:00:41
lukego
OK maybe I need to start digging in on a package-by-package basis and see what it is trying to do when it errors on r/o fs
11:08:11
phoe
lukego: that's the way to go, sadly. There are no general rules that can be applied here to help you.
11:08:50
lukego
I suppose that my "nuke them from orbit" option would be to setup some kind of read-write overlay/temp file system but let's see if it comes to that.
11:09:28
lukego
I'm looking at CL-UNICODE .asd now and seeing :perform (load-op (o c) (symbol-call :cl-unicode '#:create-source-files)) and wondering if it generates *every* time loaded?
11:12:11
phoe
or at least that is what https://github.com/edicl/cl-unicode/blob/911767add6656990e523cf3840f9f59a36a05385/build/dump.lisp#L241 tells me
11:12:57
jackdaniel
phoe: if you specialize load-op to do something more, then it will be done at each load
11:13:00
phoe
there is no need to generate this every time the system is loaded, only every time it is compiled
11:14:08
phoe
my question, though, is - can't CL-UNICODE cache the results of its compilation, probably by only generating and dumping the data in COMPILE-OP instead of LOAD-OP?
11:15:23
lukego
lemme see how to coax ql2nix to override the standard quicklisp version of cl-unicode with a local one that moves this from LOAD-OP to COMPILE-OP..
11:19:44
edgar-rft
https://edicl.github.io/cl-unicode/ says "CL-UNICODE builds parts of its source code automatically the first time it is compiled. This is done by parsing several Unicode data files which are included with the distribution and might take some time. This happens only once." but don't ask me if and how it works.
11:21:07
jackdaniel
so it is generated only on the first load (unless the .lisp source files get wiped out for some reason before next load)
11:22:14
phoe
but then it means that cl-unicode only dumps the new .lisp files on the first call to load-op
11:26:44
phoe
ASDF manual: "compile-op depends on prepare-op which itself depends on a load-op of all of a component’s dependencies"
11:37:11
phoe
so I don't get what is the problem with ql2nix in that case - by the time cl-unicode is compiled, all cl-unicode/build FASLs are in their proper places
11:39:10
jackdaniel
afaict lukego was only asking if it generates them every time, not stated that it does
11:44:22
lukego
I'm a little confused but I should note that I don't see the files present on the read-only file system. So the problem is consistent with jackdaniel being correct about the files not being regenerated unnecessarily - root problem might be that they failed to be created earlier for some reason and that's why they're being created now.
11:45:43
lukego
thanks for pointing this out jackdaniel. I'm rusty at reading system definition files. I saw :cl-unicode/build system and just assumed that was one of the dependencies, but now as you say I see that it's only conditionally a dependency.
11:46:50
Shinmera
for define-declaration, is there a way to determine whether the declaration occurs as a top-level declaration or within a function body?
11:49:01
Shinmera
Or, better yet, how would I retrieve the current function the declaration is expanded in?
11:54:36
Shinmera
I was hoping that for top-level functions I could gain access to the function name.
12:01:55
Shinmera
just because that exists doesn't mean the implementation isn't going to have named function objects, or the name available during compilation of a defun
12:07:01
phoe
But I have no idea how to retrieve the function name from inside SB-IMPL::NAMED-LAMBDA.
12:33:02
lukego
Curently I suspect the issue is not with cl-unicode/quicklisp/asdf but rather that it's local to the nix packaging and the shell script that tries to copy the results of asdf builds: https://github.com/bradleyjensen/ql2nix/blob/master/nixlispBundle.nix#L53-L77
12:38:12
lukego
Oh hm... my ignorance of both quicklisp and ql2nix is staring me in the face here... maybe it only does the ASDF:LOAD-OP for discovery purposes and then uses a quicklisp "bundle" API to make the actual package, which might skip the load step
12:40:15
lukego
Yes that's my working hypothesis now: that this tool is using ASDF:LOAD-OP to discover dependencies, then creating a "bundle" with ql:bundle-systems, then putting that bundle onto a read-only file system. But the generated artifacts for e.g. CL-UNICODE are being lost when creating the bundle.
12:42:43
lukego
So maybe I just need to add another load-op after the bundle is created but before it is migrated onto read-only fs
12:55:41
lukego
So the question now is, given a quicklisp bundle directory, how do I force load all the systems in that bundle
13:27:06
shka__
i have an unusual question: i would want to access and manipulate numpy arrays from CL
13:54:42
lottaquestions
Hi all, I inserted a breakpoint in a function using (break), and it worked as expected. However, once I was done with the breakpoint, I deleted it from the code and recompiled the whole buffer, C-c C-k, and the breakpoint kept being hit. I tried other options like M-x slime-eval-region, and even C-c C-c to compile the toplevel form, but the
13:57:31
lukego
maybe the function object was captured with #'foo? in that case seems tricky, have to work out where the reference would be stashed
13:58:29
lukego
maybe recompile the cl-ppcre:regex-replace-all usage in case that has expanded into code that captured the old definition?
13:58:43
lukego
you need to rerun th at code that does #'foo and make it tak the new function object
13:59:13
lukego
(and consider changing it to just 'foo if you want to indirect through the symbol to always use the current function definition in the future)
14:00:30
lukego
if you're in the debugger maybe you can patch the function object in the stack frame but that's beyond my ken..
14:02:46
lukego
another fun solution would be to hit the BREAK, get a reference to the old function object in the inspector, and then rewrite the heap to replace all pointers to the old function object with the new one. but I don't know how to do that either :) likely sbcl has some internal API for it..
14:07:09
lottaquestions
yeah, that would be fun. I know how to do somewhat similar things in gdb and C, and woudn't mind getting to the same skill level with sbcl
14:32:51
trittweiler
Clearly slime needs an interactive hex-editor of the dissasembly of function objects :)
14:33:41
trittweiler
lukego: Are you aware of sbcl's 20th birthday celebration in vienna in roughly 2 weeks, and if so, are you going to join?