freenode/#lisp - IRC Chatlog
Search
17:03:41
stylewarning
(Echoing in here from #sbcl): We are running into a problem where we compile and load an entire system (a vanilla ASD), and we get some performance baseline of the application, then we do (load (compile-file X)), performance improves by about 20%, then (load (compile-file Y)), performance regresses back to normal. Y depends on X, and X precedes Y in the system definition. I looked for compiler policy leakage and that
17:06:05
stylewarning
(It might also help to know neither X nor Y were modified between compilations/loading, cache has been cleared, and X just contains structures+functions, Y just contains variables and functions)
17:06:47
phoe
beach: also, I think that Shinmera has done work that is superior to mine - he has done some work on something he calls a wire protocol that is usable for serializing data over the network into S-expressions.
17:07:10
phoe
His implementation is much better than mine and I would recommend to base any Eclector-related code on his work.
17:07:41
Bike
if you had function definitions before structure definitions and the functions use parts of the structure definitions i can imagine that on the second compile it will incorporate inline structure accessors etc that it didn't know about the first time and that could speed it up
17:08:04
phoe
Also the first part of the protocol specification at https://shirakumo.github.io/lichat-protocol/
17:09:27
stylewarning
Bike: that was my first thought as well, but doesn’t explain why loading Y again makes things slower
17:12:08
jackdaniel
stylewarning: and what if both files are in the same compilation unit? (i.e files are appended)
17:22:49
Bike
i figured if y defines methods it could mess up caches but that would only be a transient performance drop
17:23:29
jackdaniel
what if sbcl has a slot in the discriminating function which holds the last defined method?
17:23:52
jackdaniel
I'm not saying it has, but I can imagine such optimization, that could be done for instance for accessors
17:48:41
shrdlu68
How do I convince SBCL that I'm doing fixnum arithmetic here: https://imgur.com/a/LQOlHm2
17:52:24
pfdietz
You might try not assigning the smaller value into RES, but using a temporary? Also, what is the type of MASK-START? If it could be very negative, FIRST-LEN could be arbitrarily large.
17:55:41
pfdietz
(truncate -61 62) ==> 0, -61, so (- 61 mask-start) could be as large as 122, and the ASH does not give a fixnum.
17:58:40
pfdietz
I suggest tightening some of the ranges from FIXNUM, as allowed by correctness. Like (integer 0 61) for some of those, if that's right? Then remove declarations until it can't figure it out.
18:00:04
pjb
shrdlu68: don't worry, I've started writing an OCR software in Common Lisp. As soon as it's debugged, …
18:19:02
sjl
If you want modulo-62 arithmetic, consider defining arithmetic functions that do that explicitly, rather than using fixnum and hoping the compiler will let it overflow like you want.
18:20:18
sjl
that way if you run on a machine/compiler where fixnums don't happen to be 62 bits you'll get slow results instead of completely bonkers ones
18:28:57
warweasle
Is there a human readable version of circular lists/graphs? I'd like something like #1=(1 2 3 . #1#) but with human names like #beginning=(1 2 3 . #beginning#)
18:36:33
Bike
implementing #n# completely is tricky because you need to be able to iterate over structures and stuff.
18:39:31
sjl
Could you implement your own function that kept a map of symbol -> integer and funcalled (get-dispatch-macro-character #\# #\=) to do the actual work?
18:40:19
warweasle
I'm trying to get a non-programmer to enter formatted lisp...so I can do almost nothing and looks like I'm doing work.
19:42:10
anamorphic
Would it make much of a difference at runtime if I had compiled all the ASDF systems my code depended on, along with my own code from one, large, concatenated source file?
20:02:36
jasom
anamorphic: the compiler is allowd to inline functions that are defined in the same source file as they are called in without an inline declaration.
20:04:10
jasom
also when you load a file, it rebinds certain dynamic variables, so it's not 100% the same if you are modifying state at load-time.
20:22:03
anamorphic
So I've been looking at some CFFI-based projects, and sometimes the project will defined a file "reload.lisp" which includes only (define-foreign-library ...) and (use-foreign-library) calls and then in a separate file, the project will have all the CFFI (defcfun ...) forms. Is there a reason for the separate "reload.lisp" file?
20:24:01
Bike
i think so that if you load the project and then save image, it's easy for a restored image to dlopen the thing again.
20:47:28
sjl
anyone know of any tricks to have a standalone .lisp file that can be both executed as a script (via a shebang like #!/usr/local/bin/sbcl --script) and LOADed?
20:48:21
sjl
sbcl --script ignores the shebang, but trying to LOAD the file doesn't, and #! isn't a reader macro by default
20:48:27
Shinmera
It's not really possible unless you define a reader macro beforehand that causes the #! to be ignored
20:48:59
Shinmera
I've tried that sorta shenanigan for a while, and typically just end up making a shell wrapper script instead
20:49:42
pjb
sjl: the trick is to have a reader macro for #! as comment, and to use a feature in the repl, so you don't call the main function of the script. I use #-testing-script (main)
20:50:38
sjl
pjb: Yeah, getting the reader macro is the hard part, because the #! has to be the first two bytes of the file for the shebang to work
20:52:28
sjl
Sure, if you load another file first it's trivial. Or even sbcl --eval "(set-dispatch-macro-character #\# #\! #'read-line)" --load scratch.lisp
20:54:12
pjb
Also, you don't have to use #! on linux and other unices. There are kernel modules to interpret files as scripts depending on other things, such as file name or path.
20:54:51
pjb
But since it requires a custom installation of a module or configuration, you couldn't use it for distributed files.
21:01:56
sjl
Yeah, there's lots of ways to do it. I was just hoping there was some magic trick to get it all self-contained in a single file.
21:02:41
aeth
You wouldn't want that because you want the script to call (main) and you would want the loadable file to leave calling (main) to the user.
21:07:31
aeth
I would personally just use: sbcl --noinform --non-interactive --load foobar.lisp --eval "(foo:main)"
21:08:49
aeth
bash will ignore #| as a comment, then you run sbcl (and this way the path isn't hardcoded to sbcl)
21:09:43
pjb
Yes, you can always write bash scripts that call lisp implementations to run pure lisp files.
21:10:59
pjb
This is what I do from Makefile to generate executable images. as bash scripts: #!/bin/bash \n exec ccl --no-init --load "$0".lisp
21:11:10
aeth
I think you need something like exit or something before the CL part? And then you wouldn't need # |# and can just have |#
21:12:46
sjl
there are twenty different ways I could do it, but I don't think any will satisfy my goals of "single standalone file", "executable on its own", "loadable with LOAD" on its own
21:13:20
sjl
Not a big deal, I was just curious. I usually end up dumping binaries for these kinds of things anyway
21:13:49
pjb
your requirement is loadable with load, not loadable with load in an image that has been prevented to load its rc files.
21:14:02
aeth
sjl: Bilingual sh/lisp where the first part is sh that reloads the file as CL meets the requirement!
21:15:23
pjb
Yes, but "executable on its own" doesn't need a reader macro: it's interpreted by the kernel!
21:16:38
sjl
for the script to be executable on its own, it needs "#!" as the first two bytes in the file, barring any extra kernel extensions and such, right?
21:17:43
sjl
For the file to be LOADable, you need a reader macro that knows to ignore that line. But there's nowhere to *put* that reader macro aside from some other file somewhere, because the #! has to come first.
21:20:10
pjb
If you change the requirements in the middle of the problem solving, then I don't play.
21:21:36
sjl
I thought "standalone file" clearly implied "other files aren't allowed". Anything you could do with a non-rc file you could certainly stuff into an rc file instead, so disallowing only other-files-that-arent-rc-files doesn't make much sense.
21:22:44
sjl
I refuse to use clisp for moral reasons -- they chose a name that confuses the heck out of new lispers.
21:23:45
aeth
sjl: That is if I'm remembering the hack properly. I saw something similar to this once
21:27:12
Shinmera
it's not a feature of any kind, it's just using # as a comment, reloading itself as lisp, then exiting before the |# can be read
21:29:26
pjb
hard coding of things in script is not a problem, if you have a Makefile with an install target to do the hard coding.
21:29:52
sjl
> If the execl() function fails due to an error equivalent to the [ENOEXEC] error, the shell shall execute a command equivalent to having a shell invoked with the command name as its first operand, with any remaining arguments passed to the new shell.
21:35:41
aeth
sjl: idk, as someone who plays around with languages and with bilingual files, this is pretty trivial
21:36:12
aeth
It's imo the Wrong Thing™, though. The right thing is to have a separate, trivial .sh file that runs the .lisp file imo.
21:36:46
sjl
I mean the POSIX behavior of "no shebang -> yolo, just pipe whatever this thing is to a shell" is horrifying
21:37:25
sjl
Lisp's multiline comments and bash's comments happening to use the same # character is just a... "lucky" coincidence
21:37:57
anamorphic
How does roswell's scripting differ with cl-launch? (apart from probably finding systems and stuff)
21:38:29
aeth
sjl: Technically, you could do the reader hack in the CL part and have a #! at the top as well
21:39:15
sjl
aeth: Yeah, but if you "do the reader hack" you need a separate file for the reader macro to go in, and if you're gonna have a separate file you may as well just do this all in a more reasonable way
21:40:02
aeth
Shinmera: That's possible, but now you have to have a different implementation for every CL implementation
21:40:20
aeth
Shinmera: Right now, it's pretty clear what to substitute to make it run in ECL or CCL etc.
21:40:34
aeth
Once you start having to do something other than calling main it might not be as portable
21:40:55
Shinmera
sjl: I mean a combination of aeth's trick with a shebang at the top to make it "standard"
21:44:34
sjl
If you do the reader macro in the sbcl args, you don't need aeth's trick, you just use a shebang of sbcl
21:45:34
Shinmera
no, I mean aeth's trich invocation will include the reader macro in /its/ call to sbcl
21:45:41
jasom
you can do it without the args by using a here document as well; that might be able to be made portable (modulo parsing native namestrings).
21:46:12
aeth
sjl: You still need the block comment trick. The block comment trick is for bash, not for SBCL
21:47:02
aeth
Shinmera: I think what you were saying was to include the reader macro definition in the --eval ? I don't think it would work, though, because isn't that run after the load?
21:47:39
aeth
There might be a way to run something before the load but that's probably implementation-specific
21:48:01
aeth
And it would be messy to have non-trivial Lisp in a bash string in a Lisp block comment
21:49:14
aeth
sjl: but yeah, you basically have to write bilingual files by (ab)using coincidences like the #| coincidence.
21:52:25
jasom
https://gist.github.com/jasom/7a02f6f228dcb469845ff1db86537e85 <-- no --eval needed. More changes would be needed to correct the output of course though.
21:54:16
jasom
that's doable by not having the shebang, but then non-posix compliant shells won't run it
21:56:37
aeth
Thanks to uiop:run-program and uiop:launch-program, CL is your shell. So why would you use any other shell?
23:43:45
emaczen
Does the CFFI groveller parse each include file, find the struct definitions and then write the defcstruct forms?
23:47:40
jasom
emaczen: I think it writes a C program to print out size and offsets, then compiles and runs the C program.
23:50:11
emaczen
jasom: I wrote a tiny program to do what you are saying to find constants and the sizes of structs
23:52:02
jasom
emaczen: it can (non-portably, but usually works) be implemented as #define offsetof(st, m) ((size_t)&(((st *)0)->m))
0:12:52
jasom
cl-autowrap actually parses and extracts types and things but I didn't think the groveller did.
0:13:26
jasom
emaczen: C11 lets you do it, but most programs don't target that version yet so may not work in C11 mode.
0:16:19
jasom
emaczen: especially since many APIs define what fields there are and their types, but not their order.
1:09:02
emaczen
Bike: I don't see how sizeof and offsetof could distinguish between types with the same sizes
1:13:57
|3b|
at least i hope people wouldn't make APIs where something is a float sometimes or an int other times
1:16:07
|3b|
i'd expect most actual changes in struct definition per platform to be things like using different sized ints, assuming just taking native int size wasn't good enough
1:19:40
Bike
for the groveler you can specify the types in a struct. dunno if that's required, it's a &key
1:20:08
|3b|
looks like it does https://github.com/sionescu/iolib/blob/master/src/sockets/grovel.lisp#L89
1:23:20
|3b|
and cffi-grovel includes ctype https://common-lisp.net/project/cffi/manual/html_node/Groveller-Syntax.html
1:35:06
emaczen
Bike: I had written my own groveller minus the C structs a few weeks ago, and I just added the C structs
1:35:58
emaczen
I looked at the types of structure members earlier today when I was manually typing defcstruct and I noticed differences
2:38:16
russellw
what's the shortest way to add an entry to an alist? I mean, you can use a pair of cons calls of course, but is there a shorter form?
2:39:37
no-defun-allowed
(push (cons key value) alist) and (setf alist (acons key value alist)) are common