libera/commonlisp - IRC Chatlog
Search
0:34:29
samps
Hi folks. I've been working on some pet projects in Common Lisp. I'm using SBCL and Iḿ having trouble trying to understand a compilation error. Would anyone be able to help me out? https://gist.github.com/lsmag/ebb517a8702f454c4bbf624d7c44b182 ... I tried typing out roughly the same faulty line on the REPL and it seems to work
0:34:50
samps
changing the line just to (merge-pathnames (first argument)) also spawns the same error
0:36:53
samps
I've read about it but haven't dipped my toes into condition signalling yet, though I have found quite a few while playing with the adopt library
0:37:06
EdLangley[m]
Anyways, that's not an error it's a warning because write-line expects a string and it's getting a pathname
0:39:00
EdLangley[m]
samps: yeah, I'm just mentioning that because using the expected terminology helps people understand the problem :)
0:39:48
samps
damn, I was caught off guard. The signal message actually starts with `IN (WRITE-LINE ...)`. Thanks, folks :)
0:41:27
EdLangley[m]
I'm not sure, my guess is that just means sbcl able to print the form that's triggering the error for some reason
1:22:27
Xach
White_Flame: i seem to remember sbcl putting a finalizer on closing the fd on a fd-stream, but I'd have to double-check. I don't know if that would make it "typical" or not.
1:50:21
Bike
so thanks to mfiano nerd sniping me, i now know that sbcl actually does maintain backpointers from arrays to arrays displaced to them (sb-kernel:%array-displaced-from). however the general form of the with-array-data macro does still do a loop to get at the simple vector backing
1:50:21
Bike
(https://github.com/sbcl/sbcl/blob/master/src/compiler/array-tran.lisp#L1629-L1661). so does sbcl not actually collapse displacements despite going through some effort to maintain the backpointers?
1:53:44
EdLangley[m]
Is there any way to promise to the compiler that an argument to a function is const?
1:54:55
EdLangley[m]
e.g. if I have (defun mapping (fn) (lambda (list) (mapcar fn list))) It'd be nice if I could convince the compiler that it doesn't need a closure
1:57:30
EdLangley[m]
What I'm trying to figure out if there's anyway to use functions like this without blocking optimizations of things like (compose (mapping #'1+) (mapping #'1+))
1:58:35
EdLangley[m]
If the compiler assumes FN never changes, I'd think you could do some sort of inlining trick to avoid extra indirection.
2:01:19
EdLangley[m]
It should be able to replace (mapping #'1+) with (lambda (list) (mapcar #'1+ list))
2:01:40
Bike
if it's willing to believe that MAPPING will never be redefined, yeah. you could just proclaim it inline.
2:02:25
Bike
but MAPPING as a function itself can't behave as though its argument is unchanging, because it doesn't know what its argument is until it's called.
2:03:51
EdLangley[m]
It can't know that the value named by FN isn't mutated, but it can know that FN is never re-bound
2:06:28
mfiano
How would it eliminate it if calling thing function is the only way to bind it to a value?
2:07:19
EdLangley[m]
I think what I'm really interested in is whether something like (compose (mapping ...) (mapping ...)) can be optimized at all
2:08:41
Bike
so (compose (mapping #'1+) (mapping #'1+)) will end up as something like (let ((m1 (mapping #'1+)) (m2 (mapping #'1+))) (lambda (&rest args) (funcall m1 (apply m2 args))))
2:09:04
Bike
then after inlining each (mapping #'1+) ends up as (let ((fn #'1+)) (lambda (list) (mapcar fn list)))
2:10:21
Bike
that said, it will probably not be smart enough to avoid consing an intermediate list, due to side effect restrictions
2:11:18
EdLangley[m]
I've always wondered how much of an optimization barrier passing LAMBDAs around is
2:11:54
EdLangley[m]
And it sounds like compilers can, in theory at least, be smart enough to inline them like any other function
2:13:19
qhong
why not do the same for compiler, assume the best scenario, and update dependent if the assumption is broken
2:14:40
Bike
for avoiding consing up the intermediate list, i think what the compiler would have to do is observe that the first mapcar's result is only ever used by the second mapcar, so the storage can be reused
2:14:48
qhong
When lambda get passed around, most of the case that means they are stored in mutable variables that confuses compiler
2:15:11
Bike
i mean, sure, but in this case the variables aren't actually mutable, so it's pretty easy to see how it goes.
2:15:25
qhong
Bike: there's systematic western adhoc method called stream fusion, and chad Soviet method called super-compilation
2:16:34
qhong
Bike: yes I think the above cases the only problem is global functions getting redefined
2:17:26
sm2n
but more or less any nontrivial language does not necessarily reach a normal form in finite time
2:18:19
Bike
"I have interpreted the major steps in biological and cultural evolution, including the emergence of the thinking human being, as nothing else but metasystem transitions on a large scale. Even though my Ph.D. was in theoretical physics, I" and i'm out.
2:18:20
sm2n
with supercompilation you can get bit-identical compiler output for all programs with the same denotational semantics
2:19:21
qhong
sm2n: are we talking about the same thing? there're two totally different thing supercompilation and superoptimization which has confusing name
2:19:53
pillton
Bike: Have you seen work which allows users to perform optimisations of compositions (H X) <=> (G (F X)) without relying on compiler macros / inlining? I was thinking something like (LET ((GRAPH (G (F (LAZILY X))))) (OPTIMIZE-GRAPH GRAPH)) <=> (LAMBDA (X) (H X)).
2:21:09
Bike
seen some of that in the haskellverse, i think, though i don't think i understand your example.
2:30:52
pillton
Bike: Assume that the evaluation of (G (F X)) is equivalent to the evaluation of (H X). I could extend both F and G to support being able to "record" their operations in a graph structure when given an "lazy" object. A "lazy" object allows F to inspect the object to obtain its type but does not perform the operation. The function F returns another "lazy" object which allows G to record its operation in the same graph as F and
2:32:08
pillton
Bike: The function OPTIMIZE-GRAPH takes the full graph and identifies that #'H can be used instead of (compose #'G #'F) and returns the "optimized" graph as a closure.
2:35:20
Bike
okay, sure. seems pretty simple phrased like that. your graph is something like <input> -> F -> G -> <output>, your optimizer replaces F -> G with H.
2:41:13
pillton
Sorry, I think realistically, the output of OPTIMIZE-GRAPH would be (LAMBDA () (H X)).
3:27:15
yottabyte
if I'm reading lines from a file, and I see "^M" in my repl, that means newline, right? how do I check for equality for that? as in, how do I check the line is a newline?
3:36:43
White_Flame
CR = 13 = ^M, LF = 10 = ^J. "Newline" technically is LF, but in practice the output encoder will render it as whatever's default for the platform
3:37:31
White_Flame
well, ASCII calls LF Newline as well, afaik. I guess #\Newline is independent of that
3:56:03
remexre
are displaced vectors expensive to have as temporaries? wondering why e.g. find has :start, instead of there being a convenient (slice vec 3 nil) or something to make displaced vectors, and the common pattern being to use them
3:58:53
pillton
remexre: (slice vec 3 nil) requires consing the displaced array where (find ... :start 3) does not.
3:59:56
remexre
shouldn't a really basic escape analysis be able to determine that the one created by (find x (slice vec 3 nil)) has dynamic extent though?
4:02:15
remexre
pillton: this wouldn't be something declared on slice, it'd be something the compiler could recognize after inlining it
4:02:31
minion
remexre: look at universal greeting time: It is always morning when a person enters a channel, and late night when they leave. You may want to read http://www.total-knowledge.com/~ilya/mips/ugt.html for further information
4:05:41
pillton
remexre: Sure, but it wouldn't catch every case where (find vec :start 3) is trivial to define, doesn't cons, interacts easily with :from-end, and doesn't require a "sufficiently smart" compiler.
6:13:27
beach
But we have only a small portion of the x86 database, and we have no other architectures.
6:15:50
beach
kathe: I mean, you can write a traditional assembler and a traditional liker if you like, of course, but it wont be related to the project I meant in that list.
6:17:17
EdLangley[m]
Most implementations don't have direct access to the hardware but, generally, they don't use the OS for much else
6:17:51
kathe
i can understand a 'cl' system generating object files, but how could they build an executable without having access to a linker?
6:19:46
mfiano
But nothing is wrong with here, if you don't mind people assuming you have a decent knowledge of CL
6:20:46
beach
kathe: In Common Lisp, there is typically just one executable, which is the entire Common Lisp system. FASL files are loaded into the image when that executable is running. There is a process that ties the contents of the FASL to the image, and that process is similar to linking, but there is no traditional linker creating an executable.
6:21:42
beach
kathe: Some Common Lisp systems can create an executable file, but that is usually done by turning the contents of memory directly into an executable file, and not by combining object files.
6:24:25
kathe
but, some part of a common lisp system has to know about "elf" or similar formats, right?
6:25:08
beach
In a typical Common Lisp system, only the executable of the entire system is an ELF file.
6:26:13
beach
Cluster takes a list of standard objects, each representing some abstract instruction or a label, like ADD or SUB. It then figures out, from the arguments, what machine instruction to use. That's the essence of an assembler. Reading text and writing object files is secondary.
6:27:27
kathe
moon-child: so a 'cl' binary is actually an entire operating environment host for the new application?
6:27:33
beach
The Common Lisp compiler produces a FASL (if it is the file compiler) or if it is COMPILE, it does everything in memory. Typically, at least.
6:29:22
kathe
actually, in olden times, microsoft access projects needed a microsoft access "runtime" to work when deployed in the wild.
6:32:03
mfiano
You say you are only familiar with batch-compiled languages, and then know all this about Smalltalk?
6:32:33
kathe
mfiano: i've worked with c++ majorly. but i've also spent around 10 years with smalltalk.
6:33:54
kathe
mfiano: and lisp was actually birthed because one of mccarthy's students/assistants thought it better to implement an interpreter instead of a pure compiler.
6:34:50
beach
kathe: Quote from Alan Kay: "I’ve written about this variously elsewhere: including in “The Early History of Smalltalk”, and I’ve emphasized especially that “the idea of Lisp” pervades the underpinnings of Smalltalk as it was developed at Xerox Parc."
6:35:35
kathe
beach: again, smalltalk the language isn't all there is, there's also smalltalk the architecture.
6:37:25
kathe
kind-a off-topic, does everybody who uses emacs on a regular 'pc' keyboard use the "esc" key in place of "meta"?
6:40:04
moon-child
I don't know how your keyboard is laid out ('pc' is not very descriptive). I am just telling you that the alt key generally corresponds to meta in emacs. You should be able to change which physical keys are associated with which virtual keycodes; the details will depend on what software you are using to perform input translation
6:40:07
kathe
semz_: i also have a "happy hacking keyboard", so i guess i'll just have to pull it out.
6:41:10
sm2n
I don't understand the appeal of these weird keyboards, are they really such a large improvement?
6:41:21
semz_
moon-child: interesting, it seems to get modifier keys right and doesn't lack F keys like so many
6:42:23
moon-child
sm2n: it is split and bowled, which avoids wrist deviation, and the keys are arranged ortholinearly, which avoids unergonomic sideways movement
6:42:55
moon-child
sm2n: it also places many important keys on the thumbs (vs just space on a traditional keyboard), which are much stronger than other fingers
6:43:46
moon-child
sm2n: long travel distance also reduces stress of a given keystroke, though this can also be mitigated by with care
6:44:31
White_Flame
kathe: for tab completion in a source code buffer, I use Esc Tab, as M-Tab/Alt-Tab is already taken by the OS :-P
6:46:30
moon-child
sm2n: I use the key labeled 'page down' as both compose and mode_switch (depending on press vs tap)
6:47:03
moon-child
sm2n: not sure how worthwhile it is to look at the idiosyncrasies of my keyboard layout :P it works for me but
6:49:28
moon-child
but, again, this is my idiosyncratic keyboard layout. If you want something with more buttons there is maltron, but I have not found kinesis lacking
6:51:14
moon-child
also one-handed keyboards which--I've always wanted to get two, one for each hand
6:52:07
sm2n
why would that be useful? Are you trying to replicate a hollywood movie scene in real life?
6:52:50
ck_
I can see the benefit, for all character tuples that you'd have to move the same finger for with only one keyboard
6:53:34
ck_
for example you could type "vr" maybe four times as fast. That's a significant improvement
6:55:52
White_Flame
are there "null modem" type in-memory streams anywhere, where a reader can block until a writer writes into it?
6:57:32
White_Flame
no, an in-memory stream where you write into it which will store into a buffer, and read from it which will block if the buffer is empty
6:58:45
White_Flame
hmm, sounds like it's a fair bit of overhead for just connecting a stream-reader loop to a writer in the same process
6:59:08
White_Flame
I'm using the zip library, where to compress a file you give it a stream to read
7:02:56
sm2n
Yeah, but for this to make sense you need some notion of multitasking, which is not part of CL
7:09:44
White_Flame
ah yeah, looking at the gray-streams-pipe, that's a proper in-memory buffering solution
7:09:46
EdLangley[m]
It looks like it's attempting to provide something on implementations that don't supply gray-streams
7:11:32
sm2n
right, but what implementations don't have gray-streams support but can also be assumed to run on unix?
7:17:25
sm2n
but this is unrelated and interesting: http://git.savannah.gnu.org/cgit/gcl.git/tree/gcl/doc/format
7:20:01
mfiano
Why would they use the difference #-(or ...) to load a file that uses an implementation-specific reader extension?
7:23:02
EdLangley[m]
It's also interesting that, like clisp, their git repo seems pretty active without ever having a release
7:25:16
EdLangley[m]
All I mean is they don't seem to care about cutting a new release, despite having a more or less continuous stream of commits
7:28:27
mfiano
My understanding is that they is a person and probably only uses it for personal things.
8:39:50
kathe
_death: i read it in passing while going through various articles/blogs/forums about it.
8:41:27
kathe
looks like mcclim, as it's name suggests is an interface-manager that uses gtk+ for widgets.
8:41:33
_death
kathe: it does not.. although old documentation refers to a gtkairo backend, which isn't part of the main branch
8:43:54
_death
kathe: https://github.com/mmontone/clim-doc contains documents on clim.. you can try the guided tour or other tutorials/guides.. probably not a good idea to _start_ with the spec
8:45:13
lisp123w
I don't have any idea of the topic, but I read online that memory safety is important, but I wonder if this is just an issue for languages like C and if CL already solved this problem before: https://news.ycombinator.com/item?id=30091980
8:46:16
_death
kathe: in climspeak, widgets are called gadgets.. here's code for drawing gadgets like push buttons, radio boxes, scroll bars, etc. https://github.com/McCLIM/McCLIM/blob/master/Core/clim-core/gadgets/concrete.lisp
8:46:24
lisp123w
kathe: A very good introduction to CLIM: https://franz.com/support/documentation/10.1/doc/clim-ug.pdf
8:49:40
_death
kathe: there are some videos on youtube talking about mcclim, and you can look at the demos by loading "clim-examples" and evaluating (clim-demo:demodemo)
9:01:56
kathe
in the lisp world, just like in the 'oss' world, everybody is hell-bent on doing their own thing.
9:03:00
kathe
emacs + slime/sly are so well entrenched that even through symbolics/genera had advanced graphical 'ide' systems, there's nothing for the present day common lisp systems.
9:03:44
kathe
i see rainer joswig's tweets about his experiments with graphical integrated development systems for common lisp and get serious heart-burn.
9:04:59
kathe
even though there are so many free common lisp systems, including the 'gpl' one called "clisp", 'gnu' people are hell-bent on developing their own common lisp, and that too a variant.
9:06:26
kathe
i'm taking time out to figure if i really want to go ahead with common lisp or simply learn enough common lisp to understand the "condition system" and "metaobject protocol" based 'clos' and return to the world of smalltalk via squeak.
9:06:45
phoe
or rather just one or a small group of developers who are hacking at it in their free time
9:08:04
White_Flame
lisp123w: yes, it's memory safe. basically, raw pointers aren't exposed and accessing out-of-bounds array indices or using the wrong accessors won't destroy your memory
9:08:05
phoe
you might want to ask on lisp-hug; Libera Chat is kind of more about FOSS implementations
9:11:42
ns12
White_Flame: "yes, it's memory safe" - What if I use a foreign function interface (such as CFFI)?
9:12:03
White_Flame
then that's not lisp, and you're interfacing with & includign non-safe language stuff
9:13:53
ns12
White_Flame: Is the Lisp part still memory safe in the presence of non-Lisp parts accessed through a foreign function interface?
9:16:03
ns12
"no, because the non-safe stuff can trample lisp" - Doesn't this have significant implications, since many Common Lisp libraries rely on CFFI?
9:16:44
White_Flame
ns12: correct, if you pull in a buggy/exploitable non-lisp library, you might have problems
9:17:00
phoe
ns12: in practice, this has as many implications as using these unsafe libraries without Lisp
9:17:23
phoe
you use a library, you get to use its bugs as well, no matter if these are about memory safety or logic bugs
9:18:19
phoe
...unless you can evaluate Lisp code, at which point not only pwn the machine, but can also abuse DX to destroy memory
9:18:41
semz_
Maybe more notable than CFFI (which obviously can't give you much more than the pittance C provides) is safety 0 code
9:18:43
_death
in general, nothing is safe.. there are risks and you can try avoid their triggers, place control mechanisms, or mitigate bad consequences
9:19:15
semz_
which isn't THAT common, but I've still seen (declaim (safety 0) (speed 3)) way too often
9:21:31
ns12
semz_: "it isn't as easy to spot" - Why not? Isn't it a simple matter of using grep to find "(safety 0)"?
9:22:40
flip214
ns12: no, something using mcclim or similar... allowing to click open subcalls and so on.
9:23:34
ns12
semz_: "CFFI dependency is usually documented" - I don't think so. In any case, CFFI would be listed in the ASD file.
9:27:06
ns12
Does the setting apply to everything that I compile, including all libraries and their dependencies?
9:30:52
mfiano
You shouldn't ever declaim optimization qualities at the toplevel anyway. The semantics are not clearly defined.
9:31:58
mfiano
Years ago I remember taking a couple days to find a bug that was due to some library declaiming at the toplevel.
9:32:10
phoe
therefore, after restricting compiler policy, you can e.g. (ql:quickload :jsown) without feeling guilty
9:34:58
phoe
mfiano: I am giving the programmer a choice at https://github.com/phoe/damn-fast-priority-queue/blob/main/damn-fast-priority-queue/src.lisp#L20, not set the defaults for them
9:36:09
mfiano
I already informed phoe of some large optimizations that can be made. I'm not sure if he applied them yet
9:36:09
jdz
I've been playing around implementing a straight-forward pairing heap: https://github.com/jdz/simple-pairing-heap/blob/master/bench.org
9:37:01
phoe
mfiano: not yet, https://github.com/phoe/damn-fast-priority-queue/issues/12 is still there
9:43:35
flip214
scymtym: some UI that allows to trace a function call, and displays the data graphically (X11, McClim).
10:34:17
beach
Although probably all implementations of Common Lisp are safe with high values of the safety quality, the language itself is not as safe as we would like to believe. Hence WSCL.