freenode/#lisp - IRC Chatlog
Search
7:09:26
jackdaniel
so what was the yesterday's conclusion? undefined beahvior, use do or something more sophisticated? :)
7:11:24
jackdaniel
heh, I now see mail from gitlab.cl.net that phoe proposes to disable tests as undefined behavior
7:11:27
beach
For WSCL, I am debating whether to explicitly specify that the behavior is undefined or whether to define the behavior. And if the latter, what alternative to opt for.
7:12:05
ck_
I couldn't attend the debate, is there a short summary of the behavior in question, other than "loop" ?
7:16:27
jackdaniel
beach: if it were a vote for a preferred alternative I'd say that "6" is more useful; it makes the mental model of what's going on easier: increment always goes after the iteration and finally goes after all iterations
7:17:35
aeth
(loop for i of-type (integer 0 5) from 0 to 5 do (print i)) ; should this give a type-error when 6 is reached even though 6 is only used in terminating the loop?
7:18:35
jackdaniel
right, I've left it out because of-type seems to have clear semantics: if we choose to increment x to 6 that should signal a condition, if we choose not to increment to 6 then it will meet criteria
7:22:49
aeth
jackdaniel: well, I had to bring it up because if it's unspecified then maybe some implementations would choose both interpretations, i.e. unless the varaible is referred to in a finally (in its final, one above, form) do not have the type-error
7:52:56
fiddlerwoaroof
It's always interesting to see how many different interpretations of a specification arise
7:54:38
fiddlerwoaroof
I encounter this semi-regularly at work, where we plan out a project and then, when implementation-time comes around, the team implementing often realize that there still remain at least two interpretations of the plan
8:29:55
pjb
beach: didn't you argue at els 2016 that MIT loop was bugged on this point? That it should print 5?
8:32:27
pjb
I would note that this is related to C for(unsigned char i=0;i<=MAX_UCHAR;i++), which if not compiled carefully, can be an infinite loop…
8:51:36
beach
pjb: Yes, that sounds plausible, and that's how I implemented it for SICL LOOP, like I said.
10:29:43
phoe
Yesterday I was trying to figure out how to treat the LOOP issue for a long while and it left me kinda exhausted
10:30:00
phoe
I was hopeful that the conclusion of "let's treat it as undefined" would be decent enough
10:31:24
pjb
phoe: well, since all implementation are not conforming on this point, as a programmer you need to consider it as undefined, until they're all corrected. Ie. do not use the loop variables in the finally clause! But as an implementer, you need to correct it.
10:43:12
phoe
my question is, if it's practically impossible to agree what should the answer be than should we agree on an ANSI-TEST that tests this at all
10:45:02
White_Flame
its your very own "is the dress white and gold, or blue and black". Congratulations
10:47:32
pjb
Perhaps we'd need a comitee of implementers to decide these matters. At least a mail-list. But where all implementers, free and commercial would participate.
10:50:54
phoe
if we are down to implementations, is anyone around with a working cl-all? I'd need to see what cl-all says on the issue
10:51:24
pjb
phoe: 6 for sbcl ccl abcl ecl, which are the implementations I have currently running on macOS.
10:52:19
pjb
phoe: now, we may consider that, in a way, the implementation is free to set the loop variable to "anything" it wants in the finally, and that the bug is in the user giving a wrong type.
10:53:12
pjb
phoe: I wonder why the implementation don't set it to NIL for finally. for x to 10 finally (return x) could return 20, or 12, if the implementation thinks it's efficient to add more than the by, and to remove some before next loop…
10:53:45
Shinmera
phoe: how would it know better if the bounds are dynamic (but only static within a user-known range)
10:53:52
pjb
that's basically my point, with respect to type declarations! The implementation knows better!
10:54:23
phoe
Shinmera: the implementation can compute it - from 1 to 10 by 5 can be at most (integer 1 14)
10:55:01
pjb
(loop for i to (foo z)) the implementation can compute the type of (foo z), and know if it's bound or not.
10:55:53
phoe
it could in theory do a dispatch based on the compiled versions of the code it prepared aheat of time
10:56:34
phoe
but then if the user does OF-TYPE but passes values not of that type it's not the problem of the implementation
10:57:39
Shinmera
if it can set it to literally whatever, then of-type is entirely useless because you could never pass a type that is correct other than T
11:04:53
phoe
So what is the user supposed to do in case of (loop for x of-type fixnum from 0 to most-positive-fixnum)
11:07:12
phoe
because if the answer to the original question is 6 then the above code does not allow X to be a fixnum which makes it non-conforming with ANSI CL
11:09:21
pjb
phoe: the question is more general: what use does the implementation make of type declarations. If it uses them to effectly allocate low-level memory for untagged/unboxed data, then the secondary question is what happens when you (incf x) to go beyond most-positive-fixnum (or any other internal maximum of the data type used): does it do checks? If yes, then what optimization is that? If the check fails, it has to allocate new mem
11:09:21
pjb
store the result or what? Or does it blindly use modulo arithmetic and fail lamentably?
11:10:22
pjb
In any case, it sounds very silly to do such a thing (allocate memory for unboxed data).
11:13:44
phoe
(lambda () (loop for x of-type fixnum from (1- most-positive-fixnum) to most-positive-fixnum do (progn) finally (print x)))
11:15:41
pjb
(let ((x (1- most-positive-fixnum))) (if (< x (1- most-positive-fixnum)) (incf x) 'done)) #| --> done |#
11:16:17
pjb
But of course, this is less optimized, since you have a test, a subtraction and an addition, instead of an addition and a test that fails…
11:17:18
phoe
(declaim (optimize (safety 0))) (lambda () (loop for x of-type fixnum from (1- most-positive-fixnum) to most-positive-fixnum do (progn) finally (print x)))
11:22:52
pjb
The thing is that the optimization works 1152921504606846974/1152921504606846975 of the time…
11:25:33
pjb
If the implementation can optimize some specific integer ranges (eg. 0..255, or -128..127, or etc), then it can do it, but it needs to be able to determine the type specified by the user, and process the last element of the range specially.
11:26:50
pjb
:of-type (integer 0 5) should not pose any problem, since 3 bits can store up to 7. So internally x can be 6, and the user blamed for specifying the wrong type, or the final value be adjusted.
11:27:19
pjb
:of-type (unsigned-byte 1) :to 255 and similar should be processed specially, for the last iteration.
11:28:35
pjb
phoe: yes. Which is why I don't think that x=6 is a good thing for :to. But since x can go way beyond with (loop for x by 3 to 10 finally (return x)) #| --> 12 |# perhaps we should allow final x > max. and the user :of-type is wrong.
11:30:43
phoe
if the implementation needs to make X any type it requires to then it should be impossible to provide OF-TYPE to that variable
11:31:12
phoe
because what if the implementation requires it to be a bignum but the user tells it to be a fixnum
11:31:56
pjb
phoe: it may be because the programs runs on a different implementation with a different most-positive-fixnum. Then the implementation must signal the type error.
11:32:58
pjb
And if the bound are variable, either you know they're within most-positive-fixnum or you don't. If you don't, don't declare it fixnum…
11:33:12
phoe
(loop for x of-type fixnum from (1- most-positive-fixnum) to most-positive-fixnum do (progn) finally (print x)) is theoretically within fixnum
11:34:05
pjb
but what about (loop for x of-type fixnum from (1- most-positive-fixnum) by 3 to most-positive-fixnum do (progn) finally (print x)) ?
11:34:33
phoe
in that case it looks like it would have to be (1- most-positive-fixnum) by the same logic
11:34:35
pjb
The last iteration is with (1- most-positive-fixnum) so in finally x = (1- most-positive-fixnum) is consistent.
11:35:39
phoe
but it makes no sense from the optimization point of view if the implementation has a temp variable of type integer
11:36:15
phoe
since we do bignum arithmetic anyway then we could drop the fixnum variable altogether and ignore the OF-TYPE
11:39:06
phoe
or do we just close our eyes and pretend not to notice that we are doing the same shit as C when it comes to edge cases
11:41:04
phoe
this means that setting X to 6 in the original question is incorrect in general and all implementations known to me right now are buggy
11:42:51
pjb
phoe: more specifically, I could not find in clhs the specification of what value should be bound to the loop variables in the finally clause.
11:44:11
phoe
pjb: neither could I. The above conclusion comes from the fact that it cannot be safely bound to 1+ most-positive-fixnum if it is declared to be a fixnum and, from the user point of view, it looks like it's a fixnum all the time.
11:44:26
pjb
phoe: so we must decide. If we decide on the value of the last iteration, then things are clear. If not, then it will be mostly implementation dependent. (Again, setting it to NIL would be a valid choice. For example, the variables can be bound to NIL before the loop starts…
11:51:06
phoe
"A loop macro form expands into a form containing one or more binding forms (that establish bindings of loop variables) and a block and a tagbody (that express a looping control structure). The variables established in loop are bound as if by let or lambda. "
11:51:19
phoe
this means that everything inside LOOP (prologue, body, epilogue) is in scope of the bindings
12:23:20
pfdietz
(mapcar #'funcall (loop for x from 1 to 5 collect (lambda () x))) ==> (5 5 5 5 5) ;; maybe?
12:31:32
pjb
pfdietz: this is implementation dependent, you could get (1 2 3 4 5). In that case, the variable in the finally may be yet a different binding.
12:41:21
pfdietz
"step v.t., n. 1. v.t. (an iteration variable) to assign the variable a new value at the end of an iteration, in preparation for a new iteration."
12:55:52
phoe
In safe code, must (let ((x 0)) (declare (fixnum 0)) (setq x :zero)) always signal an error? Or are the consequences undefined?
12:56:09
phoe
http://clhs.lisp.se/Body/d_type.htm tells me, 2. During the execution of any setq of the declared variable within the scope of the declaration, the consequences are undefined if the newly assigned value of the declared variable is not of the declared type.
13:08:12
phoe
this implies that (locally (declare (optimize safety)) (let ((x 0)) (declare (fixnum 0)) (setq x :zero))) is safe code
13:11:50
phoe
uh I mean (locally (declare (optimize safety)) (let ((x 0)) (declare (fixnum x)) (setq x :zero) x))
13:15:48
phoe
"the consequences are undefined if the newly assigned value of the declared variable is not of the declared type.
13:18:10
phoe
But clhs 3.3.1 says, "In general, an implementation is free to ignore declaration specifiers except for the declaration, notinline, safety, and special declaration specifiers."
13:18:29
phoe
So if the implementation does not ignore it, then, uhhh. It's implementation-dependent what happens?...
13:20:44
pjb
phoe: what is implementation dependent is how the implementation generates the code. But the semantics of type declarations are rather clear.
13:22:11
pfdietz
The difference between implementation dependent and undefined is that a conforming program may have code with the former behavior, but not the latter
13:22:29
phoe
pjb: I need the standard's words on it. Is the above ALWAYS supposed to generate a runtime error in safe code or not?
13:24:51
phoe
The question therefore is: if we loop from 0 to most-positive-fixnum, is this allowed to be OF-TYPE FIXNUM or not?
13:25:39
phoe
Because if yes, then the variable's value must not be (1+ most-positive-fixnum) in epilogue.
13:26:09
phoe
And if it is (1+ m-p-f) then OF-TYPE FIXNUM is invalid which is contrary to user expectations I think.
13:26:13
pfdietz
Personally, it would be more useful if the var's value in the epilogue is at most the upper bound (for positive stepping)
13:29:29
pfdietz
the IN case is interesting. (loop for x of-type foo in list-of-foos do ….) What happens when the list is empty?
13:29:32
Shinmera
Eg having the last cons in the epilogue for ON stepping allows doing easy modification of the tail without having to do your own stepping.
13:30:17
pfdietz
The wording about initializing the var to a value that is of the type... I don't think I tested for that?
13:30:57
jackdaniel
and more seriously it should do the same thing as for implicit initialization of structures with slots of specified types
13:33:05
pfdietz
"If the optional type-spec argument is supplied for the variable var, but there is no related expression to be evaluated, var is initialized to an appropriate default value for its type. For example, for the types t, number, and float, the default values are nil, 0, and 0.0 respectively. "
13:33:21
pfdietz
(that's for local variable initializations, not iteration variables, but the question remains)
13:35:36
pfdietz
Makes me think there should be a bottom value, which is undefined to compute anything on, or even compare, but that is formally in every type (even NIL).
13:35:43
Shinmera
ACTION muses about defining a predicate that tests for a specific algorithm and using this to have the compiler generate an appropriate program as an appropriate default
13:36:54
jackdaniel
one could argue that if a valid default is not known compiler should signal an error
13:41:38
pfdietz
The practical solution would be to have the OF-TYPE be done with a (LOCALLY …) around the loop body, and another LOCALLY with a declaration of type (OR NULL …) around the epilogue.
13:42:44
phoe
the standard doesn't say anything about setting variables to NIL once the epilogue is reached
13:46:05
phoe
OK - please review this, https://gist.github.com/phoe/335fecfdc195bddd47ab0928b0e62e52
13:48:11
phoe
jackdaniel: and I am not, since I've found a way to invalidate the option where 6 is returned
13:50:24
pfdietz
I keep throwing stuff into my copy of ansi-tests that isn't for public inclusion. I need to refactor.
13:53:06
phoe
If yes, then you could add it to ansi-test but perhaps export it as a separate function or set of functions, and describe those in the README.
13:53:32
pfdietz
It should be, yes, and I've applied it to many implementations. But I feel it's a separate thing.
13:54:37
pfdietz
One new tweak to it, the mutational generator, is pretty sbcl specific. It assumes that if the compiler throws an error, that's a bug. But that's only an sbcl design principle, not something required by the standard.
14:02:01
pfdietz
I am working on a testing thing that will be very sbcl-specific. But it's intended to test test suites, not implementations, so that's ok.
14:11:34
pfdietz
A generalization of that: mutation testing. That is, a way to mutate the source code of some software under test, to confirm that every mutant (that doesn't leave behavior unchanged) is detected by the test suite.
14:12:10
pfdietz
Figuring out if mutants are sterile (do not change behavior) is the central problem, but there are interesting tricks for that.
14:13:07
pfdietz
I was applying a prototype to some of the internal code from SBCL (like, the implementation of APPEND and such) to see if ansi-tests killed all the mutants.
14:19:31
phoe
OK, SBCL/CCL/ECL tickets created. I can't log into the tracker for ABCL just yet and I don't know where to create CLISP bugs.
14:29:56
Bike
clasp copies ecl and none of the few changes i've made to the loop implementation would involve this
14:35:29
jackdaniel
"code is conforming (...) since the user wants to perform an iteration for every fixnum" is hardly a argument for what is conformant (it may be argument for adopting one behavior), I'm sure that there are some mechanisms with undefined consequences with overlap with user "wanting"
14:36:33
phoe
jackdaniel: that's the weakest point of my argument since it literally is up for human interpretation
14:36:46
jackdaniel
my point is that "user intention" is not a universal guide of what is a valid common lisp
14:36:54
phoe
but my question is, if this is not true, then what does (loop from m-n-f to m-p-f ...) mean
14:38:35
jackdaniel
I don't care much of what is returned from the finally clause; I'm just afraid that if CL is implemented from the standard in 20000 years from now by alien archeologiests code depending on this interpretation may not run :)
14:40:18
phoe
jackdaniel: if OF-TYPE declaration is not good enough in that case then why is it allowed there
14:40:26
pjb
Imagine we try to remake dynosaures from DNA, and instead of getting dynosaures, we get toads! Oops, the specifications was buggy, the actual implementation was dynosaures, but the DNA was wrong!…
14:40:35
phoe
we only want the variable I to go between m-n-f and m-p-f since this is exactly what we specify
14:40:53
phoe
and if that is true, then it is obvious that the variable I is of type fixnum, since we do not intend it to be anything else
14:41:09
phoe
if the implementation assigns something out of the permitted type then it is the problem of the implementation
14:43:15
jackdaniel
"kinda makes sense" and "is the least surprising behavior" - again - is a good argument for unifying behavior, but not for declaring code conforming
14:43:46
phoe
if you find better wordings for these, please let me know - I'm more than happy to edit my writeup
14:49:46
jackdaniel
that quotation is incorrect, because it stipulates that I've expressed that opinion while I was answering the question "how this could be better phrased"
14:51:49
phoe
nope, I need external support - someone who would like to proofread this once more, please do
14:56:25
phoe
jackdaniel is correct but I'm too braindead at the moment to integrate his comment into the text.
15:29:39
phoe
I never even imagined that (make-condition '(or foo bar) ...) would be permitted by the spec
15:30:58
phoe
beach: does WSCL take care of cases such as https://gitlab.common-lisp.net/ansi-test/ansi-test/blob/master/conditions/make-condition.lsp ?
15:36:30
Kabriel
phoe: jackdaniel: remember, only you can tell what your program's intent is, not your compiler!
15:37:49
phoe
pfdietz: is there a way to run the tests without the ones noted with these ansi-cl problems?
15:50:32
phoe
huh, if inside CCL I do (load "/tmp/ansi-test/gclload1.lsp") then it tells me that File "compile-and-load.lsp" does not exist.
16:15:20
pfdietz
My original version of this assumed running the tests from the ansi-test directory. There was some churn on the loading code later (LPNs and such).
16:15:43
pfdietz
I did not want to use defsystem (later asdf) because those may not have been available on the lisp under test.
16:22:36
phoe
pjb: that would imply that (deftype foo () 'condition) (make-condition 'foo) should work
16:23:05
phoe
pjb: also if foo and bar have different slots, then what is the type of the thing that you get from the '(or foo bar) thing
16:37:55
jackdaniel
pfdietz: I want to replace makefile-based "start" with another lisp which runs tests against some other implementation(s)
16:38:17
jackdaniel
that would also enable nice result intercepting and reporting even on implementations which do not fully implement cl
16:38:20
phoe
if it encounters "#\G))))" it cannot infer that it should stop reading when it encounters parens; maybe the reader macro just gobbles up five characters whatever they are
16:43:01
phoe
an unknown reader macro can consume an arbitrary number of characters from the reader string before returning, it can read a Malbolge program and execute it
16:43:23
phoe
so when read-suppress encounters an unknown sharpsign macro, I guess it loses in the general case
16:43:37
phoe
the best thing we can do is to assume that a standard Lisp object follows the sharpsign macro I guess
16:49:27
phoe
https://gitlab.common-lisp.net/ansi-test/ansi-test/blob/master/reader/read-suppress.lsp#L72 <- where does it say in the spec that ";; Undefined macro dispatch characters should not signal an error"?
16:57:44
phoe
and it is impossible to win in this case since https://plaster.tymoon.eu/view/1540#1540
17:01:27
phoe
therefore I'd argue that it is impossible to win with an unknown sharpsign macro and the only sensible option is to signal an error
17:25:53
phoe
this constraint is impossible to satisfy - it is easy to confuse the SBCL reader as well even though it tries to work around this
17:31:04
Bike
i think signaling an error with an unknown compiler macro with read suppress would break real code.
17:34:59
jackdaniel
because someone may depend on the specified behavior for a trivial case and he will point at the spec: there, why you don't support this?
17:35:53
fiddlerwoaroof
Bike: I've come across at least one implementation that does signal an error in that situation
17:36:39
phoe
Bike: jackdaniel: what is a sensible default in that case? When one has an unknown reader macro, then literally anything can follow.
17:37:29
Bike
fiddlerwoaroof: i'm thinking of code that's like, conditionalized to use ccl macros or something
17:37:38
jackdaniel
phoe: if someone puts code for which behavior is not defined (like the case you mention), then it is their fault
17:38:27
phoe
jackdaniel: what exactly is doable in the spec in that case? One of my questions was, where in the spec it says that unknown sharpsign macro characters must be supported - and what does it mean to have them "supported"
17:39:03
fiddlerwoaroof
phoe: for most characters, without a reader macro definition, they're just consituent characters
17:39:27
jackdaniel
phoe: a reasonable assumption is that they consume the next sexpression (either atom or a list). of course that will break for many edge cases
17:39:53
phoe
fiddlerwoaroof: jackdaniel: I understand that one. My question is what is defined, is it possible to satisfy it, and what is actually implemented
17:40:18
phoe
because if the spec requires unknown sharpsign characters to be supported, it requires us to solve the halting problem due to reader macros being Turing-complete.
17:42:23
phoe
My actual question is - since the spec is either unclear or unsatisfiable, what is the behaviour that we actually want to settle on and "standardize" in ansi-tests or beyond-ansi
17:42:54
jackdaniel
phoe: we can't settle or standarize undefine behavior in ansi-tests because that is *not* standarizedx in ansi
17:43:14
fiddlerwoaroof
Initially, every character in the dispatch table associated with the char has an associated function that signals an error of type reader-error.
17:43:31
phoe
(list #\GARBAGE) is unsatisfiable in general either, even though it is in the (non-normative) characters
17:44:02
phoe
fiddlerwoaroof: if that is the case, then reading an unknown dispatch macro character should signal an error
17:45:09
phoe
Dispatching macro characters continue to parse an infix numerical argument, and invoke the dispatch function.
17:45:29
Bike
it says for example that invalid uses of the dot character are suppressed, but also that ') signals an error.
17:45:59
phoe
> No matter what the value of *read-suppress*, parentheses still continue to delimit and construct lists;
17:46:27
fiddlerwoaroof
So, I guess the question is can the reader handle the resulting READER-ERROR
17:46:59
phoe
but this still doesn't rule out e.g. having spaces in input that could be consumed by the reader macro
17:47:35
Bike
you could for example have a situation where cl-interpol may be used, so you have #?")" in read suppressed text
17:48:57
fiddlerwoaroof
I've had to fix code that does things like that inside read-suppressed code so I could load them
17:49:33
Bike
yeah my basic thought is that ansi tests should probably not test read suppress with unknown reader macros.
17:52:37
fiddlerwoaroof
From my tests, it looks like sbcl and abcl implement some sort of heuristic and ccl, ecl, lw and clisp all signal an error
17:55:19
Bike
so the "heuristic" is, i think, just that #? or whatever is ignored, so it just reads foobar
17:58:52
fiddlerwoaroof
I think given the specification of make-dispatch-macro-character, this should print something: (handler-bind ((reader-error (lambda (c) (print c)))) (read-from-string "#+(or) #?foobar 1"))
18:00:48
Bike
i don't think read suppress isn't allowed to put in a handler or do something equivalent, though