freenode/#slime - IRC Chatlog

7:49:27 Shinmera Doing something like (let ((string (make-string 1000))) (map-into string (lambda () (code-char (random #xFFFF)))))

7:49:50 Shinmera will very often result in an error

7:49:51 Shinmera Invalid protocol message: Error during string-to-utf8: Unable to encode character 28320 as :UTF-8.

7:50:25 Shinmera This is very annoying if the string occurs in a backtrace, as SLDB fails to launch at all

8:11:30 luis Shinmera: that is actually a valid character isn't it? Did you make up that number?

8:11:59 Shinmera I pasted the error.

8:12:38 Shinmera You can try the snippet yourself, it'll very likely produce a similar error with different code points.

8:13:10 Shinmera But yeah, all #xFFFF code-points are valid unicode.

8:14:22 Shinmera I stumbled on this because I'm implementing unicode algorithms and want to test my stuff against random garbage attacks. Debugging that without SLDB is... troublesome.

8:19:56 luis My memory is fuzzy but there are some non-characters but it's probably only CCL that's pedant about it.

8:21:25 Shinmera Hmm, ok, let me double-check then

8:22:24 luis (I ran into that when implementing UTF-8B support for Babel)

8:23:14 Shinmera Looks like D800 to DFFF are considered invalid code points indeed

8:23:46 luis But 28320 is not one of those is it?

8:24:02 luis (I'm on my phone. :-/)

8:24:35 Shinmera That's 6EA0

8:24:53 Shinmera Another one is 9FB0

8:27:59 luis Can you come up with a direct string-to-utf8 call that fails? Perhaps outside SLIME.

8:29:05 Shinmera Yeah, I'll try

8:30:27 Shinmera Hmm. I think the error message is misleading and points to a character close to an invalid one.

8:30:48 Shinmera Because if I explicitly ignore the D800-DFFF range in my random generator there's no more faults

8:32:53 luis That's... Evil error reporting.

8:33:36 Shinmera I can't seem to create an example myself though.

8:37:42 Shinmera Yeah, all the barfing strings do contain characters in that range

8:37:50 Shinmera But the char reported as invalid seems... random?

8:49:10 luis It might be the previous character, maybe?

8:49:34 Shinmera If I try to construct the test it reports the correct character as faulty

8:53:06 Shinmera I don't have time to debug this further. The behaviour is too confusing for me to know what to do at this point

8:54:00 Shinmera Ignoring the invalid range in my constructed examples is good enough for now, but it would be better if Slime either replaced or removed invalid characters when sending stuff over, since the current semi-crashing behaviour is far less than ideal.

8:54:58 luis Definitely. Can you open issue with the test case that does fail?

8:55:07 luis *an issue

8:55:10 Shinmera https://github.com/slime/slime/issues/527

8:58:01 luis What Lisp are you using?

8:58:04 Shinmera SBCL

8:58:10 Shinmera I pasted the exact steps to get my error

8:58:18 Shinmera https://github.com/slime/slime/issues/527#issuecomment-530733179

8:59:14 luis OK. Thanks.

8:59:23 Shinmera Hope that helps!