freenode/#lisp - IRC Chatlog
Search
23:05:27
pjb
aeth: #+genera (defmacro lambda ((&rest bindings) &body body) `(cl:function (cl:lambda ,bindings ,@body)))
23:15:34
aeth
pjb: I think it was Genera. I'm not sure because it has been a few years. I think the proper way of doing it, though, is pushing it to *features* iirc.
1:04:00
slyrus1
scymtym: if you've got a moment, can you take a look at https://github.com/sharplispers/closure-common/pull/2 ? thanks!
3:12:30
kenster
Streaming some epic lisp and C++ programming: https://youtu.be/lEL0Nly2CE0 or https://www.twitch.tv/kingherring
3:13:27
asarch
If I have a git directory, how could I know the full address of it on the GitHub server?
3:14:00
Colleen
Bike: karlosz said 20 hours, 59 minutes ago: I found a possible cause. Code like this (let ((*x* (or ... ...)))) can introduce a lexical location that is defined in two branches. Thus, we have our no dominating definer case we were looking for. I generalized the code to handle placing it in the lowest common ancestor node in the dominance tree.
3:14:01
Colleen
Bike: karlosz said 20 hours, 57 minutes ago: i meant that the let can introduce a lexical location which can be captured and has no dominating definer.
3:39:33
ym
There is a paper by Jogn Alan McDonald called "Object-oriented programming for linear algebra." and dated July 10, 1989. It tells about "a system for numerical linear algebra and optimization, implemented in Common Lisp[l3] and CLOS[l], called Cactus.". Cannot google this Cactus system location in the Internets.
3:41:44
asarch
One stupid question, does Scheme have an ANSI standard just like Common Lisp has (ANSI X3J13)?
3:53:13
beach
asarch: There are several standards organizations in the world. Each country typically has at least one. Then you have IEEE, ISO, ECMA, etc.
3:56:26
beach
asarch: It is important to understand that a standards organization is not necessarily appointed by anyone in particular. It can be an association, or it can be a for-profit company. The important role of a standards organization, as far as programming languages are concerned, is that it is independent of the organizations that provide the implementations.
4:05:50
beach
Sure. It's an important concept that is, unfortunately, often unknown to software developers. In fact, it is often unknown even to project leaders whose task it is to choose a programming language. The result is that they often choose programming languages that do not have an independent standard, with potentially dire consequences to the projects they are leaders of.
4:08:13
drmeister
If an array of unsigned byte8's only contains values between 0x01 and 0x7f inclusive - then it can be stored in a simple-base-string - correct?
4:10:44
drmeister
I thought that might be the answer - so I asked. What about printable ascii characters? 0x20 to 0x7f?
4:12:51
drmeister
Right - it says nothing about the encoding of characters. I need to decide if an array of bytes can be coerced into a simple-base-string.
4:13:52
drmeister
I could coerce those to arrays of bytes - I was looking for some other way of testing rather than failing utf-8 encoding.
4:14:08
White_Flame
" Whether a character is a base character depends on the way that an implementation represents strings, and not any other properties of the implementation or the host operating system."
4:14:27
beach
As far as I know, a UTF-8 string with bytes only between 0 and 127 contains only ASCII characters.
4:15:16
drmeister
I'm working on code that receives messages from the pzmq library - they are foreign-data. The foreign-data is being run through: (cffi:foreign-string-to-lisp (pzmq:msg-data msg) :count (pzmq:msg-size msg) :encoding cffi:*default-foreign-encoding*) - occasionally it fails.
4:15:46
beach
drmeister: I don't understand why you need to coerce an array of bytes into an array of bytes.
4:16:36
drmeister
I need to coerce the foreign data into strings when they can be and coerce them into arrays of bytes when they cannot be.
4:19:09
drmeister
When the cl-jupyter library reads a message using the pzmq library the message contains a part called the identity followed by the message.
4:19:54
drmeister
The very first message that is read has a pzmq supplied identity - which is a 5-byte array. Every other identity is a text uuid.
4:20:36
drmeister
The cl-jupyter library doesn't seem to be aware of this and tries to coerce every part of every message (including identities) into simple-base-string's
4:22:14
drmeister
I believe that the only reason the cl-jupyter library has worked for the last two years is some fluke of the communication protocol that allows it to sync up with the server after many failed messages have gone back and forth. I believe it all comes down to this 5 byte array.
4:23:10
drmeister
So - to deal with it - I need to recognize when a foreign-data array of bytes can not be encoded as a simple-base-string and in that situation - encode it as an array of bytes.
4:24:08
drmeister
I think I will apply the #'graphic-char-p predicate to every byte - and if they all pass - then I will convert it to a simple-base-string.
4:26:09
White_Flame
isn't this just a simple state machine, where the first message is binary and the rest are text?
4:27:33
drmeister
The first part of the first message is binary and all parts after that are text - as far as I understand.
4:28:58
drmeister
I'm not sure what I said is correct - I'm not sure if the identity should be considered part of the message.
4:29:28
drmeister
I'm still feeling my way around here. What I have figured out is if you treat all parts of all messages as strings - there is trouble.
6:36:00
phoe
These two aren't disjoint. There's binary data that also decodes to a simple-base-string.
6:39:04
drmeister
In every message except for the very first one - the identity(ies) is a uuid string.
6:39:29
drmeister
In the very first message - the identity isn't provided by the jupyter notebook framework. Ident = None
6:40:52
drmeister
https://github.com/jupyter/jupyter_client/blob/master/jupyter_client/session.py#L598
6:42:11
phoe
I'm looking at https://github.com/jupyter/jupyter_client/blob/master/jupyter_client/session.py now to see if the implementation shows anything to me.
6:42:14
drmeister
As a historical note, ZeroMQ v2.2 and earlier use UUIDs as identities. ZeroMQ v3.0 and later generate a 5 byte identity by default (0 + a random 32bit integer). There's some impact on network performance, but only when you use multiple proxy hops, which is rare. Mostly the change was to simplify building libzmq by removing the dependency on a UUID library.
6:43:29
drmeister
The very first message I'm getting has a 5-byte identity that can not be translated into a simple-base-string using the UTF8 encoder - in cl-jupyter it errors out saying this.
6:44:31
drmeister
cl-jupyter has been generating this error for two years - Fredrick Peschanski apologized to me about it two years ago when I started using cl-jupyter. I don't understand why cl-jupyter worked at all - but I suspect it was some fluke of the protocol.
6:45:06
phoe
I don't understand how the identities are generated and then how they are told apart after they are sent.
6:45:35
phoe
The whole message looks like [ident1, ident2, ..., DELIM, HMAC, p_header, p_parent, p_metadata, p_content, buffer1, buffer2, ...]
6:46:14
drmeister
A couple of weeks ago it stopped working. I've been debugging this for three days and it's pretty clear that because this first identity can't be encoded into a simple-base-string that the return message from the kernal is missing the identity and so zeromq is dropping it.
6:47:03
drmeister
Yes - the identities come before the DELIMiter. The DELIMiter is the string "<IDS|MSG>"
6:48:21
drmeister
So - if I were to store anything before the DELIM as (array (unsigned 8)) and pass that through to the return messages - everything would (hopefully) be fine.
6:48:50
drmeister
Most of the time though - the identity(ies) are text UUID's - so it's fine to store them in simple-base-strings.
6:49:28
phoe
"As a historical note, ZeroMQ v2.2 and earlier use UUIDs as identities. ZeroMQ v3.0 and later generate a 5 byte identity by default (0 + a random 32bit integer)."
6:49:38
drmeister
The original cl-jupyter code converted every message part as a string - I think that was the problem.
6:51:45
drmeister
Note how he uses (cffi:foreign-string-to-lisp (pzmq:msg-data msg) :count (pzmq:msg-size msg) :encoding encoding) to try to convert the foreign-string into a lisp string with the default encoding (UTF-8) and then if it fails it prints a warning and uses LATIN-1 and says it's ugly.
6:52:03
phoe
I don't know the full context, but now it smells to me that there was a type assumption that these identities are text.
6:52:16
drmeister
It's more than ugly - it's mangling the identity and then I can't send a response using the mangled identity because zeromq drops the message.
6:52:46
phoe
The first byte of zmq3.0 identity is #x00 - that's not a textual character in any charset I know.
6:53:45
drmeister
I am very grateful for your feedback - I am not at all convinced that I'm on the right track.
6:55:20
Shinmera
The standard's charset is pretty irrelevant to anything we do nowadays and have been doing for the past twenty years
6:56:28
phoe
Theoretically, (flex:octets-to-string #(0)) doesn't error; in practice, I'd freak out seeing the resulting "^@" anywhere in my program.
6:57:08
drmeister
The tricky thing here is that zeromq switched to using these 5-byte 0x0 W X Y Z (where W X Y Z are 4 bytes of an int) identities when the user doesn't provide a UUID identity. UUID identities can be coerced to simple-base-string. 0x0 W X Y Z can not be - so I thought I would coerce it to an (array (unsigned 8))
6:58:26
drmeister
But I would only coerce it to an (array (unsigned-byte 8)) if it contains any characters for which graphic-char-p is NIL.
7:00:05
drmeister
Because the cl-jupyter code needs to work with simple-base-strings for everything that isn't this one identity.
7:00:40
drmeister
I don't think it ever looks into the contents of identities - it only passes them around and uses them to generate responses for requests
7:00:45
phoe
Then it seems that cl-jupyter was based on the false assumption that identities are strings.
7:01:20
phoe
If it's just passing them around, then it doesn't seem that there are so many call sites that would need to be edited to make it work with arrays.
7:02:32
phoe
But, in their case, you could completely throw out the try-to-coerce-into-string-or-whatever functionality and go for a much simpler one: read five bytes, write five bytes.
7:04:12
drmeister
I think the big problem is that zeromq switched to these binary identities by default when previously they used uuid's
7:06:34
drmeister
I'm a lot confused by this. These components of the wire protocol are converted to JSON.
7:08:25
phoe
From Python docs: `Bytes literals are always prefixed with 'b' or 'B'; they produce an instance of the bytes type instead of the str type. They may only contain ASCII characters; bytes with a numeric value of 128 or greater must be expressed with escapes.`
7:08:59
phoe
So basically, 'asdf' in Python is "asdf" in Lisp, but b'asdf' in Python is (flex:string-to-octets "asdf") in Lisp.
7:09:21
Shinmera
Or, if your implementation supports it, a static-vector with element-type base-char.
7:12:05
Shinmera
Or, if you're SBCL, (sb-sys:with-pinned-objects (string) (.. (sb-sys:vector-sap string)))
7:12:20
Shinmera
drmeister: static-vectors is a library that implements lisp vectors for which you can retrieve a pointer that you can pass to C
7:12:24
phoe
drmeister: there's a library called STATIC-VECTORS that allocates Lisp vectors suitable for CFFI usage.
7:13:44
phoe
Basically, you get a chunk of memory that is guaranteed to act the same when read/written from Lisp (via Lisp primitives) and read/written from C (via pointers). It gets funny when the memory is allocated as a base-char vector, because in C you get bytes, but in Lisp you get base-chars.
7:15:01
drmeister
Rather, I started using cffi because pzmq only accepts lisp strings and foreign data objects.
7:17:15
drmeister
The wire protocol dictates python byte arrays are used. So I send and receive (array (unsigned 8)) arrays.
7:18:14
drmeister
Then I pass those parts to wire-deserialize - and it converts what it needs to into lisp strings.
7:18:55
drmeister
Later I pass dictionaries containing lisp strings to wire-serialize - that converts the strings into (array (unsigned 8)) arrays and then I send those back to zeromq.
7:20:15
drmeister
Identities are not encoded by wire-serialize or wire-deserialize - they are only prefixed or removed from the start of the message.
7:20:53
phoe
Identities are basically tokens that you receive and send but don't anyhow interact with, right?
7:24:34
drmeister
I think my current approach will work - but it's weird because for one thing - it encodes strings immediately when things come in to recv -
7:30:03
phoe
drmeister: that process is weird, yes. Instantly encoding everything into strings is weird, especially that these things aren't strings.
7:30:58
unanimousarc
I have a Q, I'm going through Land of Lisp and there's this part where the lisp reader creates an instance of a structure directly from the printed representation: "(defparameter *that-guy* #S(person :name "Bob" :age 35 :waist-size 32 :favorite-color "blue"))" However, I get an error when I do this in SBCL
7:36:22
unanimousarc
I wrapped the defstruct but that doesn't seem to change it, should I wrap everything in the eval-when?
7:38:43
beach
phoe: So why does it work when he evaluates it? And why does the error mention "dump"?
7:39:27
trittweiler
Yes but it's still what beach hints at. In the case of #S(...) the person struct will be created at read time, and the object will have to be dumped into the fasl, to be bound to *that-guy* at load time
7:43:35
trittweiler
unanimousarc, Yeah, there's a way around that involving make-load-form but it's really quite advanced,I would suggest you to just go with MAKE-PERSON instead of the #S(...) notation. Note that the #S(...) notation would also create a read-only object.
7:45:26
phoe
There are several "times" in Lisp that are basically reader/compiler/etc. passes from raw text to actual Lisp code and data.
7:46:07
phoe
The file compiler's role is to grab Lisp forms, turn them into code and dump all of the resulting code into compiled files.
7:46:56
phoe
#S(person ...), when it's read, actually results in a Lisp *instance*. The reader turns it into an instance of PERSON.
7:47:42
phoe
The file compiler does just fine with lists, code, functions, and so on. But it chokes on actual instances of objects, for a good reason.
7:47:57
unanimousarc
hm, I'll have to read more about this compilation stuff eventually, for now I'll just treat lisp like an interpreter
8:28:19
no-defun-allowed
AeroNotix: [here is cl-decentralise in its god awful hackish form](https://gitlab.com/Theemacsshibe/cl-decentralise)
9:53:28
phoe
AeroNotix: you should make it possible to declare what the delimiter is. It should be possible to customize the delimiter, just like in HTTP content-type multipart/form-data. It has a custom boundary.
10:20:16
beach
v0|d: Because most modern garbage collectors don't touch objects that are no longer live. Such collectors could not call any destructor for dead objects.
10:21:30
beach
v0|d: Some Common Lisp implementation have what is known as "finalizers" that can accomplish some of what a destructor does. However, it is still not predictable when the garbage collector will detect that an object is no longer live.
10:21:32
Zhivago
The more important point is that by the time the object is collected it is no-longer reachable by a destructor. If it were, it could be re-linked into the graph by the destructor. Leading to things like resurrecting objects from the dead (once only) in Java.
10:22:55
Zhivago
Finalizers get around this by not operating on the dead object, but usually share some (still live) substructure.
10:24:57
heisig
v0|d: You could, however, implement your own generic function (destroy-instance X), which would internally do a (change-class X 'destroyed). Then CLOS would have destructors :) Of course this would be unrelated to the GC.
10:34:06
phoe
the basic use case for a destructor is freeing resources - Lisp has garbage collection instead.
10:34:27
phoe
another basic use case is closing closeable resources when they are no longer used - Lisp has WITH-* macros for that.