freenode/lisp - IRC Chatlog
Search
20:24:42
Xach
I really wanted something so simple that everything you see on screen is easy to map back to its line of code.
20:25:40
rumbler31
does anyone know why apis that expose udp sockets don't give the application a stream to read, instead insist on returning vectors, and even then, only the amount of the packet that you asked for while discarding the rest?
20:26:47
slark
what do you think as a rule of thumb to use (gensym) for all macro parameters to avoid one of the leak describe in chap 8 of PCL ?
20:28:46
Bike
you should use gensym for any bindings you generate. you might not use a macro parameter that way.
20:28:48
slark
rumbler31: not sure what you want to mean, but as udp is not a reliable way to exchange information this is probably the reason
20:30:52
Shinmera
rumbler31: Because that's how UDP works. It gives you messages, not a continuous stream.
20:31:12
rumbler31
usocket, and I think ccl's socket api does the same. at least from what I'm seeing, repeat calls to socket-receive on a udp socket give me the beginning of new packets. If the whole packet is available somewhere, just queue them up. at the lowest level I'm sure that there is some vector backing a tcp socket, its just that the kernel will give you the vector in the right order
20:31:46
rumbler31
but is there really a use case for wanting part of the data in a packet and discarding the rest, as provided for by the api?
20:32:16
rumbler31
yes. but streams are an abstraction. the underlying data is still a vector even if thats not exposed to the user
20:32:49
Shinmera
UDP does not give you a stream, it's as simple as that. UDP gives you messages that may or may not arrive and they may arrive in any order.
20:33:06
Xach
tcp sequences those vectors and delivers them to the user level in a guaranteed order (or it doesn't give you any data at all)
20:33:56
Xach
ACTION always liked the "Interconnections" terminology of "best effort" rather than "unreliable"
20:34:16
rumbler31
yes, but if I go to receive from a udp socket, a whole packet was received, and unless I ask for the entire thing, the next time I call receive will return the next new vector, not what was left over and "not asked for" from the first vector
20:35:04
Bike
slark: you want to use gensym when a macroexpansion establishes some bindings. like, if the macro expands into (let ((foo ...)) ...), and foo is not supposed to be used by the user, it should be a gensym.
20:36:26
_death
rumbler31: udp has no packets, but datagrams.. it has no connection, so it is stateless, "fire and forget"
20:37:11
slark
rumbler31: you can still create your own protocol based on udp but it is overkill as tcp already exists and it will be still not reliable cause you can just works on the "application" layer
20:37:30
rumbler31
a datagram doesn't span a packet (afaik correct me if i'm wrong) so I consider datagram equivalent to a network packet
20:38:06
_death
rumbler31: tcp is a much more heavyweight protocol that creates a "virtual circuit", i.e. a connection between endpoints
20:38:31
slark
rumbler31: well i think you can say this, a network packet which doesnt care of his predecessor or the sucessor
20:39:04
Shinmera
rumbler31: I'm not sure what part you don't get anymore. Are you asking about the length argument to socket-receive?
20:39:29
rumbler31
I understand that udp has no guarantees. at the moment i'm wondering why, at the api level, I have to provide an expected size, which if there is more data than this gets dropped on the floor instead of being available for the next call for "more data"
20:40:03
rumbler31
Shinmera: and further why if there is more data left than this max size that the api drops it on the next receive call
20:41:56
Shinmera
As for why there's a length, your buffer might be less than the udp max size, so you can fill only what you need.
20:42:08
rumbler31
is it the case that if I haven't called socket-receive, that datagrams will get dropped on the floor, or will datagrams be queued (until some max os buffer size), st that socket-receive *will* return all datagrams seen by the interface to this point in time?
20:42:43
Shinmera
datagrams that you do receive should be queued by your OS, but there's probably parameters in your OS that control this behaviour.
20:43:24
rumbler31
Shinmera: so if you fill only what you need, what I'm trying to figure out is what kind of protocol would benefit from the default usage of "drop the rest of the datagram on the floor" vs "queue the rest of the datagram for a subsequent read"
20:43:24
_death
rumbler31: there is a queue, but you cannot rely on receving all datagrams, on receiving them as sent, on receiving them not being duplicated..
20:44:03
Shinmera
rumbler31: a datagram encompasses a "message". If you don't need the max size of the udp datagram for your messages you can read less than that.
20:46:13
_death
rumbler31: like I said, it is a stateless protocol.. so why should a udp implementation keep state for partial datagrams
20:48:31
rumbler31
if the datagrams arrive out of order, I will still need access to the whole packet, because I can not simply peek the datagram. If I peek it and I don't need it, great. but if I peek it and I need it, then I lost the rest of the data
20:50:20
Shinmera
UDP just gives you a thing to copy contents out of a buffer. Why is this so difficult.
20:50:55
rumbler31
what purpose is there, because there must be some, for a protocol that only cares about part of a datagram?
20:51:46
_death
rumbler31: that's the way it goes with udp, you tend to need more guarantees and you pay for what you need by building it yourself.. at some point it may cross the threshold where tcp may make sense
20:52:10
Shinmera
rumbler31: The length is there for the length of /your/ buffer, not the length of the datagram.
20:52:52
Shinmera
Right, so if you know your messages are only going to be 10 bytes, you make a 10 byte buffer and say the length is 10
20:52:59
Bike
your problem is that you don't see why the OS doesn't have a storage for packets inbetween receive calls? right?
20:54:10
Shinmera
Or you make a buffer that's more than 10 and store multiple things in it by still specifying the length as 10.
20:55:09
rumbler31
Bike: yes. although it probably doesn't have to be the os. Shinmera: yes, although really the datagram could be any size I suppose, or so i've been told.
20:56:06
Bike
so you could have a protocol where all messages are going to be ten bytes. and then you can save a couple picoseconds not bothering with partial reads and such
20:56:40
rumbler31
but thats up to the network, I might have sent 1000 bytes but the network could have made them arrive as 2 500 byte datagrams. and at that point there is no problem, I just read 500 expected more, read more
20:58:14
Shinmera
It gives you messages which have an upper bound in size. You send a message and you might get it at the other end.
20:58:40
rumbler31
I know it doesn't implement a stream. but If I write the max datagram size, I do not receive 1 64k packet, I receive 1 MTU sized packet
21:00:30
Shinmera
And the message you get is also internally consistent, so the UDP message itself will be properly consecutive. Individual UDP messages may be reordered though.
21:02:08
rumbler31
that part I should have known, but thank you for clearing it up. but even with this in mind, would anyone feel constrained with an api that presents the byte vectors from udp datagrams received on the interface so far as a stream?
21:02:56
Shinmera
You already have a full message there. There's no point in turning it into a stream.
21:06:19
rumbler31
but I guess, if I find that i've ended up with datagrams out of order and thats important, a stream interface alone would not tell me how much of the following bytes were received in the datagram
21:08:38
rumbler31
whereas if I have to keep the individual messages around, I know that when I start examining the buffer and its not what I need I can throw it out
21:09:47
rumbler31
and that the udp datagram reception call, if a stream is returned, its only temporary because I'd have to read from a new stream on a new call
21:46:44
MrSleepy
Hello do people generally do graphics and gui programming in lisp using some sort of foreign function interface, or is there a more native approach to that sort of thing?
21:49:52
MrSleepy
Well generally just drawing to the screen. I am totally not very knowledgable about it graphics in general. My main experience is through tcl/tk and Qt in C++ but that isn't really drawing in the low level sense.
21:55:28
MrSleepy
phoe_, I just checked it it seems really useful, I also peeked at #lispgames and some of the resources in their motd seem useful too
21:59:09
z3t0
MrSleepy: For advanced things look at cl-opengl and lispbuilder-sdl and for a more high level interface look at sketch
22:51:02
z3t0
I have a large library that I am trying to profile and find the bottleneck in but it has a lot of functions
6:16:54
z3t0
stylewarning: honestly I only just began commmon lisp and the only issues i have had are libraries or lack there of for many "common" application uses such as graphics, templating etc
6:17:34
aeth
If you don't think there are many libraries for graphics, you'll be disappointed for literally everything else except maybe web stuff.
6:57:40
stylewarning
People use it commercially for anything from GUIs to embedded systems to backend infrastructure
7:09:02
stylewarning
And, unless I've missed something big, there are no usable libraries for it that cover it anywhere near comprehensively.
7:10:40
stylewarning
It's nice that there's a cliki page, but attempt to use any of the libraries to, say, do the utterly trivial thing of computing the conjugate transpose of a complex float matrix.
7:14:10
stylewarning
The problem will be one of the following: complex numbers not supported, complex numbers broken with FFI, library-included F77 LAPACK is patched or out-of-date, library won't load at all, source code is undocumented and inspecting the source requires you to untangle 11 layers of macros and a home-rolled re-implementation of generic functions
7:19:23
stylewarning
This is just to transpose a matrix and conjugate it's entries. Of course, rolling your own on a 2D array is <= 5 lines of code. But unfortunately the next level of basic things like numerically stable SVD are orders of magnitude more difficult to implement, and almost surely a bad idea.
7:22:54
stylewarning
This sort of project is of course possible to implement in Lisp. But it requires careful programming, excellent interface design skills, a good understanding of compilers and runtimes, and a no-nonsense approach to generalization.
7:23:36
stylewarning
shka: If we—the collective of Lisp programmers—can't even steal LAPACK, there's no hope to stealing SciPy.
7:25:05
stylewarning
shka: I am a little bitter, because this makes my day job harder. And it has been an impediment in 3 professional settings.
7:26:19
Shinmera
A lot of people that are drawn to CL are individualists and avoid anything that would force them to do what they don't want to.
7:28:56
Shinmera
I don't really like it when people talk about what "we" need, or need to do. If you want people to be interested in what you want, do it, and do it well enough to get them interested.
7:30:29
beach
Shinmera: I totally agree about disliking opinions about what "we" need or need to do.
7:30:30
Shinmera
I for one am not a strong enough person to be able to juggle community management alongside coding, so I just don't bother and just write what I want. I don't have the energy to try and convince people to help me out.
7:32:47
Shinmera
Handle input, manage assets, get you a GL context, load collada models, present a shader pipeline, do hierarchical shader composition, and other stuff I forget.
7:33:39
Shinmera
Here's a 4 hour in-depth stream about the engine internals. https://www.youtube.com/watch?v=0v6iv_3BLeo&index=8&list=PLkDl6Irujx9Poirb63aATXXWHCVCHwlQA
7:33:56
Shinmera
The playlist also has some gamedev streams in it, which show a more practical approach.
7:35:09
Shinmera
On that note, this weekend there's going to be another Ludum Dare contest and I'll be streaming during that as much as possible.
7:37:11
stylewarning
If anyone is actually interested in some Lisp linear algebra writing, let me know, you can possibly get paid to do it. (Likewise with SBCL development.)
7:43:02
Shinmera
z3t0: As of right now I can't advise the use of Trial for anything but the development of itself. I can't guarantee the stability of any of its components due to the experimental nature of the project.
7:45:35
Shinmera
Most people use git & some hoster for vcs. There's a brazillion testing frameworks out there and more are being made by the day (probably).
7:46:53
Shinmera
But anyhow, vcs and ci are only really useful when your software is already stable.
7:49:08
stylewarning
Reducing interactive features of your software helps stability. Reducing "options at a distance" helps. Reducing generality. Not relying on undocumented APIs. Staying portable or de facto portable.
7:49:54
Shinmera
Documenting your own stuff and getting others to try it also helps tremendously, I find.
7:51:51
Shinmera
Other people are much more useful because they don't have the same cognitive predisposition as you do