libera/#commonlisp - IRC Chatlog
Search
22:20:36
antonv
directly to some TLS port, or to the unencrypted NNTP port 119 and then upgrade with STARTTLS command?
23:05:07
aeth
Afaik, usocket is the most popular, but having used it, it has a few annoying limitations in TCP that in my IRC bot tests basically lead me to ping out eventually (over the course of days, usually, I think, iirc).
23:06:26
aeth
Unfortunately, one of the other popular ones depends on something called 'libfixposix' (you know, so it can run on Solaris and NetBSD and stuff), which complicates things by adding in a(n obscure) C dependency.
23:13:21
aeth
The problem I've had with usocket is that it relies on CL streams. http://www.lispworks.com/documentation/HyperSpec/Body/c_stream.htm
23:14:42
aeth
To use it on bytes (i.e. any arbitrary encoding that you then decode) you depend on LISTEN
23:15:03
aeth
"Returns true if there is a character immediately available from input-stream; otherwise, returns false. On a non-interactive input-stream, listen returns true except when at end of file[1]. If an end of file is encountered, listen returns false. listen is intended to be used when input-stream obtains characters from an interactive device such as a keyboard."
23:16:21
aeth
The problem being "If an end of file is encountered, listen returns false." This means you might have disconnected and usocket knows this and the stream is now an EOF, but you are still LISTENing as if you might have a connection still.
23:16:38
antonv
This should depend on the protocol. I don't know the IRC protocol, but most of the protocols define a way for communicating parties to know exactly how many bytes to read.
23:16:48
aeth
there is no READ-CHAR-NO-HANG equivalent for bytes, but if you need to do the text decoding on your end, you have to do this as bytes instead of chars
23:20:18
aeth
Because LISTEN doesn't work properly for TCP but usocket relies on it, I have to write my own timeout where I do a hanging READ-BYTE to check for EOF (unless there's another way? it's non-obvious), but every now and then there's some kind of concurrency issue which eats a byte, which will eventually result in a missed ping
23:21:18
aeth
"Reading from a stream which has been closed at the remote end signals an END-OF-FILE condition, meaning that reading from the stream and detecting that condition is the way to do it." https://usocket.common-lisp.dev/api-docs.shtml
23:21:59
aeth
But reading hangs unless it's READ-CHAR-NO-HANG or guarded by LISTEN, which doesn't check for END-OF-FILE
23:30:46
aeth
looks like usocket is supposedly the successor to trivial-sockets according to its documentation, though. https://usocket.common-lisp.dev/
23:33:14
fiddlerwoaroof
fsocket ( https://github.com/fjames86/fsocket ) is a bit more complete in some ways
23:34:05
fiddlerwoaroof
I've been writing some code that needs to send and receive on multicast sockets and, afaict, there's no way to do this even with the no-portable sb-bsd-sockets library
23:44:34
antonv
aeth: maybe you could chose some other approach, I don't know - a bit difficult to follow your explanation because don't know your task and maybe never done such task. Why do you need LISTEN? Are you trying some kind of async-io, where one thread handles multiple connections?
23:45:26
antonv
If you have a dedicated thread per connection, you could just use READ-CHAR, READ-SEQIEMCE, etc.
23:47:11
antonv
In this appraoch, the only problem to deal with, as far as I can see, is some stuck remote peer, who does not send anything, but keeps the connection open - your thread will remain occupied forever. This can be approached with read timeouts. Usocket docs advertise possibility to set read timeouts on sockets (SO_RCVTIMEO).
23:50:37
aeth
antonv: it's necessarily async to some degree because you can READ-BYTE and WRITE-BYTE
23:51:12
antonv
aeth: to convert bytes to characters, it's possible by wrapping the binary stream of the socket into a flexi-stream (you probably know that)
23:51:14
aeth
it's disappointing how something so simple as an IRC client isn't obviously easy to implement correctly in usocket
23:51:21
fe[nl]ix
aeth: the main reason libfixposix exists isn't to run on Solaris or NetBSD, it's to be able to wrap C macros
23:53:06
antonv
aeth: example irc bots that server this channel, as far as I understand: https://github.com/stassats/lisp-bots
23:56:12
antonv
I saw the word 'listen' mentioned in this code... so maybe you're right about LISTEN
23:58:01
aeth
it definitely does something. https://github.com/stassats/lisp-bots/blob/09bfce724afd20c91a08acde8816be6faf5f54b2/irc-bot/connection.lisp#L42-L56
0:02:36
aeth
no, it has to be this, I think. iomux:wait-until-fd-ready https://github.com/stassats/lisp-bots/blob/09bfce724afd20c91a08acde8816be6faf5f54b2/irc-bot/connection.lisp#L58
0:04:33
aeth
so usocket *is* insufficient, and it uses https://github.com/sionescu/iolib to work around the same issue I encountered.
0:05:20
char[m]
aeth: for a read-byte-no-hang that doesn't depend on listen, what about a flexi stream that is latin1 external format that is read-char-no-hang ed from then convert that latin1 char to byte?
0:05:40
aeth
it's workable on a server, perhaps, but for something distributed to end users, libfixposix makes things approximately 87,000,000x more complicated than having no foreign dependencies
0:06:21
aeth
char[m]: could work if usocket works like that... I think gilberth in #lispcafe suggested something similar
0:09:44
char[m]
aeth: it would still require the usocket stream to be bytes so that flexi stream can wrap it.
0:12:05
aeth
but there is no read-char-no-hang for bytes and any kind of timeout at this level of abstraction will eventually have a race condition that eats a PING (and thus doesn't PONG and thus times the bot out)
0:12:29
aeth
The central issue is simply that LISTEN is ambiguous between a hypothetical alternative of having, say, no-message-p and end-of-file-p which apparently makes usocket unusable alone.
0:13:16
aeth
At least if you need to be able to both read and write like IRC. Hanging a READ-BYTE in another thread is an easy solution if you don't afaik.
0:23:18
fe[nl]ix
I'm the author of iolib/libfixposix. if I provived an SBCL/x86_64 with libfixposix statically compiled, would you use it ?
0:25:03
fe[nl]ix
I need to choose a host compilation system, to avoid problems of unresolved symbols in glibc
0:31:25
aeth
All I know is that iolib as far as I know contains the only portable (how portable? Windows, too?) solution to detecting an IRC ping out without hanging used in code that I know of.
0:37:35
aeth
fe[nl]ix: my assumption is that it would have to go back 5 years? or would that be too stable?
0:37:48
yitzi
fe[nl]ix: Do you maintain bordeaux-threads? If so, I opened a PR to add atomic support for Clasp if have time to review sometime.
0:39:14
aeth
fe[nl]ix: Is it worth supporting Debian 8 (7 years) instead of Deiban 9 (5 years)? I'm assuming that's going to be the oldest.
1:17:20
fe[nl]ix
aeth: that seems too old. I'll compile it on a Ubuntu 18.04 LTS and you'll tell me if any users complains
2:46:18
nij-
Hi! A new line is printed whenever the main function is called. This gets annoying when the main function has to be called several times. However, I cannot locate where the printer is..
2:49:14
aeth
(defun foo () (print "hi")) (defun bar () (print "hello")) (defun main () (foo) (bar)) (trace main) (main)
2:53:40
Bike
honestlly, whenever i've run into this i just grep my sources for whatever's being printed.
3:32:33
aeth
nij-: trace what you think the midpoint call is, see if the newline shows before or after the midpoint
3:48:43
doulos05
I cannot figure out how to install it. I've tried `yum install sbcl`, but it didn't work.
3:52:17
doulos05
beach: Yeah, I was just hoping to be able to use a package manager to handle keeping it up to date
3:52:47
doulos05
But since it isn't really wanting to work, I'm just going to curl the binary onto the server
3:53:17
doulos05
If I figure out how to get it via dnf or yum, I can always just rm the binary and that should work
3:58:14
aeth
that's probably the modern Red Hat approach and Oracle Linux is just Ctrl+C Ctrl+V RHEL so that would be the way to go, I think.
3:59:02
aeth
then you'd just take the smallest docker image (but through podman) that has the latest SBCL
3:59:11
aeth
I use https://hub.docker.com/r/daewok/lisp-devel in CI but that might be too heavyweight and it might not have the latest
4:01:02
mason
doulos05: sbcl is in EPEL 7: https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/s/sbcl-1.4.0-1.el7.x86_64.rpm
4:01:18
doulos05
I was developing a project in Django (only because I didn't have time to learn Hutchentoot before the deployment date) and despite using a venv, managed to get all my system packages into my virtual env
4:01:55
doulos05
Ment I had to spend about 20 minutes going through my requirements.txt nuking `thefuck` and other apps that I don't need on a server.
4:03:07
doulos05
mason: Yeah, I think I'll just do the podman setup. Part of the point of these servers is to figure out proper partitioning for services, and docker/pods is one of the more straightforward way to do that.
4:03:54
kakuhen
I would help maintain the Fedora package but I have no idea where to even begin with the package repos... that and I need to learn how RPMs work.
4:04:38
kakuhen
MacPorts is also fairly out of date (2.2.6 i believe?) but I won't comment on it to prevent venting frustration with the process of updating it.
4:04:51
mason
doulos05: Hrm, another option would be to download https://sourceforge.net/projects/sbcl/files/sbcl/2.2.8/sbcl-2.2.8-x86-64-linux-binary.tar.bz2/download and manage it with GNU Stow or something. Fewer moving parts than a container.
4:05:58
doulos05
mason: Thanks! The joy of learning on hardware you aren't using for anything important is that if I fuck it up, I can just nuke the instance and start over.
4:06:21
aeth
Fedora has clisp, ecl, and sbcl, but no ccl. Only the sbcl seems particularly out of date. The clisp is a random one from git rather than the last stable version (from July 2010).
4:07:08
kakuhen
I know this because I recently modified the MacPorts package to pull source code from that commit