Search
Thursday, 20th of September 2018, 13:22:31 UTC
14:11:14
Shinmera
Biggest thorn in my side at the moment is the problem in lichat-tcp-server, where it somehow gets into a state that leaves connections on a pending close, and then after a while stops accepting new connections, causing any attemp to time out
14:16:48
Shinmera
It happens on the production server often enough, but reproducing it locally is probably gonna be tough
14:17:26
Shinmera
might also have something to do with the way that the production server runs in multi-mode with the ws-server, but I really don't know
16:06:18
phoe
Shinmera: okay, I'll try to take a look at it during the weekend
16:06:55
phoe
leaves connection on a pending close? what do you mean?
16:08:36
Shinmera
the connection is left in the CLOSE state
16:08:46
phoe
they look like a possible source of trouble there
16:10:19
Shinmera
Here's the script to run the server http://plaster.tymoon.eu/view/926#926
16:10:19
Colleen
plaster.tymoon.eu/view/926#... Website (XHTML), Title: - Plaster
16:11:03
Shinmera
let me remove the ldap stuff
16:11:26
phoe
bt:interrupt-thread also smells icky to me
16:16:27
Shinmera
And here's the client I use that seems to cause things to go awry in some cases http://plaster.tymoon.eu/view/926#927
16:16:27
Colleen
plaster.tymoon.eu/view/926#... Website (XHTML), Title: - Plaster
16:17:58
Shinmera
anyway, any improvemets at all (removing the gross locks, etc) would be welcome
16:18:20
phoe
I'm working on a very similar server right now, anyway
16:18:35
phoe
something event-driven that has a basic event loop and no gross locks like that
16:18:50
phoe
and no thread per connection
16:19:38
Shinmera
can't really do that here because I don't control the ws connection handling
16:20:20
Shinmera
lichat-ws-server uses hunchensocket
16:20:30
Shinmera
hunchentoot is threaded.
16:20:53
Shinmera
which would be another thing to do though: make the same tcp server also handle ws requests
16:21:01
Shinmera
though that would be pretty trick'
16:23:24
phoe
hunchensocket uses a single thread per connection?
16:23:40
Shinmera
well yea, hunchentoot does
16:25:50
phoe
https://edicl.github.io/hunchentoot/#taskmasters
16:25:51
Colleen
edicl.github.io/hunchentoot... Website (HTML), Title: Hunchentoot - The Common Lisp web server formerly known as TBNL
16:26:26
phoe
it should be possible to write a taskmaster that has a thread waiting for input on a list of connections and utilizes an lparallel kernel to deal with the requests
16:28:43
phoe
I guess nobody did it just yet though
16:29:03
phoe
This is more or less parallel to me writing my https://octo.sh/Gateway/Gateway/blob/master/doc/CONNECTOR.md
16:29:03
Colleen
octo.sh/Gateway/Gateway/blo... Website (HTML), Title: doc/CONNECTOR.md · master · Gateway / Gateway · GitLab
16:29:26
Shinmera
if I remember correctly there's a thing about hunchentoot forcing a close so you can't do it async
16:29:39
phoe
forcing a close? what do you mean?
16:30:08
phoe
oh, via unwind-protect of sorts?
16:30:14
Shinmera
it forces a close on exit from connection handling
16:30:20
phoe
so you have to do it in a lexical environment
16:30:45
phoe
(unwind-protect (handle-connection) (close-connection))
16:32:11
Shinmera
I ain't a fan of async and rewriting it all to be async instead of just debugging this seems like overkill
16:32:38
phoe
I'd focus on the locks first
16:32:46
Shinmera
and if you were going to rewrite it all then make a merged tcp/ws server that can do both
16:33:30
phoe
though I have no idea if interrupting threads is a good idea
16:34:04
Shinmera
interrupting threads is not great, no, but if you're not the thread to shut down there's often no other way
16:34:18
phoe
can't you pull the carpet that the thread is standing on?
16:34:30
phoe
and literally close the connection's socket while the thread is executing?
16:34:41
phoe
the thread will then *&#&@*&^[ and die in pain
16:34:46
phoe
which will achieve the goal
16:35:05
Shinmera
that would prevent a proper close handshake
16:35:31
phoe
I can't see a handshake there
16:35:39
phoe
https://github.com/Shirakumo/lichat-tcp-server/blob/master/server.lisp#L179
16:35:40
Colleen
github.com/Shirakumo/lichat... Website (HTML), Title: lichat-tcp-server/server.lisp at master · Shirakumo/lichat-tcp-server · GitHub
16:36:10
phoe
but again, I don't know that code too well
16:37:01
Shinmera
the unwind in lichat-serverlib causes it to try and send a close message
16:37:12
Shinmera
if I remember correctly
16:38:50
Shinmera
ech, I don't remember how this clusterfuck is supposed to work
16:39:00
phoe
hold the connection's lock, emit an exit message, handshake, close the socket
16:39:45
phoe
I'm kind of used to Erlang concurrency where everything is async and you can freely pass messages around
16:39:54
Shinmera
Colleen: look up lichat 4.3
16:39:54
Colleen
4.3 connection closure https://shirakumo.github.io/lichat-protocol#4.3_connection_closure
16:41:02
Shinmera
the lichat spec is one thing I'm as close to being proud of something as I ever got
16:41:23
phoe
I'll first try finishing my code
16:41:24
Shinmera
so when in doubt, check that instead of the shitty code
16:41:32
phoe
and then try looking into the spec to understand it
16:41:49
phoe
and then figuring out what's going on in this cluste^Wcode
16:42:31
Shinmera
it's a shitpile and you don't have to be afraid of calling it that
16:43:10
phoe
I'll get to know it better and then decide what to call it
16:43:23
phoe
so far I've enjoyed your code style
16:44:38
Shinmera
the spec should be pretty easy to read, nothing fancy
16:45:21
phoe
Where do you have the implementation code for the wire protocol?
16:45:52
Shinmera
click an to-wire, then on the [src] link
16:46:26
Shinmera
staple is nice like that :^)
16:47:33
phoe
I'll possibly shamelessly steal your wire protocol because it seems to be better than my safe-read library
16:50:14
Shinmera
the null-termination allowing skipping of malicious messages is a real nice touch that I still like
16:50:38
Shinmera
forces encoding of binary though, which may or may not be a problem
16:54:32
Shinmera
just noticed that this has a good use case for :allow-other-keys, heh
17:05:29
phoe
Shinmera: it seems that someone has already done that
17:05:31
phoe
https://quickref.common-lisp.net/quux-hunchentoot.html#Exported-classes
17:05:31
Colleen
quickref.common-lisp.net/qu... Website (HTML), Title: The quux-hunchentoot Reference Manual
17:05:36
phoe
look at thread-pooling-taskmaster
17:06:02
phoe
you should be able to spawn the Hunchentoot acceptor and specify this as the taskmaster
17:06:13
Shinmera
man quickref is maddening to browse
17:06:35
Shinmera
where the fuk is the repo link
17:06:45
phoe
https://gitlab.common-lisp.net/qitab/quux-hunchentoot/blob/master/thread-pooling.lisp
17:06:45
Colleen
gitlab.common-lisp.net/qita... Website (HTML), Title: thread-pooling.lisp · master · qitab / quux-hunchentoot · GitLab
Friday, 21st of September 2018, 1:22:31 UTC