libera/#commonlisp - IRC Chatlog

3:52:18 ns12 White_Flame: Most sb-posix functions do not seem to be documented in the SBCL manual http://www.sbcl.org/manual/

3:52:40 White_Flame right, check their docstrings & refer to posix knowledge

3:53:04 ns12 I guess the full list can only be found in the source code https://github.com/sbcl/sbcl/blob/sbcl-2.2.1/contrib/sb-posix/interface.lisp

3:54:16 ns12 Does sb-posix work on Windows?

3:54:38 White_Flame inasmuch that windows has APIs for posixy things, probably

3:57:19 ns12 Is see that there is #'sb-posix:fork but no exec()?

3:58:55 White_Flame I don't think that would work very well with the lisp image

3:59:10 White_Flame why would you fork/exec instead of just launch?

4:00:28 White_Flame I guess open FDs would be a thing if you want to share them with pipes or whatever

4:02:31 beach Good morning everyone!

4:03:44 ns12 White_Flame: It is also missing select(), poll(), and signal handling functions.

4:03:46 ns12 Oh well.

4:03:54 ns12 Good morning beach.

4:04:01 White_Flame there are other libs specifically for that functionality

4:04:43 White_Flame and signal handling should not be done with raw posix calls from inside lisp. Use the implementation's wrappers for them

4:04:54 White_Flame because you'll probably mung the image's use of them

4:06:08 ns12 There's trivial-signal for signals https://github.com/guicho271828/trivial-signal

4:06:08 ns12 But how do I get poll() and select()?

4:06:20 White_Flame look in cliki for libraries

4:07:32 White_Flame I know usockets uses select() in its backend

4:07:40 White_Flame as one example to poke into

4:09:24 White_Flame I do wonder what you're doing, though, as many of these things should more be viewed in terms of the functionality you're going for, not the specific posix api

4:11:36 beach Maybe ns12 is writing a Unix shell.

4:11:59 ns12 White_Flame: I have no particular project. I have been learning about the POSIX API, and I am evaluating Common Lisp's existing ability in this area.

4:12:45 beach ns12: It is not about the language Common Lisp.

4:14:05 ns12 beach: Is it off-topic here?

4:14:51 beach No.

4:15:03 beach But you said you are evaluating Common Lisp.

4:17:40 beach Maybe I should mention this embryonic idea for POSIX protocols: http://metamodular.com/POSIX-API/

4:17:46 ns12 Yes, the Common Lisp ecosystem. Not the language itself.

4:31:48 Guest74 how many implementations have posix functions? is there much difference between the implementations? I figure everybody would stay pretty close to the c definitions.

4:35:14 ns12 Guest74: "how many implementations have posix functions?" - From my observations, it is not a priority for many Common Lisp implementations.

4:35:40 beach Guest74: It gets tricky when the C API uses C structs.

4:36:11 ns12 There's osicat https://osicat.common-lisp.dev/manual/osicat.html but it provides limited features.

4:36:45 White_Flame and uiop has its own limited set to support what asdf needs

4:37:09 Guest74 beach: i've been thinking about that, and maybe just treating structs as arrays.  It's the pointers that make things tricky.

4:37:17 White_Flame (mostly around the filesystem)

4:37:51 White_Flame also the actual C memory layout needs to be consistent, and I don't know how packing plays into that

4:38:04 Guest74 ns12:  isn't osicat ffi?  I was wondering about implementations.

4:38:07 White_Flame eg if there's a mix of 32-bit and 64-bit fields

4:39:05 Guest74 just treat everything as arrays of ub8 with accessing functions that assemble the bits.

4:39:32 Guest74 Something I'm toying with for writing reader/writers for binary formats.

4:39:50 White_Flame and now you have to deal with endianness ;)

4:40:09 Guest74 just call a different accessor.

4:40:28 White_Flame but yeah, I did write a big declarative spec for binary file formats

4:40:52 White_Flame I only ever implemented the reader, though. Can read an entire java .class file into list-based structures

4:41:06 Guest74 any links? always curious how others attack things.

4:41:31 ns12 Is there any Common Lisp implementation that provides a larger POSIX API than what SBCL has?

4:41:31 ns12 I am used to the POSIX functionality provided out of the box in OCaml (https://ocaml.org/api/Unix.html) and in Standard ML (https://smlfamily.github.io/Basis/posix.html).

4:42:01 beach Guest74: There is at least one library that deals with bits in vectors like that.

4:43:01 beach But I don't think that's the protocol you would want to expose to Common Lisp application code.

4:43:05 White_Flame Guest74: https://white-flame.com/classfile.lisp

4:43:18 White_Flame the implementation is a mudball as I was adding features

4:43:30 Guest74 I'm sure there's a few.  I've taken to using nibbles for the reading/writing that everything deconstructs to.

4:43:43 White_Flame but it's the spec (and knowledge that it is unambiguously implementable) that's the important part

4:44:48 ns12 Does LispWorks and Allegro have comprehensive POSIX APIs?

4:45:01 ns12 s/Does/Do/

4:49:58 Guest74 white_flame: I hope to have some that reads more like common lisp with a small precise specification.

4:50:33 White_Flame well, this is a declarative data specification. the lisp stuff then uses it to interpret how to pull in a file into a tree of values

4:50:54 White_Flame and take a tree of values and write it into bytes

4:51:24 White_Flame because all sorts of file formats do all sorts of crazy things, it needs a very comprehensive library

4:51:33 White_Flame I didn't want a bunch of black-box imperative code defining things

4:51:52 White_Flame s/library/vocabulary/

4:52:47 Guest74 I'm thinking I don't want to support stupid things.

4:53:24 Guest74 Many years ago I thought I would write a buffer replacement for clx to work with modern extensions.  but it's just too stupid.

4:53:36 beach Guest74: We seem to be talking about different aspects. I think as an implementation technique, stuffing things in a Common Lisp vector might work. I am more interested in what data structures will be presented to the Common Lisp application programmer. That's what my page on that link is about.

4:54:10 beach ... and that's why I mentioned that structs are tricky.

4:56:09 White_Flame ns12: ocaml probably fits the C process structure more cleanly?

4:56:21 White_Flame lisp really grew big _as_ an OS

4:56:38 Guest74 beach: for my current work on ioctls I autogenerate a lisp struct with the same fields, and also generate a reader for that struct that populates the c struct. I find it a bit tedious. I also have the option for the lisp function to take an alist of slots and values which populates the c structs/arrays for the ioctl.

4:57:04 Guest74 s/reader/reader & writer/

4:57:51 Guest74 I'm guessing lisp function built upon those autogenerated structs/readers/writers will just take keywords?

5:00:19 Guest74 though I think I need still allow to pass lisp structs(and maybe even c structs?) for things where it makes sense. e.g. force feedback effects for controllers are nicely contained within c-structs, and having a library of them you could send to the device seems like something useful.

5:03:22 White_Flame it wouldn't seem like the end of the word to marshall between a reused lisp struct and a reused C struct

5:03:34 Josh_2 Good morning

5:03:41 White_Flame but I get that some of them can get quite large

5:03:54 White_Flame and of course that wouldn't work if they're holding volatile values

5:04:24 Guest74 I don't reuse any c structs at the moment.  A new one is created for every ioctl call.  I don't know if that's as bad as I think it might be.

5:04:50 White_Flame if it's stack-allocated that's reasonable

5:05:03 Guest74 It only reads or writes to a lisp struct if it was provided by the caller.

5:05:48 Guest74 i.e. the default is it will just return values as a plist of slot value.

5:12:35 Guest74 thankfully it doesn't seem like there's anything too crazy in c stuff for ioctls.  The worst I've run across so far is nested structs and pointers to arrays within the struct, but it's all seemed to work out so far.

5:13:28 Josh_2 beach: Has anyone worked on a practical implementation of a posix interface like the one you have started specifying?

5:16:41 Guest74 sb-posix doesn't seem so bad.

5:17:54 Guest74 well, except sb-posix:ioctl doesn't really work and I have to use sb-unix:unix-ioctl, but besides that, i'm not sure how much more lispy you can get for posix calls.

5:20:34 Guest74 actually, i shouldn't say it doesn't work.  It probably works as specified by posix but linux changed how it works and bsd users were made sad.

5:33:34 pillton There must be a better way to expose OS functionality then via C APIs.

5:35:36 Guest74 i guess it depends on what OS functionality you are talking about.  We could write our own interfaces to file systems.

5:37:10 Guest74 and love it or hate it, since everything is a file in unix, you can just open-file.

5:38:01 edgar-rft ACTION suggests using assembly interrupts again instead :-)

5:38:45 pillton I am not sure that is true. I can't open a file which connects to a website for example. I also can't open a file to execute a command.

5:40:19 Guest74 execute a command?

5:40:45 Guest74 you can talk directly to the virtual terminals if you want.

5:40:51 opcode anyone here run a swank/slynk repl on a remote server accessible via the internet? if so how do you secure it?

5:41:32 pillton Guest74: Start another process e.g. fortune.

5:41:47 White_Flame pretty sure you can open a file to a website in Plan9

5:42:15 White_Flame unix just sucks ;)

5:42:29 Guest74 but i'm not sure how that's even a complaint when we have drakma/dexador?

5:42:35 White_Flame but as long as OSes have a C heritage, it will expose its APIs in C

5:43:50 Guest74 So all you have to do is write your OS in lisp.

5:47:05 White_Flame yep

5:47:11 White_Flame or do something other than function calls

5:47:21 White_Flame message passing, software interrupts, etc

5:48:04 pillton Why do you have to write your OS in lisp? What does it solve?

5:48:15 White_Flame the OS API wouldn't be exposed in C

5:48:40 pillton Then the people who like C will complain about it being in CL.

5:48:49 White_Flame stuff 'em

5:48:51 White_Flame ;)

5:48:55 Josh_2 opcode: Yes, you should use ssh tunnels

5:49:12 opcode i was going to use wireguard

5:49:20 White_Flame opcode: swank by default only listens to localhost, not external ports

5:49:40 beach Josh_2: I don't think so.

5:50:16 opcode White_Flame: can't you just set swank::*loopback-address* ?

5:50:35 beach pillton: Do you not think my proposal is better than the C API?

5:52:38 beach Actually, never mind. I should be quiet.

5:52:48 pillton beach: I'd have to think about it. I'd rather people write libraries which solve specific problems rather than offer interfaces to APIs of other languages.

5:53:02 beach I fully understand.

5:55:18 White_Flame opcode: I mean its default is secured away from outside connections, so it's less of a concern. But anybody with access to execute code on the box can access swank

5:56:01 opcode White_Flame: maybe i'm not clear, but I want to have a long-running lisp process on a server, and i want to expose a repl so I can hack on it from home

5:56:08 White_Flame right, I've done that

5:56:11 opcode but obviously as you said, the repl presents a security concern

5:56:31 White_Flame it wasn't on an internet-accessible box, technically, btu we could tunnel in

5:56:54 opcode yeah, I was planning to use nftables to only expose swank on a wireguard tunnel

5:57:00 White_Flame and I don't think you should open the listener to anything but the loopback device, and externalize the access via ssh or whatever

5:57:00 opcode it's my box, I'm not worried about other users

5:57:08 opcode ah ok

5:57:37 opcode that's probably much simpler than what I'm trying to get going here

6:16:28 Josh_2 You should just use ssh tunnels

6:16:57 Josh_2 I have 3 lisp images running on one of my VPS's and I can just open my ssh connection with tunnelling and then I can connect to them

6:20:33 pillton beach: For example, asynchronous I/O, synchronous I/O and non-blocking I/O. I think CL can do a much better job of offering interfaces to these I/O strategies that actually helps the user organise /their/ application. The underlying C API does not do that. The functionality is "exposed" by a convoluted set of C constants, functions and file descriptor modes, obfuscating each individual strategy. I think the C constants aren't even

6:20:34 pillton consistent across the different unixes. And, once you're done with the unix world, you also have to consider the non-Unix world as well.

6:21:31 beach pillton: I see.

6:23:03 beach The idea of my suggested protocols was to do it in 2 layers. The lower layer would be closer to the C API, and the upper layer would present something much more convenient to the Common Lisp application programmer. Perhaps the upper layer would be the place to do a better job.

6:23:28 beach Actually, I didn't come up with the 2-layer idea. But I forget who it was who did.

6:26:33 beach Not that it matters. I won't have time to work on it in the near future anyway.

6:26:58 pillton beach: Sure. I am just giving my two cents.

6:27:14 beach Yes, what you say makes a lot of sense.

6:48:26 dirtcastle i installed slime. but If I M-x slime , it shows " searching for program: No such file or directory,lisp"

6:56:27 White_Flame dirtcastle: you probably have to define which lisp it's going to call, and is just defaulting to "lisp"

6:56:52 dirtcastle I fixed it .

6:56:54 White_Flame however it's easier to use portacle for an all-in-one solution, or quicklisp-slime-helper to install slime individually

6:56:57 dirtcastle u r right

6:56:57 White_Flame ah, k

6:57:15 dirtcastle I set inferior lisp program sbcl

6:57:54 dirtcastle it's there in my init but it won't load properly because my init produces errors

7:02:27 molson yeah, emacs doesn't load the init if it has errors.

7:29:02 jmes Question for Parenscript users: how do I call a macro inside a PS form? When I want to use MY-MACRO (which should spit out some Parenscript code) and I write (ps (my-macro)) it gets compiled to myMacro. I tried to RTFM but my brain might be too small

7:32:26 jmes I have a situation more like this, where if I use Parenscript's LISP like so (ps (let ((something 1)) (lisp (my-macro something)))) then SOMETHING will be unbound

7:46:53 hashfunca51 White_Flame: i did the exercise of constructing a templated macro without using quasiquotes. but i've been stuck on how i get around splicing lists without using ,@

7:50:29 Josh_2 jmes: PS has its own defmacro, try that

7:55:48 Josh_2 jmes: (ps:defpsmacro js%console-log (val) `((ps:getprop console 'log) ,val))

7:58:08 hashfunca51 White_Flame: especially arguments given to an &body parameter in said macro, because the argument passed is wrapped in a list ... (1 2) => ((1 2)) ... thus the list must be spiced, since, if not, an erroneous function call is made. e.g., ... (1 2) => ((1 2)) => "illegal function call". i've been at it for a long time and still haven't found a way around not using ,@

7:59:09 jmes Josh_2: ah, somehow I missed that, thanks! I hope this does the trick :)

8:15:20 hashfunca51 White_Flame: sorry, you were involved in that conversation but it was @moon-child that recommended that exercise

8:15:53 hashfunca51 annnd he/she isn't online

8:21:04 White_Flame you have to use append

8:21:10 White_Flame aaaaand he/she isn't online

8:36:03 beach I think this has been said already. Oh well.

8:38:30 flip214 White_Flame: list*, concatenate, dot notation, explicit CONS calls...

8:40:25 White_Flame hmm, you sure about explicit CONS calls? also dot notation

8:40:44 White_Flame for splicing in the middle of lists

8:51:07 flip214 depending on the input data an (alexandria:flatten (cons list1 (cons list2 (cons list3 nil)))) might work for you as well ;)

9:32:47 neominimum Anyone know of projects that are looking for hands? I have two brain cells that I would like to put to good use.

9:35:34 beach Tons! What do you know how to do, and what kind of project would you prefer?

9:44:20 neominimum I'd like to take the opportunity to work in a variety of domains to further my knowledge. I don't think I really have a preference as to what I want to work on, as I hope the problems can be interesting in and of themselves. That being said I have recently begun some study into basic computer architecture, up through to high-level language compilers and I found it pretty interesting. I am sorely lacking in theoretical foundations, but am

9:44:20 neominimum willing to learn.

9:47:41 beach I'll keep that in mind. You could also start a new project.

9:48:11 beach http://metamodular.com/Common-Lisp/suggested-projects.html

9:49:22 beach We would help you with advice, code reviews, etc.

9:51:22 beach Or you could check with Shinmera (games) and jackdaniel (McCLIM, ECL).

9:56:08 neominimum I've been developing an actor runtime for message passing style concurrency, it's rather primitive still, but it has been the vehicle by which I've been learning common lisp. I have coupled that with a basic syntax for coordinating the execution of lisp code using an asynchronous evaluation semantics, so that one can continue to code in direct style as opposed to a continuation passing style.

9:56:28 neominimum Okay great I'll take a look, thanks.

10:42:48 Shinmera Games touch on a ton of domains

11:47:21 neominimum I like the world building aspect of game development, not so much gameplay mechanics though, but I'd like to have a go at creating some imersive 3d environements one day. AIUI there is an absolute ton of work that goes into the implementation of game engines.

12:09:42 jackdaniel usually you start from something that is interesting to you and you end god knows where :)

12:30:08 neominimum true! :) Honestly sometimes I feel overwhelmed with how many forks and detours there seem to be in the path. Theres way to much to master in a single lifetime, I get a bad case of analysis paralysis at times.

12:34:19 jackdaniel no worries, if it is true that the world is just, then everyone must have the same share- in other words we reincarnate as each possible person in each possible time with all possible scenarios ,-)

12:47:51 neominimum Personally I can't wait to reincarnate as a sea cucumber. No care in the world, gently being tossed around by the warm sea currents. Magnifique.

12:51:39 neominimum Night all

12:51:58 beach 'night neominimum.