libera/#commonlisp - IRC Chatlog
Search
7:52:15
lukego
okay I've successfully run CFFI-GROVEL for the first time and now I'm definitely having fun :P
9:53:37
White_Flame
lukego: if you M-. into TIME and such, you'll see that SBCL already has wired up references to such things directly in the runtime, without CFFI
9:56:54
borodust
lukego: link me to the header you want to wrap, i'll generate bindings for it with claw
9:57:39
lukego
borodust: that would be very interesting to see as an option, thanks. it's https://github.com/torvalds/linux/blob/master/include/uapi/linux/perf_event.h
9:58:52
lukego
White_Flame: Yeah. I'd take that route except that this is a relatively big interface - literally hundreds or thousands of new time sources instead of just a few clocks - and is probably equally useful to users of CCL/ECL/etc.
9:59:40
lukego
also it's a raw system call interface with pretty elaborate structures so I really want some help with correctly defining all of that without too much fuss
10:00:58
lukego
White_Flame: This is the relevant manpage which is pretty long https://man7.org/linux/man-pages/man2/perf_event_open.2.html
10:06:06
lukego
There's a whole bunch of hardware support for profiling in CPUs these days that I'd like to expose to Lisp. Simple things are counters for cycles/instructions/cache-misses/branch-mispredictions/etc. More elaborate are automatic sampling and tracing on the hardware level e.g. logging addresses of instructions that took a long time to complete and so on. Seems like something people would find uses for.
10:06:31
White_Flame
yeah, I'd love to see proper access to that sort of thing right from within the image
10:07:40
White_Flame
I'm not enough of a unix junkie to get into all the linkage details of all that myself
10:14:38
lukego
Is there a canonical "portable" way to make a raw system call? I want to call Linux perf_event_open from CFFI-using code and it doesn't have a C wrapper in any standard library.
10:15:04
lukego
I see code in Osicat for defining syscalls but maybe that is One More Thing that's not needed
10:22:20
borodust
lukego: okay, that won't be fast, i'm away from my main developer machine and i don't have :claw set up over here :(
10:29:15
jackdaniel
lukego: what do you mean by "raw" system calls? the most "raw" way is to shovel numbers into registers and trigger the interruption isn't it?
10:36:16
Nilby
In my o/s wrapper set, sometimes the libc wrapper isn't right and I've been forced to use the entry point libc uses, or even lower and call _syscall with the right number. None of that is what I would call "portable", more like it's sometimes "ported".
10:41:53
Nilby
Hopefully on good compilers the lisp wrapper ends up mostly disappearing in compiled code, but there's of course always the issues of data type conversion, so unless you really need performance and keep everything in C data on the Lisp side, or everything works out with unboxed data, there will be some overhead.
10:50:58
lukego
jackdaniel: The level of rawness I have in mind is basically osicat:defsyscall. maybe I'll just use that and not overthink things. (I don't have a dependnecy on oscicat before this but hey why not.)
10:52:09
lukego
er though hm not that's not right, it seems to be expecting to call a named C wrapper function
10:54:11
lukego
CodeBitCookie[m]: there are a lot of styles to choose from. starting with imitating PCL sounds like a good idea to me.
11:01:30
Nilby
Hmmm. That perf_event_open looks like one of the harder syscalls to wrap, which foolishly makes me want to try.
11:55:58
lukego
borodust: no worries. Maybe auto bindings won't save so much in this case anyway, there is a lot of information that will be communicated in custom / hardware dependent format via mmap region
11:59:38
lukego
borodust: but I will be very curious to see what claw produces if you happen to get a chance :)
12:36:15
Nilby
So it seems loop collect works by keeping a tail pointer. Is there some way to do a manual :collect, e.g. for use in not first level forms, without manually doing the whole tail pointer thing or worse a push and nreverse?
12:39:21
semz
after running into this problem myself quite a bit I just decided to roll a WITH-COLLECT macro that does the tail pointer thing under the hood
12:39:49
beach
Nilby: Then the nested form can always return the empty list of you don't want to "collect" anything.
12:41:53
Nilby
beach: Interesting. So loop ‘append’ unlike cl:append keeps a tail pointer or somehow doesn't need the traverse the list?
12:43:25
yewscion
beach: Yeah, I'm new here. Been on IRC for a long while, but new to Libera and #commonlisp.
12:44:32
beach
Nilby: You may want to run a performance test to confirm that it is not doing something quadratic on some Common Lisp implementations.
12:53:34
semz
Thinking about it, one could also treat (cons list end-pointer) as a data structure and turn collecting into a normal function. Gets rid of the macro but still a little awkward to introduce a new structure when you're really just working with lists...
12:56:17
beach
Nilby: Certainly, on SBCL, using the APPEND LOOP keyword does not result in quadratic behavior.
13:21:22
lukego
Hey can CFFI-GROVEL work with bitfields in structs at all? I was thinking I'd just treat them as integers but I'm not sure how to declare that... it wants a c-name and all the names I have are for bitfields, and none of them work because you can't take offsetof() on them.
13:22:25
lukego
Maybe a compromise would be to use CFFI-GROVEL to define all the various enum's but not for structs and function signatures.
13:29:35
Nilby
lukego: I wish I could help, but I'm very contrary and don't use cffi-grovel, so I usually leave bitfields as ints and either make bit field macros, or more frequenly wrap it in a nicer lispy interface.
14:48:09
raeda
STAPLE::WHOLE-PARAMETER-GROUP is unbound when I load staple. Maybe it's a bug in code that uses staple?
16:07:09
lisp123
If I want to load some elisp code for a program, I have two options. Either I get the end user to download and load the .el file themselves (which encourages them to tinker with it and improve it for themselves), or I can just do a slime:eval-in-emacs....I am leaning towards the latter as it reduces the installation process, any thoughts?
16:14:19
mfiano
Which has a similar piece of functionality, so you'll need to use read time conditionalization
16:15:08
lisp123
Ok, I will create a SLY front-end and make that adjustment....and also create some other front end for other users
16:16:27
lisp123
Actually thanks, this is very helpful. I will create an alternate version that works directly in the REPL, and leave the Emacs version there for those who want a slightly better experience
17:09:17
jmercouris
it always bothered me, maybe there is like a sethash in some library or something
17:14:06
jmercouris
Bike: well, I can push to a list, I feel like I should be able to push to a hash table
17:15:10
Bike
but that's what (setf gethash) does. is it just the order of arguments you don't like?
17:15:37
lisp123
hould the language have every possible gadget like this that anyone can write in a minute? In Common Lisp, there are included batteries, but they are other kinds of batteries: things that are actually hard to do. For example, a compile function for dynamically compiling code at run time. It would be prohibitively difficult for most users to develop such a thing from scratch compared to pulling keys or values from a hash table in half a doze
17:16:17
jmercouris
well, it's not a big deal, perhaps I am just procrastinating and not realizing it :-)
17:16:54
lisp123
Think of it as a feature ;) It encourages more programmers to write small bits of code vs. being spoon fed --> The end result making them better programmers
17:43:54
Guest17
saw the asdf post on reddit. I keep running into this perennial problem where I need to dump a program with asdf dependencies into another source tree such that anyone can check out the code and make the program from scratch as opposed to distributing it via the system package manager or as a quicklisp package.
17:44:35
Guest17
after reading that post, I figured that the only thing I really needed was to prevent asdf from searching the user directories for packages and only use one provided by project configuration
17:45:20
Guest17
additionally, in my particular case, it would be nice to keep the build objects (fasls) in the source tree in a particular spot, as that is where we put build artifacts of other code in this repo. That part I think I read in the documentation
17:45:57
lisp123
jcowan: I don't disagree, it can come across as an excuse - but everyone has different requirements, one person's sethash is another's setf gehash
17:47:04
Guest17
the way I see it, I can check in a whiole lisp distribution, like a checkout of a working ccl folder, the result of calling quicklisp bundle, ??? calling code to build a kernel+image ??? only using asdf packages found in the local directory and preventing the user directory defaults ??? profit
17:52:00
lisp123
Guest17: Not entirely sure what you are after, but if I recall correctly QuickLisp Local Projects might help
17:53:07
lisp123
Otherwise you just need to figure out where ASDF scans first (don't know off the top of my head) and put your .asd files there -> In general I think its better not to have multiple versions of the same dependency running around
17:53:27
Guest17
so my ultimate goal is that I have a work repo for non lisp code. I made a utility that I don't want to rewrite in common lisp. I want any yahoo that checks out the main repository to essentially run make and get my utility as a standalone program. My code depends on packages sourced from quicklisp and I know how to create a bundle of the
17:54:05
Guest17
right, the last part is important for repeatable builds, I don't want asdf to scan the default directories, in case someone like me has a local install of quicklisp
17:54:44
lisp123
Your users have Common Lisp installed I imagine and are CL users right, otherwise you wouldn't be sharing them the source code
17:55:51
Guest17
They don't, not yet anyways. Actually since we use ros, sbcl is pulled down as a dependency, so yes they all do ;-) but I'd prefer it if the build process didn't depend on any system-installed or user-installed files/packages, and that such things wouldn't get accidentally pulled in if they exist
17:58:01
Guest17
I know i've had issues with the fact that in ubuntu 16.04, many lisp packages are available from the apt repo, and those conflict with installing things assuming those aren't available (which I'm always doing)
17:59:38
lisp123
I'm trying to check the IRC Logs on that, I think that might be a better version of ASDF that could help you
18:05:29
Shinmera
There's also no way to scan for projects, or even define them outside of writing source code
18:09:38
jcowan
lisp123: That case doesn't bother me. But on that argument, why does CL standardize LENGTH? It's about five lines of code. The CxxxxR functions are even shorter.
18:11:17
jcowan
Sure. But the passage that lisp123 quoted claims that there is a rational argument for excluding functions with short definitions.
18:11:34
pjb
lisp123: symbols are string designators, and string= and string-equal take string designators…
18:12:49
lisp123
pjb: Thanks, I think I'm all good on Symbols today after my earlier confusion in the morning
18:13:31
lisp123
I am more or less finished on my autologging tool (based off IBCL), will share shortly for feedback and if anyone finds it useful
18:15:37
pjb
* The append keyword causes its list values to be concatenated into a single list, as if they were arguments to the function append.
18:15:37
pjb
* The nconc keyword causes its list values to be concatenated into a single list, as if they were arguments to the function nconc.
18:16:46
pjb
Indeed, nothing says that the implementation cannot optimize by keeping a reference to the last element, but note that since loop often cannot easily guess what will be the last list appended to the result, in the case of append it has to keep a reference to the start of the last appended list, to be able to copy if if a new one needs to be appended.
18:18:23
pjb
Both could also be implemented directly as: (loop collect list into lists finally (apply appender lists)) with appender = (function append) or (function nconc), and therefore defer the optimization to loop collect, and to the append and nconc functions.
18:21:09
lisp123
jcowan: I take back my earlier comment partly, there's no good reason for not having more standardisation as jmercouris queried earlier re sethash. However, I think "just write your own" is a fair response in the sense (IMO?) that seems to be one of the essential styles of programming in Lisp (reshaping the language to suit your needs)
18:22:00
pjb
I note that no implementation returns true for (let ((last (list 1 2 3))) (tailp last (loop for list in (reverse (cons last (quote ((4 5 6) (7 8 9))))) append list))) ; which means that they are all non-conforming: (loop append) doesn't behave like (function append)!
18:23:05
jmercouris
lisp123: well, sure, but the problem is not everyone will use whatever layer you design, the common denomenator is CL
18:23:12
jcowan
I agree with that, certainly. But I think it's worth standardizing trivial but commonly used things for two reasons: everyone will use the same name for them, and to avoid this problem: "Data type X would be optimal here, but I have a better library for data type Y, so I'll use Y.
18:23:17
jmercouris
so if you are using some super set of CL, in order for users who are not using your superset you have to be able to compile to CL
18:24:09
jcowan
Well, if it comes to that, every function and macro added to an image creates a superset of CL.
18:24:54
jmercouris
I guess I am thinking of things differently, because I am thinking in the context of Nyxt, which is a application framework
18:24:59
pjb
or any non-standard operator specific to the implementation. Including semi-standard stuff like the MOP, gray-streams or less standard stuff like ffi and sockets.
18:25:09
jmercouris
and I'm thinking OK, things that users build with Nyxt, how will they be broadly applicable to the CL community as well?
18:25:50
Bike
in this case the function at issue wouldn't just be short, it would be exactly another operator with its arguments permuted
18:25:50
pjb
jmercouris: if you don't hide the fact that some stuff comes from the CL package and some from a NYXT package, users should be able to write library code that only use CL…
18:26:05
Bike
at which point maybe there should just be mechanisms for doing that (like haskell has, off the top of my head)
18:33:19
lisp123
jmercouris: I assume you would have a fair bit of custom API for various widgets, so nyxt code writers will largely work with your standard library vs. CL?
18:34:09
lisp123
So as long as you document reasonably well, most CL users will be able to navigate through your codebase and figure out where you are abstracting away -> Thus, IMO write for the non-CL user
18:37:42
etimmons
Guest17: I think either CLPM or Qlot would help you out if you want semi-repeatable builds
18:37:45
borodust
lukego: bindings look smth like this: https://github.com/borodust/claw-linux-perf/blob/master/bindings/x86_64-pc-linux-gnu.lisp
18:39:01
etimmons
If you already use Roswell though you might lean toward Qlot, as it's fully embraced Roswell, for better or worse
18:39:48
borodust
lukego: i didn't test them though and they are also for 5.12 so if your pc blows up it's not on me ;p
18:41:17
lukego
are these rational numbers okay? https://github.com/borodust/claw-linux-perf/blob/master/bindings/x86_64-pc-linux-gnu.lisp#L508
18:46:25
etimmons
Guest17: it also rubs me the wrong way. I think probably because it tries to do too much
18:48:13
etimmons
Guest17: If you decide to poke at CLPM, feel free to jump to #clpm if you have questions. Also happy answering things here, so long as we don't swamp it
18:49:36
borodust
https://common-lisp.net/project/cffi/manual/cffi-manual.html#foreign_002dslot_002doffset
18:57:15
lukego
Just now I suspect that I'll use CFFI-GROVEL to resolve all my named enum constants to concrete values and then otherwise probably use NIBBLES to construct my foreign data. That wouldn't be a good strategy in general but I think makes sense here especially since there will be a bunch of other in-memory structures not defined as C structs at all iirc
18:57:23
etimmons
Any mods here also have a matrix account? Or are willing to briefly give mod powers to a matrix user?
18:57:47
etimmons
IRC -> Matrix messages appear to be broken because someone on the matrix side isn't fully connected
18:58:14
etimmons
It's a security feature (on by default) so that anyone sending a message on IRC knows who's all in the room when they send a message
19:37:43
q3cpma
Hello, does anyone know a way to avoid calling gethash more than once when you want to check the value before setting it? Maybe a way to store a place in a variable, or something?
19:40:30
_death
when you read from a hash-table, you use gethash, and when you write, you use (setf gethash)
19:43:48
q3cpma
_death: but I have to call gethash twice for each entry, and the standard obviously doesn't say the hash won't be computed twice
19:46:12
_death
a call to gethash is not the same as a call to (setf gethash), although it's true that the hash may be computed twice