libera/#commonlisp - IRC Chatlog
Search
3:21:48
hayley
Messing with my regex-to-CL compiler, which now seems to scan at somewhere around 570 to 700 Mcharacters/s.
3:36:41
hayley
Is there much that can be done to optimise a state machine translated to CL code? The only things I can think of are that it is branch heavy, and that memory bandwidth isn't infinite.
3:51:19
hayley
Well, I can't really do much about the branches, but the code performs a linear scan over a vector, which should be pretty good.
4:24:40
moon-child
I mean, the branch predictor will learn if, after state X, you usually transition to state Y. But if your input requires you to transition to state Z, there's nothing you can do about that
4:27:03
moon-child
'_same_ way each time' branch predictor can predict much more complex patterns than that, new zen even has a perceptron, though obviously if there's _no_ entropy there's nothing to be done
5:56:46
lisp123
I'm struggling a bit with Symbols, since equality is based on being the exact same symbol, not just whether their print names are the same. That problem I can resolve with a custom equality test. The other complexity I have is symbols being interned in different packages - so if I just rely on symbol name to identify symbols, I am missing this important information
5:57:38
lisp123
For example, the elisp reader does not have access to CL's package definitions, so if I read something via elisp, its a bit tricky to associate the symbol with the correct package. SYMBOL-PACKAGE helps here
5:57:57
lisp123
I want to know if there is a better way than just using SYMBOL-NAME & SYMBOL-PACKAGE
5:59:50
lisp123
moon-child: I would like a 'better' (for my purposes) way of working with symbols, sort of like having the symbol, its package information, its symbol name - all rolled upinto one (e.g. like a SYMBOL CLOS object?)
6:00:19
lisp123
beach: I have written a module for users to select which functions to log, this is in elisp since its I/O
6:01:05
moon-child
do you want package FOO to pass a designator for a function defined in package BAR? You can do that?
6:01:20
beach
lisp123: I'll be quiet since this obviously is not Common Lisp, so I have no advice to give.
6:02:35
lisp123
moon-child: I was just wondering whats the best way to work with symbols --> looks like the SYMBOL object is the right way and has SYMBOl-NAME / SYMBOL-PACKAGE all embedded
6:03:26
lisp123
I was thinking of creating a list structure to store all this information (since I was confused) --> Now I realise I don't have to do that
6:04:30
lisp123
Thanks both, its very helpful - sorry I was just confused earlier on, but now I know that I need to just use the SYMBOL object within the CL code and then destructure it when transferring to elisp & back
6:05:46
lisp123
I guess I should SYMBOL-NAME & SYMBOL-PACKAGE to split a SYMBOL up, and then to find a symbol, use FIND-SYMBOL
6:06:01
beach
lisp123: By the way, the term "CLOS object" is meaningless. Every Common Lisp object is an instance of some class, and the concept of classes and instances of classes is a CLOS thing.
6:08:49
beach
lisp123: By the way(2): In a typical Common Lisp implementation, SYMBOL is a built-in class, but in SICL it is a standard class: (defclass symbol () ((... :reader symbol-name) (... :reader symbol-package)))
6:10:27
lisp123
beach: Thanks, that was the missing link (thanks for clarifying). I didn't realise SYMBOL itself was a class which stored various pieces of information
6:11:17
beach
No, I just said that SYMBOL might be a built-in class which is different from a standard class.
6:11:59
beach
When the Common Lisp HyperSpec says "system class", it can be a built-in class with special representation, which is different from a standard class which has a standardized representation.
6:12:36
beach
This is so that pre-ANSI Common Lisp implementations could get away with special representations of various objects even after CLOS was bolted on.
6:14:16
lisp123
Would it be a big deal to rewrite an implementation for converting symbol as a built in class to a standard class?
6:15:46
beach
Yes. Typically, that would completely ruin their bootstrapping technique. Most Common Lisp implementations add CLOS at the end of bootstrapping, so standard classes are not available by the time symbols are needed.
7:19:27
lukego
Hey I'm noodling around trying to access the Linux kernel perf_event interface from Lisp e.g. to get free profiling data and sample performance counters like # cycles/instructions/etc. anyone happen to have already been there? also, CFFI seems like the obvious approach, but I'm finding it kind of confusing and wondering about doing something more direct e.g. constructing C structs using NIBBLES
7:20:46
lukego
though it is neat that the CFFI groveller can fill in the magic numbers from the C header file. Just that it's not obvious to me how to encode some basic C-isms like struct bitfields and anonymous unions. Maybe I should just skip those for now. It seems like what CFFI calls a "bitfield" is something else - like an enum of values 1<<0, 1<<1, 1<<2, etc.
7:25:17
lukego
probably I need to stop overthinking this and start running cffi-grovel to see it in action...
7:39:41
lukego
Thanks. Could well be that CFFI-GROVEL is exactly the sweet spot in terms of automation. Just have to get to know it a bit.
7:40:17
lukego
(SB-GROVEL specifically says that it doesn't support bitfields in structs but I don't see anything in the CFFI docs about this.)
7:41:38
lukego
... though, this FFI API is really low-level internal plumbing talking directly to a Linux syscall, so might make sense to auto-gen a messy binding with c2ffi/autowrap/claw or something, since not much client code will ever be written... but those take some work just to setup.
7:52:15
lukego
okay I've successfully run CFFI-GROVEL for the first time and now I'm definitely having fun :P
9:53:37
White_Flame
lukego: if you M-. into TIME and such, you'll see that SBCL already has wired up references to such things directly in the runtime, without CFFI
9:56:54
borodust
lukego: link me to the header you want to wrap, i'll generate bindings for it with claw
9:57:39
lukego
borodust: that would be very interesting to see as an option, thanks. it's https://github.com/torvalds/linux/blob/master/include/uapi/linux/perf_event.h
9:58:52
lukego
White_Flame: Yeah. I'd take that route except that this is a relatively big interface - literally hundreds or thousands of new time sources instead of just a few clocks - and is probably equally useful to users of CCL/ECL/etc.
9:59:40
lukego
also it's a raw system call interface with pretty elaborate structures so I really want some help with correctly defining all of that without too much fuss
10:00:58
lukego
White_Flame: This is the relevant manpage which is pretty long https://man7.org/linux/man-pages/man2/perf_event_open.2.html
10:06:06
lukego
There's a whole bunch of hardware support for profiling in CPUs these days that I'd like to expose to Lisp. Simple things are counters for cycles/instructions/cache-misses/branch-mispredictions/etc. More elaborate are automatic sampling and tracing on the hardware level e.g. logging addresses of instructions that took a long time to complete and so on. Seems like something people would find uses for.
10:06:31
White_Flame
yeah, I'd love to see proper access to that sort of thing right from within the image
10:07:40
White_Flame
I'm not enough of a unix junkie to get into all the linkage details of all that myself
10:14:38
lukego
Is there a canonical "portable" way to make a raw system call? I want to call Linux perf_event_open from CFFI-using code and it doesn't have a C wrapper in any standard library.
10:15:04
lukego
I see code in Osicat for defining syscalls but maybe that is One More Thing that's not needed
10:22:20
borodust
lukego: okay, that won't be fast, i'm away from my main developer machine and i don't have :claw set up over here :(
10:29:15
jackdaniel
lukego: what do you mean by "raw" system calls? the most "raw" way is to shovel numbers into registers and trigger the interruption isn't it?
10:36:16
Nilby
In my o/s wrapper set, sometimes the libc wrapper isn't right and I've been forced to use the entry point libc uses, or even lower and call _syscall with the right number. None of that is what I would call "portable", more like it's sometimes "ported".
10:41:53
Nilby
Hopefully on good compilers the lisp wrapper ends up mostly disappearing in compiled code, but there's of course always the issues of data type conversion, so unless you really need performance and keep everything in C data on the Lisp side, or everything works out with unboxed data, there will be some overhead.
10:50:58
lukego
jackdaniel: The level of rawness I have in mind is basically osicat:defsyscall. maybe I'll just use that and not overthink things. (I don't have a dependnecy on oscicat before this but hey why not.)
10:52:09
lukego
er though hm not that's not right, it seems to be expecting to call a named C wrapper function
10:54:11
lukego
CodeBitCookie[m]: there are a lot of styles to choose from. starting with imitating PCL sounds like a good idea to me.
11:01:30
Nilby
Hmmm. That perf_event_open looks like one of the harder syscalls to wrap, which foolishly makes me want to try.
11:55:58
lukego
borodust: no worries. Maybe auto bindings won't save so much in this case anyway, there is a lot of information that will be communicated in custom / hardware dependent format via mmap region
11:59:38
lukego
borodust: but I will be very curious to see what claw produces if you happen to get a chance :)
12:36:15
Nilby
So it seems loop collect works by keeping a tail pointer. Is there some way to do a manual :collect, e.g. for use in not first level forms, without manually doing the whole tail pointer thing or worse a push and nreverse?
12:39:21
semz
after running into this problem myself quite a bit I just decided to roll a WITH-COLLECT macro that does the tail pointer thing under the hood
12:39:49
beach
Nilby: Then the nested form can always return the empty list of you don't want to "collect" anything.
12:41:53
Nilby
beach: Interesting. So loop ‘append’ unlike cl:append keeps a tail pointer or somehow doesn't need the traverse the list?
12:43:25
yewscion
beach: Yeah, I'm new here. Been on IRC for a long while, but new to Libera and #commonlisp.
12:44:32
beach
Nilby: You may want to run a performance test to confirm that it is not doing something quadratic on some Common Lisp implementations.
12:53:34
semz
Thinking about it, one could also treat (cons list end-pointer) as a data structure and turn collecting into a normal function. Gets rid of the macro but still a little awkward to introduce a new structure when you're really just working with lists...
12:56:17
beach
Nilby: Certainly, on SBCL, using the APPEND LOOP keyword does not result in quadratic behavior.
13:21:22
lukego
Hey can CFFI-GROVEL work with bitfields in structs at all? I was thinking I'd just treat them as integers but I'm not sure how to declare that... it wants a c-name and all the names I have are for bitfields, and none of them work because you can't take offsetof() on them.
13:22:25
lukego
Maybe a compromise would be to use CFFI-GROVEL to define all the various enum's but not for structs and function signatures.
13:29:35
Nilby
lukego: I wish I could help, but I'm very contrary and don't use cffi-grovel, so I usually leave bitfields as ints and either make bit field macros, or more frequenly wrap it in a nicer lispy interface.
14:48:09
raeda
STAPLE::WHOLE-PARAMETER-GROUP is unbound when I load staple. Maybe it's a bug in code that uses staple?