freenode/#lisp - IRC Chatlog
Search
0:08:06
jasom
blargh. Suggestions for debugging compiler macros? Specifically cffi:foreign-alloc appears to be not expanding to a fast path
0:09:03
Bike
jmercouris: (describe (find-class 'cl-string-match:re-match)) might also be informative.
0:09:10
pjb
Otherwise, to debug it, you can just call it directly (extrating the body in a normal function).
0:10:09
jasom
Bike: well I know it's not expandingas expected because I end up with foreign-alloc on the stac rather than %foreign-alloc
0:17:21
jasom
specifically if you specify initial contents, then it completely loses all optimizations for the type being constant
0:21:40
jasom
I changed my with-foreign-alloc macro to perform the initialization of the memory manually and got a ~1000X speedup
0:24:12
Bike
so with constant type and unconstant but specified count you'd want it to expand to `(%foreign-alloc (* ,count ,(foreign-type-size (eval type))))?
0:24:40
Bike
for initial contents you'd have to once-only the contents and take the length, i guess.
1:06:00
Ober
huh. 'At the request of John McCarthy, Lisp’s creator, no single language that is a member of the Lisp family is to be intended to be the definitive dialect; that is, none is to be called just “LISP.” '
1:29:07
k-stz
another tagged pointer example: (setf *a* (1+ most-positive-fixnum) *b* (1+ most-positive-fixnum)) (eq *a* *b*) => nil, but set *a* and *b* to the same fixnum number then (eq *a* *b*) ==> T ! For sbcl at least, because EQ does pointer comparison. I guess, then, with EQ one can test if a value is represented by an immediate tag pointer
1:41:23
aeth
There's nothing restricting with-accessors to only be for accessors to CLOS objects, is there? It says undefined, but it should in most if not all implementations just be a trivial symbol-macrolet. http://www.lispworks.com/documentation/HyperSpec/Body/m_w_acce.htm
1:45:55
aeth
I'll probably copy the basic symbol-macrolet pattern (and change it slightly) for things that don't necessarily have that slot concept
1:47:01
Bike
if an implementation did actually restrict it to standard objects, you could probably decently make a case against that to them
1:55:11
aeth
Sorry, I mean, you can do (defmethod meh ((foo foo) x) (values foo x)) and that's separate from (defmethod meh (y x) y) with the former being (FOO T) and the latter being (T T). Also, you can do (class-of (make-foo)) and get a structure-class just like you can do (class-of (make-instance 'bar)) and get a standard-class.
1:57:50
Bike
structure objects are certainly instances of a structure class, and structures are described as having slots and accessors. the question is whether with-accessors means the general senses of those terms or the CLOS-specific senses.
1:59:43
aeth
Although I don't think that would apply to my particular case, where I want to use it on a vector struct. I could definitely see this potentially failing.
2:08:49
aeth
It's a bad argument because CL is already inconsistent, so arguing that something would keep things simple and consistent wouldn't really be relevant in an interpretation of the specification.
3:16:21
pfdietz
The random tester appears to have exhausted itself on sbcl. Ah well, the latest campaign was fun while it lasted. Interesting how bugs can breed over time.
3:20:42
pfdietz
"object n. 1. any Lisp datum. ``The function cons creates an object which refers to two other objects.''"
3:23:16
pfdietz
"access n., [...] 2. n. (of a place) an attempt to access[1] the value of the place."
3:24:50
pfdietz
I think (with-accessors ((x car) (y cdr)) (cons 1 2) (values x y)) ==> 1, 2 would be valid.
3:27:03
Zhivago
Sounds reasonable to me -- it looks like a convenience for establishing a symbol-macrolet.
3:28:50
Bike
I thought maybe it could do CLOS-specific optimizations like using more direct slot access when possible, but it seems to pretty specifically invoke the given accessors.
3:30:26
whoman
hmm what is this error from MOP - forward referenced class? not sure how that is going on
3:31:19
Bike
(defclass foo (bar) ()): if bar is undefined when this is evaluated, it's forward-referenced.
3:47:04
loke
pfdietz: CLHS has the following to say about WITH-ACCESSORS: “The consequences are undefined if any accessor-name is not the name of an accessor for the instance.”
3:54:26
loke
aeth: “instance” is defined as: “instance n. 1. a direct instance. 2. a generalized instance. 3. an indirect instance.]
3:54:40
loke
“direct instance n. (of a class C) an object whose class is C itself, rather than some subclass of C. ``The function make-instance always returns a direct instance of the class which is (or is named by) its first argument.''”
3:55:01
loke
“generalized instance n. (of a class) an object the class of which is either that class itself, or some subclass of that class. (Because of the correspondence between types and classes, the term ``generalized instance of X'' implies ``object of type X'' and in cases where X is a class (or class name) the reverse is also true. The former terminology emphasizes the view of X as a class while the latter emphasizes the view of X as a type
3:55:16
loke
“indirect instance n. (of a class C1) an object of class C2, where C2 is a subclass of C1. ``An integer is an indirect instance of the class number.''”
3:56:11
aeth
It definitely looks like with-accessors should work on any accessor with a reader (foo x) and (setf (foo x) new-object) writer, e.g. cxr
3:56:33
aeth
The HyperSpec states all the cxr things are accessors. http://www.lispworks.com/documentation/HyperSpec/Body/f_car_c.htm
3:57:54
loke
aeth: Yes. There is a non-normative note in the documenation for WITH-ACCESSORS suggesting the typical implementation is simply a MACROLET
3:58:41
aeth
accessors are definitely something that the spec gets right imo. reader (foo ...) and writer (setf (foo ...) new-object) symmetry, that can apply to just about anything.
3:59:20
aeth
Isn't everything an instance since everything has a class, and so with-accessors should really just work on everything with a conventional accessor pair?
4:00:02
loke
aeth: That's the thing. That reading is certainly a valid one... Here's the documentation for "accessor":
4:00:23
loke
“access n., v.t. 1. v.t. (a place, or array) to read[1] or write[1] the value of the place or an element of the array. 2. n. (of a place) an attempt to access[1] the value of the place.”
4:01:07
aeth
It seems natural that with-accessors is an abstraction over conventional/idiomatic CL accessors. (And that's probably one reason why it's better than with-slots, it's abstract.)
4:01:21
aeth
Bike: After this conversation, I'm tempted to do it over anything that has accessors, not just structs.
4:01:23
loke
Bike: Sounds unlikely. Why would any implementation deviate from the standard SYMBOL-MACROLET implementation?
4:02:28
aeth
with-accessors on anything with well-defined accessors (i.e. a matching setf foo for any given foo) seems reasonable
4:03:12
loke
ACTION can pretty much guarantee that it'll work on all implementations, regardless of whether it's allowed by the spec or not.
4:03:21
aeth
The implementation is trivial (although SBCL, CCL, and ECL all have slight differences in the macroexpand-1, try it!), so there probably could be an implementation-specific workaround on implementations that don't use this interpretation.
4:03:59
loke
But, if the original intent of WITH-ACCESSORS was to be allowed for anything, what is the meaning of this statement: “The consequences are undefined if any accessor-name is not the name of an accessor for the instance.”
4:07:12
aeth
Two possibilities. (1) they were opening the door for optimizations or (2) they were opening the door for the trivial implementation, which technically makes things work for non-accessors like print.
4:45:53
aeth
I really like the CL concept of accessors. Compared across languages, even. e.g. https://en.wikipedia.org/wiki/Mutator_method
4:51:24
Zhivago
Alternately you can see it as other languages preferring not to pollute their function namespace.
4:52:46
Zhivago
Of course, then you need some boiler plate to make something like foo_b(a) = c work, or you need another explicit function. like foo_b_set(a, b).
4:58:49
Zhivago
Yeah, something you could pass around. You can always wrap a use of a place in a lambda, but then you need one for each kind of use.
5:00:14
Zhivago
Although I guess you could do that and then use that object as a place which forwarded.
5:01:14
aeth
whoman: (defun lists-have-accessors-too (list) (flet ((caddddr (x) (cadddr (cdr x)))) (with-accessors ((first car) (second cadr) (third caddr) (fourth cadddr) (fifth caddddr)) list (values first second third fourth fifth))))
5:01:48
Zhivago
Devon: Yeah, I'm not sure why they went out of fashion -- but I suspect it's because they didn't support place semantics.
5:06:54
aeth
Zhivago: You indirectly raise a good question. Does the spec allow implementing everything (including hash tables) as lists? (Not like anyone should try to do this, though!)
5:09:02
aeth
Zhivago: or just make the car a symbol that means that it's a hash-table. Any list without a type symbol in the car is a regular list
5:14:48
Zhivago
Sometimes I wonder if starting with dictionaries and then adding schema to them would have produced a better outcome.
5:23:06
Zhivago
They're probably just one of those things that sounds like a good idea until you really think about it.
6:14:05
pfdietz
One can defined a pseudo-pointer class in CL. It would hold reader and writer functions for some actual place. Define DEREF and (SETF DEREF) methods for it.
6:16:14
dmiles
"class-of or type-of would not produce the expected results." I am litterally stuffing classof and typeof ijn the objects properties
6:20:50
dmiles
(get-opv a-similar-point) ==> ((typeof . POINT)(classof . claz_u_point)(ref . #<claz_u_point 5>)(sname . "znst_5")(instance . claz_u_point)(x . 3)(y . -4)(z . 12) )
6:22:16
dmiles
(get-opv 'sym) ==> ((classof . claz_symbol)(name . "SYM")(PACKAGE . #<PACKAGE COMMON-LISP-USER>)(typeof . SYMBOL) )
6:25:12
Zhivago
The point being that if you re-use something but have class-of report something else, then it no-longer has the semantics of what you were re-using.
6:25:47
dmiles
well i wanted to proably switch out the underlying structure as i get more experience
6:26:40
dmiles
i started out using asserted relations (typically that is the fastest) ... but it was messing with my setcdrs
6:28:11
dmiles
now i on first setf move the property onto the dictionary .. so symbol-plist and values are on the dictionary
6:30:22
dmiles
(i just mentioning it because i wonder if i'd be happy ever putitng oeverthing on one datastructure)
6:32:25
dmiles
oh where i was going is that i have sometimes a tree of storage pacakge[string]->symbol->value->classof
6:38:07
dmiles
some props i jsut synthesize there: https://github.com/TeamSPoon/wam_common_lisp/blob/master/prolog/wam_cl/soops.pl#L372-L394
6:40:22
dmiles
the language does at least come with a locative :P https://github.com/TeamSPoon/wam_common_lisp/blob/master/prolog/wam_cl/typeof.pl#L21
7:00:35
dmiles
has anyone done a setf expander that is based solely on gettign first class locatives?
7:02:40
dmiles
i am temped to continue doing it traditionally first with (get-setf-expander ..) returning a function call.. but would be nice so see something else
7:05:02
dmiles
in prolog a variable, is insufficent was a locative i have to use '$LOC'(TermPtr,Offset) % a bit like C.. but at leas the GC moves my pointers
7:07:59
dmiles
so it looks like: TermPtr = cons(a,b), L = $LOC'(TermPtr,2), set_locative(L,b2), to 'replcd
7:19:12
beach
jackdaniel: He is making statements about your ancestors and your dexterity, as compared to his own.
7:22:24
dmiles
There is a very strong genetic line that has moved in and accrossed europe over the centuries.. many of the people ended up in russia (my fammily did)
7:23:14
dmiles
The head of Cycorp and many many programmers that have been hired at cycorp have ended up comming from this genetic line
7:24:45
dmiles
most of us are either left handed or close to it.. actually a few yearts ago at a dinner 25 out of 28 of us relize we are all left handed
7:27:03
dmiles
yeah.. i was trying to confirm it was "a thing" i could point jackdaniel's to and ran acrossed the high mobidity rate :(
7:44:16
jackdaniel
Xach: regarding top 100 downloaded projects in Quicklisp: you count raw downloads, or things which are quickloaded? For instance if I type (ql:quickload 'foo) and foo depends on alexandria (so both are downloaded) – is the counter increased for both foo and alexandria?
7:52:40
aeth
This... is incredible really. I can copy the basic concept of with-accessors (let above a symbol-macrolet above the body) and abstract everything, with the only issue being that M-. doesn't like it, i.e. M-. looks for the definition of the symbol and fails to find it because the symbol is just a shorthand for some accessor-like form.
8:22:10
aeth
All that's left to settle is how many ;s to put at a comment that serves as a section title within a long file.
8:25:17
whoman
i have a feeling that some people have gone crazy or are going crazy with lisp. shoehorning it into how they are thinking, rather than kind of meeting half way. there is a secret whispering in the clay to be sculpted into this or that shape ... just got to listen... and get a decent keyboard.. =)
8:26:37
whoman
i wonder how the Matrix would benefit from our human intervention of giving values to numbers. especially them pretty pesky pickly adjectivies
8:28:43
whoman
if we cannot work out our problems linguistically aka symbolically, i strongly feel certainly confident that surely we are getting too deep into the self-projected communal holograms
8:30:44
whoman
lists/trees of words/symbols, which mean different at different times and contexts; eval, compile, read, write, etc... this lisp thing should be rationed to the common folk. digital acid over here
8:33:39
whoman
people be getting all stuck up in that there them ideas and ideals of real words meaning real things. its good to experiment. programmers are magicians =) the current alchemy is to.. something human something computer something earth something science... well breaking habits. we should give ourselves more credit - look at all of this stuff we made !! holy hell what potential is life! ahem i hope everyone is inspired to their magnum opus
8:39:35
shrdlu68
Hello, is anyone aware of a library that has functionality for generating combinations with repitions?
8:53:59
mrSpec
shrdlu68: I don't. But you know that you can use remove-duplicates with :test #'set-equal? ;)
8:54:37
alandipert
does anyone have ideas about how i could further speed the ev2! function? https://github.com/alandipert/advent-of-code-2017/blob/master/day05.lisp
8:55:54
alandipert
i tried inlining it in the run function, but that didn't seem to make any difference, so stuck with funcall
8:58:35
hajovonta
mrSpec: that would not remove one of '(a b) and '(b a) because these are not identical.
9:01:40
aeth
alandipert: That's because input is a global variable, and those are going to be slow because they're not lexical (or at least, the portable ones aren't)
9:02:48
hajovonta
XachX: even with custom test function, it's probably better to come up with a smart generation algorithm from a performance viewpoint
9:03:35
aeth
alandipert: why so many copies? You copy it twice, once when you convert it into a vector and then again with copy-seq
9:04:50
alandipert
aeth because ev2! mutates it.. i'm not sure the copies dominate since they're not in the hot loop (lines 36-37)
9:10:32
aeth
also, if you're sure they're always fixnum, try benchmarking with '(simple-array fixnum (*)) instead of 'simple-vector
9:11:08
alandipert
aeth the input is user-dependent, but mine is here if you want to try it: https://gist.github.com/alandipert/63e4502350d90d7b251e99cac40a3895
9:14:54
alandipert
the time to beat is 130 ms.. that's how fast a similar and type-hinted clojure algorithm does it on my machine. currently my lisp goes to ~230ms
9:18:52
beach
alandipert: Do you have to use simple vectors? It might be more efficient to use specialized vectors.
9:19:45
aeth
If you want to make a higher-order-function faster, you can usually do that by converting it into a macro, afaik.
9:20:03
alandipert
beach i don't think they have to be simple, i'm ignorant of this swath of CL :-) i went with simple because it seemed like.. the simplest
9:21:10
beach
alandipert: Well, in Common Lisp a simple vector has element type T. It might be better to use a vector type that stores numbers.
9:21:12
aeth
All I did was replaced the funcall with (progn ,@body) and now I define a function instead of passing in a higher order function. Real code should also use gensyms in macros.
9:23:22
aeth
Giving it a fixnum type doesn't make a big performance difference because fixnums are already unboxed (unlike e.g. double-float), fixnums will fit in a vector already, and it's being told they're fixnums with (the fixnum ...)
9:24:08
aeth
Even if it made no difference, though, '(simple-array fixnum (*)) in your coerce and removing the (the fixnum ...) feels more idiomatic
9:30:56
aeth
alandipert: Make sure that the length is known. That's the big thing for vectors. e.g. (declare (optimize (speed 3)) ((simple-array fixnum (1000)) code)) And use a macro instead of a higher order function, i.e. put the loop in a macro, and make the part that differs the body.
9:34:58
alandipert
aeth i can see how inlining via macro is faster, i was kind of hoping there would be a way to parameterize via defun name at least. and use (declare (inline ...))
9:35:25
alandipert
aeth https://gist.github.com/alandipert/a15a83cb49d9cabf95781b7d987bbcab is my attempt at that, no faster though
9:35:44
aeth
SBCL doesn't trust you, btw. It doesn't believe that it's not a fixnum because you're doing addition, afaik. When I disassemble, I see three object-not-fixnum-errors. If the numbers have a tighter bound than fixnum, you can save some checks each iteration of the loop (or at least, some of them might be in the loop)
9:38:45
aeth
i.e. instead of calling a function, you're inlining the whole thing into the loop iteration manually, with the macro.
9:40:00
aeth
SBCL cheats with its built-in higher order functions, and even then the cheating is a leaky abstraction and you can easily ruin performance.
9:44:58
alandipert
jdz its this clojure code, on java 9/macos, on i7 skylake https://github.com/bhauman/advent-of-clojure-2016/blob/master/src/advent_of_clojure_2017/day05.clj#L50-L59
9:52:17
aeth
And afaik you can meet the desired performance if you replace (evfun code ptr) with (progn ,@body) and redefine the functions to use this macro instead of to be passed in. Additionally, declaring it as a (simple-array fixnum (1000)) instead of * (unless it's going to be of a variable size)
9:54:05
aeth
You get a big, scary "SB-KERNEL:HAIRY-DATA-VECTOR-SET/CHECK-BOUNDS" in the disassembly without a known length, and it's every iteration of the loop because it's in ev2!
9:56:05
alandipert
jdz cool, i'll give that a look later. off to bed for now, thanks all for the help!
9:56:23
aeth
schweers: calling that function to check bounds 2000 times might have an impact in the performance if you're trying to minimize it
9:57:00
aeth
A lot of the time, it's not important. This is something that's trying to be made as fast as possible afaik.
9:59:29
aeth
Found the source: https://github.com/sbcl/sbcl/blob/7aa16ffc71fb5c04458297978292daa7f3f37dac/src/code/array.lisp
10:06:05
aeth
jdz's version gives me a consistent 0.69, which is twice as fast as the macro-ified version of ev2+run. Giving it a known length of 1000 speeds it up to about 0.64. More than I thought, but fairly insignificant. It gets rid of several bounds checks, but the bounds checks probably aren't being called every iteration anymore.
10:06:58
aeth
The macro version probably gave such a dramatic improvement because the compiler didn't have to bounds check as often.
10:09:22
aeth
known size is 2 bounds checks and 3 fixnum checks; unknown size adds 2 bounds checks; safety probably removes all the checks because the algorithm is flawless
10:12:34
alandipert
jdz your code is awesome, thanks for sharing. i like not using fancy loop stuff. ok now to bed for real :-)
10:12:36
aeth
You might be able to remove some more checks if you're clever, without resorting to safety 0. That can get you some of the way there.
10:13:37
aeth
e.g. maybe they're 32 bit signed integers? Then maybe the compiler can assume fixnum longer.
10:15:39
schweers
aeth: your measurements pretty much confirm what I suspected: bounds checks are not that expensive, since running times of programs on modern hardware are pretty much limited by the speed of memory operations.
10:19:57
aeth
Individually, the checks are pretty cheap. Collectively, they apparently add up. I don't get half the time, though. I get 0.04 seconds in the safety 0 version. And half the checks that are being removed are checks for fixnum.
10:20:57
schweers
you can go from 0.69s to 0.04s? not bad, that on the other hand is not what I expected
10:22:29
aeth
Also, that's why you should always get in the habit of putting the leading zero! 00.64 would have been obvious
10:25:50
aeth
Here's the overhead, btw: (let ((foo #(0 1 2 3 4 5 6 7))) (time (dotimes (i 10000000000) (sb-kernel:hairy-data-vector-set/check-bounds foo 4 42))))
10:29:04
aeth
Or if that's not good enough: (let ((foo (coerce (loop for i from 0 to 10000 collect i) 'simple-vector))) (time (dotimes (i 10000000000) (sb-kernel:hairy-data-vector-set/check-bounds foo 400 42))))
10:30:34
aeth
In order for it to matter, your data has to be huge or you have to have a very strict time constraint (maybe 200 frames per second?)
11:15:23
shrdlu68
When working with arrays of integers less than 128, is there a performance benefit in using :element-type '(unsigned-byte 7) rather than '(unsigned-byte 8) ?
11:17:42
schweers
according to the CMUCL manual there are (or were?) even implementations which used even worse representations in such cases
11:20:14
jmercouris
I have the following code https://gist.github.com/78247fc1030e208f023c50323d46b3e8
11:20:59
jmercouris
Additionally, the contents of "help-contents" is not actually being used within ps:ps somehow
11:23:49
schweers
if it’s a macro, and doesn’t use its argument ... on the other hand the SETF expression does use it anyway. I’m also confused
11:26:37
mercourisj
sorry, my internet keeps cutting out, I am checking the logs though, so don't worry
11:27:03
mercourisj
from the project description: "The ps macro takes Parenscript code in the form of lists (Parenscript code and Common Lisp code share the same representation), translates as much as it can into constant JavaScript strings at macro-expansion time, and expands into a form that will evaluate to a string containing JavaScript code."
11:31:34
jmercouris
Whereas it should look like "document.body.innerHTML = "contents of help-contents variable";"
11:36:04
schweers
shrdlu68: As far as I know this cannot be done. I wish there was a way though. Or rather I wish this could happen automatically.
11:37:15
schweers
jmercouris: can you verify that SETF has any effect? Also, have you tried factoring the (PS:PS ...) expression into a function and calling that with HELP-CONTENTS as an argument?
11:38:46
jmercouris
schweers: I haven't tried that, but I don't see how that would help, I didn't write ps:ps it is part of parenscript
11:39:40
schweers
I know, but I don’t know anything about parenscript, so I have no idea what the macro actually does.
11:39:42
mercourisj
schweers: setf most definitely has an effect, if you look at the outputted html it has an assignment on the innerhtml of the body
11:43:44
jmercouris
schweers: Correct, it must be in JS the point is to set the text of the web page
11:44:18
jmercouris
schweers: Correct, it must be in JS the point is to set the text of the web page
11:50:54
mercourisj
If you look in the reference here:https://common-lisp.net/project/parenscript/reference.html
11:51:14
mercourisj
You'll see the section "Symbol Conversion" which is what is happening with the value of help-contents, rather than an eval to its value
11:54:09
jmercouris
alright, so reading the manual again, I retried ps:lisp, apparently ONLY at runtime does the lisp form get evaluated, so I was doing the macro expansion and it looked wrong to me, but did actually work
11:54:41
jmercouris
so to anyone reading the log in the future, the following is how I solved my problem (ps:lisp help-contents) instead of just using help-contents which was being translated to a js symbol helpContents