libera/#commonlisp - IRC Chatlog
Search
13:41:13
jackdaniel
I've only plagiarised Bike's subconcious conclusion where he hinted remove-duplicates
13:41:14
phoe
jackdaniel: ...the only issue is that EQ won't work, because R-D is allowed to cons up a new list that is EQUAL but not EQ
13:42:07
jackdaniel
(most likely, if there are no duplicates, the original list will be returned though; sure, relying on undefined behavior is bad)
13:43:17
resttime
I'm curious, what use cases have/are there to define new SBCL intrinsics? Been reading a little about them, trying to figure out the 'what', and figure that knowing more 'why's would help. Here's one I found https://pvk.ca/Blog/2014/08/16/how-to-define-new-intrinsics-in-sbcl/ so my current impression is that it's modification of the assembly that gets generated on output
13:44:49
jackdaniel
i.e when you have a running system with a bottleneck but you can't avoid the particular calculation - then you may try to squeeze something that way
13:48:06
resttime
So would an example be like if (optimize (speed 3)) is not fast enough, then delving into intrinsics lets you optimize (or at least modify the DISASSEMBLE) even further?
13:53:05
resttime
Hmmm, might have to look more into it. I somewhat been learning how to optimize vector maths for my raytracer. Declared types, ftypes, speed 3, safety 0, dynamic-extent and stuff wherever I could. I hope intrinsics stuffs aren't too advanced for me
13:55:35
phoe
resttime: I know that heisig has been pushing SBCL to its limits with regard to vectorized code, maybe he can give you some pointers
13:58:21
_death
sbcl also uses hash-tables sometimes.. makes me wonder why I remember remove/delete-duplicates performing badly (doubt I was using a nonstandard test)
14:01:27
resttime
Oh, okay I'll throw out a question out here after working through rest of tutorial. Been following this: https://raytracing.github.io/books/RayTracingInOneWeekend.html but using Common Lisp instead. Current performance is 'decent' such that I can get something like a 400px wide render in 'decent' time but starting to slow down as more features like materials are added.
14:01:56
resttime
I figure it can be much faster somehow and it might be the vector maths as bottleneck
14:08:13
phoe
Bike: thanks for that SICP pointer, I've managed to recreate the standalone REQUIRE in https://plaster.tymoon.eu/view/2789#2789 - my only worry is if I should define what happens when there is a REQUIRE without any AMB, since in my current implementation it means THROWing without a matching CATCH
14:08:51
phoe
and I've written some basic tests from the SICP exercises, more might come later - triples and dwellings and 8 queens are probably the most important ones
14:09:52
sm2n
resttime: As someone who did the same thing, iirc the biggest wins came from declaring float array types and inlining
14:10:43
sm2n
But the one I was working on also used GFs, so there was some balance between ergonomics and performance-at-all-costs
14:15:37
heisig
sm2n, resttime: SIMD vectorization and manual/semi-automated lowering of AREF to ROW-MAJOR-AREF can boost the performance a lot, at least for number crunching.
14:16:08
resttime
jackdaniel: sm2n: ah yeah I did do a bunch of profiling and inlining as well before adding materials: https://i.imgur.com/S8gKKDh.png MAKE-RAY and MAKE-HIT-RECORD are default DEFSTRUCT constructors, I dunno how to make these faster at all. My vectors used to be defstruct too then I switched their implementations to simple-array single-float after some advice and similarly found super fast
14:16:10
resttime
speed up. Maybe I need to do it again somehow with these too then redefine how my vector operations work on this theoretical new representation of data
14:18:04
heisig
resttime: The current state is at https://github.com/marcoheisig/sb-simd. But I recommend waiting a few more weeks until it is published.
14:19:17
heisig
This also means now is the time to send me some feedback :) If anyone on this channel has some time to read/think things through, I'd really appreciate it.
14:19:56
heisig
The preferred format for handing in feedback on sb-simd is via Github issues, or by writing me on #petalisp.
14:21:41
sm2n
resttime: what are you calcuating in MAKE-RAY? mine just makes a unit vector from a direction and sticks it in an object with an origin point
14:23:29
resttime
heisig: I wish I was experienced enough to provide feedback xD but I'll make sure to check it out
14:24:22
heisig
mfiano: SBCL is not good at open coding AREF. If you hoist all stride and index calculation outside of the innermost loop, and replace (AREF A i1 ... iN-1 iN) by (ROW-MAJOR-AREF A (+ ... iN)), the code usually runs much faster.
14:24:25
resttime
sm2n: Yeah I think so, let me double check and I've definitely inlined whole bunch of stuff too
14:26:05
heisig
mfiano: Doing so by hand is tedious, but I am working on a loop optimization framework that does it automatically.
14:27:42
sm2n
heisig: How difficult would it be to add support for other archs? For example, aarch64
14:29:28
heisig
sm2n: Extending sb-simd is really simple, adding FMA on x86-64 took me less than a day. The tricky part is to get the SIMD instructions into SBCL in the first place.
14:30:35
sm2n
I see, I'll download it and play with it a bit. Don't know if I'll have any meaningful feedback though
14:30:53
resttime
sm2n: Oh misunderstood question, yeah MAKE-RAY just default constructor origin is either at the camera or the point on object the ray scatters/bounces off. Direction is where camera is shooting ray or calculation off of the surface scatter/bounce
14:32:43
jackdaniel
also, if you don't specialize on the structure class, then you may request the allocation to be a vector
14:40:25
resttime
jackdaniel: Hmmm, gonna have to think about how to reuse object if possible. I did do something similar althought just for my vector maths lib where possible with a pattern to supply a third parameter to set in a function like (vec3 out a b) with inspiration from https://github.com/mfiano/origin (thanks mfiano, gave your repo a look a while back for an example). What do you mean about
14:40:28
resttime
specialization and request allocation to be vector? (unfamiliar with it, wondering if it's just type declarations)
14:41:32
resttime
I went through the whole Raytracing in the weekend series in Rust before, when I added parallelism it was super easy, hoping it'll be same
14:44:19
rotateq
resttime: there's still a video series on youtube about how writing a raytracer in CL from which i learned much :)
14:48:17
resttime
Found it, section on structures here: https://github.com/guicho271828/sbcl-wiki/wiki/Object-representation-(lowtags,-widetags,-memory-consumption,-arrays)
14:56:20
resttime
Oh yeah, I think the one by Lehmann is only one I can find. I'll make sure to check it out thanks!
14:57:01
sm2n
I think jackdaniel was assuming you were using a structure-object, not a struct? Not sure, I'm not really familiar with structure-objects
15:01:51
jackdaniel
and I've meant, that when you use the operator defstruct, you may add a type parameter, then it will construct i.e a vector, not a structure-object when created
15:06:01
resttime
jackdaniel: Ooooh interesting, I did not know that was possible. I had only been declaring types for the slots
15:06:34
Bike
it kind of makes it a different operation. instead of defining a new kind of storage, you're defining a bunch of accessors for a normal vector or whatever.
15:07:00
phoe
sm2n: I don't understand the difference then, every instance of a class defined via DEFSTRUCT is a STRUCTURE-OBJECT
15:07:15
mfiano
You can however use (vector size) and then create an alias with deftype to be a simple-array, and SBCL will be smart enough
15:09:26
jackdaniel
" type---one of the type specifiers list, vector, or (vector size), or some other type specifier defined by the implementation to be appropriate. "
15:09:58
mfiano
resttime: Note however, that doing this means you can no longer specialize on a nominal type.
15:10:06
sm2n
phoe: huh, for some reason I thought there was a difference. Never realized you could dispatch on structs
15:10:08
jackdaniel
and it seems that sbcl assumes that in (vector size) size is the type specifier, i.e (vector fixnum) :)
15:10:43
phoe
sm2n: you can! structure classes are classes like all others and they obey CLOS dispatch normally
15:10:48
Bike
yes, well, down the page in defstruct it says you can do list, vector, or (vector element-type)
15:11:05
Bike
which is less dumb than the "arguments and values" section saying (vector size), since that's not actually a type specifier
15:12:18
resttime
mfiano: looked up nominal just now, not exactly sure meaning, is it that the defgeneric/defmethod functions won't work anymore then? (unable to distinguish the type)
15:12:27
phoe
I wish DEFSTRUCT was actually split into a proper chapter and glossary like all other parts of CL
15:12:55
mfiano
resttime: Using standard-object or structure-object means that you can create custom printed representations without clobbering existing vectors'. I mean you could use the priority argument to set-pprint-dispatch, but...
15:13:52
Bike
me, i think the defstruct that defines accessors for sequence type should be split off, and the normal defstruct should just be a :metaclass option to defclass. i can only dream
15:14:07
mfiano
resttime: A structure with a :type specified is no longer a user type, basically. It's just an alias with convenient named accessors (in addition to integral indexing)
15:14:13
sm2n
Wait this is actually really neat — you could use this to destructure some fixed size binary protocol/format trivially
15:14:52
user_
hi, anybody willing to hint me regarding fast-http and parsing request from string? I am using (parse-request (make-http-request) callbacks *request*), where callbacks is made with make-callbacks and *request* is a string of request (= "GET / HTTP..."). But I am getting invalid HTTP method error, what am I doing incorrectly?
15:18:09
resttime
Oh hmmmm, I'll have to think about tradeoffs with design and such if DEFSTRUCT isn't a user type
15:18:40
jackdaniel
this is only useful when you'd use bare vectors either way - you have somewhat useful accessors
15:19:16
mfiano
Common Lisp has a weird hybrid of nominal and structural typing, and this is one area that is troublesome in that regard.
15:20:08
mfiano
I would use a regular structure-object if you want a print-object method, rather than increasing the priority argument for set-pprint-dispatch which is a hack
15:21:14
Bike
in what sense does CL have structural typing at all? i mean, you can define generic functions on whatever, but that's not part of the type system
15:25:44
jackdaniel
hands on, who's staring at "Structural type system" article on wikipedia right now? :)
15:26:54
mfiano
with a nominal type system (deftype foo () 'bar) would be a distinct type, and not just an alias to the structural of BAR
15:29:17
Bike
anyway, deftype is just a macro mechanism for types. it's just like a type alias in c or haskell, not really even a factor in the type system per se
15:30:13
Bike
it doesn't actually let you define a "new" type in a meaningful way, unlike defclass for example
15:31:55
mfiano
Right. What I was getting at earlier is a non-structure-object struct's name behaves just like this nominative aliasing
15:33:06
Bike
the page has an example (defstruct (binop (:type list)) ...) and says "binop is a conceptual data type in that it is not made a part of the Common Lisp type system. typep does not recognize binop as a type specifier, and type-of returns list when given a binop structure. There is no way to distinguish a data structure constructed by make-binop from
15:36:45
Bike
the type of a lisp object is generally closely tied to its class, which is like the name in a nominal system. in a structural type system there is no dependence on classes in this way
15:38:24
Bike
say (struct foo) specifies the type of objects with a foo accessor. then if a function is specified to take a (struct foo) argument, you can just as well pass in a (struct foo bar) without bothering to "define a subclass" or whatnot
15:41:44
Guest74
heisig: anywhere to look on this framework you're working on? Could always insight to help optimize my stuff. I only use 1d arrays anyways to represent 2d rasters, but i'm sure my padding/width(to fake conformal arrays) stuff could probably be better.
15:43:23
Bike
might be kind of neat if the lisp type system could be extended in some way like this, but it can't, oh well
15:44:35
jackdaniel
you could define (deftype has-foo (object) `(find-method (... standard-reader-method ... ) ))
15:45:28
Bike
well ok, sure, satisfies lets you do literally anything, and you could probably finagle it to work with subtyping
15:46:59
Bike
maybe extended sequences are a good example? crhodes's extension lets you specialize protocol methods to treat whatever as a sequence... but to make it work fully, you also need to hook up the nominal typing element by making your sequence class a subclass of SEQUENCE
15:47:27
Bike
the type system works in terms of subclassing rather than "does it work with this protocol"
15:50:09
jackdaniel
I get that generic functions can't specialize on other types than classes (+ eql) - do we limit ourself to what clos recognizes as a specializable type?
15:51:14
Bike
structural types are usually not conceived of in terms of being able to test them at runtime, i don't think
15:51:44
Bike
(i mean, types in general aren't, but with a nominal type system you can just do tagging)
15:52:14
jackdaniel
so how extensible sequences are a good example then? the fact that /something in the system/ requires object to have a sequence as a superclass is an implementation decision?
15:53:14
Bike
well, what i mean is you could define your own class, specialize all the sequence generic functions, but it still wouldn't work unless you also did the subclassing
15:57:51
Bike
it takes some work to wrap your head around, especially if like me you haven't actually programmed with it
15:59:22
mfiano
I am understanding the general ideas here, being familiar with a lot of Blubs, but I am not very into PL or type theory.
16:04:38
mfiano
It's sort of unfortunate that all the interesting extensions to CL require implementation support: extensible sequences, Gray streams, PLN, etc.
16:06:53
Bike
if it can be done within the language we usually call it a "library" rather than an "extension"
16:08:07
mfiano
Although macros are often touted as a way to extend the language without implementation support.
16:08:54
yitzi
The Gray stream protocol doesn't depend on subclassing from from Gray stream classes.
16:10:28
yitzi
Technically if you define all the methods it should work, unless a particular CL implementation chooses to dispatch on the subclass.
16:10:46
mfiano
Well I inherit from trivial-gray-streams:fundamental-binary-input-stream in this one usage
16:12:13
Bike
macros are a way to extend the language without implementation support (well, beyond the implementation supporting macros). doesn't mean they can do everything
16:12:20
yitzi
The protocol proposal says that the classes are there for convenience so you can reuse `stream-write-string` etc.
16:13:11
yitzi
If you define `output-stream-p`, etc but have your own superclass it should still work.
16:13:30
Bike
but without subclassing from stream, any type declaration saying something is a stream won't be correct
16:16:48
Bike
well, the point is just that the actual type system can't know your thing can be treated as a stream without the subclassing
16:34:20
jackdaniel
class-name of anonymous class is nil, but is the term "named class" appropriate?
16:40:23
jackdaniel
yes, I saw this passage, I'm wondering whether "named class" applies to the class-name or the class name in the environment
16:43:16
jackdaniel
also "proper name n. (of a class) a symbol that names the class whose name is that symbol."
16:54:01
yitzi
Bike and mfiano: I was able to actually make a functional character output stream that does is not a subclass of stream on ECL that works. Not so on SBCL since it is doing typechecks in terpri, etc.
17:24:56
Bike
i see, and this works by specializing the mop method applicability methods. so typep wouldn't work. which is fine, it's still interesting. i should put more thought into what an extensible type system could look like.
17:48:21
phoe
jackdaniel: aren't all classes named in portable CL if we skip the MOP? even classes that have been removed from the global environment still have their names intact, AFAIK
17:51:15
jackdaniel
the gist of deliberations was whether the term "named" applies to the state in the environment or to the result of class-name invoked on the class
17:51:46
Bike
you can use (setf (find-class name) nil) and (setf (class-name class) nil) to remove any name association even without the mop.
17:52:02
Bike
and in fact it looks like there's a note on class-name saying "The name of an anonymous class is nil."
17:52:36
jackdaniel
nope, it is accessor: http://www.lispworks.com/documentation/HyperSpec/Body/f_opsetf.htm
17:52:36
phoe
also (defclass foo () ()) (setf (find-class 'bar) (find-class 'foo)) (class-name (find-class 'bar)) ;=> FOO
17:53:14
Bike
(setf find-class) mutates the environment but does not affect the actual class at all. (setf class-name) does affect the class (and not the environment)
17:53:28
phoe
"If S is a symbol such that S =(class-name C) and C =(find-class S), then S is the proper name of C. For further discussion, see Section 4.3 (Classes)."
18:24:28
jackdaniel
alright, starting from tomorrow I will be rather offline until after Holiday, so Merry Christmas and Happy Hannukah everyone :) o/
18:41:47
jeosol
I have a question guys relating to maintaining deployed CL resources. Now I have looked at two options: 1) Docker containers and 2) just a repl. For updates, rebuilding my images takes longer for sure (I also use multi-stage build) but with repl option, I push updated code to the machine and can send update command the running repl is updated
18:42:45
jeosol
Now I get for easier deployments, CI/CD, devops, etc, containers are better. I wanted what you guys are doing especially at is also related to image-based development in CL. Thanks
18:43:19
jeosol
I should add the second option appears ok for now, I just have a small machine (dev) and another for the prod testing, so nothing super complicated
19:51:04
opalvaults[m]
I'm a little confused on the use of a lambda here in Practical Common Lisp.... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/674d99d9c802ef3b4a88a04363f1155bb0fa6425)
20:09:18
opalvaults[m]
Bike: I feel like remove-if-not could also be used here to select optional arguments?
20:10:29
Bike
what's wrong with keyword arguments and what does that have to do with returning a function
20:16:29
concrete-houses
I remember paul graham saying in one of his essays that they didn't use sql database... in VIAWEB
20:17:23
hayley
And see <https://plover.com/~mjd/misc/hbaker-archive/letters/CACM-RelationalDatabases.html>
20:20:33
opalvaults[m]
Bike: nothing wrong with keyword arguments, I'm just curious if there's a way to select keywords out of a list of database entries in an easier way than using a lambda expression. Also, do arguments in a definition automatically get passed to a lambda function?
20:22:02
hayley
concrete-houses: Most of the time I am parsing whatever data format I am given, but arguably that's not a lot of data.
20:24:09
Bike
opalvaults[m]: for a real program using a database you would probably use a better representation. this is just an introductory example for the book.
20:28:12
opalvaults[m]
Bike: gotcha, okay i'll read on then. it looks like this intro is just a quick and dirty and not meant to explain everything all at once. that's interesting how lambda (cd) is able to parse over every argument passed into a procedure
20:30:24
opalvaults[m]
i get the scope of what it has access to, i suppose what is confusing and interesting to me is that lambda automatically iterates over all define arguments. it's not obvious that lambda should be able to do that coming from an Algol related family.
20:31:06
opalvaults[m]
I'm conditioned to need to iterate over **kwargs (or some such), and test accordingly
20:37:01
hayley
concrete-houses: To a zeroth approximation, having direct pointers rather than looking through an index would be faster. But I'd still like to see that if the join exposes multiple matches, i.e. the program follows a one-to-many relation.
20:39:35
concrete-houses
interesting that the wikipedia page for transitive closure mentions graphs -- aren't they all the buzz now?
20:40:26
concrete-houses
wow maybe I have been watching too much Alan Kay but it seems a lot of decades old computing was smarter
20:43:24
concrete-houses
What did Paul Graham mean when he said we faked continuations with closures?
20:53:57
concrete-houses
Work does a hunting license lottery...... So they have all these licenses fir 4 seasons and different game.....and need to randomize for 15k or so custoemrs, and then the first 'randomly' pciked customer gets the license that is the top one in a list of licenses they want I think 10 it limit to that list.....as licenses are won the availabel licesne count must be reduced by 1 or if the winner
20:54:03
concrete-houses
is a team leader, by the team size, and each team member gets the same lciense.... now in our implementation the non team leader sare in the lottery which seems silly to me.... but would a lisper be able to get random numbers adn run such a lottery pretty easily?
20:54:51
concrete-houses
if the license count is not availble then the second choice is won, if available
20:55:13
White_Flame
concrete-houses: of course non-relational databases are faster. Data is clustered and in specific application-aware format
20:59:08
concrete-houses
https://plover.com/~mjd/misc/hbaker-archive/letters/CACM-RelationalDatabases.html yes I think he does.
22:18:55
scymtym
Bike: the protocol definition would also make associated specializers and type specifiers for the "roles" of the protocol: https://gist.github.com/scymtym/c4b842f04fe37ef52745d3ed3ea97e50
22:29:17
opalvaults[m]
i have to wonder if people who dive into CL just dread working in other languages and/or lose interest in all other languages after learning CL.
22:32:00
scymtym
Bike: mostly just CLOS and MOP. being able to spell the specializer like that uses SBCL's specializer parsing/unparsing machinery which is historically but not technically related to generalizers. that's only syntax though. one could construct the method by other, slightly less convenient, means
22:33:24
scymtym
and the type specifier is just syntax for (satisfies GENERATED-GENERIC-FUNCTION) where GENERATED-GENERIC-FUNCTION returns true for that same specializer and false otherwise
22:34:33
scymtym
so in summary, a class conforming to a "role" in a protocol can be "statically" checked, used as a specializer and used as a type
22:50:35
rotateq
hayley: i often see much good DSL potential as a niche one can do, maybe for the future
22:50:43
dre
opalvaults[m], I am a python programmer as well, but I'm learning common lisp and I want to find a way to cram it into more of my projects. (keep in mind, I don't often make projects)
22:51:21
hayley
In my opinion (which involves no experience with business), using a language you don't know at work is pretty risky.
22:52:55
hayley
I didn't like Java or Python very much when I did university courses with them, but I attribute that to the university.
22:54:06
rotateq
yes here at the university there's a "functional programming" course with Haskell (of course) but the prof is not good at all and this gets the people a superficial idea of things
22:54:06
dre
hayley, i really liked python, at university, simply because it was an alternative to matlab.
22:54:09
hayley
But trying to do interactive development in Python was tricky, as it appeared things would refuse to reload unless I didn't do a topographical sort in my head.
22:55:18
hayley
...and we actually had to write somewhat fast code for one assignment, which scared me, but I somehow managed to find the best solution by accident, and the performance problems were really in a library we didn't write.
22:56:00
dre
I had like, half a day left and we had to implement some sort of robotics algorithm. There was some "matlab on computer to rpi robot, sent over wifi" contraption that was a pain
22:56:32
dre
and we were like, getting so late, and panicky about it (cause robot is not moving, close to deadline)
22:58:17
dre
and I'm like "welp nothing to lose lets try this" - python on robot. "if direction is not None and indicator is type(int): foo()" -- worked, saved the assignment, and have been diggin python for a while since.
22:59:05
dre
but yeah, gotten to the point where there's complaints, and having dabbled in lisp, I get it; I have my foot in the door, and I want more :)
23:04:22
dre
as a newcommer, Land of Lisp was a much more fun start into the language, but PCL has some great tibbets in it
23:05:57
dre
I'll probably work my way though PCL soon. I did some of AoC in common lisp which was also a great experience
23:06:14
hayley
It's very loose with names. Been a while since I read it, but beach had a list of mistakes in the first few chapters <http://metamodular.com/Books/land-of-lisp.html>
0:14:37
opalvaults[m]
*no-defun-allowed* I love that guys writing. I believe he wrote a paper about a common lisp operating system that I really liked.
0:15:05
opalvaults[m]
hard to find through the website, thankfully i bookmarked it. For anyone interested: http://metamodular.com/closos.pdf
0:28:21
Spawns_Carpeting
I am not exactly sure what the type is in the first place, whether its a vector or hwat
0:29:00
Spawns_Carpeting
i could setup a local irc server for testing but it would be easier to uh, just know the type
0:29:30
Xach
Spawns_Carpeting: it's been a while, but i believe it's a vector of octets. octets-to-string will convert it if you like.
0:35:51
Spawns_Carpeting
how can I use sb-bsd-sockets inside of an asdf package? Do I need to :use something, or just require it?
0:43:21
Spawns_Carpeting
I am trying to logically work my way through this so I know how to do it in the future. I see that sb-bsd-sockets is itself an asdf "system", but I am not sure if the :use part of a defpackage imports systems, packages, or what exactly
0:45:35
Xach
Spawns_Carpeting: so, if you have a project with an .asd system file, :depends-on (#:sb-bsd-sockets) is a thing you could have in your file.
0:46:11
Xach
to refer to external symbols in the sb-bsd-sockets package without a prefix, you could :use #:sb-bsd-sockets in your defpackage form.
0:48:24
Xach
Spawns_Carpeting: it makes it so you can refer to their external symbols without a package prefix
0:48:47
Xach
so after (defpackage #:foo (:use #:cl)) (in-package #:foo), you can refer to CL:CAR without the "CL:" part
0:49:15
Xach
If you don't :use, you can still refer to them just fine - but you have to write the prefix.
0:49:59
Xach
there are a few ways to manage that sort of thing. importing is one, and shadowing comes into play when you :use multiple packages when there's a conflict
0:56:00
Xach
Spawns_Carpeting: it's a way to use a symbol for its name without interning it in any package
0:56:20
Xach
the real critical part is the name of the symbol - the #:... is syntax for a particular kind of symbol.
1:05:53
rotateq
dre: this dialect seems to be domain specific for programming robots too: https://staff.aist.go.jp/t.matsui/eus/
1:13:06
hayley
Per https://staff.aist.go.jp/t.matsui/eus/manual/node6.html#SECTION02013000000000000000 it seems unlikely.
1:28:50
rotateq
there's this interesting paper metacircular semantics for CL special ops surely you know it