libera/#commonlisp - IRC Chatlog
Search
1:10:54
Josh_2
I keep getting a very annoying bug when making HTTP requests using Dex I get an error complaining about writing to closed SSL, https://plaster.tymoon.eu/view/2564#2564 however if I try the request once again it works just fine, five minutes later it will signal that error again..
1:21:46
etimmons
Josh_2: Try with `:use-connection-pool nil`. Dexador's connection pool support is broken and unsafe in general.
1:23:48
etimmons
No idea. It's also worth noting it's possible to do connection pooling in Drakma, it's just not built in.
1:24:19
etimmons
I tend to only use Dexador on Windows since its story for https on Windows is better
1:27:38
etimmons
I wish I had the time to integrate winhttp into drakma, then I wouldn't have to worry about dex at all
1:32:10
moon-child
the sicl meme precedes it. I kinda wonder how that happened--it's not like it has a large marketing budget
3:07:56
beach
moon-child: I think it's the published papers. They have been peer reviewed, so there must be some substance there.
3:10:26
pillton
I have a design question which relates to a chronology I have observed many times. An application originally invoked a function f(x). Over time, the function changed to f2(x) = h(x;p) where p is a parameter provided by the user of the application. My question is, how would you structure the application given that f(x) and f2(x) or potentially f3(x) are useful implementations? Would you 1) add a slot to the application state which stores
3:10:26
pillton
a closure which closes over the parameter p or 2) implement f(x) using a dynamic variable *VAR* which invokes the generic function (EXTENDABLE-F X *VAR*) or 3) just change f(x) to h(x;p) and provide a default value for p or 4) something else?
3:14:11
beach
pillton: I think the right answer depends a lot on the details. But I can tell you that, for SICL, we have developed a style that is close to number 2, but not for the reason you give. We have a first argument CLIENT that the library does not specialize to, and client code can write methods with specific clients.
3:15:53
beach
This style allows us to provide lots of useful default functionality in the library, while still allowing complete customization for clients who need it.
3:17:37
beach
This style also allows the same library to play different roles in a single Common Lisp image.
3:24:53
beach
I guess that's true. But I didn't generalize what Josh_2 said to be a "positive public image".
3:29:30
moon-child
I feel that there is a general perception of sicl as somewhat of a holy grail in lisp implentation, and that this perception has more than a little to do with performance
3:29:48
moon-child
how many times is it now people have asked whether sicl is ready yet, or when it will be ready? :P
3:45:56
beach
It is true that we have come up with techniques that are likely faster than the analogous technique used in current Common Lisp implementations (like generic dispatch, path replication, call-site optimization), but my main goal is for SICL to provide excellent debugging support.
3:48:41
beach
In fact, until I came up with the call-site optimization technique, I was convinced that SICL would never be able to compete with the best existing Common Lisp implementations in terms of performance.
3:56:44
beach
Another issue is that, although we are getting closer to generating a native executable for SICL, there is a lot of work left to implement all the optimizations we want, and that is going to take time.
4:11:15
phantomics
One key to performance will be allowing the generation of ASM code like SBCL does, I think I asked you about that before
4:12:10
beach
That's an easy one. And it is taken care of. But that alone won't imply good performance.
4:12:35
phantomics
People have complained about the stagnation of CL, but the define-vop in SBCL is a major point of distinction that makes much higher performance possible, it could lead to a division of CL between SBCL and non-SBCL
4:14:17
beach
I don't see that. The Cleavir compiler framework is quite easy to customize, but it doesn't have a define-vop.
4:19:24
beach
Well, first of all, we don't generate "ASM". We generate a graph of standard objects representing "instructions" that are closer and closer to native with different passes. And each pass is defined by a bunch of generic functions that can be customized, so that new "instructions" can be provided.
4:20:06
kakuhen
Is there a function like FIND-CLASS that does not signal error if class is not found? I know that you can write (find-class 'name nil), but it does not feel "idiomatic" to me.
4:20:09
beach
I haven't studied SBCL internals much, but I suspect define-vop is just an implementation echnique.
4:22:08
beach
kakuhen: It is a slippery slope to start writing code for people who might now only part of the language.
4:41:58
beach
phantomics: SBCL is great, of course, but there are some issues with it as well. For one thing, the garbage collector is not that great. And the technique used for generic dispatch was designed at a time when memory access and register operations were roughly equally fast. For call-site optimization, SBCL addresses a number of special cases, but in general, I don't see any attempt to get good performance out of keyword arguments an
4:45:45
phantomics
beach: I see, I'm sure there's room for improvement. It seems like SBCL is ahead of other impls in the area of generating custom ASM, which you need for SIMD. That's why things like the sb-simd project are SBCL-only, and why if I add SIMD support to April it may only be available in SBCL. It would be great to see more support for use cases like this
4:47:33
phantomics
Also, all CL impls have some shortcomings related to array handling: for instance, you can't have a vector of 8 64-bit ints and create another reference to it as a vector of 64 8-bit ints like you can in C, this would help in some situations
4:55:56
beach
Most such "shortcomings" are not about the implementation(s), but about language semantics. This particular case might be possible, though.
4:57:16
kakuhen
a little birdie told me sbcl is getting a new garbage collector and it's apparently going to be great
4:58:40
|3b|
i suspect sb-simd being sbcl only is more about # of users than anything else (and ecl has had a simd extension for a while i think)
4:59:39
|3b|
(and also that "sbcl is fast" so the users that care about performance enough to write simd code are using sbcl)
5:00:41
kakuhen
there's pretty much only three reasons i use ccl: objective-c bridge, more "efficient" gc, and starts up faster on my computer; it seems inferior to sbcl as far as number crunching and saving heap images goes
5:01:00
kakuhen
so for instance, on sbcl you can save a heap image and compress it down quite a bit, with minimal impact on performance; ccl has nothing like this
5:02:51
|3b|
the nibbles library helps with a lot of the situations where you want to read a chunk of memory as various types, though not transparently and you have to start from a vector of 8-bit ints
5:04:30
|3b|
(and most of the cases where i care about that sort of thing, transparent "8x64-bit <-> 64x8-bit" isn't what i would want anyway, since i usually care about "Nx8-bit <-> some arbitrary layout of mixed types")
5:07:43
|3b|
i think ccl has "LAP" for defining things in asm, and according to the docs that supports altivec on ppc, so you could make a portable simd lib that supported that if you happened to still have some PPC somewhere :p
5:15:12
kakuhen
i was planning to build a small blackbird workstation, but the power9 chips are on backorder
5:15:39
kakuhen
the power architecture is forwards compatible, based on what little i was able to find online
5:37:25
beach
kakuhen: That's great news about the SBCL garbage collector. Do you know any details about it?
5:38:40
hayley
|3b|: Right, there's a Lisp assembler in Clozure. Though I don't know if the assembler knows any SIMD instructions on PowerPC or x86-64. (It does, however know about the instructions to do an atomic increment rather than a CAS loop, but that is something else.)
5:39:36
hayley
beach: SBCL is going to support use of the Memory Pool System. The Boston Lisp Meeting I attended was about it, and I believe I provided a link for the slides...
5:41:26
hayley
I forgot exactly how DEFINE-VOP works, but it ends up generating another kind of instruction, which eventually is replaced by a user-defined sequence of instructions. This would not be awfully difficult to do in Cleavir.
5:42:55
hayley
We could create a HIR instruction class which eventually is replaced with a sequence of LIR instructions that the user provided -- well, yes, that's pretty close.
5:45:45
hayley
But generally I hope that DEFINE-VOP is the last resort for doing SIMD work; rather the compiler should vectorise where possible, and then a programmer could use "intrinsic" functions like in C or SB-SIMD if the compiler cannot vectorise for them.
5:47:25
hayley
Such functions are probably more important in Common Lisp, as they can perform type and bounds checks and are generally safe. Although VOPs do require precise type definitions, it is not hard to generate completely broken and unsafe code.
5:51:16
hayley
Of course, in an APL compiler like the one phantomics is writing, there are relatively few tight loops that need assembly to be written for them. But otherwise measuring the performance of anything with explicit SIMD instructions tends to be more of a challenge of who memorised the fastest instructions.
5:55:15
kakuhen
I was using remove and comparing the car of each cons cell; but this kept giving me an error that the alist was not the type SEQUENCE
5:59:15
kakuhen
ok thanks so much; this solved my issue... not using #'car in particular, but I put in a small lambda that did exactly what I wanted. thanks
6:44:37
kakuhen
so I was adding custom conditions and signalling errors, and offering restarts for these errors
6:44:48
kakuhen
in some random cases, the restarts simply wouldn't appear, and then the restarts would provide errors
6:47:27
hayley
It could help to do the work of the macro in a function, and then have the macro expand to something like (call-with-foo (lambda () body ...)), provided that there is enough code that doesn't need to be generated every time.
6:58:42
beach
Speaking of SIMD, one thing I would like in SICL some day is a set of excellent FFT functions, and those would very likely use SIMD.
7:11:56
hayley
I have an implementation of the Cooley-Tukey algorithm written in Petalisp, and Petalisp will likely be an early user of SB-SIMD.
7:16:35
beach
That's optimal with respect to arithmetic operations as I recall, but hard to manage because of the space.
7:49:57
asarch
If I have: '(:tacos 10 :beer 12 :pizza 8 … :tacos 7), how could I get the second :taco value?
7:50:53
beach
Use POSITION to find the position of the first one. Then use FIND with a :START position beyond that.
7:51:53
mfiano
That requires repeating traversing the intersection. Instead, just walk it until you find it "again". A good job for LOOP
7:54:29
beach
Or you can use MEMBER to find the first one, and then MEMBER on the CDDR of the value the first MEMBER returned.
8:00:09
|3b|
there is also GET-PROPERTIES that returns the tail of the list starting at the found property as 3rd value, and could be used similarly to member if there is a risk of the key also being a value
8:07:00
hayley
Sorry to ask, is this representative of any real data you are playing with? Would it be easier to structure the data we are searching some other way?
8:07:28
pjb
asarch: (member :tacos '(:beer 12 :prefered-meal :tacos :tacos 3)) #| --> (:tacos :tacos 3) |#
8:08:03
|3b|
for example start from (loop for (k v) on plist by #'cddr when (eql k key) collect v) to get all the values for a key, if the task isn't always "get the value for the second instance of the key"
8:08:49
pjb
asarch: but indeed, the point of a-lists and p-lists, is to shadow the following keys, so it may be preferable to structure your data differently if this is a normal thing you need to do.
8:08:56
hayley
Say, are we dealing with separate collections of foodstuff, which might be represented as '((:tacos 1 :pizza 3 :beer 4) (:tacos 4 :ramen 4))?
8:09:42
pjb
or: (let ((plist '(:ramen 2))) (push 3 (getf plist :tacos)) (push 4 (getf plist :tacos)) plist) #| --> (:tacos (4 3) :ramen 2) |#
8:09:44
hayley
Perhaps I shouldn't ask, but it seems like a weird thing to do to retrieve the second value, as pjb said.
8:10:06
asarch
Yeah: '((:tacos 1 :pizza 3 :beer 4 :tacos 4 :ramen 4) (:tacos 1 :pizza 3 :beer 4 :tacos 4 :ramen 4) (:tacos 1 :pizza 3 :beer 4 :tacos 4 :ramen 4) …)
8:12:24
|3b|
ACTION would probably add a "clean up the data into an easier to use form" step between "query" and "use", or else fix the sql to generate what you want :)
8:15:07
asarch
However, when I parse the query, I get it as list in that form: (:tacos 1 :pizza 3 :beer 4 :tacos 4 :ramen 4)
8:17:12
hayley
Well, I'm not sure if I'd count on the order the columns appear to start with. Guess there is a defined order, but foo.tacos coming first in the list and bar.tacos coming later is too subtle.
8:18:18
hayley
(Though it is also weird for the database library to strip off the foo. and bar. off the column names in my opinion. And does your database care if you use a product then a WHERE constraint rather than JOIN?)
8:20:09
hayley
Again, sorry for the hard questions, but having to use such a list just sounds like a bad idea to start with.
8:22:57
pve
asarch: you should probably use result-type :lists instead of :plists, and examine the second return value to figure out what is where
8:24:29
hayley
I mean, the database would not let you make columns with duplicate names. So it is weird that the database library ends up producing duplicate names.
8:28:13
pjb
SELECT foo.tacos as footacos bar.tacos as bartacos foo.pizza foo.beer bar.ramen FROM foo, bar WHERE foo.baz = bar.id ORDER BY foo.id;
8:36:18
hayley
Still, doesn't the database name your columns foo.tacos, bar.tacos, foo.baz, foo.id, etc?
8:48:08
asarch
Well, it seems that I will have to do two separate queries and then join each result in a new plist
9:30:22
asarch
Yeah, it works: '(:tacos 1 :pizza 3 :beer 4 :tacos 4 :ramen 4 :taquitos 1 :tacotes 4)
9:37:04
hayley
https://plover.com/~mjd/misc/hbaker-archive/letters/CACM-RelationalDatabases.html provides some reasons, which I would summarise as that 1. the relational theory didn't come first and 2. joins are messier than pointers at times.
9:38:02
hayley
Personally, I prefer that good programs are based around protocols of functions, whereas SQL has no such concept and is only concerned with storing homogenous rows. This is called the "object-relational impedance mismatch". Some people use it to claim object-oriented programming is bad, but I use it to claim that relational databases are bad.
9:39:33
pjb
lisp has not. lambda was just a keyword, when he introduced it, JMC didn't understand lambda-calculus.
9:39:56
hayley
That doesn't change that doing things in SQL or Haskell can get very hairy if you're not doing "the right things".
9:42:41
hayley
I wonder if https://counterexamples.org/ has any bugs in Haskell. Some Haskell examples, sure, but I can't remember if there are bugs.
9:50:11
pjb
The point is that because you're have a good mathematical model doesn't mean that you have a practical system. The universe is still messy.
9:52:04
tfb
pjb: and if you try and program in a version of lambda calculus which *does* make sense mathematically, but leave out all the stuff you actually need, you rapidly find out how little the good mathematical underpinnings actually help you
9:53:53
moon-child
pjb: that doesn't explain anything. What does it _mean_ to have a solid mathematical base? I can model lisp, c, sql, haskell... in a mathematical fashion. So what distinction are you trying to draw?
10:19:24
tfeb
moon-child: it should mean among other things there is a consistent model which would le you predict the behaviour of a program (modulo halting problem etc)
10:19:42
hayley
Is a mathematical specification any better than any other specification in this regard?
10:21:05
hayley
https://plover.com/~mjd/misc/hbaker-archive/MetaCircular.html "we believe that a carefully fashioned system of metacircular definitions can achieve most of the precision of denotational semantics."
10:21:17
kakuhen
suppose I have a restart-case that skips executing some pure function entirely, but I do not want to abort evaluation; does it matter if I make this restart return t or nil?
10:22:39
kakuhen
basically, I have a macro that can add a key to an alist, but I want to avoid duplicate keys in the alist, so I signal a duplicate-key condition when someone attempts to insert a key that already exists
10:22:51
hayley
From memory, the RESTART-CASE evaluates to whatever the last form in the restart clause is. So it should return something that makes sense for the operation.
10:23:06
kakuhen
there are two restarts I have: one just deletes the key and its values entirely, and another one just "does nothing" and it's intended for when the user wants to cancel redefining the key
10:23:59
kakuhen
oh and to clarify, the former restart deletes the already existing key and then inserts what would've been a duplicate key
10:25:48
kakuhen
if i want to return t, it's probably more meaningful to return the original alist anyway
10:27:00
hayley
Well, if you have a pure function, then you are going to return the modified alist, right?