libera/#commonlisp - IRC Chatlog
Search
4:30:04
aeth
I just need to see if an exact list has shown up before and if it has return the value and otherwise do a thing and store the value
4:46:39
larix
i suppose there are enough situations where its faster to just use eql, as im assuming they compare addresses in memory instead of whole thing
4:49:13
moon-child
whichi s annoying because they could have been defined to consistently behave as though they had addresses with basically no performance hit
4:50:15
moon-child
semantically, you say that + etc. nondeterministically chooses whether to hash-cons its result or cons up a new one
4:52:37
jcowan
The trouble arises with EQUAL when you compare two circular lists. Supposing A is a circular list, then (eq a a) returns t, whereas equal does not necessarily return t
4:53:51
bike
"constructively implies"? is that a real term? it sounds almost real even though i just made it up
4:53:57
moon-child
(modulo https://outerproduct.net/trivial/2023-11-29_nonblocking-cycles.html, but)
4:55:00
bike
"eql returning true constructively implies equal returns true" would mean that if eql returns true, equal actually returns, and it returns true
5:04:13
jcowan
Scheme returns #t for (equal? a a) with two exceptions: circular objects that aren't eq?, and two NaNs
5:07:54
aeth
larix: the problem with equal vs equalp is... equalp is case insensitive, which is probably not what you want, but equal uses eql instead of = (so the type has to match) to test numbers when it crawls through your sequences recursively, which is probably not what you want
5:18:23
larix
my library takes a plist of options in defmemo, so i can just add :test #'your-equal as an option
5:18:36
aeth
a lot of people are probably OK with the numeric tower of everything ending up as a long-float (which is probably just the same as a double-float) eventually, while a lot of people want their single-floats as single-floats and their integers as integers
5:19:24
aeth
although the former would most properly be done as, as soon as a float shows up, replace the memoized rational (integer or fraction) with a float
5:25:42
jcowan
The nice thing about using EQUAL is that multiple arguments are handled correctly without extra ado
5:30:27
aeth
ah, I brought up hash tables (equal, equalp, or perhaps just nested) because I'm doing something more limited than a library
5:31:43
larix
jcowan yes, but im about to push a :test option for the defmemo so its more versatile (defaults to #'equal)
5:33:01
aeth
SPIR-V requires type metadata bound to an ID so if there's a vec3, that requires generating OpTypeVector <ID for 32-bit float> 3, which is my particular problem at the moment. I've never considered list-key hash tables before
5:33:43
aeth
This means that on the first sight of a vec3, it generates or looks up the ID for OpTypeFloat 32 and for OpTypeVector <whatever the ID is for 32-bit float> 3
5:34:25
aeth
and if it generates an ID, it also needs to generate the instruction that binds it in the SPIR-V itself
7:39:16
hayley
"nondeterministically chooses whether to hash-cons its result or cons up a new one" But I wanna hash-consing GC and/or copying immutable objects whenever I find it convenient.
8:22:23
rainthree
beach: https://paste.rs/s2Pc0 is this similar in any way to the way Common Lisp does it?
8:31:47
hayley
beach: It uses "mutable value semantics" which is mostly value semantics, but allowing references in certain cases which cannot cause references to alias or outlive their referencees.
8:32:09
beach
rainthree: But you are using "pass by value" and "pass by reference" in a way that is not what it means, though the C++ people seem to use it your way, thereby not respecting established terminology.
8:33:15
beach
rainthree: "pass by value" just means that the arguments of a function are evaluated before the function is applied. It says nothing about the nature of those values, that can be references or something else.
8:33:25
rainthree
they are using it that way, I only copy pasted what they said, and I hope to learn better the concepts and the right definitions. I am reading http://metamodular.com/Software-engineering/uniform-reference-semantics.html
8:35:08
beach
rainthree: "pass by reference" is when the caller passes a reference to an object. Again, it says nothing about the nature of that object. But then, typically, you can not use an arbitrary expression as the argument. It must be what is known as an l-value.
8:36:40
moon-child
in some languages, if you pass something other than an lvalue, a temporary location will be created
8:36:59
beach
Though early versions of Fortran always used pass by reference, even when the argument was not an l-value. So you could do something like f(1) and then in f (not using Fortran notation) f(x) {x = 2} and from then on, 1 would be 2.
8:37:42
madnificent
Searching for common lisp things online the top hits tend to be from quickref.common-lisp.net ; I feel totally lost when arriving there. If not for quickref having arrived earlier on I'd have no idea what it would be about nor how to use what I read.
8:39:19
hayley
Quickref leaves much to be desired. Like why the hell are there sections on the files and internal definitions in my code? That's /my/ business, get your nose out of there.
8:39:30
madnificent
I _think_ I'm missing context when arriving there. Perhaps some text at the top stating that it's a library for Common Lisp which can be installed (for example through QuickLisp). Perhaps also a search for similar packages. Not sure.
8:42:31
hayley
Some of the definitions aren't even my business because I generate a whole bunch of definitions for hash consing. But I digress.
8:43:13
madnificent
Oh hayley looks like I've caught you online. Is it possible that the changes you made to luckless fix an infinite loop on insertion? Or did we just not encounter it anymore due to changed heuristics? We had a case I could reproduce with the version on QuickLisp.
8:44:24
hayley
I don't remember, but Luckless and 42nd-at-threadmill really don't like bad hash functions which generate lots of collisions.
8:47:01
hayley
Maybe try increasing reprobe-limit in hashtable.lisp? 10 might be low; 42nd-at-threadmill goes for 256.
8:47:59
hayley
moon-child: That's good when you have any entropy to start with; I've had cases in one-more-re-nightmare where SBCL would generate a total of 13 hash values for something like 10,000 sets of assignments (represented as lists-of-lists).
8:49:20
madnificent
hayley: The code path goes (setf gethash) -> put-if-match -> %put-if-match -> copy-slot-and-check -> %help-copy -> copy-slot -> %put-if-match.
8:50:30
moon-child
only real problem is you don't get good eq hashes for arrays, or any eq hashes at all for conses, and need to reach into internals stuff for eq hashes for structs. Otoh you can diy eq hashes for structs if you know you need them, and not pay any space cost if you never ask for sbcl-internal eq hashes. so
8:50:41
hayley
That does ring a bell. I still am guessing a bad hash function at the moment, though I'd usually (eventually) get stack overflow because of too many recursive resizes.
8:53:00
moon-child
didn't somebody make a scalable concurrent tree thing in cl? Based on that one thing by the one java guy. Ideally should be possible to fall back to that
8:53:08
madnificent
I should be able to reproduce the keys (and hashes) that were put into the system. They're strings and the strings do share some structure. I don't know the constrainst of sbcl's hash function so can't estimate.
8:57:33
hayley
Depending on what performance you need, the "fallback" sharded table in https://github.com/no-defun-allowed/concurrent-hash-tables might be good enough for now.
8:58:13
hayley
(Make sure you don't have Luckless or Threadmill loaded before loading that library, else the library will instead wrap those.)
9:00:46
madnificent
I'm using it as concurrent set in the case where the issue occurred. It was introduced when multiple threads were reading/writing but today there's a single thread actively writing so the specific issue doesn't matter too much. However, I'm a bit worried about the issue arrising in the same system on other castables where concurrency is an important part.
9:05:19
madnificent
What do the bars in the flamegraphs of concurrent-hash-tables mean? Which color means which implementation?
9:07:25
hayley
They're different threads in one program, from an older version of <https://github.com/scymtym/clim.flamegraph>. The picture compares sharding to using one lock, and nothing of other implementations. However https://raw.githubusercontent.com/telekons/42nd-at-threadmill/master/Documentation/performance.png compares implementations; sure, it doesn't make sharding look very good.
9:10:37
hayley
(I'm not suggesting to use Threadmill here, because it's even more picky about hash functions.)
9:10:47
madnificent
Right! I saw that graph before and it's interesting. Perhaps the concurrent-hash-tables implementation could be used to abstract over the functionality offered by the other libraries so it'd be possible to select the desired implementation fairly late in the game and perhaps use multiple implementations at runtime?
9:37:38
madnificent
hayley: In what sense do you not have a good idea for the multiple implementations part?
9:41:04
hayley
Doing the dispatch is tricky; TYPECASE is inextensible and generic functions are slow (in current Common Lisp implementations and relative to the time taken to do hash table operations).
9:47:32
madnificent
Perhaps a typecase in a function that could be inlined? With the necessary type annotations that could fold to the specific implementation? Flexibility has a price.
9:47:56
rainthree
beach: to the friend who presented me the Hylo programming language and who said "Lisp doesn't have this, because it needs Linear Typing", I said that it implements Mutable Value Semantics, and that Lisp has that already, and I can see one difference is that in Hylo you have to use special syntax & while in Lisp it's automatic
9:49:20
hayley
rainthree: Lisp doesn't make any guarantees of lack of aliasing, so uniform reference semantics doesn't subsume mutable value semantics in that sense.
9:50:14
moon-child
have a general 'hash table' package that picks what other package to import everything from or whatever
9:50:33
madnificent
hayley: Are you eying a solution in which hash-table implementations can be added? In my head all of this was one big static implementation depending on other libarries.
9:50:52
hayley
moon-child: That's not too far off what I do now, but it doesn't allow for multiple implementations if I understand it right.
9:51:26
moon-child
I mean ideally you would be able to do that, but practically, why would you want to?
9:52:57
hayley
Dare I utter the words "dependency injection" to describe that picking the implementation for another library might be useful?
9:54:00
hayley
...but that can be approached with a static approach. No time-slicing #commonlisp and university forms for me.
9:55:40
hayley
No bot. That evaluates to 2; updating the value of A causes the value of B to be updated because they are the same. A and B are said to alias.
9:58:00
hayley
If one cannot alias, then it is easier to tell what some side effect does, but then it is trickier to construct some interesting data structures.
10:00:42
beach
rainthree: Languages without automatic memory management have a problem in that it is very hard to know how many references there are to a particular object. It might be tempting then to avoid that possibility. But that will make programming much harder.
10:01:31
beach
rainthree: Worse, some people design languages without automatic memory management because they falsely believe that automatic memory management is costly compared to manual memory management.
10:03:30
beach
rainthree: Not only are modern techniques for automatic memory management very efficient compared to manual memory management. But for large programs, modularity makes it very hard to know the number of references to an object. So people who write large applications in such languages often resort to smart pointers and reference counters.
10:04:25
beach
rainthree: And by doing that, they turn a simple assignment like (SETQ A B) which in Common Lisp often is a single register instruction, into a test and branch and function call.
10:06:11
beach
rainthree: With reference counters, you then first decrement the reference counter of A, then you check whether it is then zero, and if it is, you call a function to deallocate it. Then you increment the reference counter of B, and finally you assign B to A.
10:08:28
hayley
I'm trying to work out the rules for how I'd enrol for a parallel programming course, when I didn't do the prerequisite class and the prerequisite class is about "systems, networks and concurrency". I may either apply based on "formal learning" i.e. that I did a course on operating systems which covered much of the content, or "informal learning" i.e. I had some work experience which involved the
10:08:43
hayley
(This becomes related to Common Lisp in the next sentence, give me a second to write it.)
10:08:47
madnificent
hayley: I think there would be a lot of value in a manual crafted library that includes what it needs at compile-time. That would greatly simplify what'd need to be built. Then perhaps be kind for PRs to add implementations.
10:08:57
beach
rainthree: My guess is that large applications in languages without automatic memory management that use this technique would be way faster if they were written in a language with automatic memory management. But since it is rare to have a large application written in two languages for comparison, the people writing those applications probably believe they made a good choice for reasons of performance.
10:11:51
beach
rainthree: Oh, and incrementing or decrementing a reference counter requires two memory operations which we try to avoid for obvious reasons.
10:12:25
madnificent
beach: I agree with you. However, it's not a stretch to want applications which consist of small parts. With many small parts memory management needn't be a problem (except it likely is, as could be proven by checking tickets about memory issues on FOSS C applications).
10:12:43
hayley
I could cite my parallel GC as work experience because the GC is a pile of concurrent algorithms and memory management, but I don't have an employer who can write a statement for me. I also need to find two referees "who can verify the experience".
10:13:56
hayley
beach: In my opinion, if one uses reference counting they are using (rather bad) automatic memory management.
10:14:15
hayley
madnificent: I think the issue is worse with small parts, because now you have to document which part allocates and frees what.
10:14:56
beach
madnificent: If it consists of small parts, you more often need to pass objects between parts. So then, either you pass the objects themselves so that you are sure that every live object has a reference count of 1, or you pass a reference.
10:14:59
hayley
(Ahem, naive reference counting, etc, you can write a good reference counter, but it looks suspiciously like a tracing GC, here is the obligatory link to the Unified Theory of Garbage Collection <https://courses.cs.washington.edu/courses/cse590p/05au/p50-bacon.pdf>)
10:15:39
beach
madnificent: In the former case, you lose because objects are larger than references to objects, and you need to copy memory. In the latter case, you get memory leaks.
10:20:12
madnificent
hayley, beach: Right. Did not consider that enough. Anyhow, regardless of the reason, searching for issues would be good proof regardless of the reason.
10:20:29
beach
I guess the extreme situation is with microservices, where you need to not only copy every object that is sent from one part to the other, but you also need to serialize those objects, thereby getting network overhead and losing object identity.
10:25:09
hayley
I'm still undecided on if I should apply by the formal learning route or the informal learning route; I'd appreciate if anyone wants to referee for me and my GC. I guess stylewarning is closest to being my employer?
10:31:02
madnificent
I am (not your) employer but can't consciously gauge further than the libraries I've used. I'm not at home in GC to estimate that work. Do you need an employer to motivate the informal work?
10:31:08
hayley
Looks like the formal route is going to be easier, though still messy as the courses don't match up. Never mind.
10:32:35
hayley
madnificent: The policy is that the informal learning has to be through some work experience, and I need "a statement from the employer". I guess their definition of work is that I am employed by someone.
11:36:31
hayley
Shinmera: Thanks. At the moment I'm thinking it won't be necessary, though I get to overthink it until the university re-opens in a few days.
11:56:28
shka
https://www.youtube.com/watch?v=fytGL8vzGeQ one of the mighty bald lispers giving interview :)
12:09:40
jcowan
beach: 1 = 2 was never part of the Fortran language, it was a bug in an earlh Fortran compiler.
12:32:43
dnhester26
hi beach, good afternoon. Would you mind taking a look at this and could you let me know if I should change anything or I made mistakes? https://lisp-docs.github.io/docs/about/reference#but-why-arent-there-a-bunch-of-spec-projects-already
12:38:00
beach
dnhester26: There is no reason to exclude other #commonlisp participants from the proofreading.
12:38:34
dnhester26
everyone: can you guys help proof reading for both meaning and grammar? https://lisp-docs.github.io/docs/about/reference#but-why-arent-there-a-bunch-of-spec-projects-already
12:39:01
dnhester26
That's a page making the intention of the technical reference clear and how it differs from the intention of the specification and novaspec
12:41:16
dnhester26
beach: "any conforming Common Lisp code should be able..." is replacing which sentence?
12:41:54
beach
If the code is not conforming, there is no reason to expect it to behave the same across implementations.
12:43:26
beach
dnhester26: I think the Common Lisp HyperSpec was also derived from the draft, but I am not entirely sure.
12:45:05
dnhester26
beach: yeah you are right, it's only based on the standard but it's not actually the standard: https://www.lispworks.com/documentation/HyperSpec/Front/Help.htm#Authorship
12:48:33
beach
Instead of "Reading the specification is not always clear..." I would write "From reading the specification, it is not always clear..." or something like that.
12:56:05
dnhester26
is the section about the differences between novaspec and the technical reference correct?
12:57:18
beach
I think you need to ask gilberth about that. I haven't read up on his objectives with it.
13:06:38
dnhester26
shka yeah: please read: https://lisp-docs.github.io/docs/about/reference#but-why-arent-there-a-bunch-of-spec-projects-already explaining what the difference is, and please let me know if I made any errors or I should change anything in that document :D
13:12:36
dnhester26
I think the search is quite cool, although maybe a bit slow since it's hosted for free
13:14:57
dnhester26
shka: look https://lisp-docs.github.io/docs/about there's a link. the picture looks similar so I assume it's the same idea
13:16:58
dnhester26
I'm not sure 100% about breaking up the explanations in their own section entirely though. I would like some level of explanations in the tutorial and reference. Though having said that I would like much more in depth explanations in this section: https://lisp-docs.github.io/docs/category/advanced-topics
13:18:10
dnhester26
shka: yeah, I haven't added a how To guides section because I thought we should first work on the tutorial and technical reference, but if you see here: https://lisp-docs.github.io/ in the bottom left of the 4 cards I list How To Guides
13:18:40
dnhester26
Ideally someone will make guides for developing a web app, a game, etc from beginning to finish just like they have in racket and other languages
13:19:25
dnhester26
The cookbook has great info, but a lot of times things are missing, he uses different libraries in different sections so you don't really get to have a full how to guide using the same libraries, and it can be hard as a newbie
13:26:10
dnhester26
shka: I thought so with the Edit botton at the bottom of each page, but someone else told me that it forks the project. I did it and it was edited but it can be because I'm in the repo. Can you please try and see if it's complicated. I think it just gives the change to be accepted as a fork, but you don't actually have to leave the browser. Not sure though
13:30:33
shka
i wonder if it would be possible to get permission to put "practical common lisp" here in a new section "books"
13:36:30
dnhester26
shka: I wanted to add CLtL2 there making it much easier to read, but we need to ask permission from HP or some other big company who bought the last copyright owner
13:37:18
dnhester26
olnw: thanks, not sure how I missed those after having read it myself like 3 times. You have a good eye!
13:38:11
dnhester26
shka: the text is basically markdown see the tip (gree box) here: https://lisp-docs.github.io/cl-language-reference/ for the syntax note and links
13:40:31
dnhester26
shka: however the whole page is static, the markdown is compiled to html when deploying to github pages
14:22:23
olnw
dnhester26: It appears that, in the footer, "Why Lisp" and "Getting Started" don't link to the correct pages.
14:38:09
mfiano
Seems like a spell checker would go a longer way than a grammar checker like languagetool, while you're writing this, which should be easy to come by with your environment.