libera/#commonlisp - IRC Chatlog
Search
1:22:12
Bike
https://plaster.tymoon.eu/view/2874#2874 can i get a sanity check here? sbcl gives me a type warning about this, but I don't see the problem
1:31:03
moon-child
perhaps it is assuming that CONS might be nil? Obviously that can never happen, but
1:56:42
lagash
What do folks here use for a linter/code formatter for CL, particularly as a pre-commit hook?
1:57:23
moon-child
pre-commit hooked autoformat is dumb. Formatting can not always be applied automatically
1:59:13
moon-child
mfiano: it also depends on the smarts of the formatter. Slime still can not indent moderately complex loops
2:00:03
lagash
mfiano: I have been using Sly, although come to think of it, not every time I've formatted my CL code, was Sly running.. that might be the issue?
2:00:19
mfiano
Stemming from its piggy-backing off of emacs-lisp, which does not have the concept of packages.
2:01:25
mfiano
Sly can indent moderately complex LOOPs just fine though, so I'm not sure about SLIME in this regard.
2:02:13
mfiano
I have written hundreds of line loops before, and indentation has never been an issue.
2:02:18
EdLangley[m]
Because, even if the formatting isn’t always great, it reduces diffs if you apply it consistently
2:02:26
lagash
It's just that Emacs and/or Sly can't seem to make up its mind when I, say, format an entire file..
2:04:25
mfiano
I also use aggressive-indent-mode, but should point out that the code and its dependencies should also be loaded; it isn't enough to just have SLIME/SLY running due to macro definitions.
2:05:19
lagash
mfiano: so depending on if I had Sly running or had the code loaded or whatnot, Emacs might format things differently?
2:06:42
EdLangley[m]
SLIME/Sly indentation is based on introspection of function and macro information
3:19:04
EdLangley[m]
Unless it was declared inline or you have other saved references to the old definition (e.g. `(funcall #'foo …)`)
3:19:39
EdLangley[m]
If you always refer to the function by name rather than with #', it should just work.
3:24:29
Bike
redefining a function is undefined if it was part of a compilation unit. otherwise it's fine
3:39:03
seragold[m]
If in CL redefining CLOS class just works across existing instances it would be weird if redefining a normal function did not IMHO
3:43:05
Alfr
unixlisp, it's defined quite in detail, it replaces the function binding in the global environment; mostly subject to the constraints listed in
3:44:07
Bike
if you continue this routine of insisting everything is undefined despite all reason i'll ban you, alright?
3:44:40
Alfr
unixlisp, 3.2.2.3 also says that, apart form the listed exceptions the new definition takes precedence.
4:00:51
beach
unixlisp: Why do you think that? I.e., that redefining a function is always undefined. I mean, is that an opinion of yours? Or something you concluded from reading the standard?
4:03:42
unixlisp
beach: I should carefully read 3.2.2.3, redefining function is not always undefined.
4:06:49
beach
unixlisp: So let me ask you again. When you said "I think redefining function is always undefined", was that an opinion of yours, or something you concluded form reading the standards document?
4:09:55
unixlisp
Oh, I said the reading is not complete, so the conclusion is wrong. (that is my opinion from read spec.)
4:12:05
unixlisp
In general, I always wonder redefining is a defined behavior or not. uch as defparameter, etc.
4:13:39
beach
It would be very strange for a language to include operators that always have undefined consequences.
4:18:03
EdLangley[m]
As a rule, Common Lisp is defined for a workflow that involves building up a program incrementally.
4:18:54
EdLangley[m]
There are exceptions (defstruct, defconstant), but it would be strange for most of the fundamental operations not to have well-defined redefinition semantics.
4:20:59
beach
unixlisp: So here is a suggestion. The next time you "think" something like that, maybe phrase it as a question instead, like "Is redefining a function always undefined behavior"?
4:36:34
beach
masinter: It could be defined behavior only if the function is currently undefined. Also, consider SYMBOL-FUNCTION obsolete in favor of FDEFINITION.
4:37:54
beach
masinter: DEFSTRUCT is pretty much like that. The first time you use it, it is defined behavior.
4:51:32
beach
But the behavior of DEFSTRUCT is very strange for a dynamic language (which by definition is a language with semantics defined as a sequence of incremental modifications to the environment).
4:53:39
EdLangley[m]
Does uninterning the symbol first make a DEFSTRUCT with the same name defined behavior?
4:58:24
beach
unixlisp: Structure classes are defined so that the implementation can represent their instances in a very efficient way if it chooses to do so. Standard classes require instances to have some indirection that allows for instances to be altered when a class is modified.
4:59:59
mfiano
Note that if you are familiar with the MOP, you can make standard classes just as efficient as structure classes. IIRC, phoe is going to talk about my technique for doing so in the new revision of Common Lisp Recipes.
5:02:23
mfiano
One nice thing about Common Lisp is its extensibility. Just as macros extend syntax to form new language constructs by the user, the MOP extends CLOS to alter semantics of CLOS OOP.
5:08:12
mfiano
I really wouldn't recommend trying to make standard classes like structure classes though. They become less and less useful for iterative programming the more you do so, as beach mentioned.
5:09:08
mfiano
And a sufficiently smart compiler should be able to dispatch fast to an accessor/other method.
5:10:24
beach
That is partly why in SICL, instances of structure classes have the same structure as instances of standard classes.
5:12:43
mfiano
I recently started programming in Lisp again (after like 6 months!), and after using standard classes, I switched to structure classes, because a concise memory footprint and tagging of numeric fields was needed. I now realize that was foolish, and I should instead tag fixnums and unpack the individual bits I need.
5:14:24
mfiano
These structure objects with 2-6 numerical fields were to be stored in very large arrays. My plan now is to just bitwise or/shift the values together into a single fixnum, with a tag header to distinguish the different "classes"
5:15:25
beach
EdLangley[m]: You may be interested in heisig's work on fast generic functions and sealing.
5:16:04
mfiano
It is used in other languages to prevent monkey patching, but in CL, heisig's work, uses it for performance.
5:16:49
mfiano
I can't stand behind semantics-altering generic function libraries anymore though. I think it too, goes too much against the interactive development style I like, and leads to very surprising behavior.
5:17:43
EdLangley[m]
E.g. when I’m deploying a server, I’m often going to replace it by spinning up a second server in parallel and cutting over
5:18:09
EdLangley[m]
In this situation, sealing makes sense because you’re never going to encounter a redefinition in production.
5:20:57
unixlisp
It is good of "fast generic function", but it has a cost: "Once a fast generic function has been sealed, it is not possible to add, remove, or redefine methods within the sealed domain"
5:21:05
mfiano
One thing you could do is write a macro wrapper over defclass that expands to cl:defclass in development, and a custom metaclass that defines inlined regular function accessors that use c2mop:standard-instance-access in production.
5:22:50
mfiano
Also fast generic functions has issues optimizing structure class specializers in my experience, so I wouldn't start there.
5:25:03
mfiano
About my idea above to use tagged fixnums though. I'm not sure if it is a good idea yet, as I may need methods dispatching on the type that is encoded in the MSB tag bits
5:25:45
mfiano
Which, would need a lot of MOP support for little gain. I'm not sure if a CASE jump table would be efficient enough for the number of types.
5:27:20
EdLangley[m]
Doesn’t beach’s generic functions paper use some sort of trick with numbers to make dispatch efficient? It’s been a while since I read it.
5:27:27
mfiano
My other thought is usually these arrays will be homogeneous, and if I can constrain that to _always_ be the case, the type can be encoded in the class wrapping the array
5:28:02
beach
EdLangley[m]: The essence of the paper is that register operations can be used where traditionally memory operations were.
5:29:06
mfiano
I'm just rubber ducking here; prob can ignore me. I have been thinking about this all day while I do system administration.
5:31:23
beach
EdLangley[m]: There is still a lot of work to be done in the domain of language implementation, where in the past it was assumed that memory operations were roughly as fast as register operations, whereas that is no longer the case.
5:38:12
mfiano
So I can't pack any type tag into my integers anyway, so that idea is out. It turns out, the largest integer requires 64 bits of data without the type tag, and I surely don't want most implementations to upgrade the simple-array element type to T.
5:54:51
masinter
changing a defstruct both changes some defuns / compiler macros and the type name of existing data
5:57:38
beach
But the point is that existing instances are now obsolete, and attempting to access some slots in such an obsolete instance may give some arbitrary result.
6:09:14
EdLangley[m]
If you have a moving GC, in theory you could track the probability that two objects are accessed in order and then arrange them so they can be read more efficiently
6:10:14
EdLangley[m]
In relational databases, for example, if you have an index on a column, you can tell the DB to use that index to sort the data on disk.
6:13:56
lisp123win
I wanted to ask, if I have a collection of objects that I want to add to multiple hash tables (each hash table basically being an index of sorts)
6:14:25
lisp123win
I am right in my assumption the overall memory cost is not too big because all the hash tables will share the same objects (to the extent I am placing the same object in multiple hash tables and not re-creating them)
7:46:01
White_Flame
Bike: it can't prove that the LOOP will always find a match, and so could return NIL
7:47:14
White_Flame
you'd have to declare that dat is a recursing list and thus would never terminate (if that's the intent)
7:48:04
empwilli
EdLangley[m]: I'm toying with this idea for compiled languages: Instead of having the class description in, let's say, c++ dictate how data is placed in memory, let is shuffle around (or do even fancier stuff, e.g. the things that databases do, in games terminology, this would be entity component systems)
7:49:45
White_Flame
empwilli: I've had plenty of ideas similar to that as well. My notion would be that data tracks where it was allocated from, and usage feeds back into how it should be initially allocated/generated
7:57:43
EdLangley[m]
So, what got me down this road was thinking about how, if the MMU were designed for the languages people like to program in (ones with lots of pointer chasing), it could use access patterns to transparently move data around in physical memory for better locality (sort of like how SSDs move data around on disk for wear leveling)
7:58:23
EdLangley[m]
Then, I realized that the GC already walks all the objects periodically, so if you had some sort of statistics about access patterns, a moving GC could take those into account when it compacts the heap
7:58:42
empwilli
I'm not too deep in this literature but I think it has been shown to be too expensive to bring a benefit
7:58:58
EdLangley[m]
The simple case here would be, if you have a list that's iterated over frequently, to move all the cells together
7:58:59
empwilli
the other issue is, that, generally speaking, the granularity of 4k pages is too big
7:59:16
EdLangley[m]
It might be too expensive to do all the time, but maybe there's some sort of profile-guided optimization you could do?
7:59:36
EdLangley[m]
e.g. run a bit of code with a profiler and store statistics about the actual observed access patterns
8:01:28
EdLangley[m]
So, a naive way to do this would be to attach a counter to every member of a class
8:01:50
EdLangley[m]
And increment it every time you follow the pointer from the instance -> the actual member
8:02:14
EdLangley[m]
Then, the gc picks a threshold and moves members close to the instance in memory, if the counter is above the threshold
8:02:56
EdLangley[m]
This probably has a bunch of disadvantages (turning reads into writes, etc.) but it'd improve the locality of instances and commonly used members.
8:04:19
empwilli
So what I can point you to is literature on NUMA systems where this is frequently a problem (at least from the perspective where to allocate memory to/from)
9:52:17
unixlisp
Alfr: "3.2.2.3 also says that, apart form the listed exceptions the new definition takes precedence" That is "about minimize the observable differences between compiled and interpreted programs".
10:00:19
beach
Alfr: Frequently unixlisp seems to make statements without any context, and sometimes without any stated purpose. I think you need to get used to it.
10:03:27
Alfr
unixlisp, but what are you trying to say with those quotes? Or what's the problem/question?
10:04:24
beach
I think unixlisp now realizes that the page that was cited before as being about redefining functions, in fact is not. But I already said that.
10:05:44
Alfr
beach, I cited that, and I still think it's relevant. As it defines what conforming programs can expect, thus conforming implementations must arrange for.
10:07:55
empwilli
EdLangley[m]: hm, given that GCs really are notorious sources for performance bottlenecs (at least my understanding, I mostly don't do GC programming, so please correct me): Moving around members in memory would really add a large amount of overheads
10:11:03
Alfr
beach, (okay for the other case where it simply happens during runtime, DEFUN's documentation already specifies the behavior as replacing.)
10:13:59
beach
empwilli: So if you are avoiding GC for performance reasons, you may be doing the wrong thing.