libera/#commonlisp - IRC Chatlog
Search
3:19:04
EdLangley[m]
Unless it was declared inline or you have other saved references to the old definition (e.g. `(funcall #'foo …)`)
3:19:39
EdLangley[m]
If you always refer to the function by name rather than with #', it should just work.
3:24:29
Bike
redefining a function is undefined if it was part of a compilation unit. otherwise it's fine
3:39:03
seragold[m]
If in CL redefining CLOS class just works across existing instances it would be weird if redefining a normal function did not IMHO
3:43:05
Alfr
unixlisp, it's defined quite in detail, it replaces the function binding in the global environment; mostly subject to the constraints listed in
3:44:07
Bike
if you continue this routine of insisting everything is undefined despite all reason i'll ban you, alright?
3:44:40
Alfr
unixlisp, 3.2.2.3 also says that, apart form the listed exceptions the new definition takes precedence.
4:00:51
beach
unixlisp: Why do you think that? I.e., that redefining a function is always undefined. I mean, is that an opinion of yours? Or something you concluded from reading the standard?
4:03:42
unixlisp
beach: I should carefully read 3.2.2.3, redefining function is not always undefined.
4:06:49
beach
unixlisp: So let me ask you again. When you said "I think redefining function is always undefined", was that an opinion of yours, or something you concluded form reading the standards document?
4:09:55
unixlisp
Oh, I said the reading is not complete, so the conclusion is wrong. (that is my opinion from read spec.)
4:12:05
unixlisp
In general, I always wonder redefining is a defined behavior or not. uch as defparameter, etc.
4:13:39
beach
It would be very strange for a language to include operators that always have undefined consequences.
4:18:03
EdLangley[m]
As a rule, Common Lisp is defined for a workflow that involves building up a program incrementally.
4:18:54
EdLangley[m]
There are exceptions (defstruct, defconstant), but it would be strange for most of the fundamental operations not to have well-defined redefinition semantics.
4:20:59
beach
unixlisp: So here is a suggestion. The next time you "think" something like that, maybe phrase it as a question instead, like "Is redefining a function always undefined behavior"?
4:36:34
beach
masinter: It could be defined behavior only if the function is currently undefined. Also, consider SYMBOL-FUNCTION obsolete in favor of FDEFINITION.
4:37:54
beach
masinter: DEFSTRUCT is pretty much like that. The first time you use it, it is defined behavior.
4:51:32
beach
But the behavior of DEFSTRUCT is very strange for a dynamic language (which by definition is a language with semantics defined as a sequence of incremental modifications to the environment).
4:53:39
EdLangley[m]
Does uninterning the symbol first make a DEFSTRUCT with the same name defined behavior?
4:58:24
beach
unixlisp: Structure classes are defined so that the implementation can represent their instances in a very efficient way if it chooses to do so. Standard classes require instances to have some indirection that allows for instances to be altered when a class is modified.
4:59:59
mfiano
Note that if you are familiar with the MOP, you can make standard classes just as efficient as structure classes. IIRC, phoe is going to talk about my technique for doing so in the new revision of Common Lisp Recipes.
5:02:23
mfiano
One nice thing about Common Lisp is its extensibility. Just as macros extend syntax to form new language constructs by the user, the MOP extends CLOS to alter semantics of CLOS OOP.
5:08:12
mfiano
I really wouldn't recommend trying to make standard classes like structure classes though. They become less and less useful for iterative programming the more you do so, as beach mentioned.
5:09:08
mfiano
And a sufficiently smart compiler should be able to dispatch fast to an accessor/other method.
5:10:24
beach
That is partly why in SICL, instances of structure classes have the same structure as instances of standard classes.
5:12:43
mfiano
I recently started programming in Lisp again (after like 6 months!), and after using standard classes, I switched to structure classes, because a concise memory footprint and tagging of numeric fields was needed. I now realize that was foolish, and I should instead tag fixnums and unpack the individual bits I need.
5:14:24
mfiano
These structure objects with 2-6 numerical fields were to be stored in very large arrays. My plan now is to just bitwise or/shift the values together into a single fixnum, with a tag header to distinguish the different "classes"
5:15:25
beach
EdLangley[m]: You may be interested in heisig's work on fast generic functions and sealing.
5:16:04
mfiano
It is used in other languages to prevent monkey patching, but in CL, heisig's work, uses it for performance.
5:16:49
mfiano
I can't stand behind semantics-altering generic function libraries anymore though. I think it too, goes too much against the interactive development style I like, and leads to very surprising behavior.
5:17:43
EdLangley[m]
E.g. when I’m deploying a server, I’m often going to replace it by spinning up a second server in parallel and cutting over
5:18:09
EdLangley[m]
In this situation, sealing makes sense because you’re never going to encounter a redefinition in production.
5:20:57
unixlisp
It is good of "fast generic function", but it has a cost: "Once a fast generic function has been sealed, it is not possible to add, remove, or redefine methods within the sealed domain"
5:21:05
mfiano
One thing you could do is write a macro wrapper over defclass that expands to cl:defclass in development, and a custom metaclass that defines inlined regular function accessors that use c2mop:standard-instance-access in production.
5:22:50
mfiano
Also fast generic functions has issues optimizing structure class specializers in my experience, so I wouldn't start there.
5:25:03
mfiano
About my idea above to use tagged fixnums though. I'm not sure if it is a good idea yet, as I may need methods dispatching on the type that is encoded in the MSB tag bits
5:25:45
mfiano
Which, would need a lot of MOP support for little gain. I'm not sure if a CASE jump table would be efficient enough for the number of types.
5:27:20
EdLangley[m]
Doesn’t beach’s generic functions paper use some sort of trick with numbers to make dispatch efficient? It’s been a while since I read it.
5:27:27
mfiano
My other thought is usually these arrays will be homogeneous, and if I can constrain that to _always_ be the case, the type can be encoded in the class wrapping the array
5:28:02
beach
EdLangley[m]: The essence of the paper is that register operations can be used where traditionally memory operations were.
5:29:06
mfiano
I'm just rubber ducking here; prob can ignore me. I have been thinking about this all day while I do system administration.
5:31:23
beach
EdLangley[m]: There is still a lot of work to be done in the domain of language implementation, where in the past it was assumed that memory operations were roughly as fast as register operations, whereas that is no longer the case.
5:38:12
mfiano
So I can't pack any type tag into my integers anyway, so that idea is out. It turns out, the largest integer requires 64 bits of data without the type tag, and I surely don't want most implementations to upgrade the simple-array element type to T.
5:54:51
masinter
changing a defstruct both changes some defuns / compiler macros and the type name of existing data
5:57:38
beach
But the point is that existing instances are now obsolete, and attempting to access some slots in such an obsolete instance may give some arbitrary result.
6:09:14
EdLangley[m]
If you have a moving GC, in theory you could track the probability that two objects are accessed in order and then arrange them so they can be read more efficiently
6:10:14
EdLangley[m]
In relational databases, for example, if you have an index on a column, you can tell the DB to use that index to sort the data on disk.
6:13:56
lisp123win
I wanted to ask, if I have a collection of objects that I want to add to multiple hash tables (each hash table basically being an index of sorts)
6:14:25
lisp123win
I am right in my assumption the overall memory cost is not too big because all the hash tables will share the same objects (to the extent I am placing the same object in multiple hash tables and not re-creating them)
7:46:01
White_Flame
Bike: it can't prove that the LOOP will always find a match, and so could return NIL
7:47:14
White_Flame
you'd have to declare that dat is a recursing list and thus would never terminate (if that's the intent)
7:48:04
empwilli
EdLangley[m]: I'm toying with this idea for compiled languages: Instead of having the class description in, let's say, c++ dictate how data is placed in memory, let is shuffle around (or do even fancier stuff, e.g. the things that databases do, in games terminology, this would be entity component systems)
7:49:45
White_Flame
empwilli: I've had plenty of ideas similar to that as well. My notion would be that data tracks where it was allocated from, and usage feeds back into how it should be initially allocated/generated
7:57:43
EdLangley[m]
So, what got me down this road was thinking about how, if the MMU were designed for the languages people like to program in (ones with lots of pointer chasing), it could use access patterns to transparently move data around in physical memory for better locality (sort of like how SSDs move data around on disk for wear leveling)
7:58:23
EdLangley[m]
Then, I realized that the GC already walks all the objects periodically, so if you had some sort of statistics about access patterns, a moving GC could take those into account when it compacts the heap
7:58:42
empwilli
I'm not too deep in this literature but I think it has been shown to be too expensive to bring a benefit
7:58:58
EdLangley[m]
The simple case here would be, if you have a list that's iterated over frequently, to move all the cells together
7:58:59
empwilli
the other issue is, that, generally speaking, the granularity of 4k pages is too big
7:59:16
EdLangley[m]
It might be too expensive to do all the time, but maybe there's some sort of profile-guided optimization you could do?
7:59:36
EdLangley[m]
e.g. run a bit of code with a profiler and store statistics about the actual observed access patterns
8:01:28
EdLangley[m]
So, a naive way to do this would be to attach a counter to every member of a class
8:01:50
EdLangley[m]
And increment it every time you follow the pointer from the instance -> the actual member
8:02:14
EdLangley[m]
Then, the gc picks a threshold and moves members close to the instance in memory, if the counter is above the threshold
8:02:56
EdLangley[m]
This probably has a bunch of disadvantages (turning reads into writes, etc.) but it'd improve the locality of instances and commonly used members.
8:04:19
empwilli
So what I can point you to is literature on NUMA systems where this is frequently a problem (at least from the perspective where to allocate memory to/from)
9:52:17
unixlisp
Alfr: "3.2.2.3 also says that, apart form the listed exceptions the new definition takes precedence" That is "about minimize the observable differences between compiled and interpreted programs".
10:00:19
beach
Alfr: Frequently unixlisp seems to make statements without any context, and sometimes without any stated purpose. I think you need to get used to it.
10:03:27
Alfr
unixlisp, but what are you trying to say with those quotes? Or what's the problem/question?
10:04:24
beach
I think unixlisp now realizes that the page that was cited before as being about redefining functions, in fact is not. But I already said that.
10:05:44
Alfr
beach, I cited that, and I still think it's relevant. As it defines what conforming programs can expect, thus conforming implementations must arrange for.
10:07:55
empwilli
EdLangley[m]: hm, given that GCs really are notorious sources for performance bottlenecs (at least my understanding, I mostly don't do GC programming, so please correct me): Moving around members in memory would really add a large amount of overheads
10:11:03
Alfr
beach, (okay for the other case where it simply happens during runtime, DEFUN's documentation already specifies the behavior as replacing.)
10:13:59
beach
empwilli: So if you are avoiding GC for performance reasons, you may be doing the wrong thing.
10:14:32
empwilli
no worries there :) I'm avoiding it just for the convenience of the languages I feel comfortable in
10:15:22
empwilli
I'm currently working on adding common lisp to my repertoire, but I'm far from fluent in it
10:24:56
beach
empwilli: It is too bad, though, that people seem to think that automatic memory management is a "notorious source for performance bottlenecks". They are then doomed to program in a way that makes it extremely hard to get both performance and modularity in their code.
10:27:51
empwilli
this sounds as if there were design patterns to program in favour of the GC. Is this the case? Can you point me to some resources?
10:30:17
beach
I had no design pattern in mind. I am just making a general observation that "liveness is a global property" [from Paul Wilson], so not a "modular" property. If your program is modular, you either need to copy all your objects, or use reference counters if you don't have automatic memory management.
10:30:39
seragold[m]
I think it was David Ungar back in early nights who showed that no sort of reference counting and its variants overall performed as well as state of art GC then. So why would you want such in your way of whatever the most efficient and elegant patterns are it is possible to come up with in a language that gives you all the power?
11:06:22
unixlisp
beach: early about (setf (symbol-function 'foo) newdef), you said "It could be defined behavior only if the function is currently undefined". but in clhs about symbol-function entry, "setf may be used with symbol-function to replace a global function definition"
11:44:18
beach
unixlisp: Yes, notice the word "could" in order to explain to masinter that the existence of that operator doesn't necessarily mean that the behavior is always defined. But as it turn out, it is.
11:50:02
loke[m]
beach: Take your average C programmer: malloc and free "feels" free in terms of performance impact. So when someone suggests that a GC will run automatically and search for all the memory to be released, that feels "obviously" slower, since now it has to do stuff when before it didn't have to.
11:50:59
unixlisp
beach: I noticed the word "could". In clhs defun entry: "defun can be used to ... redefine an already-defined function", that word is "can".
11:51:03
loke[m]
Any argument that malloc/free are actually slow falls of deaf ears since their microbenchmarks all use stack allocation, which indeed is free.
11:54:58
flip214
loke[m]: not in terms of cache misses... alloca()te a few KB and you're out of your L1
11:55:56
loke[m]
flip214: Well sure, but that applies to all allocations, regardless of whether they are heap or stack. Unless I misunderstand your point?
11:57:17
beach
loke[m]: Yeah, but it's disturbing to me that so many decisions in the software industry are based on "gut feeling" rather than research. And I can only imagine the amount of money wasted that way.
11:57:35
flip214
empwilli: one example favoring a GC: https://people.eecs.berkeley.edu/~fateman/papers/cachelisp.pdf
11:58:11
flip214
loke[m]: no, you're understanding me correctly. My point is that stack allocation is not completely free either.
12:01:24
flip214
empwilli: and ELS2018 (https://www.european-lisp-symposium.org/2018/#programme) had "Dynamic Optimizations for SBCL Garbage Collection" which would switch a load balancer around before doing GC, so the equivalent to free() would be free here
12:01:51
beach
unixlisp: Do I really have to spell this out to you? I mean to say "masinter, you are right that there is an operator (setf symbol-function), but you say that its existence is proof that redefining functions is defined behavior. However, it COULD very well be the case [but it isn't] that this operator is defined only when the symbol does not already have a function defined, in which case definition is OK, but REdefinition is not
12:01:56
beach
unixlisp: ... So the existence itself does not guarantee that redefining a function is defined behavior. However, redefining a function IS defined behavior, as the Common Lisp HyperSpec specifies".
12:05:48
unixlisp
beach: oh. "redefining a function IS defined behavior", acording to "defun can be used to ... redefine an already-defined function"?