freenode/#lisp - IRC Chatlog
Search
0:42:03
mfiano
Speaking of compiler macros, I don't remember the last time I needed to write one, but I have a question related to them: Do I have to gensym when introducing a new lexical variable as part of its expansion to prevent unwanted capture in the same case as for regular macros?
1:56:28
_death
mfiano: yes.. e.g., (define-compiler-macro foo (x y) `(let ((z ,x)) (list z ,y))) (let ((z 123)) (foo 1 z)) ==> (1 1)
2:37:13
_death
https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node101.html#SECTION001240000000000000000
5:00:34
fiddlerwoaroof
So, Lispworks is the only implementation (of three) I can get to work on my new laptop
5:03:47
aeth
I'm surprised it works at all. These sorts of things tend not to care about obscure, GCed languages.
5:04:44
fiddlerwoaroof
mmap: Cannot allocate memory / ensure_space: failed to validate 1073741824 bytes at 0x1000000000
5:05:31
no-defun-allowed
aeth: A funny coincedence, while I'm reading about how refcounting is faster on a M1 processor.
5:10:28
no-defun-allowed
It takes 6.5ns to create and free a NSObject on M1/Objective-C, and 6.7ns to create and eventually free a standard-object on x86-64/SBCL. That's a 3% performance increase; something you'd expect from Intel marketing these days :)
5:11:12
no-defun-allowed
fiddlerwoaroof: Seriously though, do you find it any faster? Most articles I find say it's much faster than an Intel MacBook.
5:32:25
aeth
oni-on-ion: Apple is migrating to ARM because (1) its ARM chips are competitive with Intel's x86-64 chips and (2) Apple wants to eventually merge macOS (a cost) with iOS (where all of their profit is)
5:34:28
aeth
on the one hand, JVM would also be hard to emulate; on the other, there might actually be Java apps on macOS worth emulating
5:34:35
theemacsshibe
An interesting headline: "Microsoft contributes to Java port for Apple silicon Macs"
5:35:30
aeth
fiddlerwoaroof: Yes, a disturbing trend. iOS is the world's most popular locked-down platform.
5:38:40
aeth
fiddlerwoaroof: maybe it's good news for iOS, but it's bad news for macOS, and I'm not looking forward to getting bug reports from Mac users.
5:41:28
aeth
I have no idea how you'd even attempt to mix the CL workflow with stuff like this, though: https://lapcatsoftware.com/articles/unsigned.html
5:43:26
fiddlerwoaroof
And, the basic path is straightforward: sign the sbcl executable and always launch your app through that executable
5:54:04
aeth
That's basically saying to avoid save-lisp-and-die exporting even if you do that on every other platform.
5:54:43
fiddlerwoaroof
The objective-c runtime as a whole is a much nicer thing to interact with than the low-level APIs of alternative platforms
5:55:26
fiddlerwoaroof
Anyways, this is what abstraction is for: something like ASDF could make a "thing" that works on whichever platform you're targeting
5:56:07
oni-on-ion
objective-c is one of my top favorite languages, there was a point i made my own runtime. it talked to gnu smalltalk =)
5:58:22
aeth
fiddlerwoaroof: afaik, you can't sign it in a way that doesn't have scary warnings without paying Apple 100 $/year
5:59:26
aeth
According to the article, "Mac developers must sign up for the Apple Developer Program, sign a legal agreement, and pay an annual fee of USD $99 plus tax in order to obtain a Developer ID code signing certificate and upload software to Apple for notarization."
6:01:03
aeth
The only thing that saves CL here is that most non-commercial applications will just be distributed in .lisp source code form for local compilation and running on a presumably-signed already-installed CL compiler
6:01:31
fiddlerwoaroof
The thing is, most applications you could just pop up a loading screen and compile the code on the fly
6:01:56
aeth
Any end-user application written in CL (which can't assume that the user has their own CL installed) will either have to be commercial or money-losing.
6:02:26
aeth
That's not really the common workflow, though. Generally, you export an executable if you don't expect the user to have CL installed.
6:03:02
aeth
No, the output of save-lisp-and-die would, afaik, have to be signed, which costs 100 USD/year.
6:03:58
aeth
Now you have to distribute a matching SBCL version. Java can get away with this because lots of things use Java.
6:04:29
oni-on-ion
hmm lisp (fasl) can be disasm quite easily ? can one only distribute the fasl/core and are those somewhat protected ?
6:05:17
fiddlerwoaroof
But, if you're not distributing to developers, you'd need all this stuff anyways
6:06:02
fiddlerwoaroof
You also need some functions like this to grab resources out of the app bundle: https://github.com/cjdev/aws-access/blob/master/src/objc-utils.lisp
6:08:48
aeth
Alternatively, $99/year is like an entire computer a decade, on top of having to actually buy an entire Apple computer a decade in order to test/build the application.
6:11:02
aeth
Breaking the law in the process of making your software is essentially a non-starter for distributing software, anyway.
6:11:44
fiddlerwoaroof
I mean, if you're going the commercial route, you're not going to worry about $99/year
6:12:39
aeth
If you're going the commercial route and not supporting iOS, then there's a very good chance that the money you make from macOS will not be enough to make up for the increased expense for supporting macOS.
6:13:01
aeth
Especially when Apple goes out of the way to act like they're in Microsoft's position, not a < 5% position.
6:15:49
aeth
My main concern with macOS, though, would be if they require everything to go through the App Store, like with iOS. Obviously not literally, since you wouldn't be able to develop for macOS (or iOS!) if they did, but contractually, it could be required.
6:16:43
aeth
Every indication is that they're moving to a macOS-iOS merger in the long run (except touch support, which afaik is still missing from macOS, unlike, say, Windows)
6:18:03
aeth
Considering how long everything I do takes, I wouldn't be surprised if Apple locks down their desktop/laptop platform before I even complete anything that I would want to export as a binary.
6:37:23
fiddlerwoaroof
My work mac looks like this: 166.41s user 9.68s system 165% cpu 1:46.50 total
6:37:53
fiddlerwoaroof
My 16 core Ryzen 2 box like this: 140.36s user 2.52s system 202% cpu 1:10.62 total
6:42:16
aeth
Not surprising... it's only going to be competitive to roughly contemporary AMD going forward unless Intel can turn themselves around. And I wouldn't be surprised if Intel finds a way to fall into #4 or #5 by 2023.
6:43:35
aeth
We should do a gofundme to make a Lisp machine CPU on TSMC 5 nm so we can be #3 ahead of Intel
6:45:24
fiddlerwoaroof
Since everything is built on the Objective-C runtime (more or less), Apple devices are more like a Lisp Machine than most Linux devices
6:48:26
no-defun-allowed
aeth: I want to make a list of (reasonable, RISC-y) things to put in a Lisp-oriented extension to RISC-V.
6:54:45
beach
no-defun-allowed: I have probably said this before, but my bet would be on some write barrier or read barrier.
6:56:56
no-defun-allowed
That's usually done with the MMU, marking pages as read-only and trapping writes (for a write barrier), no?
6:59:10
fiddlerwoaroof
So, if I understand correctly, they reserve several bits for information about where the pointer points to
7:01:27
no-defun-allowed
I recall a similar thing happens with Nettle's replicating copying collector, but it's worse as that normalization requires a memory read (to follow the forwarding pointer).
7:02:36
no-defun-allowed
But in applicative languages like ML, which it was designed for, EQ supposedly is rarely used; though testing reference equality is favourable to testing structural equality.
7:03:53
beach
no-defun-allowed: Yes, there are tendencies like that, e.g. "Our language semantics makes everything expensive anyway, so using an inefficient mechanism here doesn't matter much".
7:06:02
aeth
no-defun-allowed: I personally wouldn't make extensions unless you actually had real programs from real implementations running and profiled them and determined what the bottlenecks are that are addressable with new instructions rather than with better compilers
7:07:44
aeth
I'm curious as to if e.g. making a 68 bit architecture with 4 tag bits might be the way to go, not for performance, but for 64-bit fixnums and unboxed double-float.
7:08:39
no-defun-allowed
Now you have the problem of finding 68-bit memory. And you may want another bit for incremental marking.
7:09:46
beach
It is entirely possible that any useful extension would not be RISC-y enough to be considered.
7:13:09
beach
Not that I know what it would be. I already know how to do it in a not-too-costly way.
7:32:39
no-defun-allowed
The code that calls a generic function would be replaced with a call to the last effective method (with a prologue that calls the generic function as usual, if the last method is no longer applicable).
7:33:40
no-defun-allowed
It has been done on normal CPUs with Smalltalk, Self, Java and JavaScript at the least; so it may not be necessary to provide any processor support.
7:34:33
beach
The technique I was thinking of consists of using type information in the caller to create a call-site-specific discriminating function, and change it when the generic-function changes.
7:35:59
no-defun-allowed
The polymorphic inline cache described in <https://bibliography.selflanguage.org/_static/pics.pdf> is a bit closer to your technique, but it still picks the most common methods from runtime information.
7:38:31
no-defun-allowed
The only support that needs is that you need to be able to overwrite the call site, but that's not a big deal; and also doable on stock hardware.
7:48:44
no-defun-allowed
The one idea I have which may be better done in hardware (which gilberth gave me) is computing the address of an element in arrays of different element types when performing a generic AREF. (SBCL calls that "hairy", which is much more fun to say than "generic" though.)
7:58:49
beach
no-defun-allowed: I am thinking that things like AREF and CAR/CDR are usually done in a loop. So then, it is possible to make the type of the object a loop invariant.
8:00:33
beach
Plus, application writers who care that much about performance would stick in a type declaration that could then be verified quite easily.
8:02:23
beach
Speaking of which, I am convinced that Common Lisp implementations and their compilers are full of "optimizations" that are basically useless.
8:04:20
beach
In the case of SBCL NIL, it makes it necessary to do two tests in a loop over a list, rather than a single test, in order to determine the end.
8:05:14
beach
Though stassats assures me that there is no performance penalty for making two tests rather than one.
8:07:18
aeth
What I'm more concerned about that maintenance is compilation times. A lot of compilers are incredibly slow to get that extra speedup at the end, and it's questionable if it's worth it, especially if you translate that sort of approach to a CL compiler, which will be compiling more. (Yes, CL has optimization levels, but they're not that flexible)
8:08:05
aeth
People have complained about SBCL compilation times, but I don't really see that, except with a few edge case libraries (is Ironclad still really slow?)
8:08:55
beach
Especially for Common Lisp where the maintenance burden is a real problem, given the limited resources we have.
8:19:58
pfdietz
You don't want to make the generated code for car and cdr check if the thing is nil. That way lies madness. So what's the alternative?
8:21:49
pfdietz
You do car and cdr a lot in a lisp program, much more than you do symbol-name, symbol-plist, symbol-value, (typep ... 'symbol), etc.
8:23:04
beach
I claim it is not madness and that most CAR/CDRs are done in loops so the test has to be made anyway.
8:24:27
beach
So a single test for CONSP is almost always going to be TRUE which is the same number of tests you need in order to check for non-NIL atoms.
8:28:03
beach
But the thing that bothers me is that "optimizations" like this are put in as a result of intuition, and we know for a fact that our intuitions are frequently wrong when it comes to performance bottlenecks.
8:28:33
beach
And as a result we have, like I said, systems that are harder to maintain without any good reason for it.
8:29:44
aeth
I never directly call car or cdr... destructuring-bind is almost always the better solution.
8:36:38
pfdietz
"it's just like a macro lambda list" (looks) "the pattern is a sample object of the type to be decomposed."
8:40:37
pfdietz
I think we can safely criticize the language definition about this, though. It would have been ok for car/cdr of nil to be an error. I think this was inherited from Interlisp when CL was being defined. For that matter, punning () and false is also bad.
8:48:16
pfdietz
I see about 26000 occurrences of car in quicklisp. Picking a few at random, they don't seem to be in loops, at least not in tight ones. On the other hand that's not all that many tests to add, especially if many could be optimized out anyway. Real data on the impact would be interesting.
8:48:50
beach
And, like I said, even if it is not in a loop, you still need to test for non-NIL atoms.
8:48:56
pfdietz
That's just from a simple grep; it ignores implicit car/car from destructuring or looping forms.
8:49:40
pfdietz
If you just want an error to be signaled in that case you don't need an explicit check.
8:53:38
phoe
also all sequence operations have implicit car/cdr calls when called with list arguments
9:00:08
pfdietz
assoc is interesting. The spec explicitly says that a nil in an alist is ignored by assoc (and assoc-if, assoc-if-not). So (assoc nil '(nil (nil . a))) ==> (nil . a)
9:03:51
pfdietz
cons n.v. 1. n. a compound data object having two components called the car and the cdr.
9:08:21
phoe
"On PPC32 and X86-64, NIL is basically a weird CONS cell that straddles two doublenodes; the tag of NIL is unique and congruent modulo 4 (modulo 8 on 64-bit) with the tag used for CONS cells."
9:08:24
pfdietz
"The types cons, symbol, array, number, character, hash-table, function, readtable, package, pathname, stream, random-state, condition, restart, and any single other type created by defstruct, define-condition, or defclass are pairwise disjoint"
9:09:12
beach
phoe: That's the kind of "optimization" I was referring to earlier in this discussion.
9:10:53
pfdietz
So: the question becomes what is the actual impact of this design choice, vs. adding a check at each car/cdr. I suppose one could also trap car/car on nil and insert do the right thing in the trap handler.
9:12:21
beach
And I think the main impact is an increased maintenance burden. Especially since modern Common Lisp code uses standard objects where in the past lists were used.
9:13:50
phoe
well, unless your internal representation somehow manages to handle NIL being both a symbol and a valid argument to CAR/CDR, checks need to be somewhere
9:14:15
phoe
either it's internally represented as an atom and therefore CAR/CDR need to check, or it's internally represented as a cons and symbol accessors need to check
9:14:32
beach
So it is entirely possible that using a sub-optimal algorithm for generic dispatch is going to be much more detrimental to performance than having an explicit CONSP test.
9:15:09
pfdietz
Surely inlining the test like that makes sense here. car/cdr are not extensible with new methods, after all.
9:16:26
beach
pfdietz: That's not what I meant. I meant that, with a limited amount of person-power for maintenance, it would be wise to add complication where they have the most positive impact on performance.
9:17:51
beach
But I can also see how design choices like this were made when the implementation was a CLtL1 implementation so that lists were used a lot more than they are these days.
9:17:56
pfdietz
Sure. I was just wondering what the increased maintenance burden is buying, with reference to actual code in quicklisp. If it's not much, your argument becomes strong.
9:18:56
phoe
unless we have some sort of an implementation that we can modify and then benchmark on code
9:19:59
phoe
by "code" I mean both synthetic benchmarks that measure the performance impacts of raw operation and real applications that do little/moderate/lots of cons operations
9:23:00
pfdietz
One could shadow car/car in your code's packages (and, perhaps, library packages) with something that is inlined and does the checks, and see what effect that has.
9:23:31
pfdietz
To handle all the libraries, use a macroexpand hook to intercept defpackage and add the shadowing.
9:25:32
pfdietz
This would not handle car/cdr introduced in loops and mapping form, but as Beach argued those should not matter.
9:30:35
aeth
I essentially always just destructuring-bind and setf because d-b handles the edge cases in the way I want, unlike LOOP's destructuring.
9:31:15
pfdietz
destructuring-bind macroexpands to code that invokes sb-c::check-ds-list and sb-c::check-ds-list/&rest. I hope those are inlined.
9:33:46
Demosthenex
so i'm tinkering with clsql and the [] notation for sql, but emacs paredit isn't recoginzing those and indentation is a problem. any suggestions?
9:38:03
pfdietz
I have [ and ] mapped to paredit-open-parenthesis / paredit-close-parenthesis in the paredit-mode-map, but I use curry-compose-reader-macros where [ and ] represent function composition.
9:43:06
pfdietz
ISLISP does not have (car nil) and (cdr nil) being valid. It's mostly historical at this point, though.
9:45:15
ldbeth
Demosthenex: https://github.com/cireu/sly-el-indent here's some hack on sly (SLIME) to indent emacs lisp code, you might want to see if thoses hacks apply to your problem
9:48:50
pfdietz
trivia:ematch seems to generate faster code that destructuring-bind, for equivalent forms.
9:59:54
pfdietz
(scrolling up) The inlining of dispatch is made more exciting by the possibility that classes and methods can be dynamically modified. What if you need to recompile a function to reflect these changes while a call to that function is on the stack?
10:03:02
pfdietz
I figure you want to compile such functions with some sort of standardized stack frame, so you can return back to an unoptimized, or differently optimized, version of it if that happens.
10:03:26
aeth
pfdietz: I think, in general, you want to be fast assuming no dynamic redefinitions, although obviously permit the slow path to happen
10:03:46
aeth
most dynamic redefinitions will be user-defined, and infrequent, at least if you're assuming e.g. running in aloop
10:04:24
aeth
I do wonder how much of this is handled by CPU's branch prediction these days, though.
10:07:01
no-defun-allowed
Amazing, I was looking to see if Self had come up in #lisp before. Someone didn't reach for the search button.
10:10:09
no-defun-allowed
Perhaps it's too late for #lisp-ing, but I needed something to do while rebuilding SBCL with the higher internal-time-units-per-second so I can get less-than-awful variance with metering.
10:36:51
phoe
but I won't use it, because I just realized that I was trying to do something that did not make sense in general
11:10:34
aeth
phoe: In my Lisp, let* will just expand to destructuring-bind with an &aux since having both is unnecessary!