libera/commonlisp - IRC Chatlog
Search
22:27:21
AeroNotix
I've said it previously when this channel was on freenode but imho CL is in need of a modernized standard
22:28:02
AeroNotix
on the face of it CL being so set in stone itself seems like a good thing but you end up with balkanization of useful features when they are implemented in poor libraries
22:28:20
moon-child
EdLangley[m]: it would be better if those portability libraries were standardised
22:28:31
EdLangley[m]
So, I've seen python people say things like "the standard library is where libraries go to die"
22:28:45
AeroNotix
Modern concurrent programming is, imho, a large feature which CL lacks. Libraries are all over the place and maintained by the few, pushed to the limits by no-one
22:28:47
phoe
if we had a standardization committee member every time this topic was mentioned then we would have multiple X3J13 by now :D
22:29:07
AeroNotix
and if they are reasonably halfway decent, they're supported by only a couple of implementations
22:29:30
AeroNotix
imho the fact CL is so stagnant is a big part of why its popularity stays so niche
22:30:04
EdLangley[m]
As far as threads goes, the implementations implement threads and the portability library standardizes the interface
22:30:07
_death
what may be good is more specifications (extensions to the standard) that are agreed upon and implemented by lisp implementation vendors.. but it requires $$$.. if only there were some rich Lisp people who were interested in pursuing this ;)
22:30:43
moon-child
EdLangley[m]: the interface is not standardised to the level of quality I would like
22:32:15
AeroNotix
my suggestion is that no standard in a platform and community such as CL's leads to terrible balkanization where no-one is the winner
22:32:33
EdLangley[m]
It's a bit strange to complain about the standard causing stagnation and then suggest standardizing more things would help
22:33:03
EdLangley[m]
So, if we come up with better interfaces and/or techniques, they can only be implemented as libraries on top of the standard
22:33:05
AeroNotix
EdLangley[m]: I think standardization is good, and CL in particular lends itself well to that model
22:33:28
EdLangley[m]
Which, to me, suggests we make the standard as minimal as possible to support a healthy library ecosystem
22:33:53
EdLangley[m]
And have libraries to sort out the vendor extensions into things that are portable between implementaitons
22:33:53
moon-child
sure. For many things that works. But, again, _threads cannot be implemented as a library_
22:33:54
AeroNotix
my opinion is that because the standard is coming up on quite literally 30 years old, it's missing out on a lot of progress
22:35:28
AeroNotix
EdLangley[m]: we have the situation you mention, 'standard as minimal as possible'
22:35:37
moon-child
sure. Where is my memory model? Can I be sure that abcl, ccl, and sbcl all implement the same semantics?
22:36:03
phoe
I think this problem is always answered best by "who's supposed to do this and who's supposed to pay them"
22:38:38
phoe
"it would be nice if CL had a revised specification" is a truism; the real meat of that issue is who are the people and whence comes the money, everything else has already been said probably hundreds of times before
22:39:11
phoe
and until this meat has an appropriately meaty answer we are bound to repeat history, except as farce
22:40:57
contrapunctus
Maybe it's time to start something akin to XEPs and SRFIs? Sounds like that way the existing standard remains untouched, but there's a library standardization process...
22:44:01
_death
EdLangley[m]: the updates could constitute clarifications etc. like WSCL.. but there's a lot of room for new specs, some of which don't actually require participation of lisp implementation vendors, but may require participation of lisp library writers in particular fields
22:44:43
AeroNotix
I personally wouldn't care which form it takes... the above sounds reasonable to me
22:45:46
AeroNotix
my point is more that people are off in the weeds implementing modern tooling on 30 year old software
22:46:10
AeroNotix
and we have pockets of brilliance which are tarnished by the fact there's no cohesion between any thing
22:46:22
contrapunctus
phoe: even append-only is fine, I guess, since newer ones can supercede older ones. cf. XEPs - each one gets a new number, it's just that the older ones are rarely changed and the current set of relevent XEPs changes; the older, deprecated XEPs don't go anywhere.
22:46:40
AeroNotix
libraries fester, are incomplete or lack the performance or usability required because implementations are well, 30 years old and are unchanging to their needs
22:46:44
_death
again, this requires time and will, and the fuel is $$$.. PG went back to roots with Arc and Bel.. maybe someone should point out that the next language starts with the letter "C" :).. there are potential patrons for such a process who were (or even are) part the Lisp world.. I dunno why, but I've not seen any of them talk about that, though they usually do show some interest in Lisp
22:48:20
fe[nl]ix
if you look at past documents, they were introduced and marked as "final" within a month
22:50:56
contrapunctus
fe[nl]ix: hm...did implementation maintainers not participate in the CDR process? And did nobody request or aid CDR implementation? 🤔️
22:50:59
EdLangley[m]
If we have the library and it's implementable on top of what existing implementations provide, it's mostly fine
22:59:29
_death
fe[nl]ix: what's interesting though is that something like PLN, which has no real specification, was implemented by several Lisp vendors..
23:00:39
fe[nl]ix
and yes, there was enough demand that mostly similar functionality was implemented
23:01:33
fe[nl]ix
if you take a look at https://github.com/phoe/trivial-package-local-nicknames/blob/master/trivial-package-local-nicknames.lisp
23:03:54
fe[nl]ix
EdLangley[m]: most of the things that are important require changes to the implementations
23:04:50
_death
right, so this could be a first target.. the variants are not too different, and it's recent enough, and could improve via pressure of standardization
23:04:58
fe[nl]ix
I'm working on making Bordeaux-threads APIv2 as good as possible, with lots of tests but there are limits
23:05:28
moon-child
(well. I guess technically you can also do completely portable reader, and your own PLN. SO maybe it does not signify much)
23:05:44
_death
(btw personally I don't use PLNs.. I just give it as an example of something not too complicated that's ready for standardization)
23:10:31
_death
fe[nl]ix: this all reminds me of what I wrote a while ago https://old.reddit.com/r/Common_Lisp/comments/j7vd25/a_curated_standard_library_for_common_lisp/g89544v/ .. if anyone's interested
23:17:15
nij-
>> etimmons> A sufficient reason to not include it is how do you reconcile two systems that depend on the same system but at different versions/hashes
23:20:33
nij-
Hmm I think you are right, etimmons, unless the system with two different hash-specifiers (versions) can be loaded simultaneously without name clashings..
23:21:55
nij-
A hack of this seems to require a hack of how package works now. E.g. instead of 'nyxt:start, we need 'nyxt-0sa98d:nyxt:start. (where 0sa98d is the hash)
23:30:42
nij-
qhong: The usual strategy used by nix or guix doesn't work immediately here. By loading two different versions of the same system, the packages will be collided.
23:32:37
qhong
it actually sounds pretty cool to implement sth like nix/guix, but totally within a lisp process (like how make compare to asdf)
23:34:47
EdLangley[m]
A way to have some sort of sub-lisp system where you can load all a system's dependencies and then only expose a specified set of packages to the "global" lisp
23:37:33
EdLangley[m]
It would let you load multiple versions of a single system into one lisp image
23:41:03
EdLangley[m]
I think sicl can be loaded in sbcl and is somewhat usable? It's been a while since I've heard much about it
23:43:04
fe[nl]ix
first-class environments means implementors have to rewrite their compilers, and requires new APIs
23:43:29
fe[nl]ix
it will be a miracle if just one of the major implementations gets this feature in the next 5 years
23:45:21
mfiano
qhong: That depends on which FCE you are referring to. There are several, each very different from each other.
23:47:35
qhong
mfiano: I rather view it as very unambiguous. Like concretely different language may have very different semantics of lambda, but we in general agree that lambda is that one thing in lambda calculus, approximated to different extent by each language
23:48:04
qhong
mfiano: FCE on the other hand is simply "the reification of the \rho in lambda calculus' standard denotational semantics"
23:48:22
qhong
just like first class continuation is "the reification of the \kappa in lambda calculus' standard denotational semantics"
23:48:29
nij-
Naive question: What ever FCGE is, it should be contained in itself, right? How does that work? Self pointing object?
23:50:54
jasom
nij-: don't know how it works, but I would assume that it would make a tree (that is the root global-environment could contain pointers to other global environments and so on)
23:54:39
jasom
It seems like it's optional for any given FCGE to contain a reference to itself. Imagine that you have a working FCGE, if you remove the pointer to itself, you only lose the ability to directly query and manipulate the environment, which you didn't have in a CL without the FCGE
23:59:22
nij-
fe[nl]ix: I'm a bit confused. Suppose SBCL wants to introduce FCGE. What would be the hard work? Well, the global environment is stored in some format already. They just need to write some API to expose that global env, right?
23:59:56
jasom
nij-: instead of "some format already" think "spread out all over the image, without any thought to making clones"
0:00:13
nij-
sb-cltl2 module provides compiler-let and environment access functionality described in Common Lisp The Language, 2nd Edition which were removed from the language during the ANSI standardization process. http://www.sbcl.org/manual/
0:03:06
moon-child
mfiano: first-class global environments are a language feature that happens to be useful for bootstrapping one implementation
0:04:03
mfiano
THe paper explicitly mentions it did not research their use for application developers, and defers to problems in doing so in related works.
0:07:03
EdLangley[m]
Anyways, sicl already uses them to bootstrap itself. So, someone could probably figure out how to write an ASDF component that uses a similar technique.
0:08:47
jasom
Hmm, reading the environments paper, the package system is not affected; does that mean you couldn't have two environments in which the set packages would have conflicting imports?
0:13:49
EdLangley[m]
Yeah, ideally you'd also use a separate reader like Eclector to virtualize the package system too
0:15:37
jasom
well I wasn't sure because it says "For a given package P and symbol name N, there is at most one symbol with the name N in P" but does not clarify what "in" means; I could potentially see that not applying to imports, though the surrounding text implies it does.
0:41:43
dbotton
Does IN-PACKAGE work with in an EVAL? in a PROGN? I get strange results one time no, one time yes
0:44:56
White_Flame
what do you mean by "work"? affect *package* for the calling environment, or for peer forms within PROGN?
0:48:26
White_Flame
the literal symol whose symbol-name is "SOMETHING" in that expression; that's an extant symbol in a sexpr list that is xyz:something, or that is a string on the REPL that is being read?
0:50:26
White_Flame
ok, so while (something) is executing, *PACKAGE* should be XYZ, as well as when it returns
0:50:45
EdLangley[m]
IN-PACKAGE would affect package, but by the time you call EVAL, it's too late to adjust the package read symbols belong to
0:50:59
White_Flame
right, if it's a constructed sexpr, though, I don't think the reader is involved
0:51:29
White_Flame
hence asking if the symbol xyz:something is already constructed before that EVAL form is reached
0:52:33
White_Flame
so, when you read "(eval '(progn (in-package ....)...))", the value of *PACKAGE* will be used to construct the symbols well before EVAL or the inner IN-PACKAGE is evaluated
0:53:25
White_Flame
so when running a file, the reader needs to be called per toplevel form and then executedevaluated before the next toplevel form is read
0:54:47
Alfr
EdLangley[m], in-package sets cl:*package*, and the compiler has to treat top-level in-package specially to have the same effect, that's about it.
0:56:15
White_Flame
EdLangley[m]: yes, lisp source code forms are mutators against the running image
0:58:13
White_Flame
no, macro use of functions are compile-time, not load time. so eval-when boosts it earlier
1:00:04
Bike
if (setf package (find-package :foo)) is a toplevel form with no eval-when etcs around it, it will run at load time
1:00:17
Bike
you can kind of think as :load-toplevel :execute as being the default state before any eval-whens come in
1:00:48
White_Flame
dbotton: heh. Well, when you _read_ the string in to a list, that's where the packages are assigned to the symbols
1:01:17
White_Flame
and that happens before eval. So you need to (let ((*package* (find-package :xyz))) (read-from-string "(progn ...)"))
1:02:45
White_Flame
so yeah, if you have a variety of packages you might want to read inside of, then that'll generally have to be sideband information outside the string
1:03:05
White_Flame
"(progn #.(in-package :xyz) ...)" is also a terrible hack that will work, but also leave the caller's package changed
1:04:06
White_Flame
specifically, that it's around the read-from-string (or whatever READ-based thing) that makes it work
1:04:54
White_Flame
yep, just like toplevel .lisp (in-package ..), the compiler has to stop and swap packages before continuing the next read
1:06:01
Alfr
dbotton, you could use read and consume what you get form by form and check for an in-package form.
1:07:34
Alfr
dbotton, if you have a string, you may want to use with-input-from-string, so that READ will like it.
1:11:18
dbotton
EdLangley[m] take a look at - https://github.com/rabbibotton/clog/blob/main/demos/03-demo.lisp
1:13:29
White_Flame
(loop for line = (read-from-string ...) while line eval line) would support things like (in-package) moreso than "(progn ~a)"
1:19:25
White_Flame
once you have teh symbol CL:PROGN... yeah, all the other symbols are already read, too
1:20:37
EdLangley[m]
to pre-process the source code with its reader and turn all the symbols into PACKAGE::FOO
1:21:20
EdLangley[m]
I've done stuff like this occasionally when I want to print the code in a way that makes symbol packages unambiguous
1:22:04
EdLangley[m]
Yeah, because the first eval changes the package and then the second one evaluates in the right package
1:23:45
dbotton
that should not affected code already running correct? (ie it has already been read)
1:25:58
EdLangley[m]
I was doing this when I was serializing the code to a db and wanted it to be as unambiguous as possible
1:26:04
White_Flame
if there is #. or #= it won't round-trip and might mess with the timing & identity of objects
1:27:23
EdLangley[m]
:use :cl has the downside of making the code a bit ambiguous, because you have to assume it's always going to be read in a package that doesn't shadow symbols from :CL
2:17:35
EdLangley[m]
If I ever really return to this, I'm going to try to figure out how to use eclector to make a reversible reader
2:20:59
EdLangley[m]
I've sort of prototyped this for #+ and #-: https://github.com/fiddlerwoaroof/lisp-sandbox/blob/master/eclector-test.lisp
2:24:25
White_Flame
a reversible reader, assuming things don't mutate in the meantime, and just store the original string it read and print that as its representation :-P
2:27:46
White_Flame
just to be specific/pedantic, the brunt of the work goes in the printer, not necessarily extending the existing reader
2:40:00
EdLangley[m]
The idea would just be to have a s-expression format that doesn't lose things like #+feature or #.(...)
2:40:59
Alfr
White_Flame, just snapshot the universe for rollback and don't forget to destroy the one you abandon.
3:45:35
moon-child
drmeister, Bike: are you using llvm's built-in support for gc in clasp? If so, how have you found it?
4:12:52
mzan
Hi, I'm learning CL. Up so far this is the most complex code I wrote https://aoc-benchmarks.dokmelody.org/linux/program.php?test=aoc2021_day03b&lang=lisp&id=3
4:14:10
mzan
I like the (iter ...) macro a lot. But the (iter ...) macro initialize some variables to "nil" sometime, and the SBCL compiler refuse to apply some type optimizations because it sees an initial nil value and sometime it is not smart enough to figure out that the "nil" value will never be used.
4:16:22
mzan
My impression is that the majority of macros works well together but sometime there is some leak. It is acceptable, but it is rather frustrating if one coded in more rigid languages like Haskell.
4:17:48
mzan
In this case specific, the leak was only related to optimizations, but not real semantic. Probably there are no many leaks on the side of the semantic.
4:19:13
moon-child
mzan: in such 'rigid' languages, you would probably not be allowed to write such code in the first place
4:21:02
moon-child
re iter/optimization: there was some loop/iter-like macro that purported to generate good code for pipelines of transformations. I can not now remember what it's called, but someone else may...
4:21:03
beach
mzan: There are several irritating style problems with your code, like mysterious blank lines, wrong number of semicolons in comments, incorrect indentation.
4:26:36
mzan
sm2n: I read something about type declarations in "iterate", but I will study it better. Thanks!
4:27:34
mzan
BTW, in case of doubts I can expand the code generated by the "iterate" macro, so for sure the problem was not obfuscated by the environment.
4:28:45
mzan
Probably in future I will try to micro-optimize the code, removing left notes signaled by SBCL, and benchmark the difference.
4:30:36
mzan
moon-child: I tried SERIES. In theory they should be superior to "iterate", because you can reuse chunks of code transforming data in different places. SERIES support this.
4:31:20
mzan
In practice, up to date, rarely I feel to write more code in "iterate" than in SERIES. And "iterate" is more flexible and more in line with the imperative semantic of some parts of CL.
4:34:16
EdLangley[m]
I think the sort of implicit stream-fusion SERIES does ends up being pretty limited in CL
4:35:14
EdLangley[m]
I've been experimented with transducers (from Clojure) that work more explicitly by passing a continuation around, and I think they're a better fit for lisps
4:40:14
mzan
Another thing I didn't liked to CL, it is that there is some lack of basic data-structures. In Haskell there is a rather efficient IntMap for example, and other data structures. C++ has plenty of them.
4:40:36
EdLangley[m]
The trick behind them is pretty simple: https://twitter.com/fwoaroof/status/1337667255727886337
4:47:16
mzan
ahhh "(1+ ...)". I din't know there were this function, but I suspected there were something of similar! :-)
4:50:41
mzan
EdLangley[m]: I don't know TRANDUCERS, but you example seems anti-intuitive to me. You are usinng a "reduce", but you pass a function that creates a stream (a vector in this case).
4:51:26
mzan
In your example, "compose" apparently do not work on streams using fusion, but only on the entire result.
4:53:47
mzan
In my ideal world, I would like stream transformers with an optional internal state, that can be fused. Then some final reducers.
4:53:59
EdLangley[m]
So, functions like MAPPING return a function that takes a continuation as an argument
4:54:54
EdLangley[m]
This returns a function with the signature you'd pass to REDUCE: (ACC NEXT) -> whatever
4:55:46
EdLangley[m]
This function, instead of building up a result with an explicit constructor, calls RF and passes an appropriate value of ACC and NEXT to it
4:56:09
EdLangley[m]
So, for MAPPING, this means something like (funcall rf acc (funcall transform next))
5:01:06
mzan
I don't know CL enough for understanding all your code. But it is usefull reading it for me.
5:04:08
EdLangley[m]
But (compose (mapping f) (mapping g)) doesn't build up an intermediate collection
5:04:35
moon-child
i.e. instead of (compose (mapping f) (mapping g)), you can say (mapping (compose f g))
5:05:24
EdLangley[m]
With a full library, you can do like: (compose (mapping #'1+) (filtering #'evenp) . . .)
5:05:58
EdLangley[m]
Basically, any operation that can be expressed through the lambda you pass to REDUCE, you can do this way
5:06:38
EdLangley[m]
Yeah, but the point is to preserve the (mapcar (lambda (b) ...) (mapcar (lambda (a) ...))) style
5:07:58
EdLangley[m]
This also abstracts from the concrete result type and from limitations like "the input sequence must be finite"
5:08:39
EdLangley[m]
You can express an operation once and pick whether it builds a list, a vector or puts results on a channel of some sort at the use-site
5:08:52
moon-child
my primary exposure to this sort of style is apl (not clojure), which is very strongly oriented _away_ from streams
5:08:54
mzan
EdLangley[m]: SERIES are composable, in the sense you can reuse a complex SERIES definition as starting point of another SERIES. Is your paradigm composable?
5:10:45
mzan
so something like (defun even1+ () (compose (mapping #'1+) (filtering #'evenp)) can be reused, while a "loop" no.