libera/commonlisp - IRC Chatlog
Search
2:34:43
sukaeto
late to the conversations, but I (think I) understand what beach is saying re: binding
2:36:03
sukaeto
they sometimes use it the way a compiler writer would - as in "this variable is bound to this value"
2:36:28
sukaeto
they also use it in the logical sense - as in "this variable is not free in this form. It is bound."
2:39:20
sukaeto
in the second sense, you're saying the variable is bound by the *context*. (progv '(*x*) () (boundp '*x*)) <- *x* is bound in the sense that it won't be susceptible to variable capture, no matter where that progv is put
5:05:52
beach
lisp123: You can figure out the answer by checking for those in existing code. So if I take SICL for instance, we seem to have 25 :before, 31 :after, and 53 :around.
5:10:11
lisp123
I was wondering how useful they were since you could also bundle some of that code into the primary methods. In Sonja Keene's book, they note it as useful for code reuse (this was for :before and :after at least), where more specific classes can do some additional side effects on top of the primary method, either before or after
5:12:14
beach
lisp123: When I taught a small class at the university of Auckland, I showed the students how, if you don't have auxiliary methods, then you might sometimes have to change the interface to obtain certain optimizations, so that's a no-no for stable APIs. They were totally convinced.
5:12:36
beach
lisp123: So you can obtain the same effect with only primary methods, but you can't then also have your software modular.
5:13:11
beach
You probably haven't seen that issue, since I bet your code is fairly small, and you write mainly for yourself.
5:13:47
lisp123
Yes, I don't use much state at all or have large code at all, hence wanted to ask what was used in bigger projects.
5:14:29
lisp123
So basically, the primary method gives the primary interface which clients use, and then you add some stuff before/after/around it?
5:14:35
beach
Then you just have to take my word for it, that auxiliary methods are a must for larger projects, and especially for stable libraries.
5:15:25
beach
Yes, one common thing to do is to have the library define an :around method that can intercept the client call, and do special things then.
5:17:43
beach
Common Lisp has a lot of that kind of stuff, i.e., features that are almost never used, but when you need them, you really can't do without them. Since other languages often don't have these features, they have to sacrifice modularity, maintainability, performance, etc.
5:18:49
beach
For instance: custom metaclasses, custom method combinations, custom function types, reader macros, macros, etc., etc.
5:19:07
lisp123
Yes, its very well thought through. I basically assume anything in CL is there for a very good reason
5:19:40
beach
So the incorrect conclusion made by people who don't like Common Lisp is that, since these features are rarely used, they aren't really useful. Very wrong!
5:30:14
jeosol
beach: I do agree that for large projects, those auxilliary methods are a must. I application have non-trivial hierarchy and those :before, :after methods do help to implements things clearly.
5:31:44
jeosol
I am you mention maintainability and modularity of the codebase, these are important for nontrival programs, otherwise, you get a mess that no one wants to deal with or maintain
5:37:09
hayley
I once used a lot of :after methods for a "reactive" programming style, but then hurt my head on concurrency. But I suppose that's orthogonal to the utility of auxiliary methods (though having behaviour in one place makes modelling marginally easier).
6:02:32
beach
Also, Common Lisp is designed to make these features easy to implement. There is no need to have a committee that votes on new syntax. It can all be done in the form of library code.
6:03:55
jeosol
beach: yes. I have coded in C++, not a huge program, but it was a pain. The CL project I have been working for a while, is way larger, but the logical and physical design is much easier to manager.
6:04:27
jeosol
it's hard to make other see, the fact you don't have to fight the language and it does help you all the way
6:05:01
jeosol
not worrying about excessive compile/link-time issues with C++ header files otherwise, your build takes forever
6:05:29
jeosol
beach: you are so right - this is something I have tried to explain to others why I use CL
6:06:08
jeosol
I am like the language is a joy to work in. I am only concerned about the concepts I am trying to implement, i have uniform syntax and don't have to worry about a specific, different language to use a given point
6:06:40
jeosol
beach: very true, unfortunately, I still have to do that in the next few months while I discuss with others
6:07:12
beach
I wish you good luck. Perhaps you will find a better way than I have been able to come up with.
6:07:35
jeosol
reading large-scale C++ software design, the author mention a project that was taking the order of weeks to build - large software indeed, but with our incremental development, we get our changes integrated right there
6:08:11
jeosol
beach: I don't think so beach, I have tried the last few weeks and evening compiling some tidbits here and there to show how things are easier in CL
6:08:33
lisp123
jeosol: Although "uniform syntax" sometimes can be a misnomer. Its true the basic syntax of lisp languages are relatively simple, But then, with special forms or certain macros, complexity creeps in
6:09:05
beach
lisp123: But it is nothing like the complexity of the surface syntax of most languages.
6:09:13
jeosol
I haven't even talked about macros with them because I use that to generate a lot of functions at compile-time so I don't have to code them by hand - a lot of my functions are generated this way (still have to estimate the amout, not sure how to do that)
6:10:35
jeosol
lisp123: good point, I was simplifying things by "uniform syntax". What I mean is that there are way fewer language syntax things I have to worry about - but it kind of goes to the fighting with the language do what you want, or just focus on the concepts you want to implement
6:11:07
lisp123
Honestly, one of the issues for me initially was 'how can lisp be so simple and yet do what I want it to' -> especially with english-named variables and functions, it feels like reading pseudocode at times, but it actually works and works well
6:12:17
lisp123
jeosol: Yep agree. Just wanted to point it out since its worth mentioning to non-lispers that there is some syntax down the track that requires memorisation (but its more to do with having forms in certain places vs. punctuation etc)
6:12:55
jeosol
I work with optimization, and at the time (grad school) working with matlab (memory and speed issues) and C++ (moved because of speed), I saw a CL implementat of genetic algorithm - wow, it was so succint, the guy had my whole C++ header files and CPP files in a few pages of his theses
6:13:18
jeosol
I was sold, because of time, I used C++ to finish, but started looking into to CL then.
6:13:58
lisp123
jeosol: I still feel a bit uncomfortable reading what appears to be very simple code and it works. It feels like its _meant_ to be more complicated. Hopefully that misperception finally goes to rest as I use Lisp more and more.
6:15:17
jeosol
beach: the finding better way is what I am trying to do - I don't have same background/expertise with CL but gotten more CL related books to read over the years.
6:15:37
jeosol
hayley: The problem is people don't like to use it or complain, it's an old language etc
6:15:44
beach
lisp123: Paul Wilson describes Lisp as having a 2-level syntax. One is the syntax defined by sequences of characters, and the other is the syntax at the S-expression level.
6:16:12
jeosol
In a previous shop mostly using python, I got complaints for using CL ( for personal tasks) and for even using emacs editor when most people used vscode
6:16:32
lisp123
hayley: Work wise it can be off putting to others who don't use Lisp, so I need to bite my lip sometimes. Also I'm demotivated on some of their projects, if they are using python or something else, I don't feel the desire to improve their code (why not just do it in Lisp)
6:17:57
jeosol
well it was an ML/AI shop, so the use of python is sort of needed. But some of the older symbolic logic AI code, were CL (using SBCL and lispworks) but no one except a programmer from the 80s and myself knew lisp.
6:20:00
jeosol
some good points on the thread regarding CL vs other languages. I know PG has this whole CL superiority thing, and once told me (when I was starting) to use clojure instead, that it's a modern lisp, but after I mentioned I need/was using CLOS, he then conceded that I should stick with CL
6:20:17
lisp123
jeosol: Yeah, my biggest gripe is when people say why are using Emacs over VSCode. Of all editors, VSCode might be the worst. I rather use notepad at this point (at least I assume there is no telemetry there)
6:21:02
jeosol
like beach, having language fights is not something I am interested in, but I have often needed to explain why CL was chosen for a project ...
6:22:11
jeosol
my OS and tools are: Linux (OS), Emacs + Slime + Paredit. I know there is sly and other emacs variant that my improve my workflow, but I just stuck with these
6:26:02
jeosol
beach: is the Paul Winston comment from his and Horn's book or paper . It is ok, if you don't remember
6:27:48
jeosol
beach: is this the Hudak and Jones paper you referred to above: https://www.cs.yale.edu/publications/techreports/tr1049.pdf
6:29:37
hayley
"For those who object to this use of functional languages, we suggest reading the rest of this paper as if it were about a video game."
6:32:21
beach
I spent a year with Paul Wilson and his research group at the University of Texas at Austin. I am not sure whether he published anything about the 2-level syntax. He may just have told me.
6:32:40
jeosol
beach: Apologies, I mixed up the names and went to pick up the Winston book from the shell
6:34:20
jeosol
beach: I will be interested in a collection of relevant references like the Hudak ones, references that talk about building programs, nontrivial ones in CL. I know this task is not specific, but I am just trying to study these more
6:39:40
jeosol
beach: skimmed the Hudak and Jones paper - very interesting. Thanks for suggesting this
7:42:25
beach
UT Austin is also where J Moore (ACL2) and Gordon Novak (AKCL) work. And it was where Dijkstra had a chair position while I was there.
8:11:31
lisp123
beach: No, I use org mode at work for knowledge management (quite useful as a database). I experimented with Common Lisp at work, but the problem is if I leave, what the next person will do
8:12:07
lisp123
The other staff use python (we are not a tech company, in investment management, but cannot afford full time developers, so do a bit here and there).
8:12:21
beach
Learn Common Lisp? If they are serious developers they should have learned it already.
8:13:10
lisp123
I just interviewed for a role at my old firm, which is much larger and has a dedicated tech team. It will be partly project management of their work...however again, Python and Java are the languages of choice there
8:15:20
lisp123
I will definitely try to do that! I think we will see more usage of CL going forward, many other languages already seem to be using similar concepts, so its just a question of jumping across
8:15:36
beach
To me, the main thing that the Hudak and Jones paper shows, is that the cost of teaching the developers a new language is ridiculously small compared to the waste of time that an inappropriate language creates.
8:17:25
beach
Carol Dweck calls it "closed mindset" and it is very dangerous for software development.
8:18:30
lisp123
Bad decisions compounded onto other bad decisions to justify them, can take away millions very quickly
8:20:12
lisp123
I'd be very curious to have insight into how big tech firms work (FAANGS). From all accounts, there are a lot of closed mindsets there, and a lot of wasteful development
8:20:21
pve
Is the "Relational Lisp" mentioned in the Hudak and Jones paper based on CL? Some searching suggests it might be the case..
8:20:53
lisp123
From what I have read online, they have become quite political. Which is interesting, because they also have some cutting edge technologies & software (partly now due to the massive R&D budget they have)
8:21:40
beach
pve: I don't know. When I discuss this paper with students (for instance) I don't mention particular languages. I concentrate on what I think is the main lesson of the paper, namely that there can be a factor 20 difference in productivity depending on the language, and that's huge so should be taken into account in projects.
8:24:42
beach
pve: The way I put it is that a 5-year project could be completed in 3 months instead.
8:28:55
beach
So you can spend 4 years and 6 months training the staff on the new language. Of course, the paper shows that productivity can be obtained after two weeks.
8:30:33
Nilby
But given all that time saved, who can resist doing something more ambitious? I can't.
8:31:30
pve
beach: 3 months vs. 5 years is so incredible that, when first hearing it, people's first thought (and response) must be something along the lines of "There's no way that's possible".
8:31:48
beach
Nilby: So you are saying we should try to be as little productive as possible to limit our level of ambition?
8:33:13
beach
pve: Right, so in the next phrase (to my students), I say that even if they don't believe a factor 20, even a factor 2 or a factor 1.2 is big enough to justify taking it into account.
8:33:31
mfiano
I came from several years of Python experience, which is known to boast "rapid application development". I read a Lisp book for a week, got dirty with some code for another week, and at that point I was already faster and writing better code.
8:33:42
beach
But that right now, decisions about languages are based on what existing programmers know.
8:34:27
Nilby
beach: I don't know what to do about it, but nearly every time I do something in CL vs some blub-lang, I do it with 10x more features or more ambitious design.
8:36:30
mfiano
What helped was to take a week, month, or even longer to write a complete design document before a line of code is even written.
8:38:04
Nilby
Except when I used to do boring stuff for work, then I would just put it off until the last hour before it's due, or do it in the first few hours then goof off for a week. Doing it in CL vs C++ allowed me to do that for years.
8:40:05
Nilby
beach: But, I know non-lazy people, e.g. like you and Shinmera actually use that time to be more productive.
8:45:05
pve
My first task related to CL was porting an NLP application (between 50-100 kloc) from Allegro CL to CMUCL during a summer job. I remember it took me about a month to get comfortable with slime (already knew emacs) and complete the port, even though I didn't really know what I was doing at the time. It was mostly mechanical work.
8:45:38
hayley
There have been similar results with TLA+ (which also relates to having a good design process). "Engineers from entry level to principal have been able to learn TLA+ from scratch and get useful results in two to three weeks, in some cases in their personal time on weekends and evenings, without further help or training."
8:57:33
pve
I should also mention that I was quite open to the idea of learning CL. Later, I saw other people join (to work on components written in Java) who were effectively unable to learn CL because they weren't open to the idea (i.e. they resisted).
8:59:48
pjb
lisp123: In general, I try to leave :before :after and :around methods to client code, but they can be very useful to implement some mechanisms. For example using an :around method to implement a mutex that will be automatically applied to the primary methods even if overridene.
9:08:15
beach
pve: The "closed mindset" is a very powerful psychological force, so incredibly hard to combat.
9:13:32
pve
beach: Yeah, no kidding! I (not an expert) spent a great deal of time showing CL, slime and interactive development to people there, but it just didn't stick. One guy did get it though, so there's that.
9:14:40
beach
I estimate that, during our 15 years of using Common Lisp in our teaching , on the average around 5 our of 125 annual students understood the message.
9:24:42
hayley
Similar is that Max Stirner once wrote that everyone is an egoist, but few are "conscious" of it and make use of it. So I'd say "conscious laziness".
9:29:11
pjb
Sometimes the message is understood later. So the success count may be higher than you think (eventually).
9:37:23
madnificent
Shinmera: I'll check out Luckless. I'm looking for/at benchmarks and info now but may not find the benchmarks you mention. If you have a direct link, that'd be rad. I'm not sure what I'll learn exactly but I hope it will give me a rough idea on what to test for.
9:58:44
madnificent
The readme states that the implementation is based off NonBlockingHashMap and its Common Lisp port in Luckless. But perhaps that's more "we read it".
10:01:10
mfiano
"Improved" I suppose would depend on usage patterns, machine, and operating environment.
10:02:33
mfiano
hayley says it may have a bug and not recommended for use yet, but I still don't see that mentioned in the readme
10:04:29
madnificent
I should care about safety first, so by the time we're at the element of speed, this may be better known.
11:49:51
madnificent
mfiano: I should have asked this way earlier. mu-cl-resources has a lot of multithreading going on in suboptimal ways too. These hash-tables might make a difference for the caches it uses.
12:30:58
hayley
madnificent: Another difference is that resizing in Threadmill is not lock free, whereas it is in Luckless.
12:32:24
hayley
I basically made up the resize logic on the spot, so I'm not so sure it's correct...but it should be, based on what I watched about NonBlockingHashMap.
12:33:11
hayley
I think there might be a bug because decentralise2 sometimes drops like 1 in 10 million messages when using Threadmill, but not any other table. I'll test again tomorrow.
12:40:30
hayley
Really, I do need to get my shit together w.r.t Telekons projects. There is also the concurrent-hash-tables portability library (which is portable in that it handles Luckless, Threadmill, and a table with sharded locks), but I haven't thought of a way to specify which table implementation to use at system load time.
13:01:41
madnificent
hayley: for mu-cl-resources the drops wouldn't be the worst. this is about caching so if a cache isn't stored it'll be calculated again at another time. not clearing would be more challenging.
13:02:42
hayley
And FWIW I don't exactly know that "it drops values randomly"; I only know that decentralise2 drops messages, which might or might not have to do with the table dropping values.
13:03:11
hayley
It might not even have to do with the table at all - just going faster with concurrent programs might lead to more opportunities to expose a bug.
13:04:05
hayley
Oh, well, such is life. And I intend to simplify the code around the table, so we'll get to find out where the bug lays.
13:05:46
madnificent
also, i failed to find the time to further check out your work, but we're running a company around distributed web tech. mostly knowledge sharing with linked data, which is what ActivityPub is built on. we also co-run ipfs-search. if these 90's buzzwords or hipster-tech ring a bell and relate to your work do ping.
13:07:33
hayley
I guess so - Netfarm is what you'd get if you gave IPFS schemas and some methods, and I've written a few (not very nice) things about the Fediverse.