libera/#commonlisp - IRC Chatlog

10:03:29 dnhester26 So I guess I'm newish to CL since I've only been doing it for about 2 years, but I just discovered this: https://medium.com/@MartinCracauer/static-type-checking-in-the-programmable-programming-language-lisp-79bb79eb068a ? Is compile time type checking widespread? I've seldom seen it before. How come? I don't really have much of a use for it, but I'm curious as to why aren't there standard facilities (maybe there are?) to use compile

10:03:30 dnhester26 time type declarations like he shows? Isn't that akin to what coalton is trying to do?

10:05:52 phoe it's different than what coalton does

10:06:15 phoe basically, you are able to do local type declarations, and the compiler is supposed to trust you there

10:10:26 scymtym implementations can (and do) use type declarations in different ways, sometimes depending on the requested optimization policy. for example, SBCL, when compiling and evaluating safe code, treats type declarations as assertions that are checked at compile-time and/or runtime

10:33:03 dnhester26 interesting, thank you for your answers. Is this a widespread practice? How come it's not very common? It would be nice when calling a function to have a compile time error instead of finding out at runtime. Furthermore, I imagine that even the IDE could say what's the type like emacs tells me the parameters of the function, it could probably also tell the declared types. Why isn't this more common?

10:33:36 dnhester26 The author worked at ITA/Orbitz/Google it seems like for a long time, so I was wondering if they used that heavily over there

10:34:10 phoe it's widespread enough because often enough achieving performance requires typing information

10:34:28 phoe you are going to have different code paths for accessing simple and non-simple arrays, for example

10:35:18 phoe ELT is going to be compiled differently depeding on whether the argument is promised to be a list, a vector, a sequence, or any kind of object (there's chance for a type error!)

10:35:28 dnhester26 I had this thought that there have been a ton of CL companies throughout the years, but maybe it was before open source was popular, because I don't really see in our ecosystem so many libraries/systems shared by big companies that are widespread in use. I don't recall seeing a ITA/Google system, or any from the game maker andy gavin, or just in general. Even that portuguese company, I don't recall seeing many libraries they've shar

10:35:28 dnhester26 ed for the benefit of the community. Maybe it's because it's a prvious generation?

10:37:33 dnhester26 yeah, I was thinking for me it doesn't make that much of a difference because performance has not been an issue, but people making games or anything that needs to optimize on performance would do. But my point was not that people don't use the declare, it was more that alexandria doesn't have a macro for doing type declarations (that I know of at least) like the one presented in the post, so it makes me think it's not such a common

10:37:33 dnhester26 practice

10:37:58 phoe serapeum has its -> macro that wraps DECLAIM FTYPE

10:42:18 dnhester26 ah I didn't know, thanks for pointing that out

11:12:37 phoe but note that you can do more than just that - you have DECLARE TYPE, DECLARE FTYPE, and THE if you need to declare types of local variables, local functions, and values

11:47:33 beach dnhester26: You can't use type declarations in Common Lisp the way they are used in a statically typed programming language.

11:48:25 beach dnhester26: And you probably wouldn't want to. There is evidence that supplying type information is a major consumer of time, and that this information often needs to be altered later during development.

11:51:11 beach dnhester26: In a statically typed programming language, the program is rejected if the (mandatory) type information is inconsistent, and this requirement imposes a lot of restrictions on the type system; restrictions that don't exist in Common Lisp.

11:55:56 beach dnhester26: Take GETHASH for instance. In general, it is undecidable to check that the object returned is of a particular type.

12:31:02 dnhester26 beach: thanks for the comments. What do you mean by "undecidable"

12:31:19 adeht ** NICK _death

12:32:46 beach dnhester26: It is a term in theoretical computer science. It can't be determined statically.

12:33:38 dnhester26 Ah interesting. I guess other languages deal with that by making the setter only accept a certain input?

12:34:22 beach They deal with it by forcing the user to make the entire program available before it can be executed, and by requiring type information of every function and every variable.

12:35:48 beach dnhester26: Static typing essentially makes interactive development impossible.

12:37:00 dnhester26 Interesting. But that's only when the return values are involved or also on the parameter definition?

12:37:55 beach "that"?

12:38:25 dnhester26 I guess the parameter definition will need to adapt to having values returned from functions, be unkonwn anyway

12:38:55 beach What are you referring to now? In statically typed languages? In Common Lisp? For interactive development?

12:39:41 dnhester26 Ah that you said the gethash return value is undecidable, so "that" is the return value, in contrast to the parameter types of a function

12:39:48 dnhester26 In CL

12:40:14 beach It is undecidable because you can't specify the types of the values of a hash table.

12:40:49 dnhester26 Ok

12:41:05 beach So if you say (defun f (x y z) (+ x (gethash y z))) there is no way you can be sure of the type returned by GETHASH.

12:41:34 dnhester26 yeah, I understood that the first time

12:41:35 beach You can declare Z to be a hash table, but you can't be sure what was put in it.

12:42:23 dnhester26 It's just that in the example from the blog post it was declaring the types of the parameters of the function; something like `(defunt foo ((string a) (string b) ... )`

12:42:49 dnhester26 I remember reading comments on how statically typed languages are easier to refactor because if a given function is changed, or type, the compiler picks up all the issues everywhere. So I thought using this existing facility could provide that functionality, that's where I was coming from

12:42:51 beach DEFUNT? Is that the person's own macro?

12:43:35 dnhester26 yeah, from the blog post, let me find the link again.

12:43:44 beach No, it's OK.

12:43:46 beach I believe you.

12:44:15 dnhester26 It's only the first part of the post, not the debugging part: https://medium.com/@MartinCracauer/static-type-checking-in-the-programmable-programming-language-lisp-79bb79eb068a

12:44:21 dnhester26 Ah, ok

12:44:34 beach So another "no-no" when you work in statically typed languages is to use implementation types in declarations. Those very often need to change later.

12:45:06 dnhester26 He posts the macro, which is also not a very long one. To do something like `(declaim (ftype (function (string) t) meh4a))` where `meh4a` is the name of a function

12:45:10 beach You should use application-defined types instead,

12:45:47 dnhester26 I don't know what that means. Use the classes defined in my application?

12:46:00 beach Classes or other types (using DEFTYPE).

12:46:46 beach I absolutely think focusing on type declarations in Common Lisp is a waste of time. They are occasionally useful, but not very often, and they take a lot of time to supply correctly.

12:48:14 dnhester26 Ok. To be honest I haven't done any type declarations, I just haven't needed to do it. My questions stem from understanding if it's possible to develop useful tools for refactoring code, or if maybe by using certain features already available in CL we could at least make it easier to refactor code written using that functionality

12:48:46 beach CHECK-TYPE is often much better as an alternative. You can use it to prevent incorrect values from propagating deep into your code.

12:51:01 dnhester26 Ok I am actually using check-type already, didn't think about it as typing as much as validating inputs for controller routes in my web application

12:51:30 beach Exactly.

12:52:02 dnhester26 So in conclusion: there are no particularly useful refactoring tools we can build that don't already exist and are not CL implementation dependent. Correct?

12:53:29 beach Oh, there are tons of things that can be built. Some of them may exist in commercial Common Lisp implementations (I don't know), but we don't have many in free Common Lisp implementations.

12:54:09 beach Like I often say, we don't even have an editor that can determine the role of each symbol in an editor buffer.

12:54:26 beach If we did, we could provide a tool for things like renaming a lexical variable.

12:55:29 dnhester26 that are not implementation dependent? Are those listed in your page of suggested projects? can you provide a list of useful things to build that are not implementation dependent (and maybe some that are)?

12:56:03 dnhester26 "we don't even have an editor that can determine the role of each symbol in an editor buffer.": The climacs project is attempting to address this IIRC?

12:56:12 beach Sure, determining the role of each symbol in a file is a simple matter of applying the first phase of a compiler to it.

12:56:47 beach dnhester26: Yes, we want Second Climacs to be able to do that.

12:57:38 beach ... and the first phase of a compiler can be largely independent of the implementation, since the semantics of Common Lisp are documented.

12:58:45 beach dnhester26: I really haven't thought of a complete list of implementation-independent refactoring tools.

13:00:12 beach dnhester26: But I can say that, whenever you need to apply some transformation that depends on the meaning of symbols in your code, you need a code walker (which can be thought of as the same as the first pass of a compiler).

13:01:08 beach dnhester26: Take this example from McCLIM: (DEFMETHOD GRAFT ((GRAFT GRAFT)) GRAFT)

13:01:27 beach There are three different meanings for GRAFT in that code.

13:01:30 dnhester26 you mean look up the source code?

13:01:36 beach No, no need.

13:02:02 beach I am just saying that you need a compiler to figure out what the meaning of each GRAFT is.

13:02:06 dnhester26 ah, realize that thee's a class, a function, and a parameter

13:02:17 beach Yes.

13:02:35 dnhester26 Yeah

13:06:41 dnhester26 Right. I feel like there's such a big gap between me a relatively new CL user and you who is writing compilers/SICL. To understand writing useful CL development applications, it feels like a requirement is to understand the compilation process and a lot of other things I don't usually run into when making say a web application in CL

13:06:43 beach Just a simple thing (one of my favorite examples) like when you put the cursor on a lexical variable, or when you hover your pointer over it and you get all the usages of it highlighted, requires a compiler.

13:09:15 dnhester26 ah right, I use `M-. C-s` all the time in my files which somewhat dumbly mimics that functionality without the meaning

13:09:19 beach Right, the application programmer often does not need to understand all that. I am explaining it so that you see the reason why we have few free tools like this.

13:09:51 beach Exactly. We don't have many alternatives now.

13:09:55 thuna` I think I remember seeing something like that in Lem's demo in ELS (or was it SBCL25?)

13:10:34 beach thuna`: That would be fantastic. But as I recall, they use SLIME so they wouldn't be able to do that correctly then.

13:11:41 dnhester26 yeah, lem IIRC doesn't have any CL development improvements over slime, it's just an emacs in CL

13:14:03 phoe lem has an option to use swank, afaik

13:14:06 phoe or it can evaluate in-image

13:14:24 beach Like suppose you have (LET ((X ...)) (MY-MACRO X ...)). There is no way to know whether the last X refers to the lexical variable without expanding MY-MACRO.

13:14:45 beach And only a compiler can do that.

13:22:26 dnhester26 Ok, I summarized what you said for myself and in case you make an entry in your suggested projects it may be helpful: https://plaster.tymoon.eu/view/4728#4728

13:56:10 phoe beach: "first phase of a compiler", you mean minimal compilation + some basic semantic analysis?

13:57:07 phoe and a way to associate the X in (MY-MACRO X) with some meaning that it has in the resulting code after minimal compilation (e.g. does it refer to a variable, a function, a class, is it quoted somewhere in the expansion, etc.)?

14:02:09 jackdaniel in short -- expanding all macros

14:02:35 jackdaniel it is somewhere in the spec, 3.x

14:26:59 beach phoe: Yes, something like that.

14:29:47 scymtym the step from character syntax to s-expression syntax is needed as well since editor buffers and source files are (or can be presented as) character sequences

14:34:44 beach phoe: And things are nontrivial because of MACROLET. The thing needs to be able to generate code for the macro function.

15:00:44 younder s-expessionist:Eclector is SICL's read package.

15:02:06 beach Elector is now completely independent of SICL.

15:37:23 svm ** NICK msv

17:24:53 ldb I've seen people been working on a Scheme Language Server Protocol server

17:26:20 ldb They are stuck on the problem of tracking variables after macro expansion

17:30:05 beach Is the Language Server Protocol complete enough for Lisp, and in particular for Common Lisp?

17:31:32 ldb I'm afraid not, it is only because many people who newly started to learning programming are more getting used to VSCode or Neovim, which only provides LSP as the interface to be extended with programming language specific supports

17:32:13 beach I see.

17:38:58 beach Given the amount of work that scymtym had to put in just to make sure the editor buffer is parsed correctly by the reader, doing something similar when you need to communicate using a wire protocol must be orders of magnitude harder. At least if you want reasonable performance.

17:46:31 ldb well the language sever typically maintains their own file buffer, and call the front end of their compiler after each update

17:47:35 beach I see, so the editor just communicates buffer updates to the server?

17:48:39 ldb and request for cross reference/completion at point information when user interacts

17:48:51 beach Ah, yes.

17:49:58 ldb lsp does transmitter what user typed, since it is not always the case file is saved to disk

17:50:46 beach Makes sense.