freenode/#clasp - IRC Chatlog
Search
5:31:00
beach
If some symbol appears several times in the expansion, it could have come from the original form, or it could have been introduced by the macro expander.
5:32:18
beach
We could do better if we had a way to track source code of the macro expander itself, but that's currently not done.
5:33:54
beach
For what it's worth, in SICL, I intend to load the entire contents of the file being compiled as a string into the image, and have source locations refer to that string.
5:34:25
beach
... as opposed to hoping that the file is still around, has the same, and has not been modified.
5:35:58
drmeister
I like to load whole files and work with them in memory - but I wouldn't think to keep them around.
5:36:59
beach
Source code will likely be paged out, so it's not necessarily RAM. People tend to ignore the existence of virtual memory.
5:39:35
beach
The entire source code of SICL is 6.4MB, which is ridiculously little in terms of RAM size today.
5:40:17
beach
But I am getting used to people no longer being able to do back-of-envelope calculations.
5:41:57
beach
It is sad, though, that people aren't able to do that, because a lot of wrong decisions are made that way, and it wastes huge amounts of human energy.
5:52:37
ecraven
my entire .quicklisp directory has 180k lines of code, so that would be less than 15MB
5:52:48
ecraven
imho there's absolutely no reason *not* to keep full source code around all the time
6:01:18
beach
But I see the same knee-jerk reaction over and over again. Maybe I am unusual. I have this thread in my brain that runs all the time and it checks orders of magnitudes of whatever I hear, read, or think.
6:04:39
ecraven
even assuming you only have 512MB of RAM (and even the 5€ raspberry pi zero w has that), that's less than 10% if you compress it a bit
6:06:24
beach
Whenever I make such an estimate in a lecture, just to make sure my students realize the order of magnitude, there are students who get furious and still claim their intuition is right.
6:07:08
ecraven
not to belittle anyone, but if they are students, they are probably too young to have experienced a time when memory *was* scarce
6:10:08
ecraven
so to summarize my thoughts on this: there's no @#$*&@#$*^ reason to not store all source code in-memory to make debugging simpler
6:10:38
beach
Absolutely. If it saves a minute of debugging time, it has been paid for many times over.
11:46:34
Colleen
Unknown command. Possible matches: 8, roll, say, login, time, help, deny, you are, notify, logout,
11:48:34
Bike
drmeister: yes, i noticed that. on sbcl it returned 10 when i tested it. i didn't look into it.
11:48:34
Colleen
Bike: drmeister said 6 hours, 27 minutes ago: I fixed the validator thing. But say there is a validator that limits the range of values of a slot between 0 and 10. If I (setf (slot object) 15), the value is set to 10 but 15 is returned. I expected 10. The CLHS says of setf: "results---the multiple values[2] returned by the storing form for the last place, or nil if there are no pairs."
13:18:08
Bike
kind of not sure what to do with THE. we can't say all THEs become type checks because some are uncheckable (to ignore the problem of what a THE instruction means in this brave new world), and even if we do make almost everything checks there are still multiple values
13:26:38
beach
It ought to be the case that all type declarations of variables turn into single-value THEs, right?
13:30:36
Bike
hm, i wonder if (symbol-macrolet ((x (values 1 2))) (declare (type (values integer integer) x)) ...) is legal
13:31:35
Shinmera
Bike: I gotta be honest, I'd be very surprised if that worked in any of the implementations
13:34:29
beach
Does the presence of user THE forms make it impossible to check whether we have (THE <type> <symbol>)?
13:35:14
Bike
what makes that a little difficult is that a symbol can be a symbol macro that expands into a non-symbol.
13:37:11
Bike
also, what i got out of yesterday was we were going to put the THE translator under implementation control
13:45:26
Bike
oh, assuming the type is fully expanded. i think you can have a deftype that expands into a values type
13:47:44
beach
I would say, try to recognize as many simple cases as possible. For example (THE <non-value-type> <form-known-to-return-exactly-one-value>).
13:50:15
Bike
it's annoying going from the mostly general way it is now to special casing, but i guess that's how it goes
13:55:02
beach
Anyway, from doing a grep in my .quicklisp directory, it looks like the vast majority of the uses of THE has a <form> that is known to return one value, like (+ ...) (1+ ...) (svref ...) etc.
13:56:21
Bike
how do i know that a function returns a single value? (so far, my thought has been: type declarations)
13:57:30
beach
And, you can trust type declarations for global functions only if they are in the CL package.
13:58:55
beach
What is wrong with checking the package of the function name and the existing type declaration of the function?
14:00:14
Bike
well if i do (if (string= (package-name (symbol-package operator)) "CL") ...) then that package-name is going to be problematic with first-class global environments, right?
14:01:46
Bike
The compiler gets an environment as an argument. It is supposed to look things up in that environment, right?
14:02:33
Bike
but cl:package-name will use... either the environment the compiler is being compiled in, or the runtime environment? not sure. neither of which are necessarily correct
14:04:39
beach
I think you can safely assume that the startup environment is the same as the one the code will ultimately be loaded into.
14:07:41
Colleen
Clhs: section 3.2.1 http://www.lispworks.com/documentation/HyperSpec/Body/03_ba.htm
14:09:32
beach
If you don't respect that condition, you may be in for problems in some cases. Not that I think that this particular case is one of them.
14:11:11
Bike
if we make that assumption we don't even need the environment argument for global information.
14:13:28
Bike
i don't see a moral difference between package-name having an environment parameter and function-information having one.
14:14:06
beach
"you can safely assume it" means "if you assume it, you are safe". Nothing more. In most cases, if you don't assume it, you are also safe. It doesn't follow that "in order to be safe, it has to be assumed", and it doesn't follow that "we always want to be safe rather than flexible"
14:15:26
beach
By the way, I haven't decided whether package names are intrinsic to the package or whether they are part of the environment.
14:15:30
Bike
not least for this. honestly, having type declarations only work for variables and results of standard function calls would be silly.
14:17:35
Bike
i assumed you just did the package name thing like that because the standard kind of works as if the name->package association is not part of the environment, but you need it to be
14:20:20
beach
Perhaps Clasp should use a compiler framework other than Cleavir, too. Then there will be even fewer problems. You can then write a compiler that doesn't use CLOS and you don't have the metastability problems you now do.
14:21:43
Bike
i want to know how this can be done well in cleavir, in an implementation independent way.
14:24:58
beach
I am currently trying to find the page in the Common Lisp HyperSpec that lists the restrictions on the environments of compilation and loading of fasl files.
14:25:17
Colleen
Parse error:URI "http://l1sp.org/cl/compilation semantics?" contains illegal character #\ at position 30.
14:25:31
Colleen
Clhs: section 3.2.2.3 http://www.lispworks.com/documentation/HyperSpec/Body/03_bbc.htm
14:27:31
beach
So, now, since Common Lisp does not have first-class global environments, no restrictions like that are present in that list, right?
14:27:40
Bike
it's just that i thought that for cleavir, the "compilation environment" is the environment passed to functions, rather than the environment the compiler is itself running in. which is why macrolet uses cleavir-env:eval rather than just eval, and stuff.
14:27:54
beach
Now imagine adding such constrains in the presence of first-class global environments that makes things work sufficiently well.
14:29:00
Bike
yeah, so i would expect something like "the name of a package is the same in the compilation environment and at runtime"
14:29:50
beach
But I am unfortunately unable to make up the additional items for first-class global environments in real time.
14:32:02
beach
In normal circumstances, the run-time environment in which the compiler runs should be unimportant. Again, I have not made a complete list of marginal cases, and I am unable to do so in real time.
14:33:56
beach
The following scenario is something I expect. Compile and load the compiler into some environment E1 which then contains all kinds of crap including auxiliary functions, code utilities, AST converters, etc....
14:34:23
beach
Then define an environment E2, that contains the functions COMPILE and COMPILE-FILE imported from E1.
14:35:33
beach
The compiler can now be executed in E2 in order to compile new stuff, including a better compiler.
14:36:53
beach
Notice that in this scenario, the compiled code is loaded into the same environment it was compiled with.
14:38:59
beach
I am saying the following: Even if we have the restriction that the same environment must be used for compilation and loading the resulting compiled code, you can still have several different compilers in different environments.
14:40:27
Bike
okay... you're saying the compiler is /executed in/ E2. I thought that you would execute the compiler in E1, but pass it E2 as its environment argument.
14:43:04
Bike
how does it call functions from E1? since we loaded it in E1 the cells are all E1 cells?
14:45:27
Bike
So something like (map 'compiler-auxiliary-function ...) has to be compiled to not call fdefinition?
14:45:43
beach
And this scenario is essential for safety. Now, no code can do (eval-when ... <change-the-code-generator>) and get away with it. Because it will change the code generator of E2 which is not what the compiler uses.
14:46:25
Bike
i get that much of the concept, it's the compiler executing in E2 that's confusing me.
14:47:46
Bike
if you run the compiler in E1 and pass it E2, and it compiles a file with an eval-when, it will evaluate that code in E2 (with cleavir-env:eval), so it can't affect the compiler. i thought that was how it worked.
14:47:55
beach
I am not sure what your example means, but, yes, again, there are no doubt certain restrictions on what code that is compiled in one environment and executed in a different environment can do, or at least, the semantics of such cases much be determiend.
14:49:58
Bike
Okay, so say the compiler does (map nil 'something ...) somewhere. Naively, that's compiled as a call to MAP with arguments nil, symbol SOMETHING, ... At runtime it calls the E1 map. The e1 map probably does (fdefinition symbol)... well, i guess the fdefinition could itself have been compiler macroed into (genv:fdefinition symbol #<E1>)
14:51:23
beach
Maybe. Or one can explain that it then uses the definition in E2 and something else would have to be done to get the definition in E1.
14:53:59
beach
OK, here is one more scenario. The source code to be compiled contains #.(<modify-somthing-global>).
14:54:23
beach
It is essential that this code is read into E2, or else the compiler can be destroyed.
14:57:21
beach
Anyway, I am pretty sure the run-time environment in which the compiler executes must be E2.
15:00:35
Bike
i mean, if i understand you correctly, the environment argument must be the same as the environment the compiler is running in, so the compiler source could just have (get-global-environment) everywhere it uses the argument.
15:00:42
beach
Just because the compiler uses E2 as a run-time environment doesn't mean you have to pass E2 to the compiler.
15:02:25
beach
"you can safely assume it" means "if you assume it, you are safe". Nothing more. In most cases, if you don't assume it, you are also safe. It doesn't follow that "in order to be safe, it has to be assumed", and it doesn't follow that "we always want to be safe rather than flexible"
15:03:45
beach
The environment argument to the compiler does not have to be the same as the global environment in which the compiler executes.
15:04:21
Bike
that's what i thought, but i understood what you said just now about compiler having to execute in E2 as meaning otherwise.
15:04:53
beach
"executing in E2" just means that its run-time environment has E2 as its global part.
15:06:49
beach
Clearly, by default, as seen by any old application-programmer, if a compilation is started, then E2 will also be the one given to the compiler.
15:07:43
beach
But if you want to do something out of the ordinary, you can definitely pass any environment you like to the compiler. You are then assumed to have read up on the semantics of such a stunt.
15:08:56
beach
I take macros and stuff from one environment and then load the code into a different environment.
15:10:55
beach
Here is a much simpler scenario: The Common Lisp HyperSpec allows (encourages?) for the compilation environment to be different from the startup environment.
15:11:08
Bike
okay, so going with the actual thing where this came up, you're saying that it's okay for function definitions to not match between the environment the compiler is running in and the environment it's passed, but that package definitions do have to match? (with the understanding that you can't necessarily invent these semantics in real time)
15:12:34
beach
By having the compilation environment distinct from the startup environment, you avoid that side effects during compilation affect the startup environment.
15:12:48
Bike
my understanding was that this unusual bootstrapping situation was basically a core feature - so that, in order to make it easier, pretty much any mismatch would be allowed.
15:13:56
Bike
mm. right. (with relation to compilation and loading the clhs has an interesting restriction on that)
15:14:27
beach
Yes, there are already restrictions, and in many cases the behavior is undefined if they are not respected.
15:14:58
beach
I think the "system programmer" will want to have such situations defined so that they can be exploited.
15:15:49
Bike
well i mean we're talking about different restrictions here, i think. CL has an environment the compiler is running in, and an environment the compiled code is loaded into. we have an environment the compiler is running in, an environment the compiler is "compiling in" (including eval-ing eval-when things, etc), and an environment compiled code is loaded into.
15:16:21
beach
The restrictions are different, sure. I was referring to the spirit of such restrictions.
15:19:34
beach
For bootstrapping SICL, I am convinced (but not 100% sure of course) that I need multiple global environments simultaneously active. When I do things like (defclass standard-class (class) ....) I need to fetch one definition of standard-class from one environment and define that class (differently) in a different environment.
15:21:02
Bike
so that it does like (setf (find-class 'standard-class E2) (make-instance (find-class 'standard-class E1) ...))?
15:21:48
Bike
and then later you alter standard-class/E2 so that it's an instance of itself instead of of standard-class/E1?
15:27:02
Shinmera
This is only tangentially relevant, but I can't seem to find it: is it specified anywhere that changing the metaclass is not allowed?
15:27:49
Shinmera
If so, how would you handle cases where this is necessary? Perhaps not in SICL directly, but generally I come across this issue sometimes when I develop, and I imagine it would be one too for runtime-update systems like your LispOS
15:28:14
Colleen
Clhs: standard generic function change-class http://www.lispworks.com/documentation/HyperSpec/Body/f_chg_cl.htm
15:30:13
beach
Cross compilation is another scenario where an explicit compilation environment is required.
15:30:29
Bike
the semantic restrictions do include a bit that the metaclass of a class can't change between compile time and runtime, but i don't know what restricts metaclasses from being changed /at all/, if anything
15:30:34
beach
You can't rely on the host having first-class global environments, and even if it did, you would want the target definition of macros and such.
15:32:25
beach
In fact, in the past, I thought it might be advantageous to bootstrap Clasp by compiling its files by having Cleavir run in SBCL, using a first-class global environment that resembles that of Clasp's global environment.
15:35:34
Bike
it would be, no question. we'd have to have a C++ runtime (as now), and a cleavir-in-sbcl that can dump fasls appropriate for that runtime. i don't know if drmeister wants a dependency on sbcl (but we already have one, so)
15:35:43
beach
Shinmera: The metaclass of a class determines how slots are represented, so it would be nearly impossible to change the metaclass of a class, at least of the class has instances.
15:36:42
Bike
change-class on sbcl signals an error. mop says, in "initialization of class metaobjects" (in chapter 6) that "Portable programs must not call change-class to change the class of any class metaobject or to turn a non-class object into a class metaobject."
15:49:00
beach
Bike: Sorry I lost it. I guess I am frustrated that I am not able to make progress faster than I am. And it worries me that I am unable to explain my vision of things to do, which puts even more pressure for me to make faster progress.
15:49:56
beach
Plus, "paging" is very hard for me, so going from CST to first-class global environments to McCLIM is not easy.
16:02:45
Bike
yeah, sorry. when i ask you about stuff it's because i'm out of ideas (or am unsure that something fits your philosophy), i try not to be dependent
16:03:32
Bike
also, i remembered the real reason we don't boot from sbcl or something: the compiler uses llvm, which is C++, so we can't use it from lisp unless it's clasp
16:04:07
Bike
writing a different llvm-sys layer in terms of the c bindings so that we could use it from sbcl might still be better than maintaining an entire compiler only used for bootstrapping, tho
16:06:20
Shinmera
Probably because there's too much of a gamble involved. It's not something one can quickly try out after all.
16:10:09
beach
But if I were drmeister, with all those grants, I might consider some parallel development of alternatives, just to see whether it would be better. I mean, it's taxpayer's money and it is meant to be spent. But then, I am not drmeister, so the issue is moot.
16:11:04
Shinmera
I figure if he were in a computer science department that would be justified, but given that he's supposed to mostly be a chemist, I don't know how much leeway he has.
16:12:58
Bike
well, i understand it not being a big priority. it's really annoying, but users aren't going to build it much
16:13:13
drmeister
We have a working system right now. I want to start using it. The funding I have is to use it to design molecules. Not to rewrite it.
16:13:16
beach
drmeister: Yeah, I am not so convinced. But then, you already know that. This is #clasp, so I should try not to air my opinions too much.
16:14:51
Shinmera
ACTION is glad he doesn't have the burden of having created a new implementation when he can't even get a chat system to be stable
16:19:42
beach
Bike: I am hoping that the CST stuff is converging. So far, whenever I have worked on CST-to-AST, I have found more stuff to add to the CST library, either because it is in cleavir-code-utilities or because it should have been in the first place.
16:20:18
beach
Like, today, I found the need to canonicalize LET bindings in the form of CSTs, preserving source information, of course.
16:20:40
Bike
yeah, that makes sense. the more i went into code utilities the more environment dependent stuff i hit
16:20:55
beach
I'll continue like that. Grab something from Generate-AST and try to port it to CST-to-AST, see what is missing from CST, implement it, iterate.
16:23:42
beach
I am still willing to cough up 3k€ over 6 months to have someone help me with some things, but I don't want to make a general announcement for candidates. I would rather select someone I know to be able to help.
16:25:17
drmeister
beach: On that - I can also support good candidates to work on Cleavir related code. Anything that speeds it up, improves debugging etc. I also have more clasp related projects and problems.
16:28:31
drmeister
It's easier if they are in the US but I'd put work into figuring out how to support good programmers anywhere.
16:31:04
drmeister
Bike is going to be here through next summer (the job posting just went up) and he and I can coordinate and guide programmers elsewhere.
16:37:33
Shinmera
I have no idea whether I'd qualify, but if I manage to finish my bachelor's by summer next year I might be open for hire.
16:40:23
drmeister
Do you know how to program in Common Lisp? Are you willing to get your hands dirty with C++ and runtime issues? Are you looking for fame and fortune?
16:42:26
Shinmera
So far I'm reserving a year of time for "whatever" after my bachelor's, so it's not like I have any urgency in the matter.
16:42:53
drmeister
I bring up the runtime because while a large part of what Bike is here to do is to improve Clasp by improving Cleavir - the runtime keeps intruding.
16:45:41
beach
My (admittedly small) family says that dinner is ready. I might check in briefly later.
16:53:41
Colleen
Standard-instance-access http://metamodular.com/CLOS-MOP/standard-instance-access.html