freenode/#sicl - IRC Chatlog
Search
4:52:21
no-defun-allowed
Looking at the bindings of the most recent CST-EVAL call in the backtrace, I spotted that the problematic function is the macro function for CALL-METHOD in WRAP-IN-CALL-METHOD-MACROLET, and the undefined lexical variable shares a name with the only lexically bound variable (that ARGUMENTS-nnn name that WRAP-METHOD-COMBINATION-FORM generates).
4:54:32
beach
I am still surprised though. If the HIR evaluator manages to execute the code, I can't see how there can be a problem in HIR.
4:56:04
no-defun-allowed
Yes, it took me a while to figure out what was going on (which isn't your fault). It's as if the usage of that closed-over variable isn't converted properly, but that would indeed be reflected in HIR surely.
4:57:44
beach
In other words, if everything works when you turn off HIR-to-MIR, then it is hard to imagine that the problem would be in HIR.
5:00:37
beach
Aha. Then there is probably something wrong in HIR-to-MIR, because nothing is using MIR other than MIR-to-LIR.
5:03:45
no-defun-allowed
Now I remember that the information in the environment object for that variable referred to a lexical location, and I think it was the same location.
5:06:53
beach
It is hard to believe that a single source function would be the origin of a problem like this, because that would mean that this function is the only one using a particular HIR instruction class, and that the HIR-to-MIR processing of that instruction class would be buggy.
5:34:00
beach
If there is an uninitialized lexical location in HIR, that must mean that the HIR evaluator never executes the instruction that uses it.
5:36:12
beach
Some AST classes and instruction classes are uniquely used by some methods for arithmetic.
5:36:42
no-defun-allowed
It is an assignment instruction, which was probably produced in order to call WRAP-IN-CALL-METHOD-MACROLET.
5:38:19
beach
It is hard for me to imagine that it would have something to do with that function specifically, because it seems to me that this function is used everywhere, so then no generic function would work properly.
5:42:30
beach
It is much more likely that it has to do with the translation of some specific AST class to HIR. An AST class that is used only in very specific situations, like floating point stuff.
5:42:47
beach
Because floating point stuff is never executed during bootstrapping. It is only translated.
5:43:13
beach
So if you see a problem in translation but not in execution of HIR then that's what I suspect.
5:46:04
no-defun-allowed
Nothing sticks out as particularly rarely executed. Maybe MACROLET, but that is about it.
5:47:04
beach
None of the methods having to do with floating point (for instance) on the arithmetic functions are ever executed.
5:47:25
no-defun-allowed
Here is the test function, and when it is run: <https://github.com/no-defun-allowed/SICL/blob/master/Code/Compiler/AST-compiler/ast-compiler.lisp>
5:50:26
no-defun-allowed
I can't think of any features that would be used as part of satiation in general which wouldn't also be used elsewhere.
5:53:01
beach
So do you see my reasoning here? There is an uninitialized lexical location in HIR. That means that the code is never executed by the HIR evaluator; only translated to MIR and then LIR. So then it can't be a problem with any general mechanism like calling methods. It has to be something that is loaded into the system, but never executed.
5:53:36
beach
What you could do, for instance, is you could remove some loads of arithmetic functions, even all of them.
5:56:01
beach
It could be that the information about the lexical locations is not always up to date.
6:07:24
no-defun-allowed
That variable is still undefined before any transformations, so I am still suspecting however AST-to-HIR handles MACROLET has broken something.
6:16:41
no-defun-allowed
The raw form from the CST makes me wonder if the ARGUMENTS-nnn variable should have been quoted, and bootstrapping succeeds (without any conversion past HIR) if I do that.
6:16:42
beach
That would mean that MACROLET is never used in code that is executed by the HIR evaluator.
6:17:48
no-defun-allowed
Here is that form, with a comment indicating where I put the quote: <https://plaster.tymoon.eu/view/2437>
6:18:58
no-defun-allowed
But it is totally a guess, based on how the ARGUMENTS- symbol is quoted in the case where (NOT (CONSP METHOD)).
6:20:44
beach
So if you quoted it and bootstrapping succeeds, then that means that the result of that code was never executed during bootstrapping. That's ... fascinating.
6:23:38
no-defun-allowed
I forgot to check after conversion, but MIR-to-LIR won't work if there are uninitialized lexical locations.
6:25:40
no-defun-allowed
Well, it now signals an error because I haven't written register allocation methods for SAVE-VALUES-INSTRUCTION.
6:27:11
no-defun-allowed
Still, it is odd that the compiler introduces a bogus lexical location, instead of reporting that the variable is unbound.
7:16:41
no-defun-allowed
I am about to disappear for dinner, so I probably won't do much more work tonight.
8:08:39
beach
Oh, I saw that question in the logs. I am not working on it at the moment. I am counting on the steady progress by scymtym.
8:43:20
beach
... so that no Common Lisp programmer can resist it, with the possible exception of lukego. :)
8:47:59
beach
zacts: But I am definitely aiming for a software-only "Lisp machine", so the Unix part is not that interesting to me.
8:48:35
beach
lukego: I read your remarks in #lisp, so I understand your point of view. I am not going to argue with you.
8:49:53
beach
So, anyway, now that I know how to program ASDF, I can throw out my incomplete kludge for traversing an ASDF system definition for loading things into first-class global environments during SICL bootstrapping.
8:50:53
beach
I just need to add a list of already-loaded files to each environment, so that a file is loaded once only.
8:51:40
beach
Once I have done that, I may be able to use ASDF to load more parts of SICL itself, rather than using explicit calls to LOAD in the bootstrapping procedure.
8:51:47
lukego
yeah sorry not meaning to pick a fight anyway. it's nice with Lisp (and SICL) that it's a modular setup permitting different approaches. I like to have my Lisp'ing partitioned between separate images/processes because that gives me peace of mind. So e.g. with CLIME it's a feature rather than a bug that the UI and the application logic are in separate processes and can't accidentally mess up each others' data structures
8:53:48
beach
The plan for SICL is to use first-class global environments to isolate applications so as to avoid such accidents. But I have no master plan for how to divide things up yet.
8:55:48
lukego
I'll have to take a look at the first class global environments thing one day. I think that ultimately I'm just interested in having a few true invariants i.e. things that I /know/ are always true. Maybe this is a mechanism for that e.g. I know that component A and B can't interfere with each other because they have completely disjoint heaps (as opposed to just because nobody found a bug in their common parts lately)
8:57:17
beach
As I defined first-class global environments, they don't address the problem of separate heaps. But as MichaelRaskin pointed out, I may need to introduce allocation quotas, either for an environment or for a "user".
8:57:37
lukego
So thread-safety and memory-safety in Lisp aren't really invariants because they can pretty easily be broken by bugs at the library/application level e.g. using a low-level memory access routine like an SBCL system-area-pointer. And that's fine: sometimes you want to break the rules. But I want limits e.g. I don't want one application to be able to corrupt another or crash the machine.
8:58:23
beach
I am not planning to support any such low-level operations in SICL, but I am sure that someone will add it anyway.
8:59:27
lukego
Yeah. Trouble is that when they add that then language-enforced memory safety goes out the window. This is why I like hardware (MMU) enforced memory safety much better at least for providing an outer perimiter.
9:00:05
beach
I mean, SICL itself must obviously use low-level access operations, but I can make sure only the high-level interface is directly accessible, by isolating the low-level interface in its own first-class global environment.
9:00:10
lukego
well that's hard if you're in a unified system writing e.g. device drivers and stuff right?
9:01:11
beach
Well, you can't prevent people from wanting to alter or improve the system, but you can prevent accidents by this environment isolation.
9:01:37
lukego
beach: I also like that the MMU takes away the taboos. You wanna use low-level pointer access and inline assembly code and so on? Great! Go ahead! No problem! Just pick appropriate boundaries so that your crashes can be cleaned up properly, as with unix process termination.
9:02:35
lukego
Sure. But then you don't have invariants. In practice stuff will crash and the answer will be "tut tut you shouldn't have written code in that way." This is what we see with C over and over again.
9:03:29
beach
But C, or at least most implementations of it, is not a safe language the way I think of Common Lisp.
9:03:37
lukego
anyway, I sense that this discussion is on the edge of hostility, so I'm going to leave it at that.
9:05:53
frodef
Hm. I'm late to the party, but fwiw I'm almost exactly opposite of lukego.. We've tried MMU and serialization for (too) long, the point of a lispm must be to try the other approach.
9:07:54
beach
But we no longer need any particular instructions. Lisp runs fine on stock hardware these days.
9:08:39
no-defun-allowed
I think there are hardware implementations of term reduction machines, but those are of course for lazy functional languages and not so much Lisp.
9:09:51
frodef
BTW the MMU can be used to enforce protection without sacrificing object identity, e.g. keep a single address space but with different access zones. 64-bit address spaces now should make this very viable.
9:20:29
no-defun-allowed
(Okay, Scheme-79 is far from running S-expressions directly, but it does some linked structure out of CONSes.)
9:29:30
no-defun-allowed
Here is a fun test (if one has EXPRESSION-TO-HIR laying around): (expression-to-hir '(let ((x 2)) (macrolet ((f () x)) (f))) *e5*)
9:30:05
no-defun-allowed
An error is signalled, which reports "Undefined variable named SICL-HIR-EVALUATOR::.UNBOUND."
9:30:38
no-defun-allowed
Replace the second X with Y and instead "The variable Y is unbound." is reported as expected.
9:32:30
no-defun-allowed
I'm going to go on a limb and guess that somehow the macro function is compiled in the environment for the body of the LET form, or something like that.
9:36:33
beach
That run-time part of the lexical environment should be removed, but maybe we don't invoke the function to do that. Or maybe that function does not (yet) exist for Trucler. I know Bike wrote a function to do that.
9:36:59
no-defun-allowed
The HIR for the macro function has an undefined location named X, just like the HIR produced as part of satiation.
9:37:55
no-defun-allowed
Random guess, the function might be TRUCLER:RESTRICT-FOR-MACROLET-EXPANDER?
9:38:58
beach
So that experiment suggests that we have a let binding that binds some variable at run-time, and it is referred to in a macro function.
9:50:52
beach
Do you want to continue investigating? I need a long-ish lunch break, so I am off for a while.
9:51:46
no-defun-allowed
Maybe I am not understanding it properly, but it appears that VARIABLE-DESCRIPTION will still return lexical variable descriptions from a restricted environment.
9:59:39
no-defun-allowed
Sure. (Though now I don't think VARIABLE-DESCRIPTION is part of the Trucler protocol, and so I don't know how the pre-HIR stages handle the environment.)
13:23:27
beach
heisig: Did you see that I managed to program ASDF and it was quite simple. Well, I haven't finished the programming quite yet, but I will soon integrate it into the bootstrapping procedure.
14:51:01
beach
I must say I am very pleases with these BleuJour OCTO computers, especially since they don't crash, but also that they are so small they can be mounted on the back of a screen. And I much prefer Mint over Ubuntu. Cinnamon on Mint is much closer to the way I want my desktop than Cinnamon on Ubuntu, and certainly than the Ubuntu default desktop.