freenode/#sicl - IRC Chatlog
Search
2:51:52
jcowan
so that (eq (read) 'foo) will produce the right result if `read` reads the string "foo"
2:52:01
pfdietz
Ok. I suggested before that what you want to get rid of is the ability to know if a symbol has been interned (and to iterate over the symbols of a package). With that, the unused symbols can be GCed.
2:53:20
pfdietz
Literal symbols in the code would stick around in their packages, but nothing else would have to.
4:04:59
no-defun-allowed
i did find an interesting message about a hardware error in dmesg after the crash, but apparently that's just a cpu fault not any other hardware
4:14:02
no-defun-allowed
that said, an early batch of r5 1600s apparently had issues, and would crash gcc sometimes
4:34:24
pfdietz
Sure, there's read. But that doesn't mean unreferenced symbols can't be GCed. You may get a 'different' symbol the next time it's read, but how could one tell?\
6:42:45
beach
Here is the current thinking with respect to code generation. Compiling a file or a form will generate an AST and the file compiler will simply store the AST in a READ-able format that is already defined by Cleavir. So the AST will be the FASL format.
6:42:56
beach
When the FASL file is loaded, the AST will be turned into a HIR graph that has a top-level ENTER instruction representing a function that will be enclosed and then called at load time. When that function executes, it will accomplish all the top-level actions that are required according to the source code.
6:43:05
beach
LOAD-TIME-VALUE forms will have been decomposed in a way that remains to be determined exactly, and that decomposition is done by using the result of heisig's fairly recent work.
6:43:14
beach
Compiling the HIR graph will create a "code object" as show in Figure 14.1 on page 47 (PDF page 57) of this document: http://metamodular.com/sicl.pdf and in particular, the code object contains a "code vector" which is a vector containing the native code for the processor.
6:43:25
beach
The question is what the top-level enclose will do and what a nested ENCLOSE-INSTRUCTION does (in particular to determine the entry point). Here is what I am thinking now: The code vector will have been allocated in a place that does not move, so the addresses of the instructions are known.
6:43:32
beach
The top-level enclose puts the first address of the code vector in the top-level function to be executed and then calls it. A nested ENCLOSE-INSTRUCTION contains an offset into the code vector that is determined at compile time.
6:43:34
beach
The ENCLOSE-INSTRUCTION accesses the static environment, then the code object, then the code vector and determines the address of the first instruction of the code vector. It adds that address to its compiled-in offset and uses it to create the nested function.
6:44:18
beach
Now, today is Monday and Monday mornings are crazy around here. In 15 minutes or so, I will vanish for around an hour.
6:46:00
beach
By using the AST as the FASL format, I eliminate an entire, possibly very large, body of code that would need to be specified and implemented, namely the code for a specific FASL format.
6:46:18
beach
If performance should turn out to be a problem, such a module can be implemented later.
6:47:04
beach
But I think this current thinking is what is going to be the fastest way of getting a native system up and running.
6:47:46
beach
It means that not only the compiler but also Eclector must be included in the initial system, but I am now convinced that both those modules are fairly small.
6:48:15
beach
It suffices to look at the size of the FASLs generated by SBCL for those modules to get an idea.
8:42:18
jackdaniel
beach: does it mean, that your FASLs will be "portable" across different cpu architectures? or did I misread something?
9:03:12
heisig
Regarding portability - maybe it would make sense to have a list of SICL variables that are possibly platform dependent and store them in the FASL, just to make sure.
9:03:59
heisig
Then one could at least detect whether the conditions at compile-time are still met at load-time.
9:23:09
beach
Cleavir does have a bug right now I think, including for Clasp. It looks to me like CST-to-AST does not process the first argument of a LOAD-TIME-VALUE form. It should be converted to an AST and then to HIR. But I think fixing it may break a certain number of things, so we need to plan for it.
9:25:07
beach
It is rare that the consequences of such a bug are detected, and, as I recall, CLISP has always left the form in there without processing.
9:31:36
heisig
The bug would only arise if the null lexical environment at compile-time would be noticeably different from the null lexical environment at load-time, right?
9:36:06
beach
There are many restrictions on such differences, which is why it does not happen often.
10:50:02
jcowan
Is it clear that rehydrating the AST is quicker than regenerating it from source? Early Ada compilers stored the AST in precisely this way, but gnat does not, because the cost of reading it was found to be higher than the cost of reading *and* reparsing the source.
11:01:50
heisig
The important thing about the SICL AST is that it fulfills the requirements of 'minimal compilation'. Otherwise, one would also have to store a copy of the environment at the time of compilation.
11:07:05
beach
jcowan: The compilers of most languages process source code in a completely empty environment. For example, a C program must include .h files in order to add some definitions to the initial environment.
11:07:05
beach
Not so with Common Lisp. A file is compiled in the "startup environment" which is the state of the global environment of the Common Lisp image when the compiler was invoked. That fact radically changes how things must be done in Common Lisp compared to in other languages.
11:07:47
beach
I intend to formalize this idea at some point, because it is the root cause of why it is harder to bootstrap Common Lisp in Common Lisp than it is to bootstrap C in C for instance.
11:27:50
jackdaniel
jcowan: environment is passed explicitly as an argument (special calling convention is required)
11:29:12
jackdaniel
also for global environment (i.e not lexical) you may have a variable env which is, heh, global
11:34:47
beach
jcowan: Sorry, slow thinker alert here. What requirement are you talking about, and why does that requirement seem to make CL->C systems more difficult?
11:53:48
jcowan
What I find surprising is that macros defined at the REPL must be taken into account by file-compiling done at the same REPL.
11:55:31
jackdaniel
jcowan: you may find this article (if you don't know it yet) very interesting: https://www.dreamsongs.com/Files/Incommensurability.pdf
11:57:43
jackdaniel
many people perceive programs (after initial compilation) as finished entities which manipulate their input (so they are static)
11:58:52
jackdaniel
but image-based applications are very much real, not a theoretical possibility. program may be changed (i.e updated, fixed) without restarting it, it is enough to recompile some entities which are requierd to be changed
12:00:04
jackdaniel
no downtime and debugging of "living" issue are two things which come to mind as direct perks of having such organic environment to work on
12:00:10
jcowan
On reflection, it makes sense, though, given that the compiler is intended to mimic the REPL's behavior.
12:01:27
jackdaniel
I think it is more: "using the same environment" than "mimicing one another", unless I misunderstand you
12:18:08
jcowan
I meant that loading a FASL file is supposed to have the same effects that loading a source file would.
12:20:04
jcowan
I note that this is not entirely true, however. A compiled defstruct with :include, where the included struct is defined at the REPL, does not produce the same results when loaded (at least in sbcl) if the included struct defined at load time differs from that at compile time.
12:21:53
beach
EQ-ness of sometimes lost, so there is a concept of "similarity" to compensate for that loss.
12:23:01
beach
I am reading this page: http://www.lispworks.com/documentation/HyperSpec/Body/03_ba.htm and I am wondering about the consequences of the evaluation environment and the compilation environment being different.
12:23:36
jcowan
I tried to reduce things to a simple case, but they seem even more broken than I thought.
12:23:55
beach
So a subsequent (eval-when (:compile-toplevel) ...) may not be able to use that macro definition. Am I right?
12:24:36
jcowan
into a file, defined (defstruct foo x) at the REPL, and loaded the file. (make-bar) returns a structure with two slots, as expected.
12:26:24
beach
So I really should implement incremental first-class global environments, i.e. where modifications are made in a "delta" so that they do not alter the original environment.
12:27:02
beach
That way, the evaluation environment would be a delta on top of the startup environment and the compilation environment would be a delta on top of the evaluation environment.
12:27:46
jcowan
But if I compile the file, throw away the world, evaluate (defstruct foo x1 x2) and load the FASL, iI get a warning and then (make-bar) is not even defined!
12:28:01
beach
I need these deltas in Second Climacs anyway, because I want to be able to "back out" some modifications to the environment when some top-level form changes.
12:29:37
jcowan
Sorry, there is a load-time error, not just a warning (I missed it in the noise of loading): "The loaded code expects an incompatible layout for class FOO."
12:31:12
heisig
beach: Implementing environment deltas sounds really error-prone. What about implementing purely functional first class global environments? Or is that the idea?
12:32:15
jcowan
The (proposed) global-environment system for Scheme allows both mutation and chaining of environments in the manner of an alist.
12:32:42
jcowan
As things now stand, a Scheme must provide multiple environments, but only one is required to be mutable.
12:33:38
jcowan
I tried the same process with clisp, where loading the FASL file provokes a different error: "The class FOO has no slot named X", although there is no reference to X in the definition of struct bar.
12:34:09
beach
jcowan: For your information, since I have first-class global environments, I also plan to make it possible to start a compilation from a fresh environment, equivalent to the one containing only the standard Common Lisp functionality.
12:34:33
beach
jcowan: If you expect stuff like that to work, I think you are going to be disappointed.
12:36:00
jcowan
I was surprised to see early binding in a basically late-binding language, that's all.
12:36:03
beach
heisig: I was thinking of an environment divided into a small number of layers. A lookup would search the layers in order. A mutation would alter the topmost layer.
12:36:11
heisig
beach: I would assume that implementing deltas would roughly double the amount of special purpose code (first look in the delta, then in the underlying environment), whereas for purely functional environments, it would be sufficient to use the right data structures internally.
12:37:34
jcowan
With the proviso that binding a special variable also must count as mutation for this purpose. Binding *print-pretty* in one environment should not affect all environments.
12:38:08
beach
heisig: I think that's correct. Every query operation would have to do a recursive call if it fails. Though it might be possible to automate the definitions so that this code does not have to be written out explicitly for each query operation.
12:39:01
beach
jcowan: Yes, I am sure there are exceptions to the general rule. I would just have to think about every operation and determine the behavior in the case of layers.
12:40:19
jcowan
Indeed, it is assoc where some of the objects on the alist are hashtables rather than key-value conses.
14:04:27
scymtym
i sometimes use the names LOOKUP, DIRECT-LOOKUP and (SETF DIRECT-LOOKUP) (and maybe (SETF LOOKUP) which just calls (SETF DIRECT-LOOKUP)) when implementing stackable environments. that way, clients can choose which lookup they want
14:06:09
beach
I guess if I systematically make my functions return a second value (T or NIL), most of that stuff could be automatically generated.
14:13:32
jcowan
I think it would be important to not allow mutations+ to affect lower layers. If you need to mutate a lower layer, you need a pointer to it so you can make it the topmost layer.