freenode/#sicl - IRC Chatlog
Search
1:01:40
karlosz
Bike: i was reading the make-load-form specification pretty carefully today; i don't know if you thought about it already but could you base an object graph copier/serializer based on the make-load-form mechanism?
1:02:46
Bike
yeah, i've tested that a bit in something else and it works. but you have to touch a file for it, which is not great.
1:05:08
karlosz
hm, i guess that's true, since the file compiler needs to be involved to actually do the work
1:06:16
karlosz
maybe just writing a 'liter' customized version of the literal dumper specifically for cloning in-memory objects which piggybacks off of make-load-form
1:07:01
Bike
well the nature of make-load-form is that it needs something compiler-like, since it checks circularity by walking the forms looking for constants
1:14:28
karlosz
i'm currently working on cleaning up some aspects of sbcl bootstrap which pretty intimately tie into the loader, file-compiler, and literals handling so i thought it could be adapted to the problem of cloning in memory stuff
1:14:58
karlosz
it's actually weird that there's this mechanism to reconstruct serialized object graphs but not also one just to copy since the protocol is basically identical
1:16:20
Bike
i think the way the compile-file mechanism is form based kind of throws a wrench in using it in any lighter weight way
1:20:16
karlosz
(m-v-b (c-f i-f) (make-load-form some-flat-struct) (eval c-f) (eval i-f)) works fine for simple stuff
1:21:27
karlosz
i guess the difficult part is actually doing the walk, since there's no easy way to do the part where other constants involved in the m-l-f form can get copied as well
1:22:04
Bike
that's what i mean. if you want to copy a whole object graph you can't just do eval since the forms will refer to constants you want to copy.
1:22:15
Bike
it would still be kind of nice if it was exposed in some way without touching a file, though
1:24:39
karlosz
right, i guess what you must've done is making a file with a macro or symbol-macro call to a value and compile-filing it?
1:27:19
Bike
basically i wrote (defvar *load* #.*save*) to a file, and then (let ((*save* object)) (compile-file file))
1:31:17
karlosz
oh yeah, so like (setq *load* #.*save*) to a file, and then (defun clone-object (object) (let ((*save* object)) (compile-file file) (load file) *load*)))
1:32:27
karlosz
there's probably a way to avoid touching the actual file system, but the cool thing is you could have a compiled object generator
1:51:38
Bike
there's no standard way to avoid it, i don't think. you can't compile-file with streams
3:06:43
no-defun-allowed
Me? Not much - I was going to call my old university (again) because they may or not be billing me when they shouldn't, but they are using some online conferencing program I can't use, as I am not a student anymore. You?
3:07:59
beach
Having problems with my bank, but those are what my (admittedly small) family calls "first-world problems". Today I should act upon the modifications to the specification that heisig brought up.
3:18:51
no-defun-allowed
Right then. I can't think of any other instructions which require tricky register allocation.
3:22:20
no-defun-allowed
I don't have a list of MIR instructions with outputs to allocate handy. That could help, but I think we handled all three cases: two-address integer instruction, one-address multiplication and division, and three-address floating point instructions.
3:23:25
beach
So, can you run the bootstrapping procedure to completion with register allocation turned on?
3:24:02
beach
Because, then I would just leave the register allocation on and move to code generation.
3:25:00
beach
There are probably a few instructions missing in Cluster, but otherwise, it shouldn't be too bad.
3:26:00
beach
Then, if, in adding more stuff to bootstrapping, the register allocator finds something it can't deal with, we will just handle that when it shows up.
3:45:40
beach
It's basically: (loop for enter-instruction in (sicl-hir-to-mir:gather-enter-instructions mir) do (sicl-register-allocation:do-register-allocation enter-instruction))
3:47:47
no-defun-allowed
Somewhere in E5: There is no applicable method for the generic function #<STANDARD-GENERIC-FUNCTION CLEAVIR-IR:ELEMENT-TYPE (5)> when called with arguments (NIL).
3:49:29
beach
We should try to stick more asserts in the code so that we catch such cases earlier, before they propagate further.
3:50:40
no-defun-allowed
I think it is trying to allocate a register for an output of an INITIALIZE-VALUES-INSTRUCTION, which has none.
3:50:48
beach
Even using FIRST, SECOND, etc. On input/outputs/successors is a problem because of the way they handle NIL.
3:51:07
no-defun-allowed
Though I am missing some stack frames. Could you remind me how to ensure SBCL compiles optimizing for (DEBUG 3) please?
3:54:37
no-defun-allowed
According to the documentation, an INITIALIZE-VALUES-INSTRUCTION should have a single output, but this one does not.
3:56:55
beach
It is usually something stupid, so you will probably find it. If not, just let me know.
3:58:20
no-defun-allowed
It appears to happen when compiling SICL-CLOS:DEFGENERIC-EXPANDER, which is odd.
4:05:52
no-defun-allowed
The HIR evaluator doesn't appear to produce a thunk with an output. And I can't come up with a good output for that instruction.
4:07:24
no-defun-allowed
The Cleavir book says "This instruction is used for multiple-value-call. Its single output is a location holding an arbitrary number of values to pass as arguments to multiple-value-call."
4:09:14
no-defun-allowed
I was looking at the method for INITIALIZE-RETURN-VALUES-INSTRUCTION and not INITIALIZE-VALUES-INSTRUCTION. Sorry.
4:14:40
beach
And now I must try to remember why I do that, and what it means to augment those instructions.
4:16:03
beach
It used to be the case that I wanted all instructions that might turn into function-call instructions to have no outputs, just like the function-call instruction does.
4:16:51
beach
The NAMED-CALL-INSTRUCTION-likes can have outputs that can be handled by the call-site manager later on.
4:18:27
beach
Pleasure! Sorry for the confusion. That's what happens (to me at least) when I change the contracts of various phases over time.
4:24:08
beach
Related to named-calls, but unrelated to register allocation (and possibly interesting to heisig for type inference): It used to be very hard to introduce new function calls late in the translation process. But now that we have named calls handled by the call-site manager, it is easy.
4:24:17
beach
So my initial plan (which I still don't know why it can't work) for type inference was to turn calls to TYPEP with a constant type into TYPEQ instruction. Then, type inference would try to eliminate those, or simplify them.
4:24:17
beach
Finally, the remaining TYPEQ would either turn into simple instructions such as CONSP, or calls to TYPEP if the type was too complex to handle directly. But it was then hard to introduce new calls to TYPEP. That is no longer the case.
4:39:31
no-defun-allowed
Then I found I hadn't written a method for ELEMENT-TYPE on either RAW-INTEGER or RAW-FLOAT, which got me further into bootstrapping.
4:40:22
no-defun-allowed
Then I have found I have done something wrong with the COMPUTE-OUTPUT-ARRANGEMENT method for FIXNUM-DIVIDE-INSTRUCTION, and I'm looking at that now.
5:49:47
karlosz
i have a question about the spec for make-load-form. would it be valid to optimize away the make-load-form dumping in a situation like (defun f () (if t (foo) #.(make-goo))) where make-goo constructs an object of class GOO whose make-load-form creation form/init form has observable side effects?
5:50:12
karlosz
it would be nice if the compiler had the liberty to not dump the implicit load-time-value at all
5:51:47
karlosz
i guess it's pretty similar to the question of whether (defun f () (if t (foo) (load-time-value (setq *x* 4)))
5:52:46
karlosz
if we take it that a literal reference is operationally the same as (load-time-value (progn <creation-forms> <init-forms>)) then that means its not possible to optimize it
6:00:24
karlosz
well, you would need to dump the code to do the load-time computation straight away in the compilation process before you lower/optimize the code any further
6:42:43
beach
heisig: I am looking at section 30.7 (Compiler macros). I think there are still some cases where compiler macros are justified, but I will moderate the wording a bit, because some cases are definitely covered by call-site optimization.
7:00:46
beach
I updated http://metamodular.com/SICL/sicl-specification.pdf so that we have the same chapter numbers.
7:08:12
Colleen
heisig: Bike said 13 hours, 27 minutes ago: brief description of what i'm thinking wrt trucler optimize info https://gist.github.com/Bike/daa1bf795c8b718022856ac4cb15175a
7:16:54
heisig
Here is what the spec has to say about externalizable objects and compile-file (what karlosz asked):
7:17:04
heisig
CLHS 3.2.4: "compile-file, on the other hand, must produce a compiled file that, when loaded with load, constructs the objects defined by the source code and produces references to them."
7:18:00
heisig
CLHS 3.2.4.1: " The file compiler must cooperate with the loader in order to assure that in each case where an externalizable object is processed as a literal object, the loader will construct a similar object."
7:19:06
heisig
My reading is that this requires each literal to be constructed, independently of whether it is used or not.
7:20:37
heisig
The most important practical consideration is that I think programmers will expect each literal object to be dumped, so we should do that.
7:21:20
heisig
The other practical consideration is that the load forms of certain objects might contain bugs. And by not running them, we might trick the programmer into thinking there are no bugs.
7:22:37
heisig
And finally, I don't think this case (literal objects never being actually referenced) occurs a lot in practice.
7:25:31
beach
heisig: I take it by "we" you mean SICL developers. But I am not sure that SICL is the target for karlosz' work.
7:27:51
heisig
karlosz: Here is the (old, probably outdated) code for coalescing similar objects in SICL: https://github.com/robert-strandh/SICL/tree/master/Code/Cleavir/HIR-transformations/Value-hoisting
7:30:50
karlosz
an interesting consideration is that named constants might get reconstructed by (symbol-value name) instead of by the value, so that you don't need to duplicate values
7:33:05
heisig
No, I don't think I handle named constants. And I am not sure whether they should be, since CL constants are allowed to be redefined. But I'd have to give this more thought.
7:36:21
karlosz
are they? " A constant defined by defconstant can be redefined with defconstant. However, the consequences are undefined if an attempt is made to assign a value to the symbol using another operator, or to assign it to a different value using a subsequent defconstant. "
7:36:59
karlosz
well, i'm not sure whether you take "the consequences are undefined" as permission or prohibition
7:38:06
beach
That would be up to the implementation to decide. I am in favor of an error being signaled in that case.
7:39:38
karlosz
for the purposes of optimization, it's useful if (defconstant +x+ (complex-object)) (defun f () +x+) (defun g () +x+) it would be nice if both (1) there is no runtime lookup of '+x+ and (2) +x+ is only created once
7:39:50
heisig
beach: Good idea. If an an error is signaled when redefining a constant, SICL could make this optimization, too.
7:40:19
karlosz
so the only way that could happen is if named-constants get dumped as (load-time-value (symbol-value '+x+')) or something
7:42:43
heisig
And it could be implemented relatively easily - the similarity table can just be pre-populated with a special constructor for all objects that are the result of a defconstant form.
7:43:36
heisig
So when this object is encountered again as a literal object, it is handled by emitting (symbol-value name).
7:46:47
no-defun-allowed
I found the problem. For either tricky instruction, we have to ensure nothing is attributed to RAX and RDX. I wrote the function to clear up a register at a time. What would happen was it would relocate from RAX to another register, and then RDX to RAX because it was only programmed to avoid relocating back to the register to evacuate.
7:47:15
no-defun-allowed
That would have the end result of leaving RDX unattributed, but RAX attributed.
7:47:31
beach
no-defun-allowed: Heh, I kind of anticipated problems like that with the current protocol.
7:50:15
beach
So the registers used for passing arguments, argument count, and the environments must be freed up.
7:51:33
beach
Later, we can introduce special rules for internal calls, but right now, we treat all FUNCALL-INSTRUCTIONs as if they were calls to unknown functions.
7:56:46
karlosz
since the load-form for a named constant could do something different than the load-form of the literal value
8:11:07
heisig
karlosz: In what way is the value-producing form of a defconstant form different from a creation form for a literal object?
8:11:59
heisig
And both can have references to arbitrary literal objects. (With the exception that named constants cannot refer to themselves).
8:12:51
karlosz
i just mean if you have (defconstant +x+ (make-foo :a 1 :b 2)) and you have (defun f () (cons +x+ #.(make-foo :a 1 :b 2))) should you do coalescing?
8:13:33
karlosz
(load-time-value (symbol-value '+x+)) might not give you the same thing as the make-load-form for foo would
8:21:04
heisig
But in that case the problem occurs only for objects that are not built-in, right? For build-in objects, the implementation knows how they can be dumped and coalesced.
8:21:57
heisig
And your make-foo case should also work, but never coalesce, because both objects are distinct in the host.
8:36:23
heisig
CLHS 3.2.4.2.2 "A general-purpose concept of similarity does not exist for structures and standard objects."
8:56:56
heisig
Two objects that are EQ are the same object. So there isn't even coalescing involved. Or am I missing something?
9:14:31
karlosz
i just mean that +x+ will get dumped as (load-time-value (symbol-value '+x+)) while #.+x+ will have make-load-form called and everything and so actually when loaded (car (f)) won't be EQ to (cdr (f)) without coalescing, so you have to coalesce
9:14:53
karlosz
and if you do coalesce, you hide the fact that #.+x+ might not even have a make-load-form
9:21:51
heisig
If you populate the similarity table entry for the object named +x+ with a special constructor, then #.+x+ will not call make-load-form at all.
9:22:54
karlosz
isn't that weird though? suppose there's no make-load-form method on +x+, then you're supposed to raise an error
9:23:37
heisig
Let me check the standard. I was under the impression the compiler has quite some leeway when dealing with constants.
9:27:36
heisig
Fun fact: I recall one implementation used to simply eval all creation and initialization at load time, without handling complex/circular dependencies at all.
9:32:39
heisig
karlosz: I have a solution for you - the compiler calls make-load-form for that object, but discards the results.
9:33:19
heisig
This way, it satisfies the requirement from CLHS 3.2.4.4 that "The file compiler calls make-load-form on any object that is referenced as a literal object if the object is a generalized instance of standard-object, structure-object, condition, or any of a (possibly empty) implementation-dependent set of other classes."
9:36:19
heisig
My personal opinion is that this case is so rare that we can as well have the compiler send us an email, so that we can congratulate the authors of that code :)
9:38:53
karlosz
that's a fun interpretation! apparently this kind of thing does happen in user code :/
9:40:23
heisig
It does?!? Who performs read-time evaluation on constants that are standard-objects and relies on make-load-form being called?!?
9:42:29
karlosz
and on the sbcl mailing lists dougk reports that changing how this stuff is handled has user observable impact at google for their code
9:42:49
karlosz
since they rely on constant references being eq to each other, and injecting anonymous constants can cause that to fial
9:47:49
heisig
Well, better not touch these things then. As I said, this may not be the most rewarding thing to optimize anyway.