freenode/#sicl - IRC Chatlog
Search
7:16:54
heisig
Here is what the spec has to say about externalizable objects and compile-file (what karlosz asked):
7:17:04
heisig
CLHS 3.2.4: "compile-file, on the other hand, must produce a compiled file that, when loaded with load, constructs the objects defined by the source code and produces references to them."
7:18:00
heisig
CLHS 3.2.4.1: " The file compiler must cooperate with the loader in order to assure that in each case where an externalizable object is processed as a literal object, the loader will construct a similar object."
7:19:06
heisig
My reading is that this requires each literal to be constructed, independently of whether it is used or not.
7:20:37
heisig
The most important practical consideration is that I think programmers will expect each literal object to be dumped, so we should do that.
7:21:20
heisig
The other practical consideration is that the load forms of certain objects might contain bugs. And by not running them, we might trick the programmer into thinking there are no bugs.
7:22:37
heisig
And finally, I don't think this case (literal objects never being actually referenced) occurs a lot in practice.
7:25:31
beach
heisig: I take it by "we" you mean SICL developers. But I am not sure that SICL is the target for karlosz' work.
7:27:51
heisig
karlosz: Here is the (old, probably outdated) code for coalescing similar objects in SICL: https://github.com/robert-strandh/SICL/tree/master/Code/Cleavir/HIR-transformations/Value-hoisting
7:30:50
karlosz
an interesting consideration is that named constants might get reconstructed by (symbol-value name) instead of by the value, so that you don't need to duplicate values
7:33:05
heisig
No, I don't think I handle named constants. And I am not sure whether they should be, since CL constants are allowed to be redefined. But I'd have to give this more thought.
7:36:21
karlosz
are they? " A constant defined by defconstant can be redefined with defconstant. However, the consequences are undefined if an attempt is made to assign a value to the symbol using another operator, or to assign it to a different value using a subsequent defconstant. "
7:36:59
karlosz
well, i'm not sure whether you take "the consequences are undefined" as permission or prohibition
7:38:06
beach
That would be up to the implementation to decide. I am in favor of an error being signaled in that case.
7:39:38
karlosz
for the purposes of optimization, it's useful if (defconstant +x+ (complex-object)) (defun f () +x+) (defun g () +x+) it would be nice if both (1) there is no runtime lookup of '+x+ and (2) +x+ is only created once
7:39:50
heisig
beach: Good idea. If an an error is signaled when redefining a constant, SICL could make this optimization, too.
7:40:19
karlosz
so the only way that could happen is if named-constants get dumped as (load-time-value (symbol-value '+x+')) or something
7:42:43
heisig
And it could be implemented relatively easily - the similarity table can just be pre-populated with a special constructor for all objects that are the result of a defconstant form.
7:43:36
heisig
So when this object is encountered again as a literal object, it is handled by emitting (symbol-value name).
7:46:47
no-defun-allowed
I found the problem. For either tricky instruction, we have to ensure nothing is attributed to RAX and RDX. I wrote the function to clear up a register at a time. What would happen was it would relocate from RAX to another register, and then RDX to RAX because it was only programmed to avoid relocating back to the register to evacuate.
7:47:15
no-defun-allowed
That would have the end result of leaving RDX unattributed, but RAX attributed.
7:47:31
beach
no-defun-allowed: Heh, I kind of anticipated problems like that with the current protocol.
7:50:15
beach
So the registers used for passing arguments, argument count, and the environments must be freed up.
7:51:33
beach
Later, we can introduce special rules for internal calls, but right now, we treat all FUNCALL-INSTRUCTIONs as if they were calls to unknown functions.
7:56:46
karlosz
since the load-form for a named constant could do something different than the load-form of the literal value
8:11:07
heisig
karlosz: In what way is the value-producing form of a defconstant form different from a creation form for a literal object?
8:11:59
heisig
And both can have references to arbitrary literal objects. (With the exception that named constants cannot refer to themselves).
8:12:51
karlosz
i just mean if you have (defconstant +x+ (make-foo :a 1 :b 2)) and you have (defun f () (cons +x+ #.(make-foo :a 1 :b 2))) should you do coalescing?
8:13:33
karlosz
(load-time-value (symbol-value '+x+)) might not give you the same thing as the make-load-form for foo would
8:21:04
heisig
But in that case the problem occurs only for objects that are not built-in, right? For build-in objects, the implementation knows how they can be dumped and coalesced.
8:21:57
heisig
And your make-foo case should also work, but never coalesce, because both objects are distinct in the host.
8:36:23
heisig
CLHS 3.2.4.2.2 "A general-purpose concept of similarity does not exist for structures and standard objects."
8:56:56
heisig
Two objects that are EQ are the same object. So there isn't even coalescing involved. Or am I missing something?
9:14:31
karlosz
i just mean that +x+ will get dumped as (load-time-value (symbol-value '+x+)) while #.+x+ will have make-load-form called and everything and so actually when loaded (car (f)) won't be EQ to (cdr (f)) without coalescing, so you have to coalesce
9:14:53
karlosz
and if you do coalesce, you hide the fact that #.+x+ might not even have a make-load-form
9:21:51
heisig
If you populate the similarity table entry for the object named +x+ with a special constructor, then #.+x+ will not call make-load-form at all.
9:22:54
karlosz
isn't that weird though? suppose there's no make-load-form method on +x+, then you're supposed to raise an error
9:23:37
heisig
Let me check the standard. I was under the impression the compiler has quite some leeway when dealing with constants.
9:27:36
heisig
Fun fact: I recall one implementation used to simply eval all creation and initialization at load time, without handling complex/circular dependencies at all.
9:32:39
heisig
karlosz: I have a solution for you - the compiler calls make-load-form for that object, but discards the results.
9:33:19
heisig
This way, it satisfies the requirement from CLHS 3.2.4.4 that "The file compiler calls make-load-form on any object that is referenced as a literal object if the object is a generalized instance of standard-object, structure-object, condition, or any of a (possibly empty) implementation-dependent set of other classes."
9:36:19
heisig
My personal opinion is that this case is so rare that we can as well have the compiler send us an email, so that we can congratulate the authors of that code :)
9:38:53
karlosz
that's a fun interpretation! apparently this kind of thing does happen in user code :/
9:40:23
heisig
It does?!? Who performs read-time evaluation on constants that are standard-objects and relies on make-load-form being called?!?
9:42:29
karlosz
and on the sbcl mailing lists dougk reports that changing how this stuff is handled has user observable impact at google for their code
9:42:49
karlosz
since they rely on constant references being eq to each other, and injecting anonymous constants can cause that to fial
9:47:49
heisig
Well, better not touch these things then. As I said, this may not be the most rewarding thing to optimize anyway.
12:59:58
heisig
I just spent two more days thinking about whether our extensible libraries should provide a client class or not.
13:01:22
heisig
The only real benefit of having such a class would be increased run-time type safety.
13:01:48
heisig
Calls whose first argument are not a subclass of client could be detected immediately, thus preventing certain kinds of bugs.
13:03:10
beach
I think they should because we might want to use the library in a single image, but processing code for different clients.
13:04:12
heisig
I am not talking about the first argument, but just whether we should introduce a client class and have all default methods specialize on that class instead of T.
13:05:05
heisig
My other argument was that some programmers might want to override even the default methods. But in that case, it makes probably more sense to write a new library.
13:06:19
heisig
And the final argument is simplicity. Users of the protocol will appreciate if the can just supply something like 'CLASP as a client.
13:09:10
jackdaniel
having a client class allows you to do noughty (sometimes useful) things like adding a stealth mixin
13:10:15
beach
But since the default methods can't take advantage of such a mixin anyway, you may as well add such information to your subclass.
13:20:36
beach
OK, I am thinking about what heisig mentioned the other day, with respect to what the style guide says. When I wrote some initial code for SICL, and when I wrote some of the style guide early on, I had in mind that run-time errors should report which function signaled the error.
13:20:43
beach
I no longer think that, for two reasons. 1. Application code could be using a macro that expands to a call to a function that the application programmer is not aware of, so reporting the function would be confusing. 2. When SICL is used with some IDE, the source location will be available in the backtrace anyway.
13:20:48
beach
So the style guide has a rule that a system function F should call another system function G only if it is sure that G does not signal an error, because otherwise the programmer would have called F but the error is reported in terms of G. So the question now is whether we should get rid of that rule in the style guide.
13:20:51
beach
I still think it is a good rule because the problem could involve objects that were (partially) created by F, so the application programmer would not recognize the problem from the reported error.
13:27:15
beach
Run-time errors need no source information, because the return addresses in the stack, which could be from the caller or the snippet, can be used to figure out the call site, and the code object contains source information for the call sites, so the stack frames provide enough information to find the source
13:31:08
beach
Now, what about compile-time conditions. Well, there are two kinds of conditions. One is a run-time error in the compiler (which should not happen, but could if the compiler has defects in it). And the other kind is a a condition emitted by the compiler to inform the programmer of some issue.
13:31:15
beach
The second kind will obviously have source information associated with it. But, the entry point for the compiler needs to treat the two differently. One is a run-time error as I mentioned before, but the other is not.
13:31:17
beach
So, at the very least, we need a base class for compile-time errors so that we can handle them specially.
13:31:50
Bike
https://github.com/s-expressionists/Cleavir/blob/master/Conditions/program-condition.lisp#L3-L13
13:34:23
Bike
for the purposes of debugging the compiler it is sort of useful to associate source information with the first kind of condition too. so that e.g. if you compile a big file and the compiler screws up on a form, you know which form it screwed up on. that can be done with handler-bind and a wrapper condition in the right places
13:34:57
Bike
ideally i'd probably throw something in so that if the compiler signals anything but a program-condition it's wrapped in a condition BUG or something
13:35:56
beach
Now, when we report a run-time condition, we end up in the so-called "debugger". If SICL is executed in a dumb terminal, we need such a debugger, and it is not terribly hard. In fact, the portable condition system supplies one.
13:37:05
beach
But I am not particularly interested in that case, and when we report a run-time condition in some IDE we get the backtrace inspector and that should work OK.
13:38:47
beach
In SICL for the terminal, the source information contains the program text as a string, so it can be reported that way. Again not terribly interesting as long as it is possible.
13:39:32
beach
For the IDE, I imagine a source buffer being created (unless it already exists) and the problematic forms being annotated in some way.
13:41:11
beach
So does that mean that we must duplicate large parts of the text in the condition handler in the IDE?
13:42:55
beach
But the tool tip is essentially the same as the condition reporter in the terminal minus the source information.
13:44:29
Bike
well, i guess the way clasp does it there is some duplication between clasp and slime, i think
13:45:11
beach
I guess if the condition reporter does not include source information, then it can be used in both the terminal and the IDE, but the terminal would have a condition handler that also presents the source information as text.
13:48:04
beach
And the analogous condition handler in the IDE would instead use the source information to annotate the code.
13:49:07
Bike
yeah, i think i get it. that's pretty much how it works in clasp. source info is displayed by a handler in compile-file.
13:51:16
beach
And I should go look at the condition reporters to see that they make sense both with a terminal and as a tool tip or similar.
13:56:19
beach
Oh, and I also want to "get rid of" Acclimation, in the sense that I don't want English-language condition reporters to depend on it. For SICL, that would mean to structure the condition reporting so that it trampolines to methods like the ones we now define with a LANGUAGE parameter.
13:56:19
beach
For SICL-specific code, I would still write the condition reporters separately in most cases, but the English-language ones would not refer to Acclimation. And we could accept condition definitions with a :REPORT option for code written elsewhere, rather than depending on everyone using Acclimation.
14:02:30
beach
I would still like to be able to accommodate such a library, and have it report conditions normally, while being able to write new condition reporters for other languages.
14:03:15
beach
And, I would like for our own libraries like Clostrum, Trucler, etc. to be usable for other Common Lisp implementations that have no desire to use Acclimation.
14:04:51
beach
So, somehow, I want the English-language condition reporters to look "normal", i.e. the way they would if Acclimation were not present, while still retaining the possibility of adding condition reporters for other languages. It may not be possible, or easy. That's what I am trying to figure out.
14:09:52
jackdaniel
condition reporting is mediated through the print-object; perhaps a slight abuse by specializing the stream (or defining an around method) could be used
14:11:22
Bike
i don't understand the looking normal thing. all acclimation does is define a generic function that the condition report calls. it doesn't impose any kind of formatting
14:12:20
jackdaniel
Bike: yes, but you must specify :report that-function; it is hard to expect i.e alexandria to depend on acclimation to specify that report function
14:12:42
beach
Right. But if (say) Alexandria does not respect this protocol, then we can't create condition reporters for Alexandria conditions in other languages.
14:13:26
jackdaniel
alternatively expand :report foo to something like (lambda (c s) (if *lang* (that-function c s *lang*) (foo c s))
14:14:38
beach
I.e., does the standard say that the :REPORT results in a call to PRINT-OBJECT? Or is it more vague about it.
14:15:06
jackdaniel
there is even: specyfing (:report foo) in the definition is equivalent to (and the code snippet)
14:16:17
jackdaniel
so it seems that a more conforming solution would be adding an around method to the print-object
14:29:42
jackdaniel
another way which I'm not sure about is using the pretty printer dispatch; however I've never used it extensively
14:30:14
Bike
i think you're supposed to be able to print conditions human-readably even when *print-pretty* is false.
16:43:58
beach
So one common complaint about Emacs is that it creates buffers and windows in a way that can neither be predicted nor controlled. Whether that is true or not I don't know, but the default behavior annoys me as well. So suppose we have an IDE with a REPL, and we want to keep the configuration of the different, let's call them panes, way more fixed than Emacs does.
16:43:59
beach
Now suppose I start a compilation (asdf:load-system...) in the REPL, and there are compilation errors/warnings in several files. How should this situation be presented to the programmer? Maybe a tab for each file in *the* pane that shows source code?
16:57:51
splittist
Some IDEs have a compilation results tab in the 'repl' pane that shows errors, warnings etc. There are often also indications against file-names in the project tree pane, and indications in the margin of each buffer at the offending line.
17:00:09
splittist
Sometimes there is a visual/spatial differentiation between interactive 'terminal' areas and editing areas (although obviously one at least 'line edits' in a terminal).
17:00:18
pjb
beach: It's a diffcult question. Basically in emacs, windows belong to the user. In more structured IDE, windows belong to the IDE. Perhaps some people like that, I find it more difficult to work with such IDEs…
17:00:56
pjb
beach: the thing is that emacs is highly non-modal. So, compilation results are not integrated in a work flow. They're just dumped in a buffer, show in a window, and let the user deal with it.
17:01:30
pjb
beach: more structured IDEs impose, or strongly hint at specific workflows, hence a structure of windows. (I keep emacs terminology).
17:02:16
pjb
beach: so perhaps we could start with that, define some workflows, and a set of specfic windows that will be used by the different inputs or outputs of the workflow.
17:03:40
splittist
There's the whole windows in registers thing, too. Although it's always C-x 2 - no wait, I meant C-x 3; C-x 4 'where will this appear'; next window, no, not that one, this one; C-x 0 - start again (:
17:03:45
pjb
Now, tab are a good idea, until they're not. Ie when there is a small number of tabs, they're ok, but soon enough there are a lot of them and they become unmanageable… So be careful with them.
17:04:22
pjb
yes, and there's the split horizontally or vertically, depending on the task or the size (aspect ratio) of the frame!
17:04:49
splittist
Perhaps one should be thinking of multi-monitor from the ground up. If not multi-monitor+multi-virtual-desktop
17:07:36
beach
ACTION is still here because his (admittedly small) family is in a virtual meeting so is unable to serve dinner on schedule.
17:09:44
beach
The other thing I have been thinking of is to structure the workflow around "modules" as defined by ASDF system definitions. Then, the position of files in the file system would be less important. But I suppose not everybody works the way i do with such things, so it would have to be some kind of option.
17:10:57
pjb
I wouldn't mind if a new workflow would create a new frame with its fixed windows inside.
17:11:41
pjb
What people complain is probably when emacs changes the windows they've careful set up for their own workflow.
17:39:34
beach
I guess the way to go would be to let the user configure the different panes. Like we just propose different panes that work in different ways, and the user can build a custom IDE out of those.
17:47:27
pjb
I like when systems have memory. If the user moves a window, next time we can create a similar window in the same place.
18:21:17
jackdaniel
(format-graph-from-roots (list "clim-basic") #'print (lambda (s)(ignore-errors(asdf/system:system-depends-on(asdf/system:find-system s)))))
18:21:38
jackdaniel
this ignore-errors is because I didn't bother to parse dependencies like (:version "flexichain" "1.5.1") or (:require "sb-introspect")