freenode/#clim - IRC Chatlog
Search
3:58:36
beach
I recently saw some very real evidence of the result of Kahneman as reported in his book "Thinking, fast and slow", i.e. that the fast module is lousy with things like orders of magnitude and statistics, and the slow module is lazy so it tends to believe the fast module. The evidence I saw makes me very convinced that a large number of wrong decisions are made with respect to software development.
3:58:44
beach
In this case, I mentioned that I plan to track source information in SICL by keeping a string containing the text of each source file in memory with the code, and I got a pretty violent reaction.
3:58:46
beach
After some verification, it turns out that the entire source code for the Lisp machine is around 50MB, which represents 20 cents of RAM. For the SICL source code, it's 6.4MB or around 3 cents. The fast module had obviously estimated several orders of magnitude more and the fast module didn't bother to check.
4:01:12
beach
The advantages of keeping the source code with the executable code is of course that, even if the source file was not delivered or installed, or if it was modified a posteriori, then I can still have precise error reporting with respect to source location.
4:03:16
beach
I am intimately convinced that many wrong decisions about what optimizations to implement are based on a hunch delivered by the fast module, and that if the slow module had bothered to check, it might turn out that the potential gain is very small and that it would be better to spend that energy on more profitable work.
4:05:39
beach
Hmm, I might get a chapter in my book about the sorry state of the software industry out of this kind of material.
4:20:24
nyef
Note that storing the source code with the running image is one short step from treating the version of the source code in the image as the master copy, which we already discussed as being a Bad Idea.
4:20:59
nyef
(I'm not saying "don't store the source code with the running image", I'm saying "here's a line that almost certainly shouldn't be crossed".)
4:21:23
beach
It will be stored with each function, so that it will be the true version that the executable code was compiled from. That's the only usage.
4:22:53
beach
When EDIT is called, the in-memory copy should not be modified. Instead, the file should be looked for, compared to what is in memory (and warnings issued if they are not the same), and then the file should be loaded into an editor buffer.
4:24:50
beach
And this scenario should be respected for a LispOS as well. There are no "files" there perhaps, but there is an editable object for the source and there is this read-only copy of it that gets associated with the executable code.
4:25:52
nyef
I'm not convinced about "no files", but I'm happy to entertain "files are not necessarily bags of bytes".
4:26:01
beach
So storing the source code with the executable is just a very cheap way of improving the debugging experience.
4:27:32
nyef
There's still a scenario where total "memory" usage is an important consideration. And that's low-bandwidth data transfer.
4:29:13
beach
Well, my main point with this rant was not about memory usage, but about how the fast module can get it wrong by several orders of magnitude. The ultimate decision about memory usage depends on situations like the ones you mentioned, and whether the creator of the software cares about such situations.
4:48:03
beach
Another one of my favorite situations where the fast module gets it wrong is (but I have mentioned this situation several times in the past) when a project decides to use C++ "because we need all the speed we can get", but they then do not take into account the huge amount of additional developer time that it will take for them to actually get that kind of speed.
4:49:12
beach
Again, I am not saying that C++ is always the wrong decision. I am saying that the decision is often made by the fast module, without taking important considerations such as developer time into account.
4:56:00
loke
You might not say that, but I say that C++ is pretty much always the wrong decision, regardless of your requirements.
4:57:16
nyef
When is C++ not the wrong decision? When the requirement is "interfaces with C++". A better decision, however, is "renegotiate the requirements".
4:59:22
loke
ACTION wonders how often the requirement is "interfaces with C++" and at the same time it can't be solved with a C++ shim.
5:03:12
beach
slyrus: What do you mean by "failed to reproduce"? The research results? I don't things so. They have been quoted time and again, and they have been verified by other researchers.
5:05:24
beach
slyrus: Kahneman even got the "Swedish national bank's economics prize in memory of Alfred Nobel" (usually incorrectly referred to as the "Nobel prize in economics") for his work.
5:06:18
jackdaniel
on the other hand, even if it is desireable by developers - possibility to hide code encourages mostly bad hacks
5:08:32
beach
jackdaniel: There is only one situation I can imagine where it is worth not saving the source, namely when the Common Lisp system is used for commercial development so that the source code should not be distributed. But nyef cited some other considerations that are not high-priority for SICL.
5:13:50
slyrus
beach: http://retractionwatch.com/2017/02/20/placed-much-faith-underpowered-studies-nobel-prize-winner-admits-mistakes/
5:19:24
beach
loke, jackdaniel: My point can be made like this: The SBCL default heap size is 1GB. The total amount of source code in an SBCL distribution seems to be 0.5MB, but to make it simple, let's say 1MB. That is 0.1% of the heap size. If anyone has a small memory to be concerned about, they had better do something else, like remove less important modules.
5:23:23
jackdaniel
loke: hint: you may strip ECL modules even further with unix `strip' command (for native compilation)
5:24:11
loke
jackdaniel: That only saves disc sapce though, right? I mean, as ar as I know the linkk loader doesn't map the symbol and debug segments when you run a program.
5:24:46
jackdaniel
hm, maybe, I thought it loads the whole module into the memory, but you may be right
5:25:16
jackdaniel
probably I should add it to the compiler, that if optimize space equals zero, strip is called
5:26:56
beach
slyrus: Thanks for pointing that out to me. The contents of his book is still confirmed by my own observations, even though it may not be scientifically verifiable. But I should be more careful citing his work.
5:27:13
jackdaniel
loke: in any case I advise against using sstrip (super-strip), because it breaks the interactive development of such modules
5:29:01
loke
jackdaniel: Y9ou mean optimise space is _not_ zero rigth? Doesn't zero mean "i don't care about size", and 3 means "i really, really care about the size)
8:33:38
flip214
think of a sudoku GUI.... if I have some constant defined to 3 (the typical sudoku case),
8:34:47
flip214
should I do these from a tabular layout? or have them created via #. in the (:panes ...) of define-application-frame?
8:35:35
flip214
have a single pane there that uses my own function to draw? this has the disadvantage that I'd have to receive and parse mouse clicks, and determine the correct cell from the coordinates, etc.
8:36:23
flip214
not sure what the best way would be ... right now I'm thinking about #. in (:panes ...) and (:layouts ...)
8:37:39
beach
Each presentation would then have an underlying object that would be recovered automatically when clicked upon.
8:39:07
flip214
beach: thanks for the hint... grepping for grid gives me Documentation/Guided-Tour/simple-spreadsheet.lisp, which sounds good.
8:42:48
flip214
beach: yeah, I already saw that. I'm not sure whether I want to put buttons, input fields, or something else in each cell location...
8:45:36
beach
I have a minimalist Sudoku. Do you want me to give it to you, or do you want to figure this out for yourself?
8:49:29
flip214
but since the space of possible solutions is so big, and personal time so limited, having others provide suggestions is very valuable...
8:49:31
beach
And I had a group of students work on an algorithm for constructing grids. It was amusing, because it would use the various strategies that humans use in order to make sure the grid could be solved using such strategies. Different sets of strategies gave different levels of difficulty.
8:50:25
flip214
yeah, that's what I'm afraid of... doing such a thing "by hand" involves quite a big solution tree to investigate by its own.
8:51:38
flip214
I'll try to put my solver/explainer into your code, perhaps you're interested in it later on.
8:53:24
flip214
yeah, right. solves step-by-step and explains why some cell must contain some number.
8:54:00
beach
Excellent! If we join forces we have all three aspects: A GUI, a solver/explainer, and a grid creator.
8:55:01
beach
I have some additional code in a different place. I should try to merge it into the newly created repository.
8:55:32
flip214
there are quite a few different sudoku.lisp in the various subdirectories... is there one that I should take?
8:56:53
beach
Oh, I am confused. I must have either failed to create a repository because I had one, or I did something else wrong.
8:57:59
flip214
no problem.... take your time sorting that out, I'll have to leave for Family time anyway.
9:17:04
beach
flip214: You know what we should do? We should come up with a protocol for the grid. That way, we can experiment with different ways of implementing its graphic layout, and we can write solvers etc. independently of the way it is ultimately presented.
9:19:20
flip214
ie. the total area is still a 9x9, but the ranges are horizontal/vertical, and a connected 9-cell range
9:21:02
flip214
If I can store more data persistently (like possible values for a cell), the next solving step would be accelerated, of course...
9:21:55
flip214
so, as long as there's a button that calls my function with the 9x9 array, I'll just return the new array and an explaining message
9:22:19
flip214
although it would be nice to be able to colour the cells involved in this step, so that the reason is easier to see
9:24:41
flip214
but this is just a normal layout, where rows and colums are moved around. nothing special.
9:27:40
beach
My hunch is that it is possible to come up with solvers etc that only take into account the fact that an elementary cell belongs to three different (say) aggregates.
9:28:11
beach
Again, my hunch is that the rest is just a matter of organizing the elementary cells visually.
9:29:37
beach
flip214: Thanks for broadening my views. It will give me something to think about when I am bored with my work.
10:09:19
flip214
beach: yeah, exactly. My solver takes the 3 different sets of cells that one cell belongs to, and derives new knowledge from that.
11:13:43
nyef
I play a variant with irregularly-shaped subregions and instead of the grid being seeded with numbers, there are *more* subregions of varying sizes marked with the total of all the digits inside.