freenode/#clim - IRC Chatlog
Search
3:58:36
beach
I recently saw some very real evidence of the result of Kahneman as reported in his book "Thinking, fast and slow", i.e. that the fast module is lousy with things like orders of magnitude and statistics, and the slow module is lazy so it tends to believe the fast module. The evidence I saw makes me very convinced that a large number of wrong decisions are made with respect to software development.
3:58:44
beach
In this case, I mentioned that I plan to track source information in SICL by keeping a string containing the text of each source file in memory with the code, and I got a pretty violent reaction.
3:58:46
beach
After some verification, it turns out that the entire source code for the Lisp machine is around 50MB, which represents 20 cents of RAM. For the SICL source code, it's 6.4MB or around 3 cents. The fast module had obviously estimated several orders of magnitude more and the fast module didn't bother to check.
4:01:12
beach
The advantages of keeping the source code with the executable code is of course that, even if the source file was not delivered or installed, or if it was modified a posteriori, then I can still have precise error reporting with respect to source location.
4:03:16
beach
I am intimately convinced that many wrong decisions about what optimizations to implement are based on a hunch delivered by the fast module, and that if the slow module had bothered to check, it might turn out that the potential gain is very small and that it would be better to spend that energy on more profitable work.
4:05:39
beach
Hmm, I might get a chapter in my book about the sorry state of the software industry out of this kind of material.
4:20:24
nyef
Note that storing the source code with the running image is one short step from treating the version of the source code in the image as the master copy, which we already discussed as being a Bad Idea.
4:20:59
nyef
(I'm not saying "don't store the source code with the running image", I'm saying "here's a line that almost certainly shouldn't be crossed".)
4:21:23
beach
It will be stored with each function, so that it will be the true version that the executable code was compiled from. That's the only usage.
4:22:53
beach
When EDIT is called, the in-memory copy should not be modified. Instead, the file should be looked for, compared to what is in memory (and warnings issued if they are not the same), and then the file should be loaded into an editor buffer.
4:24:50
beach
And this scenario should be respected for a LispOS as well. There are no "files" there perhaps, but there is an editable object for the source and there is this read-only copy of it that gets associated with the executable code.
4:25:52
nyef
I'm not convinced about "no files", but I'm happy to entertain "files are not necessarily bags of bytes".
4:26:01
beach
So storing the source code with the executable is just a very cheap way of improving the debugging experience.
4:27:32
nyef
There's still a scenario where total "memory" usage is an important consideration. And that's low-bandwidth data transfer.
4:29:13
beach
Well, my main point with this rant was not about memory usage, but about how the fast module can get it wrong by several orders of magnitude. The ultimate decision about memory usage depends on situations like the ones you mentioned, and whether the creator of the software cares about such situations.
4:48:03
beach
Another one of my favorite situations where the fast module gets it wrong is (but I have mentioned this situation several times in the past) when a project decides to use C++ "because we need all the speed we can get", but they then do not take into account the huge amount of additional developer time that it will take for them to actually get that kind of speed.
4:49:12
beach
Again, I am not saying that C++ is always the wrong decision. I am saying that the decision is often made by the fast module, without taking important considerations such as developer time into account.
4:56:00
loke
You might not say that, but I say that C++ is pretty much always the wrong decision, regardless of your requirements.
4:57:16
nyef
When is C++ not the wrong decision? When the requirement is "interfaces with C++". A better decision, however, is "renegotiate the requirements".
4:59:22
loke
ACTION wonders how often the requirement is "interfaces with C++" and at the same time it can't be solved with a C++ shim.
5:03:12
beach
slyrus: What do you mean by "failed to reproduce"? The research results? I don't things so. They have been quoted time and again, and they have been verified by other researchers.
5:05:24
beach
slyrus: Kahneman even got the "Swedish national bank's economics prize in memory of Alfred Nobel" (usually incorrectly referred to as the "Nobel prize in economics") for his work.
5:06:18
jackdaniel
on the other hand, even if it is desireable by developers - possibility to hide code encourages mostly bad hacks
5:08:32
beach
jackdaniel: There is only one situation I can imagine where it is worth not saving the source, namely when the Common Lisp system is used for commercial development so that the source code should not be distributed. But nyef cited some other considerations that are not high-priority for SICL.
5:13:50
slyrus
beach: http://retractionwatch.com/2017/02/20/placed-much-faith-underpowered-studies-nobel-prize-winner-admits-mistakes/
5:19:24
beach
loke, jackdaniel: My point can be made like this: The SBCL default heap size is 1GB. The total amount of source code in an SBCL distribution seems to be 0.5MB, but to make it simple, let's say 1MB. That is 0.1% of the heap size. If anyone has a small memory to be concerned about, they had better do something else, like remove less important modules.
5:23:23
jackdaniel
loke: hint: you may strip ECL modules even further with unix `strip' command (for native compilation)
5:24:11
loke
jackdaniel: That only saves disc sapce though, right? I mean, as ar as I know the linkk loader doesn't map the symbol and debug segments when you run a program.
5:24:46
jackdaniel
hm, maybe, I thought it loads the whole module into the memory, but you may be right
5:25:16
jackdaniel
probably I should add it to the compiler, that if optimize space equals zero, strip is called
5:26:56
beach
slyrus: Thanks for pointing that out to me. The contents of his book is still confirmed by my own observations, even though it may not be scientifically verifiable. But I should be more careful citing his work.
5:27:13
jackdaniel
loke: in any case I advise against using sstrip (super-strip), because it breaks the interactive development of such modules
5:29:01
loke
jackdaniel: Y9ou mean optimise space is _not_ zero rigth? Doesn't zero mean "i don't care about size", and 3 means "i really, really care about the size)