freenode/#lisp - IRC Chatlog
Search
3:31:03
vtomole
beach: Saturday night hacking lisp, couldnt get any better. Making some progress on the allocator paper too
3:51:47
vtomole
beach: I still need to hear an opposing viewpoint to this. Garbage collection protects you from "shooting yourself in the foot", but there are reasons why new languages that don't have automatic memory management are still being created.
3:55:39
beach
vtomole: I think the way to go is to use a concurrent and incremental garbage collector. That way, it would be possible to take advantage of processors with multiple cores as well.
3:57:02
White_Flame
right, most larger systems tend to build up a runtime of their own that isn't going to be as good as what you get from language-supported features
4:00:38
beach
vtomole: There are three different issued here, and they are mostly orthogonal. One is overhead. There GC wins no problem. Implementing malloc() and free() efficiently is very hard. The second issue is real-time. This one is tougher, because a real-time GC does have more overhead than a non-real-time GC.
4:00:42
beach
But then, you don't get real-time automatically with malloc() and free() either, because of the complex data structures that have to be maintained. The final issue is development and maintenance cost.
4:00:44
beach
It is known that not using GC increases that cost a lot. In fact I have seen projects that were canceled after a few million Euros were spent, just because they couldn't simultaneously create a modular system and one that did free() at the right time.
4:01:39
White_Flame
both malloc and free have arbitrary data shuffling & maintenance they might do per call, so they're usually far from predictable performance
4:02:30
beach
But it is common for (not so knowledgeable) developers to believe that malloc() and free() behave in predictable ways.
4:05:43
vtomole
beach: Reading this paper, it seems like a developer would want his/her language to support multiple types of garbarge collectors; So the user can pick one based on the goal.
4:05:56
beach
vtomole: More generally, you should be aware that phrases like "we need all the speed we can get", or "we need absolute bounded response times" really mean "we are willing to spend any amount of additional development and maintenance time for even a minuscule improvement in performance (or to get bounded response)". Only when they realize that this kind of additional money is not available, it is too late and the project is canceled.
4:06:24
White_Flame
vtomole: ideally, the system would measure & detect which GC strategy would be most effective. It can be incredibly difficult to manually predict properly
4:07:29
White_Flame
and different strategies woudl apply to different data, not just 1 GC for all of memory
4:07:48
beach
vtomole: Yes, that would be nice. But we are in the business of free software (which is why we are on freenode) and already implementing a single bug-free GC is hard. Implementing multiple ones, some of which are real-time, is extremely hard, and we don't have the manpower.
4:09:44
White_Flame
GCs also don't stand alone. The readers/writers in the code need to manipulate them in certain ways, obey different views of safepoints, etc
4:09:46
beach
vtomole: There are a few branches of industry that can afford such additional manpower. Weapons, nuclear power, medical equipment. But mostly, it is too expensive.
4:10:35
adlaistevenson
Are there still tradeoffs that make having more than one collector implementation desirable in a language?
4:11:15
beach
vtomole: What White_Flame says. The GC is not standalone. The entire system with its representation of objects etc depends on the GC technique, and vice versa.
4:12:48
White_Flame
adlaistevenson: but azul's (mostly) fully concurrent GC has a lot of good claims there
4:14:21
beach
vtomole: For example, if you want an incremental GC (which you need for real-time or soft-real-time systems), then the compiler must generate code to take into account that objects may move at (almost) any point in time.
4:18:30
White_Flame
come to think of it, symbolics genera let you allocate "regions" which had their own memory configuration. I don't recall if that meant you could have actually different GC strategies per region, or just tweak some preferences per region on the same GC strategy
4:18:33
adlaistevenson
Dijkstra's Tri-Color Garbage Collector looks fine because it has no pauses, that I remember. If pauses are avoided, it is good for real-time systems.
4:19:28
beach
adlaistevenson: We have come a long way since. But even that collector requires collaboration from the mutator for write barriers.
4:20:33
White_Flame
there's GC read/write barriers, assembly-level ordering barriers, and threading synchronization barriers, that at least spring to mind
4:20:43
beach
adlaistevenson: http://dl.acm.org/citation.cfm?id=1134023 is one example of recent techniques.
4:21:55
beach
adlaistevenson: More like "If you attempt to write a reference to a white object into an object that is black, then either revert the black object to gray, or color the write object gray.
4:22:16
beach
adlaistevenson: Because that collector has an invariant that no black object must refer directly to a white object.
4:23:34
adlaistevenson
One problem with GCs I've seen, at least Go's GC, is that if there is too much allocation activity in a short time, the program will run into OS-enforced limit and crash. Garbage collector waits too long :-(
4:24:26
beach
adlaistevenson: That sounds more like a buggy implementation than a problem with a particular GC algorithm.
4:25:00
White_Flame
if you allocate a lot before discarding references to what you allocated, that's a legit out of memory situation...
4:25:25
White_Flame
the one thing that I hate the most about GCs is that most require you to specify a heap size
4:30:03
adlaistevenson
I wonder how people deal with long (multi-minute) GC pauses in production. Do they force the GC at certain times and rotate through the servers, so other instances are guaranteed not to be GCing at same time?
4:31:38
White_Flame
most huge systems are combinations of databases and distributed application servers
4:38:46
aeth
The main thing I could think of that might test a GC in CL would be a game or simulation of sufficient size.
4:39:53
adlaistevenson
I wonder if Swift figured out a workaround to cycles being leaked by ref counting.
4:46:24
aeth
C++ is absolute overkill for modern computers. They use it because they're locked in by proprietary libraries (and engines) written in C++.
4:48:14
aeth
If you're making a PC-only game, though, there's no reason but using existing libraries/engines to use C++, though, afaik.
4:49:20
aeth
Game dev literature often jumps through huge hoops to make C++ do things that languages like CL can do out of the box, like "hotloading", i.e. recompiling something while the program is running so development is faster iirc.
4:50:16
aeth
But if you're writing a game engine in a language with poor interoperability with C++, congratulations, you're now writing a physics engine, too.
4:51:42
aeth
The scripting language gamedevs have almost entirely moved to Lua and JavaScript. Python is not good for gaming. It doesn't embed well (well, it's not sandboxed) and it has some of the worst performance of any programming language (except Pypy, which targets enterprises and has its own issues)
4:52:10
aeth
I think Lua with gamedev and JavaScript with gamedev are, relatively speaking, now much larger than Python for gamedev.
4:52:44
aeth
Python was once taken very seriously. E.g. some major MMO (Eve?) has its backend partially in Python. Civ 4 uses Python (Civ 5 uses Lua, I'm not sure what Civ 6 uses if anything). etc.
4:53:37
aeth
Absolutely speaking, Python gamedev is probably larger than Lisp gamedev because Python is huge, especially with beginners. In terms of % mindshare? It's probably gone way down in the last 10 years.
4:54:54
aeth
Games are probably the only niche in *anything* where Lua is more popular than Python as a scripting language.
9:37:36
Grue``
I need a simple thread pool library that actually works (requirements: 1: must be able to wait until all tasks in the pool are finished, 2: must not use sb-thread:interrupt-thread to send signals to threads)
9:39:58
shka
anyway, I would try lparallel as It is my personal default for everything concurrency related
9:41:16
Grue``
"`end-kernel’ should not be used as a substitute for properly waiting on tasks with `receive-result’ or otherwise." what's the correct way to finish all the tasks then?
9:43:17
p_l
make all tasks have an end (triggered or natural) that gives out a result for receive-result ?