freenode/#clim - IRC Chatlog
Search
4:40:36
beach
What I mean is, if commonly used toolkits allow something useful that is hard to do with CLIM/McCLIM, we should think about ways to make that easier.
4:41:08
loke
beach: it's easy enough to build something ad-hoc that would make threads work for a specific use-case (and it may very well be that I have to do that in climaxima).
4:41:38
loke
I will be abelt o do that, since if it's anything that I have a lot of experience in, it's how to design sold multithreaded applications.
4:42:29
beach
Maybe YOU are the one I should consult about a design decision that I am contemplating in SICL?
4:42:39
loke
But most people don't know all the subtlities. Multitghreadsing is hard if you don't think about it the right way, and making a previously single-threaded application MT-aware is even harder.
4:44:15
beach
The SICL garbage collector has two generations. The nursery is per-thread, and as a first approximation I was thinking that the nursery collector should be run by the application thread itself.
4:44:37
beach
The global generation should be able to run in one or more separate threads, concurrently with the application threads.
4:45:34
beach
The application threads checks a flag at safe points, run their GC, and indicates to the global one that they are done.
4:45:58
beach
Then the global GC runs. While this is happening, we "allocate black" in the global collector.
4:46:58
beach
Not a big deal. I can have a separate thread, maybe one of the ones in the global collector, run on behalf of an application thread.
4:48:02
beach
You don't have to solve the problem in real time, but I would appreciate if you would think about it.
4:50:39
beach
Because that's the only way for the application threads to indicate any reference from a nursery to the heap of the global collector.
4:54:46
loke
Yes, but if I understood this correctly, then a global GC would be a very disruptive thing. Not only do you have to block all threads while running the GC itself, you also need to send a message to all threads, asking them do a collection and wait for that to finish.
4:55:54
beach
The nursery collection is designed to be fast, so that's not disruptive either. Like a few ms at most.
4:57:13
loke
beach: how do you global GC without blocking anything? What happens if a thread wants to follow a pointer to an object in the global heap while that object is being moved?
4:57:22
beach
Seen from an application thread, at safe points a flag is tested. If it is set, then run a very quick local GC, and set another flag to indicate that you are done.
4:59:07
beach
Yes, I want to make it useful, but simple too, given our limited manpower for maintenance.
4:59:40
loke
So your issue is how to use a separate thread to GC a nursery while its thread is blocked?
4:59:43
beach
Another key decision: there are no pointers from the global heap to the nurseries, nor from one nursery to another.
5:00:37
beach
Or possibly to always use a separate thread to GC the nursery, synchronizing with the application.
5:01:09
beach
I don't know, but a thread could set a flag before blocking and clearing it when it is done.
5:05:36
loke
Instead of making it a flag, let it be a multi-state variable. This variable will then be updated by the special gc-thread to indicate that it is, and have finished a side-channel GC. [at this point I had typed a lot, and then I realised that I had missed one aspect]
5:06:23
loke
In any case, the thread would check, after coming out of the block whether a GC is in progress and if so, wait for it to finish.
5:07:27
beach
So you are thinking a separate thread for a nursery GC as opposed to having it done by the application thread?
5:08:50
beach
Then I see some micro-synchronization problem. How does the global GC test that the thread is blocked in a safe way that does not introduce race conditions.
5:09:41
loke
(well, on Intel at least. On SPARC you'd need a MBAR instruction, if I recall correctly)
5:10:23
beach
loke: Thanks. I will write think about this and ask you more questions and/or write it down.
5:10:49
loke
so you'd just CAS, setting the gc-in-progress flag. Then you'll check what the old value was. If the old value was "normal" you'd just ask the thread to GC. Otherwise you'll fire up the spearare GC thread.
5:12:23
loke
When a thread goes into blocking, it would try to set the state to “block”, which will fail because the state is not “normal” anymore.
5:13:36
loke
In other word, this flag would communicate not just the state (running/blocked) but also the fact that there is an ouytstanding request to GC
5:15:09
loke
When the global gc sends the request to GC, the gc will presumably start very shortly thereafter (measured in microseconds I guess?) which mean you might want to use a spinlock instead of going to sleep while waiting for it.
5:17:53
loke
But I've seen implementations that are quite clever, dropping out of the spinning if they want too long etc.
5:18:23
loke
There is some certain dark magic invoklved in finding the correct balance between spinnign and sleeping. I'm sure there are papers written about it.
5:19:30
loke
OK, this ycombinator post suggests that CAS is indeed enforing a memory barrier on Intel
5:20:00
loke
Apparently ARM does not, so if you intend to target that architecture, you need to manually do it.
5:22:50
loke
Or you could do something really clever, that allows you to write Lisp code that doesn't care about the memory model at all, with synchronisation happening autoamtically, again so that a Lisp porgrammer would see the “logical” behaviour.
5:23:41
loke
I have no idea how feasibile that is. But the alternative is equally annoying. People simply do not understand threads and cache behaviour, and everybody gets it wrong.
5:23:46
beach
Most activity is going to be in the application threads, and objects in the nursery are not shared, so I think it should not be a problem.
5:24:44
loke
beach: Still, whenevr you modify data in the global heap, something as simple as a SETF, you're hgoing to have ordering issues.
5:25:18
loke
People will be confused if you have code that does (setf (aref x 0) 'a) (setf (aref x 1) 'b) and seeing 1 set to B, but 0 not set to A
5:30:15
loke
Well, I don't know how feasibile it is... but if two cores share cache, then you don' tneed to use a barrier
5:32:32
loke
I was thinking up these crazy schemes where you track which core has what parts of the heap potentially in cache, and schedule execution on those cores to avoid the barriers.
5:46:04
loke
I'm heading out to lunch now. but I'm leaving you guys with a question that I don't know the best way to solve
5:47:02
loke
Since I'm using a MENU-FRAME to draw my completions in, I can't use :INDREMENTAL-REDISPLAY
5:47:41
loke
Is there some function I can call to force an incremental redisplay of a pane, even though it's not actually living in an APPLCIATION-FRAME?
5:48:09
loke
Otherwise, I'll ahve to essentially reimplement the functionality of incremental redisplay by managing the output records manually.
5:48:17
beach
incremental redisplay is useful only if you have a large number of output records, like in an application pane.
5:49:15
beach
It preserves the sub-trees of the output-record hierarchy that are the same between two calls to redisplay.
5:50:03
loke
beach: I don't think it'll be too slow, but if I do a full redisplay, the frame will scroll and I'll get flickering, won't I?
5:51:30
loke
I'm having issues with the fact that text inside output records will generate an output record with the height of the font itself (and not its content)
5:52:44
loke
HOWEVER, if you pile on a bunch of combining marks, you can end up with an output record that contains text, but where the text extends outside the theout record bounds.
5:53:31
loke
I'm not sure there is a solution to this, and even though I've heavily used the freetype font engine for a while now, I have very rarely come across the issue.
5:56:10
loke
I've experimented with changing CLIM to make the bounds of the text be the actual text, rather than the font dimensions, and it seems to work but I didn't test much, and JD's point is valid. I can definitely think of cases where things may get weird.
5:57:05
loke
I'm not currently suggesting that anything be changed, but it's an observation I wanted to share.
10:33:41
loke
If I run it and type “Pane” to open the test window, I get a list of 10 elements and I can pick oine using the cursor keys. All good
10:34:28
loke
But... If I change the code (line 150) to create 22 lines or more, nothing is ever rnedered in the window.
11:23:44
beach
loke: I shall have to look at it later. My wife wants me to go with her to pick up a new car she bought.
13:55:56
beach
loke: I wrote to my Swedish MEPs about the copyright vote tomorrow and the day after (to ask them to vote against article 13). I already got answers from 4 of them. I am very impressed.
13:56:48
beach
I have never in my life felt so close to my representatives as I now do with the EU parliament.
14:06:16
beach
loke: I would need more time to figure that one out. FRAME-MANAGER-MENU-CHOOSE is unknown to me.