freenode/#mezzano - IRC Chatlog
Search
19:22:54
fittestbits__
A disk cache keeps a collection of disk blocks that have been read or written to the disk.
19:23:59
fittestbits__
Buffering means using a buffer to hold a portion of a file stream for reading and writing and the saving/refilling as required.
19:26:15
fittestbits__
Hmm. For a disk cache, maybe you want to have a single one for the whole system? Not sure what level makes sense. Are you thinking wired buffers or lisp arrays?
19:33:32
fittestbits__
Maybe something generic would be good, sort of analogous to hash tables - make-cache, (setf (get-cache-item cache key) value), (get-cache-item cache key). It keeps track of frequency of access so that it can determine which items to free.
19:34:48
fittestbits__
Maybe the cache contains a fixed number of items, when a new one is added, a function is called with an old item that should be freed.
19:35:33
fittestbits__
Then different disk drivers could have different parameters and cache different things - wired buffers, or lisp arrays or ...
19:37:32
fittestbits__
A list implementation might be better with newer ones at the front, move item to front when referenced, last item in list would be "least recently used" LRU item. The one to free when needed.
19:39:25
fittestbits__
This could be a problem with wired memory. And definitely a problem for the USB driver - the memory that the USB controller can access is managed manually, not by GC.
19:40:50
fittestbits__
So, if GC collects something in the cache for USB, somehow the wired memory associated with it must be freed. I don't know how the wired memory for the supervisor disk driver is managed.
19:50:43
fittestbits__
Looks like wired memory is collected by GC, so for supervisor disk driver, caching can use wired arrays and have them collected by GC.
19:52:31
fittestbits__
But the USB driver doesn't work that way. It allocates a chunk of physical memory, and than manually manages allocating and freeing buffers in that chunk of memory.
19:57:56
froggey
the GC will collect objects allocated in wired memory, but allocating physical memory directly is very much outside the GC's domain and it won't touch that
19:58:45
froggey
it can be made aware of the memory (to a limited extent) by using a finalizer tied to a proxy object to free the memory. this is how thread stacks get freed at the moment
20:00:51
froggey
no, it can be scattered around anywhere and isn't guaranteed to by contigious in physical memory
20:03:56
fittestbits__
OK that's a problem for USB. However, the finalizer could work. My buffers are arrays whose headers are lisp objects, but point to physical memory buffers. When GC "frees" one of them, the finalize could call (or be) my buffer-free routine.
20:24:39
froggey
on second thought... using a finalizer is a bad idea in this case. memory allocated with allocate-physical-memory (or whatever it is) isn't accounted properly by the GC or the pager and it can't be swapped out