freenode/#clasp - IRC Chatlog
Search
21:03:52
kpoeck
I have a question to https://github.com/clasp-developers/clasp/blob/dev/src/core/lispReader.cc#L1204
21:06:48
kpoeck
IT seems that was introduced on https://github.com/clasp-developers/clasp/commit/d375df2b98219e16d4b72a2ef19308ef22413673#diff-efdf9990892e3d7f24ab7788fd1a4701
21:07:48
kpoeck
but looking at the spec it seems counterintuitive that _sym_STARpreserve_whitespace_pSTAR is not used in read#
21:10:17
kpoeck
CLHS says that (read-from-string " 1 3 5" t nil :start 2) should return 3 5 and it does when I change #if 0 to #if 1
21:15:25
kpoeck
And (read-from-string " 1 3 5" t nil :start 2 :preserve-whitespace t) should return 3 4
21:19:05
drmeister
How about we bind _sym_STARpreserve_whitespace_pSTAR to T in cl__read_preserving_whitespace and NIL in cl__read and then in ...
21:20:28
drmeister
if (_sym_STARpreserve_whitespace_pSTAR->symbolValue().isTrue() && !recursiveP) clasp_unread_char(...)
21:22:16
drmeister
in collect_lexemes https://github.com/clasp-developers/clasp/blob/dev/src/core/lispReader.cc#L229
21:23:01
drmeister
It refers _sym_STARpreserve_whitespace_pSTAR is referenced - but there is no way to get recursiveP - does that matter?
21:30:11
kpoeck
_sym_STARsharp_equal_final_tableSTAR looks like a zombie, can't find that this is ever read
21:31:38
drmeister
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/lsp/sharpmacros.lsp#L80
21:35:48
drmeister
Expanding the call here: https://github.com/clasp-developers/clasp/blob/dev/src/core/lispReader.cc#L1057
21:38:31
drmeister
That will lead to the call to T_mv mv = lisp_object_query(sin, eofErrorP, eofValue, recursiveP); with recursiveP as false
21:38:50
drmeister
My computer is wigging out at the moment - too much parallel building going on. Pardon the pauses.
21:55:08
kpoeck
Can you paste your last thought regarding read_lisp_object somewhere so that I can test it?
21:56:12
drmeister
Sure - give me a sec - I'm suffering a laggy computer from my experimenting with hash table sizes in Cleavir.
22:02:52
drmeister
I didn't try compiling it - but it should have the same effect as the previous code - but lisp_object_query can now be called with recursiveP = false
22:30:24
Bike
drmeister: i'm thinking of taking tomorrow off - we don't have anything urgent going on, right?
22:58:30
kpoeck
drmeister: This change seems to work fine. Will create some more regression tests and than send a pr (next days , it is midnight in my timezone)
23:14:04
drmeister
I got the open addressing working with clasp's hash tables - now I'm experimenting with different rehash-size and size arguments for map-instructions-xxx
23:17:45
drmeister
Yeah - that's what I should have done before I brought it up in #sicl earlier today.
23:18:33
drmeister
The open addressed hash tables are faster than the old chained ones. Like 2x faster. And it's easier to control and reason about their size.
23:20:32
drmeister
sbcl has a different kind of hash-table doesn't it? They have some kind of data structure for the keys - am I remembering that correctly?
23:22:14
Bike
you mean add the clasp asts? no. most of them can probably be almost oneliners though, they're for array operations and stufff.
23:22:46
drmeister
We should time it against Clasp's interpreter - Clasp uses alists to look up variables. So your AST interpreter could be faster.
23:25:01
drmeister
I should be able to get some timing soon. I'm rebuilding everything now with logging turned off and working open addressed hash-tables.
1:19:24
drmeister
Damn it - this cost me the last two hours: "rehash-size specifies a minimum amount to increase the size of the hash-table when it becomes full enough to require rehashing; see rehash-theshold below. If rehash-size is an integer, the expected growth rate for the table is additive and the integer is the number of entries to add; if it is a float, the expected growth rate for the table is multiplicative and the float is the
1:19:24
drmeister
ratio of the new size to the old size. As with size, the actual size of the increase might be rounded up." - from make-hash-table
1:20:43
drmeister
Would it have killed them to add another keyword argument :how-to-grow-the-freakin-hash-table
3:59:15
beach
drmeister: So why not try something really simple, use a multiplicative growth instead of trying to guess the initial size.
5:02:19
drmeister
I've switched clasp's hash tables to open addressing - I can control the size now.
5:09:53
drmeister
Anyway - I'm trying to track down why the child threads compiling AST->native code appear to slow down the main thread.
5:18:30
drmeister
I'm having a heck of a time trying to figure out why I can compile the AST's in 0.3 seconds but when I launch children that compile the AST's to native code the AST generation appears to take 10x longer.
6:16:25
drmeister
I'm getting some evidence that my contention problem is the class database - it has an upgradable read/write lock using two mutexes.
6:17:42
drmeister
In the compile-file-parallel - I put a loop around the hir transformations - so each child thread now runs my-hir-transformations (a bunch of hir transformations) 100x. The main thread slows down about 20x.
6:17:54
beach
I guess it can happen when lots of DEFCLASS forms or lots of DEFMETHOD forms are accessed.
6:18:55
drmeister
I sample the process and I see lots and lots of read locks being acquired for the class database. This is within map-instructions-xxx
6:19:28
beach
I have no answer in real time, but I will give it some thought. Maybe some insight could be had by realizing that (setf (find-class...) nil) would be rare, i.e. few things would ever be deleted from the class table.
6:20:29
beach
Monday mornings are crazy around here, and I need to leave. But I'll give it some thought during the day.
6:22:05
drmeister
This looks interesting - reading... https://en.wikipedia.org/wiki/Read-copy-update
6:24:02
drmeister
The other big red flag is now I've got 40 threads grinding out HIR transformations and the process never uses more than %150 CPU
6:34:57
beach
Typically, no classes are added to the database during that phase, so a single-writer-multiple-reader thing would work.
6:53:20
drmeister
I just ran an experiment - I launched 10 threads that all loop and call (find-call 'double-float).
6:58:20
drmeister
I have a multiple reader, single writer lock already - is that the "so a single-writer-multiple-reader thing would work."?
6:59:17
drmeister
i think so - according to this lecture the multiple-reader/single-writer solution is very limited.
7:01:40
drmeister
I'm doing some research here myself - this is a known problem. The linux kernel uses something called Read-Copy-Update to speed up reads at the expense of writes.
7:05:51
drmeister
https://github.com/Bike/SICL/blob/master/Code/Cleavir/HIR-transformations/eliminate-catches.lisp#L7
7:08:42
beach
Maybe one day we will implement typep with constant type descriptors as a generic function, but we haven't done that yet.
7:17:37
drmeister
https://github.com/Bike/SICL/blob/master/Code/Cleavir/Intermediate-representation/map-instructions.lisp#L32
7:21:19
drmeister
Weren't you and Bike talking about this a while ago? Replacing typep calls with predicates?
8:09:47
drmeister
Yeah - the read-many/write-one lock is totally inadequate here. Multiple CPU's trying to grab the read lock is bad. The memory that represents the read lock can only be held by one core at a time and so it bounces between the CPU's.