freenode/#clasp - IRC Chatlog
Search
21:42:26
Bike
i suppose i'm basically separating that out, so that the local call analysis only analyzes one thing, whether the function being called is in the module
21:42:46
Bike
that's a pretty straightforward thing to do, unlike lowering calls or deciding what to do with a bad call, i think
21:43:15
Bike
at the moment i'll move it into maybe-interpolate in such a way that it should pretty much happen at the same time anyway
21:45:51
no-defun-allowed
I thought I had found some code to rebuild the table, but after building cboehm I couldn't find the build_commands.json file that it needed.
21:46:17
drmeister
no-defun-allowed: I just got through rebuilding the table - testing the build_cmps build now.
21:46:51
drmeister
FYI: You can do this on linux - but there are problems running it on macOS because they use a different version of the C++ std library.
21:48:15
drmeister
You can break it up - but the build_impsprep needs to be run from a clean build because it builds a database of all of the C++ code that analyze_clasp uses. If you interrupt the build_impsprep and then start it up again - you only get a partial database and all heck breaks loose.
21:48:57
drmeister
If you are running this in clasp you get a src/main/clasp_gc.cc file that is ready to go. You can run ./waf build_cmps right away.
21:49:04
Bike
uhh. wait, i can't use closure-call-or-invoke since there's no actual closure. okay, so this isn't completely trivial..
21:49:34
drmeister
If you are running this with cando it will build a src/main/clasp_cando_gc.cc file (I think that's what it's called). It's bad form that we dump a cando file in the clasp tree - but I haven't had time to clean this up.
21:50:11
no-defun-allowed
I vaguely recall there being an error related to the values used as EQ hashes (and I forgot the name) that would prevent cmps from compiling; the setter tries to return a value when setting with AWL and it casts the LHS which G++ doesn't like without AWL.
21:50:59
no-defun-allowed
But then I replaced my clasp/ directory with a new one in the end, so it might be leftover from something.
21:57:27
Bike
shoudl shelve this for a bit i think. thought it would be easier than mv-local-call but they'll both have this issue
21:57:47
Bike
having another entry point wouldn't be a win really, since it would take varargs and thus not be tail callable
22:17:22
Bike
but no, i suppose this entry point could be simpler, since the number of arguments is known (for calls, not mv-calls)
22:19:21
drmeister
no-defun-allowed: Right now I'd like to reevaluate problems in light of the new SharedMutex implementation. SharedMutex was broken - and now it should work. Maybe some of the weird bugs that occasionally popped up were due to that.
22:21:35
drmeister
This new SharedMutex might be useful for MPS. I believe MPS suffers in its multithreaded performance because of false sharing.
22:22:14
drmeister
I should run some tests comparing our old SharedMutex (after fixing it) and the new SharedMutex.
22:23:46
Bike
so i could probably do this but it would mean returning to cmp/arguments.lsp, and i don't wanna
22:34:25
drmeister
Note: This is for clasp - not for cando - I'll start the static analyzer for that now.
23:19:23
karlosz
Bike: it would be nice if we could make calls to &key with constant keys local calls but that would entail messing with cmp/arguments.lsp a lot right?
23:20:07
karlosz
also it would be nice if we could have a lighter weight entry point mechanism, cause a full llvm function with a full description and everything tail calling in is still some amount of overhead
23:20:56
karlosz
er, i take back "messing with cmp/arguments.lsp". it would just mean we need to parse lambda lists harder :/
23:27:15
Bike
if all keys are constant and the rest argument is not used it could just use the main entry tho
23:31:29
karlosz
i'm sure some of the llvm stuff can be trimmed off the XEP, i just didn't experiment with that because i don't know what all the knobs do
23:31:44
karlosz
it might help with the backtraces if we can squash the main entry frame with the XEP frame
0:00:09
Bike
i have local changes to make the xep's call of the main function use fastcc, so i guess it could just be marked tail and maybe that would improve the backtraces
0:10:15
karlosz
note: can substantially simplify local and extend to &key if we changed the actual function being local called to to only have required arguments
1:52:28
no-defun-allowed
cclasp-mps is shouting "../../include/clasp/mps/code/shield.c:675: MPS ASSERTION FAILED: shield->unsynced == 0" (and shield->depth == 0) at me.
1:57:27
no-defun-allowed
Oh, it also starts with "../../include/clasp/mps/code/poolawl.c:943: MPS ASSERTION FAILED: AddrIsAligned(objectLimit, PoolAlignment(pool))" and "Handle weak_obj_skip other weak kind -585575144" which I guess is bogus.
3:04:45
no-defun-allowed
I tried to load Quicklisp after building cmps; I'm starting to think that might not be doable yet?
3:06:45
drmeister
Could you paste your wscript.config file? You may have old settings or you don't have one and the defaults are active - that shouldn't break things - but I have a wscript.config that I've been working over for a while.
3:07:24
drmeister
I haven't tested the mps build much. I'm running the static analyzer for cando and then I'll test it with that.
3:07:49
drmeister
Do you have any ideas what might be different? Did you do anything different before when it broke vs now?
3:08:51
no-defun-allowed
I replaced everything in clasp/ to build it today; my wscript.config is wscript.config.template with the Arch-specific changes (LLVM_CONFIG_BINARY = '/opt/llvm90/bin/llvm-config') and I removed "--link-static" from the llvm_libraries line in wscript
3:10:09
no-defun-allowed
Now I'm compiling some of decentralise2 and it's still behaving well. cffi and flexi-streams have the most dots out of the dependencies.
3:10:51
no-defun-allowed
(ql:quickload dots are a good metric of code size, right? The more dots you have, the more code there is.)
3:14:33
no-defun-allowed
It does a thing with *macroexpand-hook*; but in any case everything compiled fine.
3:17:00
no-defun-allowed
Then I ran Clasp under SLIME; no problems. Then ran my tests again; also no problems. Then, for kicks I evaluated (trivial-garbage:gc :full t) and it didn't like that.
3:29:11
drmeister
I'll have to take a look at it. Note: I just got it back up and running today after several weeks of not using it.
3:33:43
no-defun-allowed
That is my networking/distributed hash table library. And yeah, that is from the latest commit.
3:34:56
no-defun-allowed
It looks like something got messed up in the AWL pool, but I can't tell immediately what uses weak references.
3:36:07
drmeister
The AWL pool is used for a couple of other things as well - but I think there is a nasty problem in there.
3:36:32
drmeister
The AWL pool doesn't support interior pointers and I've been using it as if it did for static vectors.
3:42:28
no-defun-allowed
I had the same error while running the tests, which I suppose invoked a GC.
4:00:21
drmeister
A preliminary test tells me that the new SharedMutex is 5.5x faster than the old one.
4:03:50
drmeister
I evaluate this with the old SharedMutex and again with the new one (I have them built side by side).
4:05:14
drmeister
With the old SharedMutex it takes 1.65 seconds and with the new one it takes 1.713 seconds - so the new one is just a bit slower than the old one in this case.
4:10:49
drmeister
Yeah - the new one scales much better with multiple threads. I'm very impressed by the author of that code.
4:11:32
drmeister
(time (many-find-packages 20 10000000)) Old SharedMutex 80.433secs. New SharedMutex 10.965secs
4:35:39
drmeister
MPS doesn't use a shared mutex - that's would explain why it scales so poorly with multiple threads.
4:38:45
no-defun-allowed
Are there many places where it only reads? I have barely studied the MPS though.
4:43:07
drmeister
Once the allocators run out of their current buffer they call a function that grabs new pages and that has a lock.
4:43:58
drmeister
When you allocate a lot - and lisp allocates a lot - threads spend a lot of time waiting for the lock.
4:45:00
no-defun-allowed
Sure, but do any read without writing or locking everything? If not, you'd just be exercising the writer lock.
4:51:38
drmeister
I don't know what they do. I'm sure it could be done better. Java has a multithreaded GC I believe.
4:54:40
no-defun-allowed
This reminds me that I had MPS running on ARM on Linux, but then forgot about it and lost it when reimaging SD cards.
6:27:18
drmeister
Time real(20.793 secs) run(20.792 secs) consed(1373407488 bytes) interps(191) unwinds(0)
6:27:30
drmeister
Time real(51.268 secs) run(51.269 secs) consed(1784819792 bytes) interps(191) unwinds(0)