freenode/#lisp - IRC Chatlog
Search
20:13:46
Younder
I say this in remembrance of Vladimir Voevodsky a brilliant mathematician who died before his time. This is his legacy.
20:15:53
Younder
Homtopy type theory, the univalent theorem and a program at Princeton Institute of Advanced Study for continuing it.
21:20:57
no-defun-allowed
Not in the standard but lparallel and Bordeaux threads are good starting points.
21:21:56
figurelisp
so when common lisp was popular people used these kind of libraries to handle concurrency?
21:24:37
no-defun-allowed
Well, probably not before the AI winter, concurrent machines probably had non-generic designs since those were expensive and new.
21:26:39
pfdietz
There's a common model of concurrency in CLs now, where special variables dynamically bound in a thread are thread local.
21:27:32
pfdietz
And I think there's de facto compatibility of the various thread primitives, at least through a compatibility layer?
21:33:10
figurelisp
I am still in my very early lisp phases. It was just that this question popped in my head
4:05:20
Bike
depends on implementation. the obvious is n², but i think some implementations use hash tables and stuff to reduce it
4:37:36
no-defun-allowed
i put a basic chip8->cl compiler in my emulator and it's 10 times faster, averaging 800 x86 cycles/chip8 cycle
4:38:22
no-defun-allowed
thanks! i'd like to emulate something more difficult but i'm not sure what to do next
4:39:40
no-defun-allowed
(the way it works is i save the dispatched function bodies and i try to get all strings of code where no jumps are done)
4:46:55
no-defun-allowed
it seems a lot of the issues with emulators are C problems, like not having the system compiler (well, maybe not with llvm or libgcc) and having to write half a compiler
4:46:57
beach
I don't need such a thing immediately myself, so I won't be of much use for testing it.
4:48:15
no-defun-allowed
yes, i think that would simplify a lot of emulator writing as you don't have to invent an IR and compiler for it
4:49:11
no-defun-allowed
i read someone's documentation on writing a dynamic recompiler and most of the issues they listed seemed to be "inventing your own compiler" problems?
4:50:28
beach
I do something similar (I think) in my SICL bootstrapping procedure. I take the intermediate code generated by the Cleavir compiler, and I translate it to Common Lisp and compile it with the native compiler.
4:52:23
beach
What I mean to say is that I think your analysis is right that a lot of difficulties in this domain are solved by having the compiler around.
4:52:43
no-defun-allowed
i was reading through https://github.com/marco9999/Dynarec_Guide which goes over doing an emulator/"compiler" in C++
4:57:34
drmeister
Shoot - I went to the trouble of implementing a parallel compile-file and now I'm running into a problem with special variables and quicklisp.
5:02:54
drmeister
I compile-file that form in a child thread - when I load the compile-file'd result it should create the special variable and bind it globally.
5:04:56
beach
So, there should be no compile-time reference to the value in the same file as the DEFVAR form.
5:15:47
drmeister
It might be messing up because I'm hot patching the code and reloading things. I'll wipe everything out and rebuild from scratch.
5:17:34
beach
Does it work as we have discussed, i.e. generate the ASTs sequentially and then work on each AST in parallel?
5:18:04
drmeister
Also - it doesn't fully utilize the machine - I have some mutexes around parts of the compiler that are throttling performance. Compile and discriminating functions drop back to serial.
5:20:00
drmeister
Maybe I can't do anything about the lock around COMPILE because it needs to be serial for AST generation.
5:23:17
drmeister
The parallel compile-file doesn't use all cores - I see maybe a 2x speedup - even though I see like 15 threads for the clasp process.
5:24:48
drmeister
But there are several things that could be throttling performance, my COMPILE lock (which I could get rid of with llvm7) the Boehm GC, llvm malloc.
5:25:58
aeth
drmeister: I suspect parallel compile file in CL won't get you much in general because most people have many small files.
5:28:29
drmeister
Still, we are seeing about 80% of the time being spent in llvm - if there are more than two top level forms then that should all go parallel. I'm a bit mystified.
5:29:16
drmeister
You know what I can do though - I can use dtrace and profile everything. That should illuminate what is locking.
5:30:41
drmeister
ASDF is a test case. With the serial compile-file it takes 190 seconds, with the parallel one it takes 120 seconds.
6:14:38
elderK
It's crazy how like, fundamentally, nothing is really different. But conceptually, wayyyy different. Never returning, always calling our successor. Programs get turned inside out.
6:44:57
aeth
drmeister: It might be best to ask someone who has all of Quicklisp for some large files
6:47:17
drmeister
Well, ASDF works. But I can't get quicklisp to work yet with the parallel compile-file.
6:51:38
drmeister
Yeah - it's an interesting bug. ~/quicklisp/quicklisp/package.lisp - it defines many packages, among them :ql-config.
6:51:56
drmeister
If I load the bitcode file generated by my parallel compile-file - the package is defined.
6:55:43
drmeister
Clearly something is wrong - but the compiler is generating the correct bitcode! But lowering it to object code or linking it into the final fasl is messing up.
7:34:15
dim
if you want to benchmark easily, I guess that (ql:quickload "pgloader") could be a nice test as it loads about 64 systems with the build dependencies ;-)