freenode/#clasp - IRC Chatlog
Search
11:55:16
heisig
Shinmera: That would be great. Do you already have a topic in mind? Your game engine perhaps?
12:07:32
Shinmera
Oh, wait, I already did that to a point. I completely forgot. https://reader.tymoon.eu/article/364
13:28:02
kpoeck
I was not able to complete the ansi-tests with bdw-gc (only by splitting in 2 parts)
13:28:30
kpoeck
The same for mcclim, I think I needed 3 retries to compile all of mcclim and its dependencies
13:28:42
kpoeck
Cl-bench results are in https://github.com/clasp-developers/clasp/wiki/Benchmark-Results
13:29:24
kpoeck
Please note that each Common Lisp implementations may have different optimisation settings, so comparing different implementations might be unfair
13:30:14
kpoeck
And my copy of Lispworks is very old, might not be fair as well (but there is no newer free version of Lispworks)
13:31:59
kpoeck
So it is totally possible that for example, clasp verifies every type declaration in a runtime check and another compiler believes the declaration
13:32:32
kpoeck
But the test between clasp mps and clasp boehm was done with same sources, same wscript.config, so should be fair
13:35:34
kpoeck
I run this with git dev as of commit 01b636f86351390feeb523ecbb33e3a5b6be9252 (november first)
13:36:24
kpoeck
So now I would continue with the mps version, since - at least in my tests - it consumes a lot less memory
13:58:16
drmeister
We need to get type inference working fully with dead code elimination to get the code size down.
13:58:55
drmeister
I had a lot of success this weekend switching from using a shadow stack to keep track of function arguments to using stackmaps
14:10:06
drmeister
kpoeck: That is very encouraging that the mps version works for cl-bench, ansi-tests and compiling mcclim.
14:27:12
drmeister
My best guess at why our code is slower is we have too many type checks and too much dead code.
14:31:11
pfdietz
Type inference is nontrivial when you start using complicated types (and, or, cons). Even SBCL has bugs in this still.
14:38:10
beach
I think we have a better technique than SBCL, but we should think about using the results in Jim Newton's thesis for computing with types, like unions, intersections, etc.
14:38:25
kpoeck
I think an optimize criteria is controlling in clasp whether we have run-time checks for the type
14:39:16
beach
And Clasp might consider using the results of Jim Newton's thesis as well for type computations.
14:45:06
jackdaniel
it is like with this joke: programmer had a problem so he thought "I'll use McCLIM", now he had two problems :)
14:45:09
Quike
and i'm still not sure about the closure aspect. i don't think we can do inference before marking shared variables somehow, since they can just change whenever
14:45:59
beach
Quike: I don't think it matters what Jim is interested in. I think his results should be looked into for the kind of type computations we do in our type inference code, and also for general stuff like TYPECASE, TYPEP, and SUBTYPEP.
14:47:47
Quike
i left my computer on at home and don't want to kick it off, but it's taken up the Bike name.
14:51:34
Quike
anyway, so even if i do value numbering and probably rewrite type inference again, i'll probably get wrong results if i run it before segregate-lexicals
14:58:29
kpoeck_
And compiling mcclim from scratch with mps takes like 6 hours, so i see room for improvement.
15:16:03
Quike
if cleavir treats the type system as a mystery lattice i don't know how newton's stuff would be used there. i mean it might be interesting for sub/typep implementation...
15:29:09
beach
Me and my (admittedly small) family are staying at Novotel, half a block from EPITA where the defense is taking place.
15:37:15
beach
Even though the train between Bordeaux and Paris takes only two hours these days, we are staying the night before and the night after the defense at the hotel. Makes for a short but nice vacation.
18:08:47
drmeister
I figured out how to recover info from stackmaps and answers to all of the questions we had on the weekend.
18:09:16
drmeister
I really appreciate the help you gave on Saturday - that really helped push me in a productive direction.
18:10:41
drmeister
Ok, so here's the poop: the _mh_bundle_header symbol is an internal symbol within the bundle(dylib) it points to the start of the library - but it's troublesome to use it because it isn't available if I load bitcode directly and it won't be available in linux.
18:11:41
drmeister
It was much easier and more portable to use 'dladdr' - with the address of any function in the library the Dl_info struct will return the address of the start of the library.
18:12:38
drmeister
Then on macOS we are kind of on our own. There are the header files and structs in /usr/include/mach-o but there don't appear to be any library functions to work with dylib files. the getsectionaddr is a helper function written to support otool and a few other tools.
18:13:01
drmeister
It was easier to copy and paste the getsectionaddr code into clasp and modify it a bit.
18:14:11
drmeister
I got the LLVM_STACKMAPS section/segment (whatever) and I run it through the parser that I wrote based on the llvm stackmaps documentation.
18:14:54
drmeister
I know a lot more about object files and how to use them with llvm. This will help us with startup time.
18:17:06
drmeister
Well, not so much for libraries - but there is for the ORC JIT. Basically it constructs an object file in memory and then I can subclass the memory manager and capture when it allocates data sections and I just figured out the method to overload that is called when the object file contents have been set up and relocated into memory. Now I can get the __llvm_stackmaps data in its final, relocated form.
18:18:38
drmeister
This has really helped to demystify the underlying object file and dynamic library machinery.
18:46:33
drmeister
I think we are in business now with stackmaps working for fasl files and JITted code.
19:48:31
frgo_
Man, it took me a second look to get what I see. Real backtraces with real info in them. YES!
19:58:22
drmeister
With new stackmaps cclasp bitcode --> 25,113,174 lines of llvm-ir and 147 landingpads
20:03:06
drmeister
The good thing about all the complicated shadow stack and interweaving of the shadow stack with the real stack is that I can still use it all - and it merges interpreter frames with real frames.
21:52:39
drmeister
I haven't pushed to dev yet because I stashed the changes and I'm building with without the changes to see the impact on the number of lines of llvm-ir and landingpads
21:53:13
drmeister
Building cclasp [81 of 439] - as soon as it's done 10-15 min I'll pop the stash and push the changes.
21:54:48
drmeister
At [81 of 439] there are more than 8,000 landingpads - but we knew that it would be high.