freenode/#clasp - IRC Chatlog
Search
11:07:38
Colleen
selwyn: drmeister said 22 hours, 59 minutes ago: Check the log above this notification - I describe my attempt to use the gdb/jit interface.
11:11:01
drmeister
If you hook gdb into a running clasp, gdb is supposed to put a breakpoint on this function:
11:11:15
drmeister
https://github.com/clasp-developers/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L3475
11:11:59
drmeister
I tried examining memory but I don't see a breakpoint - but gdb is probably putting it on only when we aren't looking at it.
11:13:17
drmeister
Huh - something just occurred to me. I wonder if after the break gdb continues through the function. We could put some code in there that reads the first bytes of the function and prints them out. At that point the breakpoint/interrupt must be in place.
11:14:14
drmeister
I'd like to know if gdb is actually being invoked when that function gets entered.
11:34:38
dmiles
i was lookjng that this https://drmeister.wordpress.com/2015/07/30/timing-data-comparing-cclasp-to-c-sbcl-and-python/
11:36:04
dmiles
well i am sure i could always compile my lisp code if this is a concern to me.. that is teh differnce between interpred vs compiled?
12:08:44
selwyn
if you are interested in more recent benchmarks: https://github.com/clasp-developers/clasp/wiki/Benchmark-Results the rightmost column corresponds to fairly recent clasp
12:11:40
selwyn
scroll all the way to the right of the table - there are some columns which you may not be seeing
12:15:44
selwyn
these are subject to change fairly often though. you might like to follow the instructions and run cl-bench yourself
13:13:50
drmeister
dmiles: Hi - the performance of Clasp is about 0.5x to 0.1x of SBCL. We know what the bottlenecks are and there is no reason why we can't get parity with SBCL once we start eliminating useless type checks and code branches. Clasp is doing lots of inlining now and that is how you get performance.
13:22:36
dmiles
Example of inlining at the Lisp AST.. http://dwim.hu/darcsweb/darcsweb.cgi?r=HEAD%20hu.dwim.partial-eval;a=headblob;f=/source/partial-eval.lisp
13:22:48
drmeister
Three kinds of inlining. The lisp AST, we have inlining in the HIR and llvm does inlining.
13:23:31
drmeister
Clasp uses link time optimization in a limited way now - and will support it better in the future.
13:24:42
drmeister
Yes - it's all graphs of nodes and edges. It's all Robert Strand's Cleavir compiler.
13:24:54
dmiles
Well i9 mean is the HIR inliner written mostly in lisp as transformations of someting like cones?
13:27:59
dmiles
(doing otherwise would be unnatural ! .. but other languages do unnatural things like build whole wierd class structure arround IRs)
13:29:38
dmiles
So i assume pretty much the answer is pretty much Yes.. the only reason it would be "No" is if you built LLVM isntead of HIR
13:32:09
dmiles
even if the HIR stage is 1-to-1 with LLVM .. you still have a place you can done brilliant things before making LLVM do the rest
13:34:09
drmeister
Cleavir uses Common Lisp classes for all of the AST nodes and HIR classes and makes extensive use of generic function dispatch.
13:36:03
drmeister
dmiles: Clasp has two compilers, the production compiler is Cleavir and it has a full AST and HIR representation. The bootstrapping compiler goes straight from s-expressions to llvm-ir - but it's only used for bootstrapping and generating discriminating functions for generic function dispatch.
13:38:05
drmeister
The lowering of HIR is one HIR to several llvm-ir instructions and then llvm optimizations go to town on optimizing the llvm-ir.
13:38:48
dmiles
Ah, i see, that is pretty awesome (that you didnt settle with just the one compiler)
13:40:40
drmeister
I have to warn you - the speed of the overall compilation is still a big issue. llvm is a slow compiler and currently dominates with between 50% and 90% of compilation time spent in llvm. We have gone parallel in the build system (fork and threads) to bring the build time down.
13:43:50
dmiles
people tend to forgive compiler time overheads like that :) .. You cant realy fix that
13:45:57
drmeister
Right - my concern is the performance of the final code and that we maintain interoperability with C++ and that the compiler be understandable so that we can get more people involved in improving the compiler.
13:53:17
dmiles
i have a bunch of closed source lisp code that got translated to java that i am just now translating to c++.. i wont be able to fix _that_ code but it'll be nice to use Clasp to host that C++
13:55:29
dmiles
so when users are writting expansion modules (expansions to CYC) they can write in common lisp that will be called by CYC
13:57:53
dmiles
It is a bit like STELLA, if you've heard of it (STELLA is a [common-]lisp to java/c++ translator) but it wont do as good as Clasp in think in many cases
14:04:35
drmeister
We can reduce the overhead to the absolute minimum. We use C++ exceptions handling to unwind the CL stack. So C++ stack frames interleave with CL frames and RAII works fine.
14:05:46
drmeister
selwyn: I cloned the gdb source code. I found the jit interface. We can instrument it.
14:26:44
drmeister
selwyn: No problem - I'm just debriefing you on what I've been up to in this area.
14:27:16
drmeister
Instrument as in stick printf statements into the gdb code to see if it's triggering when we expect it to trigger.
14:46:19
drmeister
No - it doesn't look like it's a command line option - it looks like it's a gdb command.
15:13:50
Bike
okay, so what i get in the log is it tries to calculate a dispatch function, and then it prints the " |" spacer a million times.
15:14:38
drmeister
It may be an infinite recursive loop where printing is invoking a generic function and so on.
15:15:44
drmeister
What is it trying to print? There is a _safe_rep_(whatever) facility and (core:safe-repr whatever) to print objects using a path that doesn't use any generic functions.
15:16:00
Bike
i don't know. the function it tries to get a dispatcher for is generic-function-name, though.
15:17:09
Bike
dispatch-miss does call generic-function-name in the logger... guess i can change that
15:18:26
drmeister
Yes - I started developing this "safe-repr[esentation]" code path so we can print things without invoking the Common Lisp printer.
15:19:04
drmeister
The code is in debugger.cc - you can add more printing code - just don't use the Common Lisp printer.
17:17:30
Bike
safe-repr is so safe that they only information it prints about instances is their address
17:50:33
drmeister
Yes - you can add more printing code. I think of it as c++ to crawl into objects to print what we need in a completely safe way.
17:51:56
drmeister
I got asked to come up with a proposal by tomorrow for a couple million dollars for a couple of years.
18:06:55
drmeister
Put lots of logging into your interpreter - print everything. Look at my dtree interpreter. I didn’t put that in because I thought it was pretty
18:30:12
Bike
okay, so i have DEBUG_MONITOR in the options, and then i can use monitor_file and stuff?
18:55:22
drmeister
#define DTLOG(x) if (_sym_STARdebug_dtree_interpreterSTAR.boundp()&&_sym_STARdebug_dtree_interpreterSTAR->symbolValue().notnilp()) { FILE* fout= monitor_file("dtree-interp"); fprintf( fout, "%s", (x).str().c_str()); fflush(fout); }
18:57:35
drmeister
The monitor_file("whatever") will return a FILE* pointer (it will open the file if it doesn't exist and close it when the thread exists). The file that is opened by monitor_file("whatever") will be in the /tmp/clasp-log-<pid>/whatever-<thread-id> file.
18:59:01
drmeister
If you want to correlate entries from one file to another print (core:next-number) - it just returns a new number that incremented using a 64bit atomic counter.
19:00:27
drmeister
I had my dtree interpreter print (core:next-number) and when there was a dispatch miss the dtree compiler code printed the (core:next-number) and then I just matched them up. One is after the other.
19:23:18
Bike
looks like a shifted/unshifted stamps problem, but then i don't understand how it could be working in my tests
19:25:08
drmeister
Wow - that's just like my weekend. You see why I was not over the moon about redoing this. I do like the look of your interpreter though.
19:27:32
Bike
i just don't get how it can possibly be working when i load it in an existing image, is all
21:08:56
Bike
prededed by T_sp tinstance = args->next_arg(); Instance_sp instance((gc::Tagged)tinstance);
21:10:29
drmeister
That's still consistent with dereferencing being the point of failure. I think your instance tagged pointer is screwed up. Can you put a print statement in front of T_sp value = instance->instanceRef(index); ?
21:14:16
drmeister
My connection sucks - if I disappear it's because my electromagnetic waves don't want to propagate.
21:14:57
Bike
also, args is a vaslist - just one passed to the function, but dispatch_args is another vaslist copied from it
21:15:10
drmeister
If you push this to a branch I could take a look at it. I've got mad debugging skillz after this weekend.
21:16:35
drmeister
Why? No idea yet. But you are mucking around with the lowest level of the runtime.
21:17:38
drmeister
You could put an ASSERT(instance.generalp()); in front of the instance->instanceRef(index); Then turn on DEBUG_ASSERT - you should probably build with that anyway.
21:25:10
Bike
now it's segfaulting while trying to print the pointer. i think it's trying to actually print the object. does %p not work in bformat?