libera/#clasp - IRC Chatlog
Search
14:11:13
drmeister
Over the last couple of years new tools have been developed for creating AST Matchers.
14:13:42
drmeister
This is a compiler explorer prototype that hosts an advanced, future version of clang-query that is excellent:
14:15:39
drmeister
This will help create matchers that we can use to create source-to-source transformations.
14:21:35
drmeister
I'll report back when I've build all of cando - then I'll push it and then you pull it.
14:21:59
drmeister
There is going to be a LOT of changes. I want to avoid merge conflicts - so I was trying to get this done over the weekend.
14:27:02
drmeister
What I'd like to do is take every class field in clasp and cando and apply a consistent naming scheme.
16:10:25
drmeister
It should be completely neutral. It was a test of the STST and gets us closer to standard C++.
16:12:01
drmeister
Source to source translation was something that Google put a lot of work into in the early 2010's and then not much happened after that.
16:12:14
yitzi
Understand...getting ready to start working on the Jupyter debugger for CLASP. SBCL is mostly complete with the exception of breakpoint insertion.
16:12:42
drmeister
I implemented STST for clasp and used it to solve some nasty problems early on and then let it lie fallow.
16:13:50
drmeister
Now I've brought our code back online and did some searching around the interwebs over the weekend and found that there has been some recent work towards fulfilling the promise of STST - but it's still not there.
16:15:09
drmeister
Achieving actual STST requires writing tricky code that does the translation. We can do it in clasp - but others need to write the tricky code in C++ or develop Python hooks or write a domain specific language (ugh).
16:16:11
Bike
yitzi: would breakpoints require compiler stuff? i guess stepping would, it might be nie if we could do that
16:18:00
yitzi
Bike: I am not sure yet. SBCL implements stepping via inserting step-form-conditions via the compiler. I seem to recall CMUCL had breakpoint insertion, but I don't know about SBCL yet.
16:18:31
yitzi
I suppose you could fiddle with the source before parsing and insert (break) ... that seems kind of kludgey
16:20:21
Bike
i've done some limited experimentation with having the compiler insert calls at stereotyped points, and it was actually pretty easy
16:21:55
yitzi
Cool, it may come to that. I checked and yes CMUCL had breakpoint insertion but its not in SBCL.
16:22:54
drmeister
We could put patch points in with a single NOP that can be changed to a BRK. Or we just use the line tables to put in BRK's and keep track of what was there before.
16:23:07
yitzi
To begin with I plan on supporting just what the implementation supports natively. If it doesn't have breakpoint insertion then it won't be available in the debugger.
16:23:42
drmeister
What I need first though is bullet proof source locations and more lexical variables.
16:30:34
yitzi
Awesome. I am implementing debugging in the lisp kernel for Jupyter...if you haven't picked up the context.
16:35:06
yitzi
No. The kernel has it's own process and can technically be on a remote machine. Inside the kernel there are various threads that respond to specific ports. SHELL is where user code runs. CONTROL is where debugging happens. There is also an IOPUB is a broadcaster and some other various thread.
16:37:54
beach
All that information is completely above my head. Anyway, if the Clasp program you are debugging runs in the same process as the debugger, what Allegro does, and what I suggest in the paper is relevant. If not, I guess you can use the same technique as other debuggers use.
16:38:55
beach
The main problem with same-process debugging is that you need to make sure you don't insert a breakpoint that is then tripped by the debugger, or your process will be unusable.
16:39:04
yitzi
Sorry about the terminology. It's a lot simplier than it sounds. Yes, I think your paper is very relavent.
16:41:03
yitzi
Right now the debugger hooks only get activated around user code so that is somewhat lessoned. But you are right if someone set breakpoint on the control thread that could be problem.
16:41:05
beach
Allegro's solution doesn't require different threads, whereas the technique I suggest has the debugger and the code being debugged run in different threads, and I make sure a breakpoint is only tripped if it is in the thread being debugged.
16:42:43
drmeister
I've transformed _Unbound<xxx>() to unbound<xxx> and done the other special values as well.
16:43:26
yitzi
beach: Yep, just scanning your paper. It sounds almost exactly like what I am doing. Using a locked queue to pass messages between the control (debugger) and the shell (the thing being debugged)
16:45:30
beach
I shall have to put that off. Dinner is imminent here, and I'll spend the rest of the evening with my (admittedly small) family.
16:48:04
beach
Oh, I have changed the plan a bit for what technique to use, compared to what is in the paper. I will probably instead use the technique with trampoline snippets that is in the most recent paper.
16:48:41
beach
So if there is no breakpoint, the only cost is an unconditional jump to the following instruction, with the new technique.
18:28:30
yitzi
It seems to track the source path, but I am getting line number 999902 for frames inside those files.
18:33:09
Bike
for one we could probably bind current-source-pos and then you'd get the line number of the top level form, and that wouldn't be too hard
18:38:06
yitzi
I am calling clasp-cleavir::cclasp-eval directly b.c. I have to override core:*eval-with-env-hook* to break into load-source's loop
18:39:15
Bike
it uses cst-from-expression, which doesn't accept a default SPI, which is a little annoying
18:41:44
yitzi
The recent Eclector allows overriding the source query function. I used that to do line/column tracking for the implementations that only return a file position.
19:01:54
Bike
still debating what would be the best thing to do. the information about classes from the scraper and analyzer are different enough that i don't think i want to conflate them, but then having more than one info for the same name would require refactoring a bunch of stuff
19:25:35
Bike
hm, or, what if we used the analyzer info only for "gc managed types", and the scraper for the lisp accessible stuff
20:07:16
yitzi
Bike: what about just reading cst right from the stream with eclector.concrete-syntax-tree:cst-read? Thereby replacing my usage of load-source
20:16:57
Bike
you could do that with uh, (cst:cons (cst:cst-from-expression 'lambda) (cst:cons (cst:cst-from-expression nil) cst-goes-here))) i think
20:39:07
Bike
is this master branch or main? insert-jump-coercion no longer exists in main, i don't think. bir-to-bmir is only a hundred lines
20:49:01
cracauer
I think the git checkout of deploy on the bb was screwed up. Let's see whether a fresh one fixes it.
21:03:45
cracauer
in fact the entire machine is screwed. Oh well. Will be some time until I know whether the assert is caused by this.
23:40:26
drmeister
I just talked with cracauer - the buildbot is an AWS machine - it went away and a new one replaced it but with OLDER software.
23:40:55
drmeister
The problem is not in the clasp source - the problem was the new AWS machine had old software/data.
23:48:08
cracauer
we sometimes get this time warp when the spot instance is replaced, but most of the time we do not.
23:48:33
drmeister
Bike: We might be able to statically analyze out what the LISP_CLASS macro defines.
23:51:23
drmeister
The LISP_CLASS -> scraper tags is easy - it's CPP generated code and the scraper knows how to extract the info.
23:53:32
drmeister
To get the static analyzer to extract the info - I have to expand the CPP macro like: LISP_CLASS(core, ClPkg, HashTable_O, "HashTable",HashTableBase_O);
23:55:38
drmeister
The ClPkg and "HashTable" are strings - I'd have to add them as constants or arguments to a function call. The 'core' - ugh - that's a namespace - I'd have to stringify it with the C preprocessor and then do whatever I did with ClPkg and "HashTable"
23:58:14
drmeister
What you said "hm, or, what if we used the analyzer info only for "gc managed types", and the scraper for the lisp accessible stuff" - that is what we are doing now - that would be easier.
23:59:15
drmeister
Bike: But then there is "what do we need" for the stamps? Do we just need the class name and the base class name listed in the macro?
23:59:47
drmeister
We have the class name - is all we need is the base class name listed in the macro?