libera/#clasp - IRC Chatlog
Search
11:59:22
lxsameer
hey folks, I read an old reddit post about Clasp using both MPS and boehm GC, does that apply to clasp today as well?
12:00:39
jackdaniel
I think that it is mostly boehm; but please wait for clasp devs, I may remember wrong
12:02:42
yitzi
We use Boehm. There is an MPS build, but it is really only used for static analysis in preparation for the Boehm build.
12:59:19
yitzi
lxsameer: drmeister, Bike or kpoeck may be able to answer your question. They know more about the details of GC than I do.
13:05:24
lxsameer
Bike: and on the scale of 0 - 10, 0 being not all and 10 being super happy, how happy are you with boehm? any issue so far?
13:07:29
yitzi
Bike: I haven't tried #1349 in gdb yet. I did answer your question about the logical hosts in the PR
13:10:23
Bike
could we use SYS for all of them by giving it some top level "directories" with specific translations?
13:13:16
yitzi
I don't think so, unfortunately. We need them to make clasp work in a build system and in an installed system also. Plus the location of lib + share folders is not always the same on all systems.
13:14:18
yitzi
I do think there is more simplifications possible in the bundle logic, though. At a minimum I think we could delete the bitcode logical host.
13:20:18
Bike
in the build system we can technically not use SYS at all since it's reserved for the implementation, no?
13:22:58
Bike
as for boehm, i'm not really confident in scoring anything without doing more in depth analysis than i've done
13:40:28
kpoeck
lxsameer: drmeister is also looking at mmtk (https://www.mmtk.io/), but they don't seem to be quite there yet
13:44:32
Bike
yitzi: does your change entail slime changes or anything? i'm not able to M-. anything, although the pathnames look correct...
13:44:54
yitzi
Bike: While koga is running it creates a GENERATED and a FASL logical host in the the "host" implementation, i.e. SBCL. It also uses SBCL's SYS hosts. It does not actually use the translation tables of SBCL since it never calls translate-logical-pathname. It only uses these hosts so it can print logical pathnames into the various build files. Currently, koga doesn't use logical pathnames to locate build files.
13:45:20
yitzi
Once iclasp is built then we are in "clasp" so the logical hosts SYS, GENERATED, etc all work.
13:46:28
Bike
so we "use" logical paths in koga but don't actually put in any pathname translations?
13:48:19
yitzi
Exactly. There are lot of "root" directories in koga and logical pathnames would probably be a PITA there. Instead I use relative pathnames and and plist of the roots.
13:51:06
Bike
yeah, swank is having problems here. should ext:source-location apply logical pathname translations do you think, or should that be the responsibility of the user (e.g. swank)
13:52:32
Bike
(sb-introspect:definition-source-pathname (sb-introspect:find-definition-source #'cons)) => #p"SYS:SRC;CODE;LIST.LISP" which isn't too different from clasp
13:52:34
yitzi
Especially since you don't actually need to translate-logical-pathname while you are still in CL. Only when you are sending the pathname to someone else.
13:54:52
Bike
ok, with translate-logical-pathname in swank/clasp??translate-location everything works again. super.
13:55:42
yitzi
I think this probably needs a translate https://github.com/slime/slime/blob/9005cdaac4c0adaa8e26ee5285c7b155762c0ce5/swank/clasp.lisp#L463
13:56:17
yitzi
As does this https://github.com/slime/slime/blob/9005cdaac4c0adaa8e26ee5285c7b155762c0ce5/swank/clasp.lisp#L524
13:57:53
Bike
translate-logical-pathname just returns any physical pathname it's given, so if we change swank it should still work with old clasps, right?
13:59:58
Bike
i'm not sure how those should work, though... i'm not sure if gdb/lldb actually probe the source pathnames to get sources or what
14:00:31
Bike
it seems like they shouldn't be limited to absolute pathnames for the same reasons those wouldn't work for lisp
14:09:49
Bike
gdb does understand source locations for something i compile-file and load, but not for something within clasp itself. i guess we still use absolute pathnames for the former?
14:12:18
Bike
the path gdb tries for clasp-cleavir::recurse is "SYS:SRC;LISP;KERNEL;CLEAVIR;TOPLEVEL.LISP" which is the right one, at least
14:15:11
Bike
based on what i've seen of dwarf so far i expect using our own metadata will be a huge poorly supported pain in the ass, and honestly i think it might be fine to just forget about gdb/lldb and merge this, but i'd like to hear what drmeister thinks
14:15:21
yitzi
I am looking at what other CL implementations do with logical hosts, for the heck of it.
14:22:37
Bike
gdb has a substitute-path thing to rewrite source locations, but i don't think it's sophisticated enough to do logical pathname translation
14:27:37
Bike
(incidentally, the fact sldb v works means we can round trip logical pathnames into dwarf and back out to lisp no problem, so that's neat)
14:28:41
yitzi
Ahh. That seems like it is problem independent of logical pathnames. If we put absolute pathnames into dwarf then we will only be able to debug binaries when they are still in the development tree.
14:29:37
Bike
somewhat, but gdb does have a facility so that relative pathnames in the debug info can be resolved https://sourceware.org/gdb/onlinedocs/gdb/Source-Path.html
14:33:49
yitzi
Well, we could still put something that gdb understands (absolute or relative) into the DIFile directory + filename slots and put our own logical pathname in a different metavalue slot.
14:38:22
yitzi
Yeah, I don't think you can glob in substitute-path, so we would have to add every source file explicitly.
14:46:55
Bike
i tried putting in a substitution for a directory and it did nothing. i suspect it tries breaking up paths by / before substituting
14:53:44
yitzi
So either the paths don't work with GDB or we do the extra metadata approach. Does that seem right?
14:58:47
yitzi
Ok. I am building right now. I was on a different branch. I need to get better with gdb apparently. Would you mind sending at least what you had to do to get to seeing the clasp-cleavir::recurse path? Maybe I'll try the enhanced metadata on a different branch.
14:59:29
Bike
i started clasp, connected to it with gdb (gdb -p pid), put a breakpoint on cl__error, and then had clasp call a function that called error
15:00:12
Bike
the alternate metadata thing is going to be pretty involved if it works at all. you'd need to rewrite at least some of how we collect backtrace information since we couldn't use DIContext:::getLineInfoForAddress as we do now, and i'm not sure how to put extra information in the DIBuilder
15:02:33
yitzi
Yeah, I see that. No matter what way we do it I'd like to make this PR doable. I can make a video showing this, but it makes viewing source code in the Jupyter debugger really slick.
15:07:53
Bike
well, like i said, i'm honestly pretty inclined to forget about gdb/lldb in this area. i mean it's not even all lisp code it would be flubbing on. but drmeister is down in gdb more often than i am so maybe he thinks differently
15:35:56
Bike
looking at the file, for example, it has wrapped_cl__error_T_spList_sp but not just cl__error, so I can't jump to that definition
15:39:49
yitzi
I don't think our source references are completely working in direct-calls. When you start stepping into those functions they report `GENERATED:CLWRAPPERS.LISP` not the c code locations.
15:41:20
Bike
i'm not totally familiar with ninja, but looking at the build.ninja file, the "build ../TAGS:" only lists some generated headers and not most of the sources
15:44:49
Bike
the stepper does its own source location dumping rather than work through dwarf, so it might be something with that rather than direct calls per se
15:45:49
Bike
or, no, maybe i don't understand... all the stepper dumps is the source form, not a location. the locations would be coming from the backtrace/dwarf stuff i think
15:46:47
yitzi
Can you try replacing all occurences of `:etags` in src/koga/target-sources.lisp with `:tags` and rerunning koga then `ninja -C build tags`>
15:49:21
Bike
build.ninja now has a lot more files, but M-. is still not working and i still don't see cl__error or cl__read for example in the TAGS
15:50:25
Bike
which is pretty weird, since i see src/core/primitives.cc in the list of files passed to tags
15:51:10
yitzi
I think that just updating in direct-calls via core:set-source-pos-info is not enough. I think the backtrace is reporting the compilation info from the `defun` in the macro expansion.
15:52:27
yitzi
In respect to TAGS: This could be the CTAGS issue. etags gets confused about some of the function declarations.
15:52:36
Bike
yeah, right, so there's like two different things here. there's the source info we store in lisp objects, which is what ext:source-location and set-source-pos-info and stuff deal with. then there's the debug info that's compiled into the fasls, which the compiler does
15:53:06
Bike
those two systems are in some respects independent, and set-source-pos-info is not going to affect the compiler
15:54:57
Bike
when looking through frames, i think it would make sense to preserve the location in GENERATED in some circumstances, like if we're dealing with the lisp separately from the underlying c++ code
15:59:03
yitzi
I dont think it is super critical either way. Its mostly just that the reference to clwrappers isnt very useful.
16:10:02
Bike
and --version gives a ctags.io website, so it's probably not some weird alias. yeah it does have that
17:55:49
kpoeck
:notify drmeister: could you have a look at the comments in https://gitlab.common-lisp.net/drmeister/asdf/-/commit/0a27ab4efb5ee7a0df2f01e5f14ffd36dfd69f71
17:56:09
kpoeck
::notify drmeister could you have a look at the comments in https://gitlab.common-lisp.net/drmeister/asdf/-/commit/0a27ab4efb5ee7a0df2f01e5f14ffd36dfd69f71
18:29:03
Colleen
drmeister: kpoeck said 32 minutes, 53 seconds ago: could you have a look at the comments in https://gitlab.common-lisp.net/drmeister/asdf/-/commit/0a27ab4efb5ee7a0df2f01e5f14ffd36dfd69f71
19:16:06
kpoeck
now as bike proposed attached as file to https://gitlab.common-lisp.net/drmeister/asdf/-/commit/0a27ab4efb5ee7a0df2f01e5f14ffd36dfd69f71
19:21:56
drmeister
Are you recommending that I apply the launch-programl.lisp.patch to my code and resubmit the merge request?