libera/#clasp - IRC Chatlog
Search
14:56:37
drmeister
Phew - I got the clbind type cast graph moved into GC managed memory. That's an important step to getting it handled by the snapshot save/load code.
14:57:07
drmeister
Currently snapshot save/load doesn't save/load the clbind type cast graph and so clbind doesn't work with loaded snapshots.
14:59:48
drmeister
What's complicated about it is that it maps what source C++ classes you can cast to what target C++ classes and uses template specialized C++ cast functions to achieve this. So I have a mess of function pointers that need to be saved and then restored on load.
15:07:36
drmeister
Bike: Where are you at with the things you are working on. How is the scraper/static analyzer stuff going?
15:08:12
Bike
i fixed the analyzer to output an actual sif, and i have the scraper loading that sif and interpreting the tags, but it's not doing anything with the tags yet
15:09:17
drmeister
Ok - I'm running head first into the problems with the old method. To build clasp with both the seqan-clasp and cando extensions I have to rename clasp_gc_cando.cc to clasp_gc_seqan-clasp.cc (or something like that name).
15:09:53
drmeister
The wscript file has a way of cooking up a clasp_gc_xxxx.cc file name based on the installed extensions.
15:11:04
Bike
right... well, the fact is i don't understand what any of these data actually mean, which makes it a little daunting to proceed. even just outputting the code stream... i'm not sure the information in the tags covers everything that the analyzer outputs in the cc file
15:12:48
drmeister
Oh - ok - I thought you were moving forward with confidence. We can talk about it some more. I'm 99+% sure everything in the cc file can be generated from the tags that I had the static analyzer write out.
15:13:37
drmeister
Because I followed what the code generator in the static analyzer was doing and wrote out the information that it needed.
15:13:53
Bike
right... the particular example i was looking at i was wrong about, so maybe it's fine
15:14:47
Bike
and yesterday i tried to figure out libtooling enough to see if source information could be provided but that's a whole other can of worms
15:14:50
drmeister
The kind of new/hardest thing is to generate the stamps - are you starting with that or are you starting with the class layouts?
15:15:20
Bike
i figured i'd start with the layouts since that can be done independently of the other scraper information, i think
15:17:56
drmeister
https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clang-tool/clang-tool.lisp#L871
15:18:49
drmeister
mtag-source is what I was using to get source info - I believe. Are you familiar with that function and what it returns? I'm pretty hazy and would have to run some experiments.
15:19:53
drmeister
The static analyzer is driven by the clang tooling C++ code and it's hard to wrestle information out of it. So I tend to put print statements into the code (and leave them there - hence the huge amount of output).
15:20:15
Bike
i'm really not. but the source info isn't a big deal, i can figure it out later if ever, it would just mean i wouldn't get slot unbound errors whenever i try to print one of the analyzer tags
15:22:16
drmeister
https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clasp-analyzer/clasp-analyzer.lisp#L1772
15:23:24
drmeister
https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clasp-analyzer/clasp-analyzer.lisp#L1670
15:25:28
drmeister
Looks up the node bound to :method in the 'minfo' match result (I think it's a match result) and returns some kind of source info.
15:26:49
drmeister
I think that will give you the source position of every method declaration in whatever C++ header file of the C++ class/struct we are interested in.
15:27:22
drmeister
Yeah - really good source info is definitely available - because I was using this to do source to source translation.
15:50:29
Bike
i mean, what i'd like is some kind of conceptual background, but i know you don't have that written down and it would take time to explain
15:50:41
Bike
you have to understand - i'm at a level where i don't know what things like "templated kind" are even suppoesd to mean
15:51:22
Bike
i can probably figure out the code generation just by copying stuff, but stamps will be more involved
15:54:41
drmeister
Understood. We have a place to put documentation now. Ask me specific questions and I'll make a section on the static analyzer. Things like "templated kind" I'd have to go into the source code and figure out what I was doing.
15:58:20
drmeister
That is to indicate that these classes: Iterator_O, Creator_O, WrappedPointer_O, ConstructorCreator_O, Closure_O, BuiltinClosure_O can be further subclassed by template classes that can be parameterized.
15:58:50
drmeister
The are the immediate base class of additional classes that are templated on other types.
16:00:53
drmeister
This recognized the templated_kind enum and stores info about them. If the code is identical to what we do with class_kind then we could switch them to class_kind.
16:02:20
drmeister
I propagate the 'templated_kind' info to the gdb/lldb debugging interface in places like here...
16:02:28
drmeister
https://github.com/clasp-developers/clasp/blob/isl/src/gctools/gc_interface.cc#L784
16:02:57
drmeister
So it may be a useful flag to keep using - basically it's a 'class_kind' flag to expose a class.
16:07:38
Bike
i'm keeping all the information i get, i think, if only because I don't understand what's important
16:38:54
Bike
ok, here's one specific question. if "atomic-smart-ptr-p" is true, the analyzer outputs a variable-field tag with some slots unbound
16:39:06
Bike
should i just... check for that? is that a different kind of thing from the other variable fields?
17:47:40
Bike
have some code generated, but there are some little problems... the tags:variable-capacity records (layout-offset-field-names array) but not (layout-offset-field-names end) and and (l-o-f-n length), which appear in the generated code... unless i can infer them somehow?
17:59:42
drmeister
The atomic-smart-ptr-p thing - I think on macOS and Linux the way atomics are defined is different.
18:00:33
drmeister
I run the static analyzer on linux usually - and you are probably running it on macOS. Could you show me the sif file entries you are getting?
18:03:18
drmeister
{ TAGS:VARIABLE-FIELD ( TAGS:OFFSET-TYPE-CXX-IDENTIFIER . "ATOMIC_SMART_PTR_OFFSET") ( TAGS:CTYPE-KEY . "std::atomic<gctools::smart_ptr<core::T_O>>") }
18:04:12
drmeister
And I had a problem with atomic values because they have internal C++ structure on MacOS or Linux (or both) and the differences drove me a bit nutty.
18:04:38
drmeister
But since they all reduce to a single word I ignore all of that and just use the offset of the std::atomic<whatever>
18:05:20
drmeister
I'm guessing you are getting confused by the internal structure that I'm ignoring?
18:07:19
drmeister
The internal structure is simply C++ template types within C++ template types that all reduce to a single word in memory. On one of the OS's the internal structure is declared 'private' and so my C++ code can't get access to it.
18:17:29
Bike
but the atomic-smart-ptr-p thing means there are two kinds of code generated corresponding to the same tag class
18:18:08
Bike
atomic-smart-ptr-p makes it generate code as if there's multiple fields, except the variable-field tag lacks the information it has when there are actually multiple fields
18:21:35
Bike
https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clasp-analyzer/clasp-analyzer.lisp#L810-L820 the other problem is here - the code uses these layout offset field names that aren't actually put in the tag
18:31:15
drmeister
I don't think I explained how the tags work together. Either information is missing and you need to add it - or it's in one of the related tags and you need to group the info together...
18:33:56
drmeister
The FIXED-FIELD's define fields in the Rack_O and the VARIABLE-ARRAY0 and VARIABLE-CAPACITY together define the stretchy vector on the tail end of the Rack_O and then there can be multiple VARIABLE-FIELD entries that define what is stored in each element of the stretchy vector.
18:34:30
drmeister
If you know that and that doesn't solve the problem - then you understand things and yes - we need to add some more entries to one of the tags.
18:34:55
drmeister
If you paste the sif file entries for the class you are talking about we can take a look.
19:19:28
Bike
i mean, conceptually. i don't get how the corresponding c++ code really defines anything
19:20:01
Bike
{ TAGS:VARIABLE-CAPACITY (( TAGS:CTYPE . "unsigned short")( TAGS:OFFSET-BASE-CTYPE . "gctools::GCArray_moveable<unsigned short>")( TAGS:FIELD-NAMES . ("_Data"))) \
19:20:43
Bike
this is incorrect, as far as i can tell, because the field name for the capacity is _MaybeSignedLength, and that's what's in clasp_gc.cc too
19:23:28
Bike
i have made some minor changes to the analyzer's sif file generation already btw (in my branch)
19:24:32
drmeister
{ variable_capacity, sizeof(short), __builtin_offsetof(SAFE_TYPE_MACRO(gctools::GCArray_moveable<short>),_MaybeSignedLength), __builtin_offsetof(SAFE_TYPE_MACRO(gctools::GCArray_moveable<short>),_MaybeSignedLength), 0, NULL },
19:24:40
Bike
in the code generator it actually uses two things which apparently both end up as _MaybeSignedLength
19:26:39
drmeister
This is an array - that means the "end" and "capacity" are equivalent and I put the same info in them.
19:28:14
drmeister
So I provide two offsets for GCVector, one for the "end" and one for the "capacity".
21:01:32
drmeister
The GC_DECLARE_FORWARDS part of the clasp_gc_cando.cc was a especially annoying thing to have to build.
21:36:16
Bike
drmeister: this isn't important, but out of curiosity, what are priority tags for? i think there's only one use in the codebase