libera/#clasp - IRC Chatlog
Search
17:30:32
Bike
since the analyzer generates more stamps, all that code needs to make its way into the scraper
18:33:22
drmeister
Right. I changed how stamps were generated several times until I settled on what we do now. For a while there were some stamps that were hard coded. I don't think that's the case anymore. Checking...
18:35:28
drmeister
No - I don't see anything in the scraper about hard coded stamps. I'd expect to find a small table containing General_O but I don't.
19:25:14
drmeister
It does generate stamps for things like General_O - we probably don't need to. But it simplifies the code if the IsA test for General_O checks the range covering STAMPWTAG_core__General_O... STAMPWTAG_xxx - maybe?
19:26:30
drmeister
I did a depth first walk of the class tree and assign stamps as I go. So everything gets a stamp - even things like General_O and Number_O although you never see those in the header of any object.
19:27:09
drmeister
There are plenty of stamps available - so I didn't see a reason to make the code more complicated.
19:27:53
drmeister
If you wanted to make the code more complicated you would walk the class hierarchy the same way but only assign stamps for objects that really show up in memory.
19:29:01
drmeister
Hmm, there is a reason to not allocate stamps for these classes that don't get allocated - it would keep the vector of fixable pointer masks smaller and more cache friendly during GC.
19:31:04
drmeister
This is a good time to switch to this approach of only assigning stamps for concrete classes that get allocated. Then when you build the IsA tests you would use stamp ranges that start on the first concrete class stamp and end on the last concrete class stamp of all the subclasses of the class you are creating the IsA test for.
19:33:06
drmeister
Do it in two passes. First pass walks the hierarchy and assigns stamps only to allocatable classes. Second pass walks the hierarchy and at every class you search for the min/max stamp under it.
19:37:02
Bike
sounds like we need to build up a class hierarchy in the first place. i guess the sif files do list parent classes
20:06:55
drmeister
There is a way to tell in the static analyzer - it may not yet be in the sif files.
20:11:53
drmeister
I'm trying to recall how the static analyzer finds classes that aren't allocated.
20:12:46
drmeister
The problem was always - there's a million classes in any C++ AST because of all the transitive includes - how do we find the ones we are exposing.
20:17:18
drmeister
I think in the 'project' class there's a 'classes' slot - that's EVERYTHING, every class.
20:17:42
drmeister
Then you have lispallocs, classallocs, rootclassallocs, containerallocs. Those are all the different kinds of allocated classes.
20:18:08
drmeister
So maybe we want to write out a sif tag that says if the class is in one of those hash tables.
20:19:55
drmeister
The static analyzer cares because it's looking for specific template classes involved in allocating those different kinds of objects.
20:20:44
drmeister
I'm not sure the entire taxonomy is useful. I'm pretty sure lispallocs and containerallocs are useful but not rootclassallocs and classallocs.
20:21:32
drmeister
So we could create a tag: allocation-taxonomy and then take a look at the output and maybe it will tell us we don't use one or two of these and we could remove them.
20:22:40
drmeister
There are different template classes responsible for allocating these different kinds of things.
20:23:09
drmeister
https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clasp-analyzer/clasp-analyzer.lisp#L1834
20:23:46
drmeister
https://github.com/clasp-developers/clasp/blob/main/include/clasp/gctools/gcalloc.h#L864
20:24:12
drmeister
https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clasp-analyzer/clasp-analyzer.lisp#L1991
20:25:55
drmeister
Those aren't allocated the same way. They are just instantiated using say: Vec0<Cons_O> ... _KeysValues;
20:27:29
drmeister
Ok, so let me restate that. The static analyzer looks for every class whose instances absolutely needs a stamp.
20:28:05
drmeister
It also gathers up info on every class. I think it then uses the classes that need stamps to find their parent classes to build a complete hierarchy. Currently everything in the hierarchy gets a stamp.
20:28:24
drmeister
But I think we are set up to only assign stamps to the classes that absolutely need them.
20:28:57
drmeister
So in a way, limiting stamps to the classes that absolutely need them kind of fulfills the purpose of the static analyzer.
20:29:25
drmeister
We just need to get the info into the sif file so that the scraper can know what classes need stamps.
20:31:59
drmeister
Now, if we don't use the static analyzer - then we only have the info provided by scraping the C++ code and we don't know exactly what classes will be allocated - so everything gets a stamp in build_cboehm.
20:33:12
drmeister
It may not be much of an optimization. How many classes do we have that we allocate and how many do we have that we don't. It may be that 95% of classes need stamps.
20:47:17
Bike
no, i just mean, i don't understand the details of it and how it affects the stamp generation
20:47:22
drmeister
The idea of the WTAG is you get those two bits and you know where to get the stamp for dispatch.
20:47:53
drmeister
You generate stamps and then shift the stamp two bits to the left and then stick in the WTAG.
20:49:40
drmeister
When you do the IsA tests on ranges you deal with the WTAG by taking the min_stamp and max_stamp and testing the range (min_stamp<<2|#b00) ... (max_stamp<<2|#b11)
20:52:52
drmeister
Because of this every stamp|ww|mm value looks like a FIXNUM in Common Lisp with the integer value stamp|ww
20:54:47
Bike
okay, so what else do we need to worry about? the root? we wanted c++ classes to show up before lisp classes in the numbering, right?
20:55:13
drmeister
Now - we've discussed getting rid of the WTAG - because we could just use the header stamp values to figure out where the dispatch stamp value lives. But WTAGs are pretty deeply ingrained now and I don't think they hurt us too much and they might actually be useful. They save us several stamp comparisons at runtime vs one WTAG comparison.
20:56:20
drmeister
Just start numbering stamps at 1 and they can go up to 65536 (I think) or 16384 - one of those. You won't hit that limit.
20:57:22
drmeister
Currently the stamps go 1...65536 (C++ classes) 65537...128K (clbind classes) 128K+... Lisp classes.
21:00:14
drmeister
Assume there is a TAGS:ALLOCATED . 1 and you will get our current behavior. By the end of today I should be able to give you sif files with TAGS:ALLOCATED . 0 for things like Number_O and General_O
21:03:05
drmeister
There's no way I could have planned this all out from the start. There's way too much experience here.
23:11:19
drmeister
Bike: It looks like I was wrong about the allocators - everything is showing up as allocated. I must have some default allocators in there. Checking...
23:12:16
drmeister
Yeah : https://github.com/clasp-developers/clasp/blob/main/include/clasp/core/object.h#L366
23:13:46
drmeister
That applies to the lispallocs category - every class under General_O will be marked as allocated.
1:26:42
drmeister
Patching llvm? linking it into clasp when it has the same name as gcc's libunwind Argh.
2:03:47
drmeister
Bike: When you start using libunwind - can you leave code in there for when libunwind isn't available? I'm adding an UNWINDER setting to the wscript file so I can set it up three (3 - sigh) different ways for now.
2:08:36
Bike
okay. so as far as the code goes there will be the libunwind version and the existing frame pointer chasing version