libera/#clasp - IRC Chatlog
Search
17:23:08
kpoeck
Yitzi i made more tests, also with your unicode branch. I tried to define 2 further tests, but get some errors both in your and in my branch. Could you please check whether the test in https://gist.github.com/kpoeck/e6cab4a26c65beb4b801e4a1acfa74ea are reasonable? For whats it worth, the work fine in sbcl
17:24:29
kpoeck
and a second question, how did you arrive the the exceptions for darwin in clasp_islower_wide? Same for clasp_isupper_wide
17:44:20
drmeister
Building ASDF with sjlj... Time real(46.737 secs) run(46.737 secs) consed(3895517400 bytes) interps(547) unwinds(0)
17:45:15
drmeister
Previously: Time real(51.441 secs) run(51.441 secs) consed(3907625424 bytes) interps(547) unwinds(6138)
18:02:01
yitzi
kpoeck: I came up with a code snippet, but forgot to save it. Give a sec and I'll reproduce the basics of it.
18:14:04
yitzi
kpoeck: This is essentially what I did. Warning, very kludgy. https://gist.github.com/yitzchak/404ee60f669b1791b035f99733115e97
18:26:17
yitzi
kpoeck: I really don't know what to do here. Seems like even the exceptions that I came up with don't fix stuff on all platforms at the same time. What do you think?
18:28:13
kpoeck
Perhaps we need to fix both what you did and also claspCharacter_downcase and claspCharacter_upcase?
18:29:35
kpoeck
what you did for clasp_isupper_wide and clasp_islower_wide looks fine (and a lot easier than my code)
18:31:10
yitzi
Or instead of comparing Linux to Mac, maybe compare to our downloaded UnicodeData.txt file?
18:32:13
yitzi
I think that would fit in what you are saying. Scan through everything for isupper, islower, toupper, tolower and check them adding exceptions as needed?
18:37:14
kpoeck
in sbcl - perhaps they still use an older unicode standard, this letter is not even recognized as lower-case-p
18:38:59
yitzi
Its kind of confusing. Is `lower-case-p` supposed to report on `Ll` category or if something has an uppercase letter? The title case letters also seem to be `lower-case-p = t`
18:42:13
yitzi
I haven't really thought about this, so I could not understanding the complete issue. But this http://clhs.lisp.se/Body/f_char_u.htm#char-upcase seems imply that a lowercase character has to have an uppercase equivalent.
18:43:27
Bike
13.1.4.3 says "Every character with case is in one-to-one correspondence with some other character with the opposite case." this is not how unicode works, so it's kind of problematic.
18:48:38
drmeister
Bike: the ASTMatcher library invokes the virtual method `run` and that invokes `_sym_run`
18:49:49
Bike
i think i have fixed some of the bit vector problems. one of the other misc ansi test failures is actually caused by an unwinding thing, think i have a fix for that as well
18:51:27
drmeister
Calling into the ASTMatcher library is done after we set up all the files we want to scan and all the matcher objects and then we call one function and it does everything.
18:51:58
Bike
how do we call that one function? do we go through a scraper-generated wrapper? do we do multiple-value-foreign-call?
18:57:06
drmeister
Bike: I think it's here: https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clasp-analyzer/clasp-analyzer.lisp#L3947
18:57:49
drmeister
llvm updates are coming that will allow it to work on macOS - it cannot right now.
19:03:22
drmeister
https://github.com/clasp-developers/clasp/blob/main/src/lisp/modules/clang-tool/clang-tool.lisp#L850
19:03:23
kpoeck
but (char-upcase #\LATIN_SMALL_LETTER_B_WITH_STROKE) should be #\LATIN_CAPITAL_LETTER_B_WITH_STROKE and it isn't with std::toupper (tested with xcode)
19:05:32
Bike
these seem like they don't go through the scraper. i've never actually understood how the .def forms work
19:06:03
kpoeck
There seem to be 1390 unicode characters that have a uppercase characted perhaps we are really better of with a table approach
19:12:38
drmeister
Before you tackled it with the scraper I was going to add the dynamic environment to the template functions.
19:22:31
Bike
i did it in the scraper because i knew how to do so, more than out of an idea that it was the right way to do it
19:30:15
yitzi
kpoeck: Just for the heck of it I am trying this: https://plaster.tymoon.eu/view/3126#3126
19:37:27
drmeister
I'm running the static analyzer for clasp+cando now - I'll get to the bottom of what's going on.
19:54:04
yitzi
drmeister: So I think this is a bug. https://github.com/clasp-developers/clasp/blob/0e207e7cfea6548d34635001477e1b7f01885147/src/koga/ninja.lisp#L704
20:01:54
Bike
ooh, there's an ErrorSimpleDepthCounter thing i missed, so there's all kinds of spurious warnings about error depth during the ansi tests
20:02:25
Bike
actually, is this something else we should remove? it again seems ot use a global variable instead of a threadlocal
20:13:36
drmeister
It looks like analyze in koga writes to clasp/src/clasp_gc.sif. It's not getting whatever clue I had waf pass to the static analyzer to put it in extensions/cando/src/clasp_gc_cando.sif
20:15:40
drmeister
yitzi: Yes - we need to tell it to put the clasp_gc_cando.sif file in the extensions/cando/src/ directory.
20:33:34
Bike
i'm inclined to simply remove ErrorSimpleDepthCounter, since we already have another depth counter for errors in general
20:39:43
yitzi
kpoeck: Can you try the unicode-3 branch when you have time? Testing on Linux now, but it passed on Mac here.
20:43:02
drmeister
yitzi: Could you change that so that it uses the name of the single extension if one exists to define the path and the name of the sif file?
20:44:56
yitzi
The `(extensions configuration)` is a keyword list of the extensions. Not sure what you would do with `:seqan-clasp`
20:46:56
yitzi
Plus, there is a lot places where `(member :cando (extensions configuration))` is treated specially. I get what you are saying, but changing the logic there is gonna really fix anything.
23:07:11
pfd
Fortunately, I discovered that Devuan Ceres contains clang/llvm 13.01.x. Therefore I'm converting my Devuan Chimaera system(s) to Devuan Ceres. Hopefully ./koga will be approve.
23:18:12
yitzi
drmeister: Two things occurred to about the static analyzer. First could we make the sif files generated by it composable? Basically have tags recognized by the scrapper that identifies which sif file the stuff from the analyzer goes in? Or if that doesn't work, maybe don't put clasp_gc_cando.sif in the cando folder. Instead put all of them in the clasp folder with a name concatenated from the extensions, i.e. `(format nil "src/clasp_gc~{_~a~}"
23:18:26
pfd
I'm still in the process of updating packages to reach llvm13, through apt in Devuan. Previously I tried building LLVM 13 and installing it afterwards, but the process didn't proceed due to my system's ID not being 'Debian' or 'Ubuntu'
23:21:23
pfd
Yes, but it's not smart enough to know that Devuan is virtually identical to Debian. It doesn't proceed with the sysId saying 'Devuan'
23:22:12
pfd
As I stated above the bleeding edge / rolling version of Devuan has LLVM 13 within APT.
23:24:35
pfd
I just changed my repos list to make this massive shift. I'll just have to ignore/be selective about a lot of updates, if I want to stay relatively stable.
23:26:41
yitzi
Well, I've already gone to the darkside with Arch long ago so the claims of "stability" with Debian are foreign to me. ;)
23:30:28
pfd
I also still use CMUCL with Hemlock. Debian/Devuan is closest and most dedicated to that project and to GNU Hurd, etc.
2:18:15
drmeister
yitzi: I'm not sure about composable. The static analyzer crawls through all the code and finds all classes that have smart_ptr in them and maps out their offset.
2:18:47
drmeister
If we have a class in clasp and we inherit from it in Cando - then we need the layout of the clasp part.
2:21:49
drmeister
We have been able to improve things. It used to be that we had to run the static analyzer for every combination of extensions. That was because back then the stamps were calculated by the static analyzer.
2:22:17
drmeister
Now we leave that for the scraper and it can merge the .sif files from each extension into a correct layout of every class.
2:30:29
Bike
like generate the clasp sif and the cando-clasp sif, and then take everything out of the cando-clasp sif that's in the clasp sif
2:30:47
Bike
or, somewhat more sophisticatedly, change the analyzer so you can give it an input base sif, and it won't bother recursing into anything in the base
2:31:04
Bike
then we could set the base as the clasp sif and run it on cando-clasp and just get the cando stuff