libera/#clasp - IRC Chatlog
Search
11:45:00
yitzi
zeus is missing some stuff. I don't have sudo there. Once the right deb packages are installed you can do `debuild -i -us -uc -b` in the root
11:45:50
yitzi
That will deposit a clasp-cl and a cando debian package in the directory containing the clasp repo
11:57:05
yitzi
Can we set a column limit in our `.clang-format` so we can use clang-format to make of the cc sources a bit more readable?
12:01:27
drmeister
What do we do about tmpnam - the suggestion of changing it to mkstemp is not useful in at least one case.
12:01:48
yitzi
No idea. You can set up triggers for git to automatically call clang-format. That is a bit aggressive for some.
12:03:01
yitzi
Maybe we just need our own function. C people are weird about what they think are security risks sometimes.
12:10:14
yitzi
So in anticipation that ext-load is gonna work, I am gonna move the experimental user-install stuff out of it since that stuff isn't done. Then eventually squash and cleanup the commit history in prep for merging.
12:13:25
Bike
making the temp filenames in the parallel analyzer relative to the compilation tool database main pathname lets it run now, even if i don't quite understand where that is
12:18:00
yitzi
I can look at some point at the paths in the analyzer and make them work better with koga.
12:24:33
Bike
i wonder how i even ran the serial analyzer before then, because my build directory is not build/
12:44:44
Bike
yeah, they're placed in /tmp, and i was getting permission denied, apparently because your run had already put files with the same names in /tmp
12:45:21
Bike
anyway, now it's crashed, and the only error messages i got are just the backtrace being confused, so i should find the log files if i can
13:39:40
drmeister
Bike: Do you think `(namestring (make-pathname :type "rc"))` should be implementation dependent?
13:43:33
Bike
`(pathname-name (make-pathname :type "rc"))` should be NIL provided there's no default name, i think. now how is that namestringed
13:44:22
drmeister
That's very new that we even run the parallel static analyzer - so change those at will.
13:45:04
specbot
Pathnames as Filenames: http://www.lispworks.com/reference/HyperSpec/Body/19_ab.htm
13:45:08
Bike
"These functions [e.g. `namestring`] convert pathname into a namestring. The name represented by pathname is returned as a namestring in an implementation-dependent canonical form." sounds like carte blanche to me, unless i'm missing something, which i might be
13:46:22
Bike
"namestring n. a string that represents a filename using either the standardized notation for naming logical pathnames described in Section 19.3.1 (Syntax of Logical Pathname Namestrings), or some implementation-defined notation for naming a physical pathname."
13:46:30
drmeister
I agree - I went in to this to see examples where they show again and again that it is implementation dependent. http://www.lispworks.com/documentation/HyperSpec/Issues/iss265_w.htm
13:46:59
drmeister
sbcl signals an error. ecl and we (did) return NIL `(namestring (make-pathname :type "rc"))`
13:47:23
drmeister
Last night at 2:00am I changed it to return ".rc" and I wanted to know if I made things worse. I don't think so.
13:47:37
Bike
that said, for this particular purpose could we do `(make-pathname :name "" :type "rc")`... i guess sbcl still complains about that
13:50:28
Bike
on sbcl, `(pathname-type #p".sbclrc") => NIL` and `(pathname-name #p".sbclrc") => ".sbclrc"`
14:00:18
yitzi
If only they didn't have the component restrictions on characters. They would be so much more useful.
14:05:55
drmeister
yitzi: We are free to create a new type of pathname like object that we could use - this is for building and bootstrapping clasp/cando.
14:07:13
drmeister
We could inherit from Pathname_O and call it ClaspPathname_O and change the behaviors that we don't like.
14:07:16
yitzi
Ok. I'll do some more thinking about if it is needed and at a minimum make the scrapper work better with koga.
14:08:05
drmeister
Ah - anything that will run in sbcl will not have that unless we re-implement pathname code.
14:08:59
yitzi
If we have to, we use the `source` path stuff I came up with. I am non convinced that we will need that though.
14:12:28
Bike
drmeister: if the problem is specifically #P"home:.rc", we could do `(merge-pathnames #p".rc" (user-homedir-pathname))`, no?
14:14:57
yitzi
It behaves like SBCL now. If the user supplies `--rc` it uses that path directly without assuming it lies in `$HOME`. Otherwise it uses `$HOME/.clasprc`.
14:14:58
drmeister
Yeah - I was more concerned if I had made things worse with my changes to cl__namestring that change what `(namestring (make-pathname :type "rc"))` now do.
14:15:34
yitzi
Still need to be improved a bit. It's a little hacky, but at least I can override via `--rc` in the snapshot build.
14:16:10
drmeister
A `LOG(...)` macro requires SOURCE_DEBUG to be turned on globally and #define DEBUG_LEVEL_FULL at the top of a file that contains the `LOG(...)` invocations.
14:20:18
drmeister
I don't know. I assume they have only one logging facility - they are pretty tidy.
14:25:32
drmeister
The core::lisp_debugIsOn(__FILE__) is kinda expensive - that's why I added SOURCE_DEBUG to compile them to nothing globally.
14:26:19
drmeister
Because we can turn on logging from individual source files using: `export CLASP_DEBUG=foo.cc,bar.cc`
14:28:09
drmeister
The __FILE__ is used to find just the filename and that is looked up in a std::set of filenames initialized from CLASP_DEBUG
14:29:03
drmeister
llvm has a logging system where you turn on logging with names of subsystems - like "link" or "lljit"
14:32:57
mns
for issue 1311, I see this: Fatal error 'mutex 0x814474b88 own 0x0 is on list 0x0 0x814474b08' at line 154 in file /usr/src/lib/libthr/thread/thr_mutex.c (errno = 2) what I'm trying to find out is which file or directory is missing (the errno = 2 part of the error). any thoughts ? this seems to be something related to threading. the header file does exist, I verified that already.
14:37:24
yitzi
We could use that, would still be nice to have a higher level interface that adds FILE, fmt, etc. So the messages are standardized.
14:37:51
drmeister
mns: Could you post the log of what you are seeing on that freebsd build? I've never seen that error - it looks like it's pretty deep in the weeds.
14:38:54
yitzi
Also, keep in mind that on the freebsd branch the snapshot stuff will be broken. I just added the minimal stuff to get it past iclasp.
14:39:36
drmeister
I'd like to know where in the build it's crashing - I have no idea if the snapshot code is being evaluated.
14:40:00
Bike
okay, so the clang tools main-directory is the actual main directory in the sources, that's probably not where we want to dump logs
14:41:46
drmeister
As far as I remember, we don't use any multithreading until cclasp - so I'm perplexed by mutex crashes. Unless it's the initialization of the multiprocessing code that's causing this.
14:43:18
Bike
alright i guess cl:equal signals that if it gets an argument it doesn't understand. that ought to be impossible, so maybe it is getting junk for threading reasons
14:43:31
drmeister
mns: Our build system goes (1) compile C++ pidgin Common Lisp interpreter (2) load minimal common lisp code and compile it -> aclasp (2) load all of common lisp and compile it -> bclasp(slow CL) (3) load the Cleavir common lisp compiler and compile it -> cclasp (fast CL).
14:47:16
Bike
https://github.com/clasp-developers/clasp/blob/main/include/clasp/core/object.h#L602 this is the error i'm seeing, don't know what you saw
14:54:22
drmeister
Bike: There is (core::safe-repr xxx) - that will try to print a representation without using the printer. If it fails it will give you the raw tagged pointer.
14:55:33
drmeister
Bike: I saw bad comparisons - I saw a variety of errors - and it jumped around - and sometimes I didn't see any error at all. That's why I guessed it was a nasty multithreading problem.
14:56:24
Bike
well, maybe. it was only in one thread out of several, and it looks like some kind of junk data, so i suspect it will be chaotic
15:01:20
mns
drmeister: yes I'm a freebsd user. relatively new (4-5 years), but its my primary server at home. I output in the issue to give an indication of where the build failed.
15:10:50
yitzi
Bike: adding `SortIncludes: false` fixed. Now format code for me clang-format minion!
15:11:47
mns
I haven't used gdb in a while, but Im familiar with it. I'll try and get a backtrace and add it to the issue. I'll need to get it installed on the freebsd system.
15:13:53
drmeister
mns: I'm guessing that when the multiprocessing code is being initialized it's doing something that freebsd doesn't like. We have a freebsd user who got this to work several months to a year ago (cracauer) but it hasn't been tested since then.
15:15:36
drmeister
It may be better for us to debug this because we are very familiar with the tools but you are very welcome to do it if you want to use this as an opportunity to get your hands dirty and better familiarize yourself with the tools.
15:16:19
drmeister
We are stretched pretty thin at the moment and cracauer is in Germany and tied up with other things. He is our freebsd expert.
15:17:17
mns
that's why I'm in it, to dirty my hands, and get back to doing things like this. :-) I'll see what I can do.
15:17:19
drmeister
The error message that you posted is pretty clearly the downstream effect of a downstream effect of a downstream effect of a problem in clasp that interacts poorly with freebsd.
15:18:23
drmeister
They don't directly illuminate what is going wrong. It looks like a crash when clasp is loading compiled code - when it's initializing itself. A low level, C-style, gdb backtrace may be more illuminating.
15:20:18
drmeister
The good news is that clasp is pretty debuggable at this low level. All stack frames in Clasp in bclasp are essentially C stack frames and gdb should be able to give us good information. Uh.... unless it doesn't. The initialization code involves a lot of compiled code that is loaded via the LLJIT and that may or may not register source information on freebsd.
15:20:21
mns
one of the impediments I have is that this is a headless freebsd server, so scrollin back to cut and paste is a little difficult with tmux/screen as compared to being in a terminal directly. Anyway, I'll see what I can do to contribute.
15:20:51
drmeister
The first thing we will learn is "do we get good backtraces". If we don't many of the function names you get will be hexadecimal addresses.
15:21:52
drmeister
Sort of - I didn't know which one you were more comfortable with. They both work.
15:22:50
drmeister
The procedure for interacting with the remote process is different in lldb than gdb.
15:43:57
yitzi
I haven't seen a successful build on debian:bookworm, so I am not judging by that. The current issue is whether the ubuntu:jammy makes it into dcando.
15:45:47
yitzi
I had to change the std::locale to get the build to work in the docker images, which breaks it if C.UTF8 isn't available. That PR would make getting the locale not needed.
16:54:47
drmeister
Bike: I tried to run under udb - it succeeded the one time I tried sadly. We should try again. If we had a recording we could mail down where it’s going wrong.
16:57:43
Bike
fffffairly quickly. sometimes it goes like half an hour and i think it might be hanging, but more often it dies within a few minutes
16:58:47
Bike
it goes into gethash, it calls cl:equal, and the dbg_safe_repr i put in to print a bad value segfaults
20:40:30
Bike
and when i say "the filenames" i mean the tmp ones, not the hardcoded references to build/
20:41:01
yitzi
I had it generate a script to call the serial analyzer versus calling the stuff in `run-XXXX` so I wouldn't walk on waf's feet.
20:42:46
yitzi
Since we don't have waf anymore, I think the correct answer to fix the various paths is to make proper entry point functions in the clasp-analyzer asdf system that accept compile-commands, log file locations, etc.
20:43:27
Bike
oh yeah and i did alter the koga script so it generates code to use the parallel analyzer instead
21:01:39
Bike
the hash table problem is specifically in memoized-probe-file in clang-tool, and that uses a global hash table
21:06:22
Bike
and both of those look like they lock correctly... maybe the read/write lock implementation itself is busted? man, i hope not
21:15:46
drmeister
Also, setf gethash and gethash don't just setf gethash and gethash - they rehash and that's where things get compilicated.
21:16:33
drmeister
The other thing would be to put a mutex in the code that accesses that hash table to test if that code is responsible.
21:17:29
Bike
i thought we did that by default, but i guess not, and the table in clang tool is NOT thread safe
21:39:13
Bike
there are a couple other non thread safe tables in clang-tool, but they look like they're not altered during use