libera/#clasp - IRC Chatlog

18:33:19 yitzi Compiling with the new pass manager...fingers crossed

18:38:08 yitzi Seems like it works. I had to disable some optimizations b.c. I don't know what the new version is yet or I haven't exposed that stuff to Lisp yet. I replaced the addPasses..andRun lisp function with EMIT-MODULE that handles usage of the old pass manager for codegen, thereby shielding us from its existance.

18:53:26 drmeister yitzi: Does the --pkg-config path need the path to the bin directory?

18:56:18 yitzi It needs the path to pkgconfig binary

18:56:41 yitzi drmeister: ^

19:03:23 drmeister Got it.

19:04:09 drmeister I screwed up and hadn't installed the arm fmt library. I `/opt/homebrew/bin/brew install fmt` and then `ninja -C build` and it failed again linking fmt.

19:04:31 drmeister I'm thinking I need to `koga --deep-clean` each time to reset everything - does that sound right?

19:05:31 yitzi No. Just --clean

19:05:48 drmeister Ok, well it worked, now it's compiling Lisp code.

19:05:54 drmeister I'll do --clean next time.

19:06:02 yitzi deep clean deletes the repos which isn't needed between archs

19:06:27 yitzi Did you see that I got the new pass manager working on llvm16?

19:06:59 drmeister I did not. Yer takin' my jerbs!

19:07:33 yitzi hehe. I had to disable some optimizations and I haven't exposed it to Lisp yet. But it compiles and runs.

19:07:34 drmeister https://www.youtube.com/watch?v=QGmhLtsK2ZQ

19:07:58 yitzi nice

19:08:24 drmeister Is this the failure you see?

19:08:32 drmeister https://www.irccloud.com/pastebin/GIpxmjYh/

19:09:35 yitzi Yep. Although frame 52 is probably erroneous. I think the error is in frame 55 or frame 54

19:09:51 drmeister Thank you. Investigating

19:10:57 yitzi Somewhere in that area (translate-logical-path "real-path") gets turned into (translate-logical-path "random-junk")

19:12:00 yitzi Which made me think it was some kind of problem in bytecode_vm or Wrapper_Variadic...

19:20:56 drmeister How do I turn on compiling with fork?

19:21:12 drmeister use-parallel? parallel-build - searching...

19:22:10 yitzi It should be parallel always unless jobs=1

19:22:28 drmeister I want to disable fork building.

19:23:16 yitzi `./koga --jobs=1` probably

19:43:39 drmeister (:parallel-build nil) works as well.

19:43:44 drmeister In config.sexp

19:44:08 yitzi Gotcha. With koga that would be `--no-parallel-build`

19:44:45 yitzi Looks they have removed the last non-opaque pointer stuff in LLVM17. So there isn't a getPointerElementType anymore.

19:46:28 drmeister Were we using that anymore?

19:47:18 yitzi One usage in cmpliteral.lisp ... not sure if it is dead code

19:47:27 drmeister How would I load all the source-code->bytecode and then drop into a repl?

19:49:51 yitzi ninja -C build load_cclasp-boehmprecise

19:58:23 yitzi Looks like I've got llvm17 compilation also. If so then we can keep 15 & 16 support.

20:01:11 yitzi The new pass builder has a textual representation of the optimization pipeline that it can parse. That might be easier then exposing the classes that represent different passes to Lisp.

20:02:05 yitzi For example, you can do stuff like `module(function(fpass1,fpass2,fpass3)) module(cgscc(cgpass1,cgpass2,cgpass3)) module(function(loop(lpass1,lpass2,lpass3)))`

20:04:21 drmeister In serial build the problem reproduces - so after lunch I'll dig deeper

20:05:51 yitzi ok.

20:09:31 yitzi bike: I am suspicious that this is unused code https://github.com/clasp-developers/clasp/blob/main/src/lisp/kernel/cmp/cmpliteral.lisp#L571-L630

20:10:12 yitzi I can't find any references aside from itself. Do you remember if it is being used?

20:33:36 bike i don't remember, but if it's not referenced outside that file i doubt it's used. all the literal stuff is in its own package, after all

20:34:27 yitzi thx

22:06:29 drmeister The M1 problem is reproducible using: `ninja -C build load_cclasp-boehmprecise` followed by `(compile-file "sys:src;lisp;kernel;lsp;generated-encodings.lisp")`

22:18:41 drmeister bike: Can we get arguments in the bytecode backtraces?

22:18:55 drmeister I don't recall if you answered that when I asked a few days ago.

22:27:42 bike getting the original arguments is tricky since you'd have to look at the lcc_args in the frame, which I do not know how to do programmatically

22:27:59 bike of course everything ought to be available in the saved lexical variables, but that's not apparent in udb

23:09:29 bike i have function cells working with everything now, i think. have to make some cleavir changes for the first time in a while

23:40:37 drmeister yitzi: There are corrupted objects in memory - this may be the cause of the problem.

23:54:52 drmeister If I start clasp with this: `./boehmprecise/iclasp --norc --disable-mpi --ignore-image --feature clasp-min --load load-clasp.lisp -- base 0 "SYS:SRC;LISP;KERNEL;STAGE;BASE;0-BEGIN.LISP"`

23:55:38 drmeister Then run `(room :test)` it reports that memory is clean. This means that every slot in every object in memory that points to another object actually points to a valid object.

23:55:42 drmeister Memory is internally consistent.

23:56:35 drmeister If I start with... `ninja -C build load_cclasp-boehmprecise` and then do `(room)` it report memory errors. If I use `(room :test)` - a very stringent test, it hangs.

23:56:44 drmeister So something is rotten in the state of Denmark.

23:56:56 drmeister Also, memory is being corrupted.

23:58:41 drmeister The latter run generates this...

23:58:43 drmeister https://www.irccloud.com/pastebin/dPtOUvMS/

23:58:49 drmeister It goes on and on and on.

0:03:27 drmeister I added a `(room)` invocation after every source file is loaded - it's going very far without errors...

0:07:43 drmeister bytecode-adaptor.lisp is where it goes off the rails.

0:07:45 drmeister https://www.irccloud.com/pastebin/plGMnO8B/

0:26:43 drmeister https://github.com/clasp-developers/clasp/blob/main/src/lisp/kernel/cleavir/bytecode-adaptor.lisp#L109

0:27:11 drmeister It's after loading that defun that the memory goes haywire.

0:30:05 yitzi So maybe when it recompiles the wrappers?

0:30:54 drmeister It's just loading that that causes the problem. I'm puzzling over it.

0:31:00 drmeister AFK for a bit

0:31:23 yitzi ok

1:17:17 drmeister #+(or) and #-(and) mean the same thing right?

1:17:25 drmeister Don't compile the following sexp

1:18:48 bike yes.

1:19:07 bike skip it while reading, technically

1:24:40 yitzi Not much happens in bytecode-adapter

1:25:03 bike fcells is all working but cleavir documentation build is broken for stupid reasons. ehhhhh.

2:55:29 drmeister I'm back - but it's late on the east coast.

3:14:56 drmeister I wonder if it's a problem with boehm

3:15:42 drmeister The memory problem manifests in different places around the bytecode-adaptor.lisp file each time I run it.