libera/#clasp - IRC Chatlog
Search
11:39:31
drmeister
This is when ASDF is performing `ASDF/SOURCE-REGISTRY:INITIALIZE-SOURCE-REGISTRY`
11:39:58
drmeister
It works by trying to open files and if they don't exist it signals an error and the stack is supposed to unwind.
11:41:08
drmeister
I added a guard to the vm stack so if we attempt to write into it then it segfaults.
11:41:42
drmeister
This is to the end of the stack - now it won't overwrite other structures and give confusing errors.
12:33:43
Colleen
Bike: drmeister said 50 minutes, 57 seconds ago: Can you check the logs this morning.
12:44:44
Bike
it's going to be pretty annoying if we have to put the unwind-protect in bytecode_call back.
12:47:56
Bike
huh. well that's where i'd expect the stack pointer to get updated properly. although... maybe unwind protect won't do it so well...
12:50:48
drmeister
that does a funcall into a bytecode wrapper and that invokes a compiled LAMBDA - line 2.
13:06:42
kaskal
Hi! I'm new in this channel, I'd like to use clasp to interface it with our quantum chemistry code in c++, but I have some doubts. So we normally use the intel compiler for production calculations, is this possible to use clasp with code compiled with icc? How easy is it to compile clasp in HPC systems with the limited amount of libraries out of the box ?
13:12:08
drmeister
Bike: If it went into unwind-protect cleanups then it would invoke sjlj_continue_unwinding - right?
13:12:16
kaskal
thank you for your answer, I'll try it out then. Is someone using clasp alongside MPI ? I guess your molecular code is MPI paralelized. Maybe there are some interesting resources to look at
13:13:37
drmeister
I'm not sure if our new build system makes it easy to build - but we should get that working again.
13:14:32
drmeister
kaskal: To be more clear... I'm not sure if our new build system makes it easy to build the MPI version - but we should get that working again.
13:15:01
drmeister
I will warn you - people have had a lot of trouble building clasp - I'm not sure why. It's something different every time.
13:16:09
drmeister
We have invested a lot of time making it easier to build. We wrote our own build system. If you have troubles - please come and ask.
13:17:15
drmeister
Bike: Can you put the unwind-protect back into bytecode_call with a CPP flag so we can turn it off and on?
13:17:34
kaskal
alright, I use nixos so for testing on my own machines I hope the derivation for clasp is good enough so that I can reproduce it easily, which seems to be quite straightforward from what I can see https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/compilers/clasp/default.nix#L121
13:18:29
kaskal
drmeister the code would be cc4s, github.com/cc4s/cc4s, I'm interested in using lisp to make it easier to evaluate classes of diagrams in many body perturbation theory
13:20:48
kaskal
hmm not that much, most of it is simple for `double` or `std::complex<double>`, nothing super fancy
13:21:54
drmeister
Ok. I've exposed everything from straightforward C++ libraries and to header only template code.
13:22:25
drmeister
The header only template code involves exposing specializations of the templates.
13:24:25
kaskal
interesting, maybe I'd start interfacing clasp with CTF (https://github.com/cyclops-community/ctf),
13:27:28
yitzi
I don't think we have built using MPI with koga (build system). I'd have to investigate.
13:52:39
drmeister
Bike: My hamfisted attempt at wrapping a funwind_protect around the call in bytecode_call didn't work.
14:21:42
drmeister
If you connect udb before loading that file you can examine the stack using `lbt`
14:26:04
Bike
if this is the problem, i think we can fix it while still avoiding funwind_protect by making the sp more of a local variable, like we did for fp and pc
14:31:34
drmeister
boehmprecise won't work until the static analyzer works and that is blocked by this unwinding issue because the static analyzer loads with ASDF.
14:32:21
drmeister
Bike: Add a quick fix under a #if 0...#else ... #endif using funwind_protect and I can test if this is the ultimate problem.
14:34:37
drmeister
Bike: Just to be really clear - in bytecode_call, the call to bytecode_vm ... going into the bytecode_vm call and on a normal return from the bytecode_vm call - the vm._stackPointer should have the same value - correct?
14:36:57
Bike
the sp after bytecode_vm exits should be the same as sp before teh push_frame, i think
14:46:10
Bike
should i just conditionalize this and push it? i _think_ this is what the unwind-protect should be doing, but it's not working
14:48:07
drmeister
The stack blowup was when the stack was unwinding. Otherwise everything worked fine.
14:48:50
drmeister
But I calculated the difference between the stackPointer before and after a normal return from bytecode_vm in bytecode_call and there were lots of times when they didn't match.
14:50:40
drmeister
That doesn't make sense to me. But if it's necessary then using funwind_protect to reset the stackPointer will break things - no? I know that sounds nonsensical - we need to get to the underlying cause of the problems and there may be more than one.
14:59:00
Bike
it looks the same - i get a bunch of messages about stack top and stack guard, and then it dies with no obvious error messages
15:00:50
Bike
am i miissing a debug flag or something maybe? it builds for you, right? or doesn't it?
15:04:58
drmeister
I did have a CLASP_STAGE_COUNT=5 hanging around - that can impact what is built. I just unset it.
15:05:25
drmeister
My experience however is that that screws up the build - it doesn't make the build work better.
15:17:14
drmeister
I misinterpreted that. I should have interpreted that as while loading cltl2.lisp
15:17:50
Bike
well, it gets through the loading, i think. it's when it compiles. you know, the [1 of 400] stuff
15:20:57
drmeister
I'm running under the debugger - the object file registration is probably slowing it down.
15:30:06
drmeister
Running under the debugger is very, very slow for several reasons. Part of why I wanted to debug running with bytecode.
15:36:56
Bike
how about this - i push my code with the conditionalization, which breaks either way, and then i go work on some stuff i have in mind to maybe improve the stack pointer handling
15:37:57
drmeister
I'm trying `ninja -C build load_cclasp-boehm` and then `(cmp::compile-file-serial "SYS:SRC;LISP;KERNEL;LSP;CLTL2.LISP")`
16:05:08
drmeister
Rather fork and the stack guards are not compatible. I'll have to think hard why that is.
16:11:52
drmeister
Bike: So - I pull your changes - then I edit bytecode_call and turn on the funwind_protect?
16:26:36
drmeister
Bike: I pulled your changes, set #if 0 to #if 1 in bytecode_call and I'm building.
16:31:55
Bike
so what i'm doing now should have the same effect as the unwind protect without actually doing one, hopefully
16:34:02
drmeister
yitzi: If I want to run the static analyzer just on clasp - and not pull in cando - is there a way to do that?
16:37:46
yitzi
If cando is included in the koga line it is included with the static analyzer. Otherwise not.
16:50:43
yitzi
If you guys can fix this issue I think we should run the analyzer on clasp and then stop there. The cando build is much better on vm-boehm-simp.
16:57:25
drmeister
It looks like it's working fine `In on-start-of-translation-unit for file 11 of 177`
17:47:47
Bike
with the sp change the vm runs for a while until it dies on some pretty normal looking code. egh...
17:48:23
Bike
are the python scripts to get better udb backtraces of bytecode well enough along to use?
17:48:48
drmeister
If you want to turn the guards on to catch stack overruns - go into the VirtualMachine ctor and dtor and uncomment the code.
17:50:17
drmeister
It will segfault - but in a clean way, right away when it tries to write into the guard.
17:58:42
drmeister
I should give you a primer on the debugger extension - I added a lot of features this weekend.
20:48:21
Bike
it looks like it works if i update the vm sp on normal returns. i do not understand why, though...
20:50:39
Bike
i guess what must be happening is we return from a bytecode function, and then a different bc function is called, but without a call instruction to update the vm sp
20:51:06
Bike
which i guess could happen if we have a native function that calls distinct bc functions.
21:42:57
drmeister
It's currently crashing on the first file and `./analyze` runs koga and rebuilds everything.
22:49:58
drmeister
It's dying deep in the static analyzer - I'm guessing when it's about to call the callback.
23:07:04
drmeister
It's failing in this function: clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::matchWithFilter(clang::DynTypedNode const&)