13:06:42kaskalHi! I'm new in this channel, I'd like to use clasp to interface it with our quantum chemistry code in c++, but I have some doubts. So we normally use the intel compiler for production calculations, is this possible to use clasp with code compiled with icc? How easy is it to compile clasp in HPC systems with the limited amount of libraries out of the box ?
13:12:16kaskalthank you for your answer, I'll try it out then. Is someone using clasp alongside MPI ? I guess your molecular code is MPI paralelized. Maybe there are some interesting resources to look at
13:12:46drmeisterBike: The OPEN is wrapped in an IGNORE-ERRORS
13:12:59drmeisterkaskal: We have an MPI version of clasp
13:13:37drmeisterI'm not sure if our new build system makes it easy to build - but we should get that working again.
13:14:24kaskaldrmeister alright, I'll take a look then and I'll get back to you, thx a bunch
13:14:32drmeisterkaskal: To be more clear... I'm not sure if our new build system makes it easy to build the MPI version - but we should get that working again.
13:15:01drmeisterI will warn you - people have had a lot of trouble building clasp - I'm not sure why. It's something different every time.
13:16:09drmeisterWe have invested a lot of time making it easier to build. We wrote our own build system. If you have troubles - please come and ask.
13:16:31drmeisterkaskal: What quantum chemistry code do you want to interface?
13:17:15drmeisterBike: Can you put the unwind-protect back into bytecode_call with a CPP flag so we can turn it off and on?
13:17:34kaskalalright, I use nixos so for testing on my own machines I hope the derivation for clasp is good enough so that I can reproduce it easily, which seems to be quite straightforward from what I can see https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/compilers/clasp/default.nix#L121
13:18:29kaskaldrmeister the code would be cc4s, github.com/cc4s/cc4s, I'm interested in using lisp to make it easier to evaluate classes of diagrams in many body perturbation theory
13:20:01drmeisterDo you use a lot of template programming?
13:20:48kaskalhmm not that much, most of it is simple for `double` or `std::complex<double>`, nothing super fancy
13:21:54drmeisterOk. I've exposed everything from straightforward C++ libraries and to header only template code.
13:22:25drmeisterThe header only template code involves exposing specializations of the templates.
13:24:25kaskalinteresting, maybe I'd start interfacing clasp with CTF (https://github.com/cyclops-community/ctf),
13:24:25kaskalif that works it could be a great tool for developing
13:24:43kaskalfor context, CTF is a massively parallel tensor contraction engine
13:27:28yitziI don't think we have built using MPI with koga (build system). I'd have to investigate.
14:21:42drmeisterIf you connect udb before loading that file you can examine the stack using `lbt`
14:22:00drmeisterThat displays the backtrace with bytecode frames showing arguments.
14:22:10drmeisterYou can also display the dynenv stack with `lde`
14:26:04Bikeif this is the problem, i think we can fix it while still avoiding funwind_protect by making the sp more of a local variable, like we did for fp and pc
14:26:22Bikeum, hang on, what target should i be building
14:26:31Bikeit doesn't know what vclasp-boehmprecise is
14:26:33yitziBike: There is no vclasp there. Just ninja -C build
14:30:22drmeisterThe dynamic environment stack is filling up with BindingDynEnv_O
14:31:03drmeisterBike: Build with `ninja -C build cclasp-boehm`
14:31:34drmeisterboehmprecise won't work until the static analyzer works and that is blocked by this unwinding issue because the static analyzer loads with ASDF.
14:32:21drmeisterBike: Add a quick fix under a #if 0...#else ... #endif using funwind_protect and I can test if this is the ultimate problem.
14:33:01Bikealright. i'm just gonna see if it builds or what here
14:34:37drmeisterBike: Just to be really clear - in bytecode_call, the call to bytecode_vm ... going into the bytecode_vm call and on a normal return from the bytecode_vm call - the vm._stackPointer should have the same value - correct?
14:48:07drmeisterThe stack blowup was when the stack was unwinding. Otherwise everything worked fine.
14:48:50drmeisterBut I calculated the difference between the stackPointer before and after a normal return from bytecode_vm in bytecode_call and there were lots of times when they didn't match.
14:50:40drmeisterThat doesn't make sense to me. But if it's necessary then using funwind_protect to reset the stackPointer will break things - no? I know that sounds nonsensical - we need to get to the underlying cause of the problems and there may be more than one.
14:58:06Bikeok, build is crashing in the same place with the old code
15:30:06drmeisterRunning under the debugger is very, very slow for several reasons. Part of why I wanted to debug running with bytecode.
15:36:56Bikehow about this - i push my code with the conditionalization, which breaks either way, and then i go work on some stuff i have in mind to maybe improve the stack pointer handling
15:37:26drmeisterOk, I'm trying to catch the problem with compile-file of cltl2.lisp
15:37:57drmeisterI'm trying `ninja -C build load_cclasp-boehm` and then `(cmp::compile-file-serial "SYS:SRC;LISP;KERNEL;LSP;CLTL2.LISP")`
15:38:48drmeisterIt's not reproducing the problem yet.
15:39:50Bikepushed. it's conditionalized out for the moment.
15:40:04yitzidrmeister: There are only 3 stages now
15:40:32yitziSo setting it too 5 which have world ending consequences
15:40:55yitziI'll add a bounds check to CLASP_STAGE_COUNT
15:58:08drmeisterSetting it to 5 means it wouldn't have any effect - right?
15:59:50drmeisterI'm removing the stack write guards - they may be screwing with fork.
16:00:59drmeisterYeah - the stack guard screws up fork
16:37:46yitziIf cando is included in the koga line it is included with the static analyzer. Otherwise not.
16:39:28drmeisterOh - I don't have cando included anywhere AFAIK
16:50:43yitziIf you guys can fix this issue I think we should run the analyzer on clasp and then stop there. The cando build is much better on vm-boehm-simp.
16:51:05yitziI can rebase from vm-boehm and continue working there.
16:55:08drmeisterThe static analyzer is running now.
16:57:25drmeisterIt looks like it's working fine `In on-start-of-translation-unit for file 11 of 177`
20:48:21Bikeit looks like it works if i update the vm sp on normal returns. i do not understand why, though...
20:50:39Bikei guess what must be happening is we return from a bytecode function, and then a different bc function is called, but without a call instruction to update the vm sp
20:51:06Bikewhich i guess could happen if we have a native function that calls distinct bc functions.
20:53:34Bikemaybe i don't know how to avoid the funwind protect after all. dammit.