libera/#clasp - IRC Chatlog
Search
9:41:36
Colleen
karlosz: Bike said 9 hours, 35 minutes ago: save-sp/restore-sp is broken with respect to unbinding special variables, for now
9:41:51
karlosz
::notify Bike I see the issue you're talking about with special bind. i don't think there's a good way to solve it generally
9:43:36
karlosz
::notify Bike the main problem is we can't unbind multiple frames in one instruction like clisp because the binding stack is implicit on the C++ stack and not explicit. so i see two things we could do to still get most of the benefit: eliminate dynenv consing by using a 0 marker to explicitly invalidate dynenvs, and also to just restrict savesp and restoresp to contexts where no special binding has been lexically done. this is easy
10:40:14
yitzi
::notify drmeister I replaced the offending fmt::fprintf calls with fmt::print https://github.com/clasp-developers/clasp/commit/aaeedd079551ff27fa02aa508067326c4fd81761
10:55:54
yitzi
karlosz: I probably won't be able to fix that until we dump the old [abc]clasp build methods. compile-kernel-file currently doesn't know that the order of compiling files is not the same the order files in the image so I've had to trick it a bit.
12:57:21
Colleen
Bike: karlosz said 3 hours, 15 minutes ago: I see the issue you're talking about with special bind. i don't think there's a good way to solve it generally
12:57:21
Colleen
Bike: karlosz said 3 hours, 13 minutes ago: the main problem is we can't unbind multiple frames in one instruction like clisp because the binding stack is implicit on the C++ stack and not explicit. so i see two things we could do to still get most of the benefit: eliminate dynenv consing by using a 0 marker to explicitly invalidate dynenvs, and also to just restrict savesp and restoresp to contexts where no special binding has been lexically done. this is easy
13:01:59
Bike
karlosz: could we not just generate a bunch of unbind instructions as part of the restore-sp jump sequence?
13:03:07
karlosz
i just forgot about the fact that the way i designed the fixup logic was to be optimizstic
13:03:29
karlosz
which works well for this case actualy, the resizer supports arbitrary amounts of stuff to add
13:03:44
karlosz
it wouldve been a roblem if we did the hole shrinkage fixup method because we wouldn't know how much space to insert at first
13:05:37
karlosz
as a side note the pc microoptimizations i did seem to have a measurable effect oncleavir self compile performance
13:17:04
Bike
drmeister: i tried building withi USE_IMPLICIT_BYTECODE_COMPILE_HOOK, but when it crashed it looked like *implicit-compile-hook* was the default rather than anything bytecode related.
13:26:50
Colleen
drmeister: yitzi said 2 hours, 46 minutes ago: I replaced the offending fmt::fprintf calls with fmt::print https://github.com/clasp-developers/clasp/commit/aaeedd079551ff27fa02aa508067326c4fd81761
13:27:18
drmeister
Edit the bottom of clasp/include/clasp/core/configure_clasp.h - and uncomment that.
13:29:50
drmeister
You can search the source code for that and see the two dynamic variables that change.
13:30:43
drmeister
You also set `(setq *echo-repl-read* t)` at the top of `init.lisp` - that will print every form as it is read so you can see where the crashes are.
13:33:43
Bike
i fixed the prog thing - just had to move up multiple-value-setq - but then it died in cmpintrinsics in a way that looks like it's not handling top level forms correctly
13:33:53
drmeister
I was also using the disassembler in the udb python extension to figure out if things were miscompiling
13:35:58
drmeister
I figured you would do a better job of straightening out the macro loading sequence.
13:37:21
drmeister
Yeah - so my next question is what the heck is up with the thing in cmpintrinsics?
13:43:27
Bike
complaining that %global-entry-point% is unbound. define-c++-struct expands into (progn (define-symbol-macro %global-entry-point% ...) ...some stuff that refers to %global-entry-point%...)
13:43:57
Bike
if top level forms aren't processed correctly it will try to compile the latter forms before it defines the symbol macro, resulting in an unbound variable error
13:43:58
drmeister
Yes - that's what I hit as well - is it a loading order thing or a compiler problem?
13:44:37
Bike
well, i'm pretty sure the toplevel form processor i wrote handles this correctly, so i'm thinking maybe it isn't happening
13:50:17
Bike
the default debugger binds implicit compile hook to the default. i guess that's probably what i was seeing.
13:53:50
drmeister
https://github.com/clasp-developers/clasp/blob/vm/src/core/bytecode_compiler.cc#L1934
13:54:05
Bike
bytecode-eval-with-env is not the correct eval hook. the eval hook has to do top level form processing, and bytecode-eval-with-env does not
13:59:02
Bike
maybe i should change the hook name so it's more obvious it's suppsoed to do toplevel processing
14:09:33
Bike
i have an error here that i think i'd be seeing if it was compiled before incf was defined
14:22:41
Bike
mhm. i see. now it's hitting a problem that looks like multiple-value-bind being defined too late
14:23:20
Bike
well, if it's interpreted this isn't a problem as long as the code isn't evaluated until after the macros are defined
14:24:19
Bike
i don't want to rewrite all these mv-binds though. would it be ok to move the definition of mv-bind into clasp-builder, or would that cause problems? maybe i should copy it instead?
14:24:51
drmeister
That was how it was intended. init.lisp was to help us climb out of the Turing tarpit.
14:26:36
drmeister
We don't need it early - although there may have been some stuff moved in there that we need to keep early.
14:27:29
drmeister
I say we fill init.lisp with everything that we need but only what we need to get clasp-builder.lisp to compile and bootstrap.
14:28:35
drmeister
init.lisp is to configure things and to make pidgin Common Lisp slightly more tolerable.
14:29:13
yitzi
I wouldn't recommend worrying too much about what is in init.lisp in the immediate. There is a lot of stuff in clasp-builder that I will nuke eventually.
14:30:29
drmeister
We need it to run now. We are about to transition from the interpreter to the bytecode-compiler.
14:30:58
drmeister
The interpreter interpreted macros on the fly - so we could load things out of order.
14:31:21
yitzi
Understand. I just mean don't worry about making it "pretty." I am working on some major rewrites of clasp-builder.
14:31:45
Bike
moving definitions into init isn't actually hard or anything. it's easier than rewriting to use multiple-value-call
14:32:51
yitzi
If we want multiple files for init.lisp, I have a way to do that with koga and pass via the config header.
14:53:49
drmeister
I'm going to check all my macro changes against the normal build and the bytecode implicit compile build and make sure that I am doing no harm to the normal build and that the bytecode implicit build gets up to the point where the toplevel form handling shut me down last night.
14:54:23
drmeister
Then I will move my early macros into init.lisp (where they are currently loaded via (LOAD "export.lisp"))
14:55:05
Bike
we should be able to make the basic control macros (when ,do, multiple-value-bind, that kind of thing) available for use in clasp-builder, so that writing it won't be entirely terrible
14:55:55
drmeister
CASE probably won't be one of them. There is a CASE in clasp-builder.lisp - change that to COND.
14:56:25
drmeister
Also, HANDLER-CASE probably won't be one of them. There are two uses of HANDLER-CASE in clasp-builder.lisp. What do we want to do with those?
14:58:05
Bike
i rewrote the multiple-value-setq to not do that, and finagled do to avoid psetq when it can, and that was apparently enough
15:06:15
drmeister
If you've got it working then you could move the macros into init.lisp and I don't need to confuse things by pushing my solution to this.
15:09:09
drmeister
Ok, then I guess it makes sense for me to push this - because it's basically what I have except now I MOVED THE MACROS INTO INLINE.LISP
15:11:46
drmeister
I gotta figure out how to get jupyterhub working really quickly on AWS to run cando.
15:11:53
Bike
alright, so just push it and i'll work it out from there, like we talked about. apparently it won't even be that hard
15:14:46
drmeister
Oh - there is one other thing. I dealt with mv-setq by rewriting them using mv-bind
15:45:33
karlosz
probably to kkeep track of the current special binding scope it makes more sense to put it on the context structure
15:46:15
karlosz
something like context-special-count or something that the dynenv-info would save, then just compare a tthe exit point and generate cleanup code there
15:50:44
Bike
we might have to do something like that if we want to inline unwind-protect at some point
15:53:53
karlosz
hm, i suppose we also have to clean up not just intermediate speial bindings, but alos intermediate block/tagbodies
15:56:15
karlosz
because usually you don't have random crap like special bindings you are exiting from or intermediate non local exits
16:00:02
Bike
yeah... i might also have to reevaluate the alternate more complicated way i was imagining doing dynenv stuff, where you wouldn't have bytecode_vm calling itself recursively
16:55:08
Bike
it would work like unwinding tables in C++ work - for each bytecode function we associate a mapping from PCs to dynamic environment information
16:55:22
Bike
like if an instruction is inside a special-bind, there'd be information recorded on how to undo it
16:55:43
Bike
then we have a try/catch whatever block, and when we do a C++ unwind we consult the table to see what to do