libera/#clasp - IRC Chatlog
Search
14:08:25
drmeister
FYI clasp/src/lisp/kernel/init.lisp is also being loaded right at the start - that was turning off *load-print*
14:08:27
Bike
it doesn't look like an infinite recurison to me, but it's hard to tell because as soon as it tries to print a bytecode compiler environment it crashes, because they're full of circularities
14:10:25
drmeister
Ok, then if you want to evaluate things a little more slimey you can use M-x run-lisp and give it iclasp-boehmprecise -I -f clasp-min -l loader.lisp
14:11:07
drmeister
I think <Shift>-<Enter> on a form in a buffer will pass that form to the run-lisp *interactive-lisp* buffer
14:11:17
Bike
the backtrace says that (var-info nil [an environment]) somehow immediately calls bytecode-implicit-compile-repl-form, which seems nonsensical
14:12:16
drmeister
Once you install (setq *implicit-compile-hook* 'bytecode-implicit-compile-repl-form)
14:12:53
drmeister
If you leave *implicit-compile-hook* alone - then you can use the clasp interpreter to evaluate things.
14:13:50
Bike
i mean, it's saying that the function var-info somehow calls the implicit compile hook
14:16:58
drmeister
I would turn on *load-print* and load the source code and see what expressions cause it to fail.
14:17:37
drmeister
Then don't load cmprepl-bytecode.lisp and bytecode compile that last expression in a run-lisp session.
14:20:27
Bike
Alright, so with that done i can see it gets all the way to... uh... itself, actually, to bytecode-compile.lisp
14:20:59
drmeister
Edit cmprepl to set *implicit-compile-hook* to the bytecode compiler/interpreter, hit a problem, unset it, load- that code and then try and bytecode compile and run it.
14:22:15
Bike
i already hooked bytecode environments into macro-function and stuff, but in this case it's load _constructing_ an environment and passing that to the bytecode compiler, rather than the other way around
14:22:57
Bike
well, it's fine, i can translate a clasp macrolet environment into a bytecode compiler one, as long as there are a few basic accessors
14:24:19
Bike
actually i really don't understand what's even making this macrolet environment, so let me find that first
14:26:40
drmeister
~/Development/clasp-vm/build/boehmprecise/iclasp-boehmprecise -I -f clasp-min -l ~/Development/clasp-vm/bloader/loader.lisp
14:26:48
Bike
i see, and eval-with-env-hook goes to t1evaluate, so it will try to do its own thing with toplevel forms before dropping to the implicit compile hook
14:27:45
Bike
so either i translate interpreter environments, or maybe i can just replace eval-with-env-hook? is there a reason i shouldn't?
14:30:39
drmeister
If you want to use udb later I can show you how to set some environment variables and then run emacs so that clasp generates debugging information for udb.
14:30:45
Bike
like what would happen, exactly? it seems like binding implicit-compile-hook means using the bytecode compiler for almost everything to begin with
14:31:32
drmeister
I don't know why load isn't working right now - where is it setting up an environment?
14:33:23
Bike
so t1evaluate sets up a macrolet environment, and then passes it to implicit-compile-hook, and bytecode-compile can't deal with it
14:34:07
Bike
other than turning off the switch in init.lisp and using eval-with-env-hook that's the code i have
14:36:40
Bike
if we replace the interpreter with the C++ bytecode compiler, of course, the macrolet thing won't come up since there won't be macrolet environments any more
14:42:13
Bike
it's in macrolet, where it's recursively compiling itself, so maybe that somehow is a problem
14:58:51
drmeister
To make udb more convenient run iclasp-boehmprecise from the clasp top level directory
14:59:55
drmeister
That will automatically connect udb to clasp and clasp will have generated debugging information for the udb extension I wrote.
15:00:19
drmeister
Also the CLASP_DONT_HANDLE_CRASH_SIGNALS=1 will put you in the debugger immediately when a crash happens rather than in the crash handling code.
15:33:17
Bike
looks like it's in the middle of call-receive-one, doing the funcall, when it segfaults
15:34:24
Bike
i think i'm giving it an untagged function_O pointer? so i guess it would have problems
15:35:51
Bike
if i do `lprint 0x55a520df42b5' that should be a tagged general, right? given that the untagged pointer is that but b4
15:41:51
Bike
if we do want to ue this bytecode compiler more comprehensively we will have to really improve its debug generation
15:49:51
Bike
ok. the problem is not that the function is not tagged. the problem is that the function is somehow the symbol progn
16:06:24
Bike
oh, we're still compiling local jumps as exits. that's gonna be the next efficiency problem after cells
16:28:56
drmeister
Look at register and pointers and get a sense of what are real pointers and what are not.
16:31:29
drmeister
The 0x55...5 has a tag #b101 - that's VASLIST0_TAG - hmmm - ok. What does RSP look like? Does it start with 0x55...?
16:38:04
karlosz
Bike: what is the efficiency problem with local exits? what can be done with less work when its local?
16:46:32
Bike
karlosz: jump instead of exit, so just changing ip instead of longjmping and changing ip
16:46:55
Bike
also it would be good if we could delete unused ENTRYs, because otherwise we're going to cons every time we start a loop
16:58:40
Bike
we don't need to longjmp to change the stack pointer. also, in some common cases (like (go ...) as a top level tagbody statement) the stack ought to be just where it was
17:05:58
Bike
but i think there would be a bit of an issue there in that we wouldn't be eliminating the consing entry needs to do for nonlocal unwinds but not otherwise
17:06:26
Bike
for local unwind... i actually don't think entry should _need_ to do anything, since the amount of stack to drop at any local go should be statically determinable?
17:06:55
Bike
so that if you do (funcall (block nil (lambda () (return)))) you get an out of extent unwind error instead of jumping to an expired stack frame
17:07:34
Bike
the other way to do it would be to invalidate on our way out, yeah, but that would be complicated in its own way
17:09:36
Bike
i think the segfault is being caused because i got the jump wrong by 1 in exit-8. gah.
17:09:59
Bike
it should jump to FDEFINITION 9 FDEFINITION 10 bla bla bla, but instead i see CELL-REF FDEFINITION 10, and CELL-REF is 9
17:45:12
Bike
and everything seems to be optimized out so udb doesn't know shit... i thought it would have recorded this stuff
17:53:33
Bike
drmeister: i need a primer or something on lprint. almost every time i use it i get a python error
18:03:58
drmeister
the python extension loads itself with every command - so if you can get into the code you can fix problems.
18:04:45
drmeister
Regarding udb - variables will become available and be optimized out from machine instruction to machine instruction.
18:05:07
drmeister
If you stepi and reverse-stepi or step/reverse-step you can sometimes see variables.
18:05:51
drmeister
You could also compile the interpreter with __attribute__((optnone)) - then you will have lots of debugging info.
18:06:18
Bike
i'm seeing ENTRY-CLOSE REF 7 ENTRY-CLOSE in the disassembly short before this segfault. that sequence is legal but the ref is discarded, so i'm a little suspicious
18:06:52
drmeister
When inspecting a variable value it uses the pc to crawl into the DWARF to find out where that variable value is at the current pc. When optimization is turned on that coverage becomes very, very spotty.
18:15:35
Bike
okay, i manually went through to try to figure out the stack position after each instruction (which capability i should add to the tracer, by the way) and i think the bytecode as written is trying to draw elements past the bottom of the stack frame for the function
19:30:26
karlosz
Bike: i though about sp being statically determinable and i don't think its true (because of multiple values)
19:32:37
karlosz
if we had ENTRY-BIND <frame-slot> and RESET-STACK-POINTER <frame-slot> that would make local unwinding fast
19:38:01
Bike
karlosz: when we hit an entry-close, the stack pointer should go back to where it was just before the corresponding entry, right?
19:39:03
karlosz
i think i debugged this quite thoroughly for the lisp vm if you want a more precise reference
19:39:43
Bike
I have some bytecode here that does ENTRY-CLOSE REF 7 ENTRY-CLOSE. would the compiler ever generate that? i mean, the ref'd data will be dropped, right?
19:49:53
Bike
i think this bytecode (not that part in particular) might be getting generated wrong, but i am not sure
19:50:11
Bike
it's also a bit hard to test on sbcl because all the macros would have to be expanded by hand, maybe
19:52:54
Bike
the runtime problem, at least, is definitely that it's doing a CALL 3 when there are only three things on the stack
20:09:13
Bike
http://ix.io/47YR here's what i'm compiling and the bytecode, with little annotations as i try to work out the stack positions
20:09:34
Bike
so i'm pretty sure the bytecode is in fact wrong, since the append call is missing the middle argument entirely
20:26:52
Bike
if i'm reading the disassembly correctly, the value of (let ((forms ...)) ...) ought to be pushed to the stack in %2, before the first FDEFINITION 4
20:27:06
karlosz
you can have all the functions in the module get dumped out by just scanning for bytecode template objects in the letrals vector
20:27:31
Bike
and i think ref 5 is supposed to be that, maybe? but it's before the entry-close instead of after
20:28:31
Bike
and that's how it would generate, since that entry-close is for the implicit block nil of the do
20:31:46
Bike
(bytecompile '(lambda (f) (list (block nil f)))) has the same problem. now that's something i ought to be able to compile in sbcl
20:34:59
karlosz
Bike: (compile-to-vm::compile '(LAMBDA () (print (BLOCK NIL (RETURN-FROM NIL))))) fails
20:43:35
karlosz
probably we should've just tested if the non-pidgen compiler could compile and run itself
20:44:27
Bike
okay, so compile-return from already compiles its form in a receiving-t context. so now we just need to have compile-block generate a push if it's in a receive-1 context
20:44:35
karlosz
i'll add the push instruction to the main cvm branch and try to see if it can self host.
20:51:05
Bike
i don't know what the sbcl vm is doing, but the disassembly is FDEFINITION 0 ENTRY ... REF 0 ENTRY-CLOSE CALL 1
20:52:17
karlosz
https://github.com/clasp-developers/clasp/wiki/Virtual-machine-design#nlx-instructions
20:52:39
karlosz
Pop a block dynamic environment from the dynamic environment stack (not the VM stack), marking it as invalid.
20:53:07
Bike
i asked "when we hit an entry-close, the stack pointer should go back to where it was just before the corresponding entry" and you said it should go to the sp saved on the corresponding entry
20:53:58
karlosz
we just unconditionally compile the block in an mv context and have entry-close also do unwinding?
20:59:51
karlosz
but for now maybe just pushing and having entry-close muck with the vm stack pointer will work.
21:00:37
Bike
i think it will work. i see what you mean about entry-close not needing to reset the stack though.
21:01:32
Bike
practically speaking, i don't think clasp could implement an exit-single that would be more efficient (i.e. that would not just use the mv anyway) without a total overhaul of its nlx, but it does make sense conceptually i think
21:09:49
karlosz
Bike: i have a compilation strategy that doesn't rely on entry-close also muning with the stack pointer
21:10:29
karlosz
for the case where block is in a 1 value receiving context (like in the above) we make the normal control flow do its usual thing
23:13:14
drmeister
What's the current problem? macrolet? The discussion above looks to be about an optimization.
1:14:37
drmeister
Communication from Lang Hames. I have three issues with LLVM and it looks like it will be a while until they are fixed.
1:14:59
drmeister
Now that I think about it - the bytecode compiler/interpreter will help us with all of these.
1:15:41
drmeister
If we shift code to bytecode compilation then there will be less DWARF code to finalize - so shutdown will be faster.
1:17:28
Bike
the current problem is the bytecode compiler not handling certain nonlocal exits correctly