libera/#clasp - IRC Chatlog
Search
18:35:39
drmeister
I have an x86 Macbook Pro on the thirdlaw VPN that can do profiling. I could profile the bytecode interpreter once it works with cvm.
18:48:08
yitzi
I think that removing application.config and project_headers.h has made it easier to add an extension that doesn't use clbind. I tried to make a non-clbind version of lila before and I was getting all kinds of weird syntax errors when it tried to compile anything that used project_headers.h. With this method it likes it was pretty easy and no errors yet.
18:49:20
drmeister
Bike, karlosz: If you do get cvm compatible with the bytecode interpreter at the end of the day - tell me and I'll do some profiling.
18:50:17
drmeister
I could sort them based on inheritance. We don't use multiple inheritance so it should be a simple matter of sorting the headers based on the precedence of classes
18:50:54
yitzi
It could be. I don't really know for sure since I was not able to fix the problem. I was getting all kinds of weird errors from templates. This new method seems to work out of the box.
18:53:10
yitzi
I am letting it run the workflow tests. Everything seems to work but just making sure. I have already tested with boehm and boehmprecise. I am gonna run the static analyzer at some point just to be extra sure that I haven't broken anything.
19:37:26
drmeister
(setf (fdefinition 'fib) (compile-to-vm::bcompile '(lambda (n) (if (eql n 1) 0 (if (eql n 2) 1 (+ (fib (- n 1)) (fib (- n 2))))))))
19:44:00
Bike
also, sidenote, since you made entry points funcallable, you can actually just use compile-to-vm::compile
19:50:46
yitzi
drmeister: I'm note sure I understand the advantage of the class/name pattern from CL_LISPIFY for CL_DEFMETHOD. Doesn't that defeat the point of the specialization?
19:54:44
drmeister
yitzi: atom and residue inherit from matter. (matter/add-matter a-residue an-atom) works.
19:55:26
yitzi
Well, if I have class A and class B both of which have CL_DEFMETHOD A::fu() and CL_DEFMETHOD B::fu() ....
19:56:21
yitzi
Then I want (fu x) to call the right one. I don't understand what a single dispatch A/fu and B/fu help there.
19:56:23
drmeister
I'm thinking of C++ exposed classes as working more like Common Lisp structs than classes.
19:57:55
drmeister
After all this I think it's more valuable to have A/fu and B/fu to make it easier to discover what methods apply to instances of A and B than to be able to write (fu x).
20:01:31
drmeister
Yeah, but listing applicable methods based on specializers isn't useful to programmers.
20:02:40
drmeister
I was looking for something to help me find what methods apply to a particular class with name completion. Where I can type atom/<tab> and get all the atom methods.
20:06:27
drmeister
There are 105 of them that are hard to rename because the same method name is used by multiple classes.
20:11:54
karlosz
OK, here are the easy VM optimizations to make: Not using AREF for the literals vector, and using casting instead of manual arithmetic for reading labels
20:13:05
karlosz
does that use the same mechanism as cclasp in terms fo handling C++ unwinding and longjmp stuff?
20:18:17
Bike
oh yeah, related. when we exit out of a nonlocal block does that hit the entry-close? i think it ought to, maybe?
20:18:37
Bike
since blocks and tagbodies aren't distinguished in the ir, we can't exit the entry as soon as we do an exit
20:19:52
karlosz
like https://github.com/clasp-developers/cvm/blob/a4bc08d869fc0b321c4aeefd46d7ca913d794ffd/compile.lisp#L868
20:20:44
Bike
so we do hit the entry-close? like we hit the exit instruction, and then the next instruction is entry-close.
20:20:46
karlosz
for tagbody https://github.com/clasp-developers/cvm/blob/a4bc08d869fc0b321c4aeefd46d7ca913d794ffd/compile.lisp#L839 you jump to whereever the go tag is which doesn't have to
20:21:09
karlosz
Bike: yes for block return-from, the exit goes to the label right before the entry-close
20:21:31
Bike
and then in tagbody it won't in general, unelss the tag happens to be right at the end of the tagbody
20:22:11
karlosz
i think this is how i wanted it to work for cclasp as well but i think the reason we didn't do it is that llvm can't really express inter-function control transfer
20:22:57
Bike
Yeah, in llvm the block pretty much needs to control where you go next. we can't encode the destination at the return-from/go/whatever like we can in the vm
20:23:48
Bike
llvm nlx is entirely based on c++ exceptions, which suck, and are also very dynamic, so encoding a destination in the jump doesn't make sense
20:25:36
Bike
anyway, i think that's all the instructions implemented except catch, throw, catch-close, progv, long
20:25:52
Bike
so if we do like aclasp and compile catch/throw/progv as thunks, that should about cover everything
20:26:28
Bike
and then we can start trying it on clasp and watch it all burn as the edge cases i haven't thought of come up
20:30:50
Bike
i have not been thinking too hard about it, since aclasp already uses thunks for all of these
20:37:34
Bike
i don't know if that's the problem, but i imagine a million lines of output doesn't help
20:39:48
Bike
the vm might not react well to interruption. the NLX will react appropriately to exits but normal use might not
20:40:45
Bike
i suppose i can make bytecode_call pop the frame in a catch (...) instead of only doing it on nlx or normal return. that should make it work better with exits, i think
20:41:19
Bike
(i moved the bulk of the vm interpreter to another function bytecode_vm, to do the recursive calls thing for nlx)
20:43:13
Bike
if we didn't do that we would (as far as i have thought this through) need to keep track for each PC of what dynamic environments need to be unwound if we exit, so that we can have the C++ code unwind during an alien unwind
20:43:44
Bike
which is more or less how C++ unwinding works, except we'd be using bytecode pcs instead of machine
20:54:56
karlosz_
for non-local exits specifically (not unwind-protect) it would seem like you could just make C++ callouts save the current stack pointer before the call out and then on reutrn check the current stack pointer against the saved stack pointer
20:56:27
karlosz_
yeah, i guess we do need a frame then. but calling recursively shouldnt be a big deal
21:15:23
Bike
although if that's the case i wouldn't really expect recursive calls to work at all, so who knows
22:10:35
drmeister
Bike: (fib 30) was hanging the first time that I tried it - it could be for an unrelated reason such as we are blowing the side-stack. Is that possible yet?
22:11:16
drmeister
If it's not possible (we have 16K words at the moment) (fib 30) should only go like 30 levels deep - right?
22:29:58
Bike
karlosz: i thought i was going to need to use it in exit, which is why i asked about that
22:30:51
Bike
drmeister: should go 30 deep max, yeah... which is why i'm wondering if the stack pointer is being maintained improperly, so that each call leaves the stack pointer higher than it ought to be upon returning
22:36:07
drmeister
CL_DEFUN Integer_sp core__side_stack_pointer() { return Integer_O::create((uint64_t) my_thread->_VM._stackPointer);}
22:39:42
drmeister
Bike, karlosz: Can we avoid taking pointers to anything on the side stack? If we can then we can dynamically resize it.
22:40:23
drmeister
If we must take pointers, use indices relative to the bottom (stack grows up) of the stack.
23:23:33
Bike
drmeister: we use pointers into the stack as vaslists to calls, so no, we kind of can't avoid that (without having to copy everything for that stuff)
23:26:06
Bike
debugging the vm is very pleasant compared to debugging the machine target. i can have it just dump out every instruction no fuss, and we could dump the entire memory state (stack) too if we wanted
23:26:46
drmeister
The problem is down at the bottom GC_obj_kinds[k].ok_freelist[lg] -> 12 = (void *) 0xb9000055e48fbd77
23:27:07
Bike
i guess if the stack isn't maintained we could be using up all memory, which would probably do weird things to boehm
23:31:27
drmeister
Bike: I check the side stack pointer after each call from the command line. If there was creep - it should show up - right?
23:33:08
drmeister
Then I print the difference (- (core:side-stack-pointer) *sp*) whenever the argument to fib is 1.
23:35:59
Bike
wow i misread you. when you said the stack was being maintained properly i read the opposite. oops.
23:45:00
Bike
ok, so other things that could break boehm... well, the vm uses a law of untagged pointers, which is not something i usually do, so i might have screwed stuff up on that front
0:03:20
karlosz
drmeister: like Bike said, we do sort of want stack pointers in the VM instead of indices to keep things fast. for dynamically allocated stacks we could just create a new one twice as big and copy the whole thing and just updating the stack and frame pointer in the vm struct
0:04:25
drmeister
The side stack should be keeping things alive - I set the whole thing as a source of roots.
0:05:09
drmeister
It looks like we are writing garbage into the boehm memory. I see a tagged pointer being written into boehm memory.
0:07:24
drmeister
I'm using posix_memalign( &mem, pageSize, VirtualMachine::MaxStackWords*sizeof(T_O**) ); to allocate it
0:07:45
drmeister
And gctools::clasp_gc_registerRoots((this->_stackBottom),(this->_stackBytes-pageSize)/sizeof(T_O*)); to register the whole thing as a source of roots.
0:08:11
drmeister
I can't currently limit it just to the live stack. That's why I recommended that we stick NULL in each word as we pop words.
0:09:24
drmeister
Actually, there might be a way to limit what part of the side-stack is live. There are ways of making boehm just check a range of memory - but the last time I tried to use it it crashed.
0:09:59
karlosz
yeah... i guess that might potentially cause problems. i had thought that that would just mean keeping objects live at worst
0:10:04
drmeister
I think we need to add more checking to the VM. Check if writing to pointers are all aligned.
0:10:40
karlosz
but i suppose boehm won't like it if an object in the dead part of the stack gets overwritten by something else
0:19:33
karlosz
just fixed jump-if-supplied for variable length labels so lambda lists should work again
0:49:31
drmeister
I don't think garbage on the side-stack will hurt anything. Boehm is very, very tolerant
0:52:20
drmeister
I think it's more likely that we are writing into a tagged pointer or a tagged pointer when we should be writing an untagged one.
2:10:11
Bike
for the exit stuff i made a tiny little new structure VirtualMachineStackState that includes the frame pointer, and ENTRY uses it to restore the stack pointer after a nonlocal exit
2:10:29
Bike
could it like... get reclaimed wrongly by boehm somehow... i guess not, since fib doesn't even do nonlocal exits
2:15:38
Bike
maybe there's a size_t overflowing into a stupid high number, and that's ending up in push_frame or something
2:41:25
drmeister
I do say this though: static constexpr size_t MaxStackWords = 16384; // 16K words for now.
3:41:03
drmeister
I evaluate (fib 33) and print the stack for every instruction and there's no problem.
4:16:32
drmeister
I evaluate (fib 33) and the stack stays at 1000 bytes or so all the way except at the end it balloons.