freenode/#clasp - IRC Chatlog

22:40:59 kpoeck drmeister could you please check https://github.com/clasp-developers/clasp/pull/1012

22:41:18 kpoeck Does not call read-from-string in the loop and is much quicker for me

22:53:29 karlosz yeah, there really is no way to contify that terminate-token function, since it's not really possible to have two functions share the same body in HIR or LLVM IR

22:53:57 Bike there are some lisp special operators that still use intervening C++ functions, like progv and complicated multiple-value-call

22:54:37 karlosz Bike: those C++ functions don't invoke exceptions though, right?

22:54:42 karlosz so wouldn't it be fine

22:54:49 Bike progv has cleanup to do

22:54:53 karlosz oh no

22:54:57 Bike multiple-value-call... maybe

22:56:02 karlosz well, actually, the call-position analysis should sitll work

22:56:17 karlosz because, if i understand correctly, you pass first class local functions to progv and m-v-c

22:56:22 Bike that's right.

22:56:55 Bike you say "call position", but things like (flet ((foo ...)) (bar (lambda () ... (foo ...)))) have to be ruled out, right?

22:57:22 karlosz call-position, as in transivitevly including every local function that encloses a RETURN-FROM

22:57:44 karlosz as long as the return-from cannot be captured and handed to a C++ function

22:58:39 Bike i mean in t his case foo i'd say foo is in "call position" but i don't think we could use set/longjmp

22:58:46 Bike "in this case i'd say"

22:59:10 karlosz yeah, foo is in call-position, but it's "inside" a local function that's not in call-position

22:59:16 karlosz i.e. the (lambda () ...)

22:59:22 karlosz that's sort of what i mean by transitive

22:59:22 Bike right, ok.

22:59:44 karlosz it's not that straightforward though, because DEFUN itself introduces an enclose

23:00:06 karlosz and it seems like all the toplevel defun stuff likes to introduce lotsa encloses

23:00:25 karlosz encloses that escape

23:00:51 Bike is that relevant? that doesn't seem relevant.

23:01:12 Bike i mean once you get higher than the BLOCK it shouldn't matter.

23:01:34 karlosz oh, true

23:02:10 karlosz i guess it's harder for HIR to know what "higher" than a BLOCK means, but should still be possible by analyzing dynenvs

23:02:25 karlosz dynenvs sort of encode lexical scoping in the graph

23:02:47 Bike yeah, i mean, for this case probably all you need to know is that the dynenv of the call is a child of the dynenv of the block

23:03:00 Bike so like, in (lambda () (foo ...)) it terminates at the lambda's dynamic environment output instead

23:03:00 karlosz yep

23:03:55 Bike and depending how sophisticated it is, either there are no bind or unwind protect environments, or that they're all the same

23:03:58 karlosz okay, yeah, that actually succinctly sums up the "transitive" notion

23:04:32 karlosz but maybe it blocks too much, since looking at the "terminate-token", case, it would also block that

23:04:40 Bike what case is this?

23:04:52 karlosz i think there still needs to be a "passed as first class value" criteria.

23:04:54 Bike oh, found it.

23:06:19 Bike hm, well in this case you could go a bit deeper with the analysis. like, terminate-token call dynenvs terminate with read-char-handling-eof's, now how about read-char-handling-eof? and it looks like in that case it does get back to the read-token environment

23:07:27 karlosz yeah, and you'd need to pair that with the passed-as-first-class-value thing to not conflate it with the example you gave

23:08:06 Bike i mean yeah you'd also need to know that the only instructions it's fed to are assignments and bla bla funcall as only the callee.

23:08:26 karlosz yep, that bit is solved though

23:09:19 karlosz i'm thinking it would be pretty straightforward to do this as a pass somewhere in hir or mir, that just change-class's the catch-cont

23:09:56 karlosz then translate-instruction just does its normal unwind thing in the general case, but setjmp/longjmp if the catch-cont is eligible for it

23:10:42 karlosz and of course a clasp only analysis, since it's only C++ that's really unfortunate

23:11:34 Bike it might still be interesting in some cases. for example if you know there are no intervening unwind-protects/binds, in sicl you could skip calling the unwinder function

23:11:39 Bike probably wouldn't be as big a gain, but it's not nothing

23:12:06 Bike anyway, implementing this for the runtime might be annoying. we'll have to figure out what type a jmp_buf has and then probably actually call setjmp and longjmp the C functions

23:12:52 karlosz yeah, i was looking into how to actually write the llvm instrinsics

23:13:02 karlosz though of course just relying on the C runtime would work too

23:13:12 Bike my impression was that llvm says to just use the runtime

23:13:18 karlosz i realize now that change-class is a great way to bypass using hash tables

23:13:33 Bike i mean, even with the C++ unwinding there's no actual unwind instruction, you're supposed to call __cxa_throw or whatever

23:13:42 karlosz i'm looking here: https://llvm.org/docs/ExceptionHandling.html#id24

23:14:05 Bike yaeh, that's for the exception handling specifically, so i'm not sure if it'll work

23:14:14 Bike it might

23:15:37 Bike quick check on godbolt has jmp_buf has {[8 x i64], i32, %struct.__sigset_t} which is not a five word buffer

23:15:42 Bike jmp_buf as*

23:16:41 Bike we also might not have the destination address... i think llvm lets you refer to the addresses of code blocks sometimes, but i don't know if that works outside of the jump table thing

23:18:07 karlosz "For SJLJ based exception handling, this intrinsic forces register saving for the current function and stores the address of the following instruction for use as a destination address by llvm.eh.sjlj.longjmp."

23:18:17 karlosz for setjmp

23:18:33 karlosz so it looks like that gives you the destination

23:18:35 Bike oh, yeah, maybe it's the backend doing it, ok.

23:21:22 Bike i'm just a little antsy about using something that's "used internally within LLVM's backend" and is specifically for SJLJ exception handling, which we are not doing

23:21:26 Bike no harm in trying it though

23:22:17 karlosz i don't know which approach requires more runtime setup

23:22:27 karlosz i think i'll try and see if the analysis is feasible first

23:22:50 Bike yeah, ok.

23:23:10 Bike man i wish there was just a set stack pointer and jump instruction.

23:23:37 Bike i guess i could try adding it to llvm myself, if i wanna do some real work

23:24:59 karlosz that might make llvm-ir analysis barf

23:25:15 karlosz since i don't think the ir itself supports non local exits

23:25:18 karlosz big shame

23:25:38 Bike it doesn't, that's what i mean

23:26:01 karlosz oh, i see

23:26:05 karlosz that sounds quite ambitious

23:26:13 Bike yeah. i'd rather not.

23:28:37 Bike "returns_twice: This attribute indicates that this function can return twice. The C setjmp is an example of such a function. The compiler disables some optimizations (like tail calls) in the caller of these functions." that's a point i hadn't considered

23:28:57 Bike i guess you can't reuse the stack space for a tail call since the code after the second return will still want access to the frame

23:30:25 karlosz you couldn't do that in lisp either

23:30:31 Bike right

23:30:35 karlosz the semantics actually match catch and unwind quite closely

23:30:38 drmeister We can reimplement progv and multiple-value-call

23:30:42 drmeister Hello

23:32:26 Bike yeah ok, since returns_twice is the only annotation for this and it's only for functions... yeah, that's difficult, since you'd want the same mark for the equivalent to catch-instruction

23:35:25 karlosz is it hard to emit annotations? or what do you mean by that

23:35:55 Bike i mean that returns_twice on a function is the only way llvm has to indicate that it can't do tail calls and stuff, as far as i can tell

23:36:15 Bike so you'd need anything that checks for a returns_twice call to check for this instruction too, i guess?

23:36:21 Bike i don't know much about the internals of llvm passes

23:38:16 karlosz oh, you mean because it's just a bare instruction and not a function that you can annotate return_twice on

23:38:27 Bike yeah

23:40:47 karlosz bleh

0:07:58 karlosz a macro like do-nested-dynenvs would help

0:48:35 yitzi drmeister: you around?

1:07:29 drmeister Hello

1:08:00 yitzi Just got cando to run in docker

1:08:14 drmeister How did you get install_cboehm to work?

1:08:41 drmeister It is saying: WafError: Could not create the directory /usr/local/lib/clasp

1:09:35 yitzi I removed the line that executed cando in the root user and forced all quicklisp activities to be in the user account

1:10:01 drmeister This is the wscript.config...

1:10:03 drmeister https://www.irccloud.com/pastebin/NsM7cz83/

1:10:55 yitzi I'm talking about the post_install in cando

1:11:04 yitzi https://paste.gnome.org/pux1cz7x7

1:11:25 yitzi line 48

1:12:04 yitzi I'll have to write a custom installer or tweak cl-jupyters installer since it thinks that cando is clasp.

1:12:45 drmeister I create a kernel file for cando and another for clasp

1:13:01 drmeister I'm trying to understand what your sed is doing

1:13:23 yitzi Removing the line that executes cando. It's a serious hack

1:13:33 yitzi replaces it with a print

1:13:48 drmeister Oh!

1:17:16 yitzi I'm probably done tonight. I'll detangle the dockerfile and fix the kernel installer tomorrow. Just thought you'd like to know that progress is being made.

1:19:05 drmeister Yes - thank you.

1:19:16 yitzi no

1:19:23 yitzi prob

1:19:42 drmeister I think I'll have to start my build again and get it to dump to /opt/clasp

1:19:47 drmeister install to /opt/clasp

1:20:00 drmeister I'm wracking my brains to avoid building the whole thing again . argh.

1:20:17 yitzi Actually it is using /usr/local ... I was wrong earlier

1:20:32 drmeister No worries - I see that now.

1:21:09 drmeister It means I have to go through the whole rigamarole to set up /opt/clasp and set permissions and so on.

1:22:30 yitzi I think /usr/local seems to work. The symlinks seem to work better.

1:27:39 drmeister Alright - now I'm getting my Docker legs

1:36:59 karlosz okay, i coded up a slightly more conservative analysis for when to do setjmp that should be correct

1:37:14 karlosz basically the idea is to hijack add-intermediate-functions from segregate-lexicals

1:38:34 karlosz Look at every catch-instruction. Look at every unwind that uses that catch-instruction. If every intermediate function of the catch and the unwind are in call-position, there is no problem

1:38:47 karlosz it's pretty conservative, but still captures cases like the terminate-token thing

1:39:32 karlosz it's simple and can be improved later, but now the next step is how to actually emit setjmp and longjmp

1:41:59 karlosz so, anything that is of the class "escaping instruction" should fall back to using c++ unwinding

1:42:24 karlosz https://paste.gnome.org/pexp1cjza

1:42:58 karlosz that... will mark block f as being escaping... https://paste.gnome.org/pjgfpckes

1:43:11 karlosz but not this: https://paste.gnome.org/pkeo4ipqq

1:56:02 Bike doesn't look like it knows about binds or unwinds at all, is that right?

1:56:08 Bike unwind protects

1:57:02 karlosz the output of a catch-instruction can only get unwound to

1:58:02 karlosz is bind relevant?

1:58:32 Bike if there's a bind between the block and the return-from it needs to be undone, no?

1:58:34 karlosz i guess i didn't think about how clasp implements bind

1:58:44 karlosz which is with... C++

1:58:47 karlosz ?

1:59:06 karlosz oh, i see

1:59:20 Bike i mean, it's the same with unwind-protect, the protection functions need to be executed on the way out

1:59:23 karlosz well, the thing is, this is only marking which catches escape

1:59:34 karlosz it doesn't mean stop doing whatever cleanups you need to do

1:59:47 Bike sure.

1:59:55 karlosz i thought we were just handling the cases where we could substitute in setjmp for unwind

2:00:18 karlosz does c++ unwind also handle specials binding?

2:00:25 Bike i mean, you can still do setjmp, you just have to do the unbind stuff first

2:00:29 Bike yes it does

2:00:46 Bike we could just stick to cases with no intervening binds or protects, that should be statically detectable

2:01:08 Bike doing them in a different dynamic context may be difficult, the way they work now is C++-like and not amenable to it

2:01:36 karlosz alright...

2:02:11 karlosz so then yeah, i guess it has to look at dynenvs

2:52:56 Bike also, i fixed up the ast interpreter so that it only maps the ast once, and only converts csts to asts once even when it gives up and uses the compiler

2:53:05 Bike but it's still a bit slower than just using the compiler on asdf

2:53:34 Bike i'm thinking the "children" relation on ASTs may form an actual tree so we don't need to do the hash table thing for the mapping, though

3:05:57 karlosz Bike: it is an abstract syntax "tree" after all

3:06:13 Bike yeah, but in some aspects it's not an actual tree

3:06:20 Bike return-from-ast refers to the block-ast, for instance

3:06:30 Bike but i don't think it's in the "children". maybe it used to be, i remember it being an issue

3:06:54 karlosz oh, i see.i would have just expected it to be a block tag, rather than the actual full ast itself

3:07:10 Bike the tag actually doesn't survive into the ast

3:07:16 Bike though maybe it should, if only for debugging

3:07:24 Bike referring to the ast itself is kind of easier

3:07:50 Bike maybe it was lexical asts that are a problem

3:07:50 karlosz to me it feels like it should be treated like a variable ast

3:10:45 Bike i think i'll look at ensuring the AST is actually a tree, and then this hash table thing can be gone. could speed up a couple different things, really, and i don't see any need to make it not a tree

3:11:06 Bike maybe if you have code with #n# in it, but fuck that

3:11:23 karlosz yeah. though its still kinda weird that that is the problem because of how much the compiler uses map instructions with hash tables

3:11:38 karlosz maybe asts are bigger than hir? that's also surprising

3:11:45 Bike yeah, i dunno.

3:12:03 Bike doing some flamegraphs with the new system might be good. it only maps once so the time usage there should be obvious

3:12:06 karlosz and its not like its the gf dispatch either, because ast->hir basically walks the same thing

3:12:13 karlosz yeah

3:14:12 beach Good morning everyone!

3:14:55 Bike good morning

3:18:54 Bike could also save consing by, instead of having the CHILDREN generic function, have a MAP-CHILDREN function that doesn't cons, and derive CHILDREN from it

3:21:53 karlosz i think you're right that the flamegraph will lead to the answer here