freenode/#clasp - IRC Chatlog
Search
2:52:56
Bike
also, i fixed up the ast interpreter so that it only maps the ast once, and only converts csts to asts once even when it gives up and uses the compiler
2:53:34
Bike
i'm thinking the "children" relation on ASTs may form an actual tree so we don't need to do the hash table thing for the mapping, though
3:06:30
Bike
but i don't think it's in the "children". maybe it used to be, i remember it being an issue
3:06:54
karlosz
oh, i see.i would have just expected it to be a block tag, rather than the actual full ast itself
3:10:45
Bike
i think i'll look at ensuring the AST is actually a tree, and then this hash table thing can be gone. could speed up a couple different things, really, and i don't see any need to make it not a tree
3:11:23
karlosz
yeah. though its still kinda weird that that is the problem because of how much the compiler uses map instructions with hash tables
3:12:03
Bike
doing some flamegraphs with the new system might be good. it only maps once so the time usage there should be obvious
3:12:06
karlosz
and its not like its the gf dispatch either, because ast->hir basically walks the same thing
3:18:54
Bike
could also save consing by, instead of having the CHILDREN generic function, have a MAP-CHILDREN function that doesn't cons, and derive CHILDREN from it
5:11:56
drmeister
This unwinding problem is biting me bad today - I've lost at least three hours waiting for this docker image with 18 cores! of an iMacPro to compile generated-encodings.lisp.
5:12:30
drmeister
kpoeck: It's not that your code was wrong - it was fine - we have a really bad problem with unwinding that bites us at the worst times.
5:13:12
drmeister
It's been compiling generated-encodings.lsp in this docker image for about an hour now - I'm giving up and going to bed.
5:14:05
drmeister
I don't think yitzi sees these looooooooooooooooong compilations on his docker image.
5:14:24
drmeister
And this same file on the same iMacPro in macOS only takes a few seconds to compile.
7:12:26
beach
If you are new to Lisp, I suggest you learn it first, independently of C++ integration.
7:13:30
beach
I am answering you because most Clasp developers are in the US, and asleep at this time.
7:58:35
beach
You need to wait for the developers to wake up for that. I know very little about C++ integration. I hang out here because Clasp uses the compiler of a project that I started.
7:59:39
beach
But, again, if you plan to use Common Lisp, I strongly suggest you learn it first, independently of C++. Otherwise, you might be very confused.
8:00:00
beach
The semantics of C++ and Common Lisp are very different, so the integration is nontrivial.
8:08:40
minion
kapil_: please see PCL: pcl-book: "Practical Common Lisp", an introduction to Common Lisp by Peter Seibel, available at http://www.gigamonkeys.com/book/ and in dead-tree form from Apress (as of 11 April 2005).
11:35:40
kpoeck
::notify drmeister Can you have a look on https://github.com/clasp-developers/clasp/pull/1012 Calls 60.000 times less read-from-string, so I hope it is faster
12:34:05
Colleen
drmeister: kpoeck said 58 minutes, 25 seconds ago: Can you have a look on https://github.com/clasp-developers/clasp/pull/1012 Calls 60.000 times less read-from-string, so I hope it is faster
12:34:35
drmeister
kpoeck: Thank you - I will implement that soon - but I want us to try and fix this in the compiler.
12:54:40
drmeister
Yesterday I had an issue that bordeaux-threads from quicklisp still refers to mp:lock.
12:55:11
drmeister
There was supposed to be a release of bordeaux-threads to quicklisp. I don't have any idea if or when that happens.
12:56:35
drmeister
2. waf REFUSES to generate output in one very frustrating situation. When I run waf in docker non-interactively - it just hangs when cando runs to build quicklisp code after install
12:57:29
drmeister
3. Clasp, when building generated-encodings.lsp within docker takes almost 700x (SEVEN HUNDRED TIMES) longer than on the exact same iMacPro running macOS.
12:58:09
drmeister
I'm trying to get a prototype of cando out in a docker container with jupyter widgets running so I can build a prototype of an application.
12:59:48
yitzi
drmeister: you are welcome. I've posted updated docker images to the hub. And updated the various branches
12:59:58
drmeister
2. I have spent the morning rerererererereading the waf documentation and then posting a question to the #waf IRC channel.
13:00:51
drmeister
3. Asked Bike and karlosz to develop a compiler optimization that will use setjmp/longjmp to unwind the stack in Common Lisp code that we can guarantee has no C++ code that could cause problems.
13:01:53
drmeister
kpoeck: I will try your new version. It's just that once I get past the 700x slowdown in the docker container I can ignore that problem for a while and hack the docker container so that I can get through the install.
13:02:39
drmeister
Also, I only discovered last night that it was a reproducible 700x slowdown. If that doesn't say FIX ME - I don't know what does.
13:02:46
kpoeck
that why I proposed to make the version that passes the problem to the runtime for the few users that use encodings
13:03:06
yitzi
drmeister: I have other info regarding cando on jupyter, but I can wait til you guys are done resolving this stuff.
13:03:47
drmeister
Nonononono. This problem has come up again and again and again - it's a problem that we have because we need to maintain compatibility with C++ from Common Lisp. I want to fix the problem.
13:05:02
drmeister
yitzi: cl-netcdf was missing and one needs to install lib-netcdf here... this is what I added...
13:05:35
drmeister
I also learned that I do see the output of ./waf install_cboehm but ONLY if I run: docker -it <image-id>
13:07:56
drmeister
I may sound a little unhinged right now - I get that way - it's ok. It's part of how I get things done. I care intensely about getting cando working smoothly and we have put a LOT of work into it. This C++ stack unwinding problem keeps popping at the worst times, in the WORST ways. Up to 700x slowdown out of the blue with no freaking rhyme or reason.
13:09:02
drmeister
Well, there is a rhyme/reason. So called "zero-cost C++ exception handling" sucks and the effort put into optimizing it has been highly variable.
13:10:25
yitzi
I am writing a subclass of cl-jupyter kernel for cando. and I just got it to work in the docker!!! see above
13:12:10
drmeister
yitzi: Could you elaborate on what a "subclass of cl-jupyter kernel for cando" is? I thought that's what we had. (sorry to make that anticlimactic)
13:12:37
drmeister
You are running that from the command line - I've never seen that before. But I'm missing a finer point.
13:14:10
yitzi
Right now you are hooking into hooks that you guys added to cl-jupyter to do code evaluation and start the kernel in "cando-jupyter". In order to do that in my kernel you need to subclass "common-lisp-jupyter" and handle the appropriate methods.
13:14:50
yitzi
https://github.com/yitzchak/cando/blob/clj-migrate/src/lisp/cando-jupyter/kernel.lisp
13:15:49
drmeister
I see - in our approach with cl-jupyter the whole program was a jupyter kernel but we used crude hooks to achieve that.
13:16:06
drmeister
You have a jupyter kernel class and you subclassed it for cando. Yes - excellent.
13:16:34
drmeister
FYI: We have two kinds of kernels that we could generate - and one is more important than the other.
13:17:44
drmeister
We use what is called "leap script" and it let's you invoke Common Lisp code by putting it in parentheses.
13:18:27
yitzi
https://github.com/yitzchak/cando/blob/cb84c9da6901a814dd7861d1214c242c23035031/src/lisp/cando-jupyter/kernel.lisp#L52
13:18:54
yitzi
And yes that is what we do in maxima-jupyter, subclass the common-lisp-jupyter kernel.
13:19:00
drmeister
We do have a little bit of an issue with case. Leap script allowed mixed case and was case sensitive for variable names.
13:19:55
drmeister
I hope no one in the last 27 years defined variables like foo, Foo, fOO and so on - within a single script.
13:21:02
drmeister
yitzi: I want to retain that post_install thing. I know the expedient thing is to remove it - but the cando build system needs to run cando once after it is installed to build all the quicklisp code.
13:22:02
drmeister
When I run docker build - there is no output generated by the post_install command - are you experiencing that?
13:23:07
drmeister
I've run: docker -it <image-id> several times now and then ./waf install_cboehm and I got it to build all the way through.
13:24:04
drmeister
It just occurred to me that we can force it to exit with a backtrace if it hangs.
13:24:21
drmeister
I'm going to change the wscript file and at least then it won't hang in the debugger.
13:25:06
yitzi
Ok, I'm going to go work the cando kernel so more. Hopefully in an hour or so I'll have a working docker with cando and jupyter.
13:27:44
drmeister
Bike: yitzi's common-lisp-jupyter code uses 'ironclad' - it compiles pretty quickly on clasp now.
13:29:58
drmeister
Here's the evidence - although no timing. I'm building a docker image and it built ironclad in just the last feew minutes.
13:32:26
drmeister
Sorry - in this docker container we install yitzi's jupyter code using pip3 and that invoked sbcl.
13:33:51
drmeister
I was just puzzled by two things but I glossed over them. 1. yitzi took out ironclad as a dependency of his common-lisp-jupyter code for clasp because clasp profides the hash generating facility that we would otherwise need from ironclad.
13:50:00
drmeister
yitzi: I've pushed a bunch of changes to the Dockerfile to my fork. Our two Dockerfiles now won't merge automatically.
13:55:42
drmeister
yitzi: We will have to do it by hand. I went through this because there are some things I wanted to make sure were in there.
14:04:01
drmeister
Bike: If we use setjmp/longjmp - we are going to have to do something with the map functions - correct?
14:04:23
drmeister
I am thinking 1. implement them in Common Lisp or 2. make sure they don't use RAII
14:06:16
Bike
it's okay for there to be intervening C++ frames, so long as they don't have catch blocks or nontrivial destructors that need to be run
14:16:20
Bike
maybe C++ has some kind of annotation so the compiler can check that. but that would probably be too convenient
14:22:18
yitzi
drmeister: Sorry, but your email about the jupyter mockup and problems with radio buttons was in my spam. just saw it
14:22:34
drmeister
Do you think this will work? An optimization to detect when we can replace C++ exception handling with setjmp/longjmp?
14:23:02
drmeister
yitzi: No worries. You demonstrated radio buttons work in your latest docker image.