freenode/#clasp - IRC Chatlog
Search
14:10:41
drmeister
Bike, karlosz: Firefox is much better at rendering those complex svg flame graphs
14:14:29
drmeister
yitzi is rapidly bringing jupyter notebooks/widgets/lab online and we have a really good idea of how we are going to use it to make user interfaces.
14:14:54
drmeister
Bike and karlosz are making great progress in improving the underlying compiler that everything builds on.
14:15:36
drmeister
Bike: What do you think about the setjmp/longjmp idea now that we've had a few days working on it?
14:18:05
Bike
the intrinsics are documented incorrectly and are listed as only being generated by internal llvm passes
14:19:29
drmeister
Build a small test case and then compile it with just the c-preprocessor. Do you know how to do that?
14:24:01
drmeister
Sure - but nobody is going to fundamentally change it or there will be heck to pay.
14:25:28
Bike
the jump buffer type is also defined by the C implementation, which is another wrinkle
14:26:32
Bike
yes, i'm nervous about some random change in glibc causing bizarre hard to reproduce unwinding crashes
14:27:03
drmeister
We can put in a sanity check to make sure that the invariants remain so. Also, keep the code in one place. Then we leave it for the future to deal with.
14:27:46
drmeister
The only other way out of this is to figure out what is going on in the kernels where we see crazy slowdowns and try and get control there.
14:30:32
drmeister
Also - can you set this up so that it's optional? So we can turn it off if we do run into problems?
14:31:35
Bike
for a start i was thinking i'd do a really stupid implementation that calls an intrinsic that uses setjmp. it will be way more inefficient than it should be, but going off my initial test, exceptions are so very terrible that it'll still be a win
14:32:01
Bike
plus it shouldn't implicate thread anything, but who knows, maybe libc will find a way
14:55:22
yitzi
drmeister: no difference between lab and notebook from your widget code perspective. To answer your question from yesterday
14:56:24
yitzi
That image also has lab in it btw. I just set notebook to be the default since the nglview kernel stuff doesn't work with lab yet.
14:56:25
drmeister
So if we develop widgets in jnotebook then we can port them to jlab once nglview works in jlab.
14:57:32
drmeister
Second question: There is a long pause when the accordion widget starts up. Do you see that as well? Have you any experience debugging issues like that?
14:59:02
drmeister
This is why it's good that we have an sbcl kernel as well. We can compare the two for performance.
15:00:55
drmeister
I run docker with this now: docker run -it -p 8888:8888 -p 4005:4005 -v $HOME:/home/app/home yitzchak/cando-clj;nglview
15:04:40
yitzi
Yep. The molecule spins fine, but the jupyter message indicator turns black b.c. the clasp is getting overwhelmed?
15:08:02
yitzi
I haven't done any attempt to look at the speed of the kernel code, mostly because SBCL seemed fine. So it could need some analysis/optimization?
16:20:39
yitzi
drmeister: I think I found the blocking issues in nglview that prevents loading in Lab. If I'm right then we could use lab with nglview now.
16:31:09
yitzi
It means that our code won't be at 2.7.5 so some stuff might break, but the widget appears to load. They added some new traits that Javascript expects to be there.
16:31:57
drmeister
The guys who are developing nglview have occasionally changed the names of functions in the Javascript and that requires changes to the kernel code.
16:33:23
drmeister
That's why it's good to keep track of the python code that we match so we can diff it against the python code that they develop and then we make corresponding changes to our Common Lisp code.
16:34:09
yitzi
I am testing it in docker right now, if it works you will have jupyterlab, nglview, and .................
16:36:12
drmeister
Good job figuring that out by the way. With everything going on I'd forgotten about that.
16:36:51
drmeister
yitzi: We are Amber developers and we have access to the Amber development code. But the Conda one should be ok.
16:37:11
drmeister
There needs to be an environment variable set so that cando knows where Ambertools is.
16:39:21
drmeister
Once yitzi gets this docker image together I think we should integrate his Dockerfile changes into our "deploy" script.
16:39:47
drmeister
yitzi: We have a thing we call the "deploy" script. It builds everything on our fledgeling CI system:
16:41:24
yitzi
drmeister: Keep in mind that I don't know if all of the common-lisp-jupyter, nglview, cando-jupyter works yet. I don't really think it is ready for mass consumption yet. It's getting close, though.
16:51:53
yonkunas
yitzi: please use ambertools20. We use /opt/amber as the install directory for the distributor so $amberhome top directory should be amber not amber20. This is all easily changeable I assume
16:52:41
drmeister
yitzi: No worries - this isn't going out for mass consumption. I'm trying to get this to a prototyping tool.
16:53:29
yitzi
drmeister: about to open a WIP PR for cando-jupyter so you can see the current changes. Not ready for merge.
16:59:10
yitzi
drmeister: the WIP cando-jupyter PR is here https://github.com/cando-developers/cando/pull/9
17:00:21
drmeister
What is "AmberTools interop" and "code inspection/completion"? Code completion I know.
17:01:03
drmeister
Regarding AmberTools interop - if it's what I'm guessing it is. In the "deploy" script that we are developing we are going to copy the Ambertools way of deploying a separate Python environment.
17:01:46
yitzi
AmberTools interop = make sure that all the ambertools stuff works from the updated cando-jupyter kernel
17:02:57
drmeister
The way that cando works with Ambertools is cando looks at the "AMBERHOME" environment variable and creates an "amber" hostname within cando. Then it refers to #P"amber:xxx" pathnames to find things.
17:04:17
drmeister
Ambertools provides 'sqm' - it's a semiempirical quantum chemistry package that carries out QM calculations that we need.
17:05:17
drmeister
We also call out to 'pmemd.cuda' - but that is not provided by Ambertools. That is part of the larger "Amber" package that includes Ambertools and provides the "pmemd.cuda" application for simulating molecular dynamics.
17:06:57
drmeister
pmemd.cuda is used in our "distributor" application. The "distributor" is a distributed "make"/scheduling facility that distributes calculations across heterogeneous computer systems.
17:07:44
drmeister
The jupyterlab environment and the widgets that we will assemble will provide an easy to use interface to all of this.
17:08:19
drmeister
That's why I want to start prototyping this. I want to get it out of my head and into peoples browsers.
17:10:29
drmeister
If you set up a notebook that has hidden input cells but visible output cells and the output cells are multicomponent jupyter widgets including things like the accordion widget...
17:10:50
drmeister
Then you will see this verticle display of widgets that you can scroll over and interact with.
17:11:18
drmeister
We could lay out the setup of a complex calculation by stringing together a collection of different widget panels that do different things.
17:12:12
drmeister
For a very naive user the input cells can be confusing. So if we fill them in and hide them - then there is less confusion and less chance that they will type something into them and break the notebook.
17:12:49
drmeister
The GenePattern notebooks appear to have lots of markdown to explain what is going on.
17:13:42
drmeister
So markdown to explain what is going on and widgets with hidden input cells are a 'kiosk mode' notebook that sets up the calculation.
17:14:12
drmeister
At any point a more sophisticated user could unhide the input cells - or create a new input/output cell to augment the basic one we give them.
17:15:20
drmeister
From what I've read (and I haven't dug into this too far) you can edit the .ipynb file and change the JSON code in there to hide input cells.
17:16:26
yitzi
https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/hide_input/readme.html
17:16:26
drmeister
-> The codecell hiding state is stored in the metadata cell.metadata.hide_input. If it is set to true, the codecell will be hidden on reload.