freenode/#clasp - IRC Chatlog
Search
12:36:11
drmeister
The compile-file-parallel fails on a macro in babel - I haven't tried to reduce it to a test case.
12:38:06
drmeister
It happens even if I run the thing in serial (counter to what I said above). So breaking up the compiler as I have has a problem.
13:52:28
drmeister
The Department of Energy just announced a $100M program to develop new desalination technologies.
13:52:28
drmeister
https://www.energy.gov/articles/department-energy-announces-100-million-energy-water-desalination-hub-provide-secure-and
13:53:44
drmeister
We are developing our own technology that looks more like lfarm - for distributing jobs across large clusters of machines.
13:57:27
drmeister
Sadly, no. Bike and I will take a look at compile-file-parallel tomorrow - it almost works - but there is something wrong with how I'm splitting up the ast generation and the subsequent steps. There's a form in babel that causes an assertion failure in cleavir.
14:00:50
drmeister
stassats: Do you every go looking for thread contention? I was going to try some of the tools in xcode to see if I can get any insight why compile-file-parallel doesn't go higher than about 200% CPU.
14:02:25
drmeister
It might even be malloc - google has this tcmalloc that is supposed to give better multithreaded performance.
14:05:36
drmeister
How do I determine GC impact. Regular profiling and look at how much time is spent in the gc? I can do that no problem.
14:34:08
drmeister
lparallel works great - I'm running multiple quantum mechanics calculations in parallel
15:03:00
scymtym
drmeister: where do the results go? are there side-effects for collecting the results?
15:04:20
drmeister
I write out a file and then use ext:system to launch a program that does the calculation and reads it and then writes the results to another file.
15:05:08
drmeister
But yes - like stassats implies - until I run 'ls' on the directory the results of the calculation are a quantum superposition of run and not run.
15:08:01
drmeister
I'm calculating partial charges for compounds out of this database: http://zinc.docking.org/browse/subsets/
15:09:59
scymtym
drmeister: i see. i was asking because one of the strengths of lparallel's PMAP* functions is the fact that results are returned in the same way the CL counterparts would return them
15:12:49
drmeister
I'm comparing the results of running this stand along executable to results that Cando generates. With Cando I will use that because it does the calculation within Cando.
15:15:02
drmeister
lparallel looks pretty straightforward. I'm going to keep muddling along with it.
15:17:41
drmeister
Like if I want to shut down the calculation because it's going too long or I want to restart it with different arguments.
15:19:57
drmeister
Right now, clasp doesn't handle Control-C or interrupts well, I'm looking into it.
15:20:35
drmeister
When I start working with Cando seriously in the jupyterlab interface - almost immediately after starting up the first calculation I want to interrupt it and change the arguments and run it again.
15:21:02
scymtym
i think the safest solution is having the tasks cooperate by, for example, checking a flag