freenode/#clasp - IRC Chatlog
Search
11:35:15
drmeister
I have some folks who want to install on macOS and it would be good to simplify the instructions.
11:36:22
drmeister
stassats: You make a very good point. I just need to point the build system at the brew installed llvm-config.
11:37:10
drmeister
I was thinking that I wouldn't have externals-clasp installed in that account - but I can ensure that it isn't used in my account by changing the wscript.config.
12:10:14
drmeister
::notify attila_lendvai I made a change to the parallel scraping code - so that it always does 8 parallel scraping jobs. It looks like every time we pass a different -j xx argument to ./waf it starts scraping everything all over again. My compromise hack is to stop that from happening. We need a better way
12:11:36
drmeister
::notify balrog - right you said that we would need to also link in libffi - digging deeper.
12:17:55
drmeister
::notify balrog I'm going to add libffi as a dependency on macOS if we can build clasp with brew llvm@5
12:51:26
drmeister
Argh - in waf you name the library FFI and then you append to LIB "LIB_FFI" to get it to link - otherwise it silently fails.
13:05:21
drmeister
::notify balrog I can now build and link iclasp-boehm on macOS after brew install llvm@5 libffi - thank you so much for your advance work on this.
13:05:50
Colleen
balrog: drmeister said 54 minutes, 14 seconds ago: - right you said that we would need to also link in libffi - digging deeper.
13:05:50
Colleen
balrog: drmeister said 47 minutes, 55 seconds ago: I'm going to add libffi as a dependency on macOS if we can build clasp with brew llvm@5
13:05:50
Colleen
balrog: drmeister said 29 seconds ago: I can now build and link iclasp-boehm on macOS after brew install llvm@5 libffi - thank you so much for your advance work on this.
13:06:26
balrog
The problem with adding libffi as a dependency is that if someone has compiled their own LLVM without libffi, that will lead to problems
13:07:25
balrog
https://github.com/llvm-mirror/llvm/blob/76b46df975a33d6a5ce0c606f09dfa88e195d243/tools/llvm-config/llvm-config.cpp#L702
13:07:57
balrog
It’s an llvm-config issue that’s been known since that source code FIXME was added, a long time ago.
13:08:48
balrog
The only way to programmatically find out if it’s needed is using otool to check if the lib is linked against libffi. Or maybe compile and attempt to run a test program (that’s the autotools way right?)
13:11:11
drmeister
waf does run a test build with each library - but this is a bit different - we want to build a test file linking llvm and if it fails, add libffi as a dependency?
13:13:15
balrog
I’m not sure if there’s a specific lib you have to link in — when I checked with otool it seemed the main llvm lib needed libffi
13:21:33
drmeister
I couldn't run the tests that I wanted last night because I didn't have your modified sicl
13:24:21
Bike
yeah what's with that? all power goes off for five seconds some time in the next eight hours?
13:28:21
drmeister
I am too much of a molecular biologist at heart - molecular biology is all about suppressors of suppressors of suppressors.
13:29:40
Bike
well it would make more sense if we did have some map kinase kinase kinase kinase floating around but in this case there's only one level
13:38:34
balrog
drmeister: otool -L /usr/local/opt/llvm@5/lib/libLLVM.dylib: (blah...) /usr/local/opt/libffi/lib/libffi.6.dylib (compatibility version 7.0.0, current version 7.4.0) (...blah)
13:40:07
balrog
just compiling a test program and linkiung the lib isn't enough, though. let me dig a bit deeper
13:44:07
balrog
(btw unrelated/offtopic, I read "Ignition" by John Clark over the past week, it's ... a pretty fun read)
15:30:41
karlosz
you probably figured it out by now, but the only incompatible change is having to rename first second and third on blocks to their semantic names
15:38:02
drmeister
I'm going to change the wiki installation page for macOS to refer to brew install llvm@5 libffi
16:07:51
balrog
the reason that not everyone runs into this is because it only happens if llvm::Interpreter::callExternalFunction() is called
16:13:21
drmeister
I updated the installation instructions: https://github.com/clasp-developers/clasp/wiki/Build-Instructions
16:14:09
drmeister
On macOS it uses brew install llvm@5 libffi - I left the libffi in there - I guess I can take that out now.
16:24:52
frgo
I just read the log. drmeister: Re the nr of scraper jobs: You said you pinned the nr of jobs to 8. Why not use something like https://github.com/muyinliu/cl-cpus to have a more dynamic solution, for e.g. high end workstations having a lot more cores? Yes, i know, one more dependency. If that's not a viable option then why not "steal" the relevant code?
16:28:20
drmeister
I just needed something that didn't keep rebuilding the C++ code whenever I changed the -j option for ./waf
16:29:45
drmeister
I wrote something in the wscript file to do parallel scraping and then attila wrote something else and his (and maybe mine) have this annoying unintended side effect that they force waf to rebuild things that it shouldn't when you change the -j option.
16:30:35
drmeister
Nobody is going to care but me, right now, because I want to get to build problems quickly by using ./waf build_cboehm -j32 and then ./waf build_cboehm -j1 to drop to serial build.
16:31:02
drmeister
Up until I added that hack the ./waf build_cboehm -j1 restarted the entire build!!!! Kinda inconvenient.
16:38:59
drmeister
Because I wanted the build to quickly get up to the point where it fails in parallel mode and then drop down to serial mode.
16:39:20
drmeister
If switching to serial mode restarts the build from the beginning then there is no point.
16:48:14
Bike
What are you eliminating, exactly? Is it that the catch instruction for the 'there' tag keeps the second return-from alive even though it's unreachable?
16:57:20
karlosz
it iterates over every lexical variable in a worklist and if it doesnt have any uses then it removes its defining instruction
17:18:43
drmeister
We were talking about adding value numbering and Kildall to eliminate unnecessary temporaries. Then we talked about how mem2reg converts our code to SSA and that eliminates unnecessary temporaries.
17:19:28
drmeister
Is value numbering a stronger approach to eliminating temporaries? Is that why we don't just convert to SSA and remove useless store/load's?
17:20:26
karlosz
I rewrote SSA entirely, using basic blocks. the algorithm is essentially straight out of the textbook
17:20:47
Bike
The issue is we'd like to analyze things about closure variables as well, which can't be SSAd in general
17:21:49
Bike
Beach would like the analysis passes to work on variables in general, including ones that have to be put in cells i.e. not ssa-converted
17:25:16
Bike
global value numbering can also be used for common subexpression elimination and stuff,right?
17:27:52
drmeister
So we could run karlosz SSA pass and then lower it to llvm-IR. Cleavir PHI nodes will translate to llvm PHI nodes - correct?
17:28:56
karlosz
i have an unphi pass that makes the allocation explicit, but thats because im using a stack machine layout. it should just be direct with the register machine ir
17:29:09
drmeister
Is there any reason to do this since llvm already does this and removes useless temporaries.
17:30:59
drmeister
How much do we need to rewrite the translate step? It seems like not much. We can use karlosz's basic blocks and we need to add a translate-simple-instruction for PHI nodes.
17:32:34
karlosz
you can use the same variable allocation you had vbefore. all SSA does is make the number of defining instructions strictly 1
17:34:37
drmeister
karlosz: llvm has alloca's - they are memory in the stack frame. llvm also has registers - they are registers.
17:37:43
drmeister
Right now we don't use them between translate-xxx-instruction, we only use them within translate-xxx-instruction
17:40:17
drmeister
Doesn't have to be right away - but I think I see the shape of it - it looks good.
17:42:20
drmeister
I'm supposed to be the cranky conservative one and you are supposed to be the adventurous young spirit that takes us to new heights.
17:43:00
Bike
Right now I'm an adventurous young spirit that's been staring at merge conflicts for two days
17:43:45
Bike
i think beach wants most transformations to run before closure conversion anyway, so we just do 1) do a bunch of shit 2) closure conversion 3) ssa 4) translate
17:53:34
karlosz
im hitting a variable unbound error at compile time for something like (let ((x 1)) (symbol-macrolet ((f x)) f))
18:05:39
karlosz
oh, it only works if i recompile the function. but there shoulldnt be any ordering problems with just defuns and defmethods...
18:30:28
Bike
You said you were in an 'mps' branch? I'll make a clasp-changes-mps branch for you to use.
18:31:07
drmeister
I don't know what the problem is: Could not find waf node for lisp file None - did you run './waf update_dependencies'?
18:32:56
drmeister
It's something to do with: src/lisp/kernel/contrib/sicl/Code/Cleavir/Abstract-syntax-tree/simple-float-related-asts
18:40:48
karlosz
i did have to #-cleavir out a check-special-form-syntax method on 'function in SICL itself though, since there's no system argument to specialize on
20:33:32
drmeister
I spent all morning simplifying the build system so that on macOS it wouldn't require a wscript.config file.
20:34:05
drmeister
Why? I have no freaking idea - for some reason waf on David's machine doesn't recognize what OS it is on.
20:34:46
drmeister
He is running High Sierra with the latest Xcode that became available today. TODAY!
20:35:12
drmeister
If that breaks wafs ability to detect the OS then I have definitive proof that god exists and that he hates me.
20:36:42
drmeister
The rest of you can sort your philosophical conundrums out yourselves - I know my place in the Universe.
20:40:04
Bike
i think letting him deal with a config file probably would have been less error prone than messing with the build system again
20:41:44
drmeister
David says that waf detects what OS it is on by checking the compiler - probably something about Xcode 9.3 or 9.4 broke that.
20:41:58
Bike
...unrelatedly, my netbook is running low on battery, so if i drop off that's the score.
20:42:48
drmeister
I think the link-type has something to do with creating functions and the linkage - can you provide more context?
20:56:58
drmeister
I pulled the latest dev and started building it - it is building in serial mode - have you tried parallel mode?
21:07:10
Bike
i'm hitting a weird build problem when using CSTs and i wanted to use bclasp+cleavir to poke at it
21:08:46
drmeister
balrog: Excellent - give me a few min - I'm going to make a few changes then ask if you could build the new 'dev'
21:09:59
drmeister
Mine says: /Applications/Xcode.app/Contents/Developer which is good - because I am a developer
21:10:26
balrog
(see, the macOS clang/clang++/ld/ar/ etc commands really go to this program called `xcrun`, which then checks the xcode-select path and then runs the appropriate tools in that prefix)
21:16:46
balrog
and one other thing: is his `python` the system shipped python 2.7, or something else?
21:17:07
balrog
(for a time, homebrew symlinked `python` to python3, which led to all sorts of problems.)
21:23:15
drmeister
He's one of the nicest guys I've met - he's still going to think I'm an idiot if I don't know what exactly is wrong. :-)
21:23:47
balrog
considering what kinds of wacky local configs I've seen developers have on their computers.... :D
21:25:19
drmeister
If there is a problem that a wscript.config option can't fix - I don't want to experience it.
21:30:58
balrog
continuum analytics makes it sound like there's anaconda or something in the path... I shouldn't be surprised there :)
21:33:38
balrog
yeah, it works for me after activating anaconda, so I wonder if his anaconda is very much out of date.
21:44:10
drmeister
I have parallel building working - I had to add USE_PARALLEL_BUILD = True to wscript.config