freenode/#clasp - IRC Chatlog
Search
18:53:28
drmeister
It seems to me that there is nothing fundamentally wrong with building the image at startup the way we do - but we don't have the right definition of a compilation-unit.
18:55:22
drmeister
Can you summarize again the problem with your make-instance optimization? What is out of place?
18:58:07
Bike
drmeister: it's been a while, so i don't remember the particular issues i was running into at the end. might have been related to the structs i used having their own constructors
19:09:48
drmeister
In the 'allinone' branch there is a version of Clasp that compiles cclasp by compile-filing all of the source files as if they are one from the point of view of compiling lexical variables.
19:10:54
drmeister
If we add the ability to compile classes as literals - then the class definitions will be created the first time they are needed and then on they would be referenced.
19:11:26
drmeister
The first time I tried it - it used a lot of memory and crashed my laptop. So I'm trying it on your machine.
19:12:02
drmeister
I don't have the classes being treated as literals yet - that would be next if I can get the thing to work at all.
19:12:34
drmeister
But if this were in place - would it solve all of our problems. I've asked this before - but I'm describing it again after mulling it over for a few days.
19:13:28
drmeister
The problem now is that an inline definition in file #5 needs Cleavir AST classes that are defined in file #350
19:15:14
drmeister
And each separate file generates its own table of literals and so even if we compiled classes as literals - they wouldn't be set up at the right time unless we compile everything as one big compilation unit.
19:16:48
drmeister
That's why I'm asking about your make-instance optimization - that ran into a startup problem. It's a useful complementary case to the problem with inlining.
19:18:13
drmeister
If I'm wrong about this allinone+literal-classes idea then I want to ditch it and move to saving core files.
19:19:49
Bike
core dump is more like how i'd want it to work. i meanideally i don't see why we have to run through a million forms assigning function names to functions and so on
19:20:08
drmeister
I did not appreciate the bootstrapping problems that would come with the approach that I chose for starting up the system.
19:22:55
drmeister
Going the core dump route means the following (1) We probably need to get rid of the boehm GC support (2) We need help from Ravenbrook to implement it (3) We need to GC code.
19:28:20
drmeister
(4) We wouldn't compile-file the system files. We would LOAD the source code into memory and the top level forms would be compiled/JITted as they load. The JITted modules would be in the MPS memory and so they would be treated like everything else.
19:29:30
drmeister
(5) We would need to figure out what side-effects in C++ are taking place in our current startup and run them before we start the system back up again.
19:31:35
drmeister
This (1) I've got a great idea to speed things up (2) I'm implementing it (3) Oh shit - the way we startup prevents me from deploying it because of <insert complicated bootstrapping explanation here> is absolute bullshit.
19:38:41
Bike
ok, cut out a few bogus source infos, so trying to M-. to a function defined in the repl properly says it doesn't know the source
19:38:47
attile_lendvai
drmeister: in bits.cc there's some templated version of the bitarray operations. are they really worth it performace-per-complexity wise? if i understand correctly it only saves a couple of memory dereference
19:42:20
drmeister
Saving dereferences in tight loops is important isn't it? That's why I implemented that.
19:43:43
attile_lendvai
drmeister: did any of those show up in a profiling session? because it's quites some extra complexity...
19:44:56
drmeister
How about we move that code into a wiki page so I can remember how to implement the template version if I ever need it.
19:45:33
drmeister
I figured out yesterday there is a big chunk of llvm time that I wasn't counting when I count llvm time.
19:46:42
drmeister
It turns out the inline pass takes about 80% of the time when I (compile nil '(lambda (x y) (+ x y))) repeatedly
20:33:39
kpoeck
I spent some time to make CL_DEFUN T_sp core__bit_array_op(int opval, Array_sp tx, Array_sp ty, T_sp tr) more correct
20:36:29
drmeister
kpoeck: We are thinking of throwing out the template version - you are talking about the non-template version - correct?
20:40:11
drmeister
This is consistent with what I've been seeing for the past two days. llvm time is substantial - about 50%
20:53:17
drmeister
When I invoke TIME from within clasp-builder.lisp - I don't see any TIME output. It's supposed to send it to *trace-output*. I'm thinking that's the same as *standard-error* and maybe waf has rerouted that?
20:53:43
drmeister
If anyone else has any other insight why I get no TIME output - I'd love to hear it.
20:54:44
drmeister
It will do that - and then it may double when it tries to link everything. For my laptop that was an extinction level event and I didn't see the other side of it.
20:55:35
drmeister
(defstruct (xxxx (:type vector) :named) ...) makes it pause for 10-20 seconds! But other than that it's chugging along.
21:42:59
drmeister
I should probably switch back to bitcode .bc files rather than human readable .ll files.
22:11:52
drmeister
The allinone build generated a .o file - so it works - but it's a lot less relevant now that we are looking towards dumping image files.
22:20:24
attile_lendvai
::notify kpoeck do you mean the changes that are already checked in the repo? if yes, then of course, that'd be a major offense... :) if you mean changes that are not in dev, then do warn me
1:57:32
drmeister
Loading and compiling foundation.lsp with the JIT - with no llvm optimization is 1.039 seconds
2:04:37
drmeister
At 0 - there is no inlining and no other optimization - just native code generation.
2:39:09
drmeister
I compile-file 69 source files with optimization level 3 using code compiled at optimization level 3 -->
3:00:03
drmeister
I'm going to try a different trade-off. Knock *optimization-level* to 0 and build everything.
3:10:10
drmeister
I might be able to do inlining in a different way - more deliberate inlining at call sites.
3:10:50
drmeister
But still - we are balanced right now with the bclasp compiler - where the it takes about half the compilation budget and llvm takes the other half.
3:12:44
drmeister
I figure if I compile everything with *optimization-level* 0 and then compile-file the 69 source files at optimization level 3 (as above) - that will be a decent comparison of optimized code vs unoptimized code.
3:53:48
beach
karlosz: &key processing, if performance critical, could be done with a compiler macro.
3:54:24
karlosz
beach: can that be done for local functions too? this is in the context of inlining local functions
3:55:32
beach
karlosz: I don't expect many local functions to 1. be performance critical and 2. to have keyword arguments.
6:28:16
Colleen
kpoeck: attile_lendvai said 8 hours, 7 minutes ago: do you mean the changes that are already checked in the repo? if yes, then of course, that'd be a major offense... :) if you mean changes that are not in dev, then do warn me