freenode/#clasp - IRC Chatlog
Search
14:04:17
Bike
defstruct has been slow to compile for a while. dunno about increasing memory, though.
15:15:54
drmeister
I'm getting an error when loading jupyter lab that chem:define-tests is not defined - is this something that you are adding? It has to do with smarts.
15:18:17
drmeister
https://github.com/drmeister/cando/blob/102ad64eb4bc9459d4c582819991da25400d4438/src/chem/chemInfo.cc#L314
16:30:34
Bike
i did somethin, and now C-c C-c resultsin different source offset numbers from a full compile file. but M-. goes to the right place either way.
16:32:41
Bike
oh. i see. emacs's idea of source position is different from clasp's, but they're both right up to next-sexp, so it works.
16:39:09
Bike
drmeister: you put in something to hedge if compiled-function-file is not passed a function?
16:39:33
drmeister
Yes - because when there are no debugging frames any error threw it into an infinite loop
16:40:32
drmeister
What we need is better way of getting backtraces what we have is hacked together bullshit.
16:41:14
drmeister
https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/lsp/top.lsp#L1010
16:42:55
Bike
with C-c C-c i got offset 161, with compile-file it was 38... but the only thing between 38 and 161 is whitespace and comments, which emacs ignores anyway
16:48:24
Bike
with inlining i hit another problem instead. also finicky. inlining makes the compile enormously slower anyway so i don't want to bother right now
16:49:25
karlosz
after inlining my system funcall instruction it complains that one of the slots are unbound
16:49:29
Bike
but i don't think that's what it's doing. i've messed up and given it incorrect offsets before, and it's caused errors
16:54:13
attile_lendvai
then it must be my changes... :/ we'll see, i rebased them to some older commit and started a build
16:55:30
karlosz
i was thinking about using stealth mixins and specializing methods on stuff like delte-instruction
16:56:35
karlosz
so delete-instruction can specailize on a liveness mixin to update the liveness information associated with the instruction
17:12:13
karlosz
Bike: im not sure what exactly the current liveness pass computes, but incremental liveness analysis requires keeping track of sets of uses, not just variables
17:14:02
karlosz
so when deleting an instruction that uses a variable, you can delete it from the live-in and live-out use sets
17:14:26
karlosz
its not absolutely required to keep track of uses instead of variables, must uses guarantee least fixed point solution
17:15:02
attile_lendvai
i'm compiling the point in the git log where clasp is switched to llvm6 and it seems to be less memory hungry and faster (but it might be just something subjective)
17:15:27
karlosz
Bike: it does, but unless the program is in SSA, the defs and uses need to be matched up
17:55:22
karlosz
so it looks like local functions with non local escapes and complex lambda lists dont get inlined either
17:57:17
Bike
optional we could do. &rest or &key would require something to allocate the list at runtime.
17:59:24
karlosz
makes sense. might have to be a new instruction since doing that isimplementation dependent
18:53:28
drmeister
It seems to me that there is nothing fundamentally wrong with building the image at startup the way we do - but we don't have the right definition of a compilation-unit.
18:55:22
drmeister
Can you summarize again the problem with your make-instance optimization? What is out of place?
18:58:07
Bike
drmeister: it's been a while, so i don't remember the particular issues i was running into at the end. might have been related to the structs i used having their own constructors
19:09:48
drmeister
In the 'allinone' branch there is a version of Clasp that compiles cclasp by compile-filing all of the source files as if they are one from the point of view of compiling lexical variables.
19:10:54
drmeister
If we add the ability to compile classes as literals - then the class definitions will be created the first time they are needed and then on they would be referenced.
19:11:26
drmeister
The first time I tried it - it used a lot of memory and crashed my laptop. So I'm trying it on your machine.
19:12:02
drmeister
I don't have the classes being treated as literals yet - that would be next if I can get the thing to work at all.
19:12:34
drmeister
But if this were in place - would it solve all of our problems. I've asked this before - but I'm describing it again after mulling it over for a few days.
19:13:28
drmeister
The problem now is that an inline definition in file #5 needs Cleavir AST classes that are defined in file #350
19:15:14
drmeister
And each separate file generates its own table of literals and so even if we compiled classes as literals - they wouldn't be set up at the right time unless we compile everything as one big compilation unit.
19:16:48
drmeister
That's why I'm asking about your make-instance optimization - that ran into a startup problem. It's a useful complementary case to the problem with inlining.
19:18:13
drmeister
If I'm wrong about this allinone+literal-classes idea then I want to ditch it and move to saving core files.
19:19:49
Bike
core dump is more like how i'd want it to work. i meanideally i don't see why we have to run through a million forms assigning function names to functions and so on
19:20:08
drmeister
I did not appreciate the bootstrapping problems that would come with the approach that I chose for starting up the system.
19:22:55
drmeister
Going the core dump route means the following (1) We probably need to get rid of the boehm GC support (2) We need help from Ravenbrook to implement it (3) We need to GC code.
19:28:20
drmeister
(4) We wouldn't compile-file the system files. We would LOAD the source code into memory and the top level forms would be compiled/JITted as they load. The JITted modules would be in the MPS memory and so they would be treated like everything else.
19:29:30
drmeister
(5) We would need to figure out what side-effects in C++ are taking place in our current startup and run them before we start the system back up again.
19:31:35
drmeister
This (1) I've got a great idea to speed things up (2) I'm implementing it (3) Oh shit - the way we startup prevents me from deploying it because of <insert complicated bootstrapping explanation here> is absolute bullshit.
19:38:41
Bike
ok, cut out a few bogus source infos, so trying to M-. to a function defined in the repl properly says it doesn't know the source
19:38:47
attile_lendvai
drmeister: in bits.cc there's some templated version of the bitarray operations. are they really worth it performace-per-complexity wise? if i understand correctly it only saves a couple of memory dereference
19:42:20
drmeister
Saving dereferences in tight loops is important isn't it? That's why I implemented that.
19:43:43
attile_lendvai
drmeister: did any of those show up in a profiling session? because it's quites some extra complexity...
19:44:56
drmeister
How about we move that code into a wiki page so I can remember how to implement the template version if I ever need it.
19:45:33
drmeister
I figured out yesterday there is a big chunk of llvm time that I wasn't counting when I count llvm time.
19:46:42
drmeister
It turns out the inline pass takes about 80% of the time when I (compile nil '(lambda (x y) (+ x y))) repeatedly
20:33:39
kpoeck
I spent some time to make CL_DEFUN T_sp core__bit_array_op(int opval, Array_sp tx, Array_sp ty, T_sp tr) more correct
20:36:29
drmeister
kpoeck: We are thinking of throwing out the template version - you are talking about the non-template version - correct?
20:40:11
drmeister
This is consistent with what I've been seeing for the past two days. llvm time is substantial - about 50%
20:53:17
drmeister
When I invoke TIME from within clasp-builder.lisp - I don't see any TIME output. It's supposed to send it to *trace-output*. I'm thinking that's the same as *standard-error* and maybe waf has rerouted that?
20:53:43
drmeister
If anyone else has any other insight why I get no TIME output - I'd love to hear it.
20:54:44
drmeister
It will do that - and then it may double when it tries to link everything. For my laptop that was an extinction level event and I didn't see the other side of it.
20:55:35
drmeister
(defstruct (xxxx (:type vector) :named) ...) makes it pause for 10-20 seconds! But other than that it's chugging along.
21:42:59
drmeister
I should probably switch back to bitcode .bc files rather than human readable .ll files.
22:11:52
drmeister
The allinone build generated a .o file - so it works - but it's a lot less relevant now that we are looking towards dumping image files.
22:20:24
attile_lendvai
::notify kpoeck do you mean the changes that are already checked in the repo? if yes, then of course, that'd be a major offense... :) if you mean changes that are not in dev, then do warn me
1:57:32
drmeister
Loading and compiling foundation.lsp with the JIT - with no llvm optimization is 1.039 seconds