freenode/#clasp - IRC Chatlog

Search
9:52:48 selwyn hi everyone
9:54:14 heisig Hi selwyn!
10:29:15 jackdaniel Bike: it is defined to be able to store functions in an array
10:29:44 jackdaniel so you may declare it as an array storing void(*)()
11:02:55 drmeister Hello everyone
11:03:46 drmeister heisig: Does that cl4py pull request to translate python keywords to common lisp keywords work?
11:34:26 heisig drmeister: I'm reviewing that PR right now...
11:43:26 heisig Yes, everything works. Using **kwargs this way is a brilliant idea.
11:54:32 heisig drmeister: The new version of cl4py is now on PyPI.
12:49:12 Bike jackdaniel: blagh.
12:50:50 Bike couldn't you just have an array of void*?
12:51:54 jackdaniel afair it is not portable to use memory pointers for functions
12:52:18 Bike "there are no conversions between pointers to functions and pointers to objects" ah, i see
12:52:22 Bike also seems silly
12:52:56 jackdaniel I don't remember well, but rationale was that on some architectures data and functions may be stored in different memories
12:53:15 Bike i'm sure
12:53:21 jackdaniel so a separate addressing scheme may be necessary
12:56:52 drmeister Hello
12:57:44 jackdaniel if you swim in the pointer swamp I recommend using uintptr_t for bit operations
12:57:52 jackdaniel hey drmeister
12:57:58 Bike yeah, that's what clasp does
12:58:03 Bike i'm just being annoyed at C on a separate leve
12:58:08 Bike l
12:58:26 jackdaniel why does python live on land?
12:58:31 jackdaniel because it is above C level :-)
13:01:03 Bike reticulated pythons swim in the ocean, apparently. that's neat
13:01:10 Bike i've already found two asOrNulls that can be isA instead
13:01:29 drmeister Note - there is IsA and isA - note the case.
13:01:34 drmeister (sorry)
13:01:36 Bike seriously?
13:01:39 Bike what's the difference
13:01:48 drmeister IsA is for _sp pointers
13:01:58 drmeister isA is lower level - you should never need to touch it.
13:02:02 Bike okay.
13:02:16 drmeister lower case - lower level. See - I'm not crazy.
13:05:38 Bike we have a file called testing.cc that defines two lisp functions that aren't used anywhere
13:05:43 Bike i'm never gonna run out of cleaning to do
13:07:08 drmeister Oh sure we will. At some point we will transition from "garbage dump" to "charming anachronisms".
13:15:39 drmeister asOrNull is using the TaggedCast (integer stamp comparison) - it only falls through to dynamic_cast in special cases.
13:15:48 drmeister So you don't need to take them all out.
13:16:07 drmeister I think we could profile for dynamic_cast to see if it is a problem.
13:16:26 drmeister I saw the dynamic_cast clearly in the profiling for read-line.
13:25:17 Bike IsA is clearer anyway, and this is easy
13:25:31 Bike also whenever i look at C++ i find stuff like testing.cc i can remove
13:26:00 Bike https://github.com/clasp-developers/clasp/blob/master/src/core/metaClass.cc#L106-L122 or this. check it out. remember when classes were different from instances? crazy times
13:26:11 Bike i wonder if there's a compielr warning about this buried somewhere. maybe it's not smart enough
13:29:09 drmeister Ok sounds good.
13:29:46 drmeister Crap - in cl__streamp I was testing if the argument was a gray stream first. I'm testing for Stream_O first now.
13:30:05 Bike oh, that might be bad. how often do we do streamp, though?
13:30:21 drmeister read-line does
13:31:02 drmeister I've been working for a few hours to remove dynamic_cast from read-line.
13:31:40 Bike i see.
13:31:55 Bike some of these i'm a little hesitant to remove because i s\till don't quite understand how we're using types
13:32:20 Bike for example: if (Instance_sp instance = strm.asOrNull<Instance_O>()) return eval::funcall(gray::_sym_open_stream_p, instance);
13:32:26 Bike couldn't we just pass funcall strm?
13:32:50 drmeister Ok, now I'm making progress again. I just made it 20% faster on top of the 4x faster from yesterday.
13:32:54 drmeister Hmmm, thinking...
13:33:16 drmeister What's bad about that is if strm is a Stream_sp object then it will invoke dynamic_cast.
13:33:46 drmeister And that's the common case.
13:35:45 Bike well i mean, say i write if(strm.IsA<Instance_O>()) return eval::funcall(gray::_sym_open_stream_p, strm); instead.
13:35:51 Bike is that bad?
13:36:42 drmeister Oh yeah - funcall doesn't care.
13:37:47 Bike or if we have a function with return type T_sp, and it does if (Package_sp pkg = name_desig.asOrNull<Package_O>()) return pkg;
13:38:10 Bike seems like we cast it and then cast it right back
13:38:19 drmeister There is zero cost involved with that.
13:38:31 drmeister So it's harmless
13:39:07 Bike you're sure? there's never going to be a dynamic_cast, even if name_desig is an instance or something?
13:39:20 drmeister I like to use that kind of prose because we went to the trouble of defining a variable pkg - so we should use it.
13:39:39 Bike ...but i mean, we don't need to define the variable pkg. we could just do name_desig.IsA
13:40:30 drmeister It's zero cost - it's reinterpret_cast. The cost is figuring out if name_desig is a Package_sp - once you know that - the casting costs nothing but compiler time and we are not going to impact compile time by changing this....
13:40:37 drmeister if (Package_sp pkg = name_desig.asOrNull<Package_O>()) return pkg;
13:40:39 drmeister To this...
13:40:44 drmeister if (Package_sp pkg = name_desig.asOrNull<Package_O>()) return name_desig;
13:40:57 drmeister That's what we are talking about - right?
13:41:05 Bike well i'm talking about changing it to "if (name_desig.IsA<Package_O>()) return name_desig;"
13:41:19 Bike that probably also won't be faster, but it's shorter and nicer, i think
13:41:23 drmeister Do we have that?
13:41:32 drmeister If you are talking about that I would use:
13:41:52 drmeister if (gc::IsA<Package_sp>(name_desig)) return name_desig;
13:41:56 drmeister So many choices.
13:42:11 Bike aw, cripes, is that how it works
13:42:35 drmeister But that last one is what I've been using. I've grown to prefer the function syntax rather than the method dot syntax.
13:42:50 Bike well, me too. guess i screwed up what i've done so far, then
13:42:51 Bike oh well
13:43:03 drmeister Yeah - I was wondering where you were getting name_desig.IsA<Package_O>() from.
13:43:26 Bike i mean, you get the idea, though
13:43:36 Bike asOrNull expresses that we actually want an object of this type
13:43:42 Bike but in fact, we don't care, we're just doing a type test
13:44:03 drmeister I wouldn't be surprised if name_desig.IsA<Package_O> is legal I've tried everything but this is what I think we should use now...
13:44:26 Bike well i'll go with the gc::
13:44:32 Bike i like function calls more than method invocations
13:44:44 drmeister if (gc::IsA<Package_sp>(name_desig)) { Package_sp pkg = gc::As_unsafe<Package_sp>(name_desig); ...do something with pkg;}
13:44:58 Bike ok, sure
13:45:03 Bike but in this case nothing is actually done with pkg
13:45:07 Bike it literally just returns immediately
13:45:08 drmeister When we do the big cleanup that's what I would switch everything to.
13:45:27 drmeister Yes, if you don't do anything with pkg then you don't need to do the gc::As_unsafe<Package_sp>(name_desig)
13:47:19 Bike there's a core__test_tagged_cast... i wonder how many "test" functions are lying around
13:47:57 drmeister Wow, the guy who is going to work with us in validating Cando is the best in the world.
13:48:09 drmeister He's talking now - brb
14:06:07 Bike you doAsOrNull with an _O type but IsA with an _sp type... ok...
14:08:21 drmeister I want to standardize on using the _sp type - I can derive one from the other.
14:08:40 Bike it's lower case, for jlower level
14:09:08 drmeister I am a lot of things - consistent is not one of them.
14:10:01 drmeister I've gotten rid of dynamic_cast from read-line.
14:11:06 drmeister Now we are comparable to ECL for read-line
14:12:01 drmeister (time (test 10))
14:12:05 drmeister Clasp 1.7 sec
14:12:07 drmeister ECL 1.9 sec
14:12:17 drmeister sbcl 0.36 sec.
14:12:27 drmeister stassats sold his soul to the devil for that.
14:12:39 drmeister Or someone did.
14:14:54 drmeister Seriously, fread is 43% of the time now.
14:15:46 jackdaniel did you use newest ecl?
14:15:56 jackdaniel Marius made some improvements to read-line
14:16:02 jackdaniel (performance is much better now)
14:16:05 jackdaniel merged yesterday
14:16:07 drmeister 16.1.3
14:16:22 drmeister Nope - not that recent.
14:17:11 drmeister I just tried to pull from here - there were no updates...
14:17:25 jackdaniel develop branch on ecl
14:17:44 jackdaniel master is for releases
14:17:49 jackdaniel develop is our HEAD
14:17:54 jackdaniel of accepted contributions
14:18:33 drmeister Ok, I'm pulling develop - I'm fine with experimental stuff.
14:18:35 jackdaniel ah, said PR does not affect read-line, only read-sequence
14:18:49 drmeister read-line will be hard.
14:18:54 jackdaniel it is not experimental stuff, just things which are not throughfully tested
14:19:09 drmeister Understood
14:19:30 jackdaniel we have a featureset which we want to complete before each release, after that comes release candidate with testing across dozen of different platforms x architectures
14:19:33 jackdaniel + cl-test-grid testing
14:21:04 jackdaniel and that usually show dreadful regressions and yields some more weeks of development before the release
14:23:58 drmeister I just built the develop branch of ecl.
14:24:05 drmeister The time is 1.94 secs
14:24:31 drmeister ECL:
14:24:52 drmeister Clasp...
14:25:32 drmeister The last one was sbcl.
14:31:56 drmeister The python bridge is going to be super important - these folks have a lot of validation and statistics code in Python that I don't want to reproduce.
15:05:56 drmeister Damn - we are in a good place with Cando. Almost everybody loves us.
15:17:45 pfdietz sbcl is just what happens when you polish a code base for a third of a century (counting its time as cmucl). Python integration sounds compelling, btw.
15:18:00 drmeister pfdietz: Hello - yes - understood.
15:18:35 drmeister This is what driving Cando from Python3 looks like...
15:18:58 Bike okay, so this instance/class thing is weird, i'm replacing the asOrNull with just As, since a superclass list should have all instances in it anyway
15:19:19 drmeister Note line 29 and 18 - where python keyword arguments are converted to CL keyword arguments.
15:19:22 Bike as far as i can see, this variable is never assigned to without being read. seems like it should be initialized to NULL, or whatever NULL is for a smart pointer
15:19:55 drmeister OTOH line 16 is a major wart in my eyes.
15:20:33 drmeister Just As will signal an error if the cast can not be made. asOrNull returns the cast pointer or NULL.
15:22:33 drmeister Instance_sp aCxxDerivableAncestorClass_unsafe; <<- This is initialized to NULL 0x0
15:22:54 Bike yeah, i want it to signal an error
15:23:00 Bike so, despite the comment, it is actually initialized?
15:23:13 Bike if so i'll just change the comment
15:23:30 drmeister If you then test: if (!aCxxDerivableAncestorClass_unsafe) { ...this will be evaluated... } else { ... this will NOT be evaluated ...}
15:23:42 drmeister Yeah - the comment is not correct.
15:23:47 drmeister The default constructor is here...
15:24:18 Bike ok, i'll change the comment.
15:24:29 Bike using uninitialized values gives me hives, i'm glad we're not
15:24:59 drmeister Very, very rarely do we use things like that.
15:25:28 Bike uninitialized values?
15:25:45 drmeister I think I can count on one hand how many times I've used it:
15:26:17 drmeister What are you calling uninitialized value?
15:26:28 drmeister Where the pointer is completely undefined?
15:26:28 Bike well, i mean, if this was just a regular variable
15:26:32 Bike right
15:26:43 drmeister Oh - that - right - we don't do that.
15:26:44 Bike if there's a constructor that initializes it properly, that's fine
15:26:52 Bike if slightly confusing to read
15:27:31 Bike must be an advantage of these smart pointers i've heard so much about
15:27:40 drmeister I wish I had initialized smart_ptr with Unbound but I went with NULL/0x0
15:28:02 drmeister Yes, the constructors are one of the reasons why I use them.
15:28:13 Bike well, we could change that i guess
15:28:18 Bike have code like this assign NULL to it
15:28:22 Bike can we do that with smart pointers?
15:28:27 Bike like, have smart_pointer = dumb_pointer
15:29:26 drmeister I think you can do T_sp foo = T_sp((gc::Tagged)NULL)
15:29:31 drmeister But bleh.
15:31:46 drmeister Bike: I think we can start pushing stuff into 'dev' again.
15:31:49 Bike could we do some weird template shit to get a NULL_SP that's a Void_sp or something so we don't ahve to care
15:32:31 drmeister I've got a working cando installed in /opt/clasp that does jupyterlab and ileap-boehm. So I'm good for demos.
15:32:41 drmeister I'm trying to get hold of cracauer to check with him.
15:33:11 Bike i'll keep holding off. i don't mind working in branches anyway
15:33:36 drmeister Can I pull from one repo on my machine to another one?
15:34:01 Bike well, not with pull, that uses the network
15:34:11 drmeister I made these read-line changes in my clasp directory and I'd like to test in cando-dev
15:34:19 Bike oh, hang on
15:34:28 Bike "Remote repositories can be on your local machine."
15:34:31 Bike now how does one do that
15:35:06 Bike oh, it's... really easy actually
15:35:16 Bike just do `git remote add remotename /path/to/repo.git`
15:35:22 Bike and then you can pull and stuff normally
15:35:25 Bike that's kind of neat.
15:35:27 drmeister Then what happens when you try to merge and update later on.
15:35:41 Bike it's exactly the same as using a network remote, apparently
15:36:22 drmeister I'll give it a whirl.
15:38:23 Bike the documentation doesn't seem tos pecify what urls are allowed, vexing
16:14:59 drmeister Bike: Have you had any time to think about inlining array accessors in discriminating functions?
16:16:05 Bike well, we could build the functions' source pretty easily
16:16:24 Bike cmpfastgf is already set up to allow arbitrary forms as outcomes
16:16:31 Bike so can just put in cleavir-primop:aref whatever
16:16:38 drmeister Oh - neat - ok.
16:17:15 Bike so i can try that out if you like
16:17:24 drmeister I would like that very much.
16:17:37 drmeister I'm curious how well it works.
16:21:16 drmeister Everything is really peppy. I'm really digging Cando these days.
16:49:34 Bike hm, doesn't work exactly actually... e.g., we don't want to call dispatch-miss
16:49:37 Bike no big issue though
16:58:06 Bike ok, first shot at it. doing a hundred million reads from a simple vector, vref takes 0.494s, dispatcher version takes 0.860s
17:00:24 drmeister Compare that with sbcl
17:00:41 drmeister I think we can do better than sbcl
17:01:17 drmeister It's basically free to compare a bunch of hard-coded integers to a stamp. So dispatch is basically free for us.
17:02:21 drmeister Can you dump the llvm-ir for the dispatcher version?
17:03:07 drmeister No hurry.
17:03:51 drmeister And when you say vref takes 0.494s - how are you doing that and how are you doing it with the dispatcher version?
17:04:07 drmeister Post sample code
17:15:44 Bike (let ((vec (make-array 37 :initial-element (gensym))) (n 12)) (time (loop repeat 100000000 do (vref vec n))))
17:16:02 Bike same with the dispatcher version
17:16:45 Bike llvm ir, one second
17:18:30 Bike this reminds me that ir disassembly doesn't respect standard output
17:18:44 Bike should fix that
17:19:41 Bike https://pastebin.com/WsTNY87x here is the disassembly, drmeister
17:20:17 Bike still got that fucking return value alloca.
17:21:02 Bike oh hey, dump_module takes a stream argument. neat-o beans
17:21:30 Bike why is it not working...?
17:21:50 drmeister You have to mess with llvm streams
17:22:02 Bike i mean, doesn't look like i should have to
17:22:07 Bike let me link it
17:22:13 drmeister Add (declare (optimize (debug 0))) to your discrminiator - it's spilling register args.
17:23:02 Bike https://github.com/clasp-developers/clasp/blob/dev/src/llvmo/llvmoPackage.cc#L437-L444 seems like it should work with *standard-output* already
17:24:17 Bike does debug 0 actually disable that? i just tried with (lambda (x) (declare ...) x) and i still see a bunch of store volatile
17:27:06 drmeister It should - I made some changes so that if that is in the function somewhere - then it gets applied to the entire function.
17:28:06 Bike well, i'm not seeing it. where is the code generated?
17:28:11 drmeister I generated a CFG graph and rendered it as a PDF - there is still a call in there but it looks pretty good.
17:28:12 Bike or rather i am seeing it, i suppose
17:28:21 Bike what's the call?
17:28:29 drmeister Maybe I didn't push it because of the moratorium on pushing to dev
17:28:45 Bike i mean there's one to error
17:29:13 drmeister I can't dump the PDF - it keeps failing.
17:29:30 Bike it's fine
17:29:58 drmeister Nothing other than test posts are working at the moment.
17:29:59 drmeister I'm sitting outside and the wifi connection sucks.
17:30:21 Bike i mean i can guess what the cfg looks like
17:33:23 Bike but i'd like to know what call you mean, and where the register spilling is generated so i can see if i have your code there or not, and if you have any idea about the disassemble output stream
17:35:13 drmeister It's hard to figure out what function is being called - we need to improve that.
17:36:29 Bike well, if it's a normal call i can get it from the HIR, probably
17:36:46 drmeister I'm going to push my stuff to dev - I just build cando.
17:38:51 drmeister I just pushed everything I had - it all built cando and the quicklisp.
17:41:08 drmeister Does your make-function-info-map in setup.lisp look like this?
17:42:08 drmeister The line 34 maphash is the most telling thing.
17:43:19 Bike make-function-info-map isn't in setup.lisp
17:43:28 drmeister No?
17:43:37 Bike i have it in introduce-invoke
17:43:41 drmeister Yeah - that.
17:43:49 drmeister I was bouncing around and lost track of what file I was in.
17:43:53 Bike and yet, it does look like that
17:44:14 drmeister Could you look that over? It may not have exactly the effect I'm looking for.
17:45:25 drmeister What I want is if the default (declaim (optimize (debug <whatever>))) is different from what is found in the function then the 'debug-on' slot is set to whatever the programmer explicitly set in the function. So the default is used unless the programmer explicitly set something different.
17:46:18 drmeister And is that what we want?
17:46:20 Bike i see the problem
17:46:37 Bike it works but it's basically unnecessary, because i already wrote a function to take care of the inner declaration thing, policy-anywhere-p
17:46:46 Bike you could just call that whenever you're deciding whether to save register args
17:47:08 drmeister Ok, make the change to what you set up.
17:47:08 Bike and the reason i'm seeing register args saved with (lambda (x) x) is different and stupid
17:47:24 Bike for nontrivial functions it doesn't save register args with debug 0
17:51:25 Bike i pushed my IsA changes as a "clean-cpp" branch
17:54:10 jackdaniel name it clean-cxx, sounds more eXtreme ;-)
17:55:58 Bike with debug 0, dispatcher version time goes down to 0.812s
18:08:25 Bike hm. it calls generalp.
18:08:30 Bike that's bad. why is it doing that.
18:08:50 Bike because i didn't put in an inline definition for generalp.
18:09:35 Bike and kind of can't, typeq won't work with general
18:28:50 kpoeck Hello
18:29:46 kpoeck drmeister In the readline-change, don't we need the pattern T_sp result = cl__copy_seq(sbuf_wide); _lisp->put_StrWNs_buffer_string(sbuf_wide);
18:29:58 kpoeck in all places where a result is returned?
18:30:51 kpoeck There are 6 more return .... in cl__read_line where this is not done
18:33:57 Bike sounds right to me
18:36:04 drmeister I’ll check when I get back. Thanks!
18:36:32 drmeister Bike: also do safety 0
18:37:00 Bike does that affect anything here?
18:37:07 Bike anyway, the generalp call is probably the main issue for time
18:51:20 Bike wait. it does work with typeq.
18:51:24 Bike alright then
18:52:45 kpoeck I cannot build dev
18:53:22 kpoeck drmeister did you build with DEBUG_MONITOR?
18:53:52 Bike with generalp replaced with a typeq (which is maybe not the right way to do it, since i think it'll test the header pointlessly) it's now 0.457s, faster than core:vref
18:53:55 Bike that was simple
18:54:06 kpoeck core::monitor_message is only defined with DEBUG_MONITOR in lisp.h
18:54:07 Bike only a lil bit, of course
18:54:55 kpoeck but used in llvm_sys__jitFinalizeReplFunction
18:58:17 kpoeck So I believe there is a #ifdef DEBUG_MONITOR missing in https://github.com/clasp-developers/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L4018
19:00:15 kpoeck + the #endif
19:00:58 kpoeck at least it compiles with the #ifdef
19:11:15 Bike i am... pretty unsure what this code is doing actually
19:11:19 Bike it loads the general header, i think
19:11:32 Bike then it subtracts 76 from it, and compares it with 5105
19:11:47 Bike like, why not just compare with 5181?
19:12:18 Bike probably doesn't need to do any comparison at all, either
19:12:29 Bike oh, maybe that lets it also test for a minimum value at once....
19:12:41 Bike does that work? i guess it would work. huh.
19:13:02 Bike that's a neat trick, then
19:13:12 Bike too bad it shouldn't even load the header in this case
19:23:07 kpoeck bike if dev is now open for business, could you merge fix-method-args-err?
19:23:47 Bike i don't know if it's open for business or that i've tested that enough
19:26:51 Bike eliminating the header check puts it at 0.442s, so not much of a change there
19:27:51 Bike might be worth noting that we do calls to unbox characters and stuff
19:28:02 Bike (not that my dumb timing here is measuring there)
19:28:41 Bike man i really want to know why the fuck it insists on keeping the primary return value in an alloca and bitcasting it and stuff
19:29:23 Bike does llvm not ssa variables in mem2reg if they're assigned too much or something? that seems unlikely