freenode/#clasp - IRC Chatlog
Search
4:30:18
robink
oops, last command should be 'objcopy --add-gnu-debuglink=iclasp-boehm.debug iclasp-boehm'
13:16:19
drmeister
I'm down at Saige trying to shake out a warning that is coming up when satiating functions.
13:35:52
drmeister
In Python keyword arguments are passed in to functions as dictionary objects with strings as keys.
13:37:08
fredokun1
you mean the closest thing to a python dictionnary (with string keys, to simplify) is a hashtable ... especially lookup is fast
13:37:34
fredokun1
one big difference for me is that python dicts are transparent (as are the alists) whereas CL's hashtables are opaque
13:37:36
drmeister
I'm not sure how much of an issue this is with other jupyter widgets but we have one widget that uses this property of keyword arguments -> dictionary with string keys -> JSON dictionary for updating widget on the client side.
13:38:47
Bike
the thing that i've noticed is that ** in calls lets you append dicts together like they were plists
13:41:34
drmeister
Sorry - my thinking is a bit jumbled at the moment. I'm trying to figure out how to clarify things.
13:43:07
drmeister
Maybe my suggestion of converting from alists to hash-tables for representing JSON dictionaries was not well thought out. I'm trying to recreate my thought processes that led me to that suggestion.
13:43:54
drmeister
Translating this function to Common Lisp - I would want it to look like a regular Common Lisp function - with keyword arguments.
13:45:42
drmeister
In that Python code they get a **kwargs dictionary and then they do a lot of manipulation with it.
13:45:50
fredokun1
A macro could do the trick ? Like a "defpymethod" (that's ugly but you see the point ...)
13:46:55
fredokun1
you want the calls to be standard keyword arguments calls... and you want the definition to have a hashtable argument 'kwargs' ?
13:47:39
drmeister
I was translating the plist of arguments into an alist for cl-jupyter to send out as a JSON dictionary.
13:49:48
fredokun1
If it's only for the JSON encoding, it's very easy to add a specific method to encode hashtables to JSON object (encode-json is generic)
13:50:12
drmeister
On further consideration I think I may have made a poor suggestion to change the way that cl-jupyter represents JSON dictionaries from alists to hash-tables. I can add add an intermediate translation from keyword arguments (plist) -> hash-table -> alist (for cl-jupyter).
13:50:18
fredokun1
for the decoding part, the simplest for now is to use an explicit alist->hashtable conversion
13:50:56
Bike
i thought the basic problem we had was that we wanted to use (non a)lists for various things, but the converter expects all lists to be alists.
13:53:58
fredokun1
I can implement hashtable and sequence encodings easily and quickly... The question is : where should I to that ?
13:56:02
drmeister
There are no JSON lists - right? Just JSON dictionaries(objects) and arrays - correct?
13:56:40
drmeister
The current encoding is here: https://github.com/fredokun/cl-jupyter/blob/master/src/myjson.lisp
13:58:11
drmeister
If I want to work with hash-tables I can write an encoder of hash-tables->alists to pass to myjson for encoding to JSON dictionaries.
13:58:48
fredokun1
yes but it's almost as simple and more efficient to directly write an hashtable encoder
13:59:11
fredokun1
for the decoding, I prefer to stick with alists so yes you would need an explicit conversion
14:00:34
fredokun1
it's a problem if you want a direct encoding of a vector that would be interpreted e.g. as a Json object
14:01:31
fredokun1
I don't see a big issue here. And I understand that hashtables might be more efficient/closer to the python way of doing things
14:02:02
fredokun1
maybe we could make the hashtable representation the default in the encoding/decoding of COMM messages ?
14:02:57
drmeister
I think my problem was I was thinking too literally - trying to write code that worked with dictionaries as alists while trying to duplicate Python code that was working with Python dictionaries.
14:04:02
drmeister
I think your first suggestion is most interesting - add an encoder that converts hash-tables to JSON dictionaries.
14:04:22
fredokun1
I think there's a separation to make at some point... It seems you would like a "python-styled" part and I think it's doable with a few macros. But it's separate from the Json things.
14:04:59
drmeister
(defmethod encode-json (stream (thing hash-table) &key (indent nil) (first-line nil)) ...)
14:06:20
fredokun1
we also need a alist->hashtable conversion function, easy because it's only json (string keys, etc.)
14:07:38
drmeister
If you could allow it to work with string or symbol keys by converting the key using (string key) - then the hash-tables could be :test #'EQ and use symbol keys. I can ensure that the keys are camel case as in :|useWorker|
14:09:26
fredokun1
agreed : symbols are the best ! I have still an important question: where should I do that ?
14:12:33
drmeister
How about this. I'll clean up the cl-jupyter code interface to cl-jupyter-widgets (I was about to do it and then I came up with this hair brained scheme). Then I'll submit a pull-request. You merge it and then add the hash-table encoder.
14:13:52
drmeister
I want to give you a cl-jupyter that runs on its own and has the absolute minimum number of hooks to interact with cl-jupyter-widgets.
14:14:44
fredokun1
That would be great... You can even create a branch in the cl-jupyter repo you have right access. Then I can make the pull request to master myself...
14:16:42
fredokun1
It will be an occasion to try cl-jupyter with clasp. I'm very interested because it seems you don't have the bug I'm experciencing with sbcl
14:20:23
fredokun1
I have to go soon. It was nice to talk to you again drmeister+bike, and sorry once again for being slow answering my emails.
14:48:34
whoman
does anyone have an intuition on how much cpp of clasp is generated and how much is hand written ? or its all hand?
15:00:42
whoman
yeah.. just that i see a lot of repetition and such in a few files so i wasnt sure how much of that is done by hand; i forget which file i was looking at last
15:01:25
whoman
;;it has been a long while since i sat down and typed a lot of [C] code though. reading and research these days
15:05:41
beach
So, in bclasp, what happens if you have (say) (let ((x1 ...) (x2 ...) ... (x257 ...)) (lambda (x) (case x (1 x1) (2 x2) ... (257 x257))))
15:07:07
beach
In other words, for that LET to work, it must use a different mechanism than the one used for function calls to create an environment.
15:07:59
Bike
yeah, you'd end up with code that has 137 lexical locations, which are translated into 137 LLVM memory locations, then obviously 137 LLVM SSA variables
15:09:10
drmeister
ACTION never finished "Let over Lambda" because his electronic copy on Apple iBooks has mangled examples.
15:10:12
beach
Bike: obviously, there are fewer than 137 registers, but I still don't understand what allows one but not the other.
15:10:36
drmeister
Could we use LET for calls with more than some number of arguments and the current mechanism for those that do not?
15:10:45
drmeister
Could we use LET for calls with more than some number of arguments and the current mechanism for those that have fewer?
15:11:18
beach
drmeister: I would like to understand why such a thing would be needed, i.e., I would like to know why the LET is fine, but the function call is not.
15:11:19
Bike
the difference isn't the number of arguments, it's that the call is literal. If you have all the arguments right there, and the number of arguments, like in the let, we can put in 137 variables, and that would work for calls
15:11:54
Bike
but with apply, the number of arguments is variable, so we'd have to compile on the fly or just write a different APPLY out depending, and i think the latter is more like what happens, and yes it's bad
15:14:29
beach
Yeah, I am not asking what might work if someone implemented it. I am asking whether they both currently work, i.e. the LET and the LAMBDA form. And if not, what is the difference in the mechanisms used? And if they both work, why is the CALL-ARGUMENTS-LIMIT not applicable to the LAMBDA form?
15:15:29
Bike
I think they would use basically the same mechanism. If the lambda form doesn't work it would be because of an explicit check on parameter counts or something, rather than anything inherent to the mechanism
15:16:26
beach
So by eliminating that explicit check, the CALL-ARGUMENTS-LIMIT could be as high as we like?
15:17:13
drmeister
Currently there is a limit on the number of arguments that can be used in function calls.
15:18:04
drmeister
LET doesn't have this problem because it allocates an arbitrarily large activation frame and evaluates expressions into it.
15:18:26
Bike
there's a difference between compiling a call with a known number of arguments, and implementing apply.
15:19:58
beach
So if, in Cleavir, I turn the LET into a lambda form, I will break programs with a large number of bindings in LET forms?
15:20:30
drmeister
The only way to call a function with three arguments is foo(a1,a2,a3) four arguments bar(a1,a2,a3,a4). How do I call with N arguments?
15:21:17
Bike
Could we have it so that functions with 1-N parameters are special cased, and all other functions have parameter lists (a1, ... aN, valist)?
15:21:40
drmeister
I write the arguments into a vector and then enter a big switch statement, with one entry for each arity and for every arity there is a call like (foo)(args[0],args[1],args[2])
15:23:43
drmeister
Maybe - it will take sitting down and thinking it through - I've rewritten this code half a dozen times in the last 5 years.
15:24:28
Bike
beach: obviously call-arguments-limit should be higher anyway, and there are a few paths we can take to raise it. work, but work that should be done, you know?
15:25:23
drmeister
In the last rewrite I had to impose this limit because of the way that CL functions can now call C++ functions directly. I don't recall exactly what the problem was. This is one of the most painful parts of the C++ interop.
15:25:29
beach
It's not a big deal. It's just that, in Cleavir I have no way of expressing a LET, other than by a function call. So to convert LET, I must basically do a function call, and then let the implementation do it differently if that is not a working solution.
15:25:57
beach
... unless, of course, I introduce some kind of AST for making environments that is separate from the ASTs currently used.
15:26:39
drmeister
beach: Oh - this is what this is about? Do what you need to do in Cleavir - we will fix the implementation on this end.
15:27:09
drmeister
Don't twist the design of Cleavir to meet the peculiar limitations that Clasp currently has.
15:28:34
beach
Bike: I would need a mechanism for creating environments, separate from the existing one.
15:28:48
Bike
I thought the idea was that we would have it as ((lambda (...) (lambda (x) ...)) ...), and then we'd do the cell conversion, and then we could still eliminate the call, and just put the make-cells where the call was.
15:30:16
Bike
Since we don't like... create environments, per se, in HIR, just cells, which are passed to ENCLOSE like arguments.
15:31:31
beach
It all requires the better control-flow and data-flow analysis that we have discussed several times.
15:33:31
drmeister
Bike: I have an issue that just came up with fastgf. I'm running the acid test and an error was triggered:
15:33:46
drmeister
((METHOD CLOS:SLOT-MAKUNBOUND-USING-CLASS (BUILT-IN-CLASS T T)) #<VA-LIST: (#<The STANDARD-CLASS STANDARD-CLASS> #<UNPRINTABLE> 'NIL #<The STANDARD-CLASS STANDARD-CLASS> #<The STANDARD-CLASS CLASS> #<STANDARD-EFFECTIVE-SLOT-DEFINITION CLOS::VALID-INITARGS> )
16:20:26
drmeister
The problem may be with the effective method functions calculated during satiation