freenode/#lisp - IRC Chatlog
Search
6:35:49
mange
I'm also not sure I understand the question, but ',(wrap-1 'macro-arg) would be closer.
6:39:04
beach
mange: Your second one gives a "comma not inside backquote". If I replace the ' by `, then the result is not the same.
6:44:52
mange
I'm not sure it will be possible. You need to pass a value into #'wrap-1 that will unquote itself in the expansion. Can you change wrap-1?
6:46:50
mange
Or, alternatively, can you rely on wrap-1 to always do a simple wrapping like it currently does?
6:49:07
beach
These are condensed examples of something much more complex, so in reality wrap-1 does much more and I want to reuse it if possible.
6:50:46
MichaelRaskin
If wrap-1 is given as a function, it is undistinguishable from (defun wrap-1 (form) (list 'bar form)))
6:54:32
no-defun-allowed
i want to write an async client for cl-decentralise which will register listener functions on cl-d channels whenever a certain message is received
6:56:56
beach
I want to write a function wrap-3 so that if I type (wrap-3 'some-form) I get the same result as if I type (wrap-2 'some-form), but instead of mentioning BAR explicitly in the body of the function, I want wrap-3 to call wrap-1 to obtain the same result.
7:00:48
MichaelRaskin
Because ,(list (first '`a) (wrap-1 (second '`,macro-arg))) happens to work on SBCL
7:03:49
beach
Anyway, thanks everyone. It appears that any solution will be more complicated than just repeating the body of the wrap-1 function inside wrap-3.
7:04:32
MichaelRaskin
You can also just put a wrap-1 call there, which will give a different expansion but the same functionality
7:04:46
mange
If you're willing to have wrap-1 be (defun wrap-1 (form) ``(bar ,,form)) then I think you can do it, but it will mean that other calls need to have an extra quote.
7:05:35
MichaelRaskin
But yeah, implementation is free to treat `form as _any_ form that evaluates to the correct result
7:05:55
jackdaniel
mange: as I understand it wrap-1 may have arbitrary expansion, this is just an example
7:06:56
mange
jackdaniel: Yeah, but the approach of "add an extra layer of quasiquotes" may be able to be applied more generally.
7:09:36
MichaelRaskin
Actually, just putting unadorned (wrap-1 macro-arg) should be a viable strategy
7:19:18
beach
Not just that. I think I can do it if the expansion of wrap-3 is allowed to contain a call to wrap-1. But I do want the immediate output of wrap-3 to be the same as that of wrap-2.
7:21:44
MichaelRaskin
I think it is provably impossible without modifying wrap-1. There are multiple posible read results of `, and if you want your code to look the same as if ` was written, input-output relation of wrap-1 is not wnough
7:28:07
beach
Anyway, I have several ideas now. Thanks to everyone. Time to go do something else for a while.
8:31:28
beach
Hmm. CALL-METHOD and MAKE-METHOD are some of the most complicated macros (or rather forms that wrap some other forms in those macros) I have ever attempted to write. In case anybody wondered, that's what the exercise was about.
9:36:35
beach
j`ey: Er, no. It doesn't provide anything except code to compile any Common Lisp form.
9:38:28
beach
SICL, on the other hand has as a goal to be a fully conforming Common Lisp implementation. But it is not finished yet, so there is still some code missing.
9:40:19
j`ey
beach: does Cleavir have textual output? Do you have a really basic example I could look at
9:42:00
beach
What kind of textual output are you referring to? Currently it generates a graph of intermediate code that client systems can then translate to LLVM or assembler or whatever.
9:43:42
j`ey
I was thinking textual output like LLVM IR has, but a picture/graph would be useful too
9:46:35
no-defun-allowed
ACTION uploaded an image: tumblr_o16n2kBlpX1ta3qyvo1_1280.jpg (137KB) < https://matrix.org/_matrix/media/v1/download/matrix.org/QKyfnMJUkJdTxClYnaEJneZJ >
9:48:01
beach
no-defun-allowed: Come on. This is a picture of a tool to view IR with. The person using that tool knows perfectly well what it means.
9:48:31
beach
no-defun-allowed: I wasn't about to make a special picture for j`ey with a legend in it.
9:48:46
jackdaniel
I expect that the guy with cigarette on the photo knows perfectly well what these sheets and lines mean :)
9:54:40
shrdlu68
I'm trying to optimize some code which subseqs a simple-bit-vector a lot, and it appears that subseq'ing conses much less that using displaced bit vectors.
9:57:57
beach
A displaced vector must set up a lot of information that needs to be stored somewhere.
10:00:27
Shinmera
right, so the contents fit into a word, meaning the overhead of a copy is going to be very small
10:00:45
beach
j`ey: I think you may have a hard time finding that kind of information. There is Lisp in Small Pieces, but you won't find IR graphs and stuff in it.
10:02:01
shrdlu68
shka_: I'm keeping that in mind, eventually I will try it . Right now I'm trying a scheme where I sxhash the substrings, converting them to fixnums.
10:02:02
j`ey
beach: what I was really thinking about was macros from CLHS, which I assume cleavir does have to implement
10:03:17
beach
j`ey: Which is fortunate, because there is not standardized expansion of standard macros.
10:04:48
shrdlu68
shka_: Indeed it isn't. Using (mod (sxhash substring) (expt 2 24)) as the indices of an array, it's about two seconds faster than the hash-table implementation. It consumes much less memory, though.
10:04:56
beach
j`ey: Cleavir sees a macro in the current environment. It calls the macro function, giving it the form and the environment. It then compiles the resulting form instead.
10:05:03
j`ey
beach: when you say 'supplied', is it just like how the defmacro from yesterday is supplied? https://github.com/robert-strandh/SICL/blob/master/Code/Evaluation-and-compilation/defmacro.lisp
10:06:18
beach
j`ey: I don't remember the defmacro from yesterday. Compilation takes place in an environment that the client defines. That environment must contain definitions of every macro that is used in the code to be compiled.
10:07:43
beach
j`ey: You can do the following experiment. In a SLIME REPL, type (macro-function 'with-output-to-string)
10:08:13
beach
j`ey: The client (SBCL or whatever) already has a definition of that macro. Cleavir just works with that.
10:10:28
beach
j`ey: Then it compiles the IF instead. Now IF it has to know how to compile, because that's a special operator.
10:16:09
j`ey
beach: ok, so the client will setup an environment that contains all the macros that CLHS has declared?
10:23:05
j`ey
still a little unclear how defmacro works. cleavir seems a (defmacro blah..) and calls the defmacro macro from the environment. does that update the environment to include the 'blah' macro?
10:25:52
beach
I can't understand the "cleavir seems a ..." part. But yes, DEFMACRO updates the environment so that it includes the definition of that macro.
10:27:19
beach
Try (defmacro foo (x) `(car ,x)), then (funcall (macro-function 'foo) '(foo (f y)) nil)
11:46:09
beach
So I think I am incapable of writing and debugging these MAKE-METHOD and CALL-METHOD wrappers without also having a version of macroexpand-all available.
12:01:48
MichaelRaskin
That's true, and if you start from the top-level/null lexical environments I only use my own implementation of lexical environments
12:02:16
MichaelRaskin
Which is, to be honest, quite limited, because I only need rough idea of what-means-what.
12:02:34
beach
So how do you deal with the possibility of the client version of MACROEXPAND being called with one of your environments?
12:03:14
MichaelRaskin
The client version of macroexpand cannot be called with one of my environments.
12:04:27
MichaelRaskin
Because my implementation's type is a class I define. And macroexpand wants implementation-specific environment object.
12:04:30
beach
It suffices to have a macro expanded in a lexical environment that calls macroexpand.
12:05:46
MichaelRaskin
If you want to macroexpand-all in a complicated lexical environment, you need one of the few tricks.
12:07:30
MichaelRaskin
Option 2: I can take the lexical environment object and a list of names I should check for macro definitions in that object.
12:08:03
MichaelRaskin
Option 3: if the local macros are used in a simple enough way, my heuristics might be enough.
12:08:46
beach
OK, I'm lost again. I don't know what a "macro-based macroexpand-all" is. And I don't know whether by "I can ..." you mean that this is something you could implement in the future, something I need to do in order to use your tool, or something else.
12:09:15
beach
I am sorry, I seem to be having this problem of understanding what is said in IRC. You need to be more explicit for me to understand.
12:10:51
MichaelRaskin
macro-macroexpand-all on a form will replace a form with some weird code that returns hthe full macroexpansion and should always be enough to take into account the local lexical environment
12:12:25
MichaelRaskin
(macroexpand-all form environment :names names) should also always work as long as all local names are listed in «names» parameter (listing extra is not a problem)
12:13:00
MichaelRaskin
Of course, it is possible you find a bug, in this case I will thank you and upload an updated version.
12:14:16
beach
I think I'll go with Cleavir anyway, because I need to expand macros that I defined in one of my first-class global environments. And I can't do it in the host, because they are macros that clash with those of the host, like DEFMETHOD.
12:15:24
MichaelRaskin
Well, yes, if you control the environment implementation it makes a lot more sense to use it.
12:15:51
beach
Yes, and in this case, like I said, I pretty much have to. Unless I rename all my macros just for the purpose of testing them.
12:36:56
Bike
was usually harder for me to understand than ir, though, because it's based on an expression mostly-but-not-actually-a-tree instead of a cfg
12:44:47
heisig
The example is nice though, it has blocks, bindings, tagbody, conditionals, type annotations and mutation of local variables. But it is definitely not a small expression :)
12:45:51
beach
The question I am asking myself now is whether an AST viewer would make it easier to understand.
12:47:30
beach
There are plenty of LOAD-TIME-VALUEs (at least in SICL code) for fetching global function cells.
12:52:41
heisig
Another possibility would be to introduce a pattern-based simplifier into AST-to-source, e.g., to eliminate superfluous PROGNs, convert direct lambda calls to LETs and so on.
12:56:29
beach
All these tools (existing and suggested) could be useful, so they should all be written. :)
13:01:46
beach
Let me start with macroexpand-all. I think it may be trivial, given that we have GENERATE-AST.
13:30:51
dim
any SBCL maintainer around? https://github.com/dimitri/pgloader/issues/832 looks like an SBCL issue
13:33:22
scymtym
dim: i think the person is trying to build with a version of nibbles that is too old for SBCL 1.4.11
13:34:18
pfdietz
These packages take terrible liberties with the naughty internal unexported bits of sbcl, and break when those change.
13:46:31
flip214
has anyone used CL-SAT already? With some quite complex input I get "the value NIL is not of type NUMBER" in CL-SAT:PRINT-CNF.
13:51:09
dim
pfdietz: pgloader build process should check out directly from QL, so I'm surprised; what might have happened is that the OP had a cache of quicklisp from a previous pgloader build
13:51:52
dim
I know nothing of the env where they try to use it, they know nothing of CL and QL and other bits, and it's all fine, usually
14:15:52
pfdietz
The problem here is that a widely used library is depending on unexported internal details of sbcl. Perhaps this means sbcl should have some sort of exported interface that the library could use instead, but sbcl developers are always going to feel free to change things others don't have license to depend on.
14:17:37
pfdietz
Any time one builds something that depends on unexported things, one is living in sin, at least to some extent.
14:28:26
beach
Bike: Oh, that's not what I meant. I meant that I could copy generate-ast and modify it to expand instead.
14:39:25
pfdietz
If you want to selectively inline it later, you declaim notinline right after the defun, then at local uses you can locally declare it inline.
14:40:04
pfdietz
The initial inline declaim causes the compiler to store the source for the later selective inlining.
14:58:23
shrdlu68
First macro I've written in a while: (defmacro post-incf (var) `(prog1 ,var (incf ,var)))
15:06:15
pfdietz
There's infrastructure in the CL standard for solving this problem, since it has to be solved for various builtin macros on places.
15:10:23
pfdietz
See the example I gave: the place expression can have side effecting subexpressions.
15:13:26
pfdietz
One also has to make sure the subexpressions of the place continue to be executed in left-to-right order. ansi-tests checked for all this.
15:15:37
shrdlu68
pfdietz: Hmm, shiftf suffers from the same shortcoming. Is there a way of avoiding that?
15:18:11
pfdietz
You can use GET-SETF-EXPANSION to explode places into pieces that can be assembled in the macro expansion. http://www.lispworks.com/documentation/lw70/CLHS/Body/f_get_se.htm
15:22:08
pfdietz
(LET* ((#:Y1080 Y) (#:G1081 (INCF I)) (#:OUT1082 X) (#:NEW1 (AREF #:Y1080 #:G1081)) (#:NEW1 Z))
15:48:15
beach
I believe I have a first version of MACROEXPAND-ALL working. It takes a cleavir environment that it operates in.
15:52:11
makomo
pfdietz: do the subforms of a place *have* to be evaluated left-to-right? i thought that was just a nice property/convention, but that it wasn't required for user-defined places
15:54:37
makomo
pfdietz: for example, i wrote an IFF place, which conditionally writes/reads one of the two places you provide to it as arguments, along with a condition form
15:56:22
Bike
' The evaluation ordering of subforms within a place is determined by the order specified by the second value returned by get-setf-expansion. For all places defined by this specification (e.g., getf, ldb, ...), this order of evaluation is left-to-right. '
15:57:43
makomo
i was going to write condf and other conditional places by basing them on iff, but i have to make iff take an optional second place first
16:26:50
slyrus1
right. I was going to say "presumably the AST stuff makes it possible to write such a thing nicely"?
16:27:28
beach
Sure. I did it the easy way. I basically translated the code from generate-ast (by hand) to cover all the cases.
17:00:40
Demosthenex
i'm just trying to confirm some edge cases in what i'm working on, and honestly i could just repeat things in the repl, but thought i should document it a bit better
17:04:37
beach
makomo: When possible, I write "random tests", i.e. I generate huge numbers of test cases to my code. when applicable, I write two version of my code, one "production" version, and one "trivial" version.
17:04:58
beach
makomo: The are supposed to behave the same way, but the "trivial" one is too slow for the final version.
17:06:00
makomo
beach: oh i see. how do you generate the random tests without basically solving the problem though? how do you bootstrap, i guess? :-)
17:06:34
Colleen
Clhs: function funcall http://www.lispworks.com/documentation/HyperSpec/Body/f_funcal.htm
17:07:59
beach
makomo: Then I check the tests against the code cover. If there are places that have not been executed, I try to modify my random-test generator so that those are included.
17:12:57
makomo
beach: hmm, i see how one could easily generate sequences of those operations, but how do you generate the results to test against, without already implementing the API? maybe i'm thinking about it the wrong way or something
17:13:34
makomo
or perhaps you rely on an already existing implementation of the same API (or a similar one)
17:15:12
beach
There could be bugs in the trivial one as well of course, but the probability that it would be "the same" bug in both versions is infinitesimal.
17:15:15
makomo
so you're relying on the fact that the trivial version is as trivial as can be, and that you will probably easily catch the errors since the code is simple
17:16:44
beach
But yeah, the "trivial" version is usually so simple that you can just look at it and be convinced that it is correct.
17:18:29
beach
The "trivial" version stays the same unless the API changes, which should not be the case (at least not very often).
17:33:44
pfdietz
The first is generative: have a grammar (explicit or implicit) and attach probabilities to different production rules.
17:34:22
pfdietz
The second is mutational: given a corpus of interesting inputs, generate new inputs by changing or combining them in various ways.
17:34:52
pfdietz
One can also add constraints to the inputs to try to bias them toward more interesting executions.
17:35:33
pfdietz
And if one can instrument the code under test then the inputs can be tweaked based on whether in the code the executions go ("gray box fuzzing").
17:35:39
beach
pfdietz: I often need my generated operations to follow a Markov process. Unless I have a certain probability of generating long sequences of "the same" operation, my coverage won't be adequate.
17:37:17
pfdietz
Yes. There are games you can play with that, like "swarm testing", where you randomly prune down the set of production rules before each run of the test generator. Empirically this tends to find more bugs. I used this technique in the CL random test generator.
17:39:08
pfdietz
The experience with random testing is, I think, that different test generators cover different parts of bug space, so for a really large system you want diversity.
17:42:53
makomo
pfdietz: interesting, but you do arrive at the same "problem" of having to already have a working version of what you want to test, right? i.e. once you generate the inputs (sequence of operations, etc.) you need to somehow generate the outputs (the actual solution to those inputs)
17:43:30
pfdietz
You identify properties the code should have, even if they may not fully specify what the code should do.
17:43:59
beach
makomo: I can assure you that the "working version" is trivial to write in many cases.
17:44:24
pfdietz
For the Lisp testing I was doing, the property was that (for conforming CL) the code should do the same thing if compiled w. various different declarations, or if eval-ed.
17:46:38
makomo
beach: mhm, but it's interesting how there's a fundamental circularity/bootstrapping problem in there. either you have an existing implementation or you use your brain (which is just another implementation) and write out a finite number of tests yourself
17:47:30
pfdietz
If your code is loaded with assertions, each of them is an opportunity for testing. No assertions should fail.
17:49:01
makomo
pfdietz: true. how did you test that the two versions of code did the same thing? i suppose the stuff that was randomly generated were the declarations, not the code itself? the code was predetermined and you knew what to test for?
17:49:56
makomo
beach: yeah, i understand that. i get that it's a non-problem in practice, but fundamentally the ""problem"" is there
17:50:06
pfdietz
I randomly generated code and randomly generated inputs, and then checked that no errors were thrown (this was conforming code that would not throw errors) and that it had the same result on the same inputs.
17:50:07
beach
It takes less time to write the implementation of the trivial version than it takes to write the tests.
17:51:36
beach
makomo: I.e., that I think it is extremely hard to find abstractions for a testing framework.
17:51:49
makomo
beach: not quite. i just wanted to comment on the need of having to have an existing implementation in order to begin testing, no matter how trivial it is
17:59:11
beach
makomo: My main point here is that I don't believe in "testing frameworks" because I don't see how such a thing could capture even the most useful testing techniques that I know of.
18:18:58
shka_
but i like how i can use asdf extension to specify :test-file and i can simply use is and ok assertion
18:19:10
j`ey
this is my testing framework: (defun test (val) (cond ((eq val nil) (princ "E")) (t (princ "."))))
18:19:52
pfdietz
The question you want to ask is not "which testing framework?" but rather "what specifically do you want from a testing framework?"