freenode/#lisp - IRC Chatlog
Search
6:28:48
lukego
I've been trying to pick a unit testing framework. Just makes my head hurt with all the options and trying to interpret the reviews over time. Going to try 1am to start with.
6:30:05
beach
My theory is that the idea of an abstraction such as a "unit testing framework" is nonsensical, and that this is why there are so many of them and none of them can do everything that is needed.
6:33:30
beach
Take Flexichain, for instance. With most testing frameworks, I would have had to enumerate all possible situations, but the number of combinations that can happen between resizing, rotation, moving the gap is truly huge, and it would not be humanly possible to figure that out manually so that every combination is tested.
6:35:39
beach
So instead, I wrote a trivial implementation of the same protocol, and I generate random operations using sort of a first-order Markov model.
6:35:44
lukego
Yeah. I'm thinking of starting with a combo of a dumb/convenient test-runner that can then call into a more "combinatorial" kind of a test case generator. I was thinking check-it but leaning towards just giving Screamer <duration> to find a bad case
6:36:01
beach
I then compare the result of the trivial implementation to the result of the real one.
6:37:28
beach
OK, so now give me the name of a testing framework that would give me support for this way of testing things, which I find way superior to enumerating individual tests maually.
6:39:39
beach
ASSERT is an excellent testing framework that has the additional advantage that it is in the Common Lisp standard.
6:43:37
fiddlerwoaroof
But, I'm more interested in "oracle" tests: for the sort of code I write, I don't usually care if the tests are exhaustive as long as they run quickly and cast common mistakes I make when I write code.
6:56:34
pyc
Was there some quote about some language which said something like it makes easy problems easy and difficult problems possible to solve? Anyone remembers in what context or where this quote was said?
7:04:14
jackdaniel
the context was: lisp is a language where easy problems are trivial to solve, and hard problems are possible to solve (in contrast to easy problems are hard and hard problems are impossible to solve in /some/ languages)
7:08:03
lukego
beach: that's how check-it is supposed to work. you define generators for test data (e.g. sequences of editing operations) and invariants that must hold (e.g. model and real impl return same results.) then it makes a randomized search for inputs that fail. if it finds a failure then it moves on to "shrinking" i.e. trying to find a smaller version of the failing input (e.g. smaller numbers, shorter lists, etc)
7:09:33
lukego
beach: but check-it doesn't seem to have the "assert" part of testing. So you'd still need some way to organize tests e.g. give them names and have a way of running some/all of them. and 1am seems like a reasonable solution to that - just a page of code where a test is a function that's registered on the *tests* list.
7:10:50
lukego
... and if I'm doing generator-driven testing then I'm also asking myself whether to use check-it, which is a relatively simple implementation tailored specifically for test-case generation, or Screamer, being more complex and more general (but potentially less suitable e.g. in how it navigates the search space)
7:13:23
pyc
jackdaniel: found the exact quote. I read it in Practical Common Lisp. It is in chapter 1. >> Perl advocates like to say that Perl "makes easy things easy and hard things possible" and revel in the fact that, as the Perl motto has it, "There's more than one way to do it."
7:22:40
lukego
beach: this kind of "differential testing" you describe does sound pretty great to me. if you have two implementations and reason to believe they won't have the *same* bugs then it stands to reason that you can compare them to shake the bugs out of both.
7:32:14
lukego
fiddlerwoaroof: thanks for the datapoint. 5am does seem like a safe option. I'm a bit cautious just because it seems like quite a bit of code and probably a lot that doesn't apply to me e.g. I'm not connecting to databases and stuff
7:36:08
lukego
Hey if you wanted to check if all the strings in a list were the same length would it seem natural to write (all #'= #'length strings)? or what's idiomatic lisp for that?
7:39:50
beach
lukego: Exactly. It is highly unlikely that the two would behave in the same incorrect way. And since one is trivial (it is an "editable sequence" implemented as a list) it is already highly unlikely that that one will be incorrect in the first place.
7:44:53
lukego
I think I'll keep it stupid and write (apply #'= (mapcar #'length strings)) because ALL sounds like such a serious function name and I'd have to think too much about how it relates to existing functions like EVERY.
7:46:46
lukego
Hey cool if there were a website like compiler explorer that would let you write a fragment of code, e.g. CALL-ARGUMENT-LIMIT, and shows you what it evaluates to in all major lisps
7:56:58
lukego
maybe Nix could handle that part. but it's not really an urgent problem for me, I'm not trying to write especially portable code, I just have a voice in my head saying "CALL-ARGUMENT-LIMIT" when I write things like (apply #'= ...)
7:57:47
jdz
lukego: Which should immediately trigger the "can't I use REDUCE, though?" question in your brain.
8:15:03
lukego
oh I like the curry solution. I'm not sure how the REDUCE would look just because there are kinda two accumulators i.e. truthiness and target length
8:17:30
lukego
maybe some kind of (reduce #'assert= strings :key #'length) but unlikely to be the simplest solution
8:18:09
fiddlerwoaroof
lukego: you can put the two accumulators in a datastructure of some kind. e.g. (list acc1 acc2) or something
8:18:23
lukego
(= (reduce #'min strings :key #'length) (reduce #'max strings :key #'length)) but now I'm just being silly.
8:20:18
lukego
I'm mostly thinking of the norvig/pitman "rule of english translation." I want to check that "strings all have equal length." so (all-equal-length strings) would be good but that's a whole function. (all #'equal #'length strings) would be fine except that such an ALL function doesn't exist.
8:21:20
lukego
but, meta-solution, the real problem here is that I feel a lot of pressure to keep my unit test code hyper-concise so that it doesn't distract from the actual implementation code, but I'll solve that by just putting it in another file and letting it grow as much as it needs to.
8:23:40
lukego
I actually enjoy this kind of obsessing about Lisp programming. it seems like one of the few languages where you can actually get value out of it. the returns may be diminishing but there's *is* always a better way that you can find...
8:26:30
jdz
The test of the machine is the satisfaction it gives you. There isn't any other test. If the machine produces tranquillity it's right. If it disturbs you it's wrong until either the machine or your mind is changed. The test of the machine's always your own mind. There isn't any other test. — Robert M. Pirsig, Zen And The Art Of Motorcycle Maintenance
8:27:29
lukego
Great quote. Pity about the whole descent-into-madness connotation but I'll take it :)
8:52:29
lukego
I'd quite like to adopt using CURRY but it seems like I'll often want to curry things that are macros rather than functions. The nearest solution that comes to mind is Serapeum's OP macro.
8:55:44
lukego
like I was happy with a test case (every #'is *valid-examples*) but then how to test *invalid-examples* not as (is E) but (signals error E)
8:57:14
lukego
though my first example is a bit weird, calling EVERY for side effects, since #'is does an assertion
9:29:47
lukego
So I'm not really satisfied with with LOOP nor with classics like DOLIST. Is ITERATE the only other real game in town?
9:34:11
beach
lukego: LOOP is fine for most cases. What problem do you have with it? I mean, it has the advantage of being in the standard, so that no external dependency is needed.
9:35:09
lukego
SERIES does CPS though right, or am I mixing it up with Screamer? I want to keep the bulk of my code compatible with the SLIME debugger
9:35:56
lukego
beach: Yeah. I'd usually use LOOP but pn this project I'm okay with taking on dependencies and being a bit idiosyncratic. I'm having two common issues with loop -
9:36:33
lukego
One is code moving towards the right margin of the screen. Maybe though this is me not taking full advantage of the quirks of the Emacs indentation of LOOP somehow.
9:37:19
lukego
The other is having to nest loops - causing more indentation - when I want to iterate not in parallel but as in (dotimes (y height) (dotimes (x width) ...)) when e.g. "list comprehensions" kind of abstractions would do that in one go
9:38:48
beach
lukego: I also find myself saving indentation by introducing a FOR clause, using it as a LET*.
9:38:49
lukego
I basically want to have two spaces of indentation in the bodies of my loops, in the spirit of &body, but that's not always easy to get with LOOP.
9:39:23
lukego
This FOR macro linked above ^ seems to have a neat solution to that i.e. having a more LET-style syntax
9:49:15
fiddlerwoaroof
It's completely undocumented so far, but I really like transducers for this: you right your code with functions like MAPCAR/REMOVE-IF-NOT but you skip all the intermediate lists
9:50:33
fiddlerwoaroof
https://github.com/fiddlerwoaroof/data-lens/blob/master/t/transducers.lisp#L246-L252
10:03:54
lukego
I'm actually fine with having lots of intermediate lists, I'm mostly looking for something that feels a bit poetic.
10:05:25
lukego
heisig: Thanks for the reminder of plain old recursion :). That's how I'd write e.g. Scheme code quite happily. Maybe with serapeum macros like NLET and COLLECTING that can be poetic too.
10:08:25
lukego
though NLET has this weird tail-position-only restriction and using ordinary recursion risks creating big control stacks that are boring to look at in the debugger.
10:59:02
lukego
Hey what's a neat way to write (cdr (assoc key a-list)) such that it errors when the key is not found?
11:06:38
beach
OR takes Boolean values and the return value of ASSOC is not a Boolean. It is either something useful, or a default value that happens to be NIL.
11:07:59
beach
lukego: Presumably, you want to hide the fact that you have an alist anyway, and present it as a dictionary. Then the protocol function would do the ASSOC and check for NIL.
11:09:15
lukego
This is some low-brow code where I'm happy to be concrete that it's an alist. Though having said that, it's the concreteness I want, not necessarily the a-list. Maybe I can punt and just use a function instead.
11:09:56
fiddlerwoaroof
beach: well, if NIL is a valid value for a key, you don't want to error when the key is present but associated with nil
11:10:27
beach
fiddlerwoaroof: I think the discussion was about return values and not about keys. Am I wrong?
11:11:16
fiddlerwoaroof
I don't think so (assoc :a '((:a . nil))) isn't equivalent to (assoc :a ())?
11:18:49
lukego
sorry about the rapid-fire but is there any setf-family macro like (updatef str #'string-upcase) ? short for (setf str (string-upcase (getf str)))
11:23:06
lukego
moon-child: yeah. just wondering there's a reason it doesn't already exist e.g. in alexandria which has maxf/minf/nconcf/etc.