freenode/#lisp - IRC Chatlog
Search
6:28:48
lukego
I've been trying to pick a unit testing framework. Just makes my head hurt with all the options and trying to interpret the reviews over time. Going to try 1am to start with.
6:30:05
beach
My theory is that the idea of an abstraction such as a "unit testing framework" is nonsensical, and that this is why there are so many of them and none of them can do everything that is needed.
6:33:30
beach
Take Flexichain, for instance. With most testing frameworks, I would have had to enumerate all possible situations, but the number of combinations that can happen between resizing, rotation, moving the gap is truly huge, and it would not be humanly possible to figure that out manually so that every combination is tested.
6:35:39
beach
So instead, I wrote a trivial implementation of the same protocol, and I generate random operations using sort of a first-order Markov model.
6:35:44
lukego
Yeah. I'm thinking of starting with a combo of a dumb/convenient test-runner that can then call into a more "combinatorial" kind of a test case generator. I was thinking check-it but leaning towards just giving Screamer <duration> to find a bad case
6:36:01
beach
I then compare the result of the trivial implementation to the result of the real one.
6:37:28
beach
OK, so now give me the name of a testing framework that would give me support for this way of testing things, which I find way superior to enumerating individual tests maually.
6:39:39
beach
ASSERT is an excellent testing framework that has the additional advantage that it is in the Common Lisp standard.
6:43:37
fiddlerwoaroof
But, I'm more interested in "oracle" tests: for the sort of code I write, I don't usually care if the tests are exhaustive as long as they run quickly and cast common mistakes I make when I write code.
6:56:34
pyc
Was there some quote about some language which said something like it makes easy problems easy and difficult problems possible to solve? Anyone remembers in what context or where this quote was said?
7:04:14
jackdaniel
the context was: lisp is a language where easy problems are trivial to solve, and hard problems are possible to solve (in contrast to easy problems are hard and hard problems are impossible to solve in /some/ languages)
7:08:03
lukego
beach: that's how check-it is supposed to work. you define generators for test data (e.g. sequences of editing operations) and invariants that must hold (e.g. model and real impl return same results.) then it makes a randomized search for inputs that fail. if it finds a failure then it moves on to "shrinking" i.e. trying to find a smaller version of the failing input (e.g. smaller numbers, shorter lists, etc)
7:09:33
lukego
beach: but check-it doesn't seem to have the "assert" part of testing. So you'd still need some way to organize tests e.g. give them names and have a way of running some/all of them. and 1am seems like a reasonable solution to that - just a page of code where a test is a function that's registered on the *tests* list.
7:10:50
lukego
... and if I'm doing generator-driven testing then I'm also asking myself whether to use check-it, which is a relatively simple implementation tailored specifically for test-case generation, or Screamer, being more complex and more general (but potentially less suitable e.g. in how it navigates the search space)
7:13:23
pyc
jackdaniel: found the exact quote. I read it in Practical Common Lisp. It is in chapter 1. >> Perl advocates like to say that Perl "makes easy things easy and hard things possible" and revel in the fact that, as the Perl motto has it, "There's more than one way to do it."
7:22:40
lukego
beach: this kind of "differential testing" you describe does sound pretty great to me. if you have two implementations and reason to believe they won't have the *same* bugs then it stands to reason that you can compare them to shake the bugs out of both.
7:32:14
lukego
fiddlerwoaroof: thanks for the datapoint. 5am does seem like a safe option. I'm a bit cautious just because it seems like quite a bit of code and probably a lot that doesn't apply to me e.g. I'm not connecting to databases and stuff
7:36:08
lukego
Hey if you wanted to check if all the strings in a list were the same length would it seem natural to write (all #'= #'length strings)? or what's idiomatic lisp for that?
7:39:50
beach
lukego: Exactly. It is highly unlikely that the two would behave in the same incorrect way. And since one is trivial (it is an "editable sequence" implemented as a list) it is already highly unlikely that that one will be incorrect in the first place.
7:44:53
lukego
I think I'll keep it stupid and write (apply #'= (mapcar #'length strings)) because ALL sounds like such a serious function name and I'd have to think too much about how it relates to existing functions like EVERY.
7:46:46
lukego
Hey cool if there were a website like compiler explorer that would let you write a fragment of code, e.g. CALL-ARGUMENT-LIMIT, and shows you what it evaluates to in all major lisps
7:56:58
lukego
maybe Nix could handle that part. but it's not really an urgent problem for me, I'm not trying to write especially portable code, I just have a voice in my head saying "CALL-ARGUMENT-LIMIT" when I write things like (apply #'= ...)
7:57:47
jdz
lukego: Which should immediately trigger the "can't I use REDUCE, though?" question in your brain.
8:15:03
lukego
oh I like the curry solution. I'm not sure how the REDUCE would look just because there are kinda two accumulators i.e. truthiness and target length
8:17:30
lukego
maybe some kind of (reduce #'assert= strings :key #'length) but unlikely to be the simplest solution
8:18:09
fiddlerwoaroof
lukego: you can put the two accumulators in a datastructure of some kind. e.g. (list acc1 acc2) or something
8:18:23
lukego
(= (reduce #'min strings :key #'length) (reduce #'max strings :key #'length)) but now I'm just being silly.
8:20:18
lukego
I'm mostly thinking of the norvig/pitman "rule of english translation." I want to check that "strings all have equal length." so (all-equal-length strings) would be good but that's a whole function. (all #'equal #'length strings) would be fine except that such an ALL function doesn't exist.
8:21:20
lukego
but, meta-solution, the real problem here is that I feel a lot of pressure to keep my unit test code hyper-concise so that it doesn't distract from the actual implementation code, but I'll solve that by just putting it in another file and letting it grow as much as it needs to.
8:23:40
lukego
I actually enjoy this kind of obsessing about Lisp programming. it seems like one of the few languages where you can actually get value out of it. the returns may be diminishing but there's *is* always a better way that you can find...
8:26:30
jdz
The test of the machine is the satisfaction it gives you. There isn't any other test. If the machine produces tranquillity it's right. If it disturbs you it's wrong until either the machine or your mind is changed. The test of the machine's always your own mind. There isn't any other test. — Robert M. Pirsig, Zen And The Art Of Motorcycle Maintenance
8:27:29
lukego
Great quote. Pity about the whole descent-into-madness connotation but I'll take it :)
8:52:29
lukego
I'd quite like to adopt using CURRY but it seems like I'll often want to curry things that are macros rather than functions. The nearest solution that comes to mind is Serapeum's OP macro.
8:55:44
lukego
like I was happy with a test case (every #'is *valid-examples*) but then how to test *invalid-examples* not as (is E) but (signals error E)
8:57:14
lukego
though my first example is a bit weird, calling EVERY for side effects, since #'is does an assertion
9:29:47
lukego
So I'm not really satisfied with with LOOP nor with classics like DOLIST. Is ITERATE the only other real game in town?
9:34:11
beach
lukego: LOOP is fine for most cases. What problem do you have with it? I mean, it has the advantage of being in the standard, so that no external dependency is needed.
9:35:09
lukego
SERIES does CPS though right, or am I mixing it up with Screamer? I want to keep the bulk of my code compatible with the SLIME debugger
9:35:56
lukego
beach: Yeah. I'd usually use LOOP but pn this project I'm okay with taking on dependencies and being a bit idiosyncratic. I'm having two common issues with loop -
9:36:33
lukego
One is code moving towards the right margin of the screen. Maybe though this is me not taking full advantage of the quirks of the Emacs indentation of LOOP somehow.
9:37:19
lukego
The other is having to nest loops - causing more indentation - when I want to iterate not in parallel but as in (dotimes (y height) (dotimes (x width) ...)) when e.g. "list comprehensions" kind of abstractions would do that in one go
9:38:48
beach
lukego: I also find myself saving indentation by introducing a FOR clause, using it as a LET*.
9:38:49
lukego
I basically want to have two spaces of indentation in the bodies of my loops, in the spirit of &body, but that's not always easy to get with LOOP.
9:39:23
lukego
This FOR macro linked above ^ seems to have a neat solution to that i.e. having a more LET-style syntax
9:49:15
fiddlerwoaroof
It's completely undocumented so far, but I really like transducers for this: you right your code with functions like MAPCAR/REMOVE-IF-NOT but you skip all the intermediate lists
9:50:33
fiddlerwoaroof
https://github.com/fiddlerwoaroof/data-lens/blob/master/t/transducers.lisp#L246-L252
10:03:54
lukego
I'm actually fine with having lots of intermediate lists, I'm mostly looking for something that feels a bit poetic.
10:05:25
lukego
heisig: Thanks for the reminder of plain old recursion :). That's how I'd write e.g. Scheme code quite happily. Maybe with serapeum macros like NLET and COLLECTING that can be poetic too.
10:08:25
lukego
though NLET has this weird tail-position-only restriction and using ordinary recursion risks creating big control stacks that are boring to look at in the debugger.
10:59:02
lukego
Hey what's a neat way to write (cdr (assoc key a-list)) such that it errors when the key is not found?
11:06:38
beach
OR takes Boolean values and the return value of ASSOC is not a Boolean. It is either something useful, or a default value that happens to be NIL.
11:07:59
beach
lukego: Presumably, you want to hide the fact that you have an alist anyway, and present it as a dictionary. Then the protocol function would do the ASSOC and check for NIL.
11:09:15
lukego
This is some low-brow code where I'm happy to be concrete that it's an alist. Though having said that, it's the concreteness I want, not necessarily the a-list. Maybe I can punt and just use a function instead.
11:09:56
fiddlerwoaroof
beach: well, if NIL is a valid value for a key, you don't want to error when the key is present but associated with nil
11:10:27
beach
fiddlerwoaroof: I think the discussion was about return values and not about keys. Am I wrong?
11:11:16
fiddlerwoaroof
I don't think so (assoc :a '((:a . nil))) isn't equivalent to (assoc :a ())?
11:18:49
lukego
sorry about the rapid-fire but is there any setf-family macro like (updatef str #'string-upcase) ? short for (setf str (string-upcase (getf str)))
11:23:06
lukego
moon-child: yeah. just wondering there's a reason it doesn't already exist e.g. in alexandria which has maxf/minf/nconcf/etc.
11:24:26
lukego
though looking at that code ^ mulitple evaluations of x will be an issue so maybe it's hard
11:26:26
lukego
w00t! cache hit! the next thing I was going to ask about already exists in serapeum under the name MAPPLY :)
11:29:02
splittist
there's probably a semantically more accurate way of describing what you're doing than UPCASE-PLIST-VALUE-IN-PLACE
11:34:43
lukego
Yeah. I have a list of lists ((#\X 0 0 1 1) ..) and I want to map that character in the first element position to a longer and more meaningful name e.g. #\X -> :XYLOPHONE.
11:35:34
lukego
though I realize that I want to change some of the numbers too and then (mappend (op (ch x0 y0 x1 y1) ...) ...) seems reasonable
11:39:32
lukego
also... now that I start accumulating some unit tests it looks an awful lot like design-by-contract... like the tests are mostly just asserting things about the inputs and outputs to functions. maybe I should be using function types instead here.
11:42:49
lukego
Can you use declaimed function types for design-by-contract in practice? What do I need to do to get SBCL to promise to check types like this at runtime?
11:49:29
fiddlerwoaroof
If you want Eiffel-style contracts, there's this: https://github.com/sellout/quid-pro-quo
11:53:11
lukego
Just experimentally declaring function types seems to work when I set optimize of SAFETY >= 1
11:53:34
moon-child
beach: that very document later (pg84) uses ‘or’ to supply a default value. Dogma is failure
11:53:49
estest
lukego: Here's what the manual says: http://www.sbcl.org/manual/index.html#Declarations-as-Assertions
11:54:06
lukego
I'll take a look at the DBC library. It would be kind of nice separation of concerns if the implementation if self-testing and the test code mostly just needs to feed it inputs and let it selftest
11:59:23
lukego
yeah nah in my own application code I prefer to limit the number of variables, e.g. platforms and compilers, to as narrow as possible. I'm not writing libraries for other people here.
12:00:24
lukego
and I suppose that I _am_ writing common lisp code if I specify the detailed types of my key functions using Common Lisp type declarations, it's not my fault if some compilers choose to ignore that :)
12:01:01
jackdaniel
declare is described in the standard as a "promise to the compiler", not a safety measure
12:03:28
lukego
ok that's on me then. but that's always been my development philosophy. cheat as much as possible to get the program to the point where it is worth porting, rather than get distracted with portability and risk that the program doesn't get finished at all
12:04:14
lukego
easier to port a working non-portable program than to finish an incomplete portable program.
12:05:46
jackdaniel
I'm not convinced that this is a proper justification of sloppy declarations, but it is your program so there is no need for me to be convinced ,)
12:10:22
lukego
Just seems like it will lead to more clutter to me. I'm already putting effort into untangling my test cases from my application logic so that I can look at each one in peace from the other
12:11:26
lukego
(make-foo :x 1 :y 1) verses (let ((foo (make-foo :x 1 :y 1) (check-type foo (satisfies valid-foo?)) foo)
12:12:49
lukego
I guess that's hard to answer since it depends on how initialization is done and how much depends on the context of the callre
12:14:21
lukego
Maybe. There's also e.g. the quid-pro-quo library that provides another valid solution, which has a nice separation-of-concerns feel about it to me, but that's not in quicklisp and frankly has quite a bit of machinery in its implementation.
12:15:03
lukego
there are also around methods and before methods and after methods... so don't tell me there isn't more than one way to do it thankyouverymuch :)
12:20:03
estest
Making use of SBCL's behavior of type checking declares is a nice bonus of developing in SBCL, much like its generally fast assembly output (especially when you declare types) is a nice bonus, though if your program actually should rely on types being checked, an explicit check-type or some other mechanism is a good idea.
12:22:25
lukego
quid-pro-quo seems to be not in quicklisp, and to have dependencies that are also not in quicklisp, which is a bit of a enthusiasm damper
12:27:13
pyc
Need feedback with code style: https://plaster.tymoon.eu/view/2316 I have auto-fill-mode enabled, so as I type long strings, Emacs automatically inserts line-breaks. Is this good code formatting? I am curious about the two spaces that Emacs has inserted as prefix after each line-break.
12:27:26
lukego
and trying to load it is giving me a "This is probably a bug in SBCL itself" error involving metaclass compilation, which sure does play into my "gee that looks like a lot of implementation machinery" first impression
12:27:36
estest
I also find leaning on the implementation against portability to only be worth it when you need a particular feature it has for which there's not a good portability layer. Like why use sb-thread:make-thread when bordeaux-threads exists. Otherwise being portable and even testing in other implementations from time to time doesn't really slow me down, and can sometimes find tricky bugs earlier.
12:28:32
lukego
Just now I'm only trying to find a sufficiently pleasant way to write testing code such that I will actually do it. The main barrier to instrumenting my code with tests is clutter and verbosity.
12:31:31
lukego
Maybe a good CL lawyer can help me find an acceptable formulation of the serapeum:-> function type declaration macro, such that function type declarations are strictly for documentation purposes, and any utility in the form of test coverage on SBCL is purely accidental
12:33:50
lukego
(and in truth I'd expect the really proper test coverage to come from approaches like beach outlined, with high-level functional testing, and what I'm trying to supplement that with now is just some scaffolding to help me get the nuts and bolts right before the structure is in place)
12:35:09
heisig
lukego: For my last project, I defined my own document-[function,variable,...] macros that expand into (setf (documentation ...) ...) and some fancy extras.
12:35:13
shka_
secondly: it allows implementation to use the inferred types to warn when you are using aref on list ans so one
12:35:53
shka_
however, you should not count on these to give you runtime errors when types mismatch
12:35:55
heisig
One such extra was that it would take a list of forms and add them (and the results of evaluating them) as examples to the documentation string.
12:36:09
estest
pyc: I'd prefer the spacing to align the text with the first character of the string, i.e. the A in "Alice is.." is on the same column as the b in "been working".
12:36:35
estest
But I'd only do that for docstrings or strings I know will be processed for display and have the newlines/lead spacing taken care of; for a raw record your example suggests you probably don't want newlines in the source string, so consider letting it be long (and word-wrapping) or concatenating broken up bits...
12:37:40
shka_
lukego: so in summary, ftype declarations are useful in a certain context, but not beyond it
12:38:45
lukego
shka_: that sounds reasonable. but if I'm honest I'm thinking of them as assertions that can be allowed to propagate (and inhibited from doing so in performance sensitive contexts)
12:38:59
shka_
my personal rule of thumb is to use ftype declarations for internal functions, but not for the interface functions
12:39:04
lukego
I'll see how I go, probably they are not actually the right tool for the job, but ask me again in ten minutes..
12:40:34
shka_
use check-type for stuff that is expected to be called by the user, you may use ftype and declare for the internal stuff (though, at least initially check-type is not a bad idea either)
12:41:40
shka_
besides, sbcl can deduce returned function type even without the explicit declarations
12:42:38
lukego
yeah but there are types and then there are types e.g. I might want to provide a (satisfies foo) type that's stricter than what sbcl would infer.
12:43:38
lukego
but maybe it's time for me to "read the room" and see that SBCL-specific hacks are not popular topics for discussion in here even in the early freewheeling days of a project that are always spent trying out and discarding bad ideas anyway
12:44:35
shka_
lukego: well, in that case you may define returned type of the function, but check-type the input types
12:45:18
shka_
overall, simply don't think about type declaration as suitable for checking the input types
12:45:26
lukego
I'm much more attracted to out-of-band top-level declarations like (-> read-floorplan (string function) valid-floorplan)) than anything that adds multiple lines to the actual function definitions
12:45:39
lukego
but maybe that will pass, I have been too busy fighting with you guys to actually try it yet :)
12:46:21
lukego
that's an extreme position since I'm using SBCL and SBCL says that it treats types as assertions and guarantees that they will be tested under specific conditions that I can ensure
12:46:22
shka_
well, as i said, it is FINE for internal functions that are not exposed, it is NOT FINE for external functions
12:53:02
splittist
Having just reviewed a project of mine with function names like FROB and PROCESS I will refrain from giving further programming advice (:
12:54:36
lukego
One happy thought is that with any kind of test instrumentation in my application code - whether with CHECK-TYPE, or FTYPE, or QUID-PRO-QUO - I can probably throw away these boring unit tests that I've been writing. The main purpose of those is just to keep silly errors from propagating e.g. a function returning the wrong object and finding out later when it's hard to track down what happened.
12:55:51
lukego
The situation I want to avoid is "oh, whoops, that innocent refactoring I did in PREINIT-FROB forgot to set the FIZZ slot, and that's why I have these twenty *sldb* buffers popping up all over the place
12:56:26
lukego
I should have known that if I found myself writing unit tests I was probably doing something wrong...
13:35:33
pyc
marcoxa: do you know why Emacs decides to insert two spaces to indent all the continuation lines of my long string in the example I shared earlier: https://plaster.tymoon.eu/view/2316#2316
13:36:59
beach
pyc: Emacs is not great when it comes to indenting Common Lisp code, but it's the best we have.
13:38:15
andreyorst
beach: I've thought that Emacs when used with SLIME or SLY gets some info from runtime to indent code more or less correctly?
13:38:56
beach
andreyorst: Yes, but it is fundamentally broken, because of the way it decides what is code and what isn't.
13:40:29
beach
andreyorst: Try the following experiment: In a buffer in Lisp mode type (let ((aaaaaaa<RETURN> and observe the indentation of the next line
13:41:41
beach
So it has not analyzed the text enough to know that we are dealing with a LET binding.
13:43:46
andreyorst
with let I don't see any problems, it aligns the indentation to last opened parenthesis
13:43:58
Nilby
The depth of my disgruntlement with Emacs is nearly unfathomable. But it's STILL the least sub-optimal.
13:45:12
andreyorst
yes, I understand, I just thought that it does a bit more analysis, and will think that prog1 is a binding, not a call, that's what kinda weird