freenode/#clasp - IRC Chatlog
Search
11:27:30
drmeister
I'm going to pull another copy of clasp and cando, merge dev into cst and debug this issue in esrap with the help of source tracing and esrap trace.
11:28:13
drmeister
Something broke in the last couple of days and I need to fix it fast - so that we can get this very important demo running for next Thursday.
11:28:33
drmeister
Or by the end of today I have to start regressing to older versions to find one that works.
11:30:03
drmeister
We can't render molecules because that requires parsing sybyl type rules and that requires esrap .
11:32:06
attila_lendvai
drmeister: do you know how to cherry-pick specific commits? maybe for the demos you should stick to the testing branch. create a branch for the demo and cherry pick selected commits if needed...
11:34:27
drmeister
It looks like it's been in the code since at least Monday - but I've been traveling and giving demos. When I tried to demo the development version it locked up - probably this problem.
11:34:47
drmeister
I dropped back to a docker demo but that precluded me from demonstrating some of the latest features.
11:37:01
attila_lendvai
drmeister: maybe for the demos you should stick to the testing branch? create a branch for the demo and cherry pick selected commits if they are needed for the demo (it's simple with gitk as long as there are no line-diff conflicts).
11:37:24
drmeister
The testing branch doesn't let me demonstrate the latest features that I need to demonstrate by next Thursday
11:39:18
drmeister
esrap:parse is a compiler macro - yes. But the problem happens when it invokes the compiled parser.
11:39:45
attila_lendvai
*if* it's one self-contained commit/feature that's needed for the demo, then cherry-picking can be useful to bring that one change over without all the instability of the other new developments.
11:41:34
drmeister
(ESRAP::EXEC-NEGATION #<CLOSURE-WITH-SLOTS@0x11ce34e88 LAMBDA :ftype :FUNCTION lambda-list: NIL :fptr 0x1261ff610> (NOT (OR #\NEWLINE #\PAGE)) "WILDATOM XX C N O S P" 8 21)
11:42:13
drmeister
It's failing - I'm not sure if that should happen or not. Is it looking for #\NEWLINE or #\PAGE in that substring and not finding it?
11:48:05
kpoeck
@bike, i see you might be in more import battles, but I did 2 pr's for 2 of the ansi-issues I reported
12:13:51
heisig
kpoeck: ELS ends typically around 17:30, but followed by the conference dinner and some pub crawl.
12:16:17
scymtym
drmeister: Bicyclidine gave the correct explanation. did you try ESRAP:TRACE-RULE? alternatively, what does (funcall (esrap::compile-expression '(not (or #\newline #\page))) "WILDATOM XX C N O S P" 8 21) return?
12:20:37
drmeister
We added source tracking to clasp over the past couple of months - but it's in a different branch because there was a hit to performance with a change in the cleavir compiler that we wanted to fix.
12:21:15
drmeister
If I had source tracking available to debug this problem I'm pretty sure it would be a lot easier. I'd be able to see source line information on every expression in the backtrace.
12:21:33
drmeister
Without this debugging is very timeconsuming and frustrating. I've spent years doing it.
12:24:54
Bicyclidine
no, we have one system that runs cando, and one with source tracking. you want one that runs cando with source tracking, and that's much dicier.
12:25:22
Bicyclidine
i think regular debugging is the way to go here. it sounds like esrap even has facilities for it.
12:27:24
Bicyclidine
i know you want to show off new features, but if we rush every feature to make some demo they're all going to be broken.
12:30:51
drmeister
Yeah - it's not isolated - but I can do the following to reproduce the problem once cando starts.
12:31:06
drmeister
(esrap:parse 'leap.antechamber-type-definition-parser::wild-atom-line "WILDATOM XX C N O S P")
12:32:16
scymtym
drmeister: did you try (funcall (esrap::compile-expression '(OR #\NEWLINE #\PAGE)) "WILDATOM XX C N O S P" 8 21) ?
12:35:57
drmeister
That "fix" I made to READ is high on my suspect list. I don't see the connection - but it was dealing with whitespace and I can see that it has problems.
12:43:19
drmeister
Why should it return successful-parse? There is no #\NEWLINE or #\PAGE in that substring.
12:44:30
drmeister
Just to be clear - I evaluated: (funcall (esrap::compile-expression '(OR #\NEWLINE #\PAGE)) "WILDATOM XX C N O S P" 8 21)
12:48:07
scymtym
the first result in that paste should be a successful-parse - and was when i tried the same on SBCL
12:48:59
stassats
drmeister: what about (funcall (esrap::compile-expression '(NOT #\NEWLINE)) "WILDATOM XX C N O S P" 8 21) ?
12:49:42
drmeister
(funcall (esrap::compile-expression '(NOT #\NEWLINE)) "WILDATOM XX C N O S P" 8 21) --> #<FAILED-PARSE (NOT #\NEWLINE) @8>
12:50:20
stassats
what about (funcall (esrap::compile-expression '(NOT #\X)) "WILDATOM XX C N O S P" 8 21)?
12:50:36
drmeister
(describe (aref "WILDATOM XX C N O S P" 8)) --> #\ - standard character code: #x20
12:51:18
drmeister
(funcall (esrap::compile-expression '(NOT #\X)) "WILDATOM XX C N O S P" 8 21) --> #<FAILED-PARSE (NOT #\X) @8>
12:52:37
stassats
what about (funcall (esrap::compile-expression '#\Space) "WILDATOM XX C N O S P" 8 21)?
12:55:22
scymtym
ESRAP::EXEC-NEGATION has one use that is at the intersection of structure predicates, structure inheritance and SETF return values. maybe one of those or their interaction is the problem
12:55:33
drmeister
Trying - I need to keep restarting because of a problem with my READ fix that appears when I try to evaluate :r1 from the repl.
13:05:03
scymtym
drmeister: line 11 should show a true return value. the structure is an instance of FAILED-PARSE which includes ERROR-RESULT
13:07:14
stassats
drmeister: and (progn (defstruct foo) (defstruct (bar (:include foo))) (foo-p (make-bar)))
13:08:52
drmeister
(progn (defstruct foo) (defstruct (bar (:include foo))) (foo-p (make-bar))) --> NIL
13:18:11
drmeister
Getting the non-segfaulting subset of the ansi regression tests working in a CI build system has now bubbled to the top of my "todo" list - along with getting the demo together for next week.
13:22:48
Bike
as for why anything works - we don't actually use defstruct without :type, for stupid boot reasons
13:34:48
kpoeck
@bike, in the example with (progn (defstruct foo) (defstruct (bar (:include foo))) (foo-p (make-bar)))
13:39:19
kpoeck
If you redefine (defun make-predicate (name type named name-offset) to do (typep x name) instead of (si::structure-subtypep (type-of x) name)
13:40:13
kpoeck
t.m. (progn (defstruct foo) (defstruct (bar (:include foo))) (foo-p (make-bar))) -> T
13:43:04
Bike
really, the defstruct code does a lot of stuff i'd like to rewrite, but defstruct is annoying to implement and i don't want to just do it myself
13:44:19
Bike
we shouldn't be typing checks inside the accessor functions, for one, it should be an ftype
13:45:29
Bike
and probably the expansion should be rewritten in the first place so it'd not doing all this fset at load time shit
13:45:41
Bike
maybe i'll do that after i fix this and also do the other stuff i was doing to do today
13:48:13
Bike
actually relatedly, we have this (load-time-value (list nil)) song and dance instead of just throwing the class in there, in the constructor i mean
13:48:42
Bike
was there some reason doing it the obvious way didn't work? i guess if it was in there at load time it would look up the class before the defclass is evaluated
14:24:45
Bike
hm, i bet with fastgf, generic function dispatch is faster than how we do typep for classes now, which is subclassp
14:29:02
Bike
meaning it would make sense to instead reduce typep to a generic function call, of course
14:29:16
Bike
really wish we could have the initial state of the discriminator built into the fasl, though
14:39:19
Bike
beach: Several things... we'd need to make sure stamps are the same at compile and load time, for one
15:28:17
Bike
the other thing i broke is allocate-instance (and therefore make-instance) with a derivable-cxx-class
15:30:01
drmeister
Ok - I have a couple of hours now while it rebuilds everything including your new fix (thanks). I'm going to put some time into travis/AWS and regression tests.
16:00:20
attila_lendvai
drmeister: do you know hu.dwim.stefil? it's our test framework that is basically just augmented defun's that interact very nicely with the normal lisp workflow, including slime and restarts. I can spend some time with a proposal in a PR to set up a clasp test suite that uses stefil. but I don't want to get into it if someone objects before the start, or if there's already something else planned.
16:01:31
drmeister
attila_lendvai: Separate test frameworks are welcome. We are talking about using the ansi tests - but we would not distribute them as part of clasp.
16:03:10
attila_lendvai
I see that others are already working with the ansi tests. those could also be integrated into the clasp unit tests as the other dependencies, including a filter for tests that break clasp for the time being
16:03:13
drmeister
We've started and will continue to put clasp specific tests in clasp/src/lisp/regression-tests/
16:05:39
attila_lendvai
drmeister: it's a test framework. a lib that provides deftest, is, signals, etc... and a general framework to run tests non-/interactively, collect errors, etc
16:07:16
drmeister
What we use in clasp/src/lisp/regression-tests/ is bare bones - something I wrote that could run in aclasp.
16:07:53
drmeister
But we have been writing tests that cannot run in aclasp and we've probably (hopefully?) moved beyond the need for testing at that low level.
16:08:06
attila_lendvai
a deftest in stefil is basically a defun with some wrapper to ensure some dynamic environment to collect assertion data. assert macros are made so that they can run without any environment, then they just print stuff, so that you can flexibly copy-paste, etc
16:09:44
scymtym
i'm in the process of extracting SBCL's test framework into an ASDF system. it has a bunch of specialized things for testing COMPILE, running side-effecting tests in sub-processes, etc.
16:15:54
attila_lendvai
the reason why we wrote stefil is that all other test frameworks introduced a new concept called 'test' that had a very high impedance mismatch with the rest of the system. different namespace, not funcallable, etc. with stefil you can copy paste stuff into the repl, start (some-test new-arg, ...) from the repl just like a defun, call a test from another test, and debug all that as you debug everything else with a couple of e
16:16:47
attila_lendvai
yeah, testing frameworks are a semi-religious thing... :) I do my best to remain impartial, but we still ended up rolling our own after trying several others.
16:18:13
drmeister
Bike and I will take a look and ask questions and figure out how to move forward.
16:19:29
Bike
stassats had the right idea a while back when he said (to paraphrase a lot) that the particular framework is less important than having tests at all
16:19:50
attila_lendvai
drmeister: sure. whatever way you take, I advice you against rolling your own infrastructure, because whatever you are looking for is probably out there already in some of the frameworks
16:20:03
beach
I am getting convinced that the reason so many people roll their own, is that there might not be very many obvious neutral abstractions to be had in a testing framework.
16:20:46
Shinmera
attila_lendvai: Ironic because someone's just writing anodda one (their second one, even). https://www.reddit.com/r/Common_Lisp/comments/84u4ob/1yet_another_1_common_lisp_testing_library/
16:23:09
drmeister
Yeah - and I'm sensing some momentum here and I'm interested in keeping people engaged and moving forward - so I don't want to enter any religious wars. I just want a good testing framework that we can build tests within and catch problems early.
16:23:45
Shinmera
drmeister: If all you want is to test, beach's advice is good enough -- just write a bunch of asserts.
16:24:14
drmeister
I have this demo coming up - so I have to focus on that - but I'm working on the travis and building stuff while I sit and watch things build.
16:24:46
drmeister
Shinmera: Agreed - and that's basically what we have in clasp/src/lisp/regression-tests
16:25:05
drmeister
Bike and I started writing tests for things that we are fixing and putting them in there.
16:25:37
Shinmera
My excuse for writing Parachute was an architectural / system exercise. I wanted to see if I could come up with a model that is flexible enough to encompass all other testing frameworks. So far I've succeeded.
16:26:09
attila_lendvai
the main issue with ASSERT is that you don't have an easy way to skip a failed assert when debugging interactively. and you probably also want to isolate test assertions, i.e. stuff may continue, from real asserts like (assert (universe-exists)) inside the program being tested
16:28:05
beach
Those seem to be minor problems to me, compared to the problems I have trying to squeeze my testing technique into any particular framework.
16:36:46
drmeister
Kevslinger's undergraduate travel grant from Temple University got approved for ELS.
16:37:22
drmeister
Or whatever else kind of malted battery acid passes for soda in the south of Spain.
16:38:34
drmeister
It's a good job because it depends on Kevin's academic standing and the work he's done and the application that he submitted. It's a travel award.
16:45:51
attila_lendvai
drmeister: yep. the problem was that clasp builds for too long and travis times out
16:46:34
drmeister
Ok, I have everything needed to build on AWS - I've heard that I can have travis build on AWS
16:49:16
drmeister
attila_lendvai: I don't see any files in the clasp root directory having to do with travis - am I looking in the wrong place?
16:53:33
attila_lendvai
drmeister: https://github.com/drmeister/clasp/commit/2354e4df354873b4758504a10be2431eeb82b5c0
16:59:10
drmeister
attila_lendvai: So if I merge that into the 'dev' branch - travis would start building but fail because of resource limitations.
17:00:50
drmeister
Bitter Kas: quinine...tincture of pine needle... "Turpentine-like" or "pine resiney" - sounds delightful.
17:13:29
Bike
it's also only with :include - which is a weird case because then the size of the structure can't be known at compile time, which messes up a few things
17:16:16
drmeister
travis is $69/month for the bootstrap deal - I'm spending about that on AWS right now - but AWS feels more flexible.
17:23:19
Bike
i guess i'd have to have the macroexpander check for include, and then if there is an include but the include is not known as a structure, insert special code for load time
17:24:59
Bike
like this applicable method cache thing, which i figured would be surprisingly simple, and is therefore not
18:46:40
drmeister
I just purchased a travis-ci account - bootstrap level - so 1 concurrent build at a time.
18:47:15
drmeister
I associated it with the 'clasp-developers' group. We may need to migrate clasp and cando to clasp-developers.
19:58:18
Bike
the fact the initargs are slot specificactions rather than actual slot definitions is a little annoying
20:21:55
drmeister
Let's take a look at it in about 40 min - I'm stuck down here doing office hours that so many students ask me about but nobody shows up to.
20:22:17
drmeister
Everybody (but me) loves the idea of lots of office hours but doesn't take the effort to show up.
20:23:20
Bike
eighteen year olds can handle CL-20 or monkey malaria or whatever it is we have up here
20:24:18
drmeister
I think it's more that some of our computational biology colleagues really like the peace and quiet.
20:28:21
shiho
We don't need to add (:file "charges") in cando.asd because charges is not file but directory.
20:53:50
Bike
so now e.g. 54 different effective methods on initialize-instance use the same emf, instead of behaviorally identical emfs being compiled 54 times