freenode/#lisp - IRC Chatlog
Search
7:12:21
phoe
beach: Have you done any work on a spellchecker? I remembering you having such a project once.
7:16:17
beach
Also, if it includes the word categories possible for each one, that would help the future grammar checker.
7:17:45
phoe
I'll try munching on https://github.com/robert-strandh/Spell/blob/master/Lisp-Wordlist/spell.lisp in a spare while today.
7:36:52
phoe
beach: it's not really a Lisp question, but, we have a tree (or even, a trie) of words like the ones that you have constructed. That way, we can look up words that are correct.
7:37:32
phoe
Now, based on that, we need an algorithm that will look up suggestion for words that were not found. For example, we have "across" in the example dictionary that you created, and we'd like it to turn up as a suggestion if we type "acros".
7:39:30
phoe
I also assume that you want to be automatically able to parse the english-* files in your repository and create entries based on each line inside them.
8:17:45
beach
shka_: Faster lookup won't be necessary. Most of the time, the exact match will be found, and when it is not, it is more important to get good advice that to get good performance.
8:18:30
xificurC
given this cl-i.asd file: http://ix.io/1lDG I can (asdf:load-system :cl-i) and get T but (asdf:test-sytem :cl-i) just returns T without actually running the tests. Is there some glaring error in the definitions of the systems?
8:19:18
beach
shka_: But we don't have that, so wee need to think about simplicity and maintainability.
8:19:38
xificurC
for the record I cannot (asdf:load-system :cl-i-tests) this way. I'm not sure what is the idiomatic way to organize a library and its tests
8:21:13
Shinmera
Here's a working example for tests https://github.com/Shinmera/3d-vectors/blob/master/3d-vectors-test.asd https://github.com/Shinmera/3d-vectors/blob/master/3d-vectors.asd
8:22:17
beach
xificurC: ASDF finds the system based on the name of the file. So if the name of the file is not cl-i-tests.asd then it won't find your system.
8:22:38
jackdaniel
beach: in fact it will, because in order to find cl-i (which is tested) it must load whole asd
8:25:31
jackdaniel
xificurC: if you define it in the same file, way recommended by asdf authors is to name it cl-i/tests
8:26:32
xificurC
I was copying this - https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/best_practices.md#trivial-testing-definition
8:27:41
jackdaniel
magic name conventions affecting library behavior are something what makes my day ;)
8:33:07
scymtym
i guess the SYSTEM/SUB-NAME naming requirement for defining multiple systems in the same file was chosen because SYSTEM/SUB-NAME.asd would not be a valid filename
8:34:06
phoe
beach: just a minor finding, you defclass NODE but don't use it as a superclass of anything later on.
8:34:13
jackdaniel
I'm aware of that, still it is just one step from making comments affecting inner workings of your library
8:36:01
xificurC
I have no idea why this isn't working. Before I had 2 asd files but I was the best practices page do it in 1 and thought it might be cleaner
8:36:26
xificurC
so now I'm in the stage where someone might say it's cleaner, but it's not working :)
8:39:10
phoe
beach: I assume that I should, for now, focus on a way of parsing the english-* files, correct?
8:41:28
scymtym
xificurC: your example seems to work here. it returns t as you observe, but i see "Success: 0 tests, 0 checks." being written to standard output
8:42:24
beach
phoe: If you store a single character per node, it may be a bit more expensive than necessary, but that might not be a problem.
8:43:49
xificurC
I'm pushing a path to asdf:*central-registry* to get my asd found, can there be a difference in what asdf does with a system found this way? Before asking, yes, I have to.
8:45:12
beach
phoe: But many prefixes will be shared, and you don't have to store every basic form with every derived form.
8:47:49
phoe
Also, one more thing - I don't even have to store the final forms of the words in the leaf nodes. The tree structure is enough.
8:58:38
phoe
like, english-1 contains words in some kind of dictionary notation where english-16 contains them in quasi-Lisp notation
9:11:27
phoe
Invalid initialization argument: :STRENGH in call for class #<STANDARD-CLASS SPELL::VERB>
9:11:55
phoe
Should I just &allow-other-keys to ignore the unknown keywords? Or should I patch the classes to include the new arguments?
9:12:34
phoe
english-16 contains lines like "expostulated" :type :verb :base "expostulate" :tense :past :strength :weak
9:18:18
Xof
London placenames with six consecutive consonants? Countries (three of them) which share no letters with "mackerel"?
9:30:02
phoe
"'un" :base "him" :type :pronoun :person :third :number :singular acc :gender :masculine
9:35:38
scymtym
C-u M-x slime then edit the command accordingly, that is something like sbcl --dynamic-space-size 4000
9:41:21
phoe
beach: "yourselves" :base "yourselves" :type :pronoun :person :second :number :plural refl
9:41:43
phoe
REFL means reflexive, I remember as much - I just don't need how to classify it with a keyword
9:57:36
no-defun-allowed
so now there is around a 1000 frame difference between my workers and ffmpeg
10:04:47
phoe
beach: woop, I found a stray NIL - "an" :base "an" :type :article :definite nil :number :singular
10:12:28
phoe
Once we have parsed the whole dictionary (I'm at 42% now), we can start thinking of refactoring the keywords and such.
10:26:30
phoe
Hm. I'm looking at ref1pl. I think I know how to translate ref3pl based on that. (I hope.)
10:30:59
no-defun-allowed
A SB-KERNEL::HEAP-EXHAUSTED-ERROR condition without bindings for heap statistics. (If you did not expect to see this message, please report it.
10:31:19
phoe
...if I understand it correctly, it means that "TO" is special, as in, it's a part of some kind of infinitive verb. "to write"
10:32:30
no-defun-allowed
i can imagine how my stupidity caused that so it's probably not very interesting
10:39:27
no-defun-allowed
cause my genius solution to coordinating writes to one stream from twelve workers is through a counter and hashtable
10:40:18
no-defun-allowed
if the writer signals a condition, handler-case hopefully sets a kill flag and the threads return early
10:40:41
no-defun-allowed
since the writer is slow, workers wait if the difference between writer frame and workers frame is greater than 500
10:42:52
no-defun-allowed
(the moral is: don't bother fuzz testing, just give me a keyboard and repl and i'll break shit somehow)
10:45:15
phoe
beach: there is only one word in the original english-1 dictionary that is PASSIVE and it seems that your VERB class does not have a slot for holding that information. Should I add a slot for that single word or should I convey this information in some other way?
10:46:26
no-defun-allowed
i really want to try getting opencl support in cl-vep cause it'd be much faster i think but i don't have any opencl gpus
10:48:24
phoe
beach: ....isn't "born" actually a perfect participle or something? Or is "passive verb" a thing in English?
11:22:48
beach
phoe: And no, "born" in that sense is not the perfect participle of any verb in active form.
11:37:51
trocado
hi all! commonqt/qtools question: is it possible to have qt *not* blocking the repl?
11:38:52
trocado
it would be nice to be able to create a window with a simple gui to inspect some things, but keep working on the repl
11:39:27
Shinmera
Qt needs to be run from the same thread always. As long as that's ensured it's fine.
11:39:55
trocado
I tried making a new thread for it but then it complains that it's not running on the main thread
11:40:24
phoe
you can't make and run the QApplication in thread A and then execute stuff in thread B
11:41:27
shka_
trocado: it is the best to build event loop for qt to run and simply post everything there
11:44:27
trocado
yes, i'm thinking this over and maybe it's ok if I have to close the window to get back to the repl
11:47:13
trocado
another thing: is it possible to create a window in the foreground (with focus)? it always goes immediately under emacs...
11:54:50
TMA
phoe: born is indeed a perfect participle of bear, as is borne; the two are used for different senses of bear -- an oddity every bear can bear (and has borne since time immemorial) but no bear has born it
12:09:18
phoe
The only thing I'll want to do is dumping the final tree structure into the FASL, so it is not regenerated on each load.
12:26:39
phoe
Like, if you get pointed to "horse", it's not obvious if it points to the verb "horse" or to the noun "horse".
12:28:23
phoe
Right now only LOOKUP is exported, which I also modified to point at the English dictionary by default.
12:33:43
phoe
I'll eventually want to ship it with some application and that's when I'll be looking into making this stuff more compact.
12:37:46
beach
phoe: One important question to answer is this: Is it OK to return different (in the sense of EQ) instances for two different queries for the same word?
12:38:03
shka_
well, i would make all nodes contain the same content: bitmask representing what children are present, another mask to represent what words are present, and vector of children
12:38:15
beach
phoe: If so, then one could represent the leaf information in a much more compact way.
12:41:40
beach
One important thing to measure is the total amount of space taken up by the leaves compared to the grand total size. It is entirely possible that the tree structure itself is a small part of the total size.
12:42:25
beach
If that is the case, then the only way to decrease the space is to encode the words differently, possibly abandoning the EQ property.
12:43:00
beach
I invoke Ousterhout's article again. Measure before working to hard on the wrong part.
12:43:58
beach
phoe: The point is that if interior nodes are a small part of the total amount of space, then optimizations like the one shka_ suggests won't buy much.
12:48:13
francogrex
in this post https://pastebin.com/0UL9APLq anyone have a clue why i am getting the error Attempt to call an undefined alien function? I am using sbcl 1.4.2 for x86 32 bit. could that be the problem, the 32 bit v of sbcl?
13:18:26
phoe
We don't have to store the strings for bases - instead, we can point to the proper leaf node.
13:18:53
phoe
For example, if we have a word that has a base "horse", we can point to the leaf node for the final letter #\e.
13:20:06
phoe
We can't really travel upwards from the leaves, so if we point to the last letter, then we still have no idea what the base is.
13:20:24
phoe
We might know there's a verb and a noun there, but we have no means of figuring out the string for the base, "horse".
13:22:45
jackdaniel
I wonder if it is a bit too much to paste an "in motion" low-level application analysis on general #lisp channel
13:24:25
jackdaniel
I'm not in position to give or take something, just a suggestion that this may be too cluttering for people who bother reading backlog
13:27:28
splittist
I was enjoying that. But I don't read logs, so the live experience means more to me than extracting information later. Back to your regular discussions about why scheme, clojure and emacs questions are off-topic.
13:55:39
phoe
this way, if you have eight possibilities, you go from 64 bits (a full pointer) to 3 bits in a bit vector
14:21:44
makomo
just my 2 cents: i enjoy reading discussions like those as well. i know why the "off-topic rule" exists, but sometimes an exception should be made imo, especially if the channel is quiet anyway
14:24:56
jackdaniel
my point was not that it was offtopic but rather that it is to narrowed and makes reading other messages harder. imagine everyone talking about nitty-gritty details of projects they work on. but if this kind of moderation is not welcome, I'll just shut up and work on my own things
14:34:17
AeroNotix
jackdaniel: my thinking is that if, during the discussion of minutiae of some topic that someone enters and asks something more related to the real topic of the channel, the in-flight discussion should drop/pause and focus on the more topic related discussion
14:35:20
AeroNotix
I missed the initial part of the discussion. It seemed like an interesting discussion on how to optimize storage/lookup of some kind of trie/tree datastructure related to phoe's queries last night on dictionaries.
14:36:28
AeroNotix
Anyway, I have a question that I think I already know the answer to: is defvar the ONLY way to allow dynamic binding?
14:41:17
makomo
AeroNotix: both defvar and defparameter unconditionally proclaim the given symbol (the variable's name) as special and then maybe do some binding/assigning of their own
14:41:46
makomo
AeroNotix: i'd suggest reading this discussion if you want more a couple more details https://groups.google.com/forum/#!topic/comp.lang.lisp/iLo1ti6rQIw
14:43:18
makomo
AeroNotix: one thing to be aware of is the definition of the word "variable", which in CL is different from the definition in other languages
14:43:35
makomo
in CL "variable" means "the association between a name and a value", while in other languages it means "an abstract memory location"
14:44:22
AeroNotix
I was more asking about the methods possible to achieve dynamic binding. I misunderstood the differences between defvar and defparameter
14:45:10
phoe
Their only difference is, DEFVAR does not reinitialize an already initialized variable
14:45:36
AeroNotix
Yeah for some reason I thought defvar was the form to set up a dynamic variable. I see now that I was wrong
14:49:36
phoe
as long as they do not poke the special variable named X, the value will be transmitted all the way to the BAR call
14:52:51
beach
AeroNotix: It is not a "local special variable". It is just a declaration that tells the compiler to treat references to that variable in that scope as references to THE special variable.
14:55:03
makomo
beach: regarding the value cells we talked about a few days ago, how do you represent the unbound state? using an internal symbol as a marker or something?
14:57:39
makomo
suppose i wasn't interested in LOAD-DICTIONARY and LOOKUP after compilation -- could i drop :LOAD-TOPLEVEL and :EXECUTE then? https://github.com/phoe-trash/Spell/blob/master/english.lisp
14:59:18
makomo
but i would still need :LOAD-TOPLEVEL and :EXECUTE for the *english-dictionary* variable itself, right?
14:59:37
makomo
and you need :COMPILE-TOPLEVEL because the dictionary is constructed within the compiler's environment, by way of the eval read macro?
15:09:33
russellw
The construct for implementation-specific code e.g. #+SBCL, what is that called so I can look up the documentation?
15:13:45
phoe
so it doesn't matter if the symbol pushed into *features* is :foo, bar:foo, baz::foo or #:foo
15:14:36
jackdaniel
that does not mean, that if you look for phoe:foo it will look in keyword package
15:15:13
jackdaniel
if you type #+flat-namespace , it will look in features for keyword:flat-namespace
15:15:26
jackdaniel
if you type #+cffi:flat-namespace, it will look in features for cffi:flat-namespace
15:16:29
jdz
Right, the correct wording is: «While reading the test, the current package is the KEYWORD package.»
15:50:43
oni-on-ion
i also notice, that when slime is running, emacs will highlight the code i've changed. i've been disabling that as it gets a bit laggy the more of these highlighted regions there are, but i've just read in this article here that one can send all of those edited lines into slime. anyone know how off hand?
15:52:41
trittweiler
Once upon a time, I was thinking of writing a small TAILCALL that would expand to the right declarations depending on the implementation (and otherwise cause compilation to fail.) Not sure if (locally (declare ..magic..) (funcall ...)) is enough on all implementation; an implementation may need optimization settings at the start of the body of the function.
15:53:09
trittweiler
oni-on-ion, it's the slime-highlighting-edits contrib. Just remove that from your (slime-setup '(...)) list
15:54:11
trittweiler
it would be nice to see if it can be sped up or where the bottleneck is, if you have the time anyway.
15:55:01
oni-on-ion
trittweiler: i will start using it now, because i find it useful if even there is no way to "eval all highlighted edits into slime connection"
15:57:00
oni-on-ion
ah, i guess it is official, just need to do C-c C-c : https://common-lisp.net/project/slime/doc/html/Highlight-Edits.html
15:57:48
oni-on-ion
i just read the whole elisp code for it, nothing seems obviously computationally demanding, i can only guess that multiple background faces make emacs slow on my system. will keep mind open on that
16:01:55
AeroNotix
I'm not sure I understand why TCO was left up to the implementation. Relying on TCO makes your code non-portable. At the time of standardization was TCO rare/difficult/something else?
16:03:04
oni-on-ion
would it affect some code to rely on TCO optimization? would that really change semantics etc?
16:03:26
oni-on-ion
that is also what i am thinking and just why i asked, AeroNotix , maybe TCO was undeveloped at the time
16:07:46
oni-on-ion
heh, from a-road-to-common-lisp: "You don’t have to track down the bug from just a stack trace, like a detective trying to piece together what happened by the blood stains on the wall. You can examine the crime as it’s happening and intervene to save the victim."
16:57:37
jasom
oni-on-ion: also, dynamic bindings can make things that look like tail-calls not actually tail calls
17:04:20
oni-on-ion
im not smart enough to imagine any but i wont make the assumption that such code is possible/exists/isbenefitial
17:05:16
oni-on-ion
yes. well im trying to figure out if TCO affects the actual execution/result of code with and without it
17:05:24
AeroNotix
The link I provided before suggests that most major/non-meme CL implementations have TCO. I wonder how SBCL deals with a (funcall dynamically-bound-function) in the tail call position
17:06:08
Bike
(defun factorial (n &optional (acc 1)) (if (zerop n) acc (factorial (1- n) (* n acc))))
17:06:49
oni-on-ion
ahhh. forgot about stack things. just read that the stack frame is gone before the call rather than after, when TCO is happening
17:07:55
Bike
if you're just talking about whether implementations CAN do TCO, there's nothing preventing it, and like aeronotix said it's pretty common
17:10:39
AeroNotix
Bike: the issue for me is that, yeah, CL has other looping constructs but if an implementation/most implementations implement TCO then what's the point if it's not a portable construct?
17:12:52
Bike
what i mean is if you treat it as an optimization it's not necessary for it to be portable
17:13:11
|3b|
well, arrays larger than 2^24 elements aren't portable either, but they are pretty useful too :)
17:13:49
AeroNotix
Bike: eh, I can agree with that but it still would make me uncomfortable about using it.
17:19:13
|3b|
ACTION has hit the "array indices are fixnums" a few times though, even on sbcl with relatively large fixnums, much less clisp with 24bit fixnums :)
17:21:25
|3b|
though i probably wouldn't expect much of anything to work on an implementation with A-T-S-L = 1024 :)
17:22:14
shka_
but at least my pipe runs 165% of normal speed and now i can use it whenever i want it with just one line of code so that's nice
17:25:38
pjb
AeroNotix: yes. Or you can write: (if (< (length x) call-arguments-limit) (apply #'+ x) (reduce #'+ x))
17:26:38
pjb
apply may sort the arguments according to type or magnitude, so it may give more precise results than reduce.
17:28:04
pjb
(reduce (lambda (a b)(+ a b)) '()) #| ERROR: Too few arguments in call to #<Anonymous Function #x302006ADAE4F>: 0 arguments provided, at least 2 required. |#
17:32:51
|3b|
if order matters, i'd probably rather have it consistent than break when you add 1 more item to list :)
17:34:33
pjb
(values (+ 1e0 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6) (+ 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e-6 1e0)) #| --> 1.0000095 ; 1.00001 |#
17:35:12
|3b|
also, some implementations set call-arguments-limit to values larger than you'd want to apply due to stack size limits anyway
17:36:30
pjb
(defun f (&rest args) args) (let ((list (list 1 2 3))) (eq (apply #'f list) list)) could return T.
17:37:31
pjb
3.4.1.3 The value of a rest parameter is permitted, but not required, to share structure with the last argument to apply.
17:57:32
oni-on-ion
Bike: i've just seen the library SERIAL, which led me to folio2 - im surprised i havent come across it before today!
18:15:57
pjb
And thankfully, file-position returns nil when it cannot seek; but it still can signal an error when the position is too big or inappropriate.
18:16:12
pjb
The problem here is that we don't know if (1+ pos) or (1- pos) are appropriate positions.
18:20:12
jasom
pjb: file n. a named entry in a file system, having an implementation-defined nature. <-- I disagree that a posix named fifo is not a file
18:20:17
pjb
That said, you're right that file-position only specifies a STREAM, so it could work on other type of streams, and it could be possible to find file-streams that are not seekable for soem reason.
18:21:25
pjb
Notably: (with-open-file (stream "/tmp/foo") (pathname stream)) could return #P"/tmp/foo" for fifos.
18:22:09
jasom
though I'm not sure what is appropriate for (open a-stream-that-is-not-a-file-stream)
18:24:06
pjb
And there are people who would like to write a new CL standard, when 22 years later, we still haven't debugged all the implementations!
18:25:09
aeth
pjb: A new standard would only help portability. Right now, there are SBCL-only libraries that don't need to be.
18:26:24
dlowe
I think a new standard would mostly be about removing things and making them more regular to make implementation simpler
18:29:13
aeth
dlowe: The not very controversial things I'd like to see are (1) lower the minimum requirements for short-float so IEEE half-precision floating-point can be used (short float was originally meant for unboxed float in 32-bit implementations) and (2) raise the minimum requirements for most other things, which seemed to have been written with 16-bit in mind (e.g. the 1024 minimum array size, which includes string length)