freenode/#lisp - IRC Chatlog
Search
19:59:23
fiddlerwoaroof
phoe a set of higher order functions like this make data manipulation much nicer
20:00:54
fiddlerwoaroof
This would probably more efficient if implemented with something like the series library, because then it could do stream fusion transparently
20:03:09
phoe
I have an octet vector of dimensions (* 4 Y X). Can I turn it into a 3D array with dimensions (X Y 4) without copying?
20:04:39
fiddlerwoaroof
I've had this issue before, but I don't remember if I ever found a good solution.
20:05:24
fiddlerwoaroof
You might be able to create a displaced array with your desired shape that points to the original one
20:06:51
fiddlerwoaroof
Yeah, one pain point for me has always been the lack of a numpy-like array type
20:15:50
fiddlerwoaroof
From the standard it looks like adjusting a displaced array to be non-displaced copies the contents
20:18:48
rpg
phoe: What do you mean loop on for values at once? You mean like a region, of index i, i-1, i+1?
20:20:00
rpg
But you could probably do something like (loop :for x :across arr :using i :as y = (aref arr (1- i)) ....)
20:21:12
fiddlerwoaroof
You could also do something like (map 'vector fun (step vec :offset 0 :by 2) (step vec :offset 1 :by 2)) if you had a step function
20:23:32
prxq
phoe: if you start with a 1d vector, you can use a "front end" vector with dimension (x y 4) using the :displaced-to option of make-array
20:24:52
prxq
_death: if you want to use dimensions x, y, and z, multiplications will happen anyway.
20:25:54
_death
prxq: usually processing is sequential, say pixel-by-pixel or row-by-row, so you can avoid some
20:26:51
pjb
phoe: adjust-array may and will copy an non-adjustable array. Use make-array with :displaced-to
20:27:56
pjb
phoe: displaced arrays are bound to be faster to your own (svref v (+ (* (+ (* x row) y) 4) z))!
20:28:51
pjb
phoe: for vectors you can loop over multiple indices as you wish, since it's O(1) indexed access!
20:34:17
pjb
(let ((v (vector 1 2 3 4 5 6))) (loop with 2v = (make-array 2 :displaced-to v :displaced-index-offset 0) for i below (1- (length v)) do (setf 2v (adjust-array 2v 2 :displaced-to (array-displacement 2v) :displaced-index-offset i)) (prin1 2v) (princ " "))) #| #(1 2) #(2 3) #(3 4) #(4 5) #(5 6) --> nil |#
21:14:35
fiddlerwoaroof
I've been working on something that needs such a library and, so far, I've been using a javascript library to do this client-side... but that's not really ideal
21:14:44
Shinmera
Well, HTML5 is "self-sanitising", so any parser of that will do. Alternatively, Plump will just slurp anything you throw at it, and allow you to serialise a DOM, so.
21:14:45
pjb
fiddlerwoaroof: I wrote something a long time ago, around html 4.01. It's entirely outdated. I guess somebody would have to write something for current html and html bugs.
21:15:01
kora9
Evening fellow Lisperatis! I'm wondering, why in for example (reduce #'+ '(1 2 3)) is the + preceded by #? I kind of understand that the + is preceded by '
21:15:27
fiddlerwoaroof
Shinmera: ? I don't mean validation/cleanup, I'm talking about removing things that could cause XSS or break styling
21:16:43
pjb
kora9: (defun add (x y) `(this is toplevel add ,x ,y)) (let ((add-q 'add) (add-f #'add)) (flet ((add (x y) `(this is local add ,x ,y))) (list (funcall 'add 1 2) (funcall #'add 1 2) (funcall add-q 1 2) (funcall add-f 1 2)))) #| --> ((this is toplevel add 1 2) (this is local add 1 2) (this is toplevel add 1 2) (this is toplevel add 1 2)) |#
21:16:51
kora9
Oh, cool. That's helpful to know. I just checked and (reduce (function +) '(1 2 3)) has the same result :)
21:17:43
pjb
kora9: however, for symbols in the CL package, rules ensure that (eq (symbol-function ',x) (function ,x)) so '+ and #'+ will always call the same function, as long as the + here is CL:+
21:17:48
fiddlerwoaroof
This works pretty well client-side: https://github.com/cure53/DOMPurify but, I'm starting to think it would be better to clean the html when I pull the RSS feed
21:18:22
fiddlerwoaroof
Also, the DOMPurify library has unfortunate interactions with another frontend library I'm using...
21:18:26
Shinmera
fiddlerwoaroof: If you know which elements and tags to remove, using lQuery to do that job is pretty easy.
21:23:18
Shinmera
Eg: (let ((dom (plump:parse "<foo style=\"bla\" a=\"b\"><bar><script/></bar><style/></foo>"))) (lquery:$1 dom "script,style,link" (remove) dom "[style]" (attr :style NIL) dom (serialize)))
21:24:43
sjl
fiddlerwoaroof: to be clear, are you looking for something like bleach https://github.com/mozilla/bleach but for CL?
21:25:10
kora9
I've been playing around with cl-who with my hunchentoot tests and it's pretty cool to make html look nice in lisp. But I don't know if that's what you're after
21:25:59
fiddlerwoaroof
I need to make sure that article content doesn't contain anything that will mess up the ui or allow arbitrary code execution
21:26:41
sjl
then yeah, a whitelist-based sanitizer like bleach would be ideal. unsure if anyone's built something like it in CL
21:27:36
fiddlerwoaroof
I wonder if bleach plays nicer with Google's closure library: the current library I'm using (DOMPurify) seems to be broken by the google library
21:28:31
sjl
I was just using it as an example of an approach that's probably ideal for your use case
21:28:59
sjl
although I think the html-entities package got removed from quicklisp a while ago for some reason
21:30:07
anticrisis
pjb: so basically, do I understand correctly that 'add always refers to top-level, while #'add refers to whatever is lexically scoped as add? Hence why it's "safer" to always use #', because programmers expect lexical bindings to take precedence?
21:30:33
fiddlerwoaroof
I always use MIT or similar but there are libraries like cl-html5-parser that that rules out.
21:30:45
sjl
anticrisis: 'add is a symbol, and when you funcall a symbol it looks up the global function binding
21:32:38
sjl
anticrisis: #'add expands to (function add) which is a special form, and can account for things like flet and labels
21:33:17
sjl
generally #' does what you'd expect more often, though there are cases where you might want to use the symbol
21:33:35
anticrisis
ok, i think i'm getting it, makes perfect sense when you think of lexical scope versus global scope
21:34:26
sjl
now (funcall *foo*) will always call that particular version of my-function, even if my-function is redefined later
21:34:40
Shinmera
fiddlerwoaroof: lQuery/CLSS allow negative matches, so you can do a whitelist based filter like this: (let ((dom (plump:parse "<foo><bar><script/></bar><style/></foo>"))) (lquery:$1 dom ":not(foo)" (remove) dom (serialize)))
21:35:05
Shinmera
Though that needs a patch to CLSS that I just pushed. Apparently the :not pseudo-selector has been broken for quite a while. Whoops!
21:35:22
anticrisis
right - seems like you'd want to avoid binding a particular function to a global variable
21:35:22
fiddlerwoaroof
Also, I like #' because it visually distinguishes functions from ordinary symbols
21:36:05
sjl
see also: https://google.github.io/styleguide/lispguide.xml?showone=__FUN_vs.__FUN#__FUN_vs.__FUN
21:37:20
anticrisis
it's not a good reason, but i'm still coming to terms with my dislike of the character # -- draws my eye's attention too much
21:37:45
sjl
the global variable thing was just an example -- you might also want to store functions in non-global variables
21:38:29
sjl
here's an example where I specifically use ' instead of #' in a little roguelike game: https://github.com/sjl/rldt/blob/master/src/main.lisp#L424-L432
21:39:20
sjl
and if I update use-healing-potion while the game is running, I odn't need to recreate all the healing potions
21:40:01
anticrisis
right, that makes sense -- it seems to me more common to think of a symbol as dynamically naming a function
21:40:40
Shinmera
It's just a question of whether you pass a literal function object, or a name that resolves to one.
21:40:47
anticrisis
so in that case, if you had used #'use-healing-potion, redefining the use-healing-potion function would not take effect for all of its users, right?
21:41:05
sjl
correct. all existing healing potions would still have pointers to the old function object
21:41:54
Shinmera
fiddlerwoaroof: Regarding licenses: All of my libraries are compatible with MIT/BSD, so you're free to use them as you deem fit.
21:42:18
sjl
anticrisis: well it does still exist, because all the old healing potions would be pointing to it ;)
21:43:28
fiddlerwoaroof
If you're using slime or something, using a symbol rather than a function literal will mean that you don't have to recompile the reference to update the dependency
21:45:31
sjl
basically you want #' if you're using it immediately, and you *might* want a plain symbol if you're storing it and using it later
21:46:06
fiddlerwoaroof
On the other hand the plain symbol has all the disadvantages of dynamic variables...
21:48:26
pjb
anticrisis: however, using a symbol to denote a function also has some advantages: it's a later binding; if you redefine the function the symbol will refer to the new definition, while #'x will refer to the old function until it's re-compiled.
21:49:15
pjb
(this is also a reason to avoid anonymous functions in this kind of places (hooks), so that you may redefine the function at run-time without recompiling everything).
21:50:09
fiddlerwoaroof
For hooks, I generally store the function in a hash-table or some kind of datastructure
21:50:30
fiddlerwoaroof
So, as long as I remember the reference, it's generally fairly easy to replace a previously added hook
21:51:56
anticrisis
pjb: thank you, that's very clear. i take it as #' means "use thing as it's defined now" versus ' means "use this thing as it's defined when you invoke it." Like storing a value versus a reference.
21:52:39
pjb
cl:function is the only special operator that creates closures and anonymous functions.
21:53:04
pjb
cl:lambda is a macro that expand to (cl:function (cl:lambda …)) <- it's cl:function that creates the anonymous function!
21:53:50
pjb
such cl:lambda macro didn't exist in some language before CL, which is why some people write #'(lambda …) but it's useless in CL, where cl:lambda expands automatically to (cl:function (cl:lambda …))
21:53:52
gremdrus
how do you get the current directory? I want to save files to a relative directory
21:53:55
gremdrus
https://stackoverflow.com/questions/8409527/common-lisp-get-path-to-file is not helpful :(
21:55:06
pjb
Usually, *default-pathname-defaults* set to #P"" will make pathnames without directory go to the posix current working directory.
21:55:42
gremdrus
but there's nothing like pythons -> https://stackoverflow.com/questions/5137497/find-current-directory-and-files-directory?
21:58:57
Shinmera
If your usual MO is to compile or load the file, then (make-pathname :name NIL :type NIL :defaults #.(or *compile-file-pathname* *load-pathname* *default-pathname-defaults*))
21:59:24
Shinmera
Note that this will not give you what you want if you compile that snippet with, say, slime's C-c C-c.
22:00:14
pjb
gremdrus: you know, there's a reason all the programs take options such as -o output.file or use save panel to select a place where to save files…
22:01:12
pjb
gremdrus: alternatively you could use a default computed path, cf. the XDG standard, but you would do it in the (user-homedir-pathname), not relative to the sources!
22:01:37
anticrisis
it's likely he's thinking about a command line utility or something, which acts on files in the directory he's invoking it from -- just guessing here
22:02:26
pjb
Never touch the files in the current working directory. Always use specifically indicated input and output files. Use command line arguments! (or configuration files).
22:02:58
pjb
Then again, there's no question about the sources! Just use the current directory: (open "file.txt" :if-does-not-exist :create) and that's it!
22:03:34
sjl
yeah that's why I manually craft my git commit hashes and pass -o./git/objects/abc123 to every git command
22:04:39
pjb
sjl: again, git doesn't save the repo in a directory relative the the fucking SOURCES OF GIT!
22:05:17
pjb
Anyways, there's also a reason why some things are hard to do: because it's not a good idea to do them!
22:06:04
pjb
gremdrus: if you want to do ( cd output ; command-to-fill-this-directory ) then you can, and you don't have to look up the directory where your sources are stored.
22:06:14
zulu_inuoe_
Hey all.. I have a question for somebody that has a pretty deep understanding of CLOS.. Is shared-initialize supposed to be calling initialize-instance ? I am reading and re-reading the documentation for 'shared-initialize' but I keep coming up with the answer that it should be the other way around: initialize-instance->shared-initialize
22:07:01
pjb
anticrisis: you write CLI tools in CL just like in C or in shell. (well, easier than in C or shell).
22:07:58
sjl
anticrisis: it pretty much works, and the command line arg handling is okay. not as nice as python's argparse or getopt or whatever its name is (I forget) though
22:08:33
sjl
anticrisis: roswell has always been kind of flaky for me though, especially when trying to upgrade it, so these days I mostly just have a makefile that runs a really small script to dump an SBCL executable
22:08:46
phoe
"The generic function shared-initialize is called by the system-supplied primary method for initialize-instance, reinitialize-instance, update-instance-for-redefined-class, and update-instance-for-different-class."
22:09:21
zulu_inuoe_
phoe: Correct. So, shared-initialize -shouldn't- be calling initialize-instance, right?
22:10:11
zulu_inuoe_
Erm. I mean. I am calling shared-initialize and it's calling initialize-instance on me
22:10:16
pjb
And we should override shared-initialize rather than initialize-instance in general, but writing a shared-initialize is more complex, since you would have to take into account the cases of reinitialize-instance and update-instance-*
22:10:16
sjl
anticrisis: but tbh I still use roswell for commandliney scripts, mostly because I haven't gotten around to rewriting them
22:11:11
anticrisis
sjl: at the very least, roswell was the only way i could install sbcl/ccl on windows
22:11:46
sjl
anticrisis: ah, yeah, I haven't used windows in years. I imagine if Roswell makes those easy there, that's a pretty big plus
22:12:09
zulu_inuoe_
phoe: That was a bad idea on my non-optimized game that is constantly spawning event objects :P
22:14:01
zulu_inuoe_
Whenever I write a CLI script I always use *load-truename*. At least for me it's always worked
22:15:32
pjb
zulu_inuoe_: you're doing two errors: 1- using (assuming) sbcl and 2- assuming load of the source.
22:16:49
zulu_inuoe_
pjb: I don't see it as an error if it's well-defined for my scripts to be run that way
22:24:47
anticrisis
say i want to implement something like jekyll: i need to loop through files in ./_posts and output them to ./_site. don't i need access to the current directory somehow? because i want users to be able to type 'jekyll serve' and be done. i feel like i'm not understanding the point you're making.
22:33:45
pjb
(mapcar (lambda (path) (list path (make-pathnames :directory '(:relative "_site") :defaults path))) (directory "_posts/*.*"))
22:35:27
sjl
2017-08-13 18:02:26 <pjb> Never touch the files in the current working directory. Always use specifically indicated input and output files.
22:36:49
pjb
sjl: yes, there was multiple points. The most important is about the directory of the sources. But it's also not a good practice to use the current working directory. It'd be better to take an option to tell the program what directory to process.
22:39:46
pjb
Also, in bash scripts, having to cd is always a problem. Either you have to do it in a subshell, or you have to deal with going back to the original working directory (cd - may work, but it's not always what you want or can do), and in all case you have to check for errors (when the directory doesn't exist or is not accessible, and it is particularly painful to do in bash (error handling), so it's rarely done, and so you have so m
22:39:46
pjb
bugs in scripts… Which again shows that it's just not a good idea to process the current directory.
22:41:12
sjl
Ideally you allow both, e.g. ls defaulting to printing the current directory but allowing args, or git defaulting to ./.git but honoring --git-dir
22:43:30
sjl
not providing the current directory as a default even when it would often be useful can be annoying, e.g. always having to use 'find .'
22:43:54
whoman
constructive or destructive operations should be targetted explicitly, noncreative and nondestructive could be implicit
22:45:41
anticrisis
directory is quite useful. I just did this: (mapcar 'delete-file (directory "./*.fasl"))
22:45:58
pjb
(which is nice when you're in your repo, but when you're lost (ie. in scripts), it can be a problem.
22:50:26
pjb
You could prehaps replace the push with a :collect new-image :into result and use :initial-images (nconc result (list (first images))) in the make-data-stream
22:55:25
pjb
As for the (skippy:… image) you can write a macrolet (macrolet ((ref (name) `(,(intern (string name) "SKIPPY") image))) (ref top-position) (ref left-position) …)
22:57:10
pjb
or ∴top-position ∴left-position, you have a lot of esthetic and symbolic choice with unicode characters.
22:57:10
fiddlerwoaroof
pjb: as far as error handling in bash goes, a good rule of thumb is to always start a script with set -o pipefail -eu
22:57:54
anticrisis
ran across this amazing use of loop last night: https://github.com/fare/asdf/blob/1b0a0fd945ade1f341f3b2d3c0596cc4aa5244f3/uiop/package.lisp#L678
22:58:38
Bike
it would be way more convenient if stuff like 'append' was a local macro instead of syntax.
22:59:19
fiddlerwoaroof
I was doing a algorithms course on Coursera, and I found that LOOP's syntax basically let me translate the pseudocode into something executable nearly word for word
22:59:32
pjb
fiddlerwoaroof: -u makes empty arrays signal an error. and -e doesn't apply everywhere anyways. So an erroneous cd in a lot of positions won't break the script.
23:00:27
fiddlerwoaroof
-u is annoying, I grant, especially when you're doing stuff like checking for possibly non-existent command line arguments
23:00:52
pjb
fiddlerwoaroof: and the worst of all #!/bin/bash -eu -o pipefail is useless when people run the script (which is named foo.sh!) with bash foo.sh !
23:01:17
pjb
fiddlerwoaroof: so you need both: #!/bin/bash -eu -o pipefail and set -eu -o pipefail # yes.
23:01:19
sjl
Bike: yeah, being able to define new iteration constructs as macros is the best part of iterate
23:01:37
fiddlerwoaroof
set -eu -o pipefail applies in both cases, so you don't need the options in the shebang line
23:02:05
pjb
fiddlerwoaroof: so it's just yet another turing tarpit. You can have fun and get paid for it, but it's just a silly time waste.
23:02:30
zulu_inuoe_
phoe: You remember how I was complaining about initialize-instance getting called? This is what my genious self was calling.. (shared-initialize (initialize-instance obj) slots-needing-init)
23:03:45
fiddlerwoaroof
anticrisis: here's the merge part of merge sort all in loop: http://paste.lisp.org/+7KP1
23:04:14
fiddlerwoaroof
I think this is conforming lisp code, but the part of the standard that specifies loop is a bit difficult to understand
23:09:36
sjl
I'm pretty sure that's technically non-conforming, but will work in most implementations
23:16:15
fiddlerwoaroof
Yeah, the confusing part is that it says "Termination-test control constructs can be used anywhere within the loop body. The termination tests are used in the order in which they appear. If an until or while clause causes termination, any clauses that precede it in the source are still evaluated. If the until and while constructs cause termination, control is passed to the loop epilogue, where any finally
23:16:42
fiddlerwoaroof
But, it makes sense if you pay attention: for .. in clauses are part of the "loop prologue"
23:18:16
sjl
a (loop ... while ... for) doesn't even parse according to the grammar defined in the spec, so everything else after that would seem irrelevant
0:14:48
yegortimoshenko
pjb: only uiop/filesystem:resolve-symlinks, unfortunately. what does one use to create a symlink, osicat?
0:16:55
yegortimoshenko
they do have make-link, it's just some stuff is absent from osicat and some stuff is absent from uiop...
2:22:40
whoman
"ros use sbcl/system" as recommended, still downloads a new copy of sbcl and quicklisp, which i do not need or want
2:51:21
fiddlerwoaroof
From what I gather, the whole point of ros is to allow it to take care of such things.
2:51:51
fiddlerwoaroof
It works a bit like the various sandbox development environments for other langages (virtualenv, stack, etc.)
4:49:54
anticrisis
whoman: you've tried adding a symlink to your implementation's installation directory in ~/.roswell/impls/...?
6:08:51
muzik
i changed to emacs from vi about a year ago, and thought the lisp channel might be interesting enough for me to follow :)