freenode/#lisp - IRC Chatlog
Search
20:25:41
makomo
hm, say there's a function F within file A and a macro M which uses F within file B. further, usages of the macro M appear within file C. assuming no EVAL-WHENs, in order to compile file C it has to hold that (1) we compiled file B and (2) we compiled and loaded file A. is this correct?
20:28:15
makomo
if the above is correct, does that mean we lose the possibility of compiling A, B and C in isolation (since now we need to actually *load* A to be able to compile C)?
20:30:52
makomo
is that dependency a bad thing? should one then wrap every DEFUN that's used at compile-time into an EVAL-WHEN? what about external system though? i guess we depend on them being loaded anyway so it's fine?
20:31:49
makomo
... external systems (which we don't control and can't change and wrap their DEFUNs)* ...
20:32:51
makomo
i'm mostly wondering whether the fact that we have such "load-before-compilation" dependencies is bad or not
20:33:15
makomo
jackdaniel: do we have to load B though? won't the macro be defined when B is compiled?
20:34:10
shka_
makomo: what jackdaniel's is saying i think is that you will need your function definition ONLY when you need it (during macroexpand)
20:34:51
makomo
jackdaniel: well i peeked at the expansion of SBCL's DEFMACRO and it is defined at compile-time
20:34:57
makomo
also, "If a defmacro form appears as a top level form, the compiler must store the macro definition at compile time, so that occurrences of the macro later on in the file can be expanded correctly."
20:37:22
elderK
Hey guys, quick question: When does it make sense to use a symbol's plist vs. an external hash table?
20:37:56
elderK
In my situation, I have a hash table that stores information about types and things. The symbol denoting hte type's name is the key of the hash table.
20:38:17
elderK
I was wondering if attaching type information, a structure instance, as a property on the types I define instead, would be better.
20:38:39
makomo
jackdaniel: alright, so only A's left. so is this "loaded-before-compilation" dependency a bad thing?
20:39:38
shka_
secondly, it they would be just symbols, defgeneric and defmethod with eql specializer would be probably a better option
20:41:19
makomo
jackdaniel: so assuming :serial t, not only does ASDF load them in that order, but it also compiles them in such a way that before compiling the file N, files 1, ..., N-1 have been loaded?
20:42:21
elderK
shka_: I'll have to come back later and describe it in more detail. As it is, I have a hash table mapping symbols to type metadata.
20:42:35
jackdaniel
makomo: that depends on what you tell asdf to do, it may only load them, or only compile them. my impression is that asdf sometimes does a random thing, but I didn't find any passage with (funcall (nth *funs*) (random 8)), so maybe I do not understand whole thing ,)
20:42:48
elderK
shka_: It's not the symbols I care about, it's the associated type information I store relating to them.
20:43:04
jackdaniel
but more seriously: asdf constructs a plan which is more complicated than simple load/compile
20:44:08
elderK
shka_: That isn't sufficient here :P The type metadata contains information I need :P
20:44:11
jackdaniel
but in principle it works like: if A depends on B, load B (or its compilation result) before A
20:45:19
makomo
jackdaniel: lol :D. right, it depends on what i want to do. whether ASDF will compile and then load the fasls if i tell it to LOAD-SYSTEM is a separate thing, so let's assume i'm doing COMPILE-SYSTEM
20:45:23
jackdaniel
makomo: given you are groking style issues, did you make some time to read about splicing?
20:46:37
elderK
shka_: I'm afraid I have to run. Please take a look at the linked binary-types thing to get a feel for well, what I'm talking about :D
20:47:02
jackdaniel
I don't know ASDF enough to describe without looking at source code what it does exactly when you compile-system
20:47:22
makomo
jackdaniel: not yet. sorry if it seems like i'm intentionally avoiding it since i'm asking all these questions, but i'm procrastinating and should be studying. however, asking these questions is different as i know that if i start analyzing that document, i'll waste too much time. :-)
20:47:28
jackdaniel
but I would suspect, that if there is dependency between two files, dependee is loaded before depending file is compiled
20:47:57
makomo
jackdaniel: my first exam is tomorrow and the whole week is packed with them. i'll get to it on the next monday the latest!
20:49:24
jackdaniel
I think you overestimate time needed to read it, but I understand that exams are imminent
20:50:19
makomo
jackdaniel: oh, i did read it (quickly though), but actually writing an anylsis will take more time (i.e. (re)searching the standard and the like)
20:51:47
jackdaniel
if you create binary on ecl on the other hand some dependencies which were available during compilation are not part of the resulting binary, and sometimes such binaries fail (that happens for instance if dependency on uiop is not expressed in asd file)
20:55:51
makomo
jackdaniel: so basically there's a separate compilation and run-time environment. wouldn't the same happen if you restarted SBCL and tried to load a FASL which uses uiop (before actually loading uiop)?
21:08:51
jackdaniel
drawing benchmarking application sketch: http://hellsgate.pl/files/c7111345-bench.gif
21:10:15
pjb
minion: memo for elderK: notice that p-lists are faster than hash-table when they have fewer than 5-35 elements, depending on the implementation. Also, hash-tables have a size overhead that is way bigger than p-list. So if you need to put just one or two entries on thousands or millions of symbols, better use the symbol-plist. On the other ahnd, if you need to put thousands of entries on a few symbols, then better hash-tables.
21:12:18
pjb
minion: memo for elderK: In both cases, if you attach attributes to symbols at compilation time, you must ensure that data is passed over to run-time if you need it at run-time. basically: (defmacro x (s) (setf (getf (symbol-plist s) 'your-attribute) 'your-value) `(progn (setf (getf (symbol-plist ',s) 'your-attribute) 'your-value) … ))
21:23:48
aeth
pjb: Using plists for an unspecified small number of elements just because they're slightly faster is a micro-optimization that this channel normally criticizes.
21:25:59
aeth
I did this: (defparameter *foo* (list :foo 42 :bar 43 :baz 44)) (defparameter *bar* (zr:hash-table* :foo 42 :bar 43 :baz 44)) (time (dotimes (i 1000000) (getf *foo* :baz))) (time (dotimes (i 1000000) (gethash :baz *bar*)))
21:27:02
aeth
I keep adding zeros and sometimes the hash table is slower (up to 4x in one case) but they're usually about the same.
21:29:26
aeth
So 25% speed improvement in niche cases for something that either way takes about as much time as +
21:35:10
aeth
SBCL doesn't appear to, but either way, I allocated one tiny object for each and was just accessing it.
21:36:07
aeth
The memory argument is a bit mixed. Conses will probably take up the least memory of a data structure for things like this, but they might not be together in memory, which is probably more important in today's software than being tiny overall.
21:36:48
aeth
fiddlerwoaroof: alist will add a bit more overhead, I think. At least, if you don't test that your plist is a proper one.
21:37:22
pjb
aeth: notice that it also depends a lot on the type of keys, and the hash function used by the hash-table.
21:40:04
aeth
plist is [key|-][value|-][key|-][value|...] and alist is [pointer-to-key-and-value-cons|-][pointer-to-key-and-value-cons|-][pointer-to-key-and-value-cons|...]
21:40:59
pjb
(print-conses '((a . 1) (b . 2))) #| ((a . 1 ) . ((b . 2 ) . ())) --> ((a . 1) (b . 2)) |#
21:44:36
aeth
Also, plists will iterate by cddr and then check car and alists will iterate by cdr and then check caar, so that's another difference.
22:06:00
aeth
pjb: Yes. I'm guessing the typical operations are (1) iterate over the entire alist/plist or (2) iterating over the alist/plist until you find a key that matches
22:07:02
aeth
pjb: so cddr iteration with car [and cadr], or cdr iteration with caar [and cdar] where the parts in brackets are only needed if either you're in case #1 or you matched in case #2
22:07:58
aeth
(Of course you'd use named accessors and I'd probably use destructuring-bind, rather than actually call those cxr functions)
22:11:25
fiddlerwoaroof
I've also thought that you could dynamically optimize an alist or plist by always prepending the last entry you looked up
22:27:34
aeth
Remember, the hash table is only 25% slower and now you're slowing down the lookup of the alist/plist.
22:57:00
aeth
Here's a fun little program to see the day's time from a different perspective. (defun countdown () (multiple-value-bind (second minute hour) (get-decoded-time) (format t "~2,'0D:~2,'0D:~2,'0D" (- 23 hour) (- 59 minute) (- 59 second))))
23:06:21
fiddlerwoaroof
Think about it this way: there's a certain number N of elements where the O(n) lookup of an alist becomes slower than the O(1) lookup of a hash-table. If you have a dataset where < N elements constitute the majority of accesses, then an alist that begins with those elements will generally be faster than a hash-table.
23:07:01
White_Flame
I think the only speedup would be if you read the head multiple times in succession. Even if you're reading just the 2nd entry of the list, you still have to shuffle or cons to shift a new head
23:08:00
fiddlerwoaroof
Well, my guess is the best way to do it would be to have a "training" period where you move accessed elements to the head and then freeze the list
23:08:40
fiddlerwoaroof
You'd probably also avoid reshuffling the list if the access is to one of the first N elements
23:19:38
pjb
Ok, in this lisp image, there are 169 symbol-value that are hash-tables. Of those, 133 have only 1 entry.
23:28:43
aeth
pjb: when you're talking about comparing two operations that are approximately as fast as +, it's still a micro-optimization
23:34:47
pjb
Here are the types of the first key in those 196 hashtables: (character (simple-base-string 45) standard-generic-function ccl:compiled-lexical-closure function (simple-base-string 2) (simple-base-string 5) standard-class (simple-base-string 11) (simple-base-string 6) hash-table pathname ccl:process cons (simple-base-string 17) (simple-base-string 22) bit readtable (simple-base-string 21) (simple-base-string 13) (simple-base-string
23:34:47
pjb
keyword nil (integer 0 1152921504606846975) symbol (simple-base-string 4) (simple-base-string 29))
23:55:46
scymtym
dim: best to ask in #clim. what i found is that i have to (setf mcclim-truetype::*truetype-font-path* #P"/usr/share/fonts/truetype/**/") after loading mcclim, but before doing anything with it. then i can use truetype fonts via (clim:make-text-style "EB Garamond Initials" "Regular" 24) where the "Regular" face must exist in the font
23:56:25
scymtym
when the *TRUETYPE-FONT-PATH* is properly set, the "Font Selector" demo should show all fonts and their respective faces
1:46:44
pfdietz
My typical use case for hash tables is to maintain a (temporary) map from objects to values. For example, if the objects are nodes in a graph, the map might represent mark bits used during a depth first search of the graph.
1:47:25
pfdietz
But I'd like to be able to write that algorithm more generally. For example, if the nodes have a mark field I'd like to be able to use that.
1:48:38
pfdietz
It should be possible to have a construct that uses such fields (or plists, or something else) to handle these maps, and fall back to hash tables as a general case.
1:49:36
aeth
99 times out of 100, when I use a plist/alist (and it's usually a plist) I'm making something that's supposed to be iterated over.
1:50:27
aeth
So alexandria:doplist (ugh, there's that missing hyphen problem so many people have in their naming conventions!), not random access.
1:50:59
|3b|
ACTION likes to just mix all 3, accumulate into a hash (to combine duplicate keys), then convert to alist to sort, then convert to plist for easier printing :p
1:51:31
aeth
If I'm taking in input, I prefer plists, but I'll probably turn that into something more suitable for internal use.
1:52:45
aeth
pfdietz: You can get it to fail with destructuring-bind instead of directly using loop's destructuring.
1:53:55
aeth
pfdietz: (destructuring-bind (key value &rest rest) sublist (declare (ignore rest)) ...)
1:56:19
aeth
pfdietz: I'll loop by #'cddr if I need to use a feature of loop, such as append or collect. I'll use alexandria:doplist instead of a simple loop by #'cddr that uses a :do (in part because it's shorter, especially with the fail-on-invalid-plist requirement)
1:57:17
pfdietz
What would be nice there would be something like cddr that fails if applied to non-conses.
2:00:23
aeth
What I want is something that is literally just loop but with s-expressions (or at least can be thought of in that way... obviously you'd want it to be a superset)
2:03:18
aeth
pfdietz: i.e. I want this as my iteration: (do-loop (:for i :of-type fixnum :from 0 :below 10) (:for j :of-type double-float := 1d0 :then (* 2d0 j)) (:for s :in foo) (:do ...) (:collect ...) ... (:finally ...))
2:04:23
aeth
*Possibly* there could be another layer of parentheses (so the for/with/etc. could be grouped in one (...))
2:06:27
aeth
pfdietz: Well a direct translation of LOOP would permit things like (:when ... (:collect ...))
2:07:38
aeth
(And, in fact, one use of this would be for implementing a LOOP* that is just LOOP with extensions and guaranteed portable behavior, so no more surprises with CLISP behaving differently than SBCL and CCL and ECL.)
2:09:49
aeth
Oh, and it wouldn't *quite* be a perfect translation. The best way to handle using s-expressions would be to make :of-type explicit in DO-LOOP all of the time instead of having certain types having :of-type be optional (portably fixnum, float, t, and nil, but everyone except CLISP seems to accept subtypes of float, which are far more useful than "float")
2:11:39
pfdietz
I mention COVER because I'm working on that right now. I want to get it to work with ITERATE, but COVER plays a bit dangerously with replacing some symbols in forms, and it breaks ITERATE when doing so.
2:12:46
pfdietz
There's an interesting general issue here. If I have two packages, and I need to add an extension that only makes sense when both are loaded (and is then necessary), where do I put that extension?
2:13:04
aeth
33 matches for 'defvar', which is exactly what I don't want in my iteration internals, personally. https://gitlab.common-lisp.net/iterate/iterate/blob/a1c47b2b74f6c96149d717be90c07a1b273ced69/iterate.lisp
2:13:23
pfdietz
I always use DEFPARAMETER, but perhaps you're objecting to special variables of any kind.
2:16:59
aeth
But this is the main usability problem I have with iterate: https://gitlab.common-lisp.net/iterate/iterate/blob/a1c47b2b74f6c96149d717be90c07a1b273ced69/package.lisp#L11-22
2:17:18
aeth
With LOOP, I just need LOOP. If you defined LOOP* that was a slightly modified LOOP you'd just need to export LOOP*
2:18:59
aeth
My style is to manually import symbols, rather than use prefixes or use a package. Iterate's symbol export style is... well, it's not pleasant when mixed with my package import style.
2:20:05
pfdietz
And it can cause symbol collision problems, if someone else is using those particular symbol names.
5:10:20
fiddlerwoaroof
Also, especially when I'm reading pseudocode for an algorithm, I use loop forms as executable pseudocode
5:13:45
fiddlerwoaroof
e.g. http://paste.lisp.org/display/353413#1 (although, honestly, I like the look of sjl's iterate version better)
6:10:37
aeth
fiddlerwoaroof: I disagree. Emacs highlights keywords. Emacs doesn't and shouldn't highlight symbols. So LOOP with keywords is highlighted a lot like a language with actual keywords would be highlighted. I rely on syntax highlighting.
6:11:01
rk[ghost]
so let's assume i have a file that is a list, something like '("this" "that" "and" "more")
6:11:02
aeth
fiddlerwoaroof: (loop :for x :in y ...) well if I typed "for x in y" in most languages that would highlight "for" and "in"
6:11:37
rk[ghost]
now, i want to load this file, but really, i just want to get the return value from the file of the list itself..
6:12:10
rk[ghost]
is there something equivalent to (load) that just returns the value back from executing the file?
6:17:13
rk[ghost]
i don't need to evaluate it, ima store it so i can map something across it to build another object..
6:18:12
fiddlerwoaroof
aeth: yeah, I don't really care about the syntax highlighting, but I find that the colons really annoy me when I'm trying to read the loop
6:20:37
beach
rk[ghost]: Sure. I don't expect many people to understand it. All I can hope is that the referees think it is worthwhile.
6:21:42
rk[ghost]
need computers to fix computers, need food to power foraging for food, new shelter to have space to build shelters
6:33:43
rk[ghost]
i suppose i am confused on how to use (read) in this case.. as how to cast a file to a stream? (read #P"/my/file.lisp") isn't correct XD
6:55:01
no-defun-allowed
yes, with-open-file as beach pointed out is much more convenient than just open since it handles closing the file after the body is run
6:57:04
rk[ghost]
i had the strange occurence of the terminal being absurdly unresponsive.. gee that was frustrating..