freenode/#lisp - IRC Chatlog
Search
11:09:31
rk[ghost]
err, if i have a list like '("hello" "my" "name" "is" "rk") and i want to apply a function with such as an argument that returns "hellomynameisrk" .. is that a fold ?
11:11:10
pjb
(if (< (1+ (length list)) call-arguments-limit) (apply (function concatenate) 'string list) (reduce (lambda (a b) (concatenate 'string a b)) list))
11:11:36
pjb
You can do: (if (< (1+ (length list)) call-arguments-limit) (apply (function concatenate) 'string list) (with-output-to-string (*standard-output*) (dolist (item list) (write-string item))))
11:11:51
shka_
rk[ghost]: if you want to do something like this, i recommend using either string stream or simply using format
11:12:01
pjb
since list is often small and call-arguments-limit nowadays is often large, apply concatenate will often work.
11:14:07
rk[ghost]
it just seems like a textbook use for a higher order function since i waht to "str" + "in" + "g" + "zzz" across a list..
11:14:09
shka_
so although i am not a fan of format, i think that in this case it is a good tool to use
11:14:49
_death
(defun string-append (&rest strings) (apply #'concatenate 'string strings)) now (apply #'string-append list) if list is short
11:14:57
rk[ghost]
and since i mapped across the list to make it, thought reducing it back to single string seemed sensible..
11:15:02
pjb
well, instead of reduce, if you have a long list, you can do (make-array (reduce (function +) list :key (function length)) :element-type 'character) and then fill it with the strings in list.
11:16:30
rk[ghost]
then i did a (car list) and everything worked right.. then i realized, doh.. i need to just jam the list in to one big string :P
11:17:05
rk[ghost]
the list is a list of html strings.. and i was trying to (write-to-string) the whole list.. which borked all the links XD
11:27:16
rk[ghost]
thanks for all the help everyone:) really appreciate _both_ answering my question (teaching me how to properly use reduce) and giving me a more elegant solution with reasonings
11:52:00
puchacz
hi, is it possible in Slime REPL to check what functions or generic functions defined in my-package are used anywhere in my-package please?
12:02:01
beach
Here is a new version of our bootstrapping paper http://metamodular.com/bootstrapping.pdf taking into account valuable remarks by splittist. Also we added more material in order to make the entire procedure a bit more clear. We still have two months to the deadline, so there might be more changes, of course. This has been a morning of hard works, so I am about to take a break. I'll check the logs in case someone has further
12:04:48
shka_
beach: back in the ELS 2017 you had this talk on portable efficient sequence functions
12:05:43
shka_
do you happen to have some tips on reducing memory usage during compilation when doing this?
12:06:26
beach
shka_: You might want to talk to heisig about it. He has agreed to work in the implementation.
12:08:46
heisig
shka_: I am running benchmarks at the moment to figure out the best strategy for SICL sequence functions.
12:09:54
heisig
You can reduce the memory consumption A LOT if you reduce the number of entries in *special-array-information*.
12:29:05
adlai
beach: STYLE-WARNING: definition 4.1 ("simple instance") is not referred to by the rest of the document.
13:23:22
|3b|
looks like either something wrong with your quicklisp, or something wrong with ql infrastructure fetching github releases, since you seem to have iolib 0.8.1, and 0.8.2 claims to fix problems with asdf 3.3 (and newest is 0.8.3, over a year old)
13:24:33
|3b|
or something odd with your asdf confifguration, does ~/quicklisp/dists/quicklisp/software/iolib-v0.8.3/ (or .2) exist?
13:27:48
|3b|
looks like fetching things by github release seems to be working, so probably not ql server-side, i'd guess overly broad asdf search path
13:29:55
pjb
Unfortunately, for ecl, it looks like I have to do that, since quicklisp prevents it to save executables…
13:30:52
|3b|
doesn't ql have some way of dumping a list of systems for building executables without ql?
13:32:32
jackdaniel
(my life would be certainly happier, if ASDF had similar funcionality: dump compilation instructions to file so it may be loaded without ASDF in the image to build a binary)
13:55:23
beach
Wow, that's vicious. Together with ASLR, pretty soon they are going to prevent us from writing decent Common Lisp implementations.
13:57:31
pfdietz
OpenBSD support in SBCL is already decayed; this will just mean the platform gets written off entirely.
13:58:57
shka_
what about julia? or any other dynamic language built on top of llvm for that matter?
14:01:55
beach
Note to self: In the CLOSOS specification, add W^X as an example (together with ASLR) of a kludge on top of a hack in order to make a broken model (i.e. UNIX) a bit safer, but also much harder to use.
14:03:44
pfdietz
What's the overhead for changing a page? Also, if you are compiling in small chunks, will you waste a lot of space in pages that need to be executable before they are filled?
14:03:57
|3b|
beach: yeah, possibly the status quo will change if openbsd users care enough to patch implementations when it is
14:04:04
ggole
It's iOS that is problematic here, as it doesn't allow user programs to change map permissions.
14:05:19
ggole
If you compile in small chunks you could either use a new page or do make writable -> write the code -> make executable
14:07:31
jackdaniel
ECL won't be affected by this either, because it either works with bytecodes or with modules load from disk (with dlopen), so there is no code segment write whatsoever
14:08:58
pfdietz
For people writing CL, I suggest they do not depend on things defined in foo.lisp in order to compile things later in foo.lisp. Stick your macros and class definitions in another file.
14:10:21
ggole
There's another cute trick that might be useful: double map the pages, with one mapping writable and the other executable
14:11:44
beach
It sounds to me like W^X is so easy to circumvent that it won't fix the problem it is intended to solve, but it will make life harder for compiler writers.
14:14:59
ggole
The reasoning, I suppose, is that the solution - ditching C - would not be acceptable. Thus they do what they can.
14:41:07
jcowan
beach: gcc generates trampolines in certain conditions, so I don't think you have to worry about being completely locked down. Furthermore, a strict interpretation of W^X would make JIT impossible, and Java depends on it.
15:04:48
jackdaniel
so here's how they implemented JIT in Firefox: https://jandemooij.nl/blog/2015/12/29/wx-jit-code-enabled-in-firefox/ . It is not that you can't write to the same pages which you execute, but rather permissions are either write or execute
15:04:52
MichaelRaskin
Double-mapping will probably not work; re-protecting memory as as RW -> RO -> RX seems to be the official recommendation and very unlikely not to be supportred. The problem they mitigate is overwriting already-executable page by buffer overflow.
15:05:39
jackdaniel
but in this scenario it is an explicit action taken by the environment (not accidental thing)
15:05:51
MichaelRaskin
Note that strict W^X without run-time protection level chage will break userspace dynamic loader
15:17:31
White_Flame
jackdaniel: "temporarily mark page RW" might be impossible in the future. allocate->RW->RO->RX->deallocate might be a one-way path
15:26:20
White_Flame
well, any time you want to update a JIT compilation, you'd have to allocate a new page with its own permissions
15:27:29
White_Flame
oh, interfere patching, correct. It'd have to effectively be manual copy-on-write
17:29:22
sjl
no-defun-allowed: jackdaniel: cl-6502's 25-line JIT is also worth reading as inspiration
18:48:59
shka_
it is modular, neatly designed, i never figured out how to write my own modules (but i didn't have to)
18:50:00
p_l
I recently did stuff in rails 5, and some of the stuff is still some of the better tooling I have ever used :O
18:50:50
p_l
But I do prefer lisp, so I'm thinking of looking what's the current landscape and either using something existing or using some lower layer stuff
18:52:47
p_l
Which is very minimal, good-for-microservice kind of thing (Sinatra and thus Caveman, that is)
20:25:41
makomo
hm, say there's a function F within file A and a macro M which uses F within file B. further, usages of the macro M appear within file C. assuming no EVAL-WHENs, in order to compile file C it has to hold that (1) we compiled file B and (2) we compiled and loaded file A. is this correct?
20:28:15
makomo
if the above is correct, does that mean we lose the possibility of compiling A, B and C in isolation (since now we need to actually *load* A to be able to compile C)?
20:30:52
makomo
is that dependency a bad thing? should one then wrap every DEFUN that's used at compile-time into an EVAL-WHEN? what about external system though? i guess we depend on them being loaded anyway so it's fine?
20:31:49
makomo
... external systems (which we don't control and can't change and wrap their DEFUNs)* ...
20:32:51
makomo
i'm mostly wondering whether the fact that we have such "load-before-compilation" dependencies is bad or not
20:33:15
makomo
jackdaniel: do we have to load B though? won't the macro be defined when B is compiled?
20:34:10
shka_
makomo: what jackdaniel's is saying i think is that you will need your function definition ONLY when you need it (during macroexpand)
20:34:51
makomo
jackdaniel: well i peeked at the expansion of SBCL's DEFMACRO and it is defined at compile-time
20:34:57
makomo
also, "If a defmacro form appears as a top level form, the compiler must store the macro definition at compile time, so that occurrences of the macro later on in the file can be expanded correctly."
20:37:22
elderK
Hey guys, quick question: When does it make sense to use a symbol's plist vs. an external hash table?
20:37:56
elderK
In my situation, I have a hash table that stores information about types and things. The symbol denoting hte type's name is the key of the hash table.
20:38:17
elderK
I was wondering if attaching type information, a structure instance, as a property on the types I define instead, would be better.
20:38:39
makomo
jackdaniel: alright, so only A's left. so is this "loaded-before-compilation" dependency a bad thing?
20:39:38
shka_
secondly, it they would be just symbols, defgeneric and defmethod with eql specializer would be probably a better option
20:41:19
makomo
jackdaniel: so assuming :serial t, not only does ASDF load them in that order, but it also compiles them in such a way that before compiling the file N, files 1, ..., N-1 have been loaded?
20:42:21
elderK
shka_: I'll have to come back later and describe it in more detail. As it is, I have a hash table mapping symbols to type metadata.
20:42:35
jackdaniel
makomo: that depends on what you tell asdf to do, it may only load them, or only compile them. my impression is that asdf sometimes does a random thing, but I didn't find any passage with (funcall (nth *funs*) (random 8)), so maybe I do not understand whole thing ,)
20:42:48
elderK
shka_: It's not the symbols I care about, it's the associated type information I store relating to them.
20:43:04
jackdaniel
but more seriously: asdf constructs a plan which is more complicated than simple load/compile
20:44:08
elderK
shka_: That isn't sufficient here :P The type metadata contains information I need :P
20:44:11
jackdaniel
but in principle it works like: if A depends on B, load B (or its compilation result) before A
20:45:19
makomo
jackdaniel: lol :D. right, it depends on what i want to do. whether ASDF will compile and then load the fasls if i tell it to LOAD-SYSTEM is a separate thing, so let's assume i'm doing COMPILE-SYSTEM
20:45:23
jackdaniel
makomo: given you are groking style issues, did you make some time to read about splicing?
20:46:37
elderK
shka_: I'm afraid I have to run. Please take a look at the linked binary-types thing to get a feel for well, what I'm talking about :D
20:47:02
jackdaniel
I don't know ASDF enough to describe without looking at source code what it does exactly when you compile-system
20:47:22
makomo
jackdaniel: not yet. sorry if it seems like i'm intentionally avoiding it since i'm asking all these questions, but i'm procrastinating and should be studying. however, asking these questions is different as i know that if i start analyzing that document, i'll waste too much time. :-)
20:47:28
jackdaniel
but I would suspect, that if there is dependency between two files, dependee is loaded before depending file is compiled
20:47:57
makomo
jackdaniel: my first exam is tomorrow and the whole week is packed with them. i'll get to it on the next monday the latest!
20:49:24
jackdaniel
I think you overestimate time needed to read it, but I understand that exams are imminent
20:50:19
makomo
jackdaniel: oh, i did read it (quickly though), but actually writing an anylsis will take more time (i.e. (re)searching the standard and the like)
20:51:47
jackdaniel
if you create binary on ecl on the other hand some dependencies which were available during compilation are not part of the resulting binary, and sometimes such binaries fail (that happens for instance if dependency on uiop is not expressed in asd file)
20:55:51
makomo
jackdaniel: so basically there's a separate compilation and run-time environment. wouldn't the same happen if you restarted SBCL and tried to load a FASL which uses uiop (before actually loading uiop)?
21:08:51
jackdaniel
drawing benchmarking application sketch: http://hellsgate.pl/files/c7111345-bench.gif
21:10:15
pjb
minion: memo for elderK: notice that p-lists are faster than hash-table when they have fewer than 5-35 elements, depending on the implementation. Also, hash-tables have a size overhead that is way bigger than p-list. So if you need to put just one or two entries on thousands or millions of symbols, better use the symbol-plist. On the other ahnd, if you need to put thousands of entries on a few symbols, then better hash-tables.
21:12:18
pjb
minion: memo for elderK: In both cases, if you attach attributes to symbols at compilation time, you must ensure that data is passed over to run-time if you need it at run-time. basically: (defmacro x (s) (setf (getf (symbol-plist s) 'your-attribute) 'your-value) `(progn (setf (getf (symbol-plist ',s) 'your-attribute) 'your-value) … ))
21:23:48
aeth
pjb: Using plists for an unspecified small number of elements just because they're slightly faster is a micro-optimization that this channel normally criticizes.
21:25:59
aeth
I did this: (defparameter *foo* (list :foo 42 :bar 43 :baz 44)) (defparameter *bar* (zr:hash-table* :foo 42 :bar 43 :baz 44)) (time (dotimes (i 1000000) (getf *foo* :baz))) (time (dotimes (i 1000000) (gethash :baz *bar*)))
21:27:02
aeth
I keep adding zeros and sometimes the hash table is slower (up to 4x in one case) but they're usually about the same.
21:29:26
aeth
So 25% speed improvement in niche cases for something that either way takes about as much time as +
21:35:10
aeth
SBCL doesn't appear to, but either way, I allocated one tiny object for each and was just accessing it.
21:36:07
aeth
The memory argument is a bit mixed. Conses will probably take up the least memory of a data structure for things like this, but they might not be together in memory, which is probably more important in today's software than being tiny overall.
21:36:48
aeth
fiddlerwoaroof: alist will add a bit more overhead, I think. At least, if you don't test that your plist is a proper one.
21:37:22
pjb
aeth: notice that it also depends a lot on the type of keys, and the hash function used by the hash-table.
21:40:04
aeth
plist is [key|-][value|-][key|-][value|...] and alist is [pointer-to-key-and-value-cons|-][pointer-to-key-and-value-cons|-][pointer-to-key-and-value-cons|...]
21:40:59
pjb
(print-conses '((a . 1) (b . 2))) #| ((a . 1 ) . ((b . 2 ) . ())) --> ((a . 1) (b . 2)) |#