freenode/#lisp - IRC Chatlog
Search
6:46:31
Necktwi
finally got rid of those ghost windows they are due to # to ## redirects of some of my autojoin channels
6:58:50
elderK
aeth: That's the thing: The macro is using functions defined in the same file, and is not encountering any issues on either SBCL or CCL.
7:00:24
elderK
Still seems weird. The macro makes several function calls, these functions generate the expansion. These helper functions call other functions, to get information that has been accumulated in a hash table. It seems bizarre that I am not encountering problems.
7:57:10
beach
elderK: One thing that can bite you is that you have already loaded your system before, so the functions are defined. Then, when you recompile, no warnings will be issued. To check that you have no such problem, it is good to compile from a fresh instance of your Common Lisp implementation.
7:58:15
aeth
That's basically the only time my build can fail, if I was referring to stale functions (i.e. an old version pre-rename), or if I have a circular dependency, etc., that the image won't expose unless it's fresh
7:58:35
beach
elderK: It is unfortunate that one has to restart the system to do that, of course. In SICL, I plan to use a fresh first-class global environment for this purpose without having to restart the system. Current Common Lisp implementation have a single global environment, so they can't do that.
8:07:59
beach
Sure, if someone makes a list of functions and put the list in a special variable, it will be around.
8:09:12
ggole
Perhaps I'm misunderstanding and the new environment won't include anything which can reference old bindings in that way.
8:09:23
beach
All I am saying here is that I can start the compilation from a fresh first-class global environment. This new environment will not contain any old definitions, so it will be obvious if a macro is referring to a function that does not exist at compile time. The compilation will generate a FASL as usual and that FASL can be loaded into the working environment (as opposed to a fresh one).
8:10:16
beach
Exactly. The new environment can be a pristine one, like when the Common Lisp system starts up.
8:13:42
aeth
at the moment you have to M-x slime-restart-inferior-lisp before pushing to git if you want to be polite and catch various stale environment bugs
8:14:44
beach
ggole: Sure. I keep thinking in terms of a Lisp operating system. Starting a fresh Lisp is to me like booting up a second version of your operating system. If you have things like editor buffers and such defined, they will not be present in the fresh system.
8:14:53
aeth
ggole: If you had first class environments, then *building* in a fresh environment will catch anything that you'd catch with a fresh Lisp image starting up, without having to kill the old running things
8:15:40
aeth
ggole: So you could run your IRC client and develop in the same Lisp image, and not have to bring your IRC client down when you want to test a fresh build/load to see if you had stale function bugs
8:17:03
aeth
ggole: of course the way to do it right now is to just run one Lisp per application and you know use like 100-1GB more RAM than you need to
8:17:31
beach
MichaelRaskin: Again, I am sorry for being dense. I am notorious for not understanding what other people refer to. I guess I am unable to fill in the blanks.
8:17:35
MichaelRaskin
I thought that creating a new global environment would probably put them into this environment in a more efficient way than starting a new image
8:18:15
MichaelRaskin
Rest assured that I _know_ you ask in good faith, and I understand the value of making sure about the intentions.
8:48:44
no-defun-allowed
has anyone ever used cl's runtime compilation to write a JIT compiling emulator?
8:53:11
no-defun-allowed
gonna write a chip8 one for the hell of it, but i think at the very least i can (ab)use (compile nil _) to generate thunks for each instruction
8:55:29
no-defun-allowed
also, how can i tell when to jit? should i do it when the machine jumps into new code or wait a bit?
8:58:13
ggole
You could do either - usually a fair amount of code is cold, so it makes sense to avoid compiling it
9:07:10
jackdaniel
no-defun-allowed: sjl wrote chip8 emulator with a fantastic blog post series explaning how it is made
11:09:31
rk[ghost]
err, if i have a list like '("hello" "my" "name" "is" "rk") and i want to apply a function with such as an argument that returns "hellomynameisrk" .. is that a fold ?
11:11:10
pjb
(if (< (1+ (length list)) call-arguments-limit) (apply (function concatenate) 'string list) (reduce (lambda (a b) (concatenate 'string a b)) list))
11:11:36
pjb
You can do: (if (< (1+ (length list)) call-arguments-limit) (apply (function concatenate) 'string list) (with-output-to-string (*standard-output*) (dolist (item list) (write-string item))))
11:11:51
shka_
rk[ghost]: if you want to do something like this, i recommend using either string stream or simply using format
11:12:01
pjb
since list is often small and call-arguments-limit nowadays is often large, apply concatenate will often work.
11:14:07
rk[ghost]
it just seems like a textbook use for a higher order function since i waht to "str" + "in" + "g" + "zzz" across a list..
11:14:09
shka_
so although i am not a fan of format, i think that in this case it is a good tool to use
11:14:49
_death
(defun string-append (&rest strings) (apply #'concatenate 'string strings)) now (apply #'string-append list) if list is short
11:14:57
rk[ghost]
and since i mapped across the list to make it, thought reducing it back to single string seemed sensible..
11:15:02
pjb
well, instead of reduce, if you have a long list, you can do (make-array (reduce (function +) list :key (function length)) :element-type 'character) and then fill it with the strings in list.
11:16:30
rk[ghost]
then i did a (car list) and everything worked right.. then i realized, doh.. i need to just jam the list in to one big string :P
11:17:05
rk[ghost]
the list is a list of html strings.. and i was trying to (write-to-string) the whole list.. which borked all the links XD
11:27:16
rk[ghost]
thanks for all the help everyone:) really appreciate _both_ answering my question (teaching me how to properly use reduce) and giving me a more elegant solution with reasonings
11:52:00
puchacz
hi, is it possible in Slime REPL to check what functions or generic functions defined in my-package are used anywhere in my-package please?
12:02:01
beach
Here is a new version of our bootstrapping paper http://metamodular.com/bootstrapping.pdf taking into account valuable remarks by splittist. Also we added more material in order to make the entire procedure a bit more clear. We still have two months to the deadline, so there might be more changes, of course. This has been a morning of hard works, so I am about to take a break. I'll check the logs in case someone has further
12:04:48
shka_
beach: back in the ELS 2017 you had this talk on portable efficient sequence functions
12:05:43
shka_
do you happen to have some tips on reducing memory usage during compilation when doing this?
12:06:26
beach
shka_: You might want to talk to heisig about it. He has agreed to work in the implementation.
12:08:46
heisig
shka_: I am running benchmarks at the moment to figure out the best strategy for SICL sequence functions.
12:09:54
heisig
You can reduce the memory consumption A LOT if you reduce the number of entries in *special-array-information*.
12:29:05
adlai
beach: STYLE-WARNING: definition 4.1 ("simple instance") is not referred to by the rest of the document.
13:23:22
|3b|
looks like either something wrong with your quicklisp, or something wrong with ql infrastructure fetching github releases, since you seem to have iolib 0.8.1, and 0.8.2 claims to fix problems with asdf 3.3 (and newest is 0.8.3, over a year old)
13:24:33
|3b|
or something odd with your asdf confifguration, does ~/quicklisp/dists/quicklisp/software/iolib-v0.8.3/ (or .2) exist?
13:27:48
|3b|
looks like fetching things by github release seems to be working, so probably not ql server-side, i'd guess overly broad asdf search path
13:29:55
pjb
Unfortunately, for ecl, it looks like I have to do that, since quicklisp prevents it to save executables…
13:30:52
|3b|
doesn't ql have some way of dumping a list of systems for building executables without ql?
13:32:32
jackdaniel
(my life would be certainly happier, if ASDF had similar funcionality: dump compilation instructions to file so it may be loaded without ASDF in the image to build a binary)
13:55:23
beach
Wow, that's vicious. Together with ASLR, pretty soon they are going to prevent us from writing decent Common Lisp implementations.
13:57:31
pfdietz
OpenBSD support in SBCL is already decayed; this will just mean the platform gets written off entirely.
13:58:57
shka_
what about julia? or any other dynamic language built on top of llvm for that matter?
14:01:55
beach
Note to self: In the CLOSOS specification, add W^X as an example (together with ASLR) of a kludge on top of a hack in order to make a broken model (i.e. UNIX) a bit safer, but also much harder to use.
14:03:44
pfdietz
What's the overhead for changing a page? Also, if you are compiling in small chunks, will you waste a lot of space in pages that need to be executable before they are filled?
14:03:57
|3b|
beach: yeah, possibly the status quo will change if openbsd users care enough to patch implementations when it is
14:04:04
ggole
It's iOS that is problematic here, as it doesn't allow user programs to change map permissions.
14:05:19
ggole
If you compile in small chunks you could either use a new page or do make writable -> write the code -> make executable
14:07:31
jackdaniel
ECL won't be affected by this either, because it either works with bytecodes or with modules load from disk (with dlopen), so there is no code segment write whatsoever
14:08:58
pfdietz
For people writing CL, I suggest they do not depend on things defined in foo.lisp in order to compile things later in foo.lisp. Stick your macros and class definitions in another file.
14:10:21
ggole
There's another cute trick that might be useful: double map the pages, with one mapping writable and the other executable
14:11:44
beach
It sounds to me like W^X is so easy to circumvent that it won't fix the problem it is intended to solve, but it will make life harder for compiler writers.
14:14:59
ggole
The reasoning, I suppose, is that the solution - ditching C - would not be acceptable. Thus they do what they can.
14:41:07
jcowan
beach: gcc generates trampolines in certain conditions, so I don't think you have to worry about being completely locked down. Furthermore, a strict interpretation of W^X would make JIT impossible, and Java depends on it.
15:04:48
jackdaniel
so here's how they implemented JIT in Firefox: https://jandemooij.nl/blog/2015/12/29/wx-jit-code-enabled-in-firefox/ . It is not that you can't write to the same pages which you execute, but rather permissions are either write or execute
15:04:52
MichaelRaskin
Double-mapping will probably not work; re-protecting memory as as RW -> RO -> RX seems to be the official recommendation and very unlikely not to be supportred. The problem they mitigate is overwriting already-executable page by buffer overflow.
15:05:39
jackdaniel
but in this scenario it is an explicit action taken by the environment (not accidental thing)
15:05:51
MichaelRaskin
Note that strict W^X without run-time protection level chage will break userspace dynamic loader
15:17:31
White_Flame
jackdaniel: "temporarily mark page RW" might be impossible in the future. allocate->RW->RO->RX->deallocate might be a one-way path
15:26:20
White_Flame
well, any time you want to update a JIT compilation, you'd have to allocate a new page with its own permissions
15:27:29
White_Flame
oh, interfere patching, correct. It'd have to effectively be manual copy-on-write
17:29:22
sjl
no-defun-allowed: jackdaniel: cl-6502's 25-line JIT is also worth reading as inspiration