freenode/lisp - IRC Chatlog
Search
3:13:37
npfaro
I'm reading a multi-megabyte file into an array and I think slime is crashing every time it tries to print it
3:40:34
pillton
ACTION has had to wait for the REPL to print out a high-definition image a few times.
3:48:46
npfaro
I have an array of values such that there are groups of 4. so I want to loop over them in groups
3:49:11
npfaro
Say I have #(1 2 3 4 5 6 7 8) I'd like to loop over them so that I can treat them as '((1 2 3 4) (5 6 7 8))
3:49:11
Bike
you mean like you have #(0 1 2 3 4 5 6 7 ...) and you want to work with 0 1 2 3 first, then 4 5 6 7, etc?
3:50:36
Bike
i don't think there's a way to do it with the vector directly. you could cook up something with indices like that.
3:53:17
Bike
i would also do it with the indices. if you want nicer syntax, you could probably design your own clause if you used ITERATE instead of loop
3:53:40
Nilby
or something like (loop with a = #(1 2 3 4 5 6 7 8) for i from 0 below (length a) by 4 do (format t "~s~%" (subseq a i (+ i 4))))
3:57:28
edgar-rft
I think displaced arrays are meant to be used for sub-arrays of big arrays but I need to fiddle out the syntax.
4:03:09
Bike
for large sections the displaced array probably uses less memory than subseq and involves less copying time
4:07:49
Bike
if the chunks are pretty short, i would guess that the copying overhead doesn't matter, and instead you hit slowness because you're using aref on a non simple array
4:07:59
ludston
What is a good word for destructuring a collection into chunks of n length? If it was pairs, you would say "zipping and unzipping"?
4:08:15
Bike
sbcl is probably smart enough to realize that if you're doing aref on subseq it's a simple aref
4:08:44
aeth
However, what I personally do in this situation is use the array macros/functions/etc. that I wrote. https://gitlab.com/zombie-raptor/zr-utils/-/blob/adfe5751a2b327569efb55e0d1c89c7aea9c61f7/array.lisp
4:09:12
aeth
I can decompose an array of length 4 into multiple values, to avoid additional consing (in most circumstances)
4:11:13
aeth
This can compose well with most things, but LOOP doesn't afaik have a way to bind multiple values at once (without a SETF in the body, of course) so it might not be the best kind of iteration for that.
4:12:18
aeth
I just recently extended my function to support offsets, so it can solve this problem in particular, e.g. (zr-utils:array-of-4 (vector 1 2 3 4 5 6 7 8) 4) => 5 6 7 8
4:13:34
Bike
a really fancy compiler could probably elide actually consing a displaced array for certain uses. i don't think any implementation actually bothers, though.
4:14:46
aeth
The problem is that "displaced arrays are slow" is common knowledge, so it's not idiomatic to use it, and non-idiomatic things aren't optimized because it's not worth the time.
4:17:00
Bike
at least the kind of immediate cursor i'm imagining being optimizable could just be done pretty straightforwardly with indices
4:17:43
Bike
passing them between functions could be nicer, but then interfunctional optimization is harder
4:18:56
aeth
For 2-4 values, I love the compositional possibilities with the multiple-values route, although the syntax is pretty awkward. At the function boundary, you're basically forced to multiple-value-call. And it's not going to help you if you're calling a non-inline function that uses double-floats (or similarly boxed things).
4:19:47
aeth
Obviously if it's closer to, say, 20+ or especially 200+, then something like displaced arrays would be way better.
4:20:40
Nilby
If displaced arrays are optimized away inside a function, then they're probably fine as just a struct for passing indices around.
4:20:41
pillton
beach: I had a read of your call-site optimization paper. I like it. I think the information about the callers would be useful for editors or reflection. I also think you could perhaps mention if it is possible to compile the callee more than once to suit the type information provided by the caller.
4:22:09
pillton
beach: I wonder if the word "type" in the last sentence of the second paragraph of page 2 should be "class"?
4:22:32
aeth
Nilby: The problem is, most of the heavy optimizations are going to be inside of a function, or at least inside of a file, and the time when you'd really, really want to used displaced arrays is to treat a random subarray as an array when calling a general purpose function that expects the latter
4:24:27
aeth
I suppose it could still help if those functions were inline, which could be a surprising amount of vector math stuff, especially if they were designed for this.
4:28:58
pillton
aeth: I'd rather compilers focus on eliminating indice checks for vector math stuff.
4:29:48
charles`
is there a way to read a file that has lisp code and get what the result is? if a file contains (+ 1 2), (return-load "file") => 3
4:32:42
beach
pillton: I don't think it would be necessary to compile the callee more than once. It would typically have the structure so that a prefix would test the types, and part of that prefix can then be skipped so that the snippet jumps to the middle of the callee.
4:34:19
pillton
beach: For most cases yes, but there are unique cases where it is important e.g. https://github.com/markcox80/template-function/wiki/Motivating-the-Template-Function-System
4:41:16
pillton
beach: I can't at the moment as I have to go soon. I am in no hurry for you to read the link. I think it highlights the issue I am talking about better than I could explain it here.
4:45:05
Nilby
charles`: maybe (eval (read-from-string (alexandria:read-file-into-string "foo.lisp")))
5:05:05
beach
pillton: I am not in favor of recompiling the callee according to the information provided by the current callers, but I can very well see compiling several versions of the callee according to the type information that ANY caller could provide to make the callee faster.
5:10:52
pillton
beach: Sure. That system I linked to provides a mechanism for users to request a specific specialization of an operator. I found this to be easier than trying to implement systems which handle all of the subtypes of REAL and the corresponding array subtypes for every CL implementation.
5:36:06
charles`
Nilby: doesn't work because eval uses some other environment. I tried putting *package* and *readtable* inside, but doesn't work still
5:42:18
Nilby
Hmmm. Maybe you could wrap it with (let ((*package* (find-package :whatever)) (*readtable* (copy-readtable))) (eval (uiop:read-file-form "foo")))
5:57:49
npfaro
Say I have a loop like (loop for (a b) on '(1 2 3 4)) and if a+b > 10 then move past both and onto the next pair
6:35:50
npfaro
Any way to make it in SBCL so that when you type a string at the repl it doesn't respond with #<(SIMPLE-ARRAY CHARACTER (9)) foo... {1004DCC80F}>
6:37:06
beach
If you type a string literal, it should print that same literal, because string literals are self-evaluating objects, and they are printed as usual.
6:38:03
beach
You must have somehow clobbered print-object, or you have some printer variable set wrong.
6:39:01
npfaro
while you're here, do you know how to convert a vector to a (VECTOR (UNSIGNED-BYTE 8)) easily
6:41:52
beach
Try something like (make-array 5 :element-type '(unsigned-byte 8) :initial-contents '(72 101 ...))
7:02:14
aeth
npfaro: COERCE also works, but you generally want to use MAKE-ARRAY because the :initial-contents part can be optimized when it's a quoted list like that...
7:02:28
aeth
i.e. (coerce #(1 2 3 4) '(simple-array (unsigned-byte 8) (*))) where * is the size wildcard so you don't have to specify it (you could say (4) instead of (*))
7:05:37
aeth
In other words, use MAKE-ARRAY with a literal list as the initial-contents if you're making the array in the first place. Otherwise, if you already have a sequence and just need to convert it to another, COERCE.
10:56:20
beach
I think Feldman0 wants to implement my generic-function dispatch technique for ECL, maybe.
10:59:51
jackdaniel
currently ecl has gf dispatch mechanism operates on caches (see the file src/c/clos/gfun.d), however before the last release I've implemented class stamps
11:03:10
jackdaniel
there are internal helper functions si::instance-obsolete-p and si::instance-sig-set to work with instances (used in clos/std-slot-value.lsp) defined in src/c/clos/instance.d
11:04:01
jackdaniel
there are a few problems: a single gf dispatch mechanism does not require additional compilation at runtime, while fast gf dispatch requires you to compile the dispatch function after gathering some information
11:04:30
Feldman0
Do you think that I'll have to implement it in C or in Lisp, I'm not quite sure in what what is done in ECL
11:04:58
jackdaniel
I'm sure that it could be implemented in lisp and compiled with the compiler to c
11:05:46
Feldman0
My main struggle with reading the code base is, I'm not sure why something are in C or Lisp, is there a rule of thumb that is used or?
11:05:47
jackdaniel
so, regarding the problem -- compilation on ecl is slow, so either gf should be compiled in a separate thread, or manually triggered by i.e #'compile
11:06:31
jackdaniel
the common language runtime is implemented in c; while the rest is implemented in common lisp
11:07:02
jackdaniel
basically some low level operators needed to boostrap the language must be written in (or transpiled to) C (unless you decide to boostrap from another common lisp implementation)
11:07:45
jackdaniel
ecl also features a bytecodes compiler/interpreter (alongside native code compiler), and bytecmp is written in C
11:08:16
Feldman0
Ah right so it's like the clos-boot approach in SBCL, so my solution could be like the more fully featured clos written in CL
11:08:36
jackdaniel
you may think of the C part of the codebase as of a VM on top of which you could implement other languages (i.e common lisp)
11:09:40
jackdaniel
sure, you could think of it like that; notice that current gf dispatch machinery is written in C
11:10:43
jackdaniel
afaik sbcl and ecl clos implementations are both based on the pcl implementation
11:11:42
jackdaniel
if you have some detailed questions about ecl internals, you may join #ecl channel to not clutter the general chat with it
11:13:13
Feldman0
I think this is probably fine for now; I just need to write up a draft project proposal and submit it to potential supervisior. It's a bit early for that even, but there's really only one good supervisor for this project so I'm trying to get him early.
12:05:13
VincentVega
Trying to pick a testing framework. Currently looking at prove. The lisp cookbook says "warning: Prove has a couple limitations and will soon be obsolete. We advise to start with another test framework." What limitations? Are there other frameworks that are better in some way?
12:05:17
VincentVega
I guess what I would want from a testing framework is that it just doesn't get in the way, so to speak. I want to have testing code at the top-level (right next the code that I need to test, I don't want to create a seperate file), but I don't want it to run during the compilation, but only be tracked by the framework, so that I can run it all
12:05:17
VincentVega
later if I want to. Eval'ing a form should run it though. Do any of the frameworks support this?
12:18:52
jackdaniel
but if you want to be "the real common lisp programmer", you should write your own testing framework - it is a long lasting tradition among devs ,-)
15:02:13
jackdaniel
Bike: equal could do that too (under the hood, after realizing that both are strings)
15:02:54
jmercouris
but I think you would be writing something really complex if equal had to realize you were slicing
15:10:20
beach
And yes, string= is more specific, so it falls under the rule that one should always use the more specific construct that will have the desired effect.
15:11:21
pfdietz
One could imagine a sufficiently smart compiler that would transform (equal (subseq s ...) (subseq s2 ...)) into a call to string=, but I don't think any existing compilers do that.
15:11:32
beach
But Common Lisp is not about minimalism. It's about giving the programmer a good toolbox.
15:11:39
jmercouris
pfdietz: that's what I was hinting at with my comment that jackdaniel did not understand