freenode/#lisp - IRC Chatlog
Search
20:06:14
jcowan
I don't see how it can be fast without incurring mutable numbers, unless you are talking about immutable bit vectors (in which case they might as well be numbers)
20:06:58
|3b|
no, i just mean being able to (setf (subseq bit-vector 10 20) (bits-from-int 3)) or similar
20:07:57
|3b|
instead of (setf (subseq bit-vector 10 20) #b000000011) or however many 0 that should be, and even worse if it is a variable
20:09:12
jcowan
I agree that packaging int->bitvector and bitvector->int make sense, but you have to accept the memory allocation penalty
20:10:00
jcowan
Because they are purely bitwise, there is no reason why they should be limited to 1d bitvectors
20:10:08
jasom
it should be pretty easy to write a setf expander that lets you use a bytespec on a bit-vector; then you can do (setf (bit-ldb bit-vector (byte 10 10)) (ldb (integer 10 10)))
20:11:38
jasom
you can use the subseq format alternatively, but I would personally prefer the ldb format for things as that makes both sides of the assignment consistent for a common use-case.
20:12:52
|3b|
and since you are writing the expander it could even recognize some special cases and copy bits directly without an intermediate bignum
20:13:32
jasom
most uses of ldb won't make a bignum just because you usually don't have larger than word bitfields
20:20:51
jcowan
ACTION examines SRFI 151 (integers as bits) to see which operations are strictly bitwise
20:21:46
jcowan
There are some that plainly make sense even though they are not bit-for-bit, like population count
20:23:54
jcowan
true, although that is probably inefficient on x86, as it's hardly likely to use POPCNT
20:26:56
|3b|
yeah, looks like it does logcount on backing buffer a word at a time, which uses popcnt
20:30:41
jasom
|3b|: you can't avoid creating a bignum in the case that the byte would be a bignum, becuase SETF needs to return the value assigned
20:33:11
jcowan
dammit, it looks like all 30 integers-as-bit ops make some kind of sense in bitvectors except arithmetic-shift, and that would be replaced by logical-shift
20:39:04
aeth
jasom: SETF needs to return the value assigned, but SBCL can ignore that value if SETF isn't in a position where its return value is used.
20:40:20
jasom
aeth: so if I only create the bignum at the very end of the storing form, then sbcl may be smart enough to remove it?
20:42:06
aeth
jasom: if an inline function (including an inline defun (setf foo)) or a macro (including defsetf?)
20:44:26
aeth
jasom: (defun foo (a) (declare ((simple-array double-float (1)) a)) (setf (aref a 0) 42d0) a) ; no allocation because it doesn't need to return 42d0
20:44:32
aeth
jasom: (defun foo (a) (declare ((simple-array double-float (1)) a)) (setf (aref a 0) 42d0)) ; allocation because it does need to return 42d0
20:45:35
aeth
jasom: So SBCL can in the case of double-float and (unsigned-byte 64) and (signed-byte 64) certainly remove the unused allocations of the return values of SETFs under certain (inline) circumstances.
20:45:56
aeth
jasom: whether or not it handles your specific case (or could be patched to do so) is unknown
20:47:22
Bike
this isn't just removing an allocation though, it's turning an allocation into a mutation.
21:39:50
jasom
naive implementation (only lightly tested): https://gist.github.com/jasom/037465e21c0908489428704a01777550
21:40:28
jasom
(let (( x (make-array 32 :element-type 'bit :initial-element 1))) (setf (bit-ldb x (byte 4 0)) #xa) x) ;; => #*11111111111111111111111111111010
21:40:48
|3b|
Shinmera: https://github.com/3b/cl-spidev/blob/master/low-level.lisp#L140-L214 is what i ended up with for non-consing ioctl-based read API so far
21:44:14
|3b|
also might clean up the API a bit, ended up trying 2 different ways, so might remove the one i ended up not using (once i decide which that is, since i think i currently have a bit of both still)
21:45:19
|3b|
one was a big destination buffer, with an xfer struct per range in the buffer, then switched to a smaller set of xfer buffers that i slide across the big buffer
21:45:44
|3b|
but now i'm back to only using a fairly small read buffer, so might go back to fixed buffer of xferss
21:46:54
Shinmera
Anyway, this is really cool. Wonder if I'm going to use it myself for some Pi stuff
21:46:55
|3b|
but i checked in code that uses it, so made a fork for it too so both can be pushed somewhere at same time
21:47:39
|3b|
ACTION might also rewrite the commits to get rid of tabs, so that fork might get reset at some point
21:49:38
|3b|
https://github.com/3b/3b-lepton/blob/master/capture.lisp is the code that uses it, still fairly ugly (and made more so trying to avoid consing bignums without restricting sizes too much)
21:50:27
|3b|
could just start a thread, and not worry nearly as much about consing intermediate values
21:51:05
|3b|
ACTION tried moving some work around in hopes of getting it fast enough to stream to ffmpeg without threads, not quite gotten far enough to test yet though
21:52:14
|3b|
i think there are is a debian fork too, and ubuntu server if you hack boot setup a bit
21:53:46
|3b|
was hoping to get this fast enough to run on pi zero, don't think it will be able to encode at same time though, even if it can display (which is probably iffy)
21:54:45
|3b|
more that just waiting on SPI for capture doesn't leave much time for doing anything else
21:56:06
|3b|
well, better than trying to run callbacks on another thread in unthreaded sbcl, which so far doesn't seem to be going to work :p
21:59:18
|3b|
currently i can just capture to memory and encode later, which is probably good enough for now
22:03:49
|3b|
not sure if the camera was configured for 0-655K or 0-6555K range, or not set for calibrated at all
22:06:15
|3b|
but should be able to do better with a 14bit LUT than they do with "gain control -> 8 bits -> 8bit LUT"
22:07:41
|3b|
well, with calibrated mode, most of the 'interesting' range is fairly narrow band (say freezing to a bit above body temp), so would want most of the variation there
22:08:34
|3b|
so maybe the hue rainbow there, and a blue->white ramp (with some added banding or cycling of value) for colder, and red->yellow->white for hotter
22:09:18
|3b|
or maybe a more complicated waveform in middle, will have to think about actual # of values, and how fast they change
22:09:46
|3b|
but that will be something to play with once i have live display (which is probably next task)
22:10:47
|3b|
can upload raw 16bit data to GPU, and a LUT texture, and remap it on display, then write various functions to generate new LUTs
22:11:43
|3b|
and for that don't losing sync for a second isn't too horrible, so don't need to worry about fancy threading or mmaping stuff like extended encoding capture
22:13:32
|3b|
also need to try hooking up small display soon, so i can make it portable, HDMI output to 24" monitor isn't too good for that :)
22:50:06
jasom
A positive fixnum, the exact magnitude of which is implementation-dependent, but which is not less than 1024. <-- it is guaranteed to fit within a fixnum
22:51:09
whartung
array-dimension-limit is the number of dimensions, not the magnitude of a single dimension I think
22:51:56
|3b|
(satisfies (lambda (a) (array-in-bounds-p array a))), for some specific array ARRAY :p
22:52:10
jasom
so that means fixnums must be at least 10 bits. There go my plans for an 8008 common lisp implementation :P
23:06:31
pjb
But be careful that (expt array-dimension-limit array-rank-limit) is usuall less than array-total-size-limit.
23:14:35
aeth
but notice that what it does support is a flexible *end* index, for e.g. stacks. it doesn't support a flexible start index
23:15:07
aeth
It's very similar to the whole ND array thing, though. Actually even simpler to implement.
23:15:52
aeth
I guess the issue is (aref foo 1 2) is unambiguously a 2D array, but (aref foo 1) isn't unambiguously the 1st element in a 1D array if it has a flexible start point for the index. So you'd make every aref slower for a rare feature unless you fully type declared everything.
23:29:35
jasom
wow sbcl's call-arguments-limit appears to be most-positive-fixnum. I don't think I'm going to exceed that any time soon
23:32:23
sjl
(defun function-with-one-hundred-thousand-arguments #.(loop :repeat 100000 :collect (gensym)) nil)
23:32:23
Colleen
sjl: PuercoPop said 22 hours, 5 minutes ago: Have you tried the command refresh-heads when StumpWM does recognize the new heads? https://stumpwm.github.io/git/stumpwm-git_9.html#External-Monitors
23:32:35
whartung
how big of a 2 element, N D array can you make before your computer ejects your swap partion through the case.
23:37:15
|3b|
hmm... for i of-type (unsigned-byte ,',(- (integer-length most-positive-fixnum) (integ\
23:46:10
aeth
I think you'll run into 64 bit RAM size limitations before you run into 63 bit fixnum limitations in most cases
23:46:42
aeth
you might be able to construct something with bit arrays... of course, then you do need more RAM than you normally get
23:49:09
aeth
|3b|: The official raspberry pi OS? I'd think you'd be able to just install the ARM64 fedora/etc. distros on it, idk
23:50:06
|3b|
ACTION tried a few other options and failed to get a good setup, will probably try some of the other options at some point
23:51:55
aeth
but practical minimums really require 32-bit or 64-bit, and 64-bit is a lot more convenient (e.g. unboxed single-float)
23:58:40
aeth
jasom: In SBCL double-float and (unsigned-byte 64) and (signed-byte 64) all heap allocate unless they're used in restrictive ways (within a function, saved to typed struct slots or specialized arrays)
23:59:51
aeth
In 32-bit ([un]signed-byte 64) are bignums in SBCL and their specialized arrays aren't there (just T arrays)
0:00:38
aeth
jasom: single-float's used a lot in graphics. Plus, you can work with double-floats and have single-float as the end result.
0:00:39
jasom
you would need to vary your range by 2**39 to ever have more precision with single-float vs unsigned-byte 64
0:01:54
aeth
jasom: single-floats are faster when you don't really care about the end result that much (e.g. gaming), and even faster since you'd have to use a third party library for fixed-point arithmetic
0:02:52
aeth
jasom: afaik in practice single-floats are just double-floats whose end results are stored as single-floats in CL implementations
0:03:44
aeth
jasom: But even then in CL you'd have to wrap it in (mod ... (expt 2 64)) after every operation to get it to be fast
0:05:06
aeth
generally this sort of thing is run with https://github.com/Shinmera/float-features/blob/91739d64c2d5c99f6add31be7367a9416a98891f/float-features.lisp#L143-L194
0:05:39
aeth
okay, it's actually generally just run with sb-int:with-float-traps-masked and a slow path for every other implementation, but that's the author's fault for not using the library Shinmera wrote for that
0:06:39
jasom
I do very few calculations where the required precision is relative to the magnitude of the value, so *shrug*
0:06:49
aeth
jasom: mask float traps when you are running your application, but keep the defaults when you run your unit tests
0:08:19
jasom
My motto is "fixnums can represent the entire solar system to a precision of under 1mm"
0:11:16
jasom
2um using the major axis of neptune: https://www.google.com/search?q=9.09e9km%2F2**62+in+um
0:11:30
aeth
jasom: usually there's some complicated algorithm that works around problems with floating point... and you can just implement the algorithm
0:12:51
aeth
jasom: If you do a solar system with single-floats (you could just go to doubles at that point) then you can divide it into zones
0:14:09
aeth
jasom: anyway, I think floating point is in general faster because CPUs build in hardware for floating point (but you might need special ASM to access it all)
0:14:15
jasom
At a minimum, my point is that integers are much larger than they were when floating-point won over fixed-point, so reevaluating ones choices might be good. Also, outside of certain scientific calculations the assumptions of floating-point may not fit your data
0:14:57
aeth
if you're doing real time use floats, if you're doing supercomputer simulations use floats.
0:15:19
aeth
Anything where you'd be using Fortran before SBCL became an acceptable Fortran, you probably want floats.
0:16:19
aeth
I've written my share of efficient integer arithmetic, too. It's a bit more painful in CL because CL actually wants correct behavior, so you need to establish your bounds or use unsigned and wrap each arithmetic operation with mod of a power of 2
0:18:04
aeth
I'm sure a compiler that enforced float correctness would make floats equally or more painful, of course. With floats you just get runtime nonsense values or a runtime exception (depending on settings), rather than a compile time possible bignum allocation.
0:20:13
aeth
I have established bounds for certain float functions with random testing, but I think that those bounds are probably best left as just documented rather than enforced.
0:26:55
aeth
Anyway, that's just my 2¢. On the other hand, if there's anyone who'd waste months implementing efficient fixed point in CL to be used in a handful of functions, it'd probably be me. :-p
0:30:46
oni-on-ion
"optimization: adding 1 to or subtracting 1 from a fixnum variable does not cons." =)
0:38:58
aeth
Hmm, doesn't appear to give correct results for me. I guess I was reading the manual incorrectly.
2:26:50
ealfonso
Why do I get different results with macro vs macrolet? https://pastebin.com/4nu0qsr9
2:28:24
|3b|
ACTION doesn't remember if there is a better way to get that local environment than with a macro or not
2:29:29
|3b|
macro/macrolet accept &environment lambda-list keyword to let you get access to the environment while expanding the macro
2:54:26
ealfonso
actually I'm still stuck... shouldn't the inner macro (defined via macrolet) be expanded before the defclass macro? https://pastebin.com/1U571DZC
2:56:08
Bike
since that's not even in a normal evaluation context, the my-local-slot-macro form will never be evaluated, so it's just a syntax error.