libera/#commonlisp - IRC Chatlog
Search
13:15:32
drmeister
We use jupyterlab all the time. Use https://github.com/yitzchak/common-lisp-jupyter/ it is way more advanced and updated. I've been supporting development of it for more than a year.
13:26:22
hexology
nice drmeister! i use jupyterlab for data science all the time, seems like a good match for the common lisp "interactive editing" workflow
13:29:07
drmeister
hexology: We are using a lot of widgets as well for molecular visualization, xyplots, drawing molecules etc.
13:29:45
hexology
i don't know much about molecules but i've been working with a lot of geospatial data recently (a new area for me)
13:30:07
hexology
seems like a lot of the heavy lifting is done by c libraries and tools like proj, so i imagine it's "only" a cffi wrapper away from being available in lisp
13:39:31
Bike
cffi is more explicit about pointers than C is. for example when you do the equivalent to declaring an auto variable a la "char s;", you'd do with-foreign-object, which gives you a pointer to s
13:41:54
hexology
this seems to be perfectly valid C as far as i've read the gnu stdlib docs, and i want to do the same from lisp: https://tio.run/##VU5LCsIwFNznFI@K0toPalUIbb2GG0Fi0s@DmJQkulB6dWOKKzczMD@G51wy1Xu/QMXlQ7RQWyck3orhRP401LNEUDm4M1TxU6NI4E0A@MAMrC00EG13ZbE/AKW0oFEVvE4biC2@2qsDDIlNFaiGMlCa/uoAowmrXRwtN8Wxu6goA@uM0yK2GaxsksxDE5mI9x/eSdZbn4dDDafU52cmZcCxFUw55F8
13:42:22
Bike
but what i'm saying is that when you do "char *s" in C, there will be an implicit char**, more or less
13:42:45
hexology
(i'm not 100% sure if it's considered correct to re-use the same pointer, or if it just happens to work by coincidence)
13:44:29
Bike
with-foreign-string allocates space for a string, but what you want is space for a pointer to a string.
13:45:34
hexology
i see. there's also with-foreign-pointer, is that less general and not useful here?
13:46:12
Bike
that would also work, but you give it a raw size in bytes. with-foreign-object will figure out the correct size based on the type.
13:48:01
Bike
another way of thinking about this is that in C, & is not a "real" runtime operation. If you do like, "*s = x" that will be a memory write at runtime, and "x = *s" will be a memory read. but & does not have a runtime equivalent - it's just telling the compiler to use the pointer it's already arranged
13:48:40
hexology
right, that makes sense. it can see at compile time that a pointer is needed, and it can stack-allocate the space for the pointer, right?
13:49:20
hexology
so i would use with-foreign-object of type (:pointer :char) and size being the size in bytes of my lisp string, then i'd (setf (mem-aref ...) ...) repeatedly to fill it with the bytes from my lisp string?
13:51:02
hexology
oh, i had wanted to use the same (non-const) string in the 1st and 2nd arguments, so i could loop over and over until the end of the string
13:52:38
hexology
although conceptually (and my weakness with c is on display here) i guess the string can be const but the pointer can be non-const? gah
13:52:42
Bike
okay. in that case i think what you would do is allocate the string and a :pointer :char end-str, then do one (setf (mem-ref end-str (:pointer :char)) str), then in the loop do (strtod (mem-ref end-str (:pointer :char)) end-str)
13:53:00
Bike
end-str is a pointer to a string, not a string, so you're not filling it with bytes or anything
13:57:02
Bike
this would be like the C "char** end_str; *end_str = str; while (true) { ... strtod(*end_str, end_str); ... }"
14:03:48
hexology
hm, what's the right syntax for using (:pointer ...) in with-foreign-object? it says "The function :POINTER is undefined." when i do: (with-foreign-object (s1 (:pointer :char)) nil)
14:07:57
jackdaniel
we should make an addition to the standard that states, that like keywords are self-evaluating, a similar thing happens to functions in the keyword package ,)
14:08:07
hexology
well it works! but the result is wrong :) thanks for the assistance w/ the pointers though
14:25:42
hexology
maybe i'll ask on stackoverflow, i doubt i'm the only person to struggle w/ things like this
18:58:54
hexology
i have a variable (type (integer 0 *) n) and i'm calling (incf x) but i keep getting compiler notes that sbcl is unable to "open-code float conversion in mixed numeric operation" because it apparently doesn't understand the type of the `1` in (+ 1 n). i have tried all manner of contortions like (+ (the (integer * 0) 1) n) but i keep getting the same compiler note. if only for my understanding: what the heck is going on here, and how do i convince
19:00:48
Bike
the notes are kind of dumb sometimes. what sbcl does is try every possible transformation it knows for (+ a b) and tells you if they don't apply. and that includes the float ones.
19:01:22
Bike
but yeah the problem with optimizing that is that sbcl can't do much with just (integer 0 *) in that circumstance.
19:02:40
hexology
Bike: i see, so it's telling me everything it tried along the way, not necessarily that it ultimately found nothing?
19:03:10
hexology
i don't know why it's even trying to "open-code a float conversion", i'm just working with ints in this case
19:03:46
Bike
like i said, it's trying every possible way to get rid of a call to +. you can see it messing around with complexes too
19:04:50
Bike
it sees (+ 1 n) and goes, okay, can i make this into (internal::float+ (float 1) n)? no, because n isn't a float. i'd better tell the user this in case they could declare n to be a float
19:06:53
Bike
which in this case is obviously (to you and i) not going to happen, but sbcl is not smart enough to realize that since you declared the thing to be an integer, you're probably not going to want it to be a float
19:07:20
hexology
i see. i guess that means sbcl not have some fancy optimized operation for incrementing an integer? (it probably is not important for performance anyway, in this code)
19:08:04
Bike
that means it could be a bignum, which is a big ol multiprecision integer in memory that can't really be dealt with very quickly
19:08:28
Bike
if it knew n could fit in a machine word, like a fixnum, it would just do a machine addition, but it doesn't, so it won't
19:09:11
hexology
hm. fwiw i get the same compiler notes if i declare it to be fixnum rather than (integer 0 *)
19:11:20
Bike
you mean doing (type fixnum n-blank), not just using the? right. i think what happens there is that sbcl doesn't believe you, i.e. doesn't believe that (+ n-blank 1) will always be a fixnum
19:11:36
Bike
because if it _did_ believe you, and you were mistaken, your program would crash badly
19:13:22
Bike
if it wrapped around to 0 that could be very weird for your program, since that's violating what + usually does
19:13:45
Bike
however if you tell it to wrap, like by doing (mod (+ 1 n-blank) ...), that's another story
19:17:01
hexology
i see. i agree that implicitly wrapping around isn't good (and isn't what i want), but 99.999% of the time this value will actually be in fixnum range. i was about to ask, is there a standard way to handle this? like creating a bignum variable but setting it aside unless you are in danger of overflowing?
19:18:51
White_Flame
handle what? the default case handles the 0.001% of the time where bignums are needed
19:18:53
Bike
honestly what most people do is just make it use fixnum arithmetic anyway, which on 64 bit sbcl will go just dandy up until you give it a file with four quintillion lines
19:19:32
Bike
in your particular case, i might just remove the type declaration, since i expect fixnum arithmetic is the least of your problems here optimization wise
19:26:12
hexology
but for the sake of the exercise, how do i convince it that i definitely want to use fixnum arithmetic and that i don't care what happens if i overflow?
19:28:27
White_Flame
if you declared a var fixnum, and then INCF'd it, then with safety off would probably just do fixnum math
19:29:38
Bike
the implementation is indeed free to blow up if THE is violated, but that would be rude of it, and i think sbcl checks THEs except on low safety
19:29:42
hexology
(setq n-blank (the fixnum (+ (the fixnum 1) n-blank))) like this? i hadn't tried that, but i get the same compiler notes if i do that
19:29:49
Bike
although i guess it just compile it to use fixnum arithmetic and then cheeck for overflow
19:31:51
hexology
aha, this worked, thank you: (setq n-blank (the fixnum (+ 1 (the fixnum n-blank))))
19:38:57
pjb
hexology: (setq n-blank (if (< n-blank most-positive-fixnum) (the fixnum (+ 1 (the (integer 0 #.(1- most-positive-fixnum)) n-blank))) (the integer (1+ the integer n-blank)))))
19:44:00
hexology
does it matter if you write (< n-blank most-positive-fixnum) or (< n-blank #.most-positive-fixnum) ?
19:45:31
Bike
The latter means the reader looks up the value of most-positive-fixnum and sticks that in instead of the symbol
19:45:58
Bike
but even the very basic compiler i wrote in four hours the other day is smart enough to compile constants as constants, so don't worry about it
19:46:03
aeth
White_Flame: what about an implementation that allows for compatibility of its compiled fasls between versions? And the version changes the constant.
19:46:38
pjb
hexology: but THE doesn't evaluate the type specifier! So it must be composed at read-time.
19:49:08
aeth
9 times out of ten when I use #. I later get rid of it with a deftype, a defmacro, etc.
19:52:49
jcowan
There are two possible interpretations of declarations: "I declare that this value is a fixnum even if sometimes it isn't, and I allow you to send me straight to Hell if it isn't" and "I declare that this is value is a fixnum, and if it isn't, I want you to signal a condition."
19:54:18
Bike
sbcl also offers "i declare that this is a fixnum and i'd appreciate if you checked but like don't worry too much man"
19:55:49
Bike
i would also like to implement "I declare that this is a fixnum and if you actually use that information signal a condition, but otherwise who cares" but that's pretty involved to do