freenode/#lisp - IRC Chatlog
Search
17:34:54
jasom
ebrasca: fast-io works for this too, though it's original purpose was for writing to byte-vectors it supports streams as well
17:35:08
minion
ebrasca: look at pcl: pcl-book: "Practical Common Lisp", an introduction to Common Lisp by Peter Seibel, available at http://www.gigamonkeys.com/book/ and in dead-tree form from Apress (as of 11 April 2005).
17:35:27
mfiano
fast-io is more for reading and writing octets. Binary parsing is at a different level.
17:35:59
jasom
mfiano: it is not for reading/writing octets, it can read binary values from 8-64 bits with specified endian
17:37:22
jasom
ebrasca: do you just want serialization/deserialization, or do you want to read externally specified binary formats?
17:37:23
mfiano
Point being, it is a tool for binary parsing, and requires no knowledge of compiler theory to function, unlike binary parsing.
17:42:37
mfiano
A friend wrote that specifically for that use case for me, because there were not any robust solutions available at the time.
17:42:39
Xach
It is also not particularly hard to write it yourself, and is quite educational. the tools cl provides are quite adequate building blocks to build up abstractions.
17:45:04
mfiano
You can check out flac-metadata, for an example of using fast-io and bitio in a mixed endian format. FLAC is BE for most fields, but does include Ogg Vorbis data which is LE.
17:47:41
Xach
i really enjoy files that mix endianness in them, it shows that real things are messy sometimes for reasons.
17:48:14
ebrasca
Here link of superblock structure https://www.kernel.org/doc/html/latest/filesystems/ext4/globals.html
17:52:28
mfiano
Mixed [byte] endianess is incredibly easy to handle. Things can get complicated without good abstractions when trying to handle bit endianness
17:53:05
ebrasca
s_blocks_count_lo is at offset #x4 and s_blocks_count_hi is at offset #x150 , both have 32 bits size and little endian
17:53:58
Xach
ebrasca: so: (if 64-bit-supported-p (+ (ash s-blocks-count-hi 32) s-blocks-count-lo) s-blocks-count-lo)
18:06:02
ebrasca
Xach: I am searching some solution because https://github.com/ebrasca/Mezzano/blob/master/file/ext4.lisp#L220 is very easy to make mistakes and it have some duplication with defstructure and writer.
18:44:44
aeth
The only problematic thing witht he new SBCL change is that SBCL went from completely ignoring :type in DEFCLASS at default optimization levels to not ignoring :type in a niche case that other implementations probably don't check (while still otherwise ignoring :type in DEFCLASS at default optimization levels)
18:46:20
aeth
The only problem with not using (error foo) as a default value to keyword (including the ones generated by :initform) or optional arguments is SLIME's minibuffer behavior, which creates a lot of noise when the default value is (error 'foo "bar baz quux") when the only relevant thing to the user of the API is that the default value is (error)
18:49:17
aeth
Unless this is more extensive than the 2.0.8 changes that got rolled back just before release, this is only for DEFCLASS's :iniform and perhaps DEFSTRUCT default slots (I didn't test the latter) and there are even more invalid APIs in the general case of a function with &optional or &keyword arguments with DECLAREd types that contradict the provided default values.
18:50:45
aeth
mseddon: In 2.0.8 and before, SBCL ignores :type entirely in DEFCLASS when (safety 3) isn't true, or when speed > safety, or something like that. Essentially, by default, :type is a meaningless thing in SBCL, so probably the majority of Common Lispers have never really faced a :type that actually, really checks.
18:51:16
aeth
In 2.0.9, SBCL does a tiny bit of type checking, to make sure that the :initform is a valid form of the slot's :type in DEFCLASS.
18:51:19
mseddon
I mean I happen to declaim that during development, but I would expect this for much lower values of safety.
18:51:36
aeth
(SBCL's official stance has always been to use DEFSTRUCT if you care about types because those are optimizable.)
18:53:15
aeth
mseddon: You wouldn't segfault if the type is invalid (except perhaps at (safety 0) when all bets are off), but you would probably get a runtime error. Except with DEFCLASS, which will only give you that runtime error if (debug 3) and otherwise act like everything's fine if you (setf (foo your-instance) 42) when that slot is, idk, of type conses or something.
18:53:41
aeth
(DEFSTRUCT will give you errors on setting or constructing things with slots of invalid types)
18:54:51
mseddon
aeth: if you find the time to put up an example, I would be happy to rock up alongside you and wave my pitchfork.
18:56:13
aeth
mseddon: (setf (foo your-instance) 42) if foo is the accessor for a DEFCLASS slot with :type single-float is an error, it's just that SBCL won't check for it at default optimization levels. It will be an error in e.g. CCL.
18:56:18
aeth
Take this class: (defclass foobar () ((%foo :initform 0f0 :initarg :foo :accessor foo :type single-float)))
19:00:26
aeth
That might error, that might not. In CCL, that does error. In SBCL, even though SBCL is otherwise the best implementation at checking types, that doesn't error unless the DEFCLASS is defined in a (debug 3) environment (iirc... it certainly doesn't at default).
19:02:13
aeth
Now, replace ":initform 0f0" with ":initform nil". In CCL, that's a runtime type error when calling (make-instance 'foobar) unless a valid value for foo is provided. The appropriate way to do this, though, is to make the initform a call to ERROR.
19:03:14
aeth
In many implementations, it's just ignored entirely and the invalid NIL default value is permitted, including SBCL 2.0.8 at default optimization levels.
19:04:54
aeth
Now according to NEWS at http://www.sbcl.org/news.html SBCL 2.0.9 does this: "the compiler signals a warning at compile-time when an initform of T, NIL or 0 does not match a STANDARD-CLASS slot's declared type."
19:05:42
aeth
SBCL (an implementation people generally use for the static checking) now does a tiny, tiny bit of static checking in a case where it previously did no checking at all, but only when the :initform is T, NIL, or 0
19:06:09
mseddon
aeth: no, this is the thing, I expect SBCL to fall at the first hurdle, and use it because it does.
19:06:10
aeth
(SBCL generally does static checking as compile time warnings that compile into runtime errors)
19:06:40
aeth
mseddon: Right, people are complaining because it does a bit of checking, whereas I got my hopes up that this edge case in SBCL type checking was finally fixed, when it's only fixed in a tiny, tiny, tiny special case.
19:07:35
aeth
Bike: I'm also curious as to why this doesn't (at least didn't in the right-before-2.0.8 release I tested before it got rolled back for 2.0.8) do a similar check for DEFUNs with DECLAREd types incompatible with the default optional/keyword argument values. This one might even catch a few bugs in my code.
19:07:56
mseddon
aeth: I had an amazing boss once who told me the story of managing a project for a simple vector CAD program for a tutorial for a university lesson. It blew up if they added >5 shapes on the screen, he raised it, they got back something that blew up with >6 shapes on the screen...
19:08:58
aeth
Bike: Depending on your perspective. You the user can see intforms as keyword argument defaults of a MAKE-INSTANCE. That's certainly how SLIME exposes them.
19:09:15
Bike
okay, but that's not how sbcl does it, and in fact it's kind of impossible with how mop works
19:09:44
aeth
Bike: Yes, it would have to be checked in a different place, it is just, from the user's perspective, an extremely similar issue.
19:10:26
mseddon
drl: unless you changed your shell, it's usually .bashrc that configures your default shell (bash)
19:10:45
mseddon
drl: so if you add EXPORT SBCL_HOME=/path/to/your/sbcl/home at the bottom of that .bashrc
19:11:03
aeth
Bike: Sorry, I'm communicating it improperly. It's the same class of bug (a default value that cannot possibly be of the valid type, which when properly done should be a call to ERROR instead). It would just have to be checked differently.
19:11:55
mseddon
drl: for least confusion, check it works in a shell after you add that, and then log out and in again, in case some old programs didn't realise that SBCL_HOME changed.
20:05:33
mseddon
drl: np, make sure your shell environment is configured correctly, lisp requires this.
20:46:13
aeth
I personally just create a 3-line shell script (including #!/bin/bash in those 3 lines) in ~/bin to export SBCL_HOME and then call the SBCL executable from there.
20:48:08
aeth
drl: what I did to get it to work is this command: INSTALL_ROOT=/home/yourusernamegoeshere/.local sh ./install.sh
20:48:45
aeth
drl: and then in the script: export SBCL_HOME=/home/yourusernamegoeshere/.local/lib/sbcl
20:49:36
aeth
Installing to /usr/local is something I don't mess with unless I have no other option, e.g. stumpwm (since I need to launch it like any other WM)
20:50:09
aeth
Installing to ~/.local and putting the binary (or a trivial shell script, like I do with SBCL) in ~/bin tends to have fewer issues ime
20:51:34
aeth
The only catch is it might create files you want to exclude from your backups, like anything in ~/.local/lib and ~/.local/bin in this case
20:53:11
aeth
this might work in general to laucnh a custom, home-installed SBCL: export SBCL_HOME=~/.local/lib/sbcl && ~/.local/bin/sbcl
20:53:37
aeth
You might need to put the second part on its own line, though, idk edge cases in shell parsing
20:55:02
aeth
drl: afaik, stumpwm creates its own standalone executable when you install it, so even if you use the ~/.local/bin/sbcl executable, it will use that newer version to build the /usr/local/bin/stumpwm binary
20:59:24
drl
aeth, If i set SBCL_HOME to anything but asdf/quicklisp I get and error message when I start slime.
21:10:18
Nilby
drl: When I'm having such problems I usually make a new user, or use a "clean" with no files, and if it works there, then carefully and reversibly remove stuff from my environment until it works. Then I can maybe fix the problematic piece of setup.
21:13:18
drl
aeth, I followed you directions, and it installed. But now I get this error message: core was built for runtime "lat-l-2020-09-28-06-53-18" but this is "lat-l-2020-09-14-16-03-37"
21:30:28
aeth
drl: I guess remove and reinstall? My guess is the ~/.local/lib/sbcl and ~/.local/bin/sbcl are using different SBCLs for whatever reason... or, alternatively, the correct SBCL_HOME isn't being used
4:04:23
beach
Omg_cholesterol: It means that the semantics of Common Lisp work as if every object you manipulate is in reality a pointer or a reference to some chunk of memory.
4:04:59
beach
But your remark about malloc/free makes me think that you are asking a different question, perhaps.
4:08:54
Omg_cholesterol
lets say I want to read a string from a file, you could just allocate a buffer and hope its big enough maybe 64K or you could calculate the size of the string then alloate a buffer of that size
4:11:55
no-defun-allowed
If you want to reuse a buffer like that, you could use make-string to make the buffer, then repeatedly read-sequence into that buffer. When read-sequence returns a position less than the length of the buffer, you've finished consuming the stream.
4:11:56
aeth
Omg_cholesterol: What I usually do in this situation is I make an adjustable string like this starting with some power of 2 size: (make-array (expt 2 4) :element-type 'character :adjustable t :fill-pointer 0)
4:12:38
aeth
Omg_cholesterol: And then at the very end I (subseq adjustable-string 0 (fill-pointer adjustable-string)) to make a fixed-size (more efficient) string. This last step will copy, though, and isn't strictly necessary.
4:13:27
aeth
You add to it with vector-push-extend, which will extend it if necessary (in my example, it will start at 0 and only extend when it reaches 16)
4:14:33
aeth
And in theory, a compiler could probably recognize this idiom and not copy with the SUBSEQ if it knows that adjustable-string is only used within a very limited scope, but I doubt that optimization is done. (Instead, it would just change the type prefix of the string, which is a dangerous operation and would have to be done at the compiler level.)
4:15:16
aeth
(Technically, a subseq is always supposed to allocate a new sequence, so this optimization would have to be very careful to not semantically change things and appear as if it is following the standard.)
4:20:23
aeth
I think that this style (although it might not be 0 to FILL-POINTER because there might be parts of the string that you want to drop on either side) is the intended style for this in the standard because SUBSEQ (if used on a vector) turns an array into a simple array that otherwise has the same type, i.e. it will no longer be adjustable.
5:21:16
flip214
and local variables might "vanish" in the sense that they're only a register in code, not a memory location anymore
5:29:45
beach
That's an interesting idea for a feature of an implementation though. I wonder how efficient it could be, i.e., how to avoid too much slowdown when such a feature is enabled.