freenode/#lisp - IRC Chatlog
12:56:40
axion
Hello. Is anyone good with bit twiddling? Been stuck on a function for 2 days now. Trying to modify it to support the other endianess. I could make a paste (not much code to parse) if anyone can give me a hand as a last resort before I give up.
13:07:53
beach
axion: the comment says you read bits out of bit buffer, but what's the role of *byte-buffer*?
13:09:01
axion
*byte-buffer* is really just an array of bytes, whose purpose is the same as this code, except for bytes. It can be used to cache large chunks of bytes from a stream, but it's not really relevant to this code.
13:09:21
pjb
axion: the only difficult part then is to convert between bit-vectors and byte vectors or integers.
13:09:59
pjb
axion: for this, you have to mix bit vector indexing, and arithmetic. Happily, you can also use LDB/DPB.
13:10:40
pjb
axion: the conversion got you because there are multiple ways to do it, and you have to choose one. (or the choices may be imposed on you, but you have to know which one it is).
13:11:23
axion
pjb: The truth is I am by no means good at this stuff, and that small bit took me days.
13:12:34
pjb
For example, on serial lines, the msb is transmitted first and the lsb is transmitted last. But on hard disks, it's often the opposite. On the other hand, on multi-track tapes, bits are usually transmitted in parallel. Do you care if they're stored on the outer track or the inner track?
13:14:15
pjb
Your bit-buffer is conspiciously missing a "fill pointer". (assuming your bit-count is, as it seems to be, a pointer to the next bit to be read).
13:14:41
axion
pjb: Ok, I'm not really sure what that means as far as my need for this code. I will be using this to parse binary formats which encode bytes in either BE or LE...and sometimes both
13:15:33
axion
pjb: I modified this from 3b's code here: https://github.com/3b/3b-swf/blob/master/low-level/swftype-read.lisp#L51
13:15:38
beach
axion: As both Xach and pjb said, endianness has to do with the order of bytes within a multi-byte word. The order of the bits within a byte is unimportant here, i.e. always the same.
13:17:42
beach
axion: Sorry, don't know. The protocol in question should tell you from which end of the second byte the additional 2 bits are to be taken.
13:18:03
axion
Ok, what you say makes some sense to me. I will have to think study some more. This is all very confusing to me
13:35:41
thetabit
What are some strategies for ensuring proper type assignment? e.g. ID is always an integer value, Text is a string value, etc. Using declare?
13:35:56
pjb
axion: but notice that since I use LDB wholesomely, bits in the byte are always read in the same order, so you get AA instead of 55 and 33 instead of CC!
13:38:34
pjb
Not calling an external library. Having a definite length. Therefore being able to read bytes in both orders.
13:38:55
dlowe
thetabit: A declaration is a promise, not a check. Use cl:check-type if you really insist.
13:39:37
pjb
thetabit: sorry, my answer was for the question of type alignment, I grossly misread your question.
13:41:53
pjb
axion: it seems you're reading the octets in the integer, octet by octet. This is useless, since ldb can read any number of bits!
13:41:54
thetabit
I am trying to ensure that a string is not bound to an integer when the sql is generated
13:43:15
pjb
thetabit: basically you have two situations. check-type is interactive (if the type is wrong, it breaks into the debugger with a restart to read a new value for the place). So you would in general restrict check-type to interface functions, functions meaningful for the user. Internally, you could just break with a type-error.
13:43:31
beach
For WSCL, the meaning of a type declaration will be defined according to the value of the SAFETY quality.
13:48:37
thetabit
so would I use typep, (when (typep (funcall new-id) 'integer) (setq object-id new-id))
13:49:37
rpg
thetabit: You're checking the return value of NEW-ID, and then setting the OBJECT-ID to the function, rather than the return value. Is that what you want?
13:49:52
axion
You can make a promise to the implementation that a function has a return value of a certain type be using an ftype declaration.
13:52:51
thetabit
That was an example, but this is someone that may be using a web application and they have updated a field. I want to make sure that the input from the user is correct.
13:53:43
thetabit
Let's think about a different example, say there was a datetime field displayed to the user and they could modify it, I would want to make sure that the only string that is bound that field is a datetime string
13:54:14
axion
Ah well for that you would probably want client and server side validation, the former by means of Javascript.
13:59:41
minion
beach: WSCL: Well-Specified Common Lisp. Pronounce it as "whistle". A project to revise the Common Lisp standard in a non-controversial way. See https://github.com/robert-strandh/Well-Specified-Common-Lisp
14:04:02
axion
beach: Can you link me to your paper from...years ago...about sandboxing Common Lisp?
14:09:03
axion
Even if you removed unsafe symbols, you would need to restrict the reader down. For example, evaluating 9d999
14:09:44
beach
The paper is not about sandboxing, so it doesn't have all the sufficient conditions for it. It is about first-class global environments which is one necessary condition for sanboxing.
14:10:26
axion
Yes, I understand. The topic of sandboxing came up with a colleague the other day, and I was looking for the section in that paper you wrote.
14:11:49
axion
Is that paper dated? I've been here quite a while, but only seemed like a couple years ago I proofread that for you :)
14:48:53
np356
Hello ladies and gentlemen. My sister is sortof late on her masters thesis, that requires about ~80 filled survays left. Could you please help out? Its about "The impact of business analysis techniques on software quality": https://www.surveymonkey.com/r/W39T9TB
14:58:36
schweers
is there a portable alternative to sbcls SB-SEQUENCE? If so, any major disadvantages to those compared to just using SB-SEQUENCE?
15:01:19
Bike
sb-sequence is based on a publically defined extension http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-20070301.pdf
15:01:50
Bike
if by "portable" you mean "without deep implementation support" than you're probably out of luck
15:02:45
schweers
hmm. ccl doesn’t seem to have it. I normally use sbcl, but also being able to use ccl would have been a bonus. I was hoping for something like trivial-sequences, but at least quicklisp doesn’t have that
15:03:53
Bike
you could define your own package and do the sequence functions yourself and so on, but you'd probably lose efficiency
15:04:26
schweers
cl:length is a compiled function, whereas sb-sequence:length is a generic function
15:07:35
Bike
this is the idea of the extension. the CL functions remain normal so that it doesn't have to do method dispatch, but for sequences other than vectors and lists it calls generic functions.
15:10:45
schweers
I feel a little weird asking this, but … in sbcl’s definition of length I see this: (sb!sequence:length sequence)
15:11:29
schweers
when I put point on it and say M-. (in emacs with slime) I am taken to a definition like this: (defgeneric sequence:length (sequence) ...)
15:12:49
schweers
what’s even more strange to me is that the last definition is located after this expression: (in-package "SB-IMPL")
15:37:32
schweers
I just have to ask: what do you folks use for generic sequence and/or iteration purposes? just concrete objects, or is there a sort of consensus on what to use?
15:41:47
schweers
I currently use iterate, yet I’m not too happy with the fact that it seems tied to certain sequence types. works fine for the built-in ones, but I’m a little worried about user defined sequence types
15:42:21
schweers
although iterate should be extensible enough to support the odd custom sequence, it doesn’t seem generic.
15:44:01
foom
If you really want generic sequences, you should use a language other than common-lisp.
15:47:24
schweers
I should add that I’m not really concerned about loosing cl:mapcar, cl:reduce and the like as long as there are viable alternatives
15:53:26
Xach
It seems to me like a chicken and egg problem. To be really useful, it has to be widely used.
15:55:44
schweers
okay, but there is nothing inherently problematic about them which I should be aware of?
15:55:45
jackdaniel
one could create my-future-common-lisp package with these functions amended. fwiw, closer-mop has such package called closer-common-lisp
15:59:29
Xach
jackdaniel: that's easier to get buy-in because there is really no other option for portable MOPpery
16:00:41
foom
The only other unfortunate part is that CL itself doesn't treat lists and vectors generically.
16:02:18
jackdaniel
yeah, I'm not saying its *the* solution, but adding generic abstraction over sequences may be done given one is willing to use such mfcl package
16:06:21
beach
The creators of Common Lisp typically had very good reasons for the decisions they made. It is too easy to complain about it for people who haven't contemplated all the parameters that went into their decisions.
16:06:52
beach
And it's boring to listen to these complaints as well, especially since nothing can be done about the problems.
16:07:42
foom
I certainly do not intend to assign blame -- I'm certain there have been good reasons for each decision, from everyone involved, which eventually necessitated the current state of affairs.
16:09:24
beach
The sequence functions treat lists and vectors uniformly. Any other uniform treatment of them would imply serious performance penalties, and performance was definitely high on the list of priorities for the Common Lisp creators.
16:10:43
beach
Requiring AREF to be used on a list, or a sequence of CDRs on an array would be totally unacceptable.
16:11:43
beach
And designing a data type that would be kind-of efficient for both use cases, would require some kind of tree structure which gives a non-negligible performance penalty for both kinds of access.
16:12:53
Bike
i think the extension's compromise is reasonable and doesn't impact performance measurably
16:16:00
beach
It is in fact a great compromise if the sequence functions remain ordinary functions which call the generic version only when the sequence is neither a list nor a vector, which is how it is implemented in SBCL.
16:20:09
Bike
i think it would be interesting if there were better mechanisms for specifying compile time type discrimination, that it could probably be done well, and that the reason such a thing isn't in the standard is just that it would be a pretty complicated project
16:22:55
foom
I do not complain that there exist specialized functions that work only on lists or vectors (like cdr or aref), there's no issue with that.
16:23:19
foom
But, in some cases, functions that work generically on both are missing (e.g. dosequence), or functions that work on both have arbitrarily different behavior (e.g. equal).
16:23:33
beach
foom: So what was it that you wanted when you said "The only other unfortunate part is that CL itself doesn't treat lists and vectors generically."?
16:24:47
Bike
equal is a little bit arbitrary. but, given the rest of the language, including eql, you can write something with different behavior
16:27:08
Bike
i haven't implemented a hash table myself, but i don't think providing those hooks is difficult?
16:27:21
beach
But now we are going to have complaints that it is not in the standard, even though most people are happy to use languages that don't even HAVE a standard.
16:29:32
Bike
but how many of those people are in this channel? at least, i doubt anybody here would prefer there to be no lisp standard to there being a flawed standard
16:32:39
Bike
and there's stuff like CFFI and BT to fill in gaps, there just needs to be enough pressure for it
16:33:04
foom
Yep. As I said at the start: Most people just make do without, because that's good enough.