freenode/#lisp - IRC Chatlog
Search
12:56:40
axion
Hello. Is anyone good with bit twiddling? Been stuck on a function for 2 days now. Trying to modify it to support the other endianess. I could make a paste (not much code to parse) if anyone can give me a hand as a last resort before I give up.
13:07:53
beach
axion: the comment says you read bits out of bit buffer, but what's the role of *byte-buffer*?
13:09:01
axion
*byte-buffer* is really just an array of bytes, whose purpose is the same as this code, except for bytes. It can be used to cache large chunks of bytes from a stream, but it's not really relevant to this code.
13:09:21
pjb
axion: the only difficult part then is to convert between bit-vectors and byte vectors or integers.
13:09:59
pjb
axion: for this, you have to mix bit vector indexing, and arithmetic. Happily, you can also use LDB/DPB.
13:10:40
pjb
axion: the conversion got you because there are multiple ways to do it, and you have to choose one. (or the choices may be imposed on you, but you have to know which one it is).
13:11:23
axion
pjb: The truth is I am by no means good at this stuff, and that small bit took me days.
13:12:34
pjb
For example, on serial lines, the msb is transmitted first and the lsb is transmitted last. But on hard disks, it's often the opposite. On the other hand, on multi-track tapes, bits are usually transmitted in parallel. Do you care if they're stored on the outer track or the inner track?
13:14:15
pjb
Your bit-buffer is conspiciously missing a "fill pointer". (assuming your bit-count is, as it seems to be, a pointer to the next bit to be read).
13:14:41
axion
pjb: Ok, I'm not really sure what that means as far as my need for this code. I will be using this to parse binary formats which encode bytes in either BE or LE...and sometimes both
13:15:33
axion
pjb: I modified this from 3b's code here: https://github.com/3b/3b-swf/blob/master/low-level/swftype-read.lisp#L51
13:15:38
beach
axion: As both Xach and pjb said, endianness has to do with the order of bytes within a multi-byte word. The order of the bits within a byte is unimportant here, i.e. always the same.
13:17:42
beach
axion: Sorry, don't know. The protocol in question should tell you from which end of the second byte the additional 2 bits are to be taken.
13:18:03
axion
Ok, what you say makes some sense to me. I will have to think study some more. This is all very confusing to me
13:35:41
thetabit
What are some strategies for ensuring proper type assignment? e.g. ID is always an integer value, Text is a string value, etc. Using declare?
13:35:56
pjb
axion: but notice that since I use LDB wholesomely, bits in the byte are always read in the same order, so you get AA instead of 55 and 33 instead of CC!
13:38:34
pjb
Not calling an external library. Having a definite length. Therefore being able to read bytes in both orders.
13:38:55
dlowe
thetabit: A declaration is a promise, not a check. Use cl:check-type if you really insist.
13:39:37
pjb
thetabit: sorry, my answer was for the question of type alignment, I grossly misread your question.
13:41:53
pjb
axion: it seems you're reading the octets in the integer, octet by octet. This is useless, since ldb can read any number of bits!
13:41:54
thetabit
I am trying to ensure that a string is not bound to an integer when the sql is generated
13:43:15
pjb
thetabit: basically you have two situations. check-type is interactive (if the type is wrong, it breaks into the debugger with a restart to read a new value for the place). So you would in general restrict check-type to interface functions, functions meaningful for the user. Internally, you could just break with a type-error.
13:43:31
beach
For WSCL, the meaning of a type declaration will be defined according to the value of the SAFETY quality.
13:48:37
thetabit
so would I use typep, (when (typep (funcall new-id) 'integer) (setq object-id new-id))
13:49:37
rpg
thetabit: You're checking the return value of NEW-ID, and then setting the OBJECT-ID to the function, rather than the return value. Is that what you want?
13:49:52
axion
You can make a promise to the implementation that a function has a return value of a certain type be using an ftype declaration.
13:52:51
thetabit
That was an example, but this is someone that may be using a web application and they have updated a field. I want to make sure that the input from the user is correct.
13:53:43
thetabit
Let's think about a different example, say there was a datetime field displayed to the user and they could modify it, I would want to make sure that the only string that is bound that field is a datetime string
13:54:14
axion
Ah well for that you would probably want client and server side validation, the former by means of Javascript.
13:59:41
minion
beach: WSCL: Well-Specified Common Lisp. Pronounce it as "whistle". A project to revise the Common Lisp standard in a non-controversial way. See https://github.com/robert-strandh/Well-Specified-Common-Lisp
14:04:02
axion
beach: Can you link me to your paper from...years ago...about sandboxing Common Lisp?
14:09:03
axion
Even if you removed unsafe symbols, you would need to restrict the reader down. For example, evaluating 9d999
14:09:44
beach
The paper is not about sandboxing, so it doesn't have all the sufficient conditions for it. It is about first-class global environments which is one necessary condition for sanboxing.
14:10:26
axion
Yes, I understand. The topic of sandboxing came up with a colleague the other day, and I was looking for the section in that paper you wrote.
14:11:49
axion
Is that paper dated? I've been here quite a while, but only seemed like a couple years ago I proofread that for you :)
14:48:53
np356
Hello ladies and gentlemen. My sister is sortof late on her masters thesis, that requires about ~80 filled survays left. Could you please help out? Its about "The impact of business analysis techniques on software quality": https://www.surveymonkey.com/r/W39T9TB
14:58:36
schweers
is there a portable alternative to sbcls SB-SEQUENCE? If so, any major disadvantages to those compared to just using SB-SEQUENCE?
15:01:19
Bike
sb-sequence is based on a publically defined extension http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-20070301.pdf
15:01:50
Bike
if by "portable" you mean "without deep implementation support" than you're probably out of luck
15:02:45
schweers
hmm. ccl doesn’t seem to have it. I normally use sbcl, but also being able to use ccl would have been a bonus. I was hoping for something like trivial-sequences, but at least quicklisp doesn’t have that
15:03:53
Bike
you could define your own package and do the sequence functions yourself and so on, but you'd probably lose efficiency
15:04:26
schweers
cl:length is a compiled function, whereas sb-sequence:length is a generic function
15:07:35
Bike
this is the idea of the extension. the CL functions remain normal so that it doesn't have to do method dispatch, but for sequences other than vectors and lists it calls generic functions.
15:10:45
schweers
I feel a little weird asking this, but … in sbcl’s definition of length I see this: (sb!sequence:length sequence)
15:11:29
schweers
when I put point on it and say M-. (in emacs with slime) I am taken to a definition like this: (defgeneric sequence:length (sequence) ...)
15:12:49
schweers
what’s even more strange to me is that the last definition is located after this expression: (in-package "SB-IMPL")
15:37:32
schweers
I just have to ask: what do you folks use for generic sequence and/or iteration purposes? just concrete objects, or is there a sort of consensus on what to use?
15:41:47
schweers
I currently use iterate, yet I’m not too happy with the fact that it seems tied to certain sequence types. works fine for the built-in ones, but I’m a little worried about user defined sequence types
15:42:21
schweers
although iterate should be extensible enough to support the odd custom sequence, it doesn’t seem generic.
15:44:01
foom
If you really want generic sequences, you should use a language other than common-lisp.
15:47:24
schweers
I should add that I’m not really concerned about loosing cl:mapcar, cl:reduce and the like as long as there are viable alternatives
15:53:26
Xach
It seems to me like a chicken and egg problem. To be really useful, it has to be widely used.
15:55:44
schweers
okay, but there is nothing inherently problematic about them which I should be aware of?
15:55:45
jackdaniel
one could create my-future-common-lisp package with these functions amended. fwiw, closer-mop has such package called closer-common-lisp
15:59:29
Xach
jackdaniel: that's easier to get buy-in because there is really no other option for portable MOPpery
16:00:41
foom
The only other unfortunate part is that CL itself doesn't treat lists and vectors generically.
16:02:18
jackdaniel
yeah, I'm not saying its *the* solution, but adding generic abstraction over sequences may be done given one is willing to use such mfcl package
16:06:21
beach
The creators of Common Lisp typically had very good reasons for the decisions they made. It is too easy to complain about it for people who haven't contemplated all the parameters that went into their decisions.
16:06:52
beach
And it's boring to listen to these complaints as well, especially since nothing can be done about the problems.
16:07:42
foom
I certainly do not intend to assign blame -- I'm certain there have been good reasons for each decision, from everyone involved, which eventually necessitated the current state of affairs.
16:09:24
beach
The sequence functions treat lists and vectors uniformly. Any other uniform treatment of them would imply serious performance penalties, and performance was definitely high on the list of priorities for the Common Lisp creators.
16:10:43
beach
Requiring AREF to be used on a list, or a sequence of CDRs on an array would be totally unacceptable.
16:11:43
beach
And designing a data type that would be kind-of efficient for both use cases, would require some kind of tree structure which gives a non-negligible performance penalty for both kinds of access.
16:12:53
Bike
i think the extension's compromise is reasonable and doesn't impact performance measurably
16:16:00
beach
It is in fact a great compromise if the sequence functions remain ordinary functions which call the generic version only when the sequence is neither a list nor a vector, which is how it is implemented in SBCL.
16:20:09
Bike
i think it would be interesting if there were better mechanisms for specifying compile time type discrimination, that it could probably be done well, and that the reason such a thing isn't in the standard is just that it would be a pretty complicated project
16:22:55
foom
I do not complain that there exist specialized functions that work only on lists or vectors (like cdr or aref), there's no issue with that.
16:23:19
foom
But, in some cases, functions that work generically on both are missing (e.g. dosequence), or functions that work on both have arbitrarily different behavior (e.g. equal).
16:23:33
beach
foom: So what was it that you wanted when you said "The only other unfortunate part is that CL itself doesn't treat lists and vectors generically."?
16:24:47
Bike
equal is a little bit arbitrary. but, given the rest of the language, including eql, you can write something with different behavior
16:27:08
Bike
i haven't implemented a hash table myself, but i don't think providing those hooks is difficult?
16:27:21
beach
But now we are going to have complaints that it is not in the standard, even though most people are happy to use languages that don't even HAVE a standard.
16:29:32
Bike
but how many of those people are in this channel? at least, i doubt anybody here would prefer there to be no lisp standard to there being a flawed standard
16:32:39
Bike
and there's stuff like CFFI and BT to fill in gaps, there just needs to be enough pressure for it
16:33:04
foom
Yep. As I said at the start: Most people just make do without, because that's good enough.
17:19:57
jackdaniel
Common Lisp is awful, but it's hard to find something less awful, so some may call it a local optimum in programming right now :-)
17:25:45
drmeister
I'd like to put a Cando demo (chemistry + Common Lisp) on a server and let people play with it - but not allow people to run arbitrary code that could bring down the server. Common Lisp has a pretty large attack surface.
17:26:31
whoman
jackdaniel, and i notice too, there *has* to be asymmetry, no matter how perfect symmetry appears
17:27:18
whoman
drmeister, is it possible to have package scoping that denies any reference outside of it? to make a little shell sandbox
17:29:34
Bike
jackdaniel: er, like the read sanitizer thing allows you to reference eval if you obfuscate it like that?
17:30:58
jackdaniel
yeah, it seems the person who wrote it didn't think about reading symbol by custom reader function
17:34:47
dim
drmeister: there's some kind of package scoping in use at https://github.com/dimitri/pgloader/blob/master/src/utils/transforms.lisp for the same reasons as the ones you're mentioning
17:38:09
whoman
perhaps a simple way is to iterate through all the forms and prepend "safe-symbols:" to all symbols =P
17:38:45
drmeister
ACTION is rereading beach's first class global environments paper - this is the way to do it.
18:22:07
jackdaniel
did I get this right? I'm fixing namestring parsing in ECL: https://gist.github.com/dkochmanski/e650126c39af74d1bfb201a680cc0126
18:51:38
pjb
jackdaniel: and to have \ in an org-mode file, you seem to have \\ (cf. to have "" vs. to have "\").
18:53:03
pjb
So: To have “" in the pathname, we’ll have to write “\\”, what is an equivalent of “\” in ordinary string. doesn't make any sense. Perhaps you should use a graphic representation of those strings with a little square per character?
18:53:09
jackdaniel
I'm not sure, here is the original snippet (without rendering: http://hellsgate.pl/files/293c1761)
18:53:56
pjb
To have "\" in the pathname, we'll have to write "\\\\", what is an equivalent of "\\" in ordinary string.
18:54:04
pjb
To have "\\" in the pathname, we'll have to write "\\\\", what is an equivalent of "\\" in ordinary string.
18:55:17
pjb
jackdaniel: I think we should definitely have the same escape syntax in strings and in pathname strings, that is, what is read after #P should be a normal lisp string, then interpreted as a path.
18:57:51
jackdaniel
yes, that's what I've meant in this snippet, maybe I did write it in weird fashion (sorry about that, and thanks for analyzing it)
18:59:42
pjb
Yes, you just missed a \ in "\" which should be "\\" to render correctly. The examples are correct.
19:03:03
pjb
(wild-pathname-p "[wild]foo?bar*") -> T (wild-pathname-p "foo?bar*") -> NIL :-) or something strange like that
19:05:06
pjb
wildcard-word---one or more asterisks, uppercase letters, digits, and hyphens, including at least one asterisk, with no two asterisks adjacent
19:06:10
pjb
Also, posix doesn't have wild paths (it's something that's implemented in shells and applications).
19:08:24
pjb
bash has some extended glob (shopt -s extglob) where you can use ?(…) *(…) +(…) @(…) !(…) with … being a pattern-list, patterns separated by |.
19:09:47
pjb
So "@(foo*(bar)|bar!(quux|foo))" would match foo foobar foobarbar barf barfo barfooo (but not barfoo), etc.
19:11:23
pjb
Of course, if you wanted to use ?*+@!| in the path using those extglob, you'd have to escape them, so "+(\\(|\\))foo+(\\(|\\))" matches ((foo)) or ))foo))))
19:12:41
pjb
So I would propose using either the simple * and ? wild characters, cl-ppcre regexps, or bash extglob, since they are familiar and practical options.
19:35:55
flip214
loading a system that depends on :mcclim causes some of its sources to be loaded repeatedly, starting with Backends/Standard/grafts.lisp.
19:38:21
flip214
; compiling file "...src/McCLIM/Backends/Standard/grafts.lisp" (written 24 APR 2017 12:33:52 PM):
21:30:48
jasom
first complete run of ql2nix results in over 3800 packages: https://paste.pound-python.org/show/XJrapdCPh9Ml58uDG2FW/
23:07:26
whoman
https://ia902603.us.archive.org/30/items/byte-magazine-1979-08/1979_08_BYTE_04-08_LISP.pdf
23:17:58
jasom
axion: also I accidentally forgot to update quicklisp before running, so it was running with september of last-year QL
23:18:26
aeth
whoman: I think the problem with alternate syntax to Lisp is that the various failed proposals (I think M-expressions were the first) were intended for general-purpose computation.
23:18:35
jasom
all packages tha depend on iolib are also missing because the libfixposix in nix was too old; I need to check if it's been updated now
23:19:17
aeth
whoman: Alternate syntax embedded in macros or reader macros or even strings for special cases can definitely work. For macros, see loop. For strings, see cl-ppcre. For reader macros, I'm not sure of any, but there probably are some.
23:19:44
White_Flame
aeth: afaik, m-expressions weren't a post-sexpr alternate, s-expressions were created as an AST structure to back m-expressions
23:21:39
aeth
Well, I can imagine someone making a CAS-like frontend to Common Lisp for some special circumstances. Macsyma/Maxima is sort of an early case, although it probably predates CL implementations and general computers being fast enough to do the whole thing in pure CL.
23:22:01
aeth
Of course, I just do scientific computing directly in s-expressions when I need to, but I'm unusual.
23:22:55
aeth
A good alternative would be a polyglot *CL* where you can nest a rough equivalent to any language you care to use.
23:23:36
aeth
Lots of people have made Python knock-offs and JavaScript knock-offs for their platform. Not trying for a 1:1 implementation would make it easier to basically make it familiar syntax on a fast SBCL core without semantics that deviate too far from CL's speed strengths.
23:25:40
aeth
One alternative to a CLR/.NET or JVM approach is just to implement languages on top of CL. Something for light scripting, something purely functional, something that is static, something that's like a CAS, something SQL-like, etc., all could coexist. CL is just about flexible enough to morph into different things.
23:26:19
aeth
Of course, even if you did that, there's not too much of a reason not to use s-expressions except maybe in the approachable scripting language and the CAS language.
23:27:01
aeth
(In case anyone is confused, I'm talking about computer algebra system when I say "CAS". Wikipedia has about 50,000 entries in the disambiguation page so that won't help you if you don't already know the acronym.)
23:29:19
aeth
What would probably be doable is building a Scheme, a Python-like language, and a JS-like language all on top of a common language core, running in CL. It wouldn't be portable due to a few issues (e.g. floating point NaN/inf/etc.), but could probably be made to run on SBCL, CCL, and ECL.
23:31:09
aeth
Nesting them all in one REPL wouldn't be pretty (if they don't use s-expressions), but once you write the ability to compile strings and files to CL, you also have the ability to use reader macros to do the same thing, assuming you used a character stream approach for all three.
23:32:00
aeth
I already have near-seemless integration between Brainfuck and CL as a proof of concept.
23:32:42
aeth
It becomes more complicated if you have a real language, though, because you'd need to add a lot more syntax and semantics, and you'd have to essentially have a clffi interface between the implemented-in-CL language and CL.
23:35:25
aeth
I suspect that CL also probably would need some extensions to be adopted by popular implementations if it suddenly became a compilation target.
23:39:41
phoe
aeth: I think you can define brainfuck subfunctions though by means of string input and output though
23:40:30
aeth
There are definitely places I could go with cl-brainfuck if I wanted to add features and revise it for 2018-04-01... or I might want to come up with something new for that event, I'm not sure.
23:41:27
White_Flame
(defun-bf q (ptr array) ...) could add the character 'q' to the language, passing in the current pointer
23:42:26
aeth
phoe: The lack of first class environments is probably the #1 issue for the idea of implementing many compile-to-CL (or merely interpreted-on-CL) languages and iteroperating them, afaik.
23:44:17
phoe
He'll most likely think twice if he wants to change the world for the better if aeth goes up to him and says "hey, you've been reading on these first class environments, possibly did some work on implementing them in Clasp/Cando, can you share it with me so I can write a proper Brainfuck compiler?"
23:44:35
aeth
Bike: Yeah, but cl-brainfuck was really a test run for some approaches I could use in cl-scheme, which I will resume work on at some point.
23:44:58
aeth
And environment workarounds for Scheme in CL are not fun, and will probably hurt interoperability and debugging.
23:45:28
phoe
making the smallest and most efficient water filters versus Brainfuck/CL interoperability:
23:46:48
aeth
phoe: Brainfuck is actually a very useful language because it is a tiny, well-defined language. Something like cl-scheme would never actually be a complete project, so it can't actually test features that require a literally complete project.
23:49:02
aeth
pillton: There's the difference between completing something and having something 100% complete. A very tiny language with a very tiny standard could actually be complete. Probably could even be formally proven.
23:50:32
aeth
phoe: Very small programs can be proven that they meet their specification. Something like a CL implementation probably cannot.
23:51:49
aeth
Correct with respect to a specification, if it halts (obviously, the halting problem is a problem).
23:52:35
jasom
ugh; updated nix disallows md5 as a hash for verifying URLs it fetches... time to calculate sha sums of all of quicklisp :(
23:58:22
aeth
jasom: Can axioms really be incorrect, or is it more of an issue of whether or not an axiom is useful?
23:59:42
pillton
There is an example in Mark Tarver's book Logic, Proof and Computation where the proposed logic system had an incorrect axiom.
0:00:25
aeth
Afaik, math is only wrong if there is an error in a proof, or if there's the standard philosophical skepticism arguments of e.g. you could just be dreaming, etc. But other than that, math is pretty certain, and it's really when we try to apply it when things get messy.