freenode/#lisp - IRC Chatlog
Search
7:17:53
Ober
beach: manardb is pure cl and it's fast as hell. full clos layer on it. leveldb is just a hotness in k/v stores. and the binding in cl is pretty good
8:35:55
dim
hi! I think I will need to upload files to Amazon S3 in a streaming fashion (keeping only a chunk of the file content in memory at a time), and I wonder if cl-s3 or sz3 know how to do that... docs aren't that clear... anyone done that before?
10:39:03
jmercouris
I'm relatively new to working with alists, and I see that getf only returns the first instance matching
10:43:57
jmercouris
here's the data that I am trying to traverse: https://gist.github.com/0d224cebaf779802a099938e7726b253
10:47:34
jdz
A survey has a name, description and fields. Each field would be a subclass of a generic field class that has at least a description field, etc.
11:23:40
_death
jmercouris: there are many ways to skin that cat.. for example http://paste.lisp.org/display/139074 or https://gist.github.com/death/4ae8de1d11202faecd2d634190e53d8c
11:26:53
_death
but.. if you "never traversed a tree composed of lists".. then you should do just that, since it's a very useful skill in lisp
13:45:39
klm2is
Are these equivalent? `(("name" . ,name) ("and" . "more") and (list (cons "name" name) (cons "and" "more"))
14:08:53
White_Flame
klm2is: to be more specific, the 2 resulting shapes are structured exactly the same. The `-based one is tied to source code literals, which make them dangerous/unspecified to mutate later
14:10:24
White_Flame
even though the first cons in the list is most likely going to be new on each invocation
14:15:18
klm2is
but the point is, I was using this in two places and seeing some odd cross-contamination in backquoted forms
15:38:13
White_Flame
remix2000[m]: there are a lot of libraries called "trivial-<x>" which are small portability shims for accessing non-standard but common features
15:39:00
White_Flame
but CL itself is a standard that hasn't changed since its inception, so conformance means strong compatibility within its scope
15:41:34
White_Flame
that could be taken multiple ways, but the CFFI library works across implementations
15:41:54
on_ion
is there a way to ask a particular CL implementation to check conformity to CL for some code ?
15:41:58
White_Flame
when you cross platforms, that all depends on how the C library works per platform, regardless of Lisp
15:43:14
rumbler31
the "not that hard to test" case is to simple compile the concerned code on all available implementations
15:43:45
on_ion
rumbler31: nope im just following the conversation. "jackdaniel | if you write a conforming Common Lisp program then yes" <-- ok so how does one know if their progrm is comforming to Common Lisp? obvious question no ?
15:45:13
White_Flame
some sort of automated test for conformance would get into halting problem territory
15:45:49
rumbler31
implementations conform to the standard. code is simply compiled by the implementation. If you write code where the standard says "the behavior is undefined" then you should expect different behavior on different implementations
15:45:59
Bike
probably not halting problem, except for a few such requirements, like that macroexpansion always terminates
15:46:03
schweers
on_ion: It is not always straightforward to know if you’re writing strictly conforming code, although with time you can gain some experience. For this reason I run my code on at least one other implementation from time to time (mostly running the test cases). Normally I use sbcl, but use CCL for this reason. This has triggered some bugs in my code and made me more aware of what can be non-portable.
15:47:02
rumbler31
I think a better way to phrase the discussion is "how to know how portable your code is across implementations"
15:47:08
White_Flame
Bike: the territory includes "figure out what the range of dynamic behavior is, by looking at the code". Same thing for C, try to write a program to scan code to see if it invokes undefined behavior
15:47:24
on_ion
"different behavior on different implementations" that already sounds like dangerous territory.
15:47:36
rumbler31
obviously, impl specific extensions aren't, and relying on undefined behavior also is a likely source of inspection
15:47:53
sjl
I do something similar to schweers -- develop primarily in SBCL but run my test suites in sbcl+ccl+ecl+abcl
15:49:39
Bike
jackdaniel: i think that's more of a "turn off nonconformant behavior we have by default" flag
15:50:25
schweers
on_ion: one example I ran into the other day: I use a third party library which exposes some classes. The slots of the objects of these classes are filled by reading input files. One of the fields I frequently access in a random order is a list. I replaced it with a simple-array and it was fine. Until I ran the tests on CCL. Turns out, the slot had a type specifier of LIST, and CCL checks for that.
16:52:57
jfrancis
I have a bunch of code that reads and processes files produced by a third party (the FAA, specifically). About a week ago, they started adding a BOM (unicode byte order mark, 0xef 0xbb 0xbf) to the front of every file. I'm pulling my hair out, tried everything I can possibly think of to skip past those first three bytes and read the rest of the file (which is just plain old ASCII). No luck. Anyone dealt with this? Any thoughts? 64-bit SBCL 1.4.5.38-9ee5e08
16:56:04
jfrancis
Perhaps, but if I can, I haven't figured out how. Adding ":foreign-encoding :utf-8" to with-open-file didn't hack it. Although at least the errors changed.
17:04:56
jfrancis
Oops, obviously, I mean :external-format, not :foreign-encoding. Confusing my languages. Using either :latin-1 or :iso8859-1 gives me the error "The bounding indices 40 and 45 are bad for a sequence of length 42."
17:05:46
jfrancis
I could strip the first three characters from each file as a pre-processing step before I feed them to my code, but that's precisely what I'm trying to avoid.
17:06:42
jfrancis
Or I could send a letter to the FAA and ask them to go back to publishing the files the way they've been doing it for ten years up until this Monday. But I doubt that will get much traction.
17:30:44
_death
jfrancis: what happens if you just (read-char stream) after opening the file to skip the BOM
17:43:38
_death
and the sbcl manual mentions a :replacement option that can be supplied to external-format, so that may also work
17:49:19
jfrancis
Sigh. It just seems horribly broken that I can't read a fairly common (though horribly broken) file format without ridiculous gyrations. Sticking a Byte-Order-Marker on the front of a file, despite the inherent evil of it, is common.
17:51:48
ZombieChicken
jfrancis: If it breaks a standard or you can explain how it's broken, then maybe the email makes more sense. Perhaps whoever made that change was just ignorant
17:53:49
jfrancis
ZombieChicken: Here's the problem boiled down to it's essence: How can I read an ASCII file in a portable way in CL that has a BOM on the front of it. Assuming, of course, that I can't mercilessly flog the person who put the BOM there in the first place.
17:55:06
ZombieChicken
jfrancis: I'm aware. I'm also not well acquanted with CL, so I can't offer any real help
17:56:39
_death
if it has a BOM, it's not ASCII.. and if you want to do it portably, you need to use a binary stream and do the decoding yourself
17:57:19
ZombieChicken
But if that appended data is nonstandard, an email and a prayer you get to someone who isn't stupid would be the best option; working around and excepting a broken file format doesn't help anyone
17:58:36
ZombieChicken
_death: Possible to grab a binary stream, read a few bytes, then convert the rest to an ASCII stream?
17:58:37
jfrancis
ZombieChicken: You're not seriously thinking an email to a US Gov't Federal Agency will get them to change a file format, are you? :D
17:58:49
pfdietz
Does sbcl provide restarts for that reader error? Maybe ask for one that would help here.
17:59:29
ZombieChicken
jfrancis: No, I don't expect it to do anything useful unless you're bribing people, but you might get lucky
17:59:35
jfrancis
Yeah, that's what I've resorted to. Opening the file as binary, discarding the first three bytes, then reading in the rest and converting it back to a string. It's unspeakably inefficient. Then again, it only ever has to be done once per file.
18:00:57
_death
ZombieChicken: yes, you can do precisely that with a bivalent stream.. in sbcl you can open a bivalent stream by providing :element-type :default
18:02:38
jfrancis
So I can open a stream, do three (read-byte ), then read the rest as strings? If so, is that portable, or is that an SBCL-ism?
18:04:21
jfrancis
Hmm. Ok. I just got my other code working (that reads it as binary, then converts the byte vector to ASCII). But that code is so embarrassingly bad, I'll go look at flexi-streams. Thanks!
18:15:18
jfrancis
Heh, it runs out of heap when loading the 182meg input file. Guess I need to go figure out the "right" way.
18:28:19
pierpal
182MB is very little data, my toaster has more than that ram. How are you reading those files?
18:40:29
antoszka
shka_: That talk is really depressing. I've seen it a few years ago, it shows how we've been pretty much running circles since the 60's.