freenode/#lisp - IRC Chatlog
Search
13:01:03
phoe
Techniques and Utilities for Farming Objects On The Net - https://www.youtube.com/watch?v=iXov5ukVVWI
13:13:32
dim
I find myself needed to pseudo-parse SQL in different contexts, from parsing PL/SQL; so sometimes it's just a query, sometimes it's a query in parentheses, and sometimes it finishes with a semicolon, but other times not, and deciding when to stop consuming what looks like part of the SQL query seems impossible without actually parsing SQL queries
13:13:51
jackdaniel
someone other than you had asked me about that, I'd say "maybe dim knows something" :-)
13:14:08
dim
current problem is that I'm consuming a closing paren that is part of the surrounding context rather than the query
13:14:59
jackdaniel
you need to introduce different terms for each context, otherwise it will get messy I think
13:15:25
dim
I believe parsing balanced parens is only possible if you know when to parse a literal or identifier expression that contains a paren (select 'foo' as "("; is valid SQL) ; but for that you need to know where to expect identifiers and literals, and then how to do that without writing the full blown SQL parser?!
13:16:09
dim
jackdaniel: I'm back to playing with https://github.com/dimitri/plconvert ; being given a nice example of an Oracle package to parse
13:16:32
dim
and the current parsing fails in many ways (mainly because the parser had only seen a limited amount of Oracle PL/SQL code before)
13:17:05
dim
I know that at some point I will have to parse Oracle SQL statements in full and a compiler that rewrites them in SQL for Postgres, I didn't want *now* to be that time
13:17:43
dim
oh yeah, I know about that, it's worse than just $$, it's $x$ ... $x$ and you can name them the way you want and nest them, pgloader knows how to do that for SQL files parsing
13:18:46
phoe
I remember a postmodern bug related to that, https://github.com/marijnh/Postmodern/issues/222
13:19:09
dim
I'm trying to think of a trick that allows me to parse SQL and stop before consuming that closing paren at the end of it, without having to recognise SQL in full...
13:19:33
dim
phoe: yeah see https://github.com/dimitri/plconvert : compiler from Oracle PL/SQL to Postgres PL/pgSQL
13:20:04
dim
at work I have colleagues who have to assist Oracle migrations to Postgres and handling huge PL code bases is part of the job
13:20:31
dim
I have this idea of writing a compiler for it, had it from a long time, so I have a good opportunity to make some progress now
13:21:01
dim
jackdaniel: they have online documentation but well, I don't like using it, it's too big for its own good, I didn't find a comprehensive grammar
13:22:33
jackdaniel
here is an example of c18 syntax reader: https://gitlab.common-lisp.net/ecl/ecrepl/-/blob/master/c18-syntax-reader.lisp scymtym improved it and wrote a preprocessor; so you probably should look in his repositories for more elaborate inspiration
13:22:35
dim
maybe I need to write a SQL “skeleton” parser now, that recognize the structure (select <list of fields> from <list of source relations> where <expression> group by <list ...> order by <list> etc etc
13:23:19
dim
I might not have to dive into each level deep, but without that how can I decide that the last closing paren is part of the SQL or part of the surrounding FOR ... IN (<sql here>) LOOP ... END LOOP; statement?
13:24:05
dim
jackdaniel: if you would click on the plconvert link you will see a quite advanced PL/SQL parser written in esrap already
13:24:27
dim
jackdaniel: parens may be used inside literals or identifiers, and then you want to skip them
13:24:57
dim
so can I implement parens counting in a simple way without writing a full parser that knows where to expect literals and identifiers? ;-)
13:27:07
dim
(my esrap fu isn't great by the way, so the existing PL/SQL parser isn't as good as it could be, but well, I spend like 3 days on it IIRC)
13:31:32
dim
didn't want today to be that day, but well, as much as I'm trying to avoid it, it seems I don't have that much of a choice
13:32:02
dim
who would fancy writing that and putting the code in https://github.com/dimitri/plconvert/blob/master/src/parser/query.lisp ?
13:32:45
dim
the next very fun step in plconvert is to use our awesome Common Lisp pretty printing technology to output PLpgSQL code from the output of the parser/compiler, and that's easy, it's just a tree of structure instances ;-)
14:23:48
ralt
I just came across this wasm sample https://github.com/mdn/webassembly-examples/blob/master/js-api-examples/memory.wat
14:32:55
jackdaniel
if you are into wasm, this description seems nice https://evilmartians.com/chronicles/hands-on-webassembly-try-the-basics
16:36:33
jasom
dim: I'm booked for the next week, but it should be pretty straightforward if I get some free time in September. IIRC Oracle has very nice state-machine diagrams in their documentatin that translate straight to code
16:37:30
jasom
esrap is nice, as long as you only ever need to parse strings. I've been known to just write recursive-descent parsers when I need to parse other things
16:40:20
jasom
I'm also curious if asarch was actually referencing https://sourceforge.net/projects/cprime/ or if the "'" was a typo
18:27:13
VincentVega
Is there a function which returns true on a proper list but not on a quoted thing (e.g. ''sym)? Instead of (and (consp x) (not (equal 'quote (car x))))
18:32:55
VincentVega
copec: actually, I was wring a macro for a cond which prepends a function call on each condition (unless t or otherwise) and, if the car of the clause is a list, prepends a call to each of the element in the list.
18:34:38
copec
Ah, I was wondering why you would need that function you asked about, that makes sense
18:36:42
VincentVega
Something like (cond-fn fn ('(x y) z)) -> (cond (((or (fn x) (fn y)) z) which makes me think why don't I just generate mapcar into the resulting expansion instead of doing it beforhand
18:38:26
White_Flame
I would use a keyword (cond-fn fn ((:special x y) x) ((normal-func param1 param2) ...))
18:40:47
phoe
(alexandria:switch ("foo" :test #'string= :key #'string-upcase) ("FOO" :good) (t :bad))
19:14:09
seok
so how many people actually use iterate or https://github.com/Shinmera/for/ instead of loop?
19:18:09
wsinatra
just good old fashioned loop here, for does look interesting though, a little easier to remember at a glance
19:19:10
White_Flame
I've built up some individual tools for iterating over structures in various ways, but I use loop or recursion otherwise
19:20:15
sjl_
I use loop in my libraries to minimize dependencies, iterate in my personal projects where I don't care as much
19:22:28
dim
jasom: thanks for your message! I'm getting started on transforming that state machine into an esrap rule set, but I will only get an hour here and there, so you might still be able to help when you get around to it ;-) also I'm only going for a partial parser at the moment, just to play with some files I've been given
19:40:30
dim
the SQL compiler is going to be most interesting that said! do you like working on compilers?
19:41:11
jasom
I do, but need permission from work before working on compilers just because my work makes compilers. For something like SQL, it will probably be zero issue to get a sign-off.
19:44:41
VincentVega
seok: I switched to iterate when I had to run through a queue. loop was ok, but (generating node next (or (cl-heap:dequeue q) (terminate))) (while (next node)) was better. Custom clauses, too. I am digging it so far.
19:47:30
dim
jasom: it's a compiler from Oracle PL/SQL to Postgres PL/pgSQL, and if we're feeling very brave including Oracle SQL into Postgres SQL
19:48:14
dim
and then if people/contributor have a passion for the result we can envision a rewriting proxy for migration environments
19:49:39
jasom
That would actually be useful at work; we have an oracle DB we are hoping to transition to pg. I still need to learn some more pg DBA stuff though like how backups and point-in-time restoration work.
20:15:34
VincentVega
Guys, given a type specifier (retrieved from a slot definition in my case), how would I say if its a (simple-array string) or a (simple-array bool)? The closest thing I found was array-element-type, but I have a specifier, not an object.
20:18:11
VincentVega
Bike: (and (subtypep '(simple-array boolean) '(simple-array string)) (subtypep '(simple-array string) '(simple-array boolean)))returns T, T
20:18:43
Bike
(upgraded-array-element-type 'string) => T on your implementation (and any real implementation), and ditto with boolean
20:19:11
Bike
array element types are about the storage format. they're not a declaration of the kinds of values it ends up holding. at least in most contexts.
20:19:17
VincentVega
Oh. Is there no way to get the exact specializer then, the one with which it was initially declared?
20:19:46
Bike
if your type specifier is a list (SIMPLE-ARRAY something) you can just do (second ts), of course.
20:20:39
White_Flame
if you used bitvectors instead of arrays of bools, you'll both save space and be more type distinguished
20:22:30
VincentVega
I don't necessarily need bitvectors, I just wanted to do different things based on different type spec in the slot. It's not a big problem though.
20:23:07
Bike
in general, since lisp is gradually and dynamically typed, having different behaviors based on declared types doesn't usually work
20:29:22
jasom
I'm always late to the party, but etypecase is usually good for dispatching on type as well VincentVega
20:30:09
VincentVega
Bike: And what if I needed this facility only at compile time, would it make any difference? Sorry, just have to make sure : )
20:30:51
Bike
subtypep is always going to give the same results for those inputs, if that's what you mean
20:31:31
jasom
VincentVega: I don't think there's a good way other than just destructuring the type specifier yourself
20:36:00
VincentVega
jasom: Oh well, can't have it all : ) Anyway, it's not such a big problem, rather just a temporary nuisance. Thanks, people!
20:37:19
jasom
there are definitely non-portable ways though, and someone may have made a portability shim for it.
20:38:41
phoe
otherwise, https://github.com/cosmos72/cl-parametric-types/blob/master/lang/typexpand.lisp#L24-L38
20:52:51
kilimanjaro
what are the common alternatives to SXHASH? i would like SOME-FN which roughly satisfies (= (some-fn x) (some-fn y)) == (equalp x y)
20:54:40
kilimanjaro
on SBCL at least, SXHASH seems to be optimized for performance and so will cause collisions for objects which a hash can't be computed quickly (e.g. vectors all seem to hash to the same value)
20:59:05
White_Flame
kilimanjaro: you could test for types that you want to be more specific on, and fall back to sxhash for the rest
21:02:12
Xach
phoe: Unable to load any of the alternatives: ("libmixed.dylib" "libmixed.so" "mac64-libmixed.dylib")
21:03:55
Xach
phoe: but i'm not sure why. i thought maybe the working mac version would add some insight.
21:13:11
kilimanjaro
White_Flame: i guess for a slow but reasonably general solution, I could serialize the object to a stream using cl-store or similar, and then compute a digest (e.g. md5) using ironclad
21:15:18
White_Flame
so I presume you're doing tons of comparisons, where the hash cost is amortized well?
21:15:31
Xach
ha! on the mac it fails because I was in a directory that had been deleted from under me, so the underlying error was with getcwd.
21:20:30
White_Flame
another goofy idea, especially for the vectors, is to hash the first N bytes of the datastructure memory being referenced
21:40:21
Josh_2
added dedicated threads for accepting and processing connections on my HTTP server and It's just slowed it down
21:42:33
Alfr_
kilimanjaro, probably won't work: (equalp 1d0 1) but the hashes for these numbers printed most likely will differ, similar problem (equalp #\a #\A) is true.
21:45:34
phoe
I mean, does the dlopen syscall provide any information about what exactly blew up, or does it just set errno/retval to EIMDED and the programmer gotta figure out what went wrong?
21:49:00
fe[nl]ix
Shinmera probably compiled that on a very recent distribution, and Xach is running an old(er) one
21:49:32
fe[nl]ix
for the same reason the officially distributed SBCL x86-64 binaries don't run on older distros
21:50:58
fe[nl]ix
one lesson here could be that if you want to distribute binaries, better compile on a distro from 2-3 years ago like Ubuntu LTS or Debian stable/oldstable
21:52:32
mfiano
Xach: It loads for me on SBCL 2.0.7, both dist version and HEAD, though has some warnings, and not just STYLE-WARNING, so probably needs to be fixed before inclusion anyway
22:19:08
jasom
kilimanjaro: the only good way I can think is to come up with a canonical serialization and hash that serialization. equalp has so many special-cases
22:20:38
jasom
kilimanjaro: also note that equalp is potentially non-portabe with non-simple characters; I don't know what actual implementations do, but the spec leaves it up to the implementation
22:21:30
jasom
I can think of reasonable implementations of equalp that would have differing behavior based upon the current locale, for example.
22:59:41
aeth
White_Flame: probably because the most generic thing is case insensitive equality, given that there are case-insensitive predicates in the language.
23:00:23
White_Flame
but to have that wrapped into the only facility which steps structure slots & makes it much less useful
23:00:30
aeth
By naming conventions of case-sensitivity vs. case-insensitivity, the case-sensitive version of EQUALP would be called EQUAL=
23:01:58
White_Flame
string= using the same stuff as char= which is the numeric equality of char codes
23:02:54
aeth
Yes, it wouldn't necessarily be EQUAL=, but there still would be an issue with naming it. I could definitely imagine postponing thinking about it for the next edition of the standard... which obviously never happened.
23:03:29
aeth
This is, of course, about why it's logical, not about what actually happened. You'd have to ask someone who was involved, if there's no written record of it.
23:03:35
White_Flame
but still, case-insensitive string comparison is not a deeper structural equality test; it's orthogonal