freenode/#lisp - IRC Chatlog
Search
1:02:38
pjb
aeth: I already indicated the right way to do it: you need to have a model of the JSON ontology in lisp (implemented eg. with CLOS classes). Then you can provide niceties such as converting p-list or a-list to and from instances of this JSON model ("DOM"). But at least, the JSON serialization and deserialization between these ontological objects can be done with 100% fidelity and clean roundtrips.
1:03:16
pjb
And if your lisp data is not of the kind expected in an application, you can always instanciate those DOM classes to create or parse exactly the JSON you have to process.
1:08:56
aeth
pjb: I was thinking about something more like this: https://gitlab.com/snippets/1690468
1:10:14
aeth
And for alists and plists, no need for the overhead of CLOS objects, just make the car :json-object-plist or :json-object-alist, which is just a trivial (cons :json-object-plist some-plist) before passing it into the JSON generator. For nested JSON generation it'd be a bit trickier, though.
1:10:25
pjb
The question I ask, is whether a CL hash-table is entirely semantically equivalent to a JSON dictionary?
1:12:07
peterppp
I just played around with this library https://github.com/fukamachi/websocket-driver
1:12:48
peterppp
but apparently the server has no way of finding out which client an incoming message came from
1:13:39
aeth
pjb: My current design is assuming that the CL user is going to try to generate sane JSON and that the JSON is where the potential incompatibilities are. Sanitizing the CL symbols, etc., is going to really hurt the performance, so even in the final thing, I'd leave that off by default.
1:15:06
aeth
pjb: I could replace it with a (format t "~S" string), though, and that will escape things automatically afaik. So then the issue is with edge cases in string escaping.
1:15:44
pjb
aeth: before saying that you will write your own buggy implementation, why don't you read the specification for JSON?
1:16:14
pjb
we already have at least 3 different JSON libraries in quicklisp, I don't think it's worth writing yet another one.
1:16:47
aeth
pjb: The problem is that all of the JSON libraries' defaults are shit. They all fail the nil test, i.e. nil <-> something-in-json is an absurdly bad choice in all of them. And defaults matter.
1:18:31
aeth
pjb: There are almost certainly bugs in all of them if the authors couldn't even be bothered to establish a reasonable cl <-> json mapping in the basic library design, which is the starting point of a JSON library, not some detail to work out later.
1:19:00
pjb
peterppp: this is the clisp function to find the IP of the client connected to a socket.
1:23:04
aeth
pjb: That could perhaps produce a cleaner result, but not a faster result. I need JSON support to read in 3D model formats that at least partially use JSON because JSON's trendy at the moment. So I will sacrifice some good design for performance, which seems to be the norm for game engines.
1:23:53
aeth
My personal design considerations are (1) a better cl <-> json mapping and (2) better performance
1:24:25
peterppp
maybe I can modify the library so that an object representing the socket is passed to the message handler
1:24:30
pjb
Just serialize directly your model, no need to go thru even lisp generic data structures.
1:25:09
aeth
So it'll probably have a bunch of optimizations that only make sense for glTF and any other JSON-based model formats that I happen to add support for later
1:25:40
aeth
At the moment, I'm going to handle .obj models and glTF models for import/export, in addition to a binary internal 3D format.
1:26:03
aeth
For textures, I'm just going to handle PNG because there's a good library for that and I don't have to write my own.
1:26:48
pjb
aeth: but if you write floating points, take into account the format specified for json floating point numbers!
1:26:56
aeth
The only other media format I think I'll have to support is sound... and I'm not going to even deal with sound yet. I'll probably need to support Opus and OGG Theora and maybe flac.
1:27:26
aeth
And the only other data format I'm going to mess with is an s-expressions-as-data format, which is fairly easy.
1:28:42
aeth
whoman: (1) their defaults are bad, (2) they usually focus on flexibility over performance
1:29:19
aeth
whoman: I want reasonable defaults and if I control the library, I can cheat and patch the library to meet all of the performance requirements necessary for the particular glTF JSON that I will be dealing with
1:29:59
aeth
And I really hope that between OBJ and glTF I don't have to support any other import/export for models... because there are so many 3D model portability formats. More than a dozen.
1:30:46
aeth
The older ones tend to use XML because that was trendy at the time. The newer ones tend to use JSON because that's trendy now.
1:32:27
whoman
used to basically design software in that syntax anyhow back in the day in C/c++/objc lands
1:33:03
whoman
ACTION loves NSDictionary also. which is also QuakeEd level editor map format and original DoomEd BSP format ........ so maybe a bit biased
1:35:05
aeth
whoman: An id tech 3 (or perhaps earlier) asset importer would be cool. There are a lot of cool maps, etc., in id tech 3. That would be a much lower priority thing to do.
1:39:49
aeth
pjb: Does JSON require a more formal specification than the diagrams on its website? I've always just used the diagrams. https://json.org/
1:41:26
aeth
ah, I see, there's an actual PDF. https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
1:43:07
pjb
aeth: the diagrams are formal enough for the syntax; remains some elements of the semantics, but since it's a data representation language, there's not too much of it. (like meaning of order in various places, what about duplicates, etc)
1:44:39
|3b|
sabrac: looks like it is just data files from unicode.org with an asd file so ql can download it. http://cldr.unicode.org/
1:46:27
aeth
So what I'm probably going to do is support: data s-expressions (.sxp), json (going to write this with the glTF support in mind), and maybe yaml. For 2D, just PNG for now (fortunately, I don't need to write this, at least for reading PNG files). For 3D, OBJ and glTF (requires JSON). For audio, probably Opus and Ogg Theora and *maybe* Flac. And everything else will be internal custom formats.
1:47:04
sabrac
|3b| yes, that is what it looks like. Just wondering who did it. I like to give credit where credit is due.
4:29:15
dmiles
whould this be adequate semantically? (defmacro pushnew (item place &rest rest) `(unless (member ,item ,place ,@rest) (push ,item ,place)))
4:30:27
dmiles
i am looking at where everyone else passes in an environment.. that is required isnt it?
4:47:23
Bike
"Can't specify the actual keywords [in the macro lambda list] since, apparently, non-constant keywords should be accepted." is an interesting if only tangentially related comment
4:54:17
dmiles
what makes me slightly crazy is that the package that keys go into or come from is so random
4:59:01
dmiles
i kinda liek how specializations happen in SICL .. i might end up using that simular trick
5:12:01
dmiles
i keep meaning to ask.. is there ever a time that "(eval-when (:compile-toplevel :load-toplevel :execute) t)" would not be t?
5:20:32
pjb
if it's a toplevel form then it always return T, but since it's a toplevel form, this T is just "compiled-in".
5:22:34
dmiles
ok so when inside a defun its compiled-in as t (since it contains :execute) i can get rid of eval-when wrapper?
5:23:02
dmiles
more like: ok so when inside a defun its compiled-in as t. Next since it contains :execute, i can get rid of eval-when wrapper?
5:24:02
dmiles
oh i was reading it as ? (eval-when (:compile-toplevel :load-toplevel :execute) (eval-when (:execute) t))
5:26:10
pjb
If you compile-file or load the compiled file then a toplevel (eval-when (:compile-toplevel :load-toplevel :execute) t) returns t (but as I said, it wouldn't matter); If you load the source file, then it would return t too. If it wasn't a toplevel form, then in the first two cases, it would return nil.
5:27:38
dmiles
i'll have to test the: (defun is-it-t () (eval-when (:compile-toplevel :load-toplevel :execute) t))
5:29:14
Bike
(defmacro eval-when ((&rest when) &body body) (if (or (member 'eval when) (member ':execute when)) `(progn ,@body) nil)) for non top level
6:50:38
beach
Is it possible to implement the #= and ## reader macros portably? For standard objects, it is possible to use the MOP to traverse them, but what about structs?
6:54:50
Zhivago
Wouldn't those take effect at the surface of the syntax of the constructing expression, as they are reader macros?
6:55:28
cryptomarauder
when you're far from your loved ones, especially your children, you'll not preocupy about priviledge so much in those moments. Rather christmas becomes something that you wish you could enjoy simply because it's an excuse to step out of the drudgery of apathy and disconnect we generally share most days. To me it's a good thing to practice even if you don't consider yourself a religious person. I for one am not religious.
6:55:29
cryptomarauder
I just say merry christmas becasue I hope you guys don't have to live far from your loved ones like I do. Just this.
6:56:07
beach
Zhivago: Consider #1=#(1 2 #1#). When you build the vector, you have to stick in a temporary object that you then fix up later, once the vector is built.
6:59:53
beach
I guess the same problem happens when you have to deal with instances of custom subclasses of CLASS.
7:04:11
beach
I guess for maximum portability, I could write the fixup routine as a generic function, and have the implementation specialize on their own classes.
7:05:17
Zhivago
If you can change the evaluation semantics to include proxies it should be straight-forward.
7:08:31
Zhivago
crypto: Fair enough. My recommendation would be to just skip the drudgery bit in the first place.
7:10:04
Zhivago
Ah, it has an opt-out clause -- but other than that it looks like it has the same sub-structural issues.
9:51:57
ym
shka, do you see how optimized ring buffers can be implemented on top of existing standard?
9:53:44
ym
I can see how it can work as sbcl extension, but on top of existing vectors, for example, this won't be nor portable, nor optimized.
9:59:34
shka
ym: more importantly: there is NOTHING that prevents you from writing optimized ring buffers in common lisp
10:15:16
ym
I got what bothered me. I want lisp machine reimplemented for FPGA board and able to self-modification. In that way I could build "hardware"-optimized ring buffers.
10:20:19
FareTower
ym: with two FPGA chips, you could have a duo-processor where one updates the other at a time.
10:22:45
ym
Or many such machines could be implemented with 9p support and distribute computing power via network for example.
10:23:00
FareTower
ym: for extra fun, read my essay "who (p)owns your computer" http://fare.tunes.org/computing/reclaim_your_computer.html
10:45:19
Shinmera
For those with a raspberry pi, here's a preview build of Portacle: https://twitter.com/Shinmera/status/945243615021977600
11:46:08
shka
i actually starting to write my own implementation of double ended queue before i reminded myself that there is very decent implementation just lying around
12:29:24
loke
shka: If you need a red-black tree, watch out though. Out of the ones I tried (3 different ones?) none of them were correct.
12:29:49
loke
I ported one myself from a reference implementation, and for the longest time it too had a bug (typo on my part), but now I think it's stable.
12:31:08
TMA
red-black trees are notorious for being easy to get wrong, especially when deletion is involved
12:33:49
loke
TMA: True. I was runnin gbroken implementations for months without really noticing the issue, other than random errors at times that I attributed to a broken red-black implementation. Finally, I added code to record every single addition and removal to the tree and stored it in a log so that I could replay it to prove there was a problem.
12:34:58
loke
I then utterly failed to fix it, because I didn't understand the implementation. So, I deicded to port my own, it worked... Until it didn't. Rinse and repeat and this time I was abel to fix the bug, which was a simple transcription error.
12:35:22
loke
But now it's run for well over a year without a single problem, on a heavily modified tree, so I think it works.
12:35:59
loke
Here's my implementation by the way: https://github.com/lokedhs/containers/blob/master/src/rbtree.lisp
12:36:35
loke
If anyone wants to adapt it into their own library so it can make its way to QL, I'd be pleased.
12:37:39
makomo
when you redefine a function that was previously loaded using Quicklisp in SLIME, how does it know which package it came from (and which definition to update) if you're not in that particular package (i.e. the current package is CL-USER)?
12:38:23
loke
makomo: SBCL tracks where a function was defined, and SLIME (SWANK, actually) uses that information for code navigation.
12:39:01
makomo
loke: hm yeah, but how does it know i don't want to maybe make a new function with the same name but in a different package
12:39:17
TMA
My RB-tree was ok until deletion, after spending a whole day debugging that I have made an AVL tree instead in about an hour
12:44:00
loke
TMA: I'd like mine to become available on QL. It's currently part of my own container library, but I'm considering extracting it out into its own library.