freenode/#lisp - IRC Chatlog
Search
7:41:28
|3b|
though now i'm thinking i may want communication in the other direction too, so A can do something with all the parts for which B wasn't called, not sure i can do that even with macroexpand
7:45:57
|3b|
pjb: overall problem: given a spec for a recursive data structure (names + types + offsets in memory), i want to generate code that efficiently writes values into memory corresponding to the names. the spec isn't known until runtime, and may change over time.
7:46:39
|3b|
if a name is given that doesn't exist in the spec, the value can be discarded. if a name in the spec isn't written to, i'd like to be able to write a default value (possibly from the spec)
7:47:19
pjb
Instead of writing something like: (macrolet ((define-foo (name) …) (get-foo (name) …)) (define-foo bar) (define-foo baz) (+ (get-foo bar) (get-foo baz))) write (macrolet ((with-foo (name &body body) `(macrolet ((,(scat 'get- name () …)) ,@body)))) (with-foo bar (with-foo baz (+ (get-bar) (get-baz)))))\
7:47:23
flip214
|3b|: is speed such a concern that some sane representation of the structure, and some "interpreter", isn't fast enough?
7:48:05
|3b|
flip214: yeah, this is happening 10s of thousands of times per second, and needs to leave room for lots of other work
7:48:37
flip214
|3b|: how about the CL-PPCRE approach? this takes some variable input, and build some closure chain from that.
7:49:24
|3b|
ACTION also wants to be able to have type declarations on the source data, so for example it doesn't need to spend time determining each value in a specialized array is a single float
7:51:15
|3b|
ACTION possibly wasn't clear... i have an output spec at runtime, and an input spec at compile time, and runtime compilation to combine the 2
7:51:15
pjb
Now, you could use (eval `(some-dsl-form ,some-data)) but it's clearly inferior to (take-some-run-time-data some-data) ;<- a functional API.
7:51:44
pjb
|3b|: I'm not surprised, since this macro example doesn't match your needs at all. You don't want macros.
7:52:38
|3b|
how do i communicate that parts of it are simple-array :single-float (3), other parts are (unsigned-byte 32), etc?
7:53:30
pjb
(spec (part one (simple-array single-float (3)) :check nil) (part two (unsigned-byte 32) :coerce t))
7:57:21
|3b|
probably not corresponding to the spec data, since they weren't written at the same time
7:57:49
flip214
as one example, "just" build a big "(DEFUN my-interpreter (input) ...)" form, and EVAL it.
8:00:08
pjb
Often, the dispatching is very lightweight, compared to the canned processing, so there's very little gain to generate lisp source and compile it.
8:00:18
|3b|
ACTION already does the job directly, i have something that takes a nested hash table of the names -> data to write
8:01:11
|3b|
but that is slow, since it does hash lookups and pointer calculations for each value, and tends to have unspecialized code so does extra type checks etc
8:01:13
pjb
Well, if it's too slow, then you can try to generate code to do it instead of doing it with those indirections. Be careful to remove the maximum of indirections in the code you generate.
8:02:06
pjb
Yes. You can fetch the data from the table, and compute the pointers in generate, and generate source code that contains only literal and pre-computed values.
8:02:42
pjb
But given that you're doing I/O, it's not sure you will gain anything. Have you profiled the current code?
8:03:52
|3b|
only IO is memory writes, and the profile did show lots of hash table and various sbcl internals from lack of declarations
8:04:26
|3b|
ACTION is stuck on windows with not very good profiler, so not completely sure, but fairly confident that was all the code i'm trying to optimize
8:05:26
|3b|
now i need to be able to specify to the code generator, what data it should be writing and how to access it
8:07:04
pjb
This is something that's so natural to do in lisp, that I must have a dozen of such generators around. But I'd have to locate them…
8:07:59
pjb
This (name type offset) data is the source code for your generator. The lisp source will be the compiled code for that DSL.
8:08:56
|3b|
generating code from the name+type+offset isn't the part i'm having a problem with, it is matching it to the lisp data to be written
8:09:11
pjb
and an interpreter is any function taking some data, and doing something according to that data (plus some other "input" data), while a generator (compiler) is any function that takes that data and generate a function that takes input data and produce the wanted results.
8:09:52
|3b|
so i want to write a DSL that takes names and lisp data, and matches them to the names in the name+type+offset spec
8:11:50
|3b|
or "following applies to struct at location named 'bar" with a set of "write value Y to location 'baz" which applies to 'baz in that struct
8:11:54
pjb
You want lisp code that concisely DOES write the value X to the location named by foo. ie.: (SETF FOO X).
8:13:08
pjb
You don't know anything bout foo at code-writing, since it will be specified at run-time!
8:13:32
|3b|
so i guess more precisely, the code i write would be saying "write X to location named by 'foo if it exists, otherwise write the default value specified for 'foo"
8:15:04
|3b|
then at runtime, when the spec was available, it would be transformed into something like (setf (cffi:mem-ref 123 some-type) x)
8:16:05
|3b|
where X would be a parameter passed to the generated function, or some function of a parameter
8:16:24
|3b|
and i could do things like calling B in a loop to write elements of an array or whatever
8:17:54
|3b|
and A would say "i'm writing to the struct at slot 'bar" or "i'm writing to the array at slot 'bar" (so really 2 different A, though i guess i could figure out which based on the spec)
8:18:51
|3b|
and if there is no slot 'foo or 'bar, no code would be generated. and if there is a slot 'foo2 for which i hadn't specified anything to be written, some default value would be written by the generated code
8:20:21
|3b|
so as far as i can tell, the part processing the "write X to Y" side needs to either be a complete code walker, or have some macros to implement the writing DSL
8:20:51
|3b|
ACTION is probably working too close to my complexity limits though, so other suggestions are welcomed :)
8:29:22
flip214
|3b|: how about writing some lisp code that translates your spec to the structure definitions used with alien code, and then use (slot-value 'foo) etc. to access the data?
8:29:59
|3b|
and as far as performance needs, i want to be able to write millions, probably tens of millions of values per second, while leaving as much time as possible for deciding which values to write and to actually use the data
8:31:27
|3b|
writing to slots that don't exist is allowed, and something happens to slots that aren't written
8:32:07
pjb
Of course, at any time in those compile-* functions, you can have non-literal specifications, ie. names that you will lookup (and call one of those compile-* function recursively).
8:32:23
|3b|
i know the exact offsets already, so specifying those to CFFI so foreign-slot-value can give them back to me is just extra work
8:33:00
pjb
And you could just look how CFFI is implemented and do the same. (Or why not, just call the CFFI internal functions!)
8:37:01
pjb
the defparameter should come below, it was written first, as a specification of the DSL.
8:37:53
|3b|
before running the program, i want to write some code to write values to named locations in memory, where the names are hierarchical, so i might have in C terms, foo.bar[123].baz
8:38:06
pjb
Which is then called at run-time to store the data (here a literal vector, but it could be some other run-time data).
8:38:35
|3b|
at run time, i want to determine the available names/hierarchy and corresponding locations, then turn the previously written code into optimized code to do actual writing
8:39:27
pjb
In compile-store-specification, in the otherwise branch, you will add a (gethash (first specification) *named-types*) and if found, call compile-store-specification recursively on the obtained type value.
8:39:32
|3b|
that set of names/hierarchy might change, in which case i need to recompile the optimized writer function, without modifying the previously written code
8:40:17
pjb
Analyzing the specifications for your named data types, and incorporating in a hash-table, I assume you can do it yourself!
8:40:42
pjb
And again, this is already implemented in CFFI, so I don't understand why you don't just call the CFFI internal functions…
8:41:26
|3b|
my problem is that i want to pass arbitrary data, possibly from multiple sources, possibly not using all of it
8:41:44
pjb
Look, I won't answer anymore question until you provide a 100-page PDF with the specifications of your DSLs.
8:43:11
pjb
In compile-store-specification, in the otherwise branch, you will add a (gethash (first specification) *named-types*) and if found, call compile-store-specification recursively on the obtained type value.
8:44:09
pjb
https://framagit.org/com-informatimago/com-informatimago/blob/master/common-lisp/data-encoding/data-encoding.lisp#L1199
8:45:37
pjb
You said that you already had it: * |3b| already does the job directly, i have something that takes a nested hash table of the names -> data to write
8:47:37
|3b|
the layout specification changes rarely, and compiling a function on change is acceptable
8:51:00
|3b|
so i need to specify and generate some complicated CL data structure, then figure out how to write to /that/ efficiently, instead of just extracting it from whatever it was already stored in?
9:14:12
|3b|
so any transformation of input data (like if i pass a clos instance and want to get the actual value from a slot of a value stored in that instance, or if i want to add a constant to it), would be implemented as more code in compile-store-specification?
9:14:48
|3b|
ACTION took a while to parse the code since it looks like it was working backwards from what i'm trying to do... still not 100%
9:15:41
pjb
of course, as a simple compiler, it has a structure that is very similar to an interpreter (to the classical EVAL function).
9:16:50
pjb
If you have to compute offsets, etc, then there may be an intermediate phase, where you will generate a list of "store operations", and then you will number this list, computing the offsets, before generating the actual operations with literal offsets.
9:18:03
pjb
Or if it stay simple, you may just pass the current offset to the compile-* functions, and let them compute the updated offset for their continuation.
9:18:50
|3b|
can just pretend the destination if a big lisp array, and there is just a mapping of names to indices in that array
9:19:42
pjb
Then now you're free to do whatever you want, to generate whatever code pleases you in this lambda expression.
9:21:41
pjb
You may also have a look at https://groups.google.com/forum/#!msg/comp.lang.lisp/gnGCIasIrKo/o9wfrltLqxsJ https://groups.google.com/forum/#!msg/comp.lang.lisp/uc9fLw7i0Fo/HDoDDhx3DRkJ
9:22:09
pjb
Here, the processing of the data is made not according an external type specification, but using the type of the data itself!
9:22:34
pjb
But the principle is the same: we have a big case/cond dispatching on the type, and then we do what needs to be done specifically for this type of data.
9:23:42
pjb
But it could be defined as a macro generating the code, instead of an interpreter function.
9:30:29
Shinmera
If I wanted to write Scheme I'd just use Scheme, not complain about how CL is not Scheme in #lisp.
9:31:49
pjb
Well, you can also use pseudo-scheme in CL. A r7rs implementation embedded in CL would be welcome too.
9:34:09
larsen
loke: http://repository.readscheme.org/ftp/papers/ai-lab-pubs/AIM-349.pdf although I guess this is not what you were looking for
9:37:13
|3b|
seems like a full DSL would either require constant updating, or end up turing complete (and thus presumably even worse than a code walker)
9:37:42
pjb
|3b|: orthogonal to your question, you can implement the translation of CL source to whatever you want by using a different package than CL. Cf. CL-STEPPER!
9:38:22
pjb
|3b|: and the nice thing is that in CL, any function in CL can be opencoded (and any macro in CL can be a special operator).
9:38:30
|3b|
and being able to work on it interactively like CL code is why i have to deal with it at runtime
9:39:38
pjb
|3b|: so you can define a DEFUN macro (and say, LAMDBA, DEFGENERIC, DEFMETHOD macros), to process those setf and expand them to stores in this vector.
9:49:04
|3b|
B was the (setf (slot-value ...) ..) calls, A was equivalent to binding A and S there, while providing type information to B
9:52:41
|3b|
and another goal is that it could have a (declare (type (simple-array :single-float (*)) z)) declaration somewhere, so the generated function wouldn't need to do any checking on the individual values from Z to decide if it could write them as a float
9:56:10
|3b|
other notable features are that the runtime spec doesn't have any data for 'foo2 in toplevel, or 'c in bar-type, and similarly, the person-written code doesn't write 'baz or 'd, but code is generated for them (and no code is written to write default values for other slots that are then overwritten by user code)
11:14:31
theemacsshibe[m]
What's a good way to save an object? I've just been formatting ~s objects to files and using read to read them again but using format feels a bit odd.
11:17:26
theemacsshibe[m]
Docs suggest that, seems pretty straightforward. I'll install Quicklisp in the morning.
13:36:03
didi
Why does https://github.com/tarballs-are-good/quickutil/blob/5adb3463d99095145325c4013117bd08a8f6cac2/quickutil-utilities/utilities/sequences.lisp#L118 declare FLET functions as dynamic-extend? I only know about dynamic-extend because of Costanza's c.l.l. post.
13:39:40
lieven
and in old speak to state that they're downward funargs. that also enables some optimisations.
13:55:17
didi
stylewarning: I am interested in why did you declare the labels functions as dynamic-extend in `equivalence-classes'.
13:56:39
stylewarning
didi: generally if I don’t want to save the function objects created by labels, I’ll declare them as dynamic extent, so the compiler knows it can allocate and free the memory right away
15:38:59
z3t0
is there a construct in lisp where i can do x and then evaluate if and then do x if the if evaluates to false?
15:39:45
z3t0
here's what i have, http://sprunge.us/eZNY Except I want the input to precede it as well
15:41:03
beach
(progn x (unless condition x))? Is that what you mean. I am having a hard time understanding what you want.
15:41:45
z3t0
okay so essentially I want to loop a request to the user for some input, as long as the input is not valid
15:48:49
z3t0
Just so I understand, the reason for funcall is because I am storing a function inside a variable?
15:56:31
z3t0_
okay so I have (until (funcall x y )) which exits the loop as needed but then how do i return y there?
16:05:30
z3t0_
Any recommendations for parsing floats from strings? There seem to be many different ways it is done?
16:20:48
z3t0
I am trying to write an (if b (something) (else)) but it seems to complain that the form is incorrect
16:25:27
Bicyclidine
that's perfectly helpful, you can see you have :type-converter set up as a keyword parameter
16:31:47
dim
using uiop:run-program :output :string here, is there a way to somehow plug babel or something in there?
16:31:51
didi
I'm back. If anyone want to comment on <https://paste.debian.net/hidden/bcf2c844>, please.
16:40:39
z3t0
can anyone recommend a resource that explains how to use loop but is also easy to approach for beginners
16:54:45
didi
I thought of using `listp' and `vectorp', but then I remembered SBCL has extensible sequence types.
17:02:22
z3t0
I am creating a program in lisp that stores a set of images with latitude and longitude coordinates for the center of the images
17:03:10
z3t0
I need to create a system so that I can store all of the images with lat and lng and then write a function that takes lat and lng and then finds the closest image
17:03:11
_death
stylewarning: uh, no... it is passed to MAP.. but then, there shouldn't be an NREVERSE anyway, unless it's meant to be a destructive operator
17:04:05
didi
_death: I used `nreverse' for performance reasons, because `subseq' creates a new sequence anyway.
17:05:10
stylewarning
didi this isn’t universal among all lisp users but I prefer to prepare my data before executing on it, especially if it contains somewhat complicated logic
17:05:46
stylewarning
Here you have the logic of slicing and reversing, and it’s nice to make that evident before mapping (IMO)
17:08:51
_death
z3t0: if you're using a database, it is also possible that it has a solution for that
17:12:28
_death
z3t0: it seems redis added spatial queries.. guess I'll have to update my lredis library at some point in the future :)
17:35:34
_death
(funcall (if condition #'do-something #'identity) datum) ; if you like convoluted code
17:35:55
oleo
(progn (when (not predicatedp) datum) (dosomething predicatep)) or if you are in an implicit progn would do too
17:37:18
didi
Maybe I should write a macro like (do-when predicatep fn datum). Doesn't look better, tho.
17:38:42
z3t0
I'm trying to understand spatial-trees and I get the idea of it but am a bit confused regarding implementation
17:43:51
z3t0
Can anyone help me understand https://github.com/rpav/spatial-trees/blob/master/tutorial.lisp line 30
17:54:37
z3t0
how do i use the package "rectangle" from https://github.com/rpav/spatial-trees/blob/master/api.org
17:55:03
z3t0
I am using quickload which gives spatial-trees as an object but i cant figure out how to access the rectangles package
17:58:10
Bicyclidine
it looks like there should be a rectangles package as part of the spatial-trees system?
18:05:08
oleo
it informs you of symbol conflicts between the rectangles and the spatial-trees package
18:21:08
oleo
by loading systems you get the packages but there's no way to get the packages alone via asdf
18:21:45
oleo
or at least know which systems contain it as a component and load one of the supersystems