freenode/#lisp - IRC Chatlog
Search
15:23:37
pjb
didi: basically, for is not a binding keyword. loop creates all the variables it need before.
15:24:17
pjb
didi: this is also what makes the variable available in the loop epilog in practice (even if using them is not well defined).
15:27:38
pjb
Ober: (defun octet-vector-to-string (os) (prin1-to-string os)) (octet-vector-to-string (coerce #(1 2 3) '(vector (unsigned-byte 8)))) #| --> "#(1 2 3)" |#
15:28:50
pjb
jdz: so it should be at the very least: (map 'string (lambda (x) (or (code-char x) #\?)) codes)
15:29:49
Ober
convert this #(159 31 191 95 191) to "there". leveldb does encoding to vector, but does not do the reverse on fetching
15:30:42
pjb
Oh! (defun convert (x) (when (equalp x #(159 31 191 95 191)) "there")) (convert #(159 31 191 95 191)) #| --> "there" |# works.
15:32:58
beach
Ober: But you never mentioned that your vector contains octets representing UTF-8-encoded characters.
15:33:51
Ober
because that is how it is written. Illegal :UTF-8 character starting at position 0. [Condition of type BABEL-ENCODINGS:INVALID-UTF8-CONTINUATION-BYTE]
15:36:42
Ober
hey pjb, have not see you at the godemiche meetup lately. good to see you still are a member though.
15:37:19
_death
(leveldb:with-open-db (db "/tmp/foo") (leveldb:puts db "hi" "there") (leveldb:gets db "hi")) => "there"
15:43:14
_death
Ober: I think I wrote this while working on some bitcoin code some years ago.. the blockchain was several gigs so didn't work with strings
15:44:09
pjb
Ober: now, look what you've done: you ask a meaningless question, I asked for clarification, you further added overly specific specifications, I provided a faithful implementation, and instead of starting to think, you started to insult.
16:12:17
_death
Ober: they weren't arbitrary objects.. I suppose if you want that you can use a serialization library like cl-store
16:13:58
Ober
yeah with a stream backend of a with-open-vector sort of thing, is what I was thinking
16:14:00
_death
Ober: then you miss some leveldb functionality, however.. for example the fact that keys are ordered
17:40:43
Fare
Xach: 3.3.1 tracks staging dependencies between systems. Its granularity is the file, not the individual top-level expression. Therefore if there are multiple stages in a sin
17:59:50
Fare
simple-date.asd has a completely bogus method at the end that is guaranteed to not do what the author intended and derail asdf
18:01:53
didi
Converting (values) to nil as in (null (values)) => t is a defined behavior? I want to depend on it.
18:07:00
Posterdati
please how can I determine the integer-length of an (unsigned-byte *) array? Thanks
18:12:18
pjb
You could compute: (reduce '+ (make-array (reduce '* (array-dimensions array) :displaced-to array) :key 'integer-length))
18:13:02
pjb
(and (subtypep '(unsigned-byte *) '(integer 0)) (subtypep '(integer 0) '(unsigned-byte *))) #| --> t ; t |#
18:17:51
aeth
Posterdati: If you mean you want to cut up an integer into smaller integers, ldb is one way.
18:20:07
Posterdati
pjb: I do not need how, I need a way to find the element size (in bit) of an array element
18:20:21
aeth
Posterdati: (array-element-type (make-array 42 :element-type '(unsigned-byte 32))) => (UNSIGNED-BYTE 32)
18:20:47
pjb
Posterdati: like: (array-element-type (make-array 1 :element-type '(unsigned-byte 13))) #| --> (unsigned-byte 16) |#
18:21:27
pjb
Posterdati: the upgraded element type designator can be of any form, and if it's T how will you determine it? And again what for?
18:21:31
Xach
Fare: http://report.quicklisp.org/2017-12-11/failure-report.html has other recent issues.
18:21:58
Posterdati
pjb: yes, but on sbcl it worked, not on ecl wich answer like (unsigned ext:byte8)
18:23:28
aeth
Posterdati: The only way to make it not implementation dependent would be to (1) only support some implementations (and avoid clisp, you're doing something numerical) and (2) wrap the parts that don't work with implementation-specific things
18:23:46
Fare
it would only appear to work when you specifically loaded cl-postgres or simple-date as the toplevel request, and not otherwise.
18:24:50
pjb
Posterdati: Right. (reduce 'max (decompose-base 257 256) :key 'integer-length) #| --> 1 |# so you will try to decompose into base 1?
18:26:07
pjb
Posterdati: 101 can be a representation in any base! Perhaps it represents (compose-base '(1 0 1) 1000000) #| --> 1000000000001 |# !
18:32:07
pjb
Posterdati: now, you could use a representation where the first slot of the vector would contain the maximum digit.
18:34:04
pjb
(compose-base (subseq representation 1) (1+ (aref representation 0))) (concatenate 'vector (vector (1- base)) (decompose-base word base))
18:34:07
pjb
(let ((representation (let ((base 256) (word #x55aa)) (concatenate 'vector (vector (1- base)) (decompose-base word base))))) (compose-base (subseq representation 1) (1+ (aref representation 0)))) #| --> 21930 |#
18:35:10
pjb
Posterdati: but obviously, it's not that good a representation because it's O(n) to wrap and unwrap. It would be better to use a structure with two fields.
18:35:34
pjb
Posterdati: but again, this is still silly, because in general you work only with one base, so you just keep it in a global variable.
18:43:53
Fare
I don't know when you found these issues, but the asdf-devel mailing-list is the right place to report them.
18:49:56
Posterdati
pjb: it is not desiderable to determine the size of element based on the maximum element in the array
18:50:57
jackdaniel
and I see such thing in postmodern system (:feature :postmodern-use-mop "closer-mop")
18:51:49
jackdaniel
:if-feature is broken as well for ECL < 16.2.0 (due to bug in ASDF which was fixed after 3.2.0)
18:59:07
Xach
http://report.quicklisp.org/2017-12-11/failure-report/f2cl.html#fishpack looks like it may be an SBCL bug?
19:07:53
Fare
OK, I fixed the build of chanl, too, but not its race conditions (that at least its test suite now find)
19:08:05
pjb
Posterdati: Again, the base is not intrisic. This is an external element, determined by convention or explicitely specified in a protocol. So you need to use an external value bound to a separate parameter.
19:15:47
Fare
Xach: I've sent a mail to asdf-devel looking for candidate maintainers to do more of that work together. If some failures are more urgent than others to address, please warn the mailing-list so we handle them first.
19:16:44
pjb
Posterdati: often, it's even hardwired (since usually you want efficiency and it's specified on paper, so it doesn't changes willy-nilly).
19:17:25
pjb
Posterdati: ie. instead of writing and using the functions I sprung'ed, you could write a macro that would generate efficient base conversion functions for hardwired word sizes.
19:21:12
Fare
Ah, OK. For the record, I don't remember :feature or :if-feature being broken since I made them work in 2.27. They are rather simple features (unlike one of the two ASDF 1 era homonymous "feature" features, which was removed in 2.27)
19:22:07
jackdaniel
but it got fixed after 3.2.0 (maybe unintentionally) so I didn't bother to report
19:23:13
jackdaniel
for normal load it worked fine, but when used with bundling op (either with delivery-op and make-build) it missed dependencies if if-feature was present (disregarding the content)
19:23:51
Fare
jackdaniel, oh. I remember touching that part of the code recently indeed, fixing issues along the way, but the details are out of cache.
19:24:55
Fare
I have a patch in a branch slated for 3.3.2 that fixes bugs that happen when you upgrade from e.g. ASDF 3.1.7
22:50:00
jmercouris
You can use a diffrent type of API search I think to look for regex/lines of code IIRC
22:50:56
Shinmera
fe[nl]ix: You can download all projects to Quicklisp's internal dist folder using: (mapc #'ql-dist:ensure-installed (ql-dist:provided-systems (ql-dist:find-dist "quicklisp")))
22:52:05
jmercouris
Here's what I was talking about: https://help.github.com/articles/searching-code/
22:52:06
fe[nl]ix
is it possible to specify a directory other than the default one ? I don't want to taint my installed systems
22:52:27
jmercouris
I think it only allows searching one repository/organization at a time but there should be a way to do it for all repositories of a specific language or something
22:58:14
Shinmera
I can't see any obvious way otherwise to get it to a specific directory. ql-dist:install has no other arguments.
22:59:33
jmercouris
What do you think a good way of implementing hooks in common lisp is? (something akin to emacs hooks)
22:59:51
jmercouris
from the emacs documentation, it seems that hook is just a list of functions that get called at a given time
22:59:59
jasom
anybody have a good pattern for I want to do N things in parallel, but I must perform a side-effect on the results in-order? Right now I'm doing this, which works but is slightly convoluted: https://github.com/jasom/sqlitestore/blob/master/sqlitestore.lisp#L231
23:00:26
jmercouris
should I have a list called something like (defparameter XY-Hooks ()) and add functions to that list to invoke upon event XY?
23:01:02
jasom
(quick description of how it works: I create a promise for each chunk of work and push the promise onto a queue; I then have a separate task poping from the queue, forcing the promise, and performing the side-effect.
23:01:44
Shinmera
jmercouris: If you store function objects in the list, then when you want to redefine your hook function, it won't be able to find the old one to remove it before adding the new one.
23:02:21
Shinmera
Usually in Lisp you want to have some kind of name->definition mapping to avoid this.
23:03:08
jasom
Shinmera: whether or not they are useufl as standalone functions is orthogonal to whether or not they should be named
23:03:37
jasom
Shinmera: i.e. my routing library gives each route a function name, which aids in debugging for backtraces
23:03:55
jmercouris
I guess one may not be able to remove an item from the hook without a reference to it, but that should be the problem of the user that adds that hook no?
23:03:56
basket
jmercouris: Because then it makes it difficult to change that function later. You have to find it somehow, remove it from the list, and add a new one
23:04:35
jmercouris
basket: Ok yes, in the scenario of (push 'named-func *xy-hooks*), one should be able to remove it though yes?
23:04:36
jasom
jmercouris: pushing an anonymous function makes it hard to redefine; pushing the name of a function makes it trivial.
23:05:25
jasom
and I disagree with Shinmera on the "polluting the function namespace issue" they are functions, they are named, ergo the function namespace is perfectly appropriate
23:05:44
jmercouris
Shinmera: When you say "redefine your hook function" do you mean, "redefine your function that is called by the hook XY"?
23:06:48
Shinmera
jmercouris: Well, when you redefine the hook, to be precise. Defining a hook usually involves creating a new function object to call.
23:06:51
jasom
jmercouris: any time you push an actual function object, e.g. (push (lambda () ...) ...) or (push #'foo () ...) it becomes difficult to locate one of those hooks later should you want to remove it or change its behavior
23:07:17
jasom
jmercouris: if you push the name of the function, you get redefinition for free, and locating it obviously becomes easier too.
23:08:52
Fare
Shinmera, calling asdf:load-system or ql:quickload from within a perform method is a big no-no
23:09:16
jmercouris
Shinmera: I was saying "define a hook" as in the creation of *xy-hook-list* and then (loop for function in *xy-hook-list* do (funcall function))
23:10:25
Shinmera
Fare: I need to do some nasty stuff to get qt-libs to work the way it should. Anyway, as I said, I'll elaborate tomorrow. Too tired now.
23:11:26
jmercouris
I don't really know anything about your issue, I'm just reminiscing on QT quality
23:11:37
Fare
is this related to there being no good way for cffi and asdf to properly declare dependencies on .so 's?
23:11:45
jmercouris
There was nothing wrong with the QT lisp system I was using, it was the QT code itself...
23:12:09
Shinmera
Fare: It's related to me needing to change how another system works, and then optionally needing to load other systems during the loading.
23:19:37
fe[nl]ix
and even one project that copied recursive locks from BT because dependencies are scary :(
23:48:24
fe[nl]ix
Shinmera: I think it would be a good thing, but apparently lots of people use recursive locks
23:50:14
fe[nl]ix
Shinmera: and the only system that uses the broken acquire-recursive-lock is simple-tasks
0:18:16
jasom
_death: because the results (which are going out to a file) could be many gigabytes, unless I'm misunderstanding you what you suggest would keep them all in ram
0:19:00
jasom
ACTION had a patch to BT that errord on acquire-recursive-lock on sbcl, not sure what happened to it
0:19:31
jasom
ACTION also had a patch that implemented proper recursive locks, but that was met with ambivalence
0:22:26
_death
jasom: makes sense, but you could still have syntax similar to the naive approach doing it efficiently
0:31:28
_death
jasom: I'm thinking (dolist (promise (mapcar #'future-it-out chunks)) (write (force promise))), no need to explicitly deal with tasks and queues
0:32:23
jasom
_death: ah, that will create a lot of promises; If we have 100GB of data and the average cunk size is ~60k what's the math on that?
0:36:00
_death
jasom: right, so it could have a mapcar-like operator or some kind of dolist/mapcar operator..
0:39:20
jasom
so the form I would like is (pipeline FUNCTION-THAT-GENERATES-1-WORK-ITEM-PER-CALL FUNCTION-THAT-DOES-WORK FUNCTION-THAT-HANDLES=-RESULT)
0:45:50
jasom
I considered using that plus adding an index, but that's a lot more code than just using a queue
1:07:52
Xach
I think there are huge opportunities for something better than grepping and googling. so specialized.
1:13:02
Xach
what packages does project XYZ define, what package(s) define package ABC, what functions have more than five required arguments, how many functions have &key and &optional, so many queries
1:16:46
Bike
i always thought gigamonkeys' "manifest" system, that displayed a state of the image to a browser, was really neat. i guess this would be like machine readable version. with export
1:52:03
pillton
jasom: Are you expecting the work items generated by FUNCTION-THAT-GENERATES-1-WORK-ITEM-PER-CALL to be independent work items?
1:54:39
pillton
Some "pipelines" can be naturally represented as a tree where the nodes at a specific depth in the tree represent independent work items.
2:02:59
jasom
pillton: the work items are independent, but the generator may be ordered; I don't recall if it is ordered in this specific use-case, but I've had similar cases in the past where it is
2:04:05
jasom
pillton: I think in this use case it is not ordered (i.e. I can generate the Nth item in O(1) time)
2:04:55
jasom
the main thing is I have something that generates data, some work to do, and something that collects the data (which requires it be collected in-order).
2:05:26
jasom
the work is by far the most expensive part, and in this case generating the data in parallel makes no sense, since it would require locking an external resource.
2:16:38
jasom
the other thing that e.g. unix pipelines have that I don't need in this *specific* case, but could use in other cases is backwards propagation of early termination. (i.e. I have "enough" now, so I need to tell the generator to stop)
2:24:15
jasom
in any event, the queue of promises is at least clear and simple enough to reason about that I don't have a big complaint.
2:25:01
eschulte
Is it possible to use ASDF build-op to build multiple binaries from a single package, or do I need to write a separate .asd file for each compiled binary?
2:27:39
jasom
if you use "/" to separate the primary name from the sub-name, then asdf will know which .asd file to look in
2:51:40
eschulte
Fare: I just converted my asd to use multiple defsystems, and that seems to be working well
2:52:43
eschulte
ah, that makes sense. I think the multiple independent defsystems will work well with my current top-level Makefile driver
2:53:27
Fare
of course, ASDF can't do it in a fork by default, since fork is not portable to all systems that sbcl runs on (e.g. Windows), and fork isn't safe if you started threads already.
2:54:30
pillton
eschutle: https://github.com/markcox80/lisp-executable/blob/master/lisp-executable-example.asd#L39
2:57:13
Fare
I haven't handed off anything. There is no new maintainer to replace me. I'm just going to drop it off.
2:57:33
Fare
I mean, rpgoldman is the new maintainer, but he has even less time than I to work on it
2:58:30
Fare
he certainly has made ASDF more conservative and higher quality than if I were still breaking things around
3:00:00
eschulte
that last part sounds pretty good, given that there already appears to be a good deal of functionality
3:00:04
eschulte
I've got to run, I'm excited how easy building packages with asdf has been so far though, many thanks for the tips!