freenode/#lisp - IRC Chatlog
Search
8:58:38
earl-ducaine
Does there seem to be a generally prefered thead portability library? I was looking at Bordeaux Threads but I'm worried the API (at least the ql) seems to have changed quite a bit.
9:00:11
|3b|
https://trac.common-lisp.net/bordeaux-threads/wiki/ApiDocumentation appears to be current docs
9:02:29
beach
Yes, except that address calculations are exposed and the objects manipulated may be machine data.
9:03:10
|3b|
so i can just start working on codegen from the HIR, since i probably wouldn't be changing much going to MIR anyway
9:04:35
beach
|3b|: Depending on the kind of HIR you generate, you will need to generate code for array referencing, CONS-cell referencing, standard-object referencing, perhaps fixnum arithmetic, etc.
9:06:36
|3b|
ACTION 's current test function needs to pass function arguments and a literal int to 2 native functions
9:07:36
|3b|
then i need to figure out how to specify that i'm compiling the function for the native calling convention, and assemble it into something i can put in an apk file
9:08:27
|3b|
(specifying the calling convention is one of the things where dropping user/implementation declarations came up)
9:09:41
|3b|
actually, i guess i can walk up the tree of environments to find it, so maybe dropping it isn't a problem after all
9:13:37
|3b|
and possibly using specialized arrays. i don't remember specifically which things that was for, but usually when i'm optimizing i want specialized arrays anyway :)
9:14:46
|3b|
(be sure to validate the declarations first if running on other lisps or running at safety 0)
9:17:43
|3b|
hmm, i guess i have one more step at IR level, need to distinguish whether i'm compiling for native or lisp
9:20:39
|3b|
anyone have thoughts on how best to represent java's super keyword in a lisp style? (super (foo this ...))? ((super foo) this ...)? (foo (super this) ..)? (funcall-super 'foo this ...)?
9:22:11
|3b|
(if i understand correctly, super is sort of generalized call-next-method, that lets you call any method or access a slot starting dispatch with superclass instead of actual class)
9:23:08
|3b|
ACTION currently is thinking of (foo (super this) ...) as best representing what it does
9:24:20
|3b|
though you could sort of generalize it to multiple, grab classes for current method, find applicable methods for requested function, remove first entry if exact match for current method
9:24:34
TMA
|3b|: is this special? [not in (declare (special this)) but extraordinary, separately handled etc.]
9:25:15
|3b|
yeah, this is for compiling code to interact with java APIs, so i need to subclass java classes, and call methods from superclass, etc
9:25:56
|3b|
so first need to distinguish it from an intentional recursive call, and 2nd need to generate different bytecode
9:27:00
|3b|
well, currently THIS is an explicit argument in my lispy code, so generating SUPER sounds a bit off
9:27:13
TMA
not (super this) ... the latter would seem to enable you to use (super foo) for any foo, which is not the case from your description anyway
9:32:04
|3b|
ACTION isn't sure only working on some things is too much of a problem, same way you can't call call-next-method on aything that would change applicable methods
9:32:45
|3b|
also seems like it would be fairly rare, since most places could just use call-next-method
9:37:24
TMA
using super for other purposes than the equivalent of call-next-method is considered bad style even in java itself
9:40:15
TMA
[the usage super(argument1, argument2); in constructors is a (call-next-method) of sorts albeit you can choose which superclass constructor you use]
9:46:31
shrdlu68
Spec says "vector is analogous to list", so is (aref <vector> n) the same as (nth n <list>)?
9:50:31
beach
|3b|: You know that the environment is gone by the time you have an AST, so you might need to define a new AST type as well.
9:51:36
|3b|
hard to tell what needs stored where until i try to use it, so probably will be a few passes of this sort of thing
9:59:42
pjb
shrdlu68: aref is not the same as nth: (nth 4 '(1)) #| --> nil |# (aref #(1) 4) #| ERROR: Array index 4 out of bounds for #(1) . |#
9:59:58
pjb
shrdlu68: elt is like aref: (elt '(1) 4) #| ERROR: 4 is not a valid sequence index for (1) |#
10:00:30
pjb
The older NTH is nice and gentle. It doesn't rely on a condition system, it just returns NIL.
10:01:23
pjb
The newer AREF and ELT are harsh and strict. They rely on the condition system, and like to dominate and control you.
10:12:18
adlai
no-defun-allowed: "a subset", sure, there are many trivial subsets (eg, bit vector operations, "logical" integer operations, etc). the interesting question is hunting for maximal feasible subsets.
10:29:33
|3b|
hmm, maybe it does eat the declarations in a way i can't get by walking up the env list
12:31:42
hjudt
is there any lisp project that has a docker file i can look into? i'd like to deploy a project to an openshift cluster.
12:56:38
dim
hjudt: https://github.com/dimitri/pgloader has a dockerfile and is available on docker hub too, if you want to have a look
12:57:32
dim
it was contributed a nice spec to have a builder image different from the runtime image, wherein there's just the lisp image itself and its .so dependencies
14:17:26
phoe
basically - socket operations are *not* thread-safe, and performing them at once from different threads causes things to explode.
14:19:33
phadthai
hmm yes if they're thread-safe or not depends on OS, thread implemenation and polling method (signals may also be sent to the whole process, handled by whatever thread is next scheduled)
14:27:56
phadthai
so usually we can disable signals we don't want in threads we don't want them (usually using pthread_sigmask), configure sockets to not send signals like sigint and block/unblock status (via setsockopt), use custom locks/mutexes around syscalls like recv/accept, etc...
16:29:02
Demosthenex
in case anyone is interested, humble bundle's doing a coder book bundle atm which includes land of lisp (ebooks). https://www.humblebundle.com/books/learn-you-some-code-books
17:08:17
cgay
I wonder where Land of Lisp's summary came from. "...but it's cryptic syntax...." Way to sell books, guys.
17:13:46
aeth
I recently looked through the book (it was part of an ebook bundle a long time ago) and it didn't look as bad as some people said. It moves pretty slowly, though. Takes a while to get to useful features of the language.
17:14:37
aeth
It's dated, though. It recommends CLISP, whereas these days people generally recommend SBCL. And it mentions Clojure and Arc as up-and-coming Lisps.
17:28:27
russellw
What would be the most idiomatic way to order parts of a name? That is, in Scheme, there are lots of names like string->number. In my opinion, it would arguably make more sense to write such names like number-string or number-of-string when we are dealing with pure functions, because the function name is written before the argument, but string->number is the Scheme idiom. What is the idiom in
17:31:27
Bike
so e.g. the thing to get a string out of a symbol or character is just called 'string'
17:32:26
russellw
right, which is why the distinction does not show up in the standard library. But as far as you are concerned, the idiom is from->to, not to-of-from?
17:33:26
Bike
i'm mostly talking about my preference. i don't know that it's something there's really an idiom for.
17:35:19
russellw
the specific use case that prompted the question is a pair of test functions, one of which tries reading and then writing, the other of which tries writing then reading
17:36:25
pjb
russellw: indeed, number<-string in scheme, number-from-string in CL would lead to more readable code. But still, left-to-right transformations are more common.
17:40:42
pfdietz
For accessors of structure types, the common part is a prefix. (defstruct human head arms torso legs) gives human-head, human-arms, etc.
17:41:45
pjb
(sign (hand-of-arm (arm-of-human (human-ceo-of-enterprise 'apple))) (employment-contract *me*))
17:42:37
pjb
the -of- or -from- or <- connectors leads to more readable code than the -to- or -> or just - connectors.
17:42:59
pjb
Notice on the other hand, that in CL, you can specify the prefix in defstruct, and you can use foo. instead of foo-
17:44:04
russellw
and I think given that the idiom is as it is, it is probably better if I follow it rather than doing things a different way
17:44:23
pfdietz
The alternate style is that of generic function accessors for standard classes. These typically omit the prefix entirely. arms, head, etc.
17:51:26
shrdlu68
I have a binary tree data structure such that each node consists of 'left-child, 'right-child, and 'data (which is some arbitrary object stored at the node). I'm currently using a vector to hold the values, initialized as (vector nil nil nil). What's a better alternative, performace-wise?
17:53:20
russellw
I would expect on theoretical grounds a structure should be slightly faster because the bounds check can be omitted, but the actual performance difference too small to measure
17:54:47
pjb
russellw: actually, I don't think structures can do without the bound check: (defstruct 2d-point x y) (defstruct (3d-point (:include 2d-point)) z) (3d-point-z (make-2d-point)) vs. (2d-point-y (make-3d-point))
17:55:06
shrdlu68
The results of profiling are: https://gist.github.com/shrdlu68/081a78c6196b676395ca518a268d9119
17:56:40
pjb
But you should realize that it's silly to opimize the access to the node slots of a binary tree, when the access to the tree is a slow O(logn), when you could have O(1) with a hash-table!?!
18:04:09
shrdlu68
The keys are bit vectors of length 1-240. I just walk the bit vector going left or right.
18:05:12
russellw
if all you want to do is lookup, I would expect a hash table to be both faster and smaller
18:10:45
shrdlu68
I tried using an octet vector rather than a bit vector, but octet vector was a bit slower than the bit vector.
18:11:52
shrdlu68
Yes. I had to subseq and mask some of the bits of the last octet where length was not a multiple of 8.
18:12:46
russellw
how many of these things are you storing, that you need to worry about 32 bytes per key?
18:14:58
russellw
supposing you have an atom to convert to a (read)able string, is this the fastest/simplest way to do it? (format nil "~s" a)
18:16:19
whartung
uh.. “all 240-bit prefixes”, we only have 64 bit addressing…arent; you going to run out? “I know, I’ll use virtual memory! — Now you have 2 problems.”
18:18:27
shrdlu68
whartung: I take a file, read it into an octet vector, convert that into a bit vector, and, for each bit, look back 240 bits, keeping track of the last (n=240) bits.
18:28:06
russellw
also '5s to process a 36k file'? on a modern computer, it should be much faster than that. Calculating hashes is nowhere near that slow. By orders of magnitude. Have you profiled it? What compiler are you using?
18:33:25
shrdlu68
russellw: Profiling: https://gist.github.com/shrdlu68/081a78c6196b676395ca518a268d9119
18:36:00
russellw
so if I understand that correctly, you are using SBCL, which is generally reckoned the fastest available compiler, and it is indeed spending a large percentage of the total time calculating hashes. Okay, that is bizarre, and I have no explanation for it. Should be several orders of magnitude faster
18:36:38
shrdlu68
russellw: Other implementations take so long I haven't really let them run to completion.
18:42:09
whartung
3.7% shouldn’t dominate the discussion. It’s interesting, but obviously not “the reason” its slow
18:42:43
whartung
I assume this: (FLET "BODY-FUN-0" :IN SB-IMPL::GETHASH3) includes the body of the function?
18:43:52
shrdlu68
The profiling without inlining, and removing one function (forgot to update repo): https://gist.github.com/shrdlu68/081a78c6196b676395ca518a268d9119
18:52:21
shka_
no idea, but i would expect sbcl to have different logic for short bitvectors and long ones
18:52:30
shrdlu68
"enhancement: The value of SXHASH on bit-vectors of length equal to the word size now depends on the vector's contents instead of being constant; its value on bit-vectors of length divisible by the word size now depends also on the contents of the vector's last word." => http://www.sbcl.org/all-news.html
18:53:58
jasom
it uses a poor mixng function though; the longer the bit vector the more the difference, but even with 30 bytes the first 9 decimal digits are identical
19:21:05
jasom
also how many bit vectors are there supposed to be? I'm seeing 4M already and it hasn't yet run out of ram on my 1GB heap
19:22:15
oni-on-ion
heh @ last bullet item "(only with users permission and for maintenance reasons)" isnt that FB
19:23:13
oni-on-ion
if you dont need to modify the data as they come from files, perhaps using a more efficient data structure than that ?
19:24:35
oni-on-ion
https://www.reddit.com/r/lisp/comments/5oezj6/how_to_interface_with_a_mysql_database_in_lisp/ ?
19:32:40
dim
shrdlu68: try CCL, the GC is much better than SBCL's one in my playing around with pgloader
19:43:00
jasom
shrdlu68: heap will almost always be exhasted during GC; its either during GC or during a point where GC is excluded.
19:44:08
jasom
it looks like the hash table is about to grow to ~600MB and we are using 500MB, so that checks out.
19:44:08
dim
vtomole: when using pgloader users often reach heap exhausted/game over message in situations where CCL pile through the work at a fraction of the memory usage; it might be my code though
19:44:42
dim
but when users report problems with SBCL and big/huge data sets being processed by pgloader, I know I can just recommend CCL and it's going to be ok.
19:44:43
jasom
sbcl is very conservative about *when* to invoke the GC, so I've seen heap exhausted when there would have been room had the GC been run sooner
19:47:36
jasom
but in this case it's jsut that its growing the hash-table to be more than 1/2 the heap which just isn't going to work
19:50:20
jasom
but it looks like we have 4M objects in less than 2k buckets, which explains a lot. However ~200MB of simple-bit-vectors seems excessive if the original statement that it's only 1 vector per bit in the file.
19:52:51
jasom
oh, that makes sense then. Space usage isn't that high; looks like its 8bytes+size of bit-vector per bit vector and then thhere's another over 100M for the single-float arrays
19:53:08
jasom
so it's ~300M of data for 100M of hash-table overhead which isn't great but not terrible.
19:54:53
jasom
however, it looks like it may be copying the data on a rehash, as that's the only thing that makes sense for a 600M allocation. I'll inspect the code
20:01:14
jasom
also it looks like it always rounds the hash table size up to a power of two, which makes the default rehash-size of 1.5 rather stupid
20:03:28
jasom
total hash-table overhead seems to be 4 times the word size of the number of elements rounded up to a power of two
20:27:22
aeth
pjb: whenever I have something where a compose macro/function/whatever might seem useful, I find that it's probably better just to combine two of the three compositions into one new, trivial, inline function
20:28:34
aeth
i.e. instead of (another-function (some-function (symbol-to-keyword foo))) I can have (another-function (some-function* foo)) where the variation of some-function converts symbols to keywords
20:29:08
aeth
(I say i.e. instead of e.g. because it's pretty much always that, usually even symbol-to-keyword!)
20:31:35
aeth
This is common when I have a function with a case working on keywords but I take in arbitrary symbols from a macro.