freenode/#lisp - IRC Chatlog
Search
17:07:51
rumbler31
with (setq *print-radix* t), (CCL:MAKE-SOCKET :REMOTE-HOST "beta.quicklisp.org" :REMOTE-PORT 80) fails with bt https://pastebin.com/1ZQK1qQu
17:12:51
rumbler31
looking to run the fast-io tests. I'm not familiar with many testing frameworks yet, but it doesn't look like an asdf test-op is defined for fast-io. How do I go about... running the tests?
18:35:17
flip214
Can I pass multiple :long-name options to CLON:DEFSYNOPSIS, so that any one of the long names can be used? Ie. aliases?
18:38:45
sjl
rumbler31: as far as I know that's the intended way to run tests for an ASDF system. if fast-io does it differently, they should document it.
18:39:20
rumbler31
so i've been looking for machinery for that, I expect that the test-op function to call would be explicitly defined in the .asd file right?
18:42:49
sjl
the define the tests in a separate system, like a lot of systems do, to avoid having to load the unit test framework to use the main system
18:43:07
sjl
usually when you do that, you add an :in-order-to to the main system like this https://github.com/sjl/cl-digraph/blob/master/cl-digraph.asd#L11
18:45:01
sjl
and then does (checkl:define-test-op :fast-io :fast-io-test) to I think add the in-order-to?
18:45:46
sjl
I dunno, it's weird and non-standard and won't work like most people expect -- (progn (ql:quickload :fast-io) (asdf:test-system :fast-io)) won't work without other stuff
18:47:21
Xach
I also find the way to set it up a bit cumbersome. Kind of a throwback to the (asdf:oos 'asdf:load-op ...) days.
18:50:08
rumbler31
so the answer seems to be that the authors usage of checkl in the -test.asd means that the correct answer to run tests is to invoke (run-all). Although the output doesn't suggest success or failure on my system. Thanks all
19:03:54
rumbler31
minion: memo for antonv: theoretically, if you wish to send a udp packet to a multicast address, you shouldn't need any more os support. The magic comes when you try to tell the os that you care to receive packets from certain multicast addresses
19:05:35
rumbler31
minion: memo for antonv: there were some patches for ccl on the mailing list long ago that added support for multicast sockets. Sending to multicast addresses *may* *just work* though
20:39:30
dim
ok, I simplified https://github.com/dimitri/pgloader/blob/master/src/pgsql/copy-format.lisp a lot, made several simpler functions and all, and I won some low-% of perfs at that
20:39:50
dim
so now that it's easier to review, I hope some perfs guru will get interested into either giving advices or sending patches ;-)
20:46:18
AeroNotix
Wonder if simple application of type declarations and safety 0 would get anything.
20:47:34
AeroNotix
dim: why not just filter the list of funs prior to the loop? https://github.com/dimitri/pgloader/blob/master/src/pgsql/copy-format.lisp#L52
20:50:42
dim
AeroNotix: bencmarking consists of using pgloader to load data into PostgreSQL or migrate a whole database from MySQL, the function format-vector-row is called with every single row that's loaded
20:51:14
dim
but anyway, some people had comments earlier on the previous version of the function, that I addressed, and it was good comments ;-)
20:51:38
AeroNotix
dim: Yes I know, i was reading before, thought I'd read through it now you've cleaned the code up. Before it was rather unwieldy to read.
20:55:56
AeroNotix
dim: does it need to be modified in an in-memory vector rather than streaming directly to file?
20:56:53
dim
the idea is that we use the high performance COPY protocol of PostgreSQL, which allows streaming data (in and out of the database)
20:57:09
dim
the thing is that COPY is (of course) transactional, so any error aborts the whole transaction
20:57:35
dim
pgloader works around transaction rollback and replay the good input, rejecting the bad one, so that you can get your data in
20:58:07
AeroNotix
ohh, ok. So maybe to retain that feature you can't stream it directly. Might be a good option to provide though :)
20:58:22
dim
when that happens we have ready-to-go data in memory, and also we know that the COPY offset reported in the PostgreSQL is the array offset in the batch
20:59:07
dim
now I'm thinking that in the case when you migrate from a source database to PostgreSQL, you don't want any reject file, you'd rather --on-error-stop (the option exists)
20:59:28
dim
and then you either fix your load command (casting rules, mostly), or fix the input database
21:00:38
dim
so I'm thinking that --on-error-stop should 1. be the default if the source of the data is a database server and 2. make it so that we don't use the in-memory batches at all but rather stream the data to PostgreSQL directly
21:02:12
AeroNotix
I've come across pgloader before a few times. Glad to see it's still being worked on.
21:03:36
dim
well the White Paper has been downloaded some already, which proves interest, and also that's tooling for the Enterprise, not much use at home
21:03:40
AeroNotix
dim: you already have a good place to filter functions here: https://github.com/dimitri/pgloader/blob/3bb128c5db9fc7b8926586113aa1b97c59a7b457/src/pgsql/copy-from-queue.lisp#L124-L126
21:03:58
AeroNotix
I don't see why https://github.com/dimitri/pgloader/blob/3bb128c5db9fc7b8926586113aa1b97c59a7b457/src/pgsql/copy-format.lisp#L52 needs to be conditional
21:04:46
dim
I could use #'identity I guess, but I'd be surprised if it came to a benefit in terms of performance
21:05:30
AeroNotix
I was going to mention #'identity. My thinking is that it's part of your inner loop, could be ran quite a lot on a huge input
21:11:05
AeroNotix
didn't read the whole scrollback but is using a vector like this optimal? Even ignoring what i mentioned before about streaming directly to PG, is it not faster to stream bytes to an in memory buffer instead of writing bytes to vector offsets and incrementing an index?
21:15:37
AeroNotix
except one takes care of things for you regarding where you are in the stream etc.
21:17:48
dim
I followed the design of Postmodern here, because it looked good to me and the author looks like he knows what he's doing
21:18:33
AeroNotix
fair enough, I'm more asking questions as well as offering what could help. I've never had to properly optimize a CL program just thinking what works in other languages
21:41:25
paule32
is the name "foo" from the string supportable to get the name of a "defparameter" struct?
21:43:02
AeroNotix
paule32: what are you trying to do apart from make a weird stringly typed program
21:45:34
AeroNotix
a better option if writing a language "vm" of sorts is have a datastructure to hold the mappings of sub language variables to implementation level objects
21:46:22
AeroNotix
rather than trying to generate defparameters from user input, parse the user input and maintain a mapping in a different datastructure and do lookups at the runtime of your program.
21:52:20
AeroNotix
but to answer your question, you could write something horrible with FIND-SYMBOL or something. Have fun
22:11:29
mishoo
on this line: https://github.com/glv2/ironclad/blob/master/src/public-key/ed448.lisp#L48
22:12:01
jmercouris
Is there way to do something like (read-line) and pre-fill a value in the user's terminal?
22:12:28
AeroNotix
mishoo: latest quicklisp -> (ql:quickload :ironclad) -> no issues. From "http://beta.quicklisp.org/archive/ironclad/2017-11-30/ironclad-v0.37.tgz
22:14:47
Shinmera
jmercouris: I use this https://github.com/Shirakumo/radiance-bootstrap/blob/master/boot.lisp#L39
22:20:10
mfiano
On older hardware it does. It's doing some expensive macro expansion before compile time.
22:20:14
mishoo
I thought it's a compiler bug because the compiler whines about #() not being a (simple-array (unsigned-byte 8) (*))
22:20:44
mishoo
but that's an empty array.. I thought it should be as good as any simple-array with no defined length
22:21:36
Bike
#() is a (simple-array t), which is a distinct thing from a (simple-array (unsigned-byte 8))
22:22:15
mishoo
then the compiler is right and ironclad is wrong → https://github.com/glv2/ironclad/blob/master/src/public-key/ed448.lisp#L35-L48
22:27:11
mfiano
Anyway, (let ((a (make-array 0 :element-type '(unsigned-byte 8)))) (list a (typep a '(simple-array (unsigned-byte 8) (*))))) #| => (#() T) |#
22:29:21
paule32
pagnol: pseudo code: IF content-listA = content-listB THEN result = true ELSE false
22:32:45
pagnol
when it comes to lists you may want to know that the LIST function always creates a new object
22:33:05
Bike
mfiano: that's just because the writer is oversimplifying. two things having the same textual representation doesn't make them the same.
22:33:51
Bike
mfiano: if you try (write (make-array 0 :element-type '(unsigned-byte 8)) :readably t) you'll get more specific output.
22:38:07
mishoo
Bike: I kinda agree, on an implementation standpoint, that sets must consider the types of their elements. but IMO an empty set should be the same as an empty set.
22:39:09
shrdlu68
The point, I think, was that Ironclad could not be faulting for receiving a zero-length octet vector where it expects an octet vector of any length.
22:39:21
mishoo
alright, arrays. an empty array of any type should pass type-checking for (array-of-certain-type)
22:39:43
Bike
it's not receiving a zero length octet vector, it's receiving a zero length T vector, and these are distinct objects
22:41:11
Bike
on that subject, regardless of how you think it should work - and i'm not rejecting your opinion there - the standard is pretty clear that being zero length is not a factor here.
22:43:07
mishoo
well to sum it up, I'd just like ironclad to load here. I'll look tomorrow for possible fixes.
22:45:40
shrdlu68
The compiler is telling mishoo that an object which prints as "#()" is not of type (simple-array (unsigned-byte 8))
22:46:57
shrdlu68
What we don't seem to be in agreement over is whether that object could be a zero-length octet vector.
22:47:53
Bike
the issue is that arrays of different specializations can be completely different kinds of objects. for example it would be fine for sbcl to store the length of a ub8 vector at a different place from the length of a t vector. then it could compile code that's known to get an ub8 vector (like this) to use the first offset and not check to see if it has to use the second.
22:48:30
pagnol
paule32, I'm sure if you open a repl and play around with mapcar and eq/equalp a little, you will find the mistake yourself
22:51:20
shrdlu68
Looking at the source, there's actually "#()" there. Yeah, that's bound to get flagged by the compiler.
22:52:25
mishoo
okay, we at least established that it's not some weird thing happening on my machine only, but a legitimate error from SBCL (surprised no one else complained thus far)
22:55:27
shrdlu68
mishoo: Why did you link to glv2/ironclad? Contemporary ironclad use sharplispers/ironclad.
22:56:22
mishoo
looks pretty much the same: https://github.com/sharplispers/ironclad/blob/master/src/public-key/ed448.lisp#L35-L48
23:06:22
pagnol
ok, I just wanted to verify my intuition that a German would never use 'seit wie lange'
23:07:17
mishoo
the change that Bike suggested works [ #.(make-array 0 :element-type '(unsigned-byte 8)) ]
23:08:06
shrdlu68
mishoo: It's strange that only you caught that bug, and while trying to load at that.
23:09:41
shrdlu68
I suspect so, whenever I start from scratch I always get problems loading static-vectors.
23:12:18
mishoo
try this.. either on some fresh VM, if you have one, or on your main account if you can afford it :) rm -rf .cache && rm -rf quicklisp
23:13:39
jmercouris
I suggest: https://stackoverflow.com/questions/30208863/how-to-enable-confirmation-alert-when-using-rm-command-to-delete-files-folde
1:31:15
pillton
aeth: I looked in to named and inlined specializations over the weekend. A few things were fixed as a result.
1:49:40
Pixel_Outlaw
I'm still rolling around the idea of a small 2D CAD but I'd have to choose a GUI first. :P