freenode/#lisp - IRC Chatlog
Search
10:07:32
beach
In fact, I save some space that way, because very few symbols will have a non-empty property list, so no need to represent them in the environment. In a typical implementation, this cell is allocated in every symbol, whether needed or not.
10:08:12
beach
The same thing goes for the function cell, the value cell, etc. In SICL, a symbol has only a NAME and a PACKAGE.
10:10:08
beach
I can even imagine some implementations having two function cells in every symbol, one for the name itself and one for (SETF <name>).
10:10:58
beach
Worse (in terms of maintainability), some implementations have a function cell for the name itself, but store the SETF name somewhere else.
11:16:01
sirkmatija
Hello, I am building a small web game and I am wondering how to serve a file (not a string) by ningle. Thanks for your help.
11:21:23
sirkmatija
I tried reading file into string and then serving it, but then it doesn't find js script file, even if I provide full (not relative) path
12:17:33
JuanDaugherty
sirkmatija, y u gotta use that no name thing when everybody wants you to use the lisp majors (hunchentoot, etc)?
12:28:36
jackdaniel
if you are more interested in shipping whole directory of files consider using (:static :path "/public/" :root #P"/my/dir/") as parameter to lack:builder (instead of just making ningle:<app> instance)
12:33:36
sirkmatija
thank you, it serves the file now, but the script isn't found - when i inspect source it say URL /script.js was not found on this server.
12:34:35
jackdaniel
are you sure it is under /script.js ? not for instance my-host:8080/my-app/script.js ?
12:35:40
jackdaniel
computers are not magic, usually they do exactly what you tell them to do (and when they *do* magic for you, then it is usually a game over [opinionated])
12:36:38
sirkmatija
I thought I don't have to serve the files separately if they are in same dir, a stupid mistake
13:10:37
Shinmera
Just a quick announcement, the emacs package to interact with plaster.tymoon.eu is now on Melpa: https://melpa.org/#/plaster
13:30:21
beach
Shinmera: I don't use "fukamachiware", because it doesn't solve the problems I want to solve, but from what I hear, people are very impressed with it, so I assumed it was high-quality stuff. Not so?
13:35:17
makomo
oh beach, i have a couple more typos to report (hopefully you won't mind me snooping around :-))
13:36:01
beach
makomo: Not at all. If stuff is freely available, it is there for you to read and comment on.
13:37:18
makomo
beach: http://metamodular.com/Software-engineering/developer-productivity.html "First, let is"
13:37:30
makomo
beach: http://metamodular.com/Psychology/continuous-improvement.html "I use computer every day"
13:39:02
beach
makomo: Let's say it's currently the only one. But who knows, maybe loke and others will create ALS one day.
13:39:32
makomo
beach: http://metamodular.com/Essays/you-know-you-have.text "you use more then" (not sure why i read this one but there you go :-))
13:41:25
makomo
beach: i agree very much with your essay on pyschology and people being inefficient at the stuff they do (talking about software specifically)
13:42:50
makomo
however, i managed to get one of my uni friends to try out emacs and he's been actually using it for the last week or so
13:43:52
makomo
i've helped him install slime and he's reading pcl currently (started with land of lisp but then i intervened)
13:48:03
makomo
beach: yeah, true. i figured that it's good to try to spread the word and work on "recruiting" new people into the whole hacker culture
13:48:45
makomo
there's only so much you can do of course, but still spreading the word is part of "giving back" imo
13:51:23
makomo
as much as i'd like to share knowledge, you still have to pick people who you're sharing with carefully
13:52:32
Shinmera
ACTION feels like he's wasting his effort on this channel more often than he'd like to admit
13:54:37
beach
makomo: Now if you could snoop around in my software documentation as well, that would be great! :)
13:56:00
makomo
beach: but i do like your approach to documentation -- nicely written and formatted whitepapers, etc.
13:57:10
makomo
of course that's just my opinion, but i had a weird feeling when i was reading LL (i didn't finish it completely though)
14:51:11
dim
code is at https://github.com/qitab/qmynd/blob/master/src/mysql-protocol/response-result-set.lisp#L197
14:52:11
dim
I have column-type 252, which should be eql to (defconstant +mysql-type-blob+ #xfc) I think, and (column-definition-v41-packet-cs-coll column-definition) is 63, which should be eql to (defconstant +mysql-cs-coll-binary+ 63), and, sorry what?
14:54:10
dim
since https://github.com/qitab/qmynd/commit/39e17d0fdcf111f7b081d8aff316af084476f198#diff-a7f8d7f40f13983046fa6d0c5bfb2272
14:56:02
beach
I suppose the logic behind how CASE works is that the compiler should be able to build a hash table, perhaps with perfect hashing, to make the choice real fast.
14:56:12
dim
(case (#. (list +mysql-type-string+ +mysql-type-var-string+ +mysql-type-tiny-blob+ +mysql-type-medium-blob+ +mysql-type-long-blob+ +mysql-type-blob+) ...)) doesn't seem to cut it
14:56:44
schweers
Does anyone use cl-launch with CCL? I’ve been using cl-launch with sbcl for a few months now and wanted to try my code on CCL too. When I start a repl, load and run my code from there it works. So thats a good start. When I use cl-launch I am dropped in a repl too (which doesn’t happen with sbcl).
14:57:06
jackdaniel
alexandria has switch, but it doesn't allow specifying many elements (just one, but it is evaluated)
14:57:30
beach
jackdaniel: That is not enough if the CASE would take longer if a hash table were used.
14:57:41
schweers
also, when I try to run my code from this cl-launch induced repl CCL bails out because the heap is exhausted (which does not happen when I use a repl directly).
14:58:09
dim
but the #. (list +...+ ) doesn't seem to work, so either I tried it wrong again, or maybe I am testing the code I didn't edit
15:00:31
beach
I suppose if the labels are rational numbers, a technique similar to my fast generic dispatch could be used. That could be a big win.
15:01:26
jackdaniel
sbcl makes constants available at read time (if defined at the same file), ccl doesn't - both behaviors are conforming
15:01:46
jackdaniel
to make code portable (while having defconstant) using eval-when is the solution
15:02:04
beach
_death: Yeah, I wasn't going to bring it up, because I saw no trace of the definitions in the same file, so I assumed they existed already.
15:03:02
beach
With 11 cases, it could definitely benefit from a technique like my fast generic dispatch, provided the entries are numbers, of course.
15:03:58
_death
a tree case analysis or some perfect hash-table thingy may work, if it's even needed (profile first)
15:05:31
beach
_death: A tree (or rather binary search using constants) is what my fast generic dispatch uses, simply because it is faster than trying to store some data structure in memory.
15:07:01
beach
It makes sense even when the number of cases is large, because the number of comparisons only grows logarithmically.
15:07:27
beach
So you would need thousands of cases in order for it to be slower than a few memory accesses.
15:08:18
beach
That's what I am saying. My technique does not access memory, but a perfect hashing scheme would.
15:09:37
dim
I don't think that code path is the most prominent to fix in qmynd btw, but it'd be nice to improve it some more
15:10:47
beach
This discussion did give me the idea that I should implement CASE like that in SICL (for the case of real numbers).
15:12:11
_death
beach: it reminds me of pkhuong's string-case.. and I wouldn't be surprised if I encountered more case operators like it in the past
15:12:14
dim
by the way with the fix the MySQL read now takes 2s while writing to PostgreSQL takes the 27s (in another thread)
15:12:45
dim
so my current performance problem is to do with format-vector-row if some of you guys want to play around
15:13:27
dim
dead easy to improve, but last I tried I broke the whole of pgloader, for wanting to do too many optimizations at once
15:18:14
_death
should split it into several functions.. and btw you can (reduce ... :key ...) instead of (reduce ... (map ...))
15:29:06
dim
I must have also considered :key for class/struct slot accesses, in my mind, but the API is way more generic than that
15:30:28
dim
then there's a huge optimisation (I think) to be done here by only considering UTF-8 convertions for text based datatypes
15:31:28
dim
and then another idea is to avoid batching altogether in the db-2-db case for pgloader, as error handling then should be taken care of at the source db rather than with a reject file
15:40:21
dim
is there any point in building a single vector of bytes compared to a vector of vector of bytes, on a pure performance standpoint?
17:07:51
rumbler31
with (setq *print-radix* t), (CCL:MAKE-SOCKET :REMOTE-HOST "beta.quicklisp.org" :REMOTE-PORT 80) fails with bt https://pastebin.com/1ZQK1qQu
17:12:51
rumbler31
looking to run the fast-io tests. I'm not familiar with many testing frameworks yet, but it doesn't look like an asdf test-op is defined for fast-io. How do I go about... running the tests?
18:35:17
flip214
Can I pass multiple :long-name options to CLON:DEFSYNOPSIS, so that any one of the long names can be used? Ie. aliases?
18:38:45
sjl
rumbler31: as far as I know that's the intended way to run tests for an ASDF system. if fast-io does it differently, they should document it.
18:39:20
rumbler31
so i've been looking for machinery for that, I expect that the test-op function to call would be explicitly defined in the .asd file right?
18:42:49
sjl
the define the tests in a separate system, like a lot of systems do, to avoid having to load the unit test framework to use the main system
18:43:07
sjl
usually when you do that, you add an :in-order-to to the main system like this https://github.com/sjl/cl-digraph/blob/master/cl-digraph.asd#L11
18:45:01
sjl
and then does (checkl:define-test-op :fast-io :fast-io-test) to I think add the in-order-to?
18:45:46
sjl
I dunno, it's weird and non-standard and won't work like most people expect -- (progn (ql:quickload :fast-io) (asdf:test-system :fast-io)) won't work without other stuff
18:47:21
Xach
I also find the way to set it up a bit cumbersome. Kind of a throwback to the (asdf:oos 'asdf:load-op ...) days.
18:50:08
rumbler31
so the answer seems to be that the authors usage of checkl in the -test.asd means that the correct answer to run tests is to invoke (run-all). Although the output doesn't suggest success or failure on my system. Thanks all
19:03:54
rumbler31
minion: memo for antonv: theoretically, if you wish to send a udp packet to a multicast address, you shouldn't need any more os support. The magic comes when you try to tell the os that you care to receive packets from certain multicast addresses
19:05:35
rumbler31
minion: memo for antonv: there were some patches for ccl on the mailing list long ago that added support for multicast sockets. Sending to multicast addresses *may* *just work* though
20:39:30
dim
ok, I simplified https://github.com/dimitri/pgloader/blob/master/src/pgsql/copy-format.lisp a lot, made several simpler functions and all, and I won some low-% of perfs at that
20:39:50
dim
so now that it's easier to review, I hope some perfs guru will get interested into either giving advices or sending patches ;-)
20:46:18
AeroNotix
Wonder if simple application of type declarations and safety 0 would get anything.
20:47:34
AeroNotix
dim: why not just filter the list of funs prior to the loop? https://github.com/dimitri/pgloader/blob/master/src/pgsql/copy-format.lisp#L52
20:50:42
dim
AeroNotix: bencmarking consists of using pgloader to load data into PostgreSQL or migrate a whole database from MySQL, the function format-vector-row is called with every single row that's loaded
20:51:14
dim
but anyway, some people had comments earlier on the previous version of the function, that I addressed, and it was good comments ;-)
20:51:38
AeroNotix
dim: Yes I know, i was reading before, thought I'd read through it now you've cleaned the code up. Before it was rather unwieldy to read.
20:55:56
AeroNotix
dim: does it need to be modified in an in-memory vector rather than streaming directly to file?
20:56:53
dim
the idea is that we use the high performance COPY protocol of PostgreSQL, which allows streaming data (in and out of the database)
20:57:09
dim
the thing is that COPY is (of course) transactional, so any error aborts the whole transaction
20:57:35
dim
pgloader works around transaction rollback and replay the good input, rejecting the bad one, so that you can get your data in
20:58:07
AeroNotix
ohh, ok. So maybe to retain that feature you can't stream it directly. Might be a good option to provide though :)
20:58:22
dim
when that happens we have ready-to-go data in memory, and also we know that the COPY offset reported in the PostgreSQL is the array offset in the batch
20:59:07
dim
now I'm thinking that in the case when you migrate from a source database to PostgreSQL, you don't want any reject file, you'd rather --on-error-stop (the option exists)
20:59:28
dim
and then you either fix your load command (casting rules, mostly), or fix the input database
21:00:38
dim
so I'm thinking that --on-error-stop should 1. be the default if the source of the data is a database server and 2. make it so that we don't use the in-memory batches at all but rather stream the data to PostgreSQL directly
21:02:12
AeroNotix
I've come across pgloader before a few times. Glad to see it's still being worked on.
21:03:36
dim
well the White Paper has been downloaded some already, which proves interest, and also that's tooling for the Enterprise, not much use at home
21:03:40
AeroNotix
dim: you already have a good place to filter functions here: https://github.com/dimitri/pgloader/blob/3bb128c5db9fc7b8926586113aa1b97c59a7b457/src/pgsql/copy-from-queue.lisp#L124-L126
21:03:58
AeroNotix
I don't see why https://github.com/dimitri/pgloader/blob/3bb128c5db9fc7b8926586113aa1b97c59a7b457/src/pgsql/copy-format.lisp#L52 needs to be conditional
21:04:46
dim
I could use #'identity I guess, but I'd be surprised if it came to a benefit in terms of performance
21:05:30
AeroNotix
I was going to mention #'identity. My thinking is that it's part of your inner loop, could be ran quite a lot on a huge input
21:11:05
AeroNotix
didn't read the whole scrollback but is using a vector like this optimal? Even ignoring what i mentioned before about streaming directly to PG, is it not faster to stream bytes to an in memory buffer instead of writing bytes to vector offsets and incrementing an index?
21:15:37
AeroNotix
except one takes care of things for you regarding where you are in the stream etc.
21:17:48
dim
I followed the design of Postmodern here, because it looked good to me and the author looks like he knows what he's doing
21:18:33
AeroNotix
fair enough, I'm more asking questions as well as offering what could help. I've never had to properly optimize a CL program just thinking what works in other languages
21:41:25
paule32
is the name "foo" from the string supportable to get the name of a "defparameter" struct?
21:43:02
AeroNotix
paule32: what are you trying to do apart from make a weird stringly typed program
21:45:34
AeroNotix
a better option if writing a language "vm" of sorts is have a datastructure to hold the mappings of sub language variables to implementation level objects