freenode/#lisp - IRC Chatlog
Search
5:23:48
fiddlerwoaroof
I don't really understand what you want, but there's a bunch of logging libraries for CL
6:38:19
jdz
flip214: Of course it is possible to make it a bit faster portably by working with fixnum-siszed chunks, and only creating a bignum at the end, but that would not fit in a single line on IRC.
6:38:37
flip214
scymtym: with :specifications I don't get a result (in the left "Runs" pane) either. hmmm.
6:38:59
jdz
The biggest problem is working with bits. I'm pretty sure implementation works with at least octet-sized chunks.
6:42:43
flip214
jdz: I'm looking at cl-qrencode, trying to speed it up. I'm currently converting a list of bits to a bit-vector, reducing garbage by ~33% so far. Now I got to the point where arithmetic (ECC) is being done on that data...
6:43:07
flip214
jdz: would you be interested to help? Then I'll just push my git-stash onto github for you to take a look at.
6:43:26
jdz
flip214: Why do you need bit-vectors? Just use numbers. Bit-twiddling in Common Lisp is so much more enjoyable than in C...
6:47:35
flip214
jdz: all the input data is collected; but depending on the mode, each input element provides 5 to 13 bits.
6:50:17
jdz
I'm looking at bstream.lisp file, and indeed some implementation choices are questionable.
6:50:27
flip214
jdz: github.com:phmarek/cl-qrencode.git, branch bit-stream-perf - the last commit shows the beginning.
6:54:48
flip214
jdz: well, there are 4 different input modes... so different types become unwieldy, I believe
6:55:20
flip214
but yeah, building a bigint out of all the input data was another idea I got during the bit-array conversion, too
6:58:18
flip214
well, I was wary of needing to shift a bignum a few hundred times (for each input item) - or needing to know _where_ each input bit needs to land, beforehand
7:02:38
splittist
What's the usecase for creating vast numbers of QR codes such that the time to produce them is a bottleneck?
7:03:27
jdz
But then I looked at the code, and it stores bits in lists, and appends at the end all the time.
7:06:56
splittist
jdz: perhaps it's just bored, waiting for the user to move their mouse or the printer to cycle in the next sheet of paper (:
7:11:07
no-defun-allowed
They ask "what is the program performance" but they never ask "how is the program performance" :(((
7:54:09
scymtym
flip214: i don't understand how WITH-RECORDING could finish without adding at least an empty run. can you try tracing (i mean CL:TRACE in this case) CLIM.FLAMEGRAPH.APPLICATION::ADD-RUN?
8:10:52
flip214
scymtym: some load in sb-sprof:w-p works; with flamegraph I get lots of "Dropping ... event since there is no corresponding enter event on the stack." and then a hang
8:11:36
jdz
flip214: Converting a bit-vector using fixnum-sized chunks is only 6.75 times faster than the simple one-liner with REDUCE (0.63s vs. 4.253s for a bit vector of length 300 repeated 500000 times).
8:12:35
lukego
Crazy thought. Maybe I could actually use CLIM layout features like stream positions and tabular layout and graph layout etc for circuit board design. I mean, I will generate the Gerber manufacturing data from the CLIM output records anyway, so it doesn't matter how I position the components, and if what I need is an NxM grid of (say) capacitors then why not call a standard CLIM routine to produce that? Seems nuts though
8:14:29
scymtym
flip214: can you put together a complete recipe for reproduction? i can install everything in a container to get the same versions if necessary
8:29:18
jdz
flip214: Funny, in CCL the fixnum chunked version is around 10 times slower than the simple REDUCE.
8:46:10
beach
jnewton: I asked you a question the other day: Did y'all ever finish your Baker-style SUBTYPEP?
8:49:32
jnewton
phoe: which reader do you use for IRC? I find the emacs ERC reader pretty difficult to get my head around.
8:50:19
phoe
I use tmux to be able to connect and disconnect from it at will, including from multiple machines at the same time
8:54:36
jnewton
abbrevs? are you reverring to emacs abbrevs, M-/? I've never used abbrevs in plain text, but I use then pretty often in programming.
9:00:12
beach
Plus, I then don't have to expose silly and hard-to-understand abbreviations on my readers. And I don't have to show the entire world that I don't master my tools.
9:03:12
jnewton
beach: by abbrev do you mean that when you type, emacs replaces what you type with what you meant?
9:04:19
jnewton
I tend to turn that feature off in editors that support it, because it too often replaces what I meant but misspelled, with what I didn't mean at all.
9:04:20
beach
Yes, like I would type "gme", and it automatically expands to "Good morning everyone!"
9:04:49
jnewton
ahh. well since I can type pretty fast, I can type good morning everyone as fast as I can remember gme.
9:04:56
beach
jnewton: Oh, but if you define your own abbrevs, it will replace only those, and you choose ones that you won't mistype.
9:05:46
beach
I don't believe for a second that you can type it as fast once you get used to using your abbrev. But if you don't want to save time, that's up to you of course.
9:06:55
beach
I forget. But I usually start with /msg <nick> ... and then the other person will create a private window.
9:08:08
nij-
Hello! I just found serapeum and am delighted by its documentation. I think it's now good time for me to start reading some awesome utils library. It has two benefits. First, I learn from more experienced people. Second, I would know what utils are available so I don't have to invent the (incomplete) wheels by myself. The problem is that I'm not sure what libs I should look into. Alexandria (1) is definitely on my list. How about
9:12:14
phoe
nij-: I guess that quicklisp usage stats can give you some numbers for most popular libs
9:22:38
nij-
phoe good idea! I've only found this: http://blog.quicklisp.org/2017/11/october-2017-download-stats.html is there newer or a programmatic version?
9:27:24
nij-
Hmm.. serapeum isn't that popular, contradicting to what I thought. What does that mean @_@..
9:27:48
jnewton
beach: you were asking about the version of subtypep we were working on at IRC. No it was never finished, and the student has graduated.
9:28:22
jnewton
beach: I really wish I could have convinced the student to release the code in an unfinished state, but he didn't want to reslease it before it worked.
9:29:19
jnewton
beach: I hadn't planned to suggest it to a new student. Why? because I think the first student did all the interesting part of the work, and the only remaining parts are the parts that are not fun, or very difficult.
9:30:06
beach
Yes, I understand. Also, there are indications that Baker's technique can't handle the CONS type well, other than the trivial one.
9:30:27
jnewton
beach: In retrospect, and just from a single data point, I believe one of the problems with Baker's algorithm is that it is conceptually elegant, but there are corner cases which are VERY difficult.
9:31:05
beach
jnewton: Makes sense. Anyway, we are probably going with Bike's work on a version that canonicalizes.
9:31:55
jnewton
The student spent the last semester trying to model intervals using a technique using multi-dimentional potentially unbounded bounding boxes. it made really pretty slides and really pretty lightning talks but couldn't in the end figure out whether such a set was a subtype of T.
9:32:51
beach
Baker writes that canonicalization can create exponential blow-up, but I doubth that it will happen in practice.
9:35:07
splittist
nij: serapeum is great, and a good read. But utils libraries grow out of abstracting from concrete requirements, so it's sometimes hard to understand why something is done in a particular way if you haven't faced particular challenges. Something like cl-ppcre, where the domain is easily understandable, might be more illuminating if you're looking for examples of lispy ways of meeting concrete challenges.
9:38:43
jnewton
beach: ahh, my private message window is hiddent, and nothing in emacs (that I noticed) notified me that I have new information to read in a hidden buffer.
9:39:55
jnewton
in which mode line? the main one? if the buffer is hidden, then I won't see its mode line. right?
9:41:33
beach
Like right now, mine says [i] (in gray) for "irc.freenode...", so I guess I have a message from the admins.
9:45:31
splittist
(I use irccloud.com, as, I think, does drmeister; costs money, is web-based, doesn't use abbrevs, but is persistent, usable from any machine, and meets my modest needs.)
11:23:54
flip214
scymtym: having a cpu-using body in with-recording seems to work... although I now see %start-time being 0, so I need to zoom 51y into the graph ;(
11:26:59
scymtym
flip214: i don't know what "work" means given the different execution and failures modes you have described. are you still trying to do deterministic profiling or are you now looking for a flamegraph based on statistical profiling? i think the incorrect duration happens when no deterministic traces have been recorded. as i said: i can only address issues if you provide a recipe for reproducing them
11:32:08
scymtym
flip214: did you adjust the SAMPLE-RATE in CALL-WITH-RECORDING? otherwise it will skip a certain ratio of calls
11:34:27
scymtym
note that you can record calls that are already in-progress when the recording starts. that's where WARNING: Dropping #S(EVENT …) comes from
11:36:42
flip214
well, if it hangs, I get _no_ swank communication any more - checked with wireshark.
11:38:34
scymtym
for the bogus duration, right-click that run and look at the start/end times and the traces
11:41:37
flip214
grrr... got 12.4seconds (correct), but clicking on "Analyze run" made the CPU spin for ~15secs - but no response any more
11:44:00
flip214
got one with 1.4sec, one with 51y now... but when I try to inspect the 51y one I get no response.
11:47:54
flip214
I added to the packages list... but had a type, ":HUNCHENTOOT" instead of "HUNCHENTOOT" - and that seems to give a loop that allocates memory until the heap is exhausted.
11:49:17
scymtym
i can look into that specific problem but generally, i don't understand what you are doing
11:51:37
flip214
seems the :record-blockers and :record-io breaks my usecase (socket IO via swank??), removing them helps tremendously
11:51:56
flip214
and not needing :specifications is good as well, now I can even trace with external load generators
12:07:53
flip214
jdz: QR is ~25% in my macro benchmark, so you've got a chance to increase performance of my usage by 1.33x! ;)
12:21:55
jdz
Apparently I've missed their previous posting about fully-remote position 6 months ago (https://old.reddit.com/r/lisp/comments/jiz3pt/remote_common_lisp_developer_vacancy_at_ravenpack/).
12:55:38
agumonkey
on a machine (linux mint + sbcl) I could load-asd "/.../...asd" then load-system :foo and it fetched dependencies
12:56:34
agumonkey
but on my remote laptop (arch + sbcl) if I load-asd then load-system `I get Component ASDF/USER::CL-HTML-DIFF not found, required by #<SYSTEM "nyxt">
12:57:57
agumonkey
hmm so asdf doesn't deal with fetching, only loading/setting up when they're already retrieved by quicklisp:quickload ?
13:01:16
Xach
agumonkey: that's right, asdf loads what is available and signals an error if something is missing. quicklisp handles that error and fetches things to retry.
13:40:32
royae
C. Queinnec once report in a paper that D. Moon told him not to use packages as a module system, as it is a reader thing.
13:41:38
beach
royae: It is often pointless to ask "why" Common Lisp is missing something. I mean, the answer is that it wasn't put in when the standard was made.
13:42:08
beach
royae: So are you asking what the state of mind of the members of the committee was when they didn't include it?
13:42:29
phoe
oh, OK - one issue I see is that it's hard to assign a piece of Lisp code to a module because it's hard to cleanly slice pieces of a Lisp image; there's side effects that can mutate the state of the whole image
13:42:33
jackdaniel
there are two rickety and deprecated interfaces for modules: require and provide
13:43:10
phoe
ASDF works around this, but it's still not 100% modular where you can plug things out as easily as you can plug stuff in
13:43:53
beach
royae: If I may guess, the reason is that they were not able to figure out a solution that was both semantically sound and that would not harm performance.
13:44:49
beach
royae: For instance, when I wrote my paper on first-class global environments, I read the paper by Queinnec, and decided that the overhead was going to be too great for Common Lisp.
13:45:15
beach
royae: Most module systems I have seen require a hash-table lookup for every function call, and that would be totally unacceptable.
13:48:47
beach
I doubt that it will be adopted by existing implementations, so I fear that the impact is going to be low.
13:49:16
beach
It can't be bolted onto an existing implementation. The implementation would have to do some major rewrites.
13:51:50
beach
Plus, implementations like SBCL that are optimized for performance will not even consider it. Although there is no overhead for a function call, there is a hash-table lookup for things like (fdefinition <non-constant-name>).
13:53:56
beach
royae: And you can't really resolve bindings completely at compile time, given that Common Lisp requires late binding, so you still need an indirection. I guess existing proposals did that with a hash table lookup and, like I said, it is not acceptable because of performance.