freenode/#lisp - IRC Chatlog
Search
5:23:48
fiddlerwoaroof
I don't really understand what you want, but there's a bunch of logging libraries for CL
6:38:19
jdz
flip214: Of course it is possible to make it a bit faster portably by working with fixnum-siszed chunks, and only creating a bignum at the end, but that would not fit in a single line on IRC.
6:38:37
flip214
scymtym: with :specifications I don't get a result (in the left "Runs" pane) either. hmmm.
6:38:59
jdz
The biggest problem is working with bits. I'm pretty sure implementation works with at least octet-sized chunks.
6:42:43
flip214
jdz: I'm looking at cl-qrencode, trying to speed it up. I'm currently converting a list of bits to a bit-vector, reducing garbage by ~33% so far. Now I got to the point where arithmetic (ECC) is being done on that data...
6:43:07
flip214
jdz: would you be interested to help? Then I'll just push my git-stash onto github for you to take a look at.
6:43:26
jdz
flip214: Why do you need bit-vectors? Just use numbers. Bit-twiddling in Common Lisp is so much more enjoyable than in C...
6:47:35
flip214
jdz: all the input data is collected; but depending on the mode, each input element provides 5 to 13 bits.
6:50:17
jdz
I'm looking at bstream.lisp file, and indeed some implementation choices are questionable.
6:50:27
flip214
jdz: github.com:phmarek/cl-qrencode.git, branch bit-stream-perf - the last commit shows the beginning.
6:54:48
flip214
jdz: well, there are 4 different input modes... so different types become unwieldy, I believe
6:55:20
flip214
but yeah, building a bigint out of all the input data was another idea I got during the bit-array conversion, too
6:58:18
flip214
well, I was wary of needing to shift a bignum a few hundred times (for each input item) - or needing to know _where_ each input bit needs to land, beforehand
7:02:38
splittist
What's the usecase for creating vast numbers of QR codes such that the time to produce them is a bottleneck?
7:03:27
jdz
But then I looked at the code, and it stores bits in lists, and appends at the end all the time.
7:06:56
splittist
jdz: perhaps it's just bored, waiting for the user to move their mouse or the printer to cycle in the next sheet of paper (:
7:11:07
no-defun-allowed
They ask "what is the program performance" but they never ask "how is the program performance" :(((
7:54:09
scymtym
flip214: i don't understand how WITH-RECORDING could finish without adding at least an empty run. can you try tracing (i mean CL:TRACE in this case) CLIM.FLAMEGRAPH.APPLICATION::ADD-RUN?
8:10:52
flip214
scymtym: some load in sb-sprof:w-p works; with flamegraph I get lots of "Dropping ... event since there is no corresponding enter event on the stack." and then a hang
8:11:36
jdz
flip214: Converting a bit-vector using fixnum-sized chunks is only 6.75 times faster than the simple one-liner with REDUCE (0.63s vs. 4.253s for a bit vector of length 300 repeated 500000 times).
8:12:35
lukego
Crazy thought. Maybe I could actually use CLIM layout features like stream positions and tabular layout and graph layout etc for circuit board design. I mean, I will generate the Gerber manufacturing data from the CLIM output records anyway, so it doesn't matter how I position the components, and if what I need is an NxM grid of (say) capacitors then why not call a standard CLIM routine to produce that? Seems nuts though
8:14:29
scymtym
flip214: can you put together a complete recipe for reproduction? i can install everything in a container to get the same versions if necessary
8:29:18
jdz
flip214: Funny, in CCL the fixnum chunked version is around 10 times slower than the simple REDUCE.
8:46:10
beach
jnewton: I asked you a question the other day: Did y'all ever finish your Baker-style SUBTYPEP?
8:49:32
jnewton
phoe: which reader do you use for IRC? I find the emacs ERC reader pretty difficult to get my head around.
8:50:19
phoe
I use tmux to be able to connect and disconnect from it at will, including from multiple machines at the same time
8:54:36
jnewton
abbrevs? are you reverring to emacs abbrevs, M-/? I've never used abbrevs in plain text, but I use then pretty often in programming.
9:00:12
beach
Plus, I then don't have to expose silly and hard-to-understand abbreviations on my readers. And I don't have to show the entire world that I don't master my tools.
9:03:12
jnewton
beach: by abbrev do you mean that when you type, emacs replaces what you type with what you meant?
9:04:19
jnewton
I tend to turn that feature off in editors that support it, because it too often replaces what I meant but misspelled, with what I didn't mean at all.
9:04:20
beach
Yes, like I would type "gme", and it automatically expands to "Good morning everyone!"
9:04:49
jnewton
ahh. well since I can type pretty fast, I can type good morning everyone as fast as I can remember gme.
9:04:56
beach
jnewton: Oh, but if you define your own abbrevs, it will replace only those, and you choose ones that you won't mistype.
9:05:46
beach
I don't believe for a second that you can type it as fast once you get used to using your abbrev. But if you don't want to save time, that's up to you of course.
9:06:55
beach
I forget. But I usually start with /msg <nick> ... and then the other person will create a private window.
9:08:08
nij-
Hello! I just found serapeum and am delighted by its documentation. I think it's now good time for me to start reading some awesome utils library. It has two benefits. First, I learn from more experienced people. Second, I would know what utils are available so I don't have to invent the (incomplete) wheels by myself. The problem is that I'm not sure what libs I should look into. Alexandria (1) is definitely on my list. How about
9:12:14
phoe
nij-: I guess that quicklisp usage stats can give you some numbers for most popular libs
9:22:38
nij-
phoe good idea! I've only found this: http://blog.quicklisp.org/2017/11/october-2017-download-stats.html is there newer or a programmatic version?
9:27:24
nij-
Hmm.. serapeum isn't that popular, contradicting to what I thought. What does that mean @_@..
9:27:48
jnewton
beach: you were asking about the version of subtypep we were working on at IRC. No it was never finished, and the student has graduated.
9:28:22
jnewton
beach: I really wish I could have convinced the student to release the code in an unfinished state, but he didn't want to reslease it before it worked.
9:29:19
jnewton
beach: I hadn't planned to suggest it to a new student. Why? because I think the first student did all the interesting part of the work, and the only remaining parts are the parts that are not fun, or very difficult.
9:30:06
beach
Yes, I understand. Also, there are indications that Baker's technique can't handle the CONS type well, other than the trivial one.
9:30:27
jnewton
beach: In retrospect, and just from a single data point, I believe one of the problems with Baker's algorithm is that it is conceptually elegant, but there are corner cases which are VERY difficult.
9:31:05
beach
jnewton: Makes sense. Anyway, we are probably going with Bike's work on a version that canonicalizes.
9:31:55
jnewton
The student spent the last semester trying to model intervals using a technique using multi-dimentional potentially unbounded bounding boxes. it made really pretty slides and really pretty lightning talks but couldn't in the end figure out whether such a set was a subtype of T.
9:32:51
beach
Baker writes that canonicalization can create exponential blow-up, but I doubth that it will happen in practice.
9:35:07
splittist
nij: serapeum is great, and a good read. But utils libraries grow out of abstracting from concrete requirements, so it's sometimes hard to understand why something is done in a particular way if you haven't faced particular challenges. Something like cl-ppcre, where the domain is easily understandable, might be more illuminating if you're looking for examples of lispy ways of meeting concrete challenges.
9:38:43
jnewton
beach: ahh, my private message window is hiddent, and nothing in emacs (that I noticed) notified me that I have new information to read in a hidden buffer.
9:39:55
jnewton
in which mode line? the main one? if the buffer is hidden, then I won't see its mode line. right?
9:41:33
beach
Like right now, mine says [i] (in gray) for "irc.freenode...", so I guess I have a message from the admins.
9:45:31
splittist
(I use irccloud.com, as, I think, does drmeister; costs money, is web-based, doesn't use abbrevs, but is persistent, usable from any machine, and meets my modest needs.)
11:23:54
flip214
scymtym: having a cpu-using body in with-recording seems to work... although I now see %start-time being 0, so I need to zoom 51y into the graph ;(
11:26:59
scymtym
flip214: i don't know what "work" means given the different execution and failures modes you have described. are you still trying to do deterministic profiling or are you now looking for a flamegraph based on statistical profiling? i think the incorrect duration happens when no deterministic traces have been recorded. as i said: i can only address issues if you provide a recipe for reproducing them
11:32:08
scymtym
flip214: did you adjust the SAMPLE-RATE in CALL-WITH-RECORDING? otherwise it will skip a certain ratio of calls
11:34:27
scymtym
note that you can record calls that are already in-progress when the recording starts. that's where WARNING: Dropping #S(EVENT …) comes from
11:36:42
flip214
well, if it hangs, I get _no_ swank communication any more - checked with wireshark.
11:38:34
scymtym
for the bogus duration, right-click that run and look at the start/end times and the traces
11:41:37
flip214
grrr... got 12.4seconds (correct), but clicking on "Analyze run" made the CPU spin for ~15secs - but no response any more
11:44:00
flip214
got one with 1.4sec, one with 51y now... but when I try to inspect the 51y one I get no response.
11:47:54
flip214
I added to the packages list... but had a type, ":HUNCHENTOOT" instead of "HUNCHENTOOT" - and that seems to give a loop that allocates memory until the heap is exhausted.
11:49:17
scymtym
i can look into that specific problem but generally, i don't understand what you are doing
11:51:37
flip214
seems the :record-blockers and :record-io breaks my usecase (socket IO via swank??), removing them helps tremendously
11:51:56
flip214
and not needing :specifications is good as well, now I can even trace with external load generators
12:07:53
flip214
jdz: QR is ~25% in my macro benchmark, so you've got a chance to increase performance of my usage by 1.33x! ;)
12:21:55
jdz
Apparently I've missed their previous posting about fully-remote position 6 months ago (https://old.reddit.com/r/lisp/comments/jiz3pt/remote_common_lisp_developer_vacancy_at_ravenpack/).
12:55:38
agumonkey
on a machine (linux mint + sbcl) I could load-asd "/.../...asd" then load-system :foo and it fetched dependencies
12:56:34
agumonkey
but on my remote laptop (arch + sbcl) if I load-asd then load-system `I get Component ASDF/USER::CL-HTML-DIFF not found, required by #<SYSTEM "nyxt">
12:57:57
agumonkey
hmm so asdf doesn't deal with fetching, only loading/setting up when they're already retrieved by quicklisp:quickload ?
13:01:16
Xach
agumonkey: that's right, asdf loads what is available and signals an error if something is missing. quicklisp handles that error and fetches things to retry.
13:40:32
royae
C. Queinnec once report in a paper that D. Moon told him not to use packages as a module system, as it is a reader thing.
13:41:38
beach
royae: It is often pointless to ask "why" Common Lisp is missing something. I mean, the answer is that it wasn't put in when the standard was made.
13:42:08
beach
royae: So are you asking what the state of mind of the members of the committee was when they didn't include it?
13:42:29
phoe
oh, OK - one issue I see is that it's hard to assign a piece of Lisp code to a module because it's hard to cleanly slice pieces of a Lisp image; there's side effects that can mutate the state of the whole image
13:42:33
jackdaniel
there are two rickety and deprecated interfaces for modules: require and provide
13:43:10
phoe
ASDF works around this, but it's still not 100% modular where you can plug things out as easily as you can plug stuff in
13:43:53
beach
royae: If I may guess, the reason is that they were not able to figure out a solution that was both semantically sound and that would not harm performance.
13:44:49
beach
royae: For instance, when I wrote my paper on first-class global environments, I read the paper by Queinnec, and decided that the overhead was going to be too great for Common Lisp.
13:45:15
beach
royae: Most module systems I have seen require a hash-table lookup for every function call, and that would be totally unacceptable.
13:48:47
beach
I doubt that it will be adopted by existing implementations, so I fear that the impact is going to be low.
13:49:16
beach
It can't be bolted onto an existing implementation. The implementation would have to do some major rewrites.
13:51:50
beach
Plus, implementations like SBCL that are optimized for performance will not even consider it. Although there is no overhead for a function call, there is a hash-table lookup for things like (fdefinition <non-constant-name>).
13:53:56
beach
royae: And you can't really resolve bindings completely at compile time, given that Common Lisp requires late binding, so you still need an indirection. I guess existing proposals did that with a hash table lookup and, like I said, it is not acceptable because of performance.
13:54:33
royae
Well I think it'a very attractive functionality for SICL. I miss it in CL implementations more often I need performance.
13:56:24
beach
The main difficulty is with having different versions of the same system loaded simultaneously.
14:06:38
beach
Another possible answer to "why" is that the vendors involved in the standard thought it would cost too much for them to implement it, given their existing CLtL1 implementations. As I mentioned, it would be a major rewrite of several parts of such an implementation. Given how programs were written at the time, it is likely that the absence of a module system permeates most of the code,
14:10:14
royae
Neverless I think the fact that packages are a reader thing make the code less clear.
14:12:58
beach
There is a long tradition in the Lisp world that new features are first implemented as libraries (that may require minor modifications to the implementation), and then tested for a significant amount of time before becoming widely adopted. CLOS is an example of that.
14:13:00
beach
The people in the standards committee would not just dream up some feature that they thought would be useful. And that is unlike the many relative newbies here in #lisp, who think that is a good way of defining new features.