freenode/#lisp - IRC Chatlog
Search
10:35:30
elderK
Hey guys, I was wondering how people parse binary formats in CL. I've seen read-sequence?
10:39:19
pjb
elderK: notice that you can also specify the element-type, so you can read files bit by bit, or 3-bit by 3-bit, etc.
10:39:52
pjb
(but other than 8-bit by 8-bit and possibly bit-by-bit on current hardware, it's probably implementation dependent how the bytes are mapped to the files).
10:42:48
elderK
pjb: I just figure you'd need to create functions so that you could say, read-u32 from some arbitrary position in the blob that you've loaded.
10:43:55
phoe
you could read the code for https://github.com/rpav/fast-io to learn how it works - it has functions for {read,write}[u]{8,16,32,64,128}{-be,-le}
10:44:03
pjb
yeah, bit is not portable or useful. clall -r '(with-open-file (out (format nil "/tmp/foo-~A" (lisp-implementation-type)) :direction :output :if-exists :supersede :if-does-not-exist :create :element-type (quote bit)) (write-sequence #(1 0 1 1 0 1 0 0 1 1) out) (file-length out))' ; ls -l /tmp/foo-* ; for f in /tmp/foo-* ; do od -t x1 "$f" ; done
10:44:31
pjb
clisp writes bit by bit, but writes first a 32-bit little endian file-length (in number of bits). the other implementations write one bit per octet, which is useless.
10:45:06
pjb
This is why the usual advice is to use (unsigned-byte 8) as element-type, and format the binary file yourself, octet-by-octet.
10:46:57
phoe
AFAIR it's pjb's script that calls the following code with all CL implementations available on the machine
11:33:34
random9899
found some good youtube vids about lisp too https://www.youtube.com/channel/UCufkSSu1trzm9nB-jYOPuvw and http://www.uio.no/studier/emner/matnat/ifi/INF4820/h17/timeplan/index.html#2-2
15:58:29
jmercouris
here's the top of the makefile: https://gist.github.com/4de76bd2e6250b5a8d54588e50c60a45
15:59:04
pjb
jmercouris: http://www.talisman.org/~erlkonig/documents/commandname-extensions-considered-harmful
15:59:18
phenoble
I'm currently learning CL. Any recommended resources on learning CL debugging techniques? I finished Seibel's "Practical Common Lisp" today. Nothing about debugging in there.
16:01:03
pjb
You may have extensions on directory names, when they're "file packages", as implemented on NeXTSTEP/OpenStep/MacOSX/iOS.
16:01:06
beach
phenoble: Unfortunately, the debugging tools available for free Common Lisp implementations are not so great.
16:01:46
pjb
phenoble: I have a little tutorial for clisp debugging. https://www.cliki.net/TutorialClispDebugger
16:01:54
beach
phenoble: Start by putting (proclaim '(optimize (debug 3) (speed 0) (safety 3) (compilation=speed 0))) in your start-up file.
16:02:06
phenoble
beach, jmercouris: that's unfortunate. Though I got the impression that some "trace" functionality would be a thing in CL debugging?
16:02:42
pjb
phenoble: not all implementation implement the STEP operator, or starting a trace from the debugger. For this, have a look at my conforming cl-stepper.
16:03:04
beach
phenoble: Otherwise, you need to stick a (break) in the place where you want to stop, then C-c C-c to compile that function, then run your program.
16:03:48
beach
phenoble: From the backtrace, you can then find local variables, and you can use the SLIME inspector to inspect them.
16:04:56
beach
phenoble: Most of the time, though, you run the program until it fails, look at the backtrace. Figure out why things were called the way they were. Hit SPACE in front of your stack frames, look at variables, inspect, etc.
16:05:22
pjb
phenoble: the thing is that since we can easily just redefine a function while running the program, lisp debugging techniques are often source-based. You just edit the source of your function, adding print and break and other forms to help you debug it, re-evaluate the function, and go on executing.
16:06:22
phenoble
beach: Yes, the standard is to be expected. Though I would not've been surprised if CL offered something extra-nice in conjunction with e.g. slime.
16:06:30
pjb
The only case where it's not possible, is when your function is on the call stack (it's being executed), such as a main loop, or when its calls are optimized (it's inlined), notably when it's called from other functions in the same file. For this, you have to declare it notinline and recompile the file.
16:06:53
pjb
(Probably a (debug 3) would make all functions notinline, check your implementation documentation).
16:06:55
beach
phenoble: It probably does, but you would have to pay for a commercial implementation.
16:07:32
beach
phenoble: There has not been enough manpower for the free implementations to become excellent when it comes to debugging techniques.
16:07:47
phenoble
pjb: I see. Yes. That's how I am increasingly working with elisp. Though my implementations are not all that nested, or dependent on each other. I could well imagine that even this rather comfortable style can get to its limits. So more sophisticated tools might become necessary.
16:08:27
MichaelRaskin
Also, Common Lisp has macros, so you can quickly add some interesting tracing where needed. Like writing a special trace-let macro that shows all the bindings as they are made
16:09:13
phenoble
MichaelRaskin: showing all the bindings as they are made... does that count as a modification of the compiler?
16:09:54
beach
phenoble: Yes, one way of looking at macros is as a means of programming the compiler.
16:11:04
phenoble
beach: I can imagine. I have a bit of a background in C++ metaprogramming, so I'm not all too new to the rationale behind it. Though this in particular, changing the way bindings are done or the like, intrigues me very much.
16:11:33
pjb
phenoble: have a look at OpenC++ https://www.informatimago.com/articles/life-saver.html
16:12:30
phoe
with Lisp macros, you simply accept some S-expressions and then output some S-expressions
16:12:40
beach
MichaelRaskin: For debugging, I would prefer not to modify the source. For one thing, it takes time to insert the code, and then I often forget to take it out. And I certainly don't want my "production" code to have debugging information in it. But it is also sometimes the case that the problem disappears when the code is modified that way.
16:13:41
phenoble
pjb: interesting link to this OpenC++ project, thanks! Most likely not an option in my professional environment though. We have our tools in place, and some momentum with it.
16:13:46
beach
MichaelRaskin: But, as pjb pointed out, that is often the only way to do it with the tools at our disposal.
16:14:05
phoe
beach: in theory, you could use the CLtL2 environments to insert additional debugging information into code only when DEBUG is 3, for example.
16:14:46
phoe
And you could do this without touching the compiler - just inside macros and/or compiler macros, because you have the ability to query &env for declaration information.
16:14:47
pjb
There's also the elpp proof-of-concept (a copy at https://github.com/informatimago/elpp ) this is an emacs lisp pre-processor, which allows you to define macros in emacs lisp to generate code for any programming language you have to use.
16:14:51
MichaelRaskin
I am too used to putting debug printing as a way of understanding the code (not only in Lisp)
16:15:09
phenoble
phoe: I know, yes. That's why things like "binding code in some trace printing" is considered normal in CL I suppose. Rather impossible in C++ (I suppose).
16:15:56
MichaelRaskin
beach: well, at least this is a tranferrable skill, I can read strace output comfortably, too
16:16:45
phenoble
beach: Interesting point about not modifying the source for debugging. I appreciate very much that this is so easy in lisp(s).
16:17:09
phenoble
beach: You write "with the tools at our disposal"... - that sounds like you had better tools in mind. That so? Like what?
16:17:41
phoe
beach: (defmacro foo (...) (if (= 3 (second (find 'speed (declaration-information 'optimize env) :key #'first))) '(progn (print-debug-stuff) (%foo) (print-more-debug-stuff)) '(%foo)))
16:17:59
phoe
where declaration-information is provided via trivial-cltl2 and described at https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html
16:18:57
phoe
beach: yes, but here I assume that FOO is a macro that is already used somewhere in the code.
16:19:27
MichaelRaskin
Hm, I guess I have all the moving parts to inject tracing the bindings without changing the source code, just with a recompilation.
16:19:42
beach
phoe: So then I would have to modify every function or macro that I use in the same way?
16:20:39
phoe
beach: the ones you want to debug, yep. Quite possibly it can be wrapped in a macro to hide the syntax, but this implies that you need to wrap your functions in special debugging sugar to be able to utilize this behavior at DEBUG 3.
16:20:42
beach
MichaelRaskin: That's another thing. I usually don't want tracing. I want a breakpoint where I can stop, examine the variables using the inspector, step from that breakpoint to some other place, etc.
16:21:40
MichaelRaskin
Well, injecting tracing and injecting breaks are comparable implementation-wise
16:22:35
pjb
shka_: not really. There's may one or two languages that implement actual message passing.
16:22:37
beach
MichaelRaskin: Sounds good. Now you just need to wrap the entire thing in an interactive application called a "debugger".
16:22:47
MichaelRaskin
Traces give me time-dimensions (with restrictions on breadth), breakpoints give broad access space-wise, but in a limited amount of points in time
16:23:02
pjb
shka_: what all the OOP implement (including smalltalk!) are function calls with some dispatching.
16:24:02
pjb
shka_: now you can still use a "message passing programming style". For example, don't let your method compute stuff and return results. Instead, if there are results to be obtained, let them send back a message to some other object.
16:24:31
beach
MichaelRaskin: Yes, I understand. Then the current debugging environment that is available to us should suit you just fine.
16:25:04
pjb
shka_: but you have to be careful, because you code can still have implicit synchronization expectations.
16:25:38
pjb
Those expectations are met until you change the implementation of your methods and "order" of message sending changes…
16:25:43
MichaelRaskin
Yes, pure SBCL debug REPL plus some macros added as-needed mostly satisfy my needs.
16:26:38
pjb
Well, multiple dispatch is rather an orthogonal point, and can trivially mapped to single-dispatch, so there's no reason to avoid it, when pertinent.
16:26:59
shka_
It seems like if you want to compare CLOS to something in vein of C++ you better of put table to compare those two
16:28:13
jmercouris
ASDF mentions a link-farm: https://common-lisp.net/project/asdf/asdf/Configuring-ASDF-to-find-your-systems.html
16:29:29
MichaelRaskin
A place with symbolic links to files scattered across a complicated directory structure
16:31:00
pjb
But it's as easy to have functions to fill asdf:*central-registry* with the directories where your systems are stored too.
16:31:55
jmercouris
Is that the question? I thought we weren't supposed to attempt to redefine asdf:*central-registry* directly anymore
16:33:34
pjb
eg. I still cannot use llvm. I don't see why it should have a different user interface than gdb (and why I should get python backtraces each time I launch it).
16:34:32
pjb
Anyways, this is the big advantage of CL, it hasn't changed since 1989 and is mostly compatible with everything done worth doing since 1959…
16:35:12
jmercouris
anyways, I'm not interested in getting into this argument, gotta go, thanks for the info!
16:35:23
pjb
This means that the little code we write now with the few resources we have, gets a chance to be usable in 20 or 50 years…
17:35:30
puchacz
hi, if I pass readonly that is known at compile time, this macro gives me deleting unused code warning (obviously): (defmacro <textarea ((readonly) &rest rest) `(if ,readonly (<:textarea :readonly "readonly" ,@rest) (<:textarea ,@rest)))
17:37:58
stylewarning
puchacz: try doing (if (constantp readonly) <do explicit if> <expand into if like above>)
17:42:15
flip214
`(<:textarea :readonly (if ,readonly "readonly" "not-readonly") ,@rest) and let the compiler be smart enough for constants
17:43:22
flip214
I guess that at compile time _more_ "constantp"-ness is known than at macro-expansion time...
17:44:53
stylewarning
The point isn’t that the compiler can or cannot figure something out, it’s that it *is* working but identifying dead code and annoyingly emitting messages about it.
17:45:37
puchacz
well, the flip214 solution muffles compiler OK, I only wonder if readonly="not-readonly" is a standard HTML
17:50:06
flip214
is that true currently? > (At present, no companies or consultants wish to advertise paid support or custom SBCL development in this manual).
17:52:20
pjb
puchacz: you seem to be wanting a function, not a macro. Why is this <textarea operator a macro?
17:53:36
pjb
puchacz: also, the recursive calls don't have the right form. The macro takes a list containing readonly. The recursive calls don't pass such a list!
18:41:09
rumbler31
beach: re: debugging, what tools have you used that are not available in free implementations?
19:02:58
pjb
rumbler31: what tool would you trust that 1- you haven't programmed yourself. 2- that are not at least available as free software?
19:24:11
ealfonso
how can I disable ldb from lisp? is ldb an sbcl-specific feature? I'm trying to use fmakunbound but can't find the right function.. is it SB-KERNEL:%LDB?
19:27:11
ealfonso
also, is there a simple way to intentionally trigger ldb to test this? other than requesting a lot of memory?
19:29:39
ealfonso
yeah. thanks. (disable-debugger) works since I'm using it from buildapp to create an executable
21:57:54
ealfonso
what is the best way to understand which objects are consuming the most memory in the heap? is it the sbcl statistical profiler?
21:59:10
aeth
I use sb-profile and if you break things up into small enough functions it's pretty easy to see where the allocations are happening.
22:02:38
ealfonso
if I'm interpreting correctly, statistical profiler or sb-profile gives me a 'consed' column or the amount of new allocs, but does not account for gc'd memory. I have a lot of functions that seem to alloc memory temporarily. what I'd like to see is which memory has been allocated and not gc'd (e.g. to optimize long-running process slow memory leak)