freenode/#lisp - IRC Chatlog
Search
19:16:49
emaczen
I don't have experience with gdb when writing a C program either... but thats what you usually see
19:17:11
makomo
johnjay: use emacs along with org-mode to write your notes and git to version control them, that's about it :D
19:18:08
emaczen
johnjay: ask in #emacs but look at magit mode for convenient key bindings for common git commands
19:19:53
johnjay
makomo: i thought you had a system for annotation source code which i've been thinking about lately
19:20:21
johnjay
i hate the bs inline stuff of javadoc and oxygen where you put @doc_string_with_long_name lots_of_text every other line
19:21:38
makomo
johnjay: ah, nope. another benefit of having your notes in plain text is that they're easily searchable using tools like grep, etc.
19:22:13
johnjay
makomo: i did come up with one idea though. You've heard of literate programming right?
19:22:48
johnjay
instead you make a literate prog file which has notes, and mentions only a subset of lines of the source
19:23:01
johnjay
then when you process it you just match it line-by-line to the source file to check it
19:23:34
johnjay
so for example if you have lines A,B,C,D in your file then your literate notes produce only A and C
19:23:50
makomo
johnjay: what does "check" here mean? if i'm understanding correctly, wouldn't you have to maintain the alignment of the positions of text within both of the files?
19:24:28
johnjay
the idea is not to be perfect but just to have some kind of notes that can match the source
19:24:56
makomo
like literally have parts of the source in those "note files", just for the purposes of matching?
19:25:20
johnjay
if your file is #include int main() { printf("hello"); printf("world"); return 0; }
19:25:30
jcowan
pfdietz: Technically a program can change state and still be functional if the state is not visible to the program
19:25:47
johnjay
then your literate notes might say "this is a file that prints hello with the lines printf(hello); printf("world");" then it matches those two
19:25:56
jcowan
a program that does a debug-print operation on the console is functional if (as usual) there is no way to read back console output
19:26:02
johnjay
the only edges case i thought of is what if you repeat a line which changes something
19:26:30
johnjay
makomo: right because you can add or delete lines in a source file, that's relatively common with source control
19:27:05
makomo
johnjay: right, but you still have to keep them in sync, although not fully, just in parts
19:27:24
johnjay
if you just have the lines hello and world from my example you can still keep it in sync
19:27:48
jcowan
another case is a program that has been instrumented to update a profile counter when some function is called; since the profile counters are invisible to the code, the functions are still functional
19:28:34
makomo
johnjay: i guess the details would then depend on the matching algorithm or whatever
19:29:30
johnjay
e.g. between printf('hello') and printf('world') 10 lines of Qt code were added or something
19:30:47
makomo
same, but i always wonder whether literate programming is feasible. it seems like a good idea to me, but i've never actually tried it on a bigger project
19:31:12
johnjay
yeah that was why i thought this might work, because with literate programming any change necessitates updating the literate source
19:31:27
johnjay
but maybe if you ask for less you can use it this way, to just kind of maintain some basic notes on something
19:32:56
makomo
it's a pretty nice talk, i've watched it multiple times https://www.youtube.com/watch?v=Av0PQDVTP4A
19:34:19
johnjay
i guess the essence of my idea was that the more fixed the code the more literate you can make it
19:34:57
johnjay
so maybe you could have 3 levels: full lp for old unchanging code, my hybrid version for code that's mostly done , and then like notes matching to specific line numbers for WIP
19:35:46
makomo
the hybrid version that uses matching could somehow work, i guess. but the last approach would be too burdensome i think
19:36:21
makomo
right, but that's even worse, because then you have to keep changing those line numbers
19:38:55
francogrex
hi can I make a region executable by using mprotect? for example if i use cffi foreign-alloc or make pointer, can I make it protected and executable?
19:39:28
johnjay
there's a game programming project i'm familiar with and i despair of any kind of documentation for it
19:40:00
johnjay
basically you have constant churn as bugs are discovered and fixed so nothing is really "safe" from modification
19:41:30
makomo
johnjay: yeah, it gets hard to keep up the "book" up to date with so many modifications
19:42:20
makomo
that's why i feel like literate programming is great for stuff that has been "solved", i.e. the idea was thought through, a good design was developed and it won't be changing much
19:42:43
makomo
there are a few literate books: knuth's tex, elliptic curve crypto, understanding mp3 and physically-based rendering
19:43:22
makomo
all of them talk about something "solved", i.e. something that has been analyzed, discussed, specified, etc.
19:44:27
jcowan
Sure it does. You just have to accept down in your bones that changing the prose is just as important as changing the code.
19:44:32
myrkraverk
johnjay: I've compiled noweb on arcane systems before, and I don't remember if I had any problems.
19:45:40
makomo
also, that, yes. the demands for LP are a bit higher than usual. you have to be a good writer
19:46:01
myrkraverk
jcowan: it's a different skillset. Many programmers can't write comprehensibel english.
19:50:50
johnjay
it specifically is only using lp in small pieces for code you know fairly well won't change
19:53:50
minion
nixfreak, memo from Jachy: https://github.com/t-sin/one (example of using CL with bash one liners)
19:55:26
pjb
nixfreak: also, with normal REPL you can use (DRIBBLE "file") but it doesn't work with slime since it uses custom threads to implement the REPL itself. One could patch swank to log the expressions (and results) like with DRIBBLE.
20:05:36
pjb
nixfreak: with slime, you have the whole slime buffer keeping the history. You can also just save it with C-x C-s
20:12:44
void_pointer
Only thing to be aware of is that when you use a previous code at the REPL, you can't add newlines to it
20:19:52
fiddlerwoaroof
emaczen: typically one doesn't use gdb on a common lisp program, unless you're doing ffi stuff
20:22:41
flip214_
Can I get the current input position (in bytes) from CXML? I'd really like to create an index (tag => position, length) into a large XML file, and then do "random" IO on it.
20:23:32
fiddlerwoaroof
Well, you can try starting gdb and then using attach <PID> to attach to an existing process
20:24:21
fiddlerwoaroof
However, I've found that various things about sbcl and ccl can make this really painful
20:25:06
fiddlerwoaroof
You can also do `gdb my_program` and then `run may arguments ...` inside gdb
20:26:10
fiddlerwoaroof
Finally, there's `ulimit -c unlimited; my_program; gdb my_program core_file` which lets you inspect the state of a process at the time that it crashed
20:26:48
fiddlerwoaroof
The most useful gdb command is `bt` that shows a backtrace of the program being debugged.
20:35:21
flip214_
jasom: I tried to use evil mode a few times in the past... but there's too much that doesn't work to make me comfortable.
20:37:37
emaczen
Does mallocing a pointer inside a lisp function and freeing it from the calling function cause a memory leak?
20:39:04
Bike
a memory leak is when you fail to free memory. so if there's some way for things to go so that the freeing never occurs that's a leak
20:39:15
void_pointer
As long as it is freed somewhere, it shouldn't cause a memory leak. That all said, in that case (as well as pretty much any case), you do need to make sure you have things setup such that it will for sure be freed once but no more than once because double free doesn't go well
20:39:29
Bike
for instance, if the mallocing function signals an error, which goes to the debugger, and then you quit the debugger back to the repl
20:41:26
void_pointer
or if you have the pointer in some containing object and put a finalizer on the containing object that has a copy of the pointer it will run free with (one does then need to be careful to not free it manually without removing the finalizer or otherwise disabling it in order to avoid double-free)
20:42:25
flip214
well, freeing in the outer function has the problem that the pointer to the memory-to-be-freed has to be communicated somehow... and if the inner function crashes before that, the outer won't know what to free
20:42:47
void_pointer
another issue is if the pointer points to a structure that itself has malloc'ed pointers. Freeing the first pointer doesnt reclaim the memory of those it was pointing to
20:42:51
emaczen
Well my this function returns a pointer, and then at the end of the calling function it frees the pointer -- I don't see how I can leak that much...
20:45:19
void_pointer
The one exception here is if you are using something like bdwgc and you use its malloc in which case you would be OK. If your implementation uses bdwgc or something similar and you are using the malloc the implementation provides, then there is a good chance you are using its malloc instead of libc malloc.
20:46:02
flip214
void_pointer: another exception is using hierarchical memory pools, like subversion does.
20:46:09
void_pointer
ECL uses bdwgc last I checked. SBCL, CCL, and CMUCL don't. Don't know about Clisp or Clasp
20:46:52
nirved
void_pointer: in slime repl - delete the last closing paren, and then can edit the rest with adding newlines
20:47:28
Bike
emaczen: if this is leaking, there is a path from the malloc that does not lead to the free, which you will have to find and excise.
20:47:33
void_pointer
emaczen: then unless you are one one implementation only, you can't assume much. Also, that is good you don't have to worry about pointer nesting.
20:50:24
fiddlerwoaroof
I think there's a way to transparently replace malloc with one that warns you about unfreed memory
22:50:21
oni-on-ion
the 'random bits' of memory, is this like that from the computer just powering up the chips? i mean, they dont start at 0 right, they've got to be 'set' in software?
22:50:54
phoe
in general, memory can be allocated and freed in arbitrary parts of the program; it's hard to tell malloc "warn me if I don't call free on this piece of memory by the time I leave this lexical/dynamic/??? scope" unless you employ other tactics, such as unwind-protect
23:29:25
no-defun-allowed
i made #netfarm (which is bridged to the matrix #netfarm:matrix.org) cause most of the talk around it isn't useful to the average lisper imo
2:39:37
astronavt
i mean, allegro, clozure, and lispworks have both employees and clients. so in theory yes? i know at least a few ppl here are lisp programmers at work
2:42:35
greenb
There's been a Scheme->WebAssembly compiler for some time now, but IDK if anyone uses Scheme (which is a shame, because it's strictly superior to Python other than in library count AFAICT).
2:45:36
greenb
I can't wait until Guile gets an AOT compiler or tracing JIT in v3.0. It does have an active community, but I meant use in the workplace.
2:50:41
greenb
astronavt: Nothing, just that you can't get a job writing in them. Well, they're not Kernel, but neither is CL.
3:10:58
greenb
astronavt: The Racket people found out how to add types to Lisp. See "Type Systems As Macros".
3:11:36
greenb
There's also the earlier Typed Racket, but that's a hairy mess of interleaving hygienic macro expansion with runtime evaluation.
3:12:41
greenb
However, an analysis of Typed Racket (which is gradually typed) showed that both pure dynamic typing and pure static typing were faster than a mix thereof.
3:13:15
astronavt
im not surprised. im also not sure i need dynamic typing, just good enough inference that i dont need to declare types everywhere
3:14:00
greenb
IDK if this result holds only for the peculiar Typed Racket system. I think "Type Systems As Macros" is applicable to unhygienic macro systems (but don't quote me on that).
3:14:22
astronavt
" To implement the type system, programmers write type checking rules resembling traditional judgment syntax. To implement the semantics, they incorporate elaborations into these rules"
3:14:56
greenb
For what it's worth, I don't know that static types are what they're hyped up to be. There's a good argument against them here: http://fexpr.blogspot.com/2011/11/where-do-types-come-from.html
3:49:30
fiddlerwoaroof
The best attempt at a typed lisp I've seen so far is Hackett ( https://github.com/lexi-lambda/hackett )
3:50:06
fiddlerwoaroof
There's a Strange Loop talk from this year that explains some of the decisions behind it
4:36:19
aeth
fiddlerwoaroof: I would much rather have an extension to Common Lisp than a separate typed Lisp. Sort of like how Typed Racket can use all of the existing Racket code that's out there instead of starting from 0
4:37:32
aeth
You would just need to be able to handle typed functions (ftype exists, but is not that useful atm) and have improved non-generic data structures (such as better typed conses)
4:42:40
LdBeth
Otherwise you have to start from applicative Lisp, which has significantly less features
4:43:14
aeth
LdBeth: You can extend Common Lisp more easily than Scheme because you can do it mostly in Common Lisp
4:45:32
aeth
Anything's possible, but if your Scheme is 90% C that means (probably) writing most of your new features in C.
4:52:04
aeth
LdBeth: Yes, but it wouldn't be a new implementation. Adding a good static type system to SBCL would be no more involved than adding threads or unicode. (The other implementations might be trickier. SBCL already is half of the way to being a statically typed lisp.)
4:54:18
jackdaniel
aeth: that's some weird and inaccuarate claim for a language which is specified to be dynamically typed ;/
4:55:27
jackdaniel
I'm afraid you mix terms. Gradual typing allows you to have some variables typed, static typing is when all types are known at the compilation time
4:56:46
jackdaniel
no, terms in this case are not a matter of opinion, please do not use them inproperly because it only increases the confusion
4:58:30
jackdaniel
(regarding SBCL having a support for gradual typing – I have no information on that topic)
5:01:44
aeth
interestingly, Wikipedia calls it "soft typing", not "gradual typing". https://en.wikipedia.org/wiki/Type_system#Combining_static_and_dynamic_type_checking
5:05:18
aeth
jackdaniel: As to how it does gradual typing: Most of SBCL's static type checking is done within one file (or within one with-compilation-unit). It also has sb-ext:*derive-function-types* which defaults to NIL that assumes function types never change (which is NIL because it's non-standard behavior).
5:06:55
aeth
It's afaik based on the function metadata, so if it knows at compile time that the x in (sin x) isn't a number it can complain (since functions in the CL package can't be redefined, it always behaves like this for functions in the CL package). e.g. (defun foo () (let ((x "foo")) (sin x)))
5:08:41
jackdaniel
I would be inclined to believe, that this one is a matter of type inference which is perfectly fine within dynamic typing paradigm. I need to go now though
5:11:25
aeth
You can DECLAIM FTYPE for the function or DECLARE TYPE for individual input variables to the function and it will use those in the same system. e.g. (with-compilation-unit () (defun foo (x) (declare (integer x)) (sin x)) (defun bar () (foo 42f0)))
5:12:38
aeth
Imo when it infers the ftype, it's type inference, when it respects the type declaration it's static type checking. (foo 42f0) will work perfectly fine in CLISP and will probably be caught only at runtime in many implementations that do something with type declarations.
5:15:12
aeth
You could also have it catch something like (in case 42f0 is too type inferency): (with-compilation-unit () (defun foo (x) (declare (integer x)) (sin x)) (defun bar (x) (declare (single-float x)) (foo x)))
5:21:21
LdBeth
aeth: Although I agree there will be no necessity distinguish static or dynamic types in foreseen future, and I think I know what you mean, that is not the thing I expected to be well integrated with CL. For example, how will the method dispatch work in such a type system?
5:21:37
aeth
What's missing is (1) a way to store non-T (just structs and specialized-arrays and declared/declaimed variables afaik) and (2) a way for you (not the compiler) to do something meaningful with ftypes.
5:22:15
aeth
LdBeth: There's always https://github.com/markcox80/specialization-store/ but not everything is exposed to it by the implementations (and what is varies).
5:24:13
aeth
(oh, I forgot that you can declare conses, I think the way it's done requires walking through the list to verify each time? Instead of just having a foo-list that you can just check the first element since its cdr must be foo-cons or nil)
5:26:15
aeth
LdBeth: Anyway, I personally think you could (for the most part) add the useful stuff from modern static type systems to an existing Lisp (which would make the result gradually typed! I'll concede that!) and have it just work alongside what's already there, just like they added the useful stuff from modern OOP to existing Lisps in its own, Lispy way.
5:30:08
astronavt
i just like types because, as a programmer, it means i need to keep less in my brain at once
5:30:42
LdBeth
Blah, I don’t think there’s any wrong using a existed implementation, but I discourage to actually do it without some common agreements on how would the type system work
5:33:24
fiddlerwoaroof
But, the "types as macros" paper focuses on a DSL for describing the type systems of DSLs while Hackett is trying to be a Ha(skell) built on top of (Ra)cket[t]t
5:33:24
aeth
LdBeth: I disagree what you say about gradual typing being useless, btw. Everything has a static type, it's just usually T. You could still do useful things as long as the part of a program that heavily uses types don't have to break out of it. Right now, you would because you basically can, with few exceptions, only store T. Put an integer in a hash table and then take it out in that same function and SBCL will forget it's an integer iirc.
5:35:03
fiddlerwoaroof
Hackett is interesting because the types are about more than correctness, because it has bidirectional type inference, the types can be used to pick which implementation of your functions to use and are not limited to just checking correctness.
5:35:32
LdBeth
If everything is T, having gradually type doesn’t make a difference on optimization or safety
5:36:07
aeth
LdBeth: As it is right now, SBCL can infer or use the declarations for a lot of things, especially numbers and arrays.
5:36:51
fiddlerwoaroof
i.e. in most type systems, if you erase the types, the program still makes sense. With a haskell-style type system, the types can be used as a kind of "global" macro
5:37:17
aeth
It would be a very unlispy thing to *force* people into declaring types, though, even if you *could* make a program that was 100% known-at-compile-time types (which you can't really do right now except for numeric code)
5:37:33
aeth
Lisp is about giving you everything and then if you want to have a certain style you discipline yourself into using that style.
5:38:03
aeth
Forcing people into static type declarations for static type checking would be very unlispy, but optionally permitting everything to have known-at-compile-time types isn't unlispy.
5:39:40
astronavt
in some cases you can make the program run faster by declaring types, which helps the JIT compiler optimize
5:40:11
aeth
LdBeth: SBCL is basically there right now with inference except (1) the main boundary is the function, (2) the even larger boundary is the file or the compilation-unit, and (3) you'll usually lose type information once you store things into a data structure.
5:40:30
astronavt
yeah im not saying Julia did it first. just that its a high quality modern example of it working very well in practice
5:40:45
aeth
And types for performance are basically only really necessary when there are generic things like + (works on any number) or map (works on any sequence)
5:41:23
fiddlerwoaroof
LdBeth: I believe type inference (at least Hindley-Milner type influence) was invented by the people that made ML
5:41:25
aeth
Otherwise e.g. even though it's T when you call CAR on foo, it's going to be a LIST-or-ERROR so the compiler can still make some assumptions
5:42:17
astronavt
i imagine you could abuse it pretty badly though, relying on IDE to tell you what's happening
5:42:23
fiddlerwoaroof
Yeah, wiki claims that ML was first: https://en.wikipedia.org/wiki/Hindley–Milner_type_system#History_of_type_inference
5:43:01
LdBeth
a first class type system, where types can be calculated and compose without a special concept of run time and compile time
5:45:09
jackdaniel
LdBeth: that makes no sense. what you call "analysis of program" I call a phase of compilation (just like when macros are expanded)
5:46:12
aeth
macros are in their own phase because they get you 95% of what you want with the alternative (what are they called? fexprs or something?) but with a ton more performance
5:47:13
LdBeth
jackdaniel: Lisp probably lacks fancy industrial level features like formal verification, which is where a sophisticated type system will contribute
5:47:25
aeth
(Well, potentially performant. The Common Lisp police aren't going to arrest you for making a slow Common Lisp interpreter.)
5:48:09
aeth
LdBeth: Imo, you can't get perfect verification with Common Lisp because too much of its behavior relies on global *foo*s.
5:51:08
fiddlerwoaroof
LdBeth: one of the major formal verification systems uses a subset of common lisp
5:52:12
aeth
LdBeth: I guess you could watch the globals to see if anything set them if you don't want to just subset CL.
5:52:51
fiddlerwoaroof
Yeah, it's total (i.e. every function must produce a value in a finite amount of time) so you don't have to deal with the halting problem
5:54:38
fiddlerwoaroof
Yeah, I've been putting off the Mojave upgrade until I'm confident that people have worked out the issues
5:55:59
LdBeth
the latest development snapshot compiles, but sometimes quit under high memory pressure.