freenode/#lisp - IRC Chatlog
Search
19:52:45
|3b|
(well, sbcl has some heuristics to try to avoid compiling trivial things, but generally it compiles everything)
19:53:36
|3b|
it does some things like disabling the debuggers, and exiting when it finishes instead of leaving you in a rfepl
19:54:55
|3b|
on errors you want it to just print some message and exit, so disabling debuggers is good
19:55:43
margaritamike
so instead of sbcl --dynamic-space-size {memlim} --control-stack-size 64 --load {path}
19:55:50
margaritamike
i want sbcl --dynamic-space-size {memlim} --control-stack-size 64 --script {path}
19:57:23
margaritamike
i need to push changes to that repo to reference --script instead of load, though
19:59:27
jcowan
elderK: The main thing with SATISIFIES is to be sure the predicate can accept *any* object and not, say, just numbers.
20:00:27
jcowan
so you may need to say something like (and (numberp x) (foo x)) in the definition of the predicate (it won't work to use AND in the type expression because it is unordered)
20:02:37
elderK
jcowan: Thank you jcowan. Knowing this, is there any reason to use and in combination with satisfies in a deftype?
20:03:21
jcowan
(and (satisfies thisp) (satisfies thatp)) comes to mind. Just make sure there is no dependency between the two
20:03:52
jackdaniel
if object is known to not be of type foo, then you may decide without invoking arbitrary function
20:04:16
jackdaniel
i.e (and integer (satisfies bloop)) ; bloop may be expensive, but we may know that #\c is not an integer
20:28:45
aeth
jcowan, elderK: Depending on the compiler, (and (integer 0 32) (satisfies foo)) might be the way to go even if #'foo then repeats the test because now the compiler knows it's an integer from 0 to 32.
20:31:45
aeth
(You'd pay more in the test, but get faster generated code in those two cases, and maybe just those two cases. I'm counting all numeric types that aren't bignum as "non-bignum numbers" here, not just fixnums.)
20:52:28
margaritamike
Anyone know if common lisp is a suitable language for cyber security, or know of instances of it being used for such?
20:53:13
margaritamike
I see python is starting to be used as a generic scripting language as a replacement for bash and windows scripting
20:54:06
margaritamike
i wonder if common lisp could do all the things python does here https://github.com/Gallopsled/pwntools
20:56:04
margaritamike
Inline: in what sense then? Obscurity? In what since do you think i'm referring to? I was asking in terms of being a fast automation tool as a replacement for what python is typically being used for in cyber right now
21:18:24
Xach
i like working with real objects and real conditions and stuff. sbcl's run-program makes it nice and it is easy to wrap to make common things simpler.
21:19:53
aeth
It depends on the length of the program. Nothing beats bash for one-liner scripts. bash does in 1-20 lines what anything else would do in hundreds. Once you're doing something substantial, though, I definitely wouldn't want to touch bash. (I say bash because bash's extensions make sh more usable so usually such scripts specifically run bash)
21:20:36
Xach
I'd really really like to wrap the pty capability of run-program into something like Expect but for SBCL.
21:20:57
Xach
unfortunately the pty stuff seems to be broken on macos for me, and that's where i do 99% of my hacking to start.
21:28:42
v0|d
I dont prospect anybody building a replacement for autotools in near future, hence bash will stay as it is.
21:43:19
jcowan
With the drastic shrinkage of platforms and increasing standardization, autotools is IMO obsolete. Set up the sensible cases and if someone wants to run it on their VAX running 4.0bsd, tell them to provide their own Makefile.
22:36:00
elderK
Man, I'm having trouble wrapping my head around what I need to be available at compile time, load time and execute time :|
22:36:27
|3b|
ACTION thinks it usually isn't worth bothering, since those times tend to overlap too much
22:37:04
elderK
Like, I've written a bunch of stuff. And, most of my macros expand into stuff that doesn't have an eval-when. Because, well, it's not necessary.
22:37:43
elderK
But for another macro, my define-binary-structure, I need a bunch of that information available at compile-time.
22:40:33
jcowan
elderK: If a function is called by a macro, wrap it in triple eval-when; otherwiser not.
22:41:22
elderK
The thing is, like, so far at least, all these "type metathings" are being added to a hash table at runtime, I guess.
22:42:03
elderK
It would be smarter to persist that information, if you say, save your stuff to a FASL.
22:46:07
|3b|
it isn't quite like a .o file in C, since loading it still has all the side effects you would get from LOADing the original .lisp file
22:46:24
elderK
And, like, let's say you ARE doing a bunch of stuff at runtime that's costing a lot of time. Now for a binary-types type library, a lot of the types a user defines will be known when they compile their program. That information could be persisted, and somehow... thawed on load, without having to go through re-executing all the stuff to recreate hte metadata. If you /did/ wanna persist it, so that loading was
22:46:41
|3b|
and it can store some fairly complex data structures, which requires more effort than just mmapping it into memory
22:49:54
|3b|
probably depends on specifics, maybe just a variable in .fasl, maybe in a separate file
22:51:33
|3b|
for example in the case of binary parsers, it would be complicated since you'd probably have data from a bunch of different files
22:52:37
|3b|
but maybe you could do something like caching them per thing that uses it in a way that was quick to merge into the main data set
22:58:51
elderK
:| Man. What started off as something so simple (define-binary-structure), I've wond up spending like, four hours just thinking...
23:04:17
jcowan
Is there any straightforward way to load the whole of Quicklisp so it can be searched?
23:05:44
pfdietz
The problem is that various systems in quicklisp are incompatible, due to package name collisions.
23:06:16
elderK
|3b|: This is my first "real thing" other than, like, the school assignments I was redoing.
23:06:35
elderK
|3b|: I don't have any ambitious things in my little binary library, just the same stuff that's in binary-types really.
23:06:55
pfdietz
I have 'kind of' loaded all of quicklisp, but only in a quick and dirty way to get stuff that looks like lisp for mutation testing purposes. This doesn't have to really read the lisp, and bails out if it encounters any reader errors.
23:07:00
elderK
Just, it's my one. And it has a few things different that are more suited for what I want to do, and how I Want to use it
23:08:27
pfdietz
I think it should be possible to do better, using Eclector and being smarter about things. But in general reading a lisp file may depend on setting up arbitrary lisp capabilities for the reader macros.
23:09:01
Xach
I think the way to go is to load each in turn and use sbcl's introspection to dump a searchable offline database.
23:09:27
|3b|
elderK: yeah, just a potentially complicated task if you want to handle all the odd things various binary formats do
23:09:47
pfdietz
It was pretty hacky. I had to be able to handle undefined packages, so I hacked the SBCL reader to just use *package* when it found a package it couldn't recognize.
23:11:13
pfdietz
I put in a ticket to enhance the SBCL reader so there were some useful restarts for the package not found error, but that hasn't happened yet.
23:12:23
Xach
I've mentioned it before, I'd really like a query system for the image so you could ask things like "show me all functions with two arguments" or "show me all functions with an argument named DOG" or other things (not necessarily only about function argument lists!)
23:13:14
Xach
i think the advantage of load + dump is you don't have to try to fake loading and work around reader macros and package problems.
23:15:39
Xach
the stuff needed to support M-. and who-calls and who-references could be fun to query arbitrarily
23:22:52
fiddlerwoaroof
elderK: I wrote a little thing that's sort of like what you're doing, if I understand correctly
23:25:22
fiddlerwoaroof
The usage looks like this: https://gist.github.com/fiddlerwoaroof/dc1cfe4738a9d82d48bce6046af53fbe#file-zipfile-lisp-L113
23:25:39
fiddlerwoaroof
Here's how I define the format: https://gist.github.com/fiddlerwoaroof/dc1cfe4738a9d82d48bce6046af53fbe#file-zipfile-lisp-L81
23:25:59
fiddlerwoaroof
And here's the entrypoint of the parser: https://github.com/fiddlerwoaroof/fwoar.lisputils/blob/master/bin-parser.lisp#L78
23:48:06
jcowan
Of course the code has long since rotted and was never Maclisp compatible in the first place, but it shows what could be done by introspection
23:50:32
jcowan
http://www.softwarepreservation.org/projects/LISP/interlisp-d/3100186-Interlisp_Oct83.pdf pp. 351-372 (of the PDF)
0:35:33
fiddlerwoaroof
elderK: if I had to guess, you're not properly escaping # when you call xdg-open
0:41:08
jcowan
I'm still trying to get a list of all quicklisp projects. (ql:system-apropos "") returns only 73; using quickdocs with an empty search field returns only 0. What's the righteous way to do this?
0:41:50
fiddlerwoaroof
git clone https://github.com/quicklisp/quicklisp-projects && ls quicklisp-projects/projects | wc -l
0:45:57
fiddlerwoaroof
I was wrong anyways, it looks like source-disabled.txt is used in cases where there is an old repository
0:52:55
fiddlerwoaroof
Hmm, that's going to be a bit tricky to do precisely, without reading in the source
0:57:43
pfdietz
(length (remove-duplicates (mapcar #'ql-dist:release (ql-dist::provided-systems (ql::dist "quicklisp"))))) ==> 1682
1:09:50
jcowan
okay, created a shell script that repeatedly calls sbcl --disable-debugger --eval '(unwind-protect (progn <import ql> <load project>) (quit)'
1:18:22
pfdietz
Goodness. ql-dist::dependency-tree can produce a very large tree for some systems. Need a dependency-dag.
1:31:20
pfdietz
There are just 344 systems in the current ql dist that are not in the required-systems list of another system.
1:48:42
aeth
pfdietz: It might be better to use projects as the basic unit and see if there are any projects where every contained system in that project is not depended on by any external project's systems.
1:58:27
fiddlerwoaroof
I believe ql-dist::provided-releases gives you all of the quickloadable projects
2:29:48
elderK
Guys, what is a good way to like, make sure that ... well, all stuff evals when it is meant to?
2:42:22
elderK
I see that you can for cons, but defining a list of a certain type that way seems kind of painful
2:53:46
elderK
I'm not even sure if I could like, write a nice type for a list of certain stuff - it would be recursive :|
2:56:44
fiddlerwoaroof
(defun all-strings (ss) (every 'stringp ss)) (the my-list (satisfies all-strings)
3:44:12
elderK
fiddlerwoaroof: :) So, what's a good way to check that like, all my stuff... uh, works when it should? :D
3:51:43
fiddlerwoaroof
the rule is fairly simple, though: every function a macro calls had to either be in a different file or in an eval-when
3:54:18
elderK
fiddlerwoaroof: It's just, well, I need to test that all the metadata I've gathered is availalbe at the appropriate time.
4:06:06
elderK
fiddlerwoaroof: :P Everything loads okay even if the things the macro calls, are not (:compile-toplevel)
4:06:22
elderK
:P I even forced the macro expansion in a :compile-toplevel eval-when, it still woriks.
4:08:38
elderK
fiddlerwoaroof: I'm not sure how to test this. I mean, asdf:load-system from REPL works just fine, even if I kill the REPL and start fresh
4:09:23
fiddlerwoaroof
if you compile each function as you write it, they will all be available to be called: to really test this, you either need to use a fresh lisp process or delete-package on the package containing your definitions before loading the sustem
4:10:18
fiddlerwoaroof
forming a mental model of file loading was the trickiest part of learning CL for me
4:10:21
elderK
fiddlerwoaroof: If I start a fresh SBCL, I can successfully load my system using asdf:load-system, :force t
4:10:50
elderK
fiddlerwoaroof: The thing is, it doens't make sense that this succeeds. I've purposely made some stuff not available at compile-time. The macro should fail, but it succeeds all the same.
4:17:10
elderK
Okay. So, if I am populating a hash table in one compilation unit, does that mean all of that data is visible to the next, at compile time?
4:17:23
elderK
I haven't like, got any eval-when to say "Hey, I want this stuff available at compile-time."
4:18:20
elderK
Well, yes. But I need to verify that the types people are using for slots in a "binary structure" are actually "binary types."
4:19:01
elderK
Since populating that hash table is a runtime thing, at least, atm. It'll generate code to do that, rather than do it at compile-time.
4:19:23
fiddlerwoaroof
but, im pretty sure that once a file has been compiled and loaded, you can rely on the side effects have being run
4:44:50
elderK
(Starting fresh CCL process, requiring ASDF, then load-systeming my system, ensuring it is built)
5:02:54
elderK
:P I removed all the eval-whens from my code, until I know I need them. I'm sure something will go wrong eventually and I'll need them :P
5:53:00
aeth
elderK: Easiest way to remove eval-whens is to put them in a separate file that's loaded first
5:53:32
aeth
elderK: Afaik you probably don't need an eval-when unless it's called unquoted in a defmacro (directly or indirectly, I guess) while being in the same file as that defmacro
5:54:12
aeth
Sometimes CCL needs eval-whens when no other implementation needs them. I think that's for constants used in macros
6:46:31
Necktwi
finally got rid of those ghost windows they are due to # to ## redirects of some of my autojoin channels
6:58:50
elderK
aeth: That's the thing: The macro is using functions defined in the same file, and is not encountering any issues on either SBCL or CCL.
7:00:24
elderK
Still seems weird. The macro makes several function calls, these functions generate the expansion. These helper functions call other functions, to get information that has been accumulated in a hash table. It seems bizarre that I am not encountering problems.