freenode/#lisp - IRC Chatlog
Search
16:31:28
shka_
gendl: how many languages assume that user is supposed to modify deployed application by editing the source code?
16:32:29
gendl
shka_: Ah I see what you're saying. CL applications are potentially programmable after they're deployed. Sure, that's a feature and it makes sense to take advantage of it in some circumstances.
16:32:47
gendl
But I think the question is more of an application-specific one than a language-specific one.
16:33:17
gendl
the question should be "how many applications assume that user is supposed to modify deployed application by editing the source code?"
16:34:37
gendl
The fact that CL can do both is a great testament to CL. But it doesn't mean that 100% of CL applications must therefore necessarily ship with a development and build environment built into them.
16:35:51
gendl
ok.. and that Z% of users can range anywhere from 0% to 100%, depending on the application domain and a bunch of other factors.
16:38:38
didi
I disagree with the "mainstream" statement, but I'm with Xach: an article about deploying applications would be nice.
16:39:32
gendl
I worked at Ford in ICAD in the 1990s. (ICAD was a CL based knowledge based engineering system with a solid modeler -- in principle an ICAD application could automate tons of manual CAD design). ICAD was able to demonstrate some fantastic, amazing, miraculous, unbelievable things from a development seat. But very few applications ever made it out into actual enterprise-wide mission-critical production. We sucked at actually
16:39:32
gendl
deploying anything into production. And I feel one reason for that is the environment is so geared toward development that it might even seem to actively _discourage_ production deployment.
16:42:32
gendl
And I suppose people are entitled to their opinions. This attitude of "well, CL people don't really have production deployment or mainstream acceptance as a priority" may be fact today, but nothing says it has to stay that way foreve.r
16:46:24
shka_
gendl: i honestly think that you are making this issue to appear larger then it really is
16:47:46
gendl
shka_: maybe I'm making a mountain out of a molehill. Someone said it's a matter of subtle mindset, and I think that's right.
16:48:19
gendl
But the practical ramification that started this rant is when things like `asdf:system-relative-pathname` show up in mainline application or library code.
16:50:01
gendl
didi: No. You can only think of it as a package which is available in a development mode, when all your ASDF systems are actually present.
16:51:00
gendl
If you want your application to be forever bound by the baggage of its ASDF system and source codebase, then that is of course your right to do so, but it should be done deliberately with eyes wide open.
16:51:43
gendl
alexandria:foo is not assuming the presence of a certain ASDF system in your filesystem. It's just doing some generic portability convenience thing for you.
16:52:08
gendl
And by the way, a large percentage of what's in alexandria is likely available in UIOP as well.
16:52:51
gendl
if UIOP can supersede cl-fad, alexandria, and several other little "trivial" libraries, I for one would find that a happier state of affairs.
17:03:35
gendl
Xach: I appreciate the way you state facts as you see them (whether happy facts or not), you sometimes express your desires ("I wish.."), and you avoid idle speculation.
17:03:45
didi
gendl: I see. Thank you. Personally I don't use ASDF functions, but despite morally, I can't see it been different from any other package.
17:05:09
Xach
gendl: I think sometimes there is a tendency to reject good things because they are difficult, rather than because they are bad, and wishing for good things is my way of acknowledging what is good without talking about how likely they are to come to pass.
17:05:26
gendl
didi: Morals has nothing to do with it. I'm talking about practicality, and the concept that a developer should be aware of what assumptions he/she/it is making when doing certain things.
17:05:37
Xach
"if tree shaking was so great then someone would already have done it for us, therefore tree shaking is bad"
17:09:03
gendl
pfdietz: Do you think docker will eventually eliminate the need for languages (e.g. CL) to support certain OSs directly, (e.g. Windows)?
17:11:06
gendl
so we just target a single platform and let Docker deal with all portability issues? Or am I barking up the wrong tree?
17:15:46
gendl
pfdietz: maybe this is a discussion for a different channel, but is it usually 1 application per container? And we're talking about just server applications right? Like webservers or web API servers right?
17:16:30
gendl
and the benefit is that the 1 application can have all its runtime libraries installed and guaranteed to be there and guaranteed not to conflict with other ones for other applications, etc.
17:17:08
pfdietz
I guess you could call them server applications, but there's no reason they have to serve more than one client. The web browser provides the UI.
17:18:01
shka_
my colleague mad for instance complete portable development env in docker with all the libs, gcc, and good damn web browser included
17:18:41
gendl
shka_: how would a desktop application running in a docker on a Linux VM on a Windows host show up on that Windows desktop?
17:19:17
gendl
shka_: ok, then I'm confused by your statement "you can deploy desktop applications inside docker"
17:19:59
jcowan
In Flavors packages were strictly hierarchical, inheritance was strictly up the tree, and symbols had names like FOO:BAR:BAZ (no ::)
17:20:02
oni-on-ion
pfdietz: are docker 'services' typically serving just one client ? then i can see the benefits
17:20:38
gendl
to me, docker sounds like a band-aid (maybe a necessary one) over a broken global ecosystem with a chaos of libraries, languages etc.
17:22:45
gendl
To me, a CL application is already kind of its own "container" (if it's done properly). Containerizing a CL application inside another (docker) container might be belts & suspenders. But I speculate. I have to know more about docker before I speak more on this.
17:23:08
jcowan
The Java trick works well for Java, which has probably a thousand times as many publicly available packages, to say nothing of all the proprietary-nobody-else-cares ones.
17:24:08
jcowan
my tagsoup library, e.g. lives in org.ccil.cowan.tagsoup, and you can tell the compiler that tagsoup is a (local) nickname for this
17:26:22
shka_
so if you can essentially save-lisp-and-die and have standalone binary, that's awesome
17:26:34
gendl
That might make a good YC startup idea -- "containerize Paul Graham so we can replicate him across the Internet"
17:26:42
didi
Maybe something like (local-package foo ...) which rewrites every DEFUN as LABELS and God knows what else. /me shivers
17:28:16
gendl
shka_: the last thing, then we'll drop the docker subject I promise, cause it certainly belongs in its own channel (which prolly exists here), but what's the basic diff between a container and a chroot environment?
17:29:11
buffergn0me
If for nothing else than to support multiple conflicting library versions in the same image
17:30:23
gendl
shka_: well how it works on OSX and Windows is no big mystery - it's running a Linux VM.
17:30:48
buffergn0me
So when loading a system, you would specify that package #:foo.bar.baz is a nickname for #:version-package-store.foo.bar.baz.2.7
17:30:51
gendl
The trick is that each container doesn't need to be its own whole VM. It's a chroot environment on that same VM. Much lighter weight than running so many actual VMs.
17:31:55
didi
buffergn0me: Sounds like a plain to fragmentation and code rot, but I guess the availability of such feature isn't bad.
17:34:29
|3b|
multiple versions available at once sounds nice at a coarse level (like separate applications having different versions of dependencies), but could get ugly if 2 things in same project used different versions
17:35:23
|3b|
like you try to handle foo:some-error, but the lib signaled foo-2:some-error and you tried to handle foo-3:some-error due to different versions
17:36:48
|3b|
ACTION wonders how hard it would be to inject versioned local nicknames into package-local-nicknames style, don't think i thought about that use case
17:38:28
buffergn0me
Basically a library would be loaded into #:version-package-store.foo.bar.baz.2.7 by the build system. The library still thinks it is #:foo.bar.baz, and the systems using the library would see it as #:foo.bar.baz
17:40:43
|3b|
you would need foo.bar.baz loaded with foo.blah-1.2, foo.bar.baz loaded with foo.blah-1.3, etc
17:41:13
|3b|
if some other libs depends on foo.bar.baz with explicit dependencies on blah-1.2 and blah-1.3
17:47:06
buffergn0me
The "name" of that package would include dependency information. Nix handles that using hash trees of the build info of each system
17:47:31
jcowan
cross-platform docker, btw, uses a virtual machine under the covers, but iirc there is only one vm for all containers in that situation
17:50:30
pfdietz
|3b|: this is what made me think about first class global environments. Load each library into its own internal-to-lisp sandbox, to avoid interference. There would have to be some mechanism for propagating information between these "worlds".
17:50:31
|3b|
i think you could do that with current package system, though would probably have to poke at asdf a bit to get fasl caching correct
17:50:44
jackdaniel
gendl: I'm not going to discuss many problems I have whenever I truly depend on UIOP, my point was that there are often better tools for a job (quality and/or abstraction-wise) and calling them superseded is unfair (especially in a context of discussion, where uiop fails due to abstraction-creep on run-program streams)
17:55:27
buffergn0me
The question with environments is, should symbols share identity across them? Or is full virtualization of packages needed?
17:55:56
gendl
jackdaniel: I understand. "Superseded" is more of an aspiration than a reality in the present tense. I wish those better tools can eventually be merged into UIOP (assuming they are not so huge that they would bloat UIOP unreasonably).
17:57:28
|3b|
(packages are unrelated to the naming provided by symbols, they only map from text to symbol objects)
17:58:08
gendl
but I guess there's a philosophical question of whether UIOP wants to provide more than the bare minimum of what ASDF needs & uses. I think it's already doing that, though. For example I don't think ASDF exercises all the options of uiop:run-program.
18:00:12
meepdeew
Hi all, so towards the end of PCL Ch 20 (Special Operators), the writing gets into EVAL-WHEN, LOAD, COMPILE-FILE, etc..
18:00:23
meepdeew
It briefly touches on some of the differences between LOAD and COMPILE-FILE, and the implications.
18:00:28
meepdeew
For example, footnote 12 - why loading a lisp file with an IN-PACKAGE doesn't change the *package* for the context loading the file was illuminating for me.
18:00:42
meepdeew
But I'm still a bit unclear on some things addressed there, like what forms COMPILE-FILE must evaluate and why.
18:01:01
meepdeew
Can anyone recommend others writings, links, or resources to learn more about those areas specifically? Or is the really best (only?) way to just dig into the hyperspec and/ or implementation source code? It's struggling with these kinds of things that make me feel like I have blind spots and should go back and read SICP.
18:02:26
Bike
the basic version is that compile-file evaluate, at compile time, toplevel forms in an EVAL-WHEN with the :compile-toplevel situation.
18:02:52
Bike
so if you look at the macroexpansion of (defmacro whatever ...) you'll probably see code wrapped in such an eval-when, so that the macro is defined during compilation.
18:06:57
Bike
keep in mind that the expansion is going to differ between implementations, and it's possible one will do something else
18:09:39
buffergn0me
meepdeew: I would also take a look at the CLtL chapter on compilation: https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node224.html
18:12:50
buffergn0me
meepdeew: Actually, this is probably even more relevant to your question: https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node227.html
18:19:31
jackdaniel
gendl: I don't share your enthusiasm for "ubiquitous" solutions (disregarding what they are), but I'm aware it is a matter of a personal aesthetic
18:23:27
gendl
jackdaniel: in general I agree, but the situation is that uiop is _forced_ to be ubiquitous by its nature and its role in supporting the (current) fundamental build system. It's run and exercised and tested constantly (still not enough to be sure, but more than your average QL library out there). So I wish there was not a perceived need to replicate so much of its functionality in other targeted libs.
18:24:16
jcowan
I don't understand this sentence: "The compiler must treat any binding of an undeclared variable as a lexical binding."
18:25:12
gendl
I suppose if UIOP could be an assembly of all the best targeted libraries (i.e. uiop.asd would just contain a bunch of :depends-on and importing/re-exporting symbols) rather than a monolithic thing, that could be an answer...
18:25:32
jackdaniel
gendl: The point is that it replicates other libraries functionality (and declares it, arogantly I'd say, deprecated). Also I don't consider m4 being anything but autoconf companion, but it is as popular on systems as uiop on asdf-enabled implementations
18:25:41
gendl
but again, that would open up UIOP to so many points of failure, by depending on so many little libraries, all with twisty passages going in different directions...
18:26:09
jackdaniel
I'd be much more at ease, if uiop were a library on ql and asdf/uiop were its copy in different package for asdf
18:26:46
jackdaniel
current state is that uiop is not updated, because uiop.asd is somewhere in implementation guts (which asdf finds), so ql doesn't see a need to download it (by default)
18:29:03
gendl
but yes, ASDF has been a bit flakey as of late in terms of whether it see it there or not, especially for the monolithic-compile-op operations. Currently my build scripts have to manually prepend the uiop--system.fasl to the build result.
18:31:04
gendl
jackdaniel: yep, it's good to know all these opinions. Too easy to be doing things one way and end up thinking that's the only way.
18:36:23
Bike
i'm just reading this as saying, if the compiler sees a binding of a variable, and there's no declaration of that variable to be special, it's lexical.
18:38:58
jcowan
What is the purpose of the :intern clause in DEFPACKAGE? (I know what it does, not why you'd use it)
18:40:09
jcowan
To intern a symbol in a package, it suffices to mention it. Under what circumstances would you wish to avoid mentioning it?
18:40:11
pjb
Packages are basically collections of symbols. So it's natural that you can enter symbols in them, when you create a package.
18:41:15
Xach
jcowan: if you want to establish all your package definitions before loading any code.
18:41:17
pjb
Also, you may want to intern it preventively, to force conflicts if you later use or import a package exporting a symbol with the same name.
18:43:30
pjb
(defpackage "A" (:use) (:export "X")) (defpackage "B" (:use) (:intern "X")) (in-package "B") (use-package "A") #| ERROR: Using #<Package "A"> in #<Package "COMMON-LISP-USER">
18:43:50
pjb
without the :intern, A could be used in B without error. This may not be what you want.
18:45:04
jcowan
However I don't understand the other argument. When you load the code, it is presumably within the (implicit) scope of an IN-PACKAGE and so will be interned at that time.
18:47:23
pjb
I have a example here: https://github.com/informatimago/lisp/blob/master/tools/dummy-asdf.lisp
18:47:49
pjb
where I define a dummy asdf package, so I may load and define some things even when asdf is not available.
18:49:40
pjb
jcowan: also, another example, in https://github.com/informatimago/patchwork/blob/dialog/src/packages.lisp
18:50:30
jcowan
In the first example, ISTM that (in-package "ASDF") would be better instead of qualifying every symbol being defined
18:51:21
pjb
This was an old program, without much structure to the code, and basically, circular dependencies between the packages, defined at run-time… Creating the asdf system was a daunting task of untangling this messy web. I had to use :intern to have those symbols existing to be able to import them in another package, line 352.
18:52:50
pjb
So granted, we'd rather write well structured programs, where the dependency graph between packages and modules is not circular. In which case, :intern might be less useful. But if you want to take advantage of the possibility to have circular dependencies between packages and use non-exported symbols in other packages, you may have to use :intern.
19:36:31
jcowan
If I had to pick just one good thing about CL, it is the ability to write debuggers that make sense to non-programmers
19:44:42
jcowan
anamorphic: I mean the ability to have restarts with user-intelligible explanations that allow a user to say how to proceed from this point
19:44:57
phoe
or, if that's not an acceptable answer, I'd pick the fact that all the partial features forming the language (homoiconicity, macros, conditions, restarts, CLOS, debugger, interactivity, livepatching, introspection, syntax macros, ..., ...) actually work together in order to empower the programmer on their task
19:45:32
phoe
oh, right - that's the ability for Lisp to notice when and what screwed up, present the user with available options, and then be able to unwind safely to a predetermined spot and continue execution from there
19:46:07
jcowan
Just so. I think that emphatically beats all the other features in a ranking of excellence, though I agree that the effects of the other features are synergistic
19:46:07
oni-on-ion
https://www.gnu.org/software/mit-scheme/documentation/mit-scheme-user/Restarting.html
19:46:52
jcowan
MIT Scheme has a lot of MacLisp features still surviving that other Schemes never adopted
19:47:23
phoe
including the ability to actually introspect the condition object using the Lisp inspector, which is an equally awesome mechanism (if not more!)
19:47:51
phoe
and, from the condition object (or from the stacktraces!), inspecting subsequent Lisp objects that happened on the way to the final INVOKE-DEBUGGER call
19:54:36
makomo
speaking of the condition system, what would be a nice way to handle a signalled condition (using SIGNAL), log it somewhere and continue going as if nothing happened
19:55:21
makomo
F wants to handle it, but only to be able to log it later on. it wants G to continue its work nonetheless
19:56:46
phoe
(handler-bind ((condition (lambda (c) (format t "WAAH! A condition happened!~%~A~%" c)))) (foo))
19:57:02
makomo
the handler that does the logging shoudln't decline the handling, i.e. it should perform a non-local transfer. but i'm not sure if there's a "pretty" way to do that?
19:58:05
Bike
if you decline to handle a condition signaled by SIGNAL, SIGNAL will still return normally (i.e. G will just keep going)
20:00:27
makomo
but how would that restart look like? don't restarts have to perform a non-local transfer as well, otherwise INVOKE-RESTART returns with w/e
20:03:30
phoe
you'll achieve what you want - the logging function will log your stuff, and execution will continue
20:03:54
makomo
i have a LU decomposition algorithm thingy. now, i want to report a "warning" when the matrix is nearly singular or similar
20:04:26
makomo
and since i already have this function returning multiple values, i really don't want to add more return values to signify "the matrix could be nearly signular"
20:04:40
makomo
and i don't want to do th elogging from within the LU algorithm itself, for obvious purposes
20:05:12
makomo
the outer function logs it, gets the results, and then maybe prints out the warning when printing out the result
20:05:37
phoe
you can have F have a handler that stores the condition in some list, and then calls the continue restart.
20:06:03
Bike
like if you use WARN (instead of signal) you automatically allow a given F to suppress warnings or not.
20:06:18
makomo
Bike: right, that might be simpler, but i somehow want to "clean up after myself". do you think that's just wasted effort or?
20:06:40
makomo
in the sense that, i want to be the only one to deal with the condition, and not let it bubble up
20:08:11
Bike
like maybe later you do something with a million matrices, and you don't want the interface for that to print a thousand warnings, but you do want a handler that just counts instances instead of logging something
20:08:50
phoe
for example, to invoke an external debugger, so the user may inspect the matrix and decide what to do
20:09:36
phoe
if your code is used as a library, then the information that your code signals a SINGULAR-MATRIX-WARNING might be valuable to programmers that write code that uses your code.
20:10:08
phoe
by catching all of those warnings yourself, you prevent them from getting that information.
20:10:16
makomo
Bike: oh, the function doing the logging is really a "top-level function", not meant to be called by other functions. like a REPL or something
20:10:29
makomo
it calls the underlying algorithms on some tasks and reports the solutions and warnings
20:10:55
makomo
well then i guess i would modify it not to call the restart, and let the signal bubble up
20:11:10
phoe
makomo: if it's a top level function, then I say that you write a handler in F that logs the thing to standard-output and then declines, and signal the condition in G.
20:11:54
phoe
if it's toplevel, then there's a very short handler stack, so efficiency won't hurt you.
20:12:36
makomo
meh yeah, not like i benchmarked it or that i really care anyway, but i just wanted to be as "pedantic" as possible
20:12:44
phoe
write it, and if it turns out to be too slow for you, I guess there'll nonetheless be many, maaany other things to fix rather than "oh this thing is signaling a condition".
20:14:12
makomo
but, putting this "let it bubble up vs. ..." issue aside, what do you think of handling this warning problem using conditions
20:14:27
makomo
i can't think of a better way to do it cleanly, and i wonder how stuff like MATLAB, etc. does it
20:14:44
makomo
how do their algorithms look like, and how does MATLAB's top-level find out that a warning occured within an algorithm
20:15:00
makomo
or Mathematica for example, which warns you when a system might not have a solution or w/e
20:16:19
makomo
well, i would like the user to know that the solution i computed might be really unstable
20:17:08
phoe
if the user wants to get notified of unstable results, they should handle your condition
20:17:22
makomo
yeah, that's what i'm going with, that's settled. i'm just wondering how other systems do it
20:17:53
phoe
a lot of other systems simply throw an exception if some flag is set, which gives you a poor man's condition system.
20:18:09
makomo
other options are returning extra values to notify the caller of warnings or setting some global warning flag or something
20:18:21
phoe
basically, if(throwOnUnstableResult) { throw new UnstableResultException(); } else { ... }
20:22:32
makomo
the other day i was writing some tests. never had so much fun writing tests in my whole life
20:32:31
HerrBlume
Hello, I'm trying to add an jdbc backend to clsql (for abcl.) To get it going i have to remove the ffi dependency from the clsql core package. Does someone know the cannonical repository or mailing list?
20:41:45
HerrBlume
phoe: It does not compile on my machine without throwing errors. And I think it should not be needed for clsql/clsql-sys. So my thought was to remove the cffi/uffi thingy. Writing a clsql-jdbc part should be trival (no other dependency, since the jdbc drivers are loaded via the classpath)
20:43:51
phoe
HerrBlume: hm - I think you might need to fork the github repostory for clsql and add your modifications there.
21:06:24
emaczen
phoe: If I create a bunch of threads with bt:make-thread and then do (format t ...) on them, each threads output gets tangled together
21:08:18
phoe
emaczen: make a lock and hold it around each call that prints something to *\standard-output*
21:08:23
sjl
Write a thread-safe-output-stream with trivial-gray-streams that wraps another output stream and uses locks from bordeaux-threads or whatever to synchronize writes to it, then (setf *standard-output* (make-thread-safe-output-stream *standard-output*)
21:24:34
emaczen
phoe: What you suggested seems to work, why go through the extra work with the gray stream?
21:25:56
phoe
and you can forget about locks altogether and just write FORMAT like you usually would.
21:26:35
phoe
it just writes to a stream; and the fact that the stream happens to be thread-safe is just a handy coincidence.
21:30:52
no-defun-allowed
are there any native machine vision libraries (similar to opencv without the floating point traps) for cl?
21:33:10
sjl
Note that you might also want to deal with *terminal-io* -- for anything but FORMAT, a stream of t means *terminal-io*, not *standard-output*
21:33:40
sjl
e.g. (format t "foo") writes to *standard-output*, but (princ "foo" t) writes to *terminal-io*
21:33:50
aeth
I'm considering implementing BLAS in native CL (and possibly also in a Lispy-lang-on-GPU as well) because the current options for this sort of thing disappoint me that much. From there it's only 3-4 steps to a useful library.
22:28:49
jcowan
Does anyone have practical experience with loading third-party plugins into their application with LOAD?
22:45:31
aeth
The only differences with that and ASDF (assuming you load the files in the same order) are, afaik, that you'll recompile every time and the FASL will be in the same directory (possibly implementation-specific?) instead of in ~/.cache/common-lisp
22:46:33
jcowan
but what I'm asking about is whether people exploit the ability to load a file whose name is computed at runtime.
22:48:28
aeth
The laziest way to do this would be to use Quicklisp to quickload an ASDF system. There's probably a proper way to do this via ASDF.
22:53:59
aeth
jcowan: I think the idiomatic way to solve that problem in modern CL would be to compute the directory containing the foo.asd file that describes the system foo and then have ASDF recognize that directory (would that be by pushing it to asdf:*central-registry*? not sure here)
22:56:18
jcowan
I see two problems, or perhaps only one: you can't sandbox what you load easily, and you don't know how to interface with it except by convention or reflection.
22:58:04
aeth
Sandboxing is apparently impossible in current implementations. Iirc, beach is working on an implementation that permits sandboxing as one of its goals.
22:58:07
aeth
https://github.com/robert-strandh/SICL/blob/e19613c61d8797a9ea586e3c513efdaeb0860d68/Papers/Global-environments/sec-benefits.tex#L92-L119
23:48:01
griddle
I've got a quick question. I've been working on implementing a small lisp interpreter and I'm interested in how '(...) and (quote (...)) relate
0:09:52
jcowan
If you had a sandbox, you could statically examine what it does and see if you are willing to load it into the live image, but of course there are no guarantees about what crimes it might commit dynamically.
3:17:45
r0sebush
question from a sbcl newbie.. What's is the best method to include libraries, like cl-ppcre into your lisp code for production release? .sbclrc or include (ql:quickload ...) in your code base?
3:39:49
gendl
2. When ready for release, use (asdf:operate 'asdf:monolithic-compile-bundle-op <your-application-system-name>)
3:40:24
gendl
that will produce one big .fasl with your whole application including its depended-upon stuff, presumably in the right order.
3:40:52
gendl
then start a fresh sbcl image, load this .fasl, then dump image. (I'm not sure the specifics of that with sbcl). ASDF also has built-in operators for doing that.
3:41:43
gendl
See this ancient blog post which needs to be updated/added to: http://gendl.blogspot.com/2013/03/saving-images-with-asdf3.html
3:43:37
gendl
it was suggested earlier today that someone needs to make a do's-and-don'ts for production application release and maybe a how-to. It's something which should be pretty routine and well known at this point but doesn't seem to be.