freenode/#lisp - IRC Chatlog
Search
4:34:45
no-defun-allowed
cl-trie has...interesting documentation. I tried to make an instance of cl-trie:trie and got a warning: "Key for trie not provided, possibly an overlook! See documentation for more information"
4:36:01
no-defun-allowed
So I did, and the example suggests I should add the initargs :verbose nil to muffle the warning.
4:36:27
beach
no-defun-allowed: You will find some very inefficient code in my repositories. For example, when I write random tests for a data structure, there is usually a "simple" version of it that uses the simplest data structure I could think of, and a "standard" version that is meant for production. I then generate random tests to compare the results of the two.
4:38:23
beach
But it could have taken as an example by rpg: "This really occurred in some code I've seen"
4:39:07
beach
So, if taken out of context, it might pass as code written by a very incompetent programmer.
4:40:37
no-defun-allowed
I ask because it's not uncommon to see simple code like that when "marketing" languages. If I get around to that, it would be polite to add a footnote: "This is an example of a <name>, which may or may not be representative of what you write in larger programs."
4:42:55
beach
Representing a matrix as a list of lists makes for some beautiful code for (as I recall) transposing the matrix.
4:44:55
beach
"Sir, I just compared the performance of your code to a C version of it. The C version is 1000 times faster. Does that mean that the language you are teaching us is useless in industry?"
4:45:01
no-defun-allowed
Agreed. It would probably be better to not include inefficient but "cute" code in that kind of material then.
4:52:50
no-defun-allowed
(And that didn't stop Python or any other scripting language, unfortunately.)
4:53:25
fiddlerwoaroof
At this point, consumer hardware is fast enough that inefficient algorithms often don't matter
4:58:10
aeth
fiddlerwoaroof: In practice, though, you'd use someone's matrix library, and the matrix library would make the correct decisions for the domain.
4:58:59
fiddlerwoaroof
aeth: matrix math is relatively rare, unless you're in a data science or game development context
4:59:18
aeth
Well, I mean, more likely, something other than matrices, like e.g. some standard GUI app or whatever
4:59:58
aeth
Imo, it's not because of the hardware that most programmers don't have to think about efficiency, it's because the work has already been done. Even to some extent us with SBCL
5:00:02
fiddlerwoaroof
Yeah, but my experience is that it's usually something like Rails or Spring which are tremendously inefficient
5:00:56
aeth
On servers, yes, you can do that, and if the efficient solution takes 5x longer to write than Rails, then using Rails makes sense.
5:01:16
aeth
I mean, it'll hurt people like me inside a bit to write inefficient code, but there are deadlines to meet
5:01:50
aeth
I think the real question is why there isn't a CL Rails, which probably should be able to be 10x faster without sacrificing developer productivity (besides unfamiliarity with the environment, of course)
5:03:09
aeth
It is very noticable when you're making your user load something in 3 seconds in JS rather than loading something instantly (* before network latency is taken into account, which should be < 100 ms afaik)
5:04:14
aeth
Sure, go for developer productivity over efficiency, but only when your servers are paying the cost, not the user with a 5 year old tablet or a 15 year old PC.
5:05:36
aeth
Fortunately (if JavaScript isn't involved) pretty much any end-user task on a PC is fast enough even on 10 year old hardware unless it's demanding like games or some professional tool.
5:06:31
aeth
I think the only thing that the user would notice as the difference between a CL GUI and an optimized C GUI is RAM usage, even if the former is 10x slower.
5:08:29
aeth
(And a lot of that speed loss is safety, i.e. not segfaulting randomly, which the user certainly notices.)
6:12:44
LdBeth
I guess many interpreted languages have claimed they've suppressed C on performance :*
6:18:18
beach
There is no such thing as an interpreted language. Interpretation is a property of the implementation.
6:31:05
fiddlerwoaroof
I'd think that certain language features (f-exprs, maybe?) preclude compilation in a meaningful sense
6:31:13
jackdaniel
if the specification says: "code is always interpreted compiling it is a spec violation" then yes ;-)
6:32:37
fiddlerwoaroof
But, if the meaning of a symbol always depends on the dynamic context and that context can always be changed arbitrarily by a function's caller, you couldn't optimize at all
6:32:41
beach
Sure, some languages make it very hard to write a compiler. But history shows that what is possible can change over time as new techniques are invented.
6:33:05
fiddlerwoaroof
(my guess is that such a language would be extremely painful to write as well, but...)
6:34:22
no-defun-allowed
I would guess self-modifying Brainfuck is a hard language to compile, at least if you ignore JIT/dynamic recompilation techniques.
6:35:14
jackdaniel
arguably any language is hard to compile, because efficient compilation is not easy ;-)
6:35:49
fiddlerwoaroof
beach: would I use Trucler if I wanted a macro to have access to declarations?
6:44:48
beach
fiddlerwoaroof: Currently, heisig is the one maintaining Trucler, and he is also writing backends for various existing Common Lisp implementations. The best thing would be to check with him what is possible.
10:38:53
francogrex
I have noticed that lately especially with java8, lisp(sbcl) is behind in terms of speed, also visible in the games for example: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/pidigits.html
10:41:25
Shinmera
Plots like these only tell you how much time each party spent optimising their benchmark.
10:44:38
scymtym
pidigits is mostly bullshit (there are exceptions like the Chapel solution) since most solutions just use gmp via ffi
10:46:57
no-defun-allowed
Seems a non-negligible set just interface with GMP (including the SBCL code!) so they should be roughly the same.
10:49:07
aap
and i've been working on pdp-6 simulation recently. as of yesterday it passses diagnostics and it turns out LISP is also working
10:52:43
aap
it's the earliest incarnation we have of what would be called maclisp a few years later
10:53:28
no-defun-allowed
francogrex: FWIW, I ran that code and it took 5.7 seconds without GMP. I figure my laptop is slower than the test machine, so Lisp is no more than 4 times "slower" than C.
10:59:05
no-defun-allowed
And that code does not look very nice, using GO and an unnecessary PARSE-INTEGER in a loop. I also think there are other things you could do to shave off a few microseconds, like using lexical variables instead of dynamic variables for the state the generator uses.
11:01:58
scymtym
no-defun-allowed: optimizing pidigits-sbcl-3 a bit made it go from around 9 to around 3 seconds on my machine
11:03:53
scymtym
no-defun-allowed: turned the global functions into local ones and got rid of the special variable accesses
11:04:47
no-defun-allowed
Also I think there is some emulation of ratios happening with the variables named num and den.
11:10:59
francogrex
no-defun-allowed: i agree with you about all other programming languages using GMP too
11:11:22
francogrex
and guys who denigrated the programming task or the benchmark: enough, grow up please
11:13:04
francogrex
I would like to see that if a head to head comparison, i.e. like as much similar code/constructs
11:13:32
no-defun-allowed
Well, I was going to say "play stupid games, win stupid prizes", but I have to go grow up now ):
11:20:47
francogrex
no-defun-allowed: let's not go into the usual debacle i have seen many times especially in stackoverflow: a question like, "can we do that in [insert favourite language here]?" and the flood of reponses/answers that are unable to directly answer the question start: "and why do you want to do this like that?" "what you are trying to do is stupid", or "would you not rather do that instead?" .... all replies that avoid the answer because
11:25:21
thijso
There are often reasons for those kinds of responses... not saying they're always _good_ reasons, but...
11:31:09
no-defun-allowed
Another FWIW: I wrote an implementation of the Bailey-Borwein-Plouffe formula and asked Petalisp to compute pi to what I think is 10,000 decimal digits. That ran in 1.83 seconds.
11:32:03
flip214
no-defun-allowed: you can calculate each _hex_ digit of pi directly with a fairly simple formula... decimal is the problem ;)
11:32:45
no-defun-allowed
Well, then you want (ceiling (integer-length (expt 10 10000)) 4) hex digits or so to get 10,000 decimal digits.
11:33:52
flip214
the idea is, when being asked for the 1e9 digit of pi, to just return the _hex_ digit in a millisec or so
11:38:00
no-defun-allowed
This is what I have so far though: https://plaster.tymoon.eu/view/1534#1534
11:46:47
francogrex
no-defun-allowed: it's good but the benchmark is about comparisons in implementing the same algorithm.
11:47:51
francogrex
I agree that the ratings reflect not only programming languages but the ability of the programmer to find alternative language constructs optimize his code to get the best implementation of the algorithm.. so we are also evaluating
11:47:55
beach
I think it is best to stay out of comparisons and arguments as long as we do not have the manpower to keep up with more mainstream languages.
11:50:41
beach
I meant the discussions on stackoverflow etc. It seems some people take them real seriously.
12:04:52
p_l
francogrex: might be a weird question, but I seem to recall from years ago that you might have done some fun stuff with Excel and ECL?
12:24:37
aap
i have the raw verilog here: https://github.com/aap/pdp6/tree/master/verilog but not the quartus project (yet)
12:27:10
lukego
I'm vaguely cooking up a Lisp HDL at the moment, built upon my complete lack of experience with hardware development or other HDLs :)
12:29:02
aap
do note that my verilog is for the most part sorta transcribed from the original schematics so it's not really idiomatic
12:30:02
lukego
I'm imagining a bottom layer that's an S-expression syntax on either Verilog or FIRRTL or Yosys RTLIL, but also an upper-layer that's a domain specific language for router/switch/firewall type of networking applications.
12:32:09
lukego
Dreaming up a DSL that could be efficiently run either on CPU or FPGA... hard to emphasise how little I understand of the problem at this stage though :)
12:35:29
lukego
Yeah. I had already talked myself out of doing this and decided to use Chisel, but that didn't seem to stick, maybe that will be Plan B if I make a mess in Lisp.
12:35:59
lukego
I probably need to try writing my own HDL before I can really appreciate and evaluate other ones anyway.
12:36:44
lukego
I had also planned to use a DSL called P4 off-the-shelf but it seems to be much too restricted for my needs.
12:38:47
p_l
there was a company that sold services based around Lisp-to-VHDL system (or maybe verilog)
12:40:27
lukego
p_l yeah that was Marc Battyani's company right? I remember his awesome talks. I am trying to "do a Marc" I suppose :)
12:42:43
lukego
I'm not an expert at P4 but the examples I've read have all been basically broken due to limitations of the language not permitting a correct implementation and it puts me off. The basic limitation seems to be that you can't write to tables, only read from them, and so you can't really write state machines in P4. Maybe can be fixed with a language extension to just allow writes but for now I'm a skeptic.
12:47:22
aap
i simulate with icarus verilog and once i'm confident it's right i'm putting it into my project with quartus (on windows)
13:02:04
lukego
I'm reluctant to use Windows but maybe I'll be lucky and never get confident enough to reach the synthesis step ;-)
13:02:27
lukego
I have a TinyFPGA BX for use with Yosys and I'm planning to upgrade to an EX or similar board at some point.
13:03:06
lukego
I suppose that initially my coding will be all Lisp running in a high-level simulator
13:22:36
jmercouris
if I compile a C library with one compiler, can I use that shared library with another compiler? If so, how does that work?
13:23:04
jmercouris
My question ultimately goes to this: how does CFFI work, on a fundamental level, wouldn't it have to hook into compiler specific details to make function calls to C functions, or am I missing something?
13:25:15
Xach
jmercouris: the system establishes an ABI that programs can use to make calls and interoperate
13:26:47
jmercouris
I guess so, but perhaps if ABI didn't exist, the performance of Unix Domain sockets would be better
13:27:40
Bike
without an ABI, to call operating system functions, you'd need to know what compiler the operating system was compiled with, and probably use it
13:27:42
d4ryus
id even argue that this ABI is the sole reason why C is still used that much, as it is the only language with a solid ABI.
13:30:03
jmercouris
Here's my question then, if the ABI, is so awesome, why aren't there other stable ABIs?
13:32:03
jmercouris
can I in SBCL write a program that exposes functions via this ABI to a C program on a Unix system?
13:32:07
Bike
i think with ECL that works because the lisp is compiled into a c program, not because of matching the ABI
13:32:28
Bike
(i mean, compiling into a C program means it gets to match the ABI, you know what i mean)
13:32:44
dlowe
so you can't suck sbcl into a C program, but you could pass a lisp function to a C function as an argument
13:33:04
jackdaniel
there is additional perk of that: if we compile ECL with CXX then we match C++ ABI of the default compiler ,)
13:33:45
Bike
sbcl has its own internal calling conventions that aren't stable or exported, but also to call a lisp function you need to have the lisp runtime, which is generally larger than libc, and i think only ecl makes it available as a shared object anyway
13:34:45
jackdaniel
Bike: but this is not a coincidence that functions are compiled to C functions (it would be in fact easier to have some things as state machines -- i.e flet functions); it was designed this way in KCL to have a common runtime for numerous languages in the same ABI as C (prolog was one of them)
13:35:58
dlowe
maybe just read the wikipedia page? https://en.wikipedia.org/wiki/Application_binary_interface
13:35:59
Xach
jmercouris: "binaries" implies multiple independent processes to me. but the ABI is about how, within a single process, to set up the registers and such to invoke functionality that is part of a single binary, provided by the OS (as a syscall), or linked in via a shared library
13:36:21
Xach
jmercouris: it is a protocol, but it's a protocol in the sense of an orderly and formal series of steps
13:36:29
Bike
it's just letting functions compiled with one thing call functions compiled with another thing, in the same process
13:36:49
jmercouris
Maybe I spent too long in Java world, but that's what protocol means to me when I think about it
13:37:07
Bike
an ABI is also why you can link a library to your C program without worrying so much about what compiled the library
13:37:10
jmercouris
why doesn't everyone implement this ABI for all sorts of FFI from any lang to any lang?
13:37:34
jmercouris
Bike: that's why my first question was about the shared library with one compiler, and the other program with another compiler
13:37:40
jackdaniel
jmercouris: C ABI has some limitations. one obvious one is that it is the caller who "owns" the callee arguments
13:40:08
Bike
like you can have more than one function with the same name but different parameter types
13:41:38
shka__
Bike: every compiler has it's own mangling rules so it is not even possible to know how it works
13:41:48
_death
there is an old book about C++ object model (Lippman) but in general the language is so complex that there is a lot of room for disagreements, and though there are multiple ABI standards, none is king
13:42:29
_death
it does not help that the C++ crowd decided to come up with a new standard every couple of years
13:42:58
Bike
you can drop by #clasp and ask drmeister stuff if he's not busy with another grant proposal
13:44:18
_death
jackdaniel: no one really cares about those.. people either care about the old standards or their implementation-specific extensions
13:44:19
jackdaniel
correct me if I'm wrong, but I think that languages which target llvm (given it is the same llvm version) may interop with each other
13:45:31
jmercouris
is there a way to avoid the segfault or somehow make that part of the memory not interfere?
13:46:06
jmercouris
dlowe: that's what I'm doing at the moment in Next, but one of our developers wishes to move to using CFFI
13:47:31
jackdaniel
if you want to invent memory protection on top of memory protection then I will leave that without comment
13:50:39
phoe
one way I think of is to mprotect() all of your Lisp pages each time you dive into C and unprotect them on your way back to Lisp
13:53:28
phoe
yep - you'd actually need to measure what is faster, doing actual IPC or spamming mprotect() inside a single process
13:53:55
phoe
the trick is, you have no idea if your Lisp implementation actually uses mprotect itself
13:54:10
jmercouris
jackdaniel: cs200 is a beginner CS class, one of the first things we learn about with C is all the types of ways a program crashes and what they mean
13:54:47
jackdaniel
jmercouris: it is hard to assume what you know and what you don't know (based on some of your questions)
13:54:52
phoe
_death: I don't think shared memory is going to work in that use case - you basically want Lisp to drive a lot of foreign code
13:55:07
jmercouris
jackdaniel: well, CS is a very wide breadth, and my knowledge is in a very specialized domain...
13:56:02
jmercouris
phoe: yes, basically, a lot of IPC, we have a very strict, limited protocol defined in remote.lisp
13:56:27
jmercouris
jackdaniel: yes, my specialty is in data ingestion, sanitation, and semantic analysis of documents
13:56:32
jackdaniel
either way, using mprotect or other ways to protect memory will save you from some crashes and will make it easier to hide bugs in your usage of the library
13:57:24
jackdaniel
jmercouris: then I can't assume you know what segfault is? I'm confused, but I won't investigate ,)
13:59:04
jmercouris
it's something Akin to smoke using gobject introspection to make bindings for languages
14:01:08
jmercouris
in terms of performance, I thought unix domain sockets typically used shared memory for IPC?
14:01:53
phoe
stackoverflow tells me https://stackoverflow.com/questions/2101671/unix-domain-sockets-vs-shared-memory-mapped-file
14:02:26
_death
phoe: I did not necessarily meant it this way.. it's also possible to have a shared memory area for storing large blobs and a message queue, perhaps using a different transport
14:07:46
dlowe
there's quite a lot. IPC means having to serialize and de-serialize your data structures
14:12:28
jmercouris
Indeed, that is extra work, on both sides, we use D-Bus which supports some primitives
14:12:42
jmercouris
and fortunately most of the information is contained within Lisp data structures that the foreign code not be knowledgable of
14:13:35
phoe
jmercouris: Second Question of Optimization: do you know where exactly in your system the bottleneck exists?
14:13:50
phoe
if it's not the IPC between Lisp and C, then you won't gain much from optimizing this place
14:14:30
phoe
otherwise you won't gain as much as you can from spending your time on speeding up the wrong thing
14:14:55
jmercouris
Yeah, the biggest argument in favor of moving to FFI is that we won't have to maintain a separate C and C++ codebase for the Webkit and Webengine ports
14:15:41
jmercouris
and that we'll be able to access more of GTK and QT without having to expand the protocol
14:15:57
jmercouris
for example, we want to add split window functionality, it means updating the C program, and the C++ program, and the Lisp program and then coordinating a release