freenode/#lisp - IRC Chatlog
Search
0:17:23
aeth
com.informatimago and xelf and ECL are officially ahead of the trend because they've been on Gitlab for forever.
0:36:58
rumbler31
pjb: regarding trust, if I held that mindset I'm not sure how I would have gotten anything done.
0:56:20
aeth
rumbler31: It definitely depends on your domain. For some domains, you have no choice but to pay thousands of dollars for some overpriced specialized software.
0:57:25
rumbler31
the discussion I meant to re-start was about debugging tools. beach mentioned something that suggested that in his experience, paid implementations have better debugging experiences and I wanted them to expound on that
1:02:58
aeth
It wouldn't surprise me. Commerical software like that tends to focus on features to justify upgrades. e.g. integrated Prologs.
3:10:55
minion
beach, memo from pjb: apropos 9:17 https://www.youtube.com/watch?v=ZJr81DtSwUc (loop for i below 10 sum i maximizing (sin i) into maxin collect (* i i) into squares finally (print (list maxin squares))) is conforming AFAIK. it has several accumulations clauses of different types… ;-)
3:12:40
beach
rumbler31: I have no experience with commercial Common Lisp implementations, but I hear they are much better than the free ones for debugging. For my own experience, I still miss using GDB back in the days when I was programming in C. I find I can't do things with our free Common Lisp implementations that I could do then.
3:15:49
beach
rumbler31: I do know what I would like to see for our free Common Lisp implementations, namely this: http://metamodular.com/clordane.pdf but it will require significant support from the implementation, so it probably won't happen with the existing free implementations.
3:48:07
jeosol
I have not gone through your linked pdf in detail, but does your design closely match gdb in c/c++ or it's different with added functionality.
3:49:38
jeosol
Also, what do you mean by "significant support from implementation", obviously man hours, but does this also mean it is likely to require lot of implementation details that it is likely to be very different for each free implementation
3:50:11
jeosol
also since SBCL is very popular, is this feasible to at least put efforts for that implementation.
3:51:42
jeosol
As the designer, you are in better place to push this because it is going to be more clearer to you, and subsequently others who take time to study the document (and who may have used other debugging systems)
3:52:30
beach
jeosol: I did not try to match GDB. I was considering something I would like for Common Lisp.
3:53:11
beach
jeosol: In particular, I won't use different processes. I want one thread to debug another thread, and the debugged thread could be the debugger.
3:54:03
beach
jeosol: I am totally confident that I would be unable to convince the maintainers of any current implementation, including SBCL, to add the support that Clordane requires.
3:55:03
dtornabene
i just finished it btw, seems like the implementation of this on sbcl wouldn't be....momentous? but then I have the barest knowledge of sbcl internals
3:56:19
beach
jeosol: I don't know SBCL well enough to give an estimate. It requires compiler support and, as I understand it, the SBCL compiler is very complicated.
3:56:23
jeosol
and possibly implement using a style with as little modification to the original code base.
3:57:13
jeosol
if it has to involve working deep in the compiler, then it will require major effort for sure
3:57:50
beach
The SBCL maintainers will likely say that it might slow down the generated code, and reject it before even considering it.
3:57:55
dtornabene
talk about a subject in need of a good book, a serious thorough book on SBCL internals would be *awesome*
3:57:59
jeosol
for my applications, I wouldn't say I have not had a need for debugger, but I ensure my systems are testing in very small units, and when CLOS objects are used, I have constructor functions for easy testing.
3:59:09
jeosol
however, my debugging probably sucks, combination of inlet and outlet statements to function (stopped this), SLDB, good old print statements, ...
4:00:03
jeosol
beach: on the issue of rejecting, it would have been nice if it was even possible to scope out requirements and what might change or be impacted in the context of SBCL
4:01:22
beach
dtornabene: You have to consider that SBCL has a long history and the internals have very likely become a bit messy over time.
4:02:53
dtornabene
beach: for sure, i spent a few hours recently just reading, taking line counts, investigating the layout. It seemed like a huge task to get on top of the system as a whole, but its something I'd like to devote a non-trivial amount of time to over the next 12 months or so
4:03:48
beach
jeosol: I totally think that we need a more modern implementation. That is why I am not willing to put any effort into SBCL. In particular, if I have to convince the current maintainers to make changes that are very unlikely to be accepted, I prefer putting my energy into something more modern.
4:05:25
beach
jeosol: One that has first-class global environments, my generic dispatch, my LOOP implementation, my garbage collector, the debugging support I want, etc, etc.
4:06:44
jeosol
Good points. I take it there is not way to implement these new functionalities or add them in existing implementation
4:08:34
beach
jeosol: There is always a way to implement them in an existing implementation, but it is a question of how much work it requires, and whether the maintainers are willing to accept it.
4:10:53
jeosol
By the way, are big companies using SBCL or some of the other free implementations, such that they could fund some of these type of projects
4:12:22
jeosol
This could be an opportunity to take CL to be more modern and add more battle tested, and proven libraries so some library class "like Boost C++ libraries" that eventually make their way into the core.
4:12:33
beach
loke: A sliding collector for the nurseries (per-thread) and a combination of mark-and-sweep and a traditional malloc/free for the global collector.
4:13:58
jeosol
I say we apply for grant, so pull some resources, especially with the merits of the new system.
4:14:17
beach
As Wilson et all showed a few decades ago, fragmentation only exists in programs that don't do anything useful.
4:14:42
beach
loke: I am planning that, but I haven't given it much thought. I want it to be correct first.
4:15:10
loke
The benefit of compaction isn't that fragmentation is sliminated per se, but that allocations can be much faster if the free memory is clustered.
4:15:40
jeosol
Or may be fork off SBCL now, just like the latter from CMUCL back then. But it will likely be more difficult if the base code is messy. For large codes, I know first hand how important it is to get the base right.
4:16:09
loke
beach: Right. That's what makes GC-language allocations so much faster than C, for example.
4:16:11
beach
loke: In the global collector, allocation is made in big blocks to promote many objects at the same time.
4:16:56
beach
jeosol: My hunch is that it would be even more work to transform SBCL into a modern implementation than it would be to create a new one.
4:17:38
jeosol
beach: I agree with you last point. Yes, especially if the design is radical. It may be better to start of a fresh base where things are layered systematically.
4:17:42
loke
akkad: Ironclad hasn't had any known vulnerabilities, AFAIK. It doesn't suffer from buffer overruns and such, and the algorithms seems to be solidly implemented.
4:18:15
beach
loke: Nothing special. I have a linked list of two-word header objects that use mark-and-sweep. When a dead header object is found, the equivalent of free() is called on the "rack" (the part that contains the data).
4:21:18
beach
Sort of, yes. The nursery could tell the global collector that it needs a bunch of objects at once.
4:22:56
beach
There are plenty of advantages of this scheme. For one thing, an object in the global heap never moves.
4:26:19
beach
I would need people who 1. are available for hire, 2. know about compiler techniques, 3. know about Common Lisp details, 4. know about garbage collection, 5. know the details of CLOS, 6. know about reusability, maintainability, portability.
4:26:59
beach
I am betting that the handful of people who fit that profile are already working on some exiting free or commercial implementation.
4:28:12
jeosol
Well, it also depends, there are people coming up the ranks that may not know everything now
4:28:48
jeosol
My code uses most CLOS but I had to read Sonja's book, and a bunch of others to get good over time.
4:29:12
jeosol
Looking at the list, I think you want people who are heavily invested in CL overall, and true some may be working on other projects now
4:29:19
beach
jeosol: Oh, I see what you are saying. I should 1. Apply for a grant, and 2. Hire you.
4:30:22
jeosol
I am not the most person qualified for this task. What I meant above is the person who has the main design and how all the parts fit, will lead the efforts early on.
4:31:22
beach
jeosol: Here is what I think. I think I should continue doing what I am doing now, i.e. working on the implementation and doing the research to invent techniques of making it modern, good, and fast. Then, people who are sufficiently knowledgeable and sufficiently interested will show up spontaneously. Like Bike, heisig, drmeister, etc. And there are already signals that some more people will join.
4:31:24
jeosol
I think it's the team that matters most as long as they believe in the final objective, agree on the common grounds, ..., off and running
4:32:06
jeosol
I didn't know you have gone far with this, hence my grant comment. I thought it is something just starting out.
4:32:09
beach
jeosol: Besides, Common Lisp has been off the radar for grant-giving institutions for the past half a century or so.
4:32:39
jeosol
If you have gone far, by any means, I would say continue, because changes are likely going to slow and impede things now.
4:33:15
beach
jeosol: The Cleavir compiler framework is already used for Clasp. And karlosz has a GSoC project to use it for CLISP. jackdaniel has plans to use it for ECL. And in Marbella, there were rumors that someone might use it to write a new SBCL compiler.
4:33:50
minion
jeosol: Cleavir: A project to create an implementation-independent compilation framework for Common Lisp. Currently Cleavir is part of SICL, but that might change in the future
4:34:40
beach
jeosol: And, obviously, the compiler is a HUGE part of a new implementation, so a big chunk of code is already done.
4:34:46
jeosol
When I started working on my code/application. I got to a point, where I was considering changing language since people start saying I should consider Clojure. I actually posted this question on Twitter and Paul Graham advised against Lisp, and only agreed when I mentioned my code is heavily CLOS-based
4:35:13
jeosol
Eventually, I stayed with it. A buddy told me to use something else as a condition to come along.
4:36:46
beach
The fact that Cleavir can adapt to totally different implementations is an indication that it is modular and maintainable.
4:37:08
jeosol
Clasp is something I planned to take a look at after discussing ML libraries and it was recommend to use clasp and link to other ML/AI libraries and make them accessible via CL
4:37:21
beach
Contrast that with the SBCL compiler that has assumptions about that particular implementation all over the code.
4:38:04
jeosol
I see your point. It is better to start fresh. For large code bases, it usually works better than way and you get more productivity as you add more layers on top.
4:39:22
beach
loke: Interestingly, the fact that the global garbage collector does not move things around, makes this ideal for interoperation with foreign code like C. But such interoperation is not one of the goals of SICL. :)
4:40:11
jeosol
beach: to echo your "reusability, maintainability, ...", point, it is very point and you can see from the use of Cleavir component in several projects.
4:41:27
beach
Right. I spend a lot of time on that stuff. I am not perfect of course, so occasionally I get it wrong.
4:41:58
karlosz
to add to beach's point, the Cleavir to CLISP interface is only about 900 lines or so
4:49:35
beach
So the goal of Cleavir is not only to provide a compiler framework, but also to include all kinds of modern compiler optimization techniques so that a client implementation can pick and choose what optimizations it wants to apply and in what situation.
4:50:53
beach
Some of these optimization techniques can be used as published. Some need to be adapted to the specific situation of Common Lisp (nested functions, threads, closures, etc). Others need to be invented from scratch (like the path replication stuff we published in ELS).
4:51:44
jeosol
beach: you got me. I am convinced and sold. Definitely not possible, so forking it wouldn't work
4:52:17
jeosol
Layered software/code is a beauty, it takes almost no effort to add new features or corner cases.
4:52:44
karlosz
one big obstacle to compilers is just visualizing the IR. sbcl still odesn't have any good tools for doing that, even after 30 years of existence
4:53:11
karlosz
whereas i can say that the cleavir visualization tools really cut down on debugging time
4:53:51
karlosz
in fact, if you want o learn about sbcl internals, its probably still better to go back to the cmucl docs
4:55:51
beach
jeosol: Since perhaps you don't fit the profile for Cleavir or SICL, you might instead want to consider a project on this list: http://metamodular.com/Common-Lisp/suggested-projects.html :)
4:58:04
jeosol
I have been trying to learn ASDF as Fare is planning to leave at some point so he told me. Beach, I can work on multiple projects. Most of the work I was doing on that application is for the backend which is mostly complete
4:59:29
loke
It processes text, and generates a sequence of glyphs and their precise location where they should be drawn
5:01:06
beach
jeosol: SICL is only one line in that list: "A Lisp system with first-class global environments"
5:04:30
loke
LdBeth: McCLIM already supports it in my Freetype branch. What Beach is talking abou tis a pure-Lisp implementation.
5:04:43
jeosol
are you considering things around data manipulation arrays (e.g., numpy, pandas as in pythong), machine learning/AI, even if down the line
5:17:15
beach
jeosol: For applications, I am only able to write a list of the ones that I personally would like to see and that I know that we don't have.
5:20:40
jeosol
people talking about optimization tend to be carried array. very impressive, I need to take a look at the stuff again.
5:21:46
jeosol
I think there are expertise littered around that can be converged which is why I mentioned the effort of the individuals involved.
5:22:19
jeosol
oh I will definitely come to ELS. I submitted an abstract for a conference in China and one due next week for a conference here. This is also the lisp work
9:44:32
beach
jmercouris: It might work with some restrictions, yes. But it would be very implementation specific.
9:44:59
shrdlu68
There are several serialization libraries, perhaps those that do binary serialization could be useful.
9:45:10
jmercouris
beach: "The pickle module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. "Pickling" is the process whereby a Python object hierarchy is converted into a byte stream, and "unpickling" is the inverse operation"...
9:45:48
beach
Sure, you can serialize things. But that's not the same as getting the binary representation.
9:46:18
jmercouris
what is the difference between serializing to binary, and the binary representation?
9:47:27
beach
PRINT is a very good serializer. It produces output in the form of characters. If you convert those to UTF-8 as most Common Lisp systems would, then you serialize to binary.
9:47:39
shrdlu68
jmercouris: Yep, so does ironclad and most packages that deal with networks protocols.
9:48:04
jmercouris
beach: well, I am not necessarily interested in how it is represented internally, just in a way to serialize and deserialize the objects
9:48:18
jmercouris
and to take the serialized form, and be able to input it in some reserializer and create an "identical" object
9:48:19
beach
jmercouris: And, you can de-serialize with READ. But there are restrictions, just like there must be in Python. You can't get an EQ object back.
9:49:02
shka
cl-store works on lists, vectors, instances of classes, structs (for sbcl), hash-tables..
9:49:19
phoe
a human-readable text representation is a very specific case of a binary representation of an object
9:50:10
jmercouris
okay fine, I would then need to take the character representation and turn it into a binary stream of some text encoding and decode it
9:50:47
_death
in many cases, READ is unsuitable.. and serialization of arbitrary objects is often not a good idea anyway.. it's better to think about your external representation without reference to any particular programming language
9:51:52
jmercouris
I just wanted a simple way to get a bunch of bits for some representation of some thing to test with
9:52:34
jmercouris
I know about existing CL QR code libs, I might end up inventing a version of my own, but I just thought it was a cool idea
9:52:45
jmercouris
you could sort of stream a whole bunch of data in video format, to transfer more substantial things
9:53:12
_death
so you can encode the string to an octet vector, then use that to produce a QR code, and do the other way around on the side side
9:53:23
shrdlu68
jmercouris: See also: https://common-lisp.net/~loliveira/ediware/flexi-streams/doc/#strings
9:54:06
jmercouris
shka: yeah sure, there will have to be something in the protocol to show "start of sequence" "end of sequence"
9:54:28
jmercouris
it's effectively a list of 0s and 1s, at least that's what I got from the wikipedia article
9:56:13
jmercouris
beach: I'm a little bit confused, is there some way to do (print "some string") --> 0010101010101011?
9:57:12
beach
jmercouris: There is no standard way (Common Lisp or otherwise) to represent a string in binary form.
9:57:39
jmercouris
beach: Is there a standard way to represent a string in binary form in lisp given a particular encoding?
9:57:41
_death
jmercouris: so, (babel:string-to-octets "foo" :encoding :ascii) gives you an octet vector.. and you can work on it bit by bit if you like
9:58:22
jmercouris
Yeah, I'm going to do that for now, I am just trying to understand a little bit more about what beach is saying
9:58:25
beach
jmercouris: Yes, of course. In the worst case, you have a table with that encoding and you concatenate representations for each character.
9:58:39
jmercouris
Okay, I was thinking for a second there might be a built in lib or something in the standard
9:58:54
beach
jmercouris: If you are lucky, your Common Lisp implementation uses Unicode internally, so that the table is already present.
10:00:26
beach
jmercouris: In which case, you can do something like (loop for char across string do (format ... "...~b" char)
10:04:20
thoxdg
if you use char-code you usually get unicode code points which is not a binary encoding
10:07:34
thoxdg
http://github.com/cffi-posix/ http://github.com/cl-remap/ http://github.com/RailsOnLisp
10:08:31
shrdlu68
A while back I ventured into compression, and wrote an arithmetic coding implementation in CL. It appears modern lossless compression has been reduced to a statistical modelling challenge, i.e, given a stream of bits, try and predict the next bit in the stream. Is anyone interested in creating a prediction function and testing it? I could create a library that makes testing out prediction functions easy
10:11:05
thoxdg
we're only lacking in semantic data about programming languages, they even self compile now
10:12:39
shrdlu68
thoxdg: Not sure, a prediction function simply receives a stream of bits and outputs the probability that the next bit will be 1.
10:15:53
thoxdg
well if you don't represent entropy of your prediction function the model should leak entropy
10:16:01
holycow
i get the following error: Failed to find the TRUENAME of /usr/share/common-lisp/source/cl-cffi/toolchain/package.lisp: No such file or directory
11:02:20
shangul
https://apaste.info/2pWi => I tried to read contents of a file. This is SBCL. Where was I wrong?
11:12:06
shangul
scymtym, I want to learn. if you give me something that I may read more, that would fine
11:13:28
scymtym
shangul: if you know how to use libraries, you can use the function ALEXANDRIA:READ-FILE-INTO-STRING to do exactly that
11:14:31
shangul
okay, but what's wrong in my code? purpose of read and with-open-file is something else than I'm supposing?
11:15:56
scymtym
the function CL:READ is for reading Common Lisp code from a stream. also, READ only reads a single form, not everything in the file
11:18:00
scymtym
you can use CL:READ-LINE to read one line (as a string) at a time. there is no builtin function to read a whole file
11:18:46
minion
shangul: please look at pcl: pcl-book: "Practical Common Lisp", an introduction to Common Lisp by Peter Seibel, available at http://www.gigamonkeys.com/book/ and in dead-tree form from Apress (as of 11 April 2005).
11:18:50
minion
shangul: please see gentle: "Common Lisp: A Gentle Introduction to Symbolic Computation" is a smoother introduction to lisp programming. http://www.cs.cmu.edu/~dst/LispBook/
11:20:35
scymtym
i don't know anything about it personally, but it is often recommended here to people without programming experience
11:21:20
scymtym
shangul: i have to go to a meeting now. good luck and maybe give ALEXANDRIA:READ-FILE-INTO-STRING a shot
11:24:01
minion
Do you like banana: An error was encountered in lookup: Parse error:URI "https://www.cliki.net/Do you like banana?source" contains illegal character #\ at position 24..