freenode/#lisp - IRC Chatlog
Search
6:41:27
fiddlerwoaroof
In distributed systems in general, the idea of "identity" is usually not really central, in the sense of reference identity
6:41:30
no-defun-allowed
Having some varying stages of going from real objects to binary blobs, I maintain a weak hash table of already-existing instances and their hashes in the server and client. Thus we can have enough object identity for things to make sense.
6:41:56
beach
fiddlerwoaroof: Exactly, which is why I don't want distributed, and I don't understand it.
6:42:32
fiddlerwoaroof
Usually you treat as identical things that are equal under an appropriately specified value of "equality"
6:42:44
beach
Abandoning Common Lisp-style identity means you are no longer programming Common Lisp really.
6:43:12
no-defun-allowed
And we continue to maintain the illusion by putting a read barrier on SLOT-VALUE-USING-CLASS, which rewrites slot values to replace "reference" objects with the real object supplied by the client.
6:43:43
no-defun-allowed
There was a technique based on "naming" objects and maintaining name translation tables in replicated object systems which I read about.
6:43:46
White_Flame
nor is a database part of your address space, nor connected to the ephemeral runtime of the lisp image
6:44:02
beach
And you have to take into account the possibility of one application object being represented by several instances. Then you have to track updates so that all instances are synchronous. That sounds like C++ programming to me.
6:44:38
White_Flame
in database applications, the database is the canonical store of information. Clients work with queries & pieces at a time, resolving all changes back into the shared database
6:45:07
fiddlerwoaroof
Yeah, no client should ever care about the state of the data in another client
6:45:12
White_Flame
any lisp runtime can come and go, working with the external data in the database, as much as little as it wants, and can do so simultaneously with others
6:46:26
no-defun-allowed
I don't think SQL has references (any more than it has indexes of rows in tables).
6:46:35
fiddlerwoaroof
I think the best way to think of a table is a relation between the primary key and 0 or more other values
6:47:17
no-defun-allowed
At the rate at which I program, it might be wiser for myself to wait for CLOSOS, and use the in-memory database implementation as a persistent database.
6:47:21
beach
Now I am beginning to see the answer to my question. It appears that the data base can store only objects that are not guaranteed to be EQ, so that creating a copy is not a problem.
6:47:23
White_Flame
and because it has no implicit references, that removes the burden fo having everything manifested at once
6:48:03
White_Flame
databases are for: 1) your data is bigger than RAM, 2) you need transactional security, 3) you have multiple users hitting your data simultaneously, 4) the data needs to persist safely
6:48:03
moon-child
if it's a concern, make your own version of eq that compares primary keys for db-sourced objects
6:49:14
White_Flame
and of course, SQL (and any query language, like prolog etc) makes traversing thorugh tons of data way simpler than imperative looping
6:49:56
no-defun-allowed
Re-reading the Postgres manual, the "limitation" that a binary value has to be materialized entirely in memory is not really a problem, as I would do that on the Lisp side briefly too.
6:50:16
fiddlerwoaroof
It's great for situations where you want to distribute work across a large number of computers
6:51:09
fiddlerwoaroof
The last system I worked on basically treated the database as a cache, and you replayed the log through all the worker nodes to rebuild or update the cache.
6:51:46
White_Flame
beach:your EQ conclusion is incorrect as well. Relational keys have very speciifc equality semantics, and those are the only things that are compared for identity
6:51:51
fiddlerwoaroof
But, in this context, the restrictions are even more severe because you have to design your data so that reprocessing the same log entry doesn't change the DB
6:52:34
fiddlerwoaroof
Because, roughly, you can't guarantee that a message in the log is only processed once (if you want to make sure that you don't lose any data)
6:52:38
beach
White_Flame: Sure, but they are separate from my Common Lisp references, so I have to program differently.
6:53:04
White_Flame
correct, they are data, they are not objects. there is no pointer or reference to them. they are stored by key
6:53:35
White_Flame
and those keys between rows can be connected in unique ways via sql to get summaries, comparisons, joins, etc in complex ways across huge datasets
6:54:18
beach
So here is a good story: My employer bought Oracle to store student information like exam results, etc. I proved to my students that 100 years of such data would fit in the RAM of the laptop I was using at the time.
6:54:29
White_Flame
actually, the key is only for fast lookup, there can be slow lookup for any match
6:55:10
White_Flame
right, you can make custom solutions for anything. Or you can have multiple machines all sharing ACID protected data in a standard way
6:55:26
White_Flame
instead of only implemented the protections you think are needed at implementation time
6:55:32
fiddlerwoaroof
White_Flame: the primary key is sort of essential to the relational model, though
6:57:03
White_Flame
any non-key column can be used for selection. just the key is unique with the table and indexed well
6:57:06
fiddlerwoaroof
e.g. your schema is largely driven by the idea that a table is a relation between the columns that compose the primary key and the rest of the columns
6:58:29
fiddlerwoaroof
I've never read Codd's book on the relational model, but my general impression is that the primary key is not just an optimization, it's part of the theory behind the model
7:01:29
fiddlerwoaroof
RDBMSes also tend to be used by multiple applicaitons simultaneously: e.g. the applications your developers make for users, the financial department's accounting software, the data science department's modeling tools, etc.
7:02:01
fiddlerwoaroof
The other big advantage is that it's a relatively rigorously specified way to represent data that lots of existing tools can take advantage of.
7:07:14
no-defun-allowed
Generally, I'm now convinced that getting a database to behave with the possibility of crashes is easier than getting a filesystem to behave.
7:09:26
fiddlerwoaroof
no-defun-allowed: this is a great read on that topic https://danluu.com/deconstruct-files/
7:11:30
fiddlerwoaroof
Some people tried testing the assumptions made by various tools that have to write files reliably.
7:11:39
fiddlerwoaroof
They discovered "every single piece of software they tested except for SQLite in one particular mode had at least one bug"
7:11:53
no-defun-allowed
Making either work correctly with all the logic, and all the stuff I keep in memory for performance, seems hard though.
7:12:24
fiddlerwoaroof
And this isn't an accident: SQLite is tested for correctness pretty thoroughly https://www.sqlite.org/testing.html
7:12:44
fiddlerwoaroof
So, my recent thought has been "don't write data to files, write it to SQLite"
7:13:30
fiddlerwoaroof
You don't even need to treat it as an actual database, just one table with a single column that you append to
7:48:03
loke[m]1
If you want to see my implementation of a object mapping for CouchDB: https://github.com/lokedhs/potato/blob/master/src/potato/metaclasses.lisp#L33
7:49:04
loke[m]1
And here is one use of it: https://github.com/lokedhs/potato/blob/master/src/potato/upload.lisp#L193
7:59:56
srandon111
hello all, is active development on common lisp and/or common lisp libraries/modules still done? or is it a lnaguage just used to mantain old software?
8:00:35
beach
srandon111: There is lots of active development of applications and system software written in Common Lisp.
8:03:15
scymtym
there is probably also some work to keep legacy common lisp software running, but my impression is that such cases often involve commercial software and the commercial implementations and are not discussed here
8:04:23
beach
srandon111: Where did you get the idea that there might not be any active development in Common Lisp?
8:10:08
beach
srandon111: I said "ignorant", but there is another category of people who will spread this kind of misinformation, namely people who have a huge amount of effort invested in some technology that might be threatened by Common Lisp.
8:12:09
beach
That last kind of people is a dangerous one. Carol Dweck calls this phenomenon a "closed mindset". And these people will sometimes do awful stuff to avoid having to abandon the technology they invested in, and learn something better.
8:13:03
beach
Often, people with a closed mindset are also very sociable and come across as leaders, which makes the problem even worse.
8:13:25
scymtym
but there is no useful answer to such questions anyway. if a single person still uses common lisp, the answer whether anybody still uses common lisp would technically be "yes", but practically "no". but at what threshold does the answer become practically "yes"?
9:29:10
White_Flame
srandon111: quicklisp really invigorated 3rd party usage, and compared to CL's age is quite recent
10:03:47
scymtym
srandon111: my answers would be yes and yes. a popular example is google's search engine for flights
10:08:45
no-defun-allowed
"Having asked about the possibility that there are real people out there who use Lisp (as opposed to AI People who are known to be non-real) and having received no answers, I can only conclude that Lisp is not being used and that it is not, therefore, a real language."
10:10:09
no-defun-allowed
The ending remarks from a short-lived section of some journal called Lisp Pointers. It failed to catch on because Lisp people call them references and not pointers.
10:10:31
beach
srandon111: I am asking because if the answer had been "no", what conclusion would you draw from it? It would be a grave mistake to draw the conclusion that Common Lisp CAN NOT be used for such thing, as opposed to drawing the more correct conclusion that programmers (and especially managers) are generally not very well trained when it comes to knowing the features of what is out there.
10:10:55
no-defun-allowed
Oh, the journal was Lisp Pointers, and that was the closing remark of "The key of Lisp".
10:33:51
srandon111
beach, i wanted to learn a lisp language and have doubts about doing clojure or common lisp
10:48:19
indathrone
srandon111: don't overthink it; just pick one and get started. If you like working in lisps, much of what you learn with one will translate into working in others.
11:10:25
Odin-
srandon111: In my experience, Clojure has more concessions to modern ways of thinking, but the community assumes that Java is familiar.
12:12:53
VincentVega
Anyone who used cffi here? How do I do automatic NULL checks? I have a bunch of functions that return various pointers, so I guess I can't just do this https://pastebin.com/zbz31xRa and would have to produce these definitions on the fly or something. Unless there's an existing mechanism for this of course, and that's why I am here.
12:23:30
VincentVega
In fact, nvm, guess I can try to define new foreign types on the fly, if cffi is ok with it...
15:11:28
Bike
of course, if you don't know the type, this could cause problems. for example if you guess something is a null terminated string and it turns out to be a two megabyte array with no zeroes in it
18:16:29
jcowan
beach: I think that if you approach the relational model without assumptions and without a filter, you find that it has its own kind of consistency and in fact is quite beautiful in the mathematical sense.
18:17:02
jcowan
Granted, SQL databases have made serious compromises with the model like allowing users to depend on the order of columns and rows.
18:17:52
jcowan
Without meaning to offend, you sound to me like someone discovering Lisp and saying "My suspicions are confirmed; this is not like Java at all!"
18:21:14
jcowan
"Why would anyone give up the rigor of statically typed OOP for this big ball of mud that allows you to write just any old thing?"
18:29:19
jmercouris
So we could have the same function redefined with different argument counts like in Java
18:31:49
Gnuxie[m]
not sure there's much need for that as none of CL (the language) needs them, so would make sense for it to be a portable library
18:33:30
Gnuxie[m]
like, there's nothing that special that is required for their implementation and i don't know how much you'll really lose in performance to dispatch either, it doesn't seem to be that complicated at a first look
18:33:34
beach
jcowan: I know the beauty of it. I took a graduate-level course at Hopkins. And it makes a lot of sense as long as you consider only numbers and strings, but it gets more complicated when you want to preserve identity.