freenode/#lisp - IRC Chatlog
Search
21:15:32
aeth
Does anyone run CL on a processor with many cores? AMD has some 16 core / 32 thread and even (for servers) 32 core / 64 thread.
21:18:58
no-defun-allowed
I usually use lparallel and bordeaux-threads for stuff so I'm not paying much attention, sorry.
21:19:58
jasom
aeth: threads predate parallelism by a lot on PCs. In the 90s a lot of code was written multithreaded to avoid io blocking.
21:20:58
no-defun-allowed
cl-decentralise spawns a thread per connection since they actually do things in SBCL instead of splitting up one interpreter's time (cough CPython cough).
21:23:12
no-defun-allowed
cl-vep uses one generator and a lock for the generator. When a worker needs a new frame it holds the lock and funcalls the generator. This method (work "source" and lock) hasn't given me issues in the past.
21:23:52
no-defun-allowed
I've done high thread counts to avoid http blocking too. That method scaled pretty well up to 64 threads.
21:24:29
no-defun-allowed
However, the ffmpeg interface for cl-vep is shit slow and only puts out 4fps and only makes a load of 1.5.
21:24:41
jasom
aeth: I just spawn more worker threads. I haven't had any issue at 16 worker threads on my 8/16 ryzen
21:25:07
aeth
jasom: anyway, what I mean is things that are architectured so that they'd benefit from e.g. if AMD released a 64 core CPU tomorrow.
21:28:32
makomo
why is the syntax grammar for DEFSTRUCT so weird? i don't get what the keywords/singleton lists are supposed to do http://www.lispworks.com/documentation/lw70/CLHS/Body/m_defstr.htm
21:29:40
makomo
why would you want just :conc-name? is that supposed to construct (within the context of the grammar) a non-list version of (:conc-name conc-name)?
21:30:27
Bike
"A defstruct option can be either a keyword or a list of a keyword and arguments for that keyword; specifying the keyword by itself is equivalent to specifying a list consisting of the keyword and no arguments."
21:31:23
makomo
Bike: guess i should have read further, but it confused me right away so i didn't bother :c. thanks
21:33:42
jasom
aeth: lparallel has bounded queues implemented with vectors; I haven't checked the code, but it ought-not be consing if implemented right.
21:34:28
aeth
jasom: I'm guessing you'd queue or pass messages with things that are unboxed (like fixnums) or with keywords/symbols
21:35:27
jasom
aeth: setf on an array is going to be non-consing. Creating the value to setf in that locatin *might* be consing
21:36:28
jasom
aeth: I'm pretty sure an unspecialized array will not cons if you do something like (setf (aref foo) 3) since 3 is a fixnum
21:36:28
aeth
Would you sync with timers or would you have a thread sleep when done until it receives a start-working-again call?
21:44:49
aeth
Oh, there *is* a Threadripper 32 core (64 thread) CPU, the 2990WX. Wikipedia just put it in a separate chart much further down because it's Zen+.
21:46:55
aeth
So there is a chance that a (rich) Lisper is already running an application with performance on architecture close to manycore in mind (if you count the 64 threads via SMT)
21:47:37
jasom
aeth: Talos is available with 128T (or maybe even more) configurations since POWER8 is 4T per core
21:49:52
jasom
2x 22-core POWER9 is the highest configuration which translates to 196 threads I think?
21:49:54
aeth
Is PPC compatible with POWER8? I'm not sure if any CL runs on POWER8 if not. SBCL seems to have PPC and PPC64.
21:50:52
jasom
POWER9 is PPC64, but I think the talos is designed to run in little-endian mode, and I don't know if SBCL does that.
21:51:31
aeth
PPC64 is marked as in progress on the SBCL download page. I wouldn't be surprised if threading doesn't work
21:52:57
aeth
I wonder if there's a 2 socket AMD Threadripper mobo. That would be 2x 2x 32 at a max (right now... expect it to double next gen), so 128.
21:52:59
jasom
and the IBM E980 can support up to 16 12 core SMT8 processors for more cores than I care to do the math on
21:55:00
aeth
jasom: But yes, IBM's POWER is definitely worth mentioning in this kind of discussion. Last I checked, IBM seems to be keeping up with Intel, which is 10ish cores behind AMD.
21:56:39
aeth
A quick search has a reddit thread where the answer is "Threadripper is single socket only, for dual you'll need to go with Epyc." https://www.reddit.com/r/Amd/comments/6tqg88/dual_socket_threadripper_max_ram/dlmt8fn/
21:56:50
aeth
Of course, random reddit comments are even less reliable than random StackOverflow answers
21:57:48
aeth
(That's probably one of the ways they get people to pay 2x to 4x as much as the consumer line.)
22:17:08
Demosthenex
is there a way to start sbcl with swank using a port number from the command line? i'm trying to work on a few different things on a remote host, and i can connect with ssh/port forwarding. just starting sbcl with swank on separate ports is awkward
22:48:29
aeth
Wikipedia says POWER9 goes up to 24-core (96 threads via SMT4?) https://en.wikipedia.org/wiki/POWER9
22:53:12
aeth
It looks like both AMD and IBM are going to release 48-core CPUs very soon, though. It looks like Qualcomm already has a 48-core ARM server.
23:00:00
Bike
programs relying on particular details of the machine they're running on is something to be avoided. surely this is a basic principle
23:03:27
aeth
Bike: The architecture can also run on an 4 core machine, yes, but there's less of a motivation for it there.
23:05:29
aeth
Bike: But consider e.g. a game or similar application that can be divided into several systems. In today's world, if it's parallel at all, each major system would probably get its own thread. That'll only scale up to the number of major systems, which is probably far less than 90.
23:07:22
Bike
in that case whether you have five or ninety processes isn't important for basic architecture.
23:08:18
aeth
Except for the most part, afaik what you'd typically see when the end user has quad core is maybe putting one task on each a thread to be "parallel", not also subdividing those tasks.
23:10:40
Bike
if you already have an idea for how to structure a program for multiprocessing, what are you even asking?
23:13:32
aeth
Bike: I'm wondering if anyone has successfully deployed a CL application that takes advantage of one of these CPUs with a "very high" core count. ("very high" being relative to the normal 2-4 cores that you would have seen for this kind of hardware until recently)
23:14:23
Bike
the machine i use to build clasp has thirty something cores. during build we have each core do a compile-file so we can compile a bunch of files at once.
23:15:04
whartung
I mainge someone one has. At those core counts, you’re getting into “super computing” kinds of designs and applications.
23:15:53
whartung
Funny Bike, back in the day, when I had the CPU meter on my desktop, I’d just kick off builds until it reach 90%. … by hand. Whenever it dipped down, I’d kick off another one.
23:18:37
whartung
My friend laments how horrible Swift builds are on the Mac — it’s apparently very CPU intensive.
23:22:37
aeth
Bike: I would personally probably just manually implement the required algorithms and data structures when needed in my case rather than using a library. Micromanaging this sort of thing is kind of a key point of the engine (when I get there), not a secondary feature.
23:24:21
aeth
It's mainly due to the requirements for 100 FPS logic and no consing at the library level in the game loop (I have consed in all of my substantial tests, of course... too hard not to... but that's in the user's code)
23:27:44
aeth
no-defun-allowed: A 3D game engine. I switched my original design (nothing is stopping it from being general purpose, of course) from first person to third person strategy (i.e. far away camera) because of fewer physics requirements (otherwise the project would become more about writing a physics engine than a game engine)
23:31:06
aeth
I just need to do mouse selection of an entity and a destination and then I have arbitrary movement on a 2D plane within the 3D world (e.g. ships on water). After that (soundless) games would technically be possible, although still fairly difficult because the text rendering isn't complete yet.
23:33:59
aeth
I'm guessing the input and the final update of the authoritative game state would have to stay on a core thread.
23:36:38
whartung
as I understand it, it’s prety difficult to multitask the (a?) rendering pipeline.
23:37:14
whartung
you can easily have a rendering thread, a state thread, sevearl AI threads. But the busy thread is typically the rendering thread.
23:37:26
Bike
you don't really have the opportunity anyway since the graphics library wants to do rendering in mysterious ways
23:37:48
jasom
Bike: there are some tradeoffs you can make; e.g. make the serial performance much worse in trade for more parallelism; the number of cores you have can decide if you will make that tradeoff
23:38:10
Bike
mostly because rendering is so hard we've farmed it off to repurposed bitcoin-mining chips
23:38:39
whartung
well, they happen to be pretty good at the job. If they weren’t we wouldn’t be handing them off to it.
23:40:26
whartung
as I understand it, the games job is to get the models oriented properly, send them to the GPU, and make sure the proper textures are loaded at the same time. The GPU does the actual rendering stuff.
23:40:45
jasom
aeth: my most recent webapp will probably scale well past 8 cores; all the state is in the database and each worker thread has its own DB connection. It's when you get past 32 cores that things can get sticky.
23:44:37
whartung
but the gpu guys have a way of getting it to work on multiple GPUs (2 GPUs, alternatiing scan lines or something like that)
23:46:35
aeth
whartung: It's generally not worth it to have multiple GPUs. The game (or application) has to specifically support it (and probably support it differently for nvidia and AMD) and it's usually better just to get the next tier up in graphics cards so it probably only applies to you if you're using Titans.
23:48:32
aeth
For nvidia it's called SLI and the general advice on forums has been to not do it, at least for the past few years.
23:49:25
White_Flame
whartung: different generations had different limitations. I know there were 3x linked card setups,a nd probably 4 as well
23:49:56
White_Flame
I think some also just send along the PCIe bus instead of having their own inter-card connectors
23:50:06
aeth
See https://en.wikipedia.org/wiki/Scalable_Link_Interface and https://en.wikipedia.org/wiki/AMD_CrossFireX
23:50:26
aeth
but apparently nvidia just abandoned SLI on the next generation (that isn't out yet) in favor of "NVLink Bridge"
23:52:52
whartung
I’ve spent enough treasure on computer equipment, I’m glad I was never really in the GPU chase :)
23:53:15
White_Flame
nvidia is going off the deep end with prices anyway, best to ignore it for now, too
23:53:28
aeth
But when I was saying that rendering is heavy, I mean heavy on the CPU. The GPU is doing most of the work, but #'draw still seems to be the most CPU intensive function at the moment for me. I think Vulkan would allow this to be split up, though.
23:53:59
White_Flame
no-defun-allowed: most lisps are fully reflective, if you use their implementation internals
23:54:21
aeth
White_Flame: You see that all over the place. CPUs, GPUs, smartphones, etc. The high end is improving, but also getting more expensive.
23:54:53
White_Flame
aeth: RAM prices are settling down, though, finally returning to where they were 3 years ago or wherever it was before that spike
23:54:59
whartung
depends on how high powerful the polisher you’re using no-defun-allowed, and what kind of compound you’re using
23:56:46
whartung
no, I understand aeth. Just getting the frames set up. A lot of that is the modeling side (modeling the car(s) for example), and things like the physics engine. Is it worth while to farm our modeling the suspension dymanics of 40 cars in a nascar sim to different CPUs per frame? I dunno. it may not even be possible if you’re modeling drafting and what not.
23:58:42
aeth
whartung: but on the other hand I think a large part of strategy games is embarassingly parallel, or close enough.
23:59:09
whartung
yea it can be, quesiton is whether it’s cpu worthy enough to bother breaking it out.
23:59:56
aeth
I've definitely played some building strategy games (games with workers doing tasks and carrying resources around a village) where it uses all the CPU
0:00:05
whartung
The only time I upgraded a machine for a better game performance (among other things) was way back when I went from 133 to 400 for Total Annihilation. So it coul dhandle more units. But even that would be difficult to paralleize because of the dependencies on each other (notably in traffic jams)
0:00:56
aeth
Even without htop up, I can tell when my desktop's fully using the CPU in a multithreaded way because I can hear the CPU fan when that happens.
0:01:44
whartung
my friends laptop turned in to an air hockey table when compiling Swift. You’d think it would float off the desk...
0:02:28
aeth
In general, simulation(ish) strategy games seem to max out the CPU and recent AAA games tend to max out the GPU
0:02:55
aeth
(I don't think both would be possible, unless you wrote specifically to the hardware combination and optimized heavily.)
0:05:17
jasom
aeth: do you sometimes render inconsistent state when you do that (i.e. some objects 10ms behind others)?
0:06:13
aeth
Oh and 100 fps for logic because its inverse is 10 ms and what you want is (integerp (* 1000 (/ n))) so both measurements are integers imo.
0:07:28
whartung
unrelated, are they any good examples of network servers in CL floating around on the net? I’ve seen the http server (Hunchenroot? Is that right?) Just curious if there are others of that ilk I could look at.
0:08:08
aeth
In case anyone's wondering about the other numbers... (loop for i from 1 to 2000 when (integerp (* 1000 (/ i))) collect i) => (1 2 4 5 8 10 20 25 40 50 100 125 200 250 500 1000)
0:10:24
aeth
Looks like it's 1 and numbers whose only prime factors are 5 and 2. (I ran my factoring function on that list to verify)
0:14:05
aeth
(limit in the sense that you can't get better than 1000, not in the calculus sense of the word "limit")
0:20:05
aeth
Bike: I didn't expect the fundamental theorem of arithmetic but I guess I should have expected it as soon as INTEGERP entered
0:20:56
aeth
Here I was working with periods and frequencies in a real world problem and then suddenly PRIMES
0:24:14
Bike
people are all like one plus one is two, well in my opinion it makes more sense to define two as the octave
0:27:07
aeth
Bike: But this is sort of the same problem. With the logic running at 100 ticks a second, some parts might actually only need to run at 50 or 25 or even 1. And this can be determined with a simple MOD
2:04:10
stacksmith
Is there a foreign pointer validation function in CFFI? To check if dereferencing would crash...
2:07:43
stacksmith
I was hoping for a less-than-perfect validation - like check if it's in-range for any areas allocated with CFFI. But yeah, that was wishful thinking.
4:01:33
no-defun-allowed
no, this is the right channel, stacksmith was asking about bounds checking for cffi arrays
4:02:54
beach
I should be quiet. Whenever I complain about the fact that half of the contents here is about other languages, I am told that I am wrong to complain.
4:12:40
no-defun-allowed
are there any good implementations of unordered sets that can be looked up in less than O(n) time?
4:13:50
beach
no-defun-allowed: All discussions of sets in the CS literature are about special subclasses of sets.
4:14:20
beach
no-defun-allowed: And that's because there is nothing better than O(n) in the general case.
4:16:56
beach
If your keys are simple and you have a lot of elements, then they are typically better than lists.
4:51:33
White_Flame
no-defun-allowed: a rule of thumb from other languages is that around 10 elements is where hash tables tend to get faster than iterating a list. I would suspect CL's tradeoff (again, for simple keys that dont' take forever to hash) would be in the same ballpark
5:14:51
aeth
even if a plist is slightly faster at 3-5 elements, I didn't notice it last time I benchmarked it because both are just so fast.
5:19:55
aeth
I'd personally say hash-table vs. plist is whatever semantically seems to make more sense for a few elements. Above a few dozen, use a hash-table unless you're only sequentially iterating over it with something like alexandria:doplist or you're in a macro (where you're probably sequentially iterating over it anyway).
5:51:03
cobax
Shinmera: any names you can throw at me would be appreciated, so I can read up some more on the alternatives
5:52:25
White_Flame
if by "orthogonal" you mean transparent, that model tends to be underrepresented because it's quite difficult and usually doesn't work well
5:53:39
cobax
That is what I mean, yes. Difficult to whom, is the question. Postgres is quite easy for its programmers but hard for whomever needs to install and maintain and deploy it. On the other hand...
5:53:43
White_Flame
if the object model is integrated with the OS itself in a fully persistent address space like some mainframe & experimental OSes, then it would work better. But application-level, not really
5:54:41
cobax
White_Flame: I'm not sure all that is needed. Ruby doesn't have that and there is Maglev for it.
5:55:05
aeth
Lispers seem to prefer just representing SQL as s-expressions (nothing is stopping someone from writing an SQL database in CL, though)
5:56:37
White_Flame
cobax: well, if you just want persistence without performance, then it wouldn't be hard to pull off
5:57:51
cobax
So now you know I'll be abusing whatever persistence library as a replacement for an RDBMS (I don't care about performance and joins) then what are the current persistence libraries?
6:00:50
cobax
White_Flame: the angle I'm coming from is, Lisp is image-based, but how can I take easy/comfortable advantage of this fact, to make small apps without a database, by persisting my environment, like Smalltalk does?
6:02:08
White_Flame
so certainly on exit of your application, you can have it save the image & exit, and your startup script would resume from that image
6:03:03
White_Flame
but if you crash, the saved image's heap wasn't constantly updated with your running heap or anything
6:03:28
cobax
Ok, so tell me where I'm wrong here, then an easy way to make a prototype website with db functionality but without postgres, instead relying on CL's image-based-ness, then I can deploy a CL app and take a snapshot every 24 hours for data backup?
6:04:21
White_Flame
if you want to use CL images, then that's taking manual snapshots of the entire CL runtime
6:04:24
cobax
I don't know what you mean by "DB-like functionality". an RDBMS has several aspects to it. For instance, I don't need the fact that it's relational.
6:06:31
White_Flame
SBCL, for instance, has SAVE-LISP-AND-DIE as its only image writing function (as far as I'm aware). I don't know how well other systems deal with saving an image but not exiting
6:06:53
White_Flame
because bundling that all up for a resumable disk state sometimes involves closing stuff out
6:08:03
White_Flame
most deployed Lisp images don't constantly re-save themselves; they're like standard deployed applications where the image on disk starts up adn then reads its application data from other files/DBs
6:08:37
White_Flame
but you can certainly try an image-based application persistence model. I don't think there's anything preventing it; it's just uncommon
6:09:14
White_Flame
especially if you're fine with it restarting every 24 hours for snapshotting & flushing purposes
6:10:39
cobax
White_Flame: I have many books on Lisp and its implementations and it seems as if the "image-based" aspect is less explored than in smalltalk, would you agree?
6:10:59
cobax
for instance, even Picolisp seems to be ahead of Common-Lisp in terms of a rich image-based-ness?
6:11:08
White_Flame
it was more useful in ye olde days where reinitializing everything would take way too long
6:11:36
White_Flame
nowadays, you can cycle in a fresh Lisp image, load & initialize all your code in seconds
6:12:26
cobax
White_Flame: I understand, but you see, then an advantage is lost, namely the ability to exploit the image-based-ness for purposes of long term storage
6:13:00
White_Flame
cobax: most people store long term data as data, not all wound up as artifacts inside a runnable Lisp heap
6:13:47
White_Flame
especially if you save non-executable images, the image format can change across minor versions
6:16:42
cobax
I think it starts with good serialization. what is the best CL serializer, is what I should seek next.
6:16:59
aeth
You probably don't want to leave an image up too long in development because you can have bugs with "stale" functions. I think the most common one is when you rename foo to foobar, but don't change all calls to foo. (1) calls to foo will no longer get your updates to foobar and (2) your program will not start in a fresh image
6:17:57
White_Flame
cobax: I never use general serializers, because I always end up wanting to serialize closures
6:17:59
aeth
Now embed a db in an image and this sort of thing becomes a lot more common, where you cannot get to where you are from a fresh start
6:19:14
cobax
for instance, think of cl-swap-file and clufs, if I store serialized lisp objects there, I pretty much have an object data base, no?
6:21:05
cobax
ok, so help me out, and I appreciate all this help so far, and understand you are away from keyboard now, but...
6:21:54
cobax
if I have serialization of the objects I'm interested in storing, and I have a pure common-lisp file system / swap thing like clufs, then I have an object database for the objects I care about?
6:36:05
MichaelRaskin
I think you won't get true orthogonality, because long-term-compatible serialisation of functions is not always available.
6:45:31
MichaelRaskin
Well, if you ask for transparency and don't list any limitations, there is a question can a slot value be a function. Or maybe even a class reference
6:46:23
cobax
MichaelRaskin: Ah, yes I understand that difficulty, but yet the few orthogonal transparency (or near that) solution I've seen for Smalltalk all do that without problems
6:49:47
cobax
If I may paste this here for reference, this is how maglev / gemstone serializes things:
6:49:49
cobax
"Ruby classes are stored as bytecode in the stone server, which is transported via shared memory for local connections, and via optimized binary protocol for remote connections, to the local interpreter and then compiled down to native machine code."
6:50:52
cobax
so if by serialization we mean down to bytecode, then serializing anything is suddenly doable, including functions but perhaps not class references?
6:57:44
MichaelRaskin
I guess it all depends on design choices, especially when you take the optimiser into account
6:57:57
cobax
hmm I found this which seems to have many answers from a GemStone engineer: https://maglevity.wordpress.com/2010/01/12/gemstone-internals-videos/
7:04:32
White_Flame
but serializing byte code compiled from source code is not equivalent to serializing runtime state
7:04:55
White_Flame
again, closure objects and such, not to mention the baseline issue of cross-module object references
7:05:40
White_Flame
I do think your problem is underspecified as well, at least from what has been said here
7:15:58
White_Flame
looking through the gemstone videos, it points back to what I said before about the object system being integrated all the way down to the running platform (not really OS in this case, but same idea) in order to implement it properly
7:22:48
White_Flame
there are also quite a number of limitations and inflexibilities compared to Lisp objects
7:24:12
cobax
wait, but Gemstone allows serialization of objects etc, so they've solved the problem somehow, no?
7:24:30
xificurC
I put "yason" into :depends-on in my my-system.asd. Now when I try (asdf:load-system "my-system") it errors with "Component yason not found, required by ...". Obviously I'm doing things the wrong way? I thought depending on a system would make asdf call out quicklisp to fetch the system
7:25:17
beach
You either need to compile your system with quicklisp or call quicklisp manually to install the dependencies.
7:25:52
White_Flame
cobax: they built a database system. They didn't bolt on serialization on top of an existing runtime
7:26:24
cobax
I came to the same realization. I guess the solution is to start with a database, and put the language I want in control of it
7:26:35
White_Flame
so if you want the same, then build up a Lisp enironment that is a database system
7:27:12
cobax
White_Flame: I'm trying to understand, and I really appreciate all your help, but I'm not sure I understand if Picolisp is that or not
7:28:47
cobax
"distributed ruby with the maglev vm" is the name of one article. maglev is gemstone. so it's distributed.
7:30:28
White_Flame
and looking at the docs, "Whenever an external symbol's value or property list is accessed, it will be automatically fetched into memory, and can then be used like any other symbol." this is not library functionality. It's again wound right into the low-level definition of memory access
7:30:57
White_Flame
now, certainly this can be done with CLOS, but only with CLOS, not any general slot reference
7:32:16
xificurC
White_Flame: picolisp is dynamic, no lexical context to carry. A function is just a list. There's no lambda, just quote, so '((X Y) (+ X Y)) is really just a list, written with QUOTE directly (quote (X Y) (+ X Y))
7:32:37
cobax
I'm not quite sure I can appreciate, from what you're saying, what is it exactly that low-level integration can afford that very good serialization cannot. The problem is around references, it seems, but I don't get it.
7:33:02
White_Flame
cobax: you can't really change how memory references are done in a library. It's part of the fundamental VM
7:33:55
White_Flame
all these systems perform special things on memory reads, loading things transparently
7:34:14
cobax
White_Flame: I understand now. So let's call that the last-mile of orthogonal persistency :)
7:35:14
cobax
xificurC: can you say a bit more about how being dynamic and not having to carry lexical context can afford Picolisp... what exactly? Distributability of serialized objects the way I am talking about?
7:36:56
White_Flame
hmm, in picolisp, external symbol names are brace-surrounded disk locations where their data is stored
7:37:18
Demosthenex
so in emacs slime, M-. is bound to slime-edit-definition which fails to jump to my function defs in the same file, but xref-find-definitions works. any suggestions for how to fix it, or shall i just override the keybindings?
7:38:30
White_Flame
cobax: yeah, hopefully your takeaway is that this sort of thing is not really library functionality. It's fundamental language/runtime functionality
7:38:40
trittweiler
xref-find-definitions might just search for a regexp in the current buffer. slime-edit-definition works on source location information stored in your Lisp session
7:39:10
White_Flame
cobax: and that there are extreme design considerations for all of these sorts of systems; they're not as flexible as a Common Lisp environment, because they need to support that persistence
7:39:49
Demosthenex
trittweiler: i'm editing a buffer that's been completely eval'ed into the running slime session
7:40:05
Demosthenex
trittweiler: and the def is two pages back in my source, so a simple backward regexp find easily can find it
7:40:39
cobax
White_Flame: I understand the loss of flexibility, but I am fascinated by how we took this path of fragmented development instead of integrated programming systems which in my opinion are more productive
7:41:27
Demosthenex
trittweiler: wait, i just had a thought. my source file is local and my sbcl process is on a remote host. though i did compile and execute the functions in emacs, is slime trying to use the filesystem maybe?
7:42:01
cobax
White_Flame: constraints help correctness if they still allow all the same programs to be made, so those are good things, or can be
7:42:26
White_Flame
I'm talking about feature & performance constraints, not towards-working-code constraints
7:42:57
White_Flame
sometimes when you're constrained you need to write spaghetti in order to implement something mor ecomlex, and that doesn't help correctness at all
7:43:10
cobax
I understand, and to me the interesting aspect is precisely the productivity of integrated environments
7:44:08
White_Flame
I've written OO systems in limited BASIC environments. The constraints of BASIC didn't help.
7:44:49
cobax
White_Flame: the constraints of Gemstone seem to be helping lots of people who keep them employed, so there seems to be a market and people that agree with me that the trade offs are worth it
7:45:14
xificurC
cobax: since functions don't carry a lexical context and are not compiled, they are just a list
7:45:32
White_Flame
certainly Common Lisp meets this definition of "integrated" because it doesn't have modern OS interactions bult in the spec (no threading, networking, process spawn, etc). That means we have to jump through hoops & ad hoc systems to get things done
7:46:24
xificurC
White_Flame: indeed persistent symbols are "special" in that regard. But you are able to work with them the same way as with any other symbol
7:46:40
cobax
hmm I don't want to talk about specs, I want to stay in the realm of practicality and productivity
7:47:14
cobax
not vague, I mean integrated like a smalltalk programming system like squeak or gemstone
7:47:47
cobax
and I want to eliminate the hoops of having to keep maintaining separate things from one's programming language, such as postgres
7:48:35
cobax
White_Flame: no, but they seem to be more productive for CRUD websites than most current development workflows
7:48:49
White_Flame
right, so you have a specific context there, of database systems. Many programs don't have databases and don't need those features; any productivity there is contextualized
7:51:16
xificurC
cobax: depends on what you're building really. There's plenty of places to keep non-persistent state in a website
7:52:03
cobax
well, be that as it may, it is usual to have a database, and I think it is more productive to have the most things integrated possible
7:52:03
xificurC
if you're building a website that shows a company's portfolio or an association they might not have e-shops and forums and registrations etc. Just a plain old hyperlinked site with no fuss
7:52:40
cobax
xificurC: sure, but I don't want to have to switch technologies between clients that want a DB and those that don't... the point is to streamline production
7:53:21
xificurC
cobax: I was just saying "the context is making website" is still not that clear. Different requirements ask for different solutions. If you need databases, sure.
7:53:54
Demosthenex
cobax: don't most websites export state information to the database so the codebase can be considered stateless?
7:54:04
shrdlu68
Well well, since I can't define a subclass of a bit-vector, is there another way to create a thing that behaves like a bit vector?
7:55:24
cobax
sorry folks I'm just tired of having to fix postgres, redis, this that and the other, I just want to code in a productive environment that doesn't switch my attention off of programming
7:55:45
Demosthenex
cobax: your complaint is about integration and systems administration for dependencies then
7:56:09
cobax
Demosthenex: yes and I feel Smalltalk solves this but is too heavy a hammer... and that there should be a lispy equivalent
7:56:27
Demosthenex
cobax: which is why cloud services are so popular now, you export the care, feeding, administration and ownership to a third party and pay for the priviledge of using it
7:56:32
White_Flame
and "it should just work!" for a plethora of use cases, each with their own individual complexities
7:57:21
Demosthenex
cobax: you said you were tired of fixing dependent products. i said cloud services allow you to pay for a maintained product.
7:58:03
cobax
Demosthenex: well then it's broader than that. having to code against amazon's services doesn't contribute to orthogonal transparency
7:58:06
Demosthenex
i am mistaken that one of the advantages of exporting all application state to the DB means that you can use concurrent functional stateless code at the webserver layer?
7:58:46
White_Flame
cobax: your problem isn't orthogonal transparency. It's something that you think would be a silver bullet to your problems, but I can assure you it isn't
7:59:10
cobax
Demosthenex: look at GOODS, it is open source. what I am saying is not Amazon Services, it is things in the direction of Gemstone and GOODS
8:00:39
cobax
if Amazon had their own Ruby in which I can call an amazon lambda natively, then that is more what I mean
8:01:00
Demosthenex
so you want a programming language pre-integrated into a cloud environment where all teh dependencies are handled for you?
8:01:56
cobax
Demosthenex: yes! otherwise what is the point of integration, if I have to "do it all over again" in code
8:02:51
cobax
White_Flame: well I'm glad nothing stopped the gemstone people from giving this crazy idea a go
8:03:38
Demosthenex
cobax: you should compare that to 4gl solutions like openedge progress, ibm's rational 4gl, etc.
8:03:46
White_Flame
right, they have a niche they're serving. If you want fully integrated, does-anything, no-config, no-admin general programming, well then wait for AI
8:04:45
cobax
White_Flame: it's interesting, when gemstone released maglev, most of the Ruby community didn't get it either. maybe we're stuck with clunky database code forever
8:05:08
Demosthenex
because that's a common thing. 4gl's were popular because appdevs wanted to work in an environment where presentation and storage were already solved. if you use a generic language (python, CL, java, etc) you have to do it all over again yourself, or use a framework
8:05:23
White_Flame
and for many, the database interfaces mesh with the transactional level and robustness considerations they need to manage
8:05:53
White_Flame
all of the "hide the details" systems don't break in survivable ways as mature DBMSes. That's why the latter are still aroudn
8:06:00
cobax
Demosthenex: everyone that codes for the web should want to "work in an environment where presentation and storage are already solved" no? :)
8:06:19
Demosthenex
cobax: no, i get that. but isn't that why there are so many web frameworks? they are the new 4gl
8:06:44
cobax
Demosthenex: but like White_Flame said, frameworks will never integrate at the binary level... hence they won't solve this problem
8:06:45
White_Flame
they focus on one particular facet of the tech; your engineering work is still to be done on top fo that
8:07:22
cobax
White_Flame: I totally appreciate your point about survivability and breakage. I agree.
8:07:50
cobax
Demosthenex: I had seen the term "4gl" but am not too familiar. perhaps there was some boat I missed.
8:08:00
Demosthenex
the tradeoff for 4gl style languages is you are bound by the limits the designers made for you
8:08:37
Demosthenex
cobax: ibm rational is all i can think of, it's really outside my area of expertise
8:09:51
Demosthenex
the 4gl's i have been exposed to are things customer applications are written in that i've had to support.
8:10:30
cobax
Demosthenex: too bad much material on openedge progress is behind a paywall, I'd love to see their REST example
8:10:47
Demosthenex
cobax: https://en.wikipedia.org/wiki/Fourth-generation_programming_language "Languages claimed to be 4GL may include support for database management, report generation, mathematical optimization, GUI development, or web development."
8:11:16
Demosthenex
frankly, i haven't ever seen a OSS 4gl outside of the xclip/harbor project, which is a OSS implementation of CLIPPER, an 80's application language
8:12:00
cobax
ah that's it then! yes, Clarion is there :) that's what I mean then, 4gl, it's a shame we don't do more 4gl...
8:12:44
cobax
Demosthenex: what would a 4gl for terminal apps look like? it would make curses first-class somehow?
8:15:00
cobax
Demosthenex: I feel someone needs to do something about my complaint and currently I am researching solutions that I may bring to the market
8:15:49
Demosthenex
i understood 4gl's to integrate common application requirements, db/object integration, reporting, forms interfaces, etc.
8:15:54
cobax
xificurC: I guess it is 4gl if the context is "operating system calls" but then we're stretching the meaning of the term beyond usefulness
8:16:16
Demosthenex
cobax: 4gl's are really good at making simple apps and for prototyping complex ones
8:16:39
Demosthenex
cobax: big apps often outgrow them because of the limitations that integration brings, however, that doesn't sound like your issue ;]
8:17:06
cobax
Demosthenex: thanks for the link, I have provided tech support for many dbase applications I suppose! also looks as/400-ish
8:17:40
Demosthenex
i grew up using Borland products that all had TurboVision for text user interfaces, and that's virtually vanished
8:18:10
Demosthenex
i absolutely hate user interface creation, or i'd have made my own curses lib by now.
8:18:17
cobax
Demosthenex: well, I just think that today with the processing power we have, we are wasting too much time on rails and jango and postgres when most clients needs would never surpass those integration limitations
8:18:42
Demosthenex
cobax: think with the processing power today how much time is wasted on the entire webstack, client and server.
8:19:18
cobax
Demosthenex: paint this picture for me so I can better understand it, what does Picolisp lack from your ideal dbase/4gl for terminal apps?
8:21:03
Demosthenex
cobax: their example app with their builtin html application stuff looks pretty simple
8:22:17
cobax
I've decided my next project will be in a language with an integrated database and picolisp seems like the suitor
8:24:24
Demosthenex
cobax: i can't find screenshots for their example html application, link if you find anyone or see one running