freenode/#lisp - IRC Chatlog
Search
18:18:58
shka_
but i like how i can use asdf extension to specify :test-file and i can simply use is and ok assertion
18:19:10
j`ey
this is my testing framework: (defun test (val) (cond ((eq val nil) (princ "E")) (t (princ "."))))
18:19:52
pfdietz
The question you want to ask is not "which testing framework?" but rather "what specifically do you want from a testing framework?"
18:27:33
beach
I am totally convince that this is one abstraction that everyone is looking for but that isn't possible in general.
18:28:49
beach
But the fact that there are so many and that people are not satisfied with what exists, tells me that there is a problem.
18:31:30
pfdietz
There are ten copies of RT in Quicklisp, last I checked. Not only are they easy to write, they're easy to copy. :)
18:39:42
jackdaniel
testing is not an easy task so many people have different ideas how the tool simplifying that should look like
18:43:29
jasom
beach: there are so many testing frameworks for the same reason I accidentally wrote a routing library for clack
18:44:06
jasom
beach: at first I didn't really need a routing library, so I didn't use one. Then I added a couple of features as I needed them. Then I realized that if I just moved it to a separate package, I had a complete routing library...
18:45:29
jasom
just like a testing framework it's easy to grow organically as you need more features *and* not much code when it's done: https://github.com/jasom/cl-fccs/blob/master/src/route.lisp
18:47:37
pjb
jackdaniel: right. IME, testing depends on the kind of software you are testing. Basically, most testing libraries are just unsuitable for a lot of software I had to test.
18:50:35
pjb
perhaps it would be more worthwhile to write down specifications, eg. of testing framework interfaces to testing report systems. eg. https://testanything.org
18:52:07
pjb
If asdf:test-op was specified to produce on *standard-output* a TAP stream, quicklisp could parse the results and produce nice reports…
18:53:14
pjb
You could use any testing framework, even multiple frameworks in a big system, as long as they would all follow a unique protocol, collected and interpreted by the surrounding tools and UI, it would be nice, integrated, and useful.
19:13:49
vsync
before I wanted to use it just because it was out of fashion and I'm a weirdo even among the weirdos
19:14:18
vsync
never had an actual use though... but lately I have found one where you want to have an inner named closure
19:16:39
vsync
though mine is for healthcare provider coordination data exchange between health systems and EMRs
19:17:07
vsync
for that purpose but generalizable! just like embedded in many other systems I'm sure, heh
19:18:33
pjb
vsync: the real use case for &aux, is when you write a macro and you need to add variables around a &body docstring-declarations-and-body <- NOT merely BODY!!!
19:19:14
pjb
I don't mind indentation. It's done automatically, and I've got a 5000+ pixel wide screen :-)
19:22:53
vsync
yeah I stumbled across it the other day when I didn't have CLOC installed and until my clean kernel compile happens the system crashes if I access a reiserfs and then a jfs
19:25:09
vsync
need to leave 3 extra characters for the minimum log graph view with diff, plus 1 for some terms wrap on the last column uselessly somewhy
19:25:37
stylewarning
vsync: sometimes a character in the last column is reserved for an overflow character
19:26:03
vsync
stylewarning: makes sense for consistency... like do you indent context lines in diff 1 char? I say yes
19:30:50
pjb
Well the max line length is not good. You'd have to look at the typical line length. My files contain outliers in comments or in data. Sometimes in code, but rarely.
20:12:48
makomo
ugh, i don't like it when i have to use composite keys for hash tables, it feels kinda clunky
21:15:32
aeth
Does anyone run CL on a processor with many cores? AMD has some 16 core / 32 thread and even (for servers) 32 core / 64 thread.
21:18:58
no-defun-allowed
I usually use lparallel and bordeaux-threads for stuff so I'm not paying much attention, sorry.
21:19:58
jasom
aeth: threads predate parallelism by a lot on PCs. In the 90s a lot of code was written multithreaded to avoid io blocking.
21:20:58
no-defun-allowed
cl-decentralise spawns a thread per connection since they actually do things in SBCL instead of splitting up one interpreter's time (cough CPython cough).
21:23:12
no-defun-allowed
cl-vep uses one generator and a lock for the generator. When a worker needs a new frame it holds the lock and funcalls the generator. This method (work "source" and lock) hasn't given me issues in the past.
21:23:52
no-defun-allowed
I've done high thread counts to avoid http blocking too. That method scaled pretty well up to 64 threads.
21:24:29
no-defun-allowed
However, the ffmpeg interface for cl-vep is shit slow and only puts out 4fps and only makes a load of 1.5.
21:24:41
jasom
aeth: I just spawn more worker threads. I haven't had any issue at 16 worker threads on my 8/16 ryzen
21:25:07
aeth
jasom: anyway, what I mean is things that are architectured so that they'd benefit from e.g. if AMD released a 64 core CPU tomorrow.
21:28:32
makomo
why is the syntax grammar for DEFSTRUCT so weird? i don't get what the keywords/singleton lists are supposed to do http://www.lispworks.com/documentation/lw70/CLHS/Body/m_defstr.htm
21:29:40
makomo
why would you want just :conc-name? is that supposed to construct (within the context of the grammar) a non-list version of (:conc-name conc-name)?
21:30:27
Bike
"A defstruct option can be either a keyword or a list of a keyword and arguments for that keyword; specifying the keyword by itself is equivalent to specifying a list consisting of the keyword and no arguments."
21:31:23
makomo
Bike: guess i should have read further, but it confused me right away so i didn't bother :c. thanks
21:33:42
jasom
aeth: lparallel has bounded queues implemented with vectors; I haven't checked the code, but it ought-not be consing if implemented right.
21:34:28
aeth
jasom: I'm guessing you'd queue or pass messages with things that are unboxed (like fixnums) or with keywords/symbols
21:35:27
jasom
aeth: setf on an array is going to be non-consing. Creating the value to setf in that locatin *might* be consing
21:36:28
jasom
aeth: I'm pretty sure an unspecialized array will not cons if you do something like (setf (aref foo) 3) since 3 is a fixnum
21:36:28
aeth
Would you sync with timers or would you have a thread sleep when done until it receives a start-working-again call?
21:44:49
aeth
Oh, there *is* a Threadripper 32 core (64 thread) CPU, the 2990WX. Wikipedia just put it in a separate chart much further down because it's Zen+.
21:46:55
aeth
So there is a chance that a (rich) Lisper is already running an application with performance on architecture close to manycore in mind (if you count the 64 threads via SMT)
21:47:37
jasom
aeth: Talos is available with 128T (or maybe even more) configurations since POWER8 is 4T per core
21:49:52
jasom
2x 22-core POWER9 is the highest configuration which translates to 196 threads I think?
21:49:54
aeth
Is PPC compatible with POWER8? I'm not sure if any CL runs on POWER8 if not. SBCL seems to have PPC and PPC64.
21:50:52
jasom
POWER9 is PPC64, but I think the talos is designed to run in little-endian mode, and I don't know if SBCL does that.
21:51:31
aeth
PPC64 is marked as in progress on the SBCL download page. I wouldn't be surprised if threading doesn't work
21:52:57
aeth
I wonder if there's a 2 socket AMD Threadripper mobo. That would be 2x 2x 32 at a max (right now... expect it to double next gen), so 128.
21:52:59
jasom
and the IBM E980 can support up to 16 12 core SMT8 processors for more cores than I care to do the math on
21:55:00
aeth
jasom: But yes, IBM's POWER is definitely worth mentioning in this kind of discussion. Last I checked, IBM seems to be keeping up with Intel, which is 10ish cores behind AMD.
21:56:39
aeth
A quick search has a reddit thread where the answer is "Threadripper is single socket only, for dual you'll need to go with Epyc." https://www.reddit.com/r/Amd/comments/6tqg88/dual_socket_threadripper_max_ram/dlmt8fn/
21:56:50
aeth
Of course, random reddit comments are even less reliable than random StackOverflow answers
21:57:48
aeth
(That's probably one of the ways they get people to pay 2x to 4x as much as the consumer line.)
22:17:08
Demosthenex
is there a way to start sbcl with swank using a port number from the command line? i'm trying to work on a few different things on a remote host, and i can connect with ssh/port forwarding. just starting sbcl with swank on separate ports is awkward
22:48:29
aeth
Wikipedia says POWER9 goes up to 24-core (96 threads via SMT4?) https://en.wikipedia.org/wiki/POWER9
22:53:12
aeth
It looks like both AMD and IBM are going to release 48-core CPUs very soon, though. It looks like Qualcomm already has a 48-core ARM server.
23:00:00
Bike
programs relying on particular details of the machine they're running on is something to be avoided. surely this is a basic principle
23:03:27
aeth
Bike: The architecture can also run on an 4 core machine, yes, but there's less of a motivation for it there.
23:05:29
aeth
Bike: But consider e.g. a game or similar application that can be divided into several systems. In today's world, if it's parallel at all, each major system would probably get its own thread. That'll only scale up to the number of major systems, which is probably far less than 90.
23:07:22
Bike
in that case whether you have five or ninety processes isn't important for basic architecture.
23:08:18
aeth
Except for the most part, afaik what you'd typically see when the end user has quad core is maybe putting one task on each a thread to be "parallel", not also subdividing those tasks.
23:10:40
Bike
if you already have an idea for how to structure a program for multiprocessing, what are you even asking?
23:13:32
aeth
Bike: I'm wondering if anyone has successfully deployed a CL application that takes advantage of one of these CPUs with a "very high" core count. ("very high" being relative to the normal 2-4 cores that you would have seen for this kind of hardware until recently)
23:14:23
Bike
the machine i use to build clasp has thirty something cores. during build we have each core do a compile-file so we can compile a bunch of files at once.
23:15:04
whartung
I mainge someone one has. At those core counts, you’re getting into “super computing” kinds of designs and applications.
23:15:53
whartung
Funny Bike, back in the day, when I had the CPU meter on my desktop, I’d just kick off builds until it reach 90%. … by hand. Whenever it dipped down, I’d kick off another one.
23:18:37
whartung
My friend laments how horrible Swift builds are on the Mac — it’s apparently very CPU intensive.
23:22:37
aeth
Bike: I would personally probably just manually implement the required algorithms and data structures when needed in my case rather than using a library. Micromanaging this sort of thing is kind of a key point of the engine (when I get there), not a secondary feature.
23:24:21
aeth
It's mainly due to the requirements for 100 FPS logic and no consing at the library level in the game loop (I have consed in all of my substantial tests, of course... too hard not to... but that's in the user's code)
23:27:44
aeth
no-defun-allowed: A 3D game engine. I switched my original design (nothing is stopping it from being general purpose, of course) from first person to third person strategy (i.e. far away camera) because of fewer physics requirements (otherwise the project would become more about writing a physics engine than a game engine)
23:31:06
aeth
I just need to do mouse selection of an entity and a destination and then I have arbitrary movement on a 2D plane within the 3D world (e.g. ships on water). After that (soundless) games would technically be possible, although still fairly difficult because the text rendering isn't complete yet.
23:33:59
aeth
I'm guessing the input and the final update of the authoritative game state would have to stay on a core thread.
23:36:38
whartung
as I understand it, it’s prety difficult to multitask the (a?) rendering pipeline.
23:37:14
whartung
you can easily have a rendering thread, a state thread, sevearl AI threads. But the busy thread is typically the rendering thread.
23:37:26
Bike
you don't really have the opportunity anyway since the graphics library wants to do rendering in mysterious ways
23:37:48
jasom
Bike: there are some tradeoffs you can make; e.g. make the serial performance much worse in trade for more parallelism; the number of cores you have can decide if you will make that tradeoff
23:38:10
Bike
mostly because rendering is so hard we've farmed it off to repurposed bitcoin-mining chips
23:38:39
whartung
well, they happen to be pretty good at the job. If they weren’t we wouldn’t be handing them off to it.
23:40:26
whartung
as I understand it, the games job is to get the models oriented properly, send them to the GPU, and make sure the proper textures are loaded at the same time. The GPU does the actual rendering stuff.
23:40:45
jasom
aeth: my most recent webapp will probably scale well past 8 cores; all the state is in the database and each worker thread has its own DB connection. It's when you get past 32 cores that things can get sticky.
23:44:37
whartung
but the gpu guys have a way of getting it to work on multiple GPUs (2 GPUs, alternatiing scan lines or something like that)
23:46:35
aeth
whartung: It's generally not worth it to have multiple GPUs. The game (or application) has to specifically support it (and probably support it differently for nvidia and AMD) and it's usually better just to get the next tier up in graphics cards so it probably only applies to you if you're using Titans.
23:48:32
aeth
For nvidia it's called SLI and the general advice on forums has been to not do it, at least for the past few years.
23:49:25
White_Flame
whartung: different generations had different limitations. I know there were 3x linked card setups,a nd probably 4 as well
23:49:56
White_Flame
I think some also just send along the PCIe bus instead of having their own inter-card connectors
23:50:06
aeth
See https://en.wikipedia.org/wiki/Scalable_Link_Interface and https://en.wikipedia.org/wiki/AMD_CrossFireX
23:50:26
aeth
but apparently nvidia just abandoned SLI on the next generation (that isn't out yet) in favor of "NVLink Bridge"
23:52:52
whartung
I’ve spent enough treasure on computer equipment, I’m glad I was never really in the GPU chase :)
23:53:15
White_Flame
nvidia is going off the deep end with prices anyway, best to ignore it for now, too
23:53:28
aeth
But when I was saying that rendering is heavy, I mean heavy on the CPU. The GPU is doing most of the work, but #'draw still seems to be the most CPU intensive function at the moment for me. I think Vulkan would allow this to be split up, though.
23:53:59
White_Flame
no-defun-allowed: most lisps are fully reflective, if you use their implementation internals
23:54:21
aeth
White_Flame: You see that all over the place. CPUs, GPUs, smartphones, etc. The high end is improving, but also getting more expensive.
23:54:53
White_Flame
aeth: RAM prices are settling down, though, finally returning to where they were 3 years ago or wherever it was before that spike
23:54:59
whartung
depends on how high powerful the polisher you’re using no-defun-allowed, and what kind of compound you’re using
23:56:46
whartung
no, I understand aeth. Just getting the frames set up. A lot of that is the modeling side (modeling the car(s) for example), and things like the physics engine. Is it worth while to farm our modeling the suspension dymanics of 40 cars in a nascar sim to different CPUs per frame? I dunno. it may not even be possible if you’re modeling drafting and what not.
23:58:42
aeth
whartung: but on the other hand I think a large part of strategy games is embarassingly parallel, or close enough.
23:59:09
whartung
yea it can be, quesiton is whether it’s cpu worthy enough to bother breaking it out.
23:59:56
aeth
I've definitely played some building strategy games (games with workers doing tasks and carrying resources around a village) where it uses all the CPU
0:00:05
whartung
The only time I upgraded a machine for a better game performance (among other things) was way back when I went from 133 to 400 for Total Annihilation. So it coul dhandle more units. But even that would be difficult to paralleize because of the dependencies on each other (notably in traffic jams)
0:00:56
aeth
Even without htop up, I can tell when my desktop's fully using the CPU in a multithreaded way because I can hear the CPU fan when that happens.
0:01:44
whartung
my friends laptop turned in to an air hockey table when compiling Swift. You’d think it would float off the desk...
0:02:28
aeth
In general, simulation(ish) strategy games seem to max out the CPU and recent AAA games tend to max out the GPU
0:02:55
aeth
(I don't think both would be possible, unless you wrote specifically to the hardware combination and optimized heavily.)
0:05:17
jasom
aeth: do you sometimes render inconsistent state when you do that (i.e. some objects 10ms behind others)?
0:06:13
aeth
Oh and 100 fps for logic because its inverse is 10 ms and what you want is (integerp (* 1000 (/ n))) so both measurements are integers imo.
0:07:28
whartung
unrelated, are they any good examples of network servers in CL floating around on the net? I’ve seen the http server (Hunchenroot? Is that right?) Just curious if there are others of that ilk I could look at.
0:08:08
aeth
In case anyone's wondering about the other numbers... (loop for i from 1 to 2000 when (integerp (* 1000 (/ i))) collect i) => (1 2 4 5 8 10 20 25 40 50 100 125 200 250 500 1000)
0:10:24
aeth
Looks like it's 1 and numbers whose only prime factors are 5 and 2. (I ran my factoring function on that list to verify)
0:14:05
aeth
(limit in the sense that you can't get better than 1000, not in the calculus sense of the word "limit")
0:20:05
aeth
Bike: I didn't expect the fundamental theorem of arithmetic but I guess I should have expected it as soon as INTEGERP entered
0:20:56
aeth
Here I was working with periods and frequencies in a real world problem and then suddenly PRIMES
0:24:14
Bike
people are all like one plus one is two, well in my opinion it makes more sense to define two as the octave
0:27:07
aeth
Bike: But this is sort of the same problem. With the logic running at 100 ticks a second, some parts might actually only need to run at 50 or 25 or even 1. And this can be determined with a simple MOD
2:04:10
stacksmith
Is there a foreign pointer validation function in CFFI? To check if dereferencing would crash...
2:07:43
stacksmith
I was hoping for a less-than-perfect validation - like check if it's in-range for any areas allocated with CFFI. But yeah, that was wishful thinking.
4:01:33
no-defun-allowed
no, this is the right channel, stacksmith was asking about bounds checking for cffi arrays
4:02:54
beach
I should be quiet. Whenever I complain about the fact that half of the contents here is about other languages, I am told that I am wrong to complain.
4:12:40
no-defun-allowed
are there any good implementations of unordered sets that can be looked up in less than O(n) time?
4:13:50
beach
no-defun-allowed: All discussions of sets in the CS literature are about special subclasses of sets.
4:14:20
beach
no-defun-allowed: And that's because there is nothing better than O(n) in the general case.
4:16:56
beach
If your keys are simple and you have a lot of elements, then they are typically better than lists.
4:51:33
White_Flame
no-defun-allowed: a rule of thumb from other languages is that around 10 elements is where hash tables tend to get faster than iterating a list. I would suspect CL's tradeoff (again, for simple keys that dont' take forever to hash) would be in the same ballpark
5:14:51
aeth
even if a plist is slightly faster at 3-5 elements, I didn't notice it last time I benchmarked it because both are just so fast.
5:19:55
aeth
I'd personally say hash-table vs. plist is whatever semantically seems to make more sense for a few elements. Above a few dozen, use a hash-table unless you're only sequentially iterating over it with something like alexandria:doplist or you're in a macro (where you're probably sequentially iterating over it anyway).
5:51:03
cobax
Shinmera: any names you can throw at me would be appreciated, so I can read up some more on the alternatives
5:52:25
White_Flame
if by "orthogonal" you mean transparent, that model tends to be underrepresented because it's quite difficult and usually doesn't work well
5:53:39
cobax
That is what I mean, yes. Difficult to whom, is the question. Postgres is quite easy for its programmers but hard for whomever needs to install and maintain and deploy it. On the other hand...
5:53:43
White_Flame
if the object model is integrated with the OS itself in a fully persistent address space like some mainframe & experimental OSes, then it would work better. But application-level, not really
5:54:41
cobax
White_Flame: I'm not sure all that is needed. Ruby doesn't have that and there is Maglev for it.
5:55:05
aeth
Lispers seem to prefer just representing SQL as s-expressions (nothing is stopping someone from writing an SQL database in CL, though)
5:56:37
White_Flame
cobax: well, if you just want persistence without performance, then it wouldn't be hard to pull off
5:57:51
cobax
So now you know I'll be abusing whatever persistence library as a replacement for an RDBMS (I don't care about performance and joins) then what are the current persistence libraries?
6:00:50
cobax
White_Flame: the angle I'm coming from is, Lisp is image-based, but how can I take easy/comfortable advantage of this fact, to make small apps without a database, by persisting my environment, like Smalltalk does?
6:02:08
White_Flame
so certainly on exit of your application, you can have it save the image & exit, and your startup script would resume from that image
6:03:03
White_Flame
but if you crash, the saved image's heap wasn't constantly updated with your running heap or anything
6:03:28
cobax
Ok, so tell me where I'm wrong here, then an easy way to make a prototype website with db functionality but without postgres, instead relying on CL's image-based-ness, then I can deploy a CL app and take a snapshot every 24 hours for data backup?
6:04:21
White_Flame
if you want to use CL images, then that's taking manual snapshots of the entire CL runtime
6:04:24
cobax
I don't know what you mean by "DB-like functionality". an RDBMS has several aspects to it. For instance, I don't need the fact that it's relational.
6:06:31
White_Flame
SBCL, for instance, has SAVE-LISP-AND-DIE as its only image writing function (as far as I'm aware). I don't know how well other systems deal with saving an image but not exiting
6:06:53
White_Flame
because bundling that all up for a resumable disk state sometimes involves closing stuff out
6:08:03
White_Flame
most deployed Lisp images don't constantly re-save themselves; they're like standard deployed applications where the image on disk starts up adn then reads its application data from other files/DBs
6:08:37
White_Flame
but you can certainly try an image-based application persistence model. I don't think there's anything preventing it; it's just uncommon
6:09:14
White_Flame
especially if you're fine with it restarting every 24 hours for snapshotting & flushing purposes
6:10:39
cobax
White_Flame: I have many books on Lisp and its implementations and it seems as if the "image-based" aspect is less explored than in smalltalk, would you agree?
6:10:59
cobax
for instance, even Picolisp seems to be ahead of Common-Lisp in terms of a rich image-based-ness?
6:11:08
White_Flame
it was more useful in ye olde days where reinitializing everything would take way too long
6:11:36
White_Flame
nowadays, you can cycle in a fresh Lisp image, load & initialize all your code in seconds
6:12:26
cobax
White_Flame: I understand, but you see, then an advantage is lost, namely the ability to exploit the image-based-ness for purposes of long term storage
6:13:00
White_Flame
cobax: most people store long term data as data, not all wound up as artifacts inside a runnable Lisp heap