libera/#commonlisp - IRC Chatlog
Search
12:18:11
lisp123
contrapunctus: re code browser, why worry about directories and files? better do based on packages and how functions depend on other functions
12:19:55
lisp123
an object browser for CLOS objects would be good - believe its already implemented in various forms, but worthwhile adding to any 'code browser'
12:20:53
lisp123
it would be good to read a function and automatically browser to the applicable method
12:21:45
lisp123
then it would be good to have a documentation browser and potentially some type inference
12:53:11
contrapunctus
lisp123: thanks for the input. Have a look at https://codeberg.org/contrapunctus/dryad#explanation to understand the reasons for it.
12:53:22
contrapunctus
«better do based on packages and how functions depend on other functions» I'm working on a view which does something similar, but using system definitions rather than packages.
12:54:27
contrapunctus
«it would be good to have a documentation browser» One part of this will be done when I implement source comment display. Another aspect is customizable views of source code, e.g. displaying any combination of code, [and wherever applicable] definition name, definition type, docstring, arglists, etc...the parsing for this is in place, but needs a UI.
13:32:03
jeosol
I am using SBCL primarily and currently investigating two different representation options for some data. Is there a function to find out (in the repl) the size of the object created?
13:34:07
beach
I don't think so, but you can make an educated guess. Are you sure your application requires you to optimize for space?
13:34:48
jeosol
To give some context, I am trying to refactor some code that does binary genetic algorithm and I have to store the genes as array of the bits #(1 0 1 0 ...). I am thinking of saving it in the objects as str "1010..." then blow it up when I need to use it for GA operations (crossover and mutation), but it's probably too much work as I'll need those
13:35:01
_death
one simple approximation is to traverse the object, summing the sizes of sub-objects.. for the size of "primitive" objects you can use sb-ext:primitive-object-size
13:35:43
jeosol
beach: Good point. I am just thinking about it, I don't think so. It's seem it will just be too much work. I am trying to see if I can store things compactly and see how things the time and space tradeoffs
13:36:38
jeosol
beach: Thanks for that. That resolves my problem. Using array of bits is my current implementation but had someone suggest if I have looked into compressing them.
13:36:39
beach
jeosol: The standard requires the existence of bit vectors, and I am pretty sure SBCL stores a single bit for each element in that case.
13:40:16
jeosol
The arrays are usually sparse. It will contain lots of 0's and 1's. But usually, I am not looking at massive datasets. I do use a some compression elsewhere but for storing things in a file where I store things as 4*1 4*0 5*1 with the first number indicating the count of the second number. It's not related to my bits question but I do process some
13:40:17
jeosol
large arrays in a 3d-grid that are mostly just 1's (have property there) or 0's (property is not there)
13:40:33
hayley
It is worth noting that SBCL can do SIMD-within-a-register for sequence operations on bit vectors, which may be faster than writing out the loop by hand.
13:43:18
hayley
Some time ago I had mentioned that I got marked down substantially for using Lisp/ML-ish pseudocode on an assignment; but I got full marks for writing an algorithm which used such bit vector tricks, so it evened out. I'd highly recommend trying to use such operations.
13:43:25
jeosol
beach: to your question of optimizing for space. I normally sacrifice space for convenience of implementation and clear code - so I often may create more variables especially if I need to track them, than perform computations to get the variables when needed.
13:44:24
jeosol
beach: I think the boosting computation speed is a better use of my time. So I search for bottle necks and see where I can replace some code with better algorithms and/or use appropriate data structures
13:46:05
hayley
Oh, and speaking of SIMD, I got a new processor recently, and all the vectorised scanning stuff in one-more-re-nightmare is much faster than I had anticipated; as the new (Zen 3) processor has a 256-bit ALU and the older processor has a 128-bit ALU. Scanning at 16GB/s with submatches is quite fun.
13:46:22
jeosol
I will try to create a version of the code so I can track the speedup as I make the improvements
13:47:11
jeosol
hayley: wow, that's very nice. I might self will look to get a machine later on, but I have to do some research, since I do some machine learning, thinking of something with GPU
13:49:17
hayley
I see. Last time I checked, the GPU market was in bad shape, but the Ethereum price went down recently, so hopefully fewer miners will inflate demand these days.
13:53:44
jeosol
hayley: good point. I have seen several posts on reddit that said the GPU prices have crashed due to crash in BTC and related coin prices
13:54:27
hayley
Long ago I had decided to only replace computer parts after they've been used for 5 years. The result is that I can't waste too much money at once, and hopefully I would partake in producing less e-waste too. But I decided to spend more money than usual on a CPU, as the cost is amortised over 5 years. That information might or might not help with picking a machine.
14:16:28
hayley
On topic, I do a fair amount of model checking and fuzzing of my code, to help find bugs; there usually are a few bugs before I run a fuzzer for the first time. That sort of workload is embarrassingly parallel, so I decided throwing money at a nice CPU would improve my productivity enough to justify the cost.
14:19:27
jeosol
Doing something similar now - my code mostly runs but there are some code paths that will result in problems every now are then. I am running stochastic optimizations so something some situations may arise and I have not accounted for nor written protective code around
14:20:45
hayley
Pratchett once said that "real stupidity beats artificial intelligence every time", but often artificial stupidity beats my intelligence ):
14:22:24
jeosol
beach: On the timing and performance issue, is there some resource (or may be there needs to be one) that emphasizes CL style for better performant code. Perhaps some of the Lisp guides and/or some repos have this. Like for python, because of multiple ways of doing things, some looping constructs run slower and others that use C underneat or
14:24:32
beach
I am unaware of any such resource. Also, I imagine that for Python, it's for a single implementation, whereas for Common Lisp, the recommendations might vary a lot between implementations.
14:26:25
jeosol
my bad, I am sorry, but butchering the name. I get confused, not sure if I have seen the other spelling somewhere or my mind is playing tricks on me
14:26:50
hayley
I always thought the rule for Python was to just use a C module, or to hope PyPy fares better.
14:27:54
jeosol
beach: good point beach. I was even thing of say things like organize let forms, loop macro constructs that are performance, albeit slightly, etc. But I do agree that will definitely vary my implementation.
14:28:32
hayley
The instructor didn't like my sense of humour for some submitted work; we had to produce an algorithm that ran quickly, which was to be written in Python, so I made a C module which did the computation. (Better yet, we were allowed external libraries, and those often use C as well.)
14:28:35
jeosol
hayley: I don't claim to be a python expert, and don't know the language fully, just use it to get some things (in the ML/AI space) done
14:29:25
jeosol
hayley: yes, for serious computation, it was recommended to use a C module or libs with C modules behind
14:31:02
hayley
Later he had admitted that my submission was fast even for a C program; perhaps I could have done it in Python properly. But I searched how to perform a count-trailing-zeroes operation in Python, and was told to make a string and search for a '0' character in the string, which scared me.
14:36:38
hayley
As for an optimisation guide for Common Lisp, I think one might be tempted to write too much about optimising for SBCL. (Though it might be good advice to recommend you try hacking on the compiler of your favourite implementation, too.]
14:37:25
hayley
Oh, it was searching for the '1' in the binary representation, which isn't much better. Ideally this operation would just use the BSF instruction on x86, I think.
14:38:16
hayley
(Arguably I didn't write my code in C, as I used a GCC intrinsic function to perform count trailing/leading zeroes though.)
14:41:53
hayley
Common Lisp is quite good for bit munging, in comparison. LDB is already amazing, INTEGER-LENGTH is find-last-set(?) and LOGCOUNT is Hamming weight.
15:21:45
jeosol
TMA: that's where I have seen that then. I was confused when I saw it and then saw his name later on. Good to know
16:27:18
phantomics
Hey, quick question, are there any libraries for CL that optimize integer division, i.e. (floor), like libdivide for C/C++?
16:33:08
pjb
(time (floor 903128712708418 12312)) #| --> 73353534170 ; 7378 |# took 4 microseconds (0.000004 seconds) to run.
16:33:48
phantomics
The naive implementation of floor can be improved on a bit, libdivide uses some strategic bit shifts and other techniques to make it faster
16:34:29
phantomics
And April's row-major-index based array transformations do a lot of (floor)ing to convert indices
16:35:06
pjb
Yes. Again, you can report a bug asking vendors to use libdivide or better in their cl:floor implementation.
16:38:05
jeosol
I recall seeing that PLN is not in SBCL a while back. What is the current status? I tried in many moons ago and got some errors.
16:38:35
phantomics
Or just FFI it directly in April, but that would be inelegant, although it appears that some recent processors have much faster int division, like M1 and Ryzen
16:40:02
jackdaniel
afair commercial implementations either picked changes up, or will do that, but that would need a confirmation
16:40:19
jackdaniel
I think that Shinmera had some fancy page that summarizes support for various extensions
16:40:21
jeosol
jackdaniel: thanks, I only use SBCL, I tried it a while ago and remember seeing something that asdf for SBCL didn't update something (i don't remember exactly)
16:52:16
jeosol
jackdaniel: I think I recall my error. I was trying to use PLN with uiop:define-package, that's why I wasn't able to get it to work, not plain defpackage
17:36:33
jeosol
Shinmera: thanks for that. It seems there is some issue using it with uiop:define-package
17:37:09
jeosol
jackdaniel: did you confirm that it gives an error too, or that I shouldn't be using it that way
17:37:55
jeosol
Shinmera: interesting. So it means I'd have to update manually because I am using latest SBCL
17:38:24
jeosol
I am not using PLN now per se, but trying to make things more maintainable, so looking at options
17:39:18
Shinmera
I've been using PLNs in all my libraries published since 2019 and it's been great.
17:40:12
jeosol
you point about dependence about is to use plain defpackage instead of uiop:define-package
17:54:42
mfiano
Shinmera: Well PLNs didn't really originate in SBCL, but it was the first implementation of the external specification
17:55:29
jackdaniel
the specification was written by nsivola for sbcl with intent to propose it as CDR after some user feedback
18:00:38
mfiano
It's pretty amazing how that feature caught like wildfire in the world of opinionated Lispers.
18:32:07
Bike
phantomics: sbcl optimizes division by small constants into shifts+etc already https://github.com/sbcl/sbcl/blob/master/src/compiler/srctran.lisp#L3093-L3119. if you want to do something like libdivide you could call compile? like use (compile nil `(lambda (x) (truncate x ,d))) where libdivide has libdivide::divider<int>;
18:32:29
Bike
if the function call overhead is too much, you could probably leverage sbcl internals to produce appropriate forms yourself and rely on the modular arithmetic optimizations
19:13:31
phantomics
By small constants does it have to be expressed as a constant in the code, like (floor x 3)? Will it work with for example (floor x (aref v y)), where v is a vector of positive integers?