libera/#commonlisp - IRC Chatlog
Search
6:42:51
Nilby
dbotton: Here's my highly unsatisfactory code for geting loadable systems https://plaster.tymoon.eu/view/3335#3335
6:49:46
Shinmera
dbotton: you can steal this https://github.com/Shirakumo/dist/blob/master/asdf.lisp
6:50:37
Shinmera
note that while what I do covers almost every case, in general what you're asking for is impossible. you need to load the ASDs (and thus potential systems) to determine what systems there are.
7:14:03
Nilby
Shinmera: A good demonstration that CL easy readability is convenient fiction. I feel like this could have been fixed a long time ago with few tweaks like a standard walker and *read-intern*, but how to fix stuff like eclector.reader:find-character => (code-char #xFFFD) ? Maybe more reader error conditions?
7:15:11
Shinmera
ASD files can be any arbitrary code, and system definitions could look like literally anything. The only way to get stuff out is to evaluate it.
7:16:49
beach
Nilby: Your code has several occurrences of incorrect indentation, and there are several violations of the rules stated on page 13 of the LUV slides.
7:18:09
Nilby
Shinmera: Yes. I completely agree, but even if everyone omitted any evaled code in ASD's it's trouble.
7:18:33
beach
Nilby: There are also some questionable practices in terms of software engineering, i.e., a function silently doing nothing when the input is not what it expects.
7:19:21
hayley
Now I wonder if there are enough hooks in ASDF, that just gleaning at the DEFSYSTEM form would give wrong results, even if the form fits the "shape" one expects.
7:24:50
Nilby
hayley: as you may know, ASDF was eventually made so you can do nearly everything without evaling, and it turns out it's possible to load much of quicklisp dist without eval, but still just the reading probalems that Shinmera has to work around. e.g. undefined package prefixes, unknown character names. I guess nobody puts structure literals in ASDs.
7:38:35
Nilby
but there's still things like :version (:read-file-form "version.sexp") or even the popular :long-description #.(uiop:read-file-string #P"README")
8:24:18
Nilby
beach: Thanks, but you can probably save your critiquing effort, since I have always been incorrigible.
9:41:56
Seok__
I found cl-intbytes useful for processing integer bytes, is there a similar library for single and double floats?
9:42:59
jackdaniel
Shinmera's library has a casting function that is a portability layer for the implementation-specific casts extension (which are cheap)
11:48:16
dbotton
I've decided to scale my project idea to still use .asd but if systems not setup up "right" as a project you will have to edit it by hand, but can still with in reason click to open files.
13:36:53
jeosol
just came here to thank all the sbcl devs, upgraded from 2.2.6 to 2.2.7 without any issues
13:38:15
jeosol
I work with Python (no language wars) but I very much appreciate this about working with CL/SBCL, most upgrades give no issues except small ones (like compression lib change with 2.2.6*)
14:18:58
rendar
i guess that getting '(' while reading a symbol name is an error, all must be space separated, right?
14:20:26
NotThatRPG
ASDF is ready for a 3.3.6 bugfix release, with a bunch of fixes. Per semantic versioning, this should be entirely backwards compatible. I encourage people to download a copy of 3.3.5.10 and try it out with their systems. Bugs can be reported at https://gitlab.common-lisp.net/asdf/asdf/-/issues Thanks!
14:26:44
phantomics
Morning everyone, does anyone know the fastest way to keep a count of active workers in an lparallel kernel? I want to make the count frequently so I can efficiently assign tasks to inactive workers and frequently calling (task-categories-running) seems like it will have substantial overhead
14:26:47
White_Flame
rendar: and the main term involved here is that #\( is a terminating macro character, eg it will terminate any token being built and start a new one
14:27:28
NotThatRPG
Get your pre-release ASDF here: https://gitlab.common-lisp.net/asdf/asdf/-/wikis/Pre-release-ASDF
14:29:36
jackdaniel
last time we had libraries versions in the topic the length was quite big, and there was no clear criteria what should be listed there
14:38:33
jeosol
does wrapping a call in handler-case result in performance hit? I am assuming it's neglible. Context: I am trying to make a task run as fast as possible
14:39:53
jackdaniel
(i.e when there is no non-local jump, then there shouldn't be any performance hit, otherwise - I guess that it depends)
14:40:58
scymtym
many implementations cons a little bit and bind (as well as later undoing the binding) a dynamic variable
14:41:37
beach
jackdaniel: The handler code, yes, but the dynamic environment must be modified somehow.
14:42:30
jackdaniel
yes, but binding a special variable is not overly expensive (unless someone is really desperate, but at that point perhaps common lisp is not the right choice)
14:44:34
White_Flame
here's the disassembly for a single hander-case around a single function call: https://plaster.tymoon.eu/view/3337#3337
14:44:36
scymtym
consing a handler binding, manipulating a dynamic variable (including undoing the binding for any kind of exit) can far exceed the execution cost of some operations
14:47:07
jeosol
right now, I can get my calls, I do a 1000 such calls, to run around 1-1.5 seconds. sbcl in docker takes 3.6 seconds
14:47:08
Bike
i mean, i said it before you even mentioned safety 1, so obviously i wasn't referring to you
14:47:54
Bike
if you're on sbcl, it has pretty good profiling tools for figuring out what particular is taking up time
14:48:06
aeth
If you have (iirc) CASE or some arithmetic, (speed 3) is basically free speed. The best kind of optimization. Might hurt compilation times or inspecting with the debugger or something, otherwise idk why it's even configurable instead of the default
14:48:53
jeosol
Bike: Good point. I will look into that. There is an external call to a Fortran exe (can't optimize this at all). Beside that call, I want everything on the SBCL side to be tight as possible
14:49:13
jackdaniel
aeth: you've said it - might hurt inspecting; examining what's wrong is quite important
14:49:43
Bike
cos like if you want to catch errors you need to have it, but if you're doing something more specific there may be alternate solutions
14:50:07
jeosol
Bike: sometimes I may get input that makes a call go wrong. I have hunchentoot in front, so I just want to fail silently
14:50:53
Bike
and your failure is in the form of an error you don't control? because in that case you're not going to be able to lose the handler-case
14:51:09
White_Flame
jeosol: so there's no loop involved, just the performance of a single chain of events?
14:51:50
White_Flame
Bike: prechecking might speed it up, because then type inference knows what's up
14:51:54
Bike
jeosol: the sbcl manual explains exactly what it does with types in 4.2 "Handling of Types", including defclass slots
14:51:59
jeosol
White_Flame: I am doing a series of calls to hunchentoot server that runs SBCL task. The ratio of server request time and sbcl task is 0.99 so far
14:53:12
jeosol
Bike: thank you. I try to specify the types at least for documentation. With recent updates, I recall SBCL complained when I have a type as float but left :initform nil (from old code)
14:54:49
NotThatRPG_away
Thanks jackdaniel -- it would be good to get feedback before the release. Especially from users of implementations I find it hard to test locally.
14:55:14
scymtym
jeosol: yes, the :INITFORM vs. :TYPE check is a relatively recent improvement. people are talking about (defclass foo () ((%bar :type integer :writer (setf bar)))) (setf (bar (make-instance 'foo)) #\c) signaling a TYPE-ERROR at runtime which only happens with elevated safety
14:59:57
Bike
on sbcl, one of the nicer things about speed 3 is that it will make the compiler give you an overwhelming amount of advice on how to make your code faster
15:02:53
jeosol
so what's a generally strategy in using speed and safety. You go up to safety 3 to ensure everything works ok, and then do safety 0 and speed 3?
15:04:00
jackdaniel
0 is for local code that you are absolutely certain that it is correct under any circumstances
15:04:45
pjb
If you don't care what the result will be, then why not use safety 0 or 1. But then also, why run the program at all???
15:05:20
jeosol
just looked at the sbcl manual. there is a comment about corrupting the heap with "no type checks" when (= safety 0)
15:06:19
aeth
maaaaaaybe there's a bounds check that you can eliminate inside of a LOCALLY where you're absolutely sure it's impossible to go out of bounds with
15:06:23
jeosol
I have never configured these before and just used default setting. Now everything runs well, I want to optimize a bit and remove unnecessary code paths
15:06:50
White_Flame
yeah, hit speed 3 and (try to) follow SBCL's notes as far as you care to take it
15:08:31
jeosol
aeth: no optimization context per se. This is following from a previous convo with you on SBCL in bare metal repl and SBCL in docker. The former is my benchmark and the latter something I am improving
15:09:08
jeosol
so far brought the docker version from 14 secons to 3.7 seconds but these are changes on the docker side of things, not SBCL related. The bare metal repl is 0.8-1.5 seconds
15:13:53
jeosol
For task B, I read in CLOS instanced serialized to disk (it's the exact same each time), and then I changed some variables based on the call, then do call another function that takes that CLOS instance
15:18:34
jeosol
For the first call, it will read the object. The issue is that this test case is smaller compared to some other cases. The slots of the instance hold 3-dimensional arrays. My small test case is 100x1x1).
15:21:19
jeosol
I am also dealing with memory issues if I hold those arrays in memory which is what I do currently.
15:21:50
jeosol
Thanks everyone for chiming it, I appreciate the help. I have the cases running now and will share some of the benchmarks soon.
15:28:33
jeosol
for the performance and tuning hints to sbcl, I need to run (declaim (optimize (safety 1) (speed 3))) at toplevel?
15:31:55
jeosol
I just want to know so I save the first results, then run with higher speed and seel what performance boost I get.
15:32:26
aeth
speed changes the verbosity, though. Lots of notes about how you could make things faster, by changing what you're doing entirely, even if you... can't.
15:33:08
Bike
i think the messages i was talking about come up on speed 3 regardless of the other settings.
15:33:10
aeth
I have to do this in a few places: (locally (declare #+sbcl (sb-ext:muffle-conditions sb-ext:compiler-note)) ...)
15:33:47
Bike
and, like aeth alluded to, these will basically all be microoptimization hints. sbcl isn't smart enough to tell you a more efficient organization of your code
15:33:50
jeosol
Bike: the hints have suggests to improve code -- haha, hopefully it's not to change things radically
15:34:57
jeosol
Bike: that was my point. I have used suggestions from here over time, so organization is ok, I guess.
15:35:43
jeosol
As long SBCL duration is only marginally higher than the external exe, everything is fine
15:46:36
NotThatRPG
I've always wondered if there's some way to selectively mute those notes, so I can go through a big swodge of them, fix the ones I can fix, and muffle the ones I can't or don't want to
15:48:32
Bike
you can use muffle-conditions like aeth said, but i think getting more specific than "all notes" is not generally possible