libera/#sbcl - IRC Chatlog
Search
21:38:43
stassats
(f (if x constant y) constant) seems to happen fairly often in the wild, i guess it's worth folding that
1:26:48
jeosol
For context, I am trying to get Google's bazelisp lisp to work: here is the link to the call in the file : https://github.com/jeosol/bazelisp/blob/219fe8c01f222e95d322d89b3518399622c81708/main.lisp#L759
1:40:45
jeosol
I do see the "compile-file" function in sbcl-2.1.7/src/compiler/main.lisp but not "compile-files"
1:49:02
jeosol
I also just check 2.0.11 which is the version they appeared to have used and it's not there either: https://sourceforge.net/p/sbcl/sbcl/ci/sbcl-2.0.11/tree/src/compiler/main.lisp#l1731
1:50:08
jeosol
I am trying to move things forward to at least 2.1.7 which is my current version on my machine
1:51:50
jeosol
as new versions come out, so I can spend that period to quickly iron out any issues - usually, I get none.
1:54:47
jeosol
stassats: are you aware of any tool or effort for parallel compilation, i.e., to speed things up
1:56:40
jeosol
I need to load a large project. Tried docker, computations are slow when run inside a container, 3 seconds vs 22 seconds (sbcl container)
1:57:31
jeosol
I was not saying my application is larger than sbcl or anything, just that I am trying to load things faster
1:57:56
jeosol
just to clear that up. I can't compare to the work you guys do and what I am working on can't happen without you guys
1:58:53
jeosol
I was thinking testing from the cloud hence me trying docker. But to slow even with multiple sequential builds
1:59:44
etimmons
jeosol: Does your build involve creating many processes from SBCL (using run-program)?
1:59:49
jeosol
Not in development, I just use CC, and I don't load the whole thing, I test different parts. I am only recently looking at any CI/CD type workflow so I can run things faster
2:00:46
jeosol
I recently went back to bazel, hacked for the last 4-5 hours and exhausted. Almost got it to work but I kept asking myself why I would rely on a poorly documented tool
2:01:19
jeosol
My aim is to do some kind of evaluation of the available tools CLMP, QL, POIU (if I get it to work) and see metrics
2:02:33
etimmons
Weird. That's the only time I've personally seen SBCL be that much slower inside a container.
2:03:49
jeosol
The machine is the same one I use with several browser tabs and other calculations running, so not sure how that impacts. But I also run the native (shell) option that way
2:04:42
jeosol
The set up involves request from django to hunchentoot (both native) and sbcl application (in docker)
2:05:41
jeosol
the runtime inside the container is not back, but the requests take long to return back.
2:07:26
jeosol
etimmons: I'd have some feedback for you in a few days. I think bazelisp evaluation is done
2:12:27
etimmons
I'm confused how this django/hunchentoot setup interacts with Bazel/POIU, so the best I can do is wish you luck! And ping me if it does turn out you're spawning a bunch of processes. There's a relatively easy workaround for that particular slowdown.
2:23:40
jeosol
etimmons: I just thought about your question again. Yes, my program when run in full mode does a lot of run-program calls. But what I was testing is a single run-program call
2:25:05
jeosol
Forget about the django part - sorry for confusion - that is just the web facing server. All other code is CL/SBCL
2:26:08
jeosol
I was trying to test bazel or poiu to build faster. It is not related to the test. I am trying to get into CI/CD - not sure which approach to go with and still investigating. Code is not hosted publicly
2:27:16
jeosol
I have given too much time to bazel/bazelisp, even if I get it to work, I still don't adequately understand the tool for my comfort. And the google guys say their aim is to get to work for their needs, and not care about much else
2:31:13
etimmons
Try adding `--ulimit nofile=1024:2048` to your `docker run` command. Here's the effect on my machine: https://plaster.tymoon.eu/view/2652#2652
2:32:57
etimmons
Docker's default settings set an insanely high limit on how many file descriptors a single process can have open (1048576 on my install)
2:33:33
etimmons
And SBCL iterates over ~every one of those to try and close it when you run-program
2:34:45
jeosol
etimmons: seriously, thanks for that. Let me do quick, almost finished rebuilding the image
2:34:48
etimmons
Lowering it to a more sane number speeds that up significantly since every call to close() needs to go through the kernel.
2:37:21
etimmons
Maybe one of these days I'll find the time to make a patch that uses close_range under the hood now that Linux has it. I just wish the libcs would add a wrapper for it.
2:45:07
etimmons
Yeah. It was under daewok, but I put them in a place the CLF has access to in order to up the bus factor (and hopefully legitimacy).
2:47:14
etimmons
The Quicklisp installer is included in the image. You can install it with the `install-quicklisp` script. But I just recommend mounting your own quicklisp folder so that it's persisted across containers
2:51:09
jeosol
I'll look it into it. Do you run some automated workflow for the updates when a new version comes out
2:53:42
etimmons
when the new version is released I run a script locally and push the results to the git repo. Then Gitlab CI picks it up and actually builds the images.
2:55:28
jeosol
I at the ulimt option to docker, saves 8 seconds: ~ 22 seconds to average of 14 seconds
2:58:38
etimmons
I'm poking at some Gitlab CI scripts that can be included into CL repos and try to automatically do something sane to run tests and the like. But that's probably not a conversation for #sbcl
2:59:54
etimmons
Well, that's progress at least. Only thing I can guess at for the remaining 11 seconds is poor filesystem performance. But that's not an SBCL specific thing.
3:01:31
jeosol
I will look into the docker settings more. It doesn't make sense. Native runs for 3 seconds, and 11 seconds is way high - but I will check it later