freenode/#lisp - IRC Chatlog
Search
20:10:57
pfdietz
cl-json turns names into keywords, jsown leaves them as strings, but even converting them to keywords manually leaves jsown much faster.
20:23:54
puchacz
hi, to pass values of dynamic variables (these defparamter / let, I think they are called special variables) to a new thread, I first bind them to local variables in the parent thread like (let ((x1 *x1*)) etc. and then rebind to global symbols again from local variables in the child thread: (let ((*x1* x)) etc. it seems to work.
20:24:18
puchacz
can somebody confirm that it is correct please? and is it a portable Common Lisp way or SBCL specific?
20:26:34
puchacz
right, so all I need to do is to bind a list variable names to *default-special-bindings* and I am done?
20:29:15
phoe
it'll bind all the dynavars via PROGV and execute your thread code in its dynamic scope
20:32:58
puchacz
and if I just want the same name in car and in cdr of alist elements, it begs for a macro :)
20:37:40
puchacz
is there anything in the standard that (let ...) variables are visible in the child thread or it just a convention that happens to be followed in all major Lisps?
20:42:21
puchacz
jackdaniel - pass a closure from the parent thread to the child thread in a (let ...) variable and it will work, no matter what variables the closure refers to?
20:43:20
jackdaniel
puchacz: yes. i.e (let ((x 42)) (bt:make-thread (lambda () (let ((*something* x)) (call-my-body)))))
20:43:53
phoe
exactly what jackdaniel posted, if for some reason you don't want to use bt:*default-special-bindings*
20:44:36
jackdaniel
phoe: technically speaking they are not. special bindings may tap into implementation mechanism without creating a closure
20:45:36
phoe
jackdaniel: tapping into implementation details - how exactly do they pass values into thread functions then?
20:45:44
puchacz
for a while I thought you were saying I could (let ((x (lambda() (do-something)))) (bt:make-thread (lambda () (funcall x))) - pass a closure like this
20:48:45
jackdaniel
closures may be more expensive than ordinary functions and given a particular implementation of a dynamic environment it may be practical (performance wise)
20:51:50
puchacz
on another note, I rented a (virtual) 32 core server, exported image on my ubuntu there, with an intention to run some numerical computations (driven from my PC, invoked via simple http calls to the remote computer), and the whole thing managed to crash with messages like "CORRUPTION WARNING in SBCL pid 27983(tid 0x7f8ea266f700):" 2 or 3 times
20:52:43
jackdaniel
do you by chance run jsown or other library which sets safety to 0 in some functions?
20:53:32
puchacz
it is supposed to calculate a goal function for numerical minimisation, each calculation takes parameters via hunchentoot handler, splits work between many threads and then merges result, the calculation takes about 4 seconds.
20:55:26
puchacz
however it seemed as it was hunchentoot that crashed last time, previous incidents, I did not check - "Session secret is unbound. Using Lisp's RANDOM function to initialize it." and then "Memory fault at 0x7f94e68f8000 (pc=0x536bf253, fp=0x7f8ea266c158, sp=0x7f8ea266c138) tid 0x7f8ea266f700
20:56:41
phoe
you could (sb-ext:restrict-compiler-policy 'safety 3 3) and recompile all of your code to make sure it is compiled with highest safety
20:57:18
puchacz
phoe: will it work and override even if there are "optimised" functions somewhere with safety 0?
20:58:49
puchacz
okay :) the thing is that sbcl image loads libssl when hunchentoot starts - I think
21:04:08
ark
puchacz : Are you running this from a lisp image/executable? If foreign memory is still allocated when the image is created (i.e. when `save-lisp-and-die` is called) it will likely crash the image when you start up again.
21:06:21
puchacz
ark - it is possible that libssl is being loaded when hunchentoot is loaded from quicklisp, not started later. I don't know. it happened only after few thousand calls, unfortunately few times already
21:07:37
phoe
if yes, how did you deploy your binary? did you close the foreign library before freezing and reopened it after thawing?
21:08:01
puchacz
yes, I generated binary on my computer, copied sbcl over to the virtual server and used a command line like this:
21:10:02
puchacz
sbcl --dynamic-space-size 8192 --eval '(progn (eval (read-from-string "(ql:quickload :swank :verbose t)"))(eval (read-from-string "(ql:quickload :bubrary5 :verbose t)"))(eval (read-from-string "(require :bubrary5)"))(eval (read-from-string "(in-package :bubrary5)"))(eval (read-from-string "(bubrary5::deliver-for-computation)")))'
21:10:27
ark
puchacz: If it helps, I blogged a little bit about using foreing memory with lisp images (in the context of lisp game development): https://recursive.games/posts/Beware-foreign-memory-in-lisp-images.html
21:11:26
phoe
puchacz: if you wrote the code, then you should know - do you close and reopen libssl?
21:12:27
puchacz
I don't touch libssl myself, and I don't start hunchentoot in deliver-for-computation
21:13:55
puchacz
shall I have a closer look what hunchentoot is doing, I hope it is not loading libssl when I is just (require ...) it and not start?
21:16:39
phoe
this is the list of libraries that need to be closed before you dump the image and reopened after you reload it
21:33:56
phoe
when you thaw the image, the pointer that refers to that foreign library is no longer valid
21:33:59
puchacz
I did what you asked me to do - manually started sbcl with the same memory settings, executed forms one by one except for "deliver")
21:37:00
phoe
one: in your dumping function, manually close the foreign library; in your reloading function, manually reopen the foreign library before you bring hunchentoot up
21:37:36
fiddlerwoaroof
Most ssl stuff on CL uses cl+ssl and you probably need to call (cl+ssl:reload) in your entry point
21:38:59
fiddlerwoaroof
The simplest way to solve this sort of problem is to deploy your source code and use ql to load it and start the server
21:40:03
puchacz
I could do it in this situation, it is for my personal computations and it is my personal rented virtual server
21:40:27
puchacz
however being able to create executable for users in general is too useful to give up on it
21:41:00
fiddlerwoaroof
Yeah, it's just more complicated and you have to know a bunch of implementation details
21:42:24
fiddlerwoaroof
The sbcl manual has some information and then your libraries can make things more complicated
21:43:02
fiddlerwoaroof
FFI in particular makes things complicated, especially if they use something like CFFI-GROVEL
21:43:47
puchacz
I don't use anything FFI myself, hunchentoot does. But I will use sqlite at some point
21:44:21
fiddlerwoaroof
One solution is to use the (mostly undocumented) feature of CFFI to statically link all your foreign dependencies into the lisp image
21:44:54
fiddlerwoaroof
I use this here: https://github.com/fiddlerwoaroof/daydreamer/blob/master/.circleci/config.yml
21:45:45
fiddlerwoaroof
The "easy way" to do this is to use my fiddlerwoaroof/sbcl-static:1.5.8 docker container, because you need to rebuild sbcl with the right options
21:46:10
ark
puchacz: my solution is to unload all foreign libs before the image is saved, then reload them when the image starts up. https://gist.github.com/realark/1b6970a13615a6e389294ab9480d1e09
21:48:08
fiddlerwoaroof
But the repositories I've needed to patch are all referenced here: https://github.com/fiddlerwoaroof/daydreamer/blob/master/clone-all-the-things.sh
21:48:41
puchacz
doesn't sound like a picnic. I will try ark's solution to unload and load back. but obviously I risk a crash when a user has different library versions
21:49:29
fiddlerwoaroof
To actually build the image, you just do this: https://github.com/fiddlerwoaroof/daydreamer/blob/master/build.lisp#L39
21:49:48
puchacz
by the way phoe, do you mean this library for delivery? https://common-lisp.net/project/ecl/static/manual/re55.html
21:51:39
puchacz
(and switching off my current setup with the virtual server so the meter is not ticking :-) )
1:11:27
dlowe
One interesting thing it does is provide a docker container so you can try it out with no setup
1:17:51
jasom
just because it seems like something that might already exist; a tool for pretty-printing durations to round amounts? e.g. print "3 seconds or "5 minutes" or "2 hours" or "3 days" or "3 weeks" given some type of time duration as input?
1:23:09
oni-on-ion
whoa -- just learned about: (format nil "~r" 1234) ==> "one thousand two hundred thirty-four"
1:24:24
oni-on-ion
jasom, i cant seem to find anything built in, perhaps this is helpful? --> https://github.com/quek/simple-date-time
1:32:23
pjb
oni-on-ion: even better: this is the (format nil "~:r" 1234) #| --> "one thousand two hundred thirty-fourth" |# time somebody found it!
1:37:35
oni-on-ion
its possible to do custom formats ? for eg., print digit from 1-26 for alphabet: "abcd -> 1234"
1:39:27
specbot
Tilde Slash: Call Function: http://www.lispworks.com/reference/HyperSpec/Body/22_ced.htm
1:44:03
no-defun-allowed
I believe it was this part that tripped me up: If name does not contain a ":" or "::", then the whole name string is looked up in the COMMON-LISP-USER package.
1:50:54
no-defun-allowed
I assume it could be so compilers could generate code for FORMAT rather than interpreting the string. If that rule wasn't there, the effect would be dependent on the current package, but that doesn't seem too difficult to manage
2:22:39
jasom
I've used that for escaping strings: (format X "Foo \"~/pkg:escape/\"" bar) is much more obviously correct that (format X "foo \"~a\"" (escape bar)), particularly when you have a long format string or many arguments.
2:26:57
pjb
(format nil "Foo ~S bar" "Hello \" World! \\ <-- ha! \"") #| --> "Foo \"Hello \\\" World! \\\\ <-- ha! \\\"\" bar" |#
2:53:24
vms14
what's the "correct" way for having a group of labelled functions with local variables, should I use let and put the labels inside that let, or make the labelled functions receive them as argument? two of them are recursive and one gets called from another, but needs an argument that the callee does not need
2:54:24
vms14
sorry if you cry with the code, but it's a bad implementation of a image format I've created
2:55:00
vms14
the image will be a list of positive and negative numbers and the first item will be the width of the image
2:55:58
vms14
positive numbers refer to a color and negative numbers mean steps or points to draw with the current color
4:20:03
vms14
Josh_2: ty but now I need to create a paint software able to export to this weird format I've created
4:21:06
vms14
and I have no idea about how I should do it, but since will be for basic pixel art stuff it should be easier
4:28:52
vms14
oni-on-ion: I'm using clx and I saw no way to import images more than monochrome bitmaps
4:30:30
vms14
new-clx has support for xpm format, but the old one not, I use the "old" one since I target sbcl
4:32:29
oni-on-ion
could always hook up opengl to the window, might be helpful for higher color and hardware acceleration if neede
4:35:12
oni-on-ion
nice =) couple months ago i got some raw pixel blitting happening and it was very fast. so one can use any image loader
4:35:28
oni-on-ion
like the recent PNG library for CL which is nice n fast (compared to the previous one, afaik)
4:36:39
vms14
if you want to get started I recommend to google for "simple clx", they're simple examples and knowing xlib you'll easily get the difference
4:37:11
oni-on-ion
i can imagine =) however i am just using raylib for now. its a nice library, than SDL(2), and works for win32 and the web as well.
4:37:35
oni-on-ion
for using opengl, i had it rendering Cairo onto opengl texture then to window in real time at hyper speed on old hardware
4:39:19
oni-on-ion
(cairo because i needed vector and also font. also the API is just about interchangeable (especially with ocaml!!) with html5 canvas so that my work could run on any modern system incl. mobile and game console)
4:43:21
oni-on-ion
cool =) i honestly would too but i want to show my friends who only have mobile, and my tablet is stuck with windows8.1
6:08:55
pjb
oni-on-ion: cliki.net references several graphic toolkits. I would suggest pgl, since it's very simple. (However it's not native). Depending on the sophistication of your paint application, and the target systems, McCLIM or CLX could be good choices too.
6:11:26
oni-on-ion
i think it was vms14 who is doing current clx work. i will regard this new information