freenode/#lisp - IRC Chatlog
Search
20:04:00
sjl
> The symbols which name the slots must not be used by the implementation as the names for the lambda variables in the constructor function, since one or more of those symbols might have been proclaimed special or might be defined as the name of a constant variable.
20:05:03
sjl
If you're expecting foos2 to have an initform of foos1, and that that would evaluate to the keyword you passed for foos1, I think that bit of the spec explicitly forbids that
20:08:53
sjl
interestingly, sbcl and ccl DO use the symbols as the parameter names if you specify a BOA constructor
20:16:23
fourier
please recommend some simple documentation generation system from CL sources. I need something what can extract docstrings from functions and generate github markdown document. could write something myself but maybe something exists already
20:20:39
rme
I'm not trying to tell you what to do, but one option would be to write actual documentation, rather than fiddling around trying to generate it from docstrings and introspection.
20:21:22
Bike
the page on defstruct says the initforms are to be evaluated in the lxeical environment of the defstruct, which pretty much forbids that
20:21:51
sjl
Bike: yeah, I don't think that can work for the keyword constructors because they're supposed to use other symbols as the lambda arguments
20:26:46
sjl
> If no default value is supplied for an aux variable variable, the consequences are undefined if an attempt is later made to read the corresponding slot's value before a value is explicitly assigned. If such a slot has a :type option specified, this suppressed initialization does not imply a type mismatch situation; the declared type is only required to apply when the slot is finally assigned.
20:35:43
Bike
new question. if the clhs defines a function can an implementation have it be a generic function instead?
21:04:10
k-stz
hey, I try to parse a formula like !A and generate ==> (not A). I'm stuck on how to approach this, do I use READ-CHAR or just convert it to a string? Problem arose when trying to parse formulas like: !(a -> b)
21:23:50
stacksmith
k-stz: suggestion: make sure you understand Lisp pretty well before attempting to implement a compiler for another language. Otherwise you will be in a world of pain. You must pay special attention to the CL reader and its limitations, symbols and packages (in terms of how your syntax maps to CL)...
21:26:17
stacksmith
Another suggestion: unless you really need a different syntax, stick with Lisp and save yourself a lot of trouble.
21:32:24
k-stz
stacksmith: thanks, I wanted to try a simple parser in lisp because I never tried it and hope to learn something. That felt like a good starting point
21:37:23
stacksmith
k-stz: I do not mean to discourage, but I speak from personal experience of trying to do something similar when I started with Lisp. It seemed like an easy enough task, but led to a serious smack-in-the-face realization of just how pathetically ignorant I was at the time. It was a great experience but your appreciation of such things may be different.
21:43:52
k-stz
stacksmith: I think I see your point, this already isn't as easy as a thought. No damage done, though. Seeing the first few problems makes me attentive to solutions, and I just might know where to look next
21:46:51
stacksmith
k-stz: Another observation is that you can do it in a Lispy way, by using the Lisp reader, macros, etc. or by parsing text, the way you'd probably do it in C or whatnot. You will learn entirely different things. Lisp is great for writing DSLs, but it is not a task for a novice.
21:51:05
stacksmith
k-stz: Read 'On Lisp' - it is full of useful information about doing things like that, including an entire Prolog implementation in a few pages of code, and information about reader macros and regular macros.
22:01:18
k-stz
stacksmith: I have this book on my radar, I should also encounter this topic again in SICP soon. Thanks again o/
22:05:13
k-stz
I found what I read about Erik Naggum enjoyable as well, I'd say because he can articulate his opinions well and he touches on many things. It doesn't appear shallow
22:06:04
Shinmera
While his writings are very insightful, they definitely often are inflammatory as well though.
22:09:35
stacksmith
I learned a tremendous amount when I started out - by reading Naggum transcripts. It was never boring, although somewhat painful on occasions.
22:11:36
dim
https://tapoueh.org/blog/2018/03/object-relational-database-management-system/ if you're interested, comparing PostgreSQL type system / function overloading with CLOS generic functions
1:18:32
Bike
you put the defun in your repl, and then the next line try to call it, and it doesn't work?
1:24:52
Bike
gethashes is a function that gets a value out of some hash tables. it has no provision for writing into hash tables
1:25:07
Bike
there is no way for the compiler to look at your loop and figure out an inverse procedure, see?
1:26:28
Bike
it analyzes the form (gethash key table) and macroexpands into something that writes into the table.
1:29:50
Bike
you can also try (macroexpand-1 '(setf (gethashes foo "two") 2)) to see why that didn't work out.
1:34:40
Bike
if you're still thinking about pointers, keep in mind c has lvalues as a distinct concept. you can't get the address of a register variable but you can assign to it no problem.
1:36:10
jasom
johnnymacs: you know how you can doo foo[bar] = baz in C, in this case SETF is like the = and gethash is like foo[bar]
1:37:21
Bike
you can do (incf (gethash foo bar)), for instance, to increment a value in a hash table
1:38:08
stacksmith
johnnymacs: unlike forth, where cells are pointers and @ and ! read and write to them, in Lisp mentioning a variable fetches it by default. setf sets it. setf can set a lot of complicated forms like gethash. You can also write a function to let setf know what to do with your gethashes.
1:53:08
Bike
so, lisp has assignment, but it is not pointers, and it is not "hacked to behave like pointers" or whatever
1:54:22
Bike
they aren't! all you're trying to do is set values in a hash table! that is easy to do with setf gethash, you just want to deal with pointers instead for some reason
1:55:19
Bike
yeah, and you already wrote that. now you just need to write a (setf gethashes) that does a loop the other way or whatever.
1:56:48
johnnymacs
If I am going to modify setf to handle gethashes the only way I can do that is with an array/hash table of functions or nested conditionals
1:58:04
Bike
this gethashes thing, it's defined so that (gethashes table a ...) = (gethashes (gethash a table) ...), right?
1:58:41
johnnymacs
Well what happens to the speed of setf as the number of cases setf can handle increases
2:01:12
Bike
(setf (gethashes table ...) new-value) will expand into (funcall #'(setf gethashes) new-value table ...)
2:02:02
Bike
of course, for most uses i imagine the number of keys will be fixed, so you could write something to do it without a loop
2:21:01
stacksmith
johnymacs: it's great that you jumped in and are coding, but you have a bit of a learning curve. Rather than fighting it, you need to get to the point where CLHS makes sense. Also not bitch that CL is getting in your way. It is your brain that's in the way of accomplishing tasks, not CL.
2:25:01
stacksmith
Bike's code should pretty much to the basics. Also read about Lisp macros - I think you may not fully appreciate them. They are kind of like Forth immediate words - they run at compile time and transform the code to your wishes.
2:42:04
jeosol
anyone encounted (sb-vm::tal-call-symbol ....) error while running multiple threads. My code runs okay when I run serial, but I switch to parallel, I get that (sb-vm::tal-call-symbol ...) error and then says a function (defined by macro) is undefined
2:49:47
jeosol
This is the link to the section of the code. Just create pastebin account. Pardon the upcase, as the code is generated from macro
2:50:41
jeosol
First of all, I will say I am not an expert lisp programmer, but I am writing an application in Lisp using SBCl for an optimization challenge with deadline in a few weeks
2:51:43
jeosol
things run serial without problems, but when I switch to parallel to gain some speed up, I get problems with GET-CPG-CELL-FACE-X- not being defined.
2:52:27
jeosol
I just simply abort. I have never been able to run the code in parallel lots of macro generated functions.
2:54:37
jeosol
the problem is an optimization problem. I am doing a bunch of F(X), where X is a vector of solutions. There can be many solutions in one iterations (think stochastic optimization algorithms), I can evaluate serially, but can get good speedup
2:55:14
jeosol
if I do things in parallel. Serially evaluation runs okay, but when I switch to parallel, I get some that SB-VM issue. I am using lparallel library
2:56:10
jeosol
@pierpa what is background/domain? the data for the challenge is not public in the sense that a request has to be made to get the data
2:56:54
jeosol
and solving the problem needs access to some other black box modeling tools. I can provide more details offline if needed
3:00:42
jeosol
@Bike, this new paste shows the restarts I get when there is an error https://pastebin.com/bJqGkceM
3:02:15
jeosol
the calling function is defined in the same package as the function that it is complaining about, except that the latter is defined by a macro
3:03:45
jeosol
@Bike, it is weird, runs okay, when serial, but with parallel it blows up. there is something in the code base, that is not correct. I thought it was race condition related so I ran parallel option but with one worker per batch, same error
3:08:49
jeosol
@pierpa, the problem is really challenging, each function takes 14 mins on my machine. and I need to run hundreds or thousands of function evaluations, so parallel is definitely the way to go
3:08:54
stacksmith
jeosol: are there any global variables in play? They may not be the same in a different thread...
3:10:00
jeosol
Thanks @stacksmith, appreciate the help. That defvar is in the same package. It is a vector of other functions defined by a macro.
3:10:37
stacksmith
I don't know if that's sufficient to be in the same package... I would print it and make sure...
3:10:52
jeosol
I posted an earlier problem, I got recommendations (I think it was @Bike, not sure) to use that instead of constructing function names dynamically using intern and format ..
3:11:14
jeosol
I posted an earlier problem, I got recommendations (I think it was @Bike, not sure) to use that instead of constructing function names dynamically using intern and format ..
3:11:35
stacksmith
It's fine to use an array, it's just that it's global. So when a new thread starts, it is probably nil.
3:13:40
stacksmith
Inside that thread that is. You may need to rebind it in whatever function that starts a new thread in lparallel.
3:20:32
jeosol
in the debugger I get this error: COMMON-LISP-USER::GET-NORMAL-CPG-CELL-FACE-X-1 is undefined
3:20:51
jeosol
Not sure why it is looking for that function in the CL-USER package when I run in parallel?
3:28:42
stacksmith
jeosol: If you create a new thread and output to *standard-output* you will crash, because inside the thread *standard-output* is nil (or maybe unbound, I can't remember).
3:30:09
Bike
not directly, but if it's looking for a symbol in the wrong package that seems like a good place to start looking for problems
3:30:39
stacksmith
Honestly, the first thing to do is print the array and see if it has functions in it...
3:45:45
pierpa
jeosol: wait, "defined by a macro" means that your macro is creating the symbol you use as the function name?
3:45:56
jeosol
@stacksmith, my application runs and writes statement to the *standard-output*, e.g., results from computations, etc.
3:46:44
stacksmith
Are you sure that the output comes from inside the thread - or from main thread after the computations are done?
3:47:50
stacksmith
Also, if you put the whole program into a thread, it's possible that some intialization function causes macros to expand inside the thread, with CL-USER as the default package...
3:48:48
jeosol
@stacksmith, no, I am not saying you are wrong or anyone. I need suggestions on what the problem could be
3:49:35
stacksmith
jeosol: if you want to rule out the issues I proposed, just output the values of *package* and your array from inside the thread...
3:52:55
stacksmith
Bordeaux-threads docs state "Global bindings are shared between threads: the initial value of a global variable in the new thread will be the same as in the parent, and an assignment to such a variable in any thread will be visible to all threads in which the global binding is visible." - but I always have to rebind *standard-output* or it does not work.
3:53:59
stacksmith
So if any of the globals at thread startup happen to be bound, the values are lost.
3:54:51
stacksmith
Hence, I suspect that package may be bound, and inside the thread it's what it was originall - CL-USER.
3:55:33
pillton
I'm not sure how bordeaux-threads can make such a claim given that it is up to the implementation to decide what happens with the environment when a new thread is created.
3:56:59
stacksmith
I would imagine that threads are started way deep, in a place where bindings are not in scope.
4:00:07
Bike
you could have all special bindings be in thread local storage. then there'd be no global bindings.
4:02:55
stacksmith
Then you would have to keep track of all special bindings separately. Do implementations do that?
4:04:30
stacksmith
Well, whatever way symbols are associated with values - you would need to keep a separate list and upon thread creation recreate it in the new TLS buffer - and you would still wind up with the initial global values...
4:06:56
stacksmith
Except, the outer bindings would contain the entire rest of the system, requiring a full recompile every time anything is changed - at least the way SBCL currently works...
4:09:23
rme
Typically, implementations make certain special variables thread-local (such as *print-base*, etc.). Other special variables see a global value. In CCL, there's a form called defstatic which is like defparameter, but asserts that the symbol in question will never be given a thread-local binding.
4:12:49
Zhivago
Personally, my preference is to use (possibly very light-weight) processes rather than threads, so that this problem doesn't exist.
4:14:38
jeosol
This is a pain, not resolved, yet, but may be I have to rethink how I structure things
4:16:07
jeosol
I will keep hacking at this as I need the parallel functionality. Thanks for all your comments. I am monitoring the chat for hints ... :-)
4:16:36
stacksmith
jeosol: have you tried to output *package* from the environment in which your macro runs? Is it possible that you have an (eval-when ...) initialzation that somehow winds up inside the thread?
4:20:21
stacksmith
I think so, if the thread initialization is around the macros. They run in a new context in CL-USER package
4:24:49
stacksmith
I try to spawn new threads after the system is loaded and all compilation and expansion is done...
4:34:49
stacksmith
I think so, but you may want to decide if you want to truncate or deal with floats or fractions, etc.
4:41:55
jeosol
@stacksmith, I don't start the threads in eval-when. I use that to define the functions when I load the system. I think the problem is related to *package*
4:48:51
jeosol
My application is very large so I am using asdf:package-inferred-system for all my files and I have to manage things carefully. See the updated repo as of January 2018 https://www.youtube.com/watch?v=sgdiRh-alfQ
4:48:56
pierpa
that EXPORT will not make the symbol accessible in any package apart than it's home package
4:49:21
stacksmith
Do you need to have the macros in eval-when? It may not be the issue, but is there a reason?
4:50:57
jeosol
@stacksmith, I only use the threads for the function evaluations. Multiple calls SIMD type, F(X), where my X's are different CLOS objects
4:52:00
pierpa
jeosol: your system is the one shown by the video, or your system is the thing which produced the video?
4:52:06
stacksmith
Well, I still think that the first thing you should've done a couple of hours ago when we started this - print or stick the value of *package* somewhere and look at it.
4:54:10
jeosol
There is a visualization for SBCL code base with GOURCE on youtube but not sure how updated it is. There are others for LINUX, python, etc
4:57:49
jeosol
The tool shows evolution and complexity of GIT repo. I was just fooling around with options to GOURCE and added elasticity to the branches etc
5:01:24
jeosol
@stacksmith. The code base does not focus on a single problem but a series of interrelated problems in a domain. The part on optimization is focused at that challenge I mentioned. There are parts doing some small ML type workflows, etc
5:02:05
jeosol
but my focus for the last few moments is in the optimization part to get it to work for that challenge problem.
5:03:53
jeosol
I asked a question yesterday here about web frameworks in CL to use. I wanted to build a web application to have public access. I know a bit of hunchentoot and cl-who, html, javascript, etc, I will do this later, but for now, I just run within slime
5:05:54
jeosol
@stacksmith, like I said before, I am not an expert on a lot of things. The part of Cl I understood the most is CLOS part because my code is essentially OOP which is easier to manage for me. How things work internally, is not all clear to me so I learn as I go
5:06:04
stacksmith
jeosol: I've been up for two days and getting fuzzy, but if your export is in the macro definition, you probably want (export 'name ,*package*), no? Splice in the package as it exists at expansion time, not when the macro was defined...
5:07:55
stacksmith
Wait, that's backwards. You want no comma, the package at expansion time, not at definition time. Can someone look at this because I am getting cross-eyed...
5:08:02
jeosol
@stacksmith, thanks for the suggestions, you can get some rest man. I will go through the messages to review the messages again
5:09:40
stacksmith
Wait, it is right there. Without a comma it expands to *package* and gets its value then and there, with a comma it spliced in the package at macro definition time.
5:10:17
pierpa
you should either create the symbol in the right package with (INTERN "NAME" the-right-package), or IMPORT the symbol in the right package later. Your EXPORT does nothing.
5:11:31
jeosol
Since I use asdf:package-inferred-system and essentially one-file defsystems, I usually import the functions one.
5:12:10
jeosol
For the error I am getting the caller and called function are in the same package. Everything blows up when I go parallel function evaluation, otherwise, it runs without problems.
5:12:57
stacksmith
jeosol, just to be clear, macroexpanding your macro in REPL and having it expand in its eval-when environment during startup is likely very different. That's why I've been urging you to figure out a clean way to see what *package* is, at that point and go from there...
5:13:32
pierpa
I suggest reading carefully the packages chapter of CLTLII, or alternatively the relevant parts of CL recipes.
5:13:33
jeosol
@stacksmith, ok let me do that. You men when I am running the application, to print the value of *package*
5:14:39
stacksmith
yeah, but be careful about when you do it. You want to modify your macro to push the package onto a global, I think - because it might expand more than once....
5:14:49
jeosol
Thanks @pierpa, I need to get a copy of that book for sure. When my system got large, after a chat with Fare and switching to one-file system, it is painful but manageable
5:41:24
jeosol
ok, I asked because it you are working with time and dates, it is easier to use one of the time-date librarieis
5:41:48
jeosol
Which one you use depends on what you are doing, time resolutions, if you want to track local-time, etc
6:32:44
stacksmith
didi: not as good as "In theory, there is no difference between theory and practice. But, in practice, there is. "
6:35:56
stacksmith
Or perhaps my favorite by Alan Perlis: "Syntactic sugar causes cancer of the semicolon."