freenode/#lisp - IRC Chatlog
Search
19:13:39
jasom
most ORMs tend to be buggy ill-specified implementations of half of a real object store; 99% of the time SQL actually maps better to the problems I have
19:15:01
jasom
though I wouldn't mind a sexp sugar on top of SQL, since that's more aesthetically pleasing to me than wrapping a DSL with ""
19:35:54
jasom
rumbler31: a tool to let programmers stay in their OO bubble and pretend that SQL doesn't actually exist ;)
19:39:25
dim
the problem space of application code objects in memory and of the relational model are not compatible, specialize each of them, the mapping is bound to fail
19:39:52
dim
the only interesting ORM style would be to map query results to app objects, as each SQL query defines a new relation
19:40:21
dim
I've seen that to some extend in POMM (PHP) and I think JOOQ (Java) has something to that approach
19:40:28
sukaeto
jasom: re: sexp sugar on top of SQL, I've been relatively happy with https://github.com/fukamachi/sxql
19:40:39
dim
in general ORM try to map the database model with the application in-memory state, and that can't help anybody
19:41:13
sukaeto
it doesn't do some of the more complicated dialect specific stuff (namely Postgres' ON CONFLICT and common table expressions), but it's extensible so you could always add those things if you need them
19:42:56
dim
disclaimer: I wrote a book on SQL this summer to teach that to app developers, available at https://masteringpostgresql.com ; I am quite opinionated on the topic...
19:44:16
jasom
dim: does it include basics of RDB? I was unable to take the DB course at university because there was no room in the class, so I've mostly just learned by doing...
19:45:25
dim
jasom: I think it does, but that depends on the specifics of the curriculum you missed ; you can get a sample with the full ToC for free (in exchange of your email) on the website
19:47:03
jasom
I limped by using key-value stores for like a decade before actually learning any SQL
19:47:17
sukaeto
anyway, to add my (potentially uneducated) opinion on ORMs to the discussion: I agree that trying to automatically map "sets of facts about the world" into "sets of objects in memory" is a fundamentally flawed errand.
20:02:46
p_l
the problem with ORM is that it's associated with ActiveRecord pattern, which is popular but not the only way that falls under "ORM"
20:08:45
sukaeto
I mean, I've used ORMs as just a way to query the database in the language the ORM was written in vs. embedding strings of SQL - that is, each ORM object is just a record representing some relation in the database, and the business logic objects consume/query the ORM objects. That works out just fine.
20:10:06
sukaeto
I still think things like SxQL are nicer for "querying the database in your language". Just one benefit of Lisp over most everything else. :-)
20:12:46
p_l
some ORMs provide explicit access to relational aspect instead of trying to shoehorn shit into objects
21:00:18
fouric
anyone have any resources on function hotpatching in CL? i've found a few pages on stack overflow that just tell me the bare minimum (how to hotpatch a single function using Emacs+SLIME), but i'm interested in knowing more, such as "how does this work?" and "how can i make my code \"aware\" of said hotpatching (e.g. add hooks that fire off when a function is replaced so i can run code that updates state)?"
21:01:21
fouric
stack overflow gives me like 2 pages that tell me what i already know, and google gives me those same pages before beginning to omit either "lisp" or "hotpatch" from my query
21:03:39
phoe
And your own DEFUN can contain code to fire any callbacks you want and call CL:DEFUN as a part of its operation.
21:04:21
fouric
i was hoping that there would be something built-in, but i'm willing to write my own variant of DEFUN if need-be
21:05:01
Bike
for how it works, it's a somewhat obscure point of compilation semantics. If a function defined in one compilation unit refers to another function defined in the same unit, e.g. by a call or #', redefining the other function is undefined behavior.
21:05:17
phoe
you also need to use something like ASDF's WITH-UPGRADABILITY for the functions that you want to be able to always replace.
21:05:20
Bike
But you can explicitly force a lookup by using fdefinition, either explicitly or by e.g. (funcall 'foo ...)
21:06:21
phoe
as long as your functions are *not* inlined by the compiler, setting a new fdefinition instantly causes the whole image to pick up the new function.
21:06:54
phoe
like, when the function is newly called, it'll use the new definition. old calls won't magically start executing the new function if the old function is in the middle of being executed.
21:07:49
jasom
fouric: under the hood, functions are typically called via indirection, so there is a pointer to the code for a function. When you redefine it, you create new code then just update the pointer
21:08:19
jasom
fouric if you know assembly at all, just look at this output from sbcl: (disassemble (lambda () (declare (optimize (speed 3) (safety 0))) (quux)))
21:09:31
jasom
then all of the call stacks &c. are roots for the GC, so no old function definition will get garbage collected until it is no longer on anybody's call stack
21:11:53
fouric
Bike: if there isn't some single document on the internet, that's fine, i'll gladly come here for help
21:12:19
fouric
...i was just trying to see if there was something that i could go to instead of re-asking questions that have already been put here multiple times
21:12:24
Bike
I guess the overall way to think of it is, the image is the image. it has some bindings, like of functions and classes. you can alter the bindings at your whim. these are basic language semantics. With functions, the compiler is allowed to integrate definitions and skip bindings sometimes, so you have to be aware of those times to do redefinition
21:12:42
Bike
But everything else is fairly safe. except there's also threading and who knows what's going on there.
21:13:52
fouric
is that how function redefinition happens? your code is running in one thread, and the SLIME client in another, and said client updates the definition of FOO or whatever, which the "main" thread picks up on the next time FOO is called?
21:14:56
fouric
Bike: i'm just trying to learn stuff while enroaching of the time of more skilled (and therefore productive) people in here as little as possible
21:14:58
Bike
the semantics are actually defined in a single threaded environment, and implementations do try to be nice
21:16:13
rumbler31
also, I personally appreciate the dive into more complicated topics that I might not yet even know how to ask about
21:20:05
jasom
also the specification lists when functions can be redefined; if they are either declared notinline at all call points, or not referenced anywhere in the file in which they are defined then it's safe.
21:20:38
jasom
basically permission to inline is logically the same as permission to assume the function will never change
21:21:24
rumbler31
jasom: what do you man about "not referenced anywhere in the file in which they are defined"
21:26:33
jasom
rumbler31: logically speeking there are 3 states for inlining a function; lets call them: inline, notinline and default. The default is "may be inlined anywhere in the same file as the function definition"
21:27:22
jasom
so a file with (defun foo () ) (defun bar () (foo)) the call to foo in bar may be inlined, absent any other declarations
21:28:39
sjl
(declaim (inline foo)) (defun foo ...) (declaim (notinline foo)) makes foo inlineable but prevents any compiler macros
21:29:18
rumbler31
so without explicitly specifying notinline on foo when loading the file, given the default inline state, a conforming implementation will warn/error when foo is redefined?
21:30:40
rumbler31
then.. I guess I still don't know what you meant to say with your second sentence
21:30:47
jasom
rumbler31: the assumption is that you know what you are doing if you redefine functions unless something very obviously is wrong (it was declared inline, you have two definitions of the same function in the same file)
21:31:27
jasom
rumbler31: a conforming implementation is allowed to assume that a function defined in file X has a fixed definition for the entire body of file X.
21:32:04
rumbler31
what does fixed definition mean? s.t. re-invocations of defun are unspecified behavior?
21:33:07
jasom
rumbler31: in practice it usually just means that the redefinition won't take effect, but I suppose it could have safety implications if the type of the function were inferred (not sure if sbcl does this).
21:34:09
jasom
For the purposes of incremental development using C-c C-k will usually mean that all functions not declared inline will work right (since you recompile the entire file).
21:37:03
jasom
macros definitions are invoked when the macro is expanded. When compiling this is the same as compile time.
21:37:27
jasom
so if you change a macro, there is no magic that finds all places in which it was expanded and recompiles them.
21:37:50
jasom
you should use asdf if you want this to work (it tracks dependencies on a file-level basis)
21:40:01
rumbler31
so the part about macros not being re-evaluated I understand. I will ponder how that affects function redefinition
21:40:33
jasom
My point is everything I said up to that sentence refers only to redefining functions. If you redefine macros, then you need to think differently
21:46:49
aeth
jasom: Return type information isn't inferred unless the function is inline afaik, i.e. (defun foo () (the single-float 1f0)) (defun bar () (+ 1f0 (foo))) (disassemble #'bar)
21:49:02
aeth
You could probably wrap a the around the function call, which would get rid of the generic-+, but still do type checking because it still won't trust you not to redefine #'foo. You'd probably need sb-ext:truly-the in SBCL, and you'd need to *really* make sure your wrapper will never fail.
21:52:45
jasom
I assume changing foo to be (random 1.0) or some such will have a similar effect. but use a single-float specialized add
21:53:20
aeth
jasom: I think that it's not a type thing but a constant thing, though. I've had the same thing happen with macros that generate functions that return constant strings. If I changed those macros in SBCL, I would then get a runtime type error on the user of that generated function if the length of the generated string changed (because length is part of the type in Pascal-style strings)
21:53:25
jasom
yup. So sbcl does assume that function-types for functions that are not declared inline are fixed
21:54:08
aeth
jasom: yeah, but I think it's specifically an optimization for functions that are returning constants
21:54:36
jasom
because I get a FPU add instead of generic+ when I change it to return a random single-float
21:55:31
aeth
You are correct, I just tested that with (random 2f0) and I get ADDSS and it tests for '(single-float 0.0)
21:57:05
aeth
I don't get a warn when recompiling foo to return "hi" (rather than the file), I just get a runtime type error when calling bar
22:00:10
aeth
(well, it would warn about the return type of foo not being compatible with +, so my example isn't perfect)
22:01:05
aeth
This is interesting, though. It means that fewer, larger files aren't just a stylistic choice. It will potentially make the code more efficient, but at the cost of possibly causing issues with recompiling just an individual function rather than the file.
22:01:31
stylewarning
aeth this is well known about compiling things within the same compilation unit
22:02:02
jasom
asdf has with-upgradability at least partly because the entire thing gets put into a single .lisp file
22:05:06
Bike
with-compilation-unit should be usable to make more things optimize together, at least in theory. hell if i know whether sbcl does
22:25:37
aeth
with-compilation-unit would only be necessary to get file-like optimizations in the REPL, though, right?
22:26:44
aeth
pjb: Is removing the file boundary possible in a way that's compatible with ASDF? It might be useful as part of a build step for distributing a final executable.
22:27:08
pjb
You would need asdf support for with-compilation-unit. I don't remember if it's available or not.
22:28:07
pjb
Also, you raise a interesting question: can we always concatenate all the files of a project for distribution, without changing semantics, vs. a separate compilation.
22:28:43
pjb
Obviously, it's not possible in general: you would have to add eval-when to be able to compile the concatenated file.
23:24:00
drmeister
That’s what I assumed with-compilation-unit was for when I first encountered it. Optimize whole compilation units together
1:29:31
pfdietz
I'd say not hard, although there are some parts you don't want to do first. Restarts are a bit tricky.
1:35:11
pfdietz
Restarts are a way to resume after a condition (like, an error) is signaled, without unwinding the stack.
4:01:59
Colleen
beach: drmeister said 1 hour, 17 minutes ago: concrete-syntax-tree-base.asd contains a dependency for :alexandria but doesn't use anything from the system - could you remove it? Also - I submitted a pull request for the improved fix for handling end-of-file in the sicl reader.
4:38:51
drmeister
Hey lispers - when people refer to "tooling" as being important in how they use their programming languages and IDE's - do you know what that means?
4:39:27
drmeister
I think I kind of know what it means - but I'm trying to broaden my understanding of what people think it is.
4:43:38
drmeister
http://reactkungfu.com/2015/07/the-hitchhikers-guide-to-modern-javascript-tooling/
4:45:57
drmeister
When I read responses on Hacker News to new language announcements - invariably someone says "Your new language X sucks because it lacks what my language Y has - which is the tooling".
4:47:28
drmeister
One of the promises of Clang's AST tools that hasn't really been realized afaik is that it would improve tooling for C++ syntax aware tools.
4:48:36
drmeister
syntax highlighting, finding locations that call a function, finding where functions are defined - these could be made more robust with cst-to-ast.
4:52:11
beach
Currently, most implementations implement TRACE by modifying the code. Then you can't debug the debugger.
4:52:12
drmeister
Imagine if someone makes a poor choice for a function name - or dynamic variable. grep/search/replace is brittle and error prone. A syntax aware search and replace would be much more robust.
4:54:05
beach
The good thing here is that I plan to use CST-to-AST also in Second Climacs, so that we can avoid code duplication.
4:55:11
drmeister
Can I just say that I think "Second Climacs" is an unfortunate name for an editor?
4:55:50
drmeister
I feel the same way about "Slime" - love the programming environment - hate the name.
4:57:34
drmeister
That won't make me stop using something - but I'd love to see a better "official" name.
5:02:30
beach
The point here is that our Common Lisp implementations need to make breakpoints "thread aware" so that we can have one thread run the debugger to debug another thread.
5:02:50
beach
As long as that is not happening, we are doomed to have inferior programming environments.
5:04:07
beach
If we do it that way, we can even allow the user to set breakpoints in standard functions like READ or GETHASH. Currently, that would almost certainly make the debugger stop working.
5:05:13
drmeister
I don't disagree. For things like that I currently use gdb or lldb - but they are not lisp aware.
5:07:33
drmeister
I say I think it compiles a wrapper function because I haven't investigated how it works. I inherited it from ECL and it just works.
5:08:21
aeth
Do people use issue trackers to track future features in large Lisp projects? Or do people use something else (e.g. org files?)
5:09:50
beach
drmeister: Then you are not only recompiling the function, but also modifying the FDEFINITION.
5:21:44
beach
Anyway, I suspect Clasp and ECL (just like SBCL) don't have thread-aware trace points or breakpoints.
6:23:56
beach
So here is an idea for the implementation of thread-aware breakpoints in x86. The important feature is that threads that are not being debugged should not take too big a performance hit.
6:24:00
beach
I suggest that, when code is compiled with a high value of the DEBUG quality, then the execution of each form starts and ends with a test of the DF status flag. If 0, then execution continues as normal.
6:24:01
beach
If 1, then some further action is taken to determine whether there is a breakpoint at this point in the code. The "further action" remains to be specified, but the slow version would be to interrogate a breakpoint table in the thread instance.
6:25:32
beach
The use of the DF flag means that, if the implementation uses specific instructions that depend on its value, then the value has to be saved and restored around the execution of these instructions.
6:33:16
rme
when I last looked into using the x86 direction flag, I found that manipulating it was expensive. it ended up being faster to set a flag in memory.
6:33:49
White_Flame
I'm kind of preferential to Java-style interruption, where an address is routinely read. If it's to be interrupted, the address range is made unreable. Now, you could extend this so each thread is reading a different address (just an indirection through a thread-local pointer), and/or use different addresses per potential breakpoint.
6:34:15
White_Flame
I think you have to push flags to stack to read the DF flag, only set & clear are part of the ISA
6:35:35
White_Flame
hmm, if you have ample room in your 64-bit address space, you could read thread_local_page + N*program_counter to get a unique memory-protection based interrupt per location in your code
6:36:45
White_Flame
and with future fixes for meltdown/crispr, it might even be quicker to interrupt than it is now ;)
6:37:09
rme
I'd want to look at trying to use the dr0 through dr3 debug registers on x86 for setting hardware breakpoints if possible
6:39:34
beach
rme: I haven't had the time to read up on hardware breakpoints, but that's an important subject.
6:43:30
beach
I still like the DF over an address in memory, because that address would have to depend on the thread, making it expensive to read because of indirection. But here is a compromise: at the entry of each function compiled with a high value of the DEBUG quality, move the DF flag to a register or to the local stack frame so that each further test inside the function is cheaper.
6:46:07
beach
Some simple hash-like technique could be used. Take the PC modulo (say) 256 and consult a table in the thread instance. If a bit is set there, then consult a complete hash table to determine whether this PC value actually has a breakpoint.
6:47:11
White_Flame
I suspect the Java way is one of the least intrusive, though it probably does tie up a cache line
6:47:59
rme
there's almost certainly going to be thread-local memory maintained by the lisp runtime. on ccl, %fs or %gs points to the thread-local memory block (except on highly advanced systems like macOS, where we have to burn a gpr for this).
6:48:23
White_Flame
beach: each thread reads its own address; you only deny read permissions for the memory page for one thread, in my idea
6:49:53
White_Flame
basically, each thread just reads a canary location in memory. When you want to halt a thread, play with the memory manager for a particular canary location