freenode/#lisp - IRC Chatlog
Search
18:28:20
beginner_supreme
Hey all, curious if high-performance compilers create contiguous objects when given element-type, so for (make-array n :element-type some-object/struct), the rough equivalent in C would be: malloc(n * sizeof(some-object/struct));
18:29:47
beginner_supreme
Is this sort of contiguous allocation done at all for these types of expressions (not just for arrays) or are only pointers contiguous, having an extra indirection
18:37:20
copec
I'm not sure if all compilers do, but the presumption is that you are giving it sufficient information so that it can.
18:40:37
beginner_supreme
I read one opinion of it on stack overflow, and the idea is that it would stress the garbage collector out if such an optimization were done. https://stackoverflow.com/questions/29008545/any-ways-to-allocate-flattened-array-of-structure-as-c-in-common-lisp
18:42:12
stylewarning
beginner_supreme: in Common Lisp, usually will not be allocated necessarily like that
18:42:19
copec
If you have reached a point that that is important enough, I would consult the specific implementation docs and/or source
18:42:44
stylewarning
beginner_supreme: it'll be more like: void **array = malloc(n * sizeof (void *))
18:43:01
stylewarning
beginner_supreme: followed by n allocations of the objects, which will inadvertently be contiguous
18:46:55
beginner_supreme
For the future, are such inquiries suitable for this channel or must I refer them to #clnoobs?
18:50:11
copec
I tend to ask in clnoobs first, and reflect the question in here or one of the other CL channels if I can't get an answer. There are some pretty knowledgable people in there though, it's not newb on newb action
19:20:16
White_Flame
beginner_supreme: garbage collectors can clump together data references by their nature. you can stack-allocate in CL (dynamic-extent). and of course, type-specified vectors as you mentioned
19:22:08
beginner_supreme
So the allocated objects themselves would be contiguous, but would the vector would still contain pointers to those objects, not the objects themselves. Right?
19:23:11
White_Flame
vectors of numbers, characters, etc, can often be packed together. Vectors of structs and such might not
19:24:19
White_Flame
of course, you need to specialize its type as you create it to try to gain such advantages
19:25:00
dlowe
even if you have a vector of structs, they're not going to be laid out in memory like they would in C
19:26:34
White_Flame
you might get the struct slots themselves, but structs themselves don't use special packed representations. If the slot value fits in a machine word, it'll be in the struct. Else it'll be a pointer
19:26:54
White_Flame
erm, you might get a typed vector to hold a linear array of the struct slots themselves...
19:27:08
makomo
dlowe: is that guaranteed or just something that's not really done by compilers or something else?
19:27:50
makomo
so hypothetically, you *could* have a vector in which the structs are laid out contigiously without any indirection
19:28:32
White_Flame
but since dereferencing is though standard functions, as the number of types of backends to those functions increases, so does the amount of type testing & dispatch
19:29:39
White_Flame
CLOS objects can be defined to migrate to new versions of their class. instantiated structs keep their old structure around if you re-defstruct
19:29:54
stylewarning
the real problem imo is the meaning of (setf (aref packed-array i) new-struct)
19:30:17
stylewarning
how referencing/pointers/etc work. You would need typed pointers into typed arrays
19:31:57
stylewarning
it would be so cool if (UPGRADED-ARRAY-ELEMENT-TYPE S) == S for a structure type S
19:33:03
stylewarning
I'd even be happy with a super special struct type, like array of packed words or something
19:34:13
White_Flame
yeah, an extended cons cell with >2 slots is something I've mused about on occasion
19:34:19
aeth
arrays of structs, typed lists (this is trivial to do yourself from structs, but it won't be as efficient), and typed hash tables are on my wish list, as well as type declarations behaving everywhere like in SBCL (permits gradual typing) and being able to declare specific optimizations (like TCO or fast floating point) rather than general categories
19:34:39
aeth
Add all of the stuff on the list I just gave and CL is a good competitor for something like C# or Java
19:35:53
Bike
if it's a structure of uniformly typed slots you could probably fake it with some special functions or macros
19:36:54
aeth
e.g. no need to put a struct of (x, y, z) single-floats in an array when I can just have a 2D array
19:37:43
aeth
I then cover this up with my array macros, which are fairly elaborate. https://gitlab.com/zombie-raptor/zombie-raptor/blob/18db1a4e34892adf4f2f4c20d52df416a3a340a9/util/array.lisp
19:38:56
aeth
e.g. (setf (array-row-of-4 foo 32) (array-of-4 some-array)) ; essentially copies the first four elements of an array (no checking, so it's faster) into the first four elements of the array row 32
19:40:06
aeth
Unfortunately, this isn't a perfect substitute for arrays-of-structs because all items have to be of the same type (which could be T, I guess)
19:41:03
aeth
(It's not really copying, either, but if you're doing specialized arrays (numbers or characters) you don't really notice.)
19:41:39
stylewarning
aeth: I've faked it by defining things like (define-struct-array <defstruct options>) which just creates a struct-of-arrays
19:42:18
stylewarning
aeth: It at least makes things a bit more efficient, even if you're not reaping all of the rewards of having a single object or having a lot of memory locality.
19:47:08
beginner_supreme
The point made about how avoiding indirection by not having an vector of pointers would require typed pointers for pointer arithmetic is a nice point..
19:58:37
aeth
copec: I tried 2-3 times to get it through roswell, which I use to test in various implementations, but after failing 2-3 times (and it's a long, C++ compile) I don't want to try again via roswell
20:44:00
antoszka
copec: https://common-lisp.net/ <- that might be another starting point, probably also worth idling less here and on #clnoobs and #lispgames
23:46:35
aeth
If you had a way to sort https://www.quicklisp.org/beta/releases.html by category you'd get basically the same result. A list of everything, but with no way of telling if it's actually up to date, worth using, etc.
23:54:13
aeth
Library selection is a tricky thing. There isn't really one Common Lisp. It's a very multi-paradigm language. Sure, you can definitely write code that's not "Lispy", but even if the code is "Lispy" there's many different valid styles.
23:57:34
aeth
Lisp is not even necessarily a dynamically typed language! SBCL did gradual typing years before it was cool.
0:05:38
black_13_
or what argument should be given to (ffi:load-foreign-library on a windows machine
1:34:26
Prof_Birch
Is lisp good for rapid prototyping an application before you are sure of exactly how you want to approach the problem?
1:35:21
Prof_Birch
I am hearing it's good for prototyping in a DSL kind of way, but I wasn't sure if it is actually good for prototyping in a general use of the word
1:37:31
Prof_Birch
I have ADHD, so it can be a bit hard for me to lay out exactly what I intend to do before hand, even if I have a general or somewhat planned idea
1:38:02
Prof_Birch
I *think* lisp could be a good choice for me to lay my ideas out, and play with them to express what I mean before I code it in something like Java
2:10:56
pierpa
unless there's a customer paying you to deliver something in some other language, I mean.
2:14:33
Prof_Birch
Hey, I'm here to ask questions and break stereotypes I read online. So feel free to enlighten me
2:15:27
Prof_Birch
I assume the answer will be yes, since I'm on the lisp channel, but is it really all that? I hear whispers online, but nobody I know uses it
2:17:46
akkad
yeah, it's a language. if you put the features into context of when they were made. it's magial
2:20:22
akkad
if you mean as a family of languages. CL might be seen as a little more strictly limited wrt innovation.
2:20:51
Prof_Birch
I don't know enough about Lisp to pick a flavor, and elisp didn't seem suited for general programming
2:21:09
Prof_Birch
I figured learning the basic syntax would be the place to start, and CL was the dominant one as far as I could tell
2:21:25
akkad
don't feel like you need to limit yourself. once you get a lisp down, the rest are much easier to grok typically.
2:22:08
beginner_supreme
I sort of like to look at (and validate my desire to study CL) as being a very very fast python with the freedom to extend it as I want/need.
2:22:24
Prof_Birch
I'm just searching for a language that can help me build programs that are just on the edge of my mind, I can't seem to pin it down but I'll know what it is when I see it
2:23:27
aeth
Prof_Birch: SBCL is gradually typed, if you don't mind relying on the not-entirely-portable behavior of type declarations.
2:23:46
Prof_Birch
I'm not worried about portable. and I barely get paid at all. I'm a bit of a hobby programmer
2:24:37
beginner_supreme
Someone in the #clnoobs room showed me how type declarations are not even necessary most of the time. Compilers like SBCL have great type inference.
2:25:20
aeth
beginner_supreme: There are two places where the lack of type inference will hurt your performance: numbers and sequences. This is because there's a kind of generic function (but not defgeneric generic) for numbers and sequences, like + and map
2:27:06
beginner_supreme
Right, but for things like using (elt) instead of (aref) - these are inconsequential as far as I'm aware. elt would just be expanded to an aref if the type is inferred to be an array
2:27:39
aeth
But it probably won't infer the type is an array if you use sequence operations on it and the array is passed in as a variable instead of created within the same function
2:28:16
aeth
Arrays have the added overhead of bounds checking when the dimensions are unknown, but the dimensions can be encoded within the type.
2:32:08
beginner_supreme
I had that internal debate before too, but decided to stick to CL. It's just easier to work with in my opinion
2:32:56
aeth
The problem with Haskell is that you have to be inventive to keep purity. CL doesn't care about mutability so you can take your mutable algorithms and data structures with you into CL.
2:33:09
Prof_Birch
I know, the problem is I don't always know what I am thinking clearly (Hence ADHD)
2:33:34
beginner_supreme
Haskell requires placing all your mutability and imperative style into constructs called Monads, which are a bit difficult for most to grok
2:33:52
Prof_Birch
that I can put down what I am thinking, and tinker. Java and python didn't do that so well. It always ended up with me rewriting my program and making no progress
2:34:26
Prof_Birch
I like the idea of perl, but I would also like to go back and read what I wrote at some point
2:35:37
beginner_supreme
In CL, when you are writing something in a spaghetti or repetitive way, you might realize "Oh this... I can write a macro for this... to keep the code size down... and focused on what I want to write."
4:33:18
beach
jeosol: It is not reserved for Europeans. We have many participants from all over the world. US in particular, but also Japan and some other places.
4:34:48
jeosol
I have finished my challenge code so I am fully in invested in CL for now. I am planning to start presenting my work soon.
4:34:50
beach
Usually, you can wait until much later, but this time ELS is held jointly with the <programming> conference, so there will be many people who need rooms.
4:36:04
jeosol
I would never have been able to do it without CL. For example, the macro facility was exception and of CLOS which my codes uses a lot.
4:36:47
jeosol
also look forward to get jobs where I can hack CL. I almost got one with some US defense guys, but security clearance issues.
4:37:03
jeosol
thanks for the link. I will look it up and start making plans. These things go by quickly.
4:37:37
jeosol
But then, I will need EU work visa right? I will be in Barcelona for the Challenge conference, so I may try to see if I can visit them.
4:42:03
rme
I understand that Spain can be rather troublesome, but why not talk to RavenPack anyway?
4:42:09
beach
jeosol: I know of at least 5 US-of-Asians who, once installed in the EU never looked back.
4:43:59
beach
rme: Oh? That would be very surprising. Sure, if you are an unskilled person from Africa, I can imagine a few obstacles, but if you have an offer of a skilled job, I don't see the problem. But maybe you know better than I do.
4:47:30
jeosol
I normally joke that "blessed is S/he who hacks lisp for a living ...". There are a few companies there have adverstise and look for CL as part of their stack, but then they want other languages too JS, Java,...
4:47:55
jeosol
that's good actually. I will start developing some contact. Conference is in September
4:48:17
aeth
jeosol: If you have European ancestors you might actually be able to get European citizenship. Depends on the country. Italy is the easiest.
4:48:56
rme
beach: I can't quite make a move to Bordeaux (or anywhere in France) work out just on my own. I'm talking to some potential employers, though.
4:50:51
beach
jeosol: The great advantage of going to ELS is that you will be able to put a face on many of the #lisp participants.
4:51:13
aeth
Personally, I think I could get Italian (1/4) or Hungarian (1/16??) citizenship if I tried. For the latter I would have to learn Hungarian, though.
4:55:26
jeosol
beach: yes, good point. Nothing like meeting people and discussing in person and finding common grounds and collaborative projects.
4:56:24
jeosol
I did plan to come this here, write my paper on building a very large project/application with CL.
4:57:17
jeosol
when I started, my code was barely running stable, issues, but after questions here and comp.lang.lisp, it is running smoothly, and runs for days and weeks without issues now.
4:58:31
jeosol
beach: on the conference bit, I think the collaboration is something that would be useful. For example, I have talked about picking up CLML for machine learning/AI but waiting to sort out the licence issues and get a go-ahead
5:01:05
aeth
I cannot easily break 10k. Every time it looks like I might get close, a new macro pops up.
5:01:59
jeosol
There are a few reasons for the productivity, lisp being one of them. I type fast, and because I can reach any file and function from within emacs, all those things help. Also, my code is essentially a document because I use long
5:02:49
jeosol
variable names which my be hard for many. The variable names and function names convey meanings. So with the code reading like a document, it was much easier to write along.
5:04:01
jeosol
I won't recommend writing it the way I did, I was also writing to code as a hobby (April 2016), then a challenge came up Feb 2017, so I decided to make the code be able to solve that challenge. Deadling was a month ago.
5:05:01
jeosol
But I must say, I have enjoyed working on the project and can't imagine if I can get to code CL and get paid for it. hmmm
5:06:46
jeosol
but one thing I would say is that, my code quality gets better with time. When I learnt the basic things I needed, I just started writing and replacing code as I go forward and using better idioms, or techniques to improve efficiency. But my focus was
5:09:18
jeosol
I would say it is a platform because the code base contains optimization engine, statistics, proxies. So application area is for optimization problems where scare resource have to be allocated to maximize some objective function.
5:15:05
jeosol
A presentation may be able to help explain things better. Hence beach suggestion to come to ELS
5:16:43
jeosol
I was hoping to reproduce some examples using CLML library and make changes to the code if necessary. I am trying to resolve with the library owners to avoid any issues.
5:19:31
shachaf
Is there something in Lisp which is like progn, but where each statement can be a sort of macro that acts on the rest of the block?
5:21:03
shachaf
I'm thinking about a language feature for a different language. But I thought maybe Lisp people would've thought about something like this.
5:21:22
makomo
what does it mean "but where each statement ... acts on the rest of the block" though
5:22:05
makomo
maybe something along the lines of this, but i don't know if that's what you're looking for: https://fare.livejournal.com/189741.html
5:22:22
shachaf
I guess it would mean that (progn (a) (b) (c)) can turn into something like (a '(progn (b) (c))) if a is a particular type of macro.
5:23:55
shachaf
OK, that looks similar. Though I'm thinking that in the common case the things in the progn block are just evaluated as regular expressions.
5:25:15
makomo
i'm not sure if it's possible to do what you want, i.e. for A to act on the whole progn
5:25:42
makomo
you could for example invent your own PROGN-like thing that will be a macro that will codewalk the body and looks for A's :D
5:26:12
shachaf
Right, that's what I was thinking. I was wondering whether something like that existed.
5:26:51
shachaf
It's also similar to https://common-lisp.net/project/cl-monad-macros/monad-macros.htm (or Haskell's do notation)
5:28:02
shachaf
And I suppose also to continuation-passing style, or first-class continuations in languages that have them.
5:29:38
makomo
shachaf: you might want to look at the implementation of contiuations in On Lisp. i don't know how complete it is, but it could come in handy as an example
5:32:09
shachaf
But continuations are opaque functions and I'm curious about applications that do more syntactical macro-style things.
5:39:30
aeth
You could try looking at various utility libraries to see if any do something similar to what you're talking about
5:39:58
shachaf
(And also I'm thinking about this in the context of some sort of low-level language that doesn't have closures.)
5:53:28
fiddlerwoaroof
shachaf: you might look at f-exprs in the kernel language, among other places