freenode/#clasp - IRC Chatlog
Search
3:37:55
Bike
http://wingolog.org/archives/2014/01/19/elf-in-guile wingo can be informative even when they're not around
4:24:38
Bike
maybe we could do some junk to dump out elf with compiled functions in it, for dispatchers.
8:15:53
beach
The global allocator has two parts, namely a list of 2-word headers (or CONS cells) and a traditional heap as used by malloc() and free().
9:07:23
beach
Doug Lea's allocator also has some features that can be eliminated when used together with a garbage collector. In particular, caching of chunks, hoping that the same chunk will be allocated again soon, is of no use.
10:05:52
beach
This is getting interesting. Doug Lea's allocator is designed to handle arbitrary sequences of calls to malloc() and free(). I wonder how the design might change when used with a garbage collector. In such a context, there will be a large number of calls to malloc() with no calls to free(), followed by a large number of calls to free() with no calls to malloc().
10:18:47
Shinmera
I assume it's not a copying one then, though? Since with copying you don't really need to free anything.
10:19:59
beach
The per-thread GC is a sliding collector. Recently, I have become convinced that the global GC should be managed as combination of a mark-and-sweep collector for the headers and as a malloc/free heap for the racks.
10:20:25
beach
For example, ignoring caching, when there is a call to free(), the preceding and/or following chunk might be free. In that case, the chunk about to be freed must be coalesced with its neighbor(s) and those neighbor(s) must be unlinked from their respective free lists. But when there are several calls to free(), then one might be able to coalesce several adjacent blocks and only unlink at the end of the entire operation.
10:31:17
Shinmera
I don't have any experience implementing or testing GCs for their viability so I can't really say anything about it.
10:33:14
beach
Most systems require much more complicated memory management, though. The two-word mandatory header of all SICL objects makes things a lot simpler.
13:00:30
drmeister
beach: A two-word manditory header of all SICL objects? I thought it was just one word - the stamp.
13:18:09
frgo
But how does one interface from SICL (or Lisp) to malloc() or free() without reaching out to C?
13:20:11
Bike
malloc and free are way more complicated, which is why they're library functions, and why beach can say things like "i want to use dlmalloc specifically"
13:30:03
frgo
(Showing of my lacking knowledge here) Ok so I use mmap to do what? Map a file into memory? or use an anonymous virtual region? Then what? I don't have read or write functions in Lisp to access that memory region - or do I?
13:30:46
Bike
you use an anonymous mapping to get memory from the OS. then you use some primitives to deal with the memory directly.
14:16:49
drmeister
So - I've been toying with the idea of generating webassembly from Clasp. Sadly - it won't work. While C++ can be compiled to webassembly, garbage collection isn't easy to achieve within the webassembly runtime.
14:17:46
stassats
i wanted to do something with webassembly, but it's poorly documented and requires a ton of support javascript code
14:20:14
drmeister
The machine registers aren't pushed onto the C++ stack. They are on a shadow stack that is inaccessible to the C++ code.
14:21:26
drmeister
So I couldn't ship MPS with the code because it can't scan the stack conservatively and find roots in registers
14:22:00
drmeister
webassembly doesn't have registers - but the machine running webassembly has registers.
14:24:08
drmeister
If the compiler optimizes some variable out of the stack frame and into a register - then the GC won't see it. I was hoping you would challenge this.
14:25:14
drmeister
You don't see a problem with that? I worried that roots could disappear into registers.
14:27:04
drmeister
Here's my thinking. I thought I could compile C++ and CL generated bitcode to webassembly. This would include the MPS garbage collector - which is written in C and compiled into Clasp.
14:27:43
drmeister
Bike: I talked with a couple developers yesterday on the #webassembly IRC channel. I posted something this morning to the webassembly/gc discussion board on github.
14:28:21
drmeister
The webassembly gc page goes into a lot of detail of what they want - everything that I already have with MPS.
14:28:23
stassats
drmeister: if the code is optimized which breaks the semantics of a stack machine, then there would be problems
14:29:39
drmeister
(a) There would be problems with conservative scanning of the stack for garbage collection - or (b) there would be problems in general that guarantees that they won't do optimizations that break the semantics of a stack machine?
14:32:56
drmeister
Shinmera: If it's easy - I think it would be a good way to disseminate it - it would also allow me to do things that require me to work with Javascript now.
14:33:20
Shinmera
you don't seriously think you can run Clasp, a 50mb binary or what, implemented in basically JS, in your browser?
14:35:39
drmeister
Shinmera: It's a bit of a lark at the moment - what is a reasonable sized binary for running in a browser?
14:36:06
stassats
my cpu has 168 registers, with only 16 exposed to the user, so it waits for all the in flight operations to complete when you access all of them, wasm could have something like that too
14:36:18
Shinmera
If you want to just run it, I don't know. I can't imagine anything wasm to be really fast if it's that fucking huge because browsers are garbage at memory.
14:39:52
drmeister
Shinmera: It's part of my (unwritten) job description not to be reasonable at times.
14:41:29
drmeister
I was thinking - wasm looks like a great and up and coming way to deliver web applications. It's getting more capable (exception handling, threads, memory page protection, garbage collection). It's llvm based.
14:42:28
drmeister
We have a Common Lisp that has a C++ runtime and uses llvm and could generate wasm.
14:43:05
Shinmera
drmeister: I just don't see this working now, or in the next three years. Don't waste your breath on this at the moment.
14:43:06
drmeister
I'm not wasting time on it. I've been finding people to ask questions while things compile for the last two days.
14:43:23
Bike
if i clicked on a webpage and it started downloading 250 MB of binary i'd probably close the tab, or more likely try to close the tab while my browser freezes
14:44:03
drmeister
The 250MB binary is the current state of Clasp - we have spent zero time optimizing size.
14:44:07
Shinmera
Bike: Even if my browser just downloaded 1mb of JS but then maxed my CPU trying to run "(print :hello)" I'd still close it.
14:44:34
Bike
clasp's present design is a big monolithic system, like most lisps. moving away from that would be interesting, but "nontrivial" doesn't begin to cover it
14:46:27
Bike
anyway, i've been sketching a manual. is the C++ integration documented anywhere already or do we just refer people to the bullet demo?
14:47:31
Bike
i can document the mop and environment extensions (also, i should export the latter a bit), but i don't know gray streams, sockets, or c++
14:47:38
frgo
Well, having a small MAIN module being the executable and separating the rest of clasp into a shared library is not that hard. I'd love to see C++ Binding, GC interfacing, CLOS, etc being in separate shared libs. We could then use the loader to handle init at load.
14:49:20
Bike
but i'd rather not copy a manual that doesn't actually apply, and i don't know how much we've changed from ecl in those modules (though i'd guess not much). lying manuals are bad
14:50:25
frgo
stassats: Yes - but only as long as you are doing clasp core development. Whenb done, create an optimized image and load this.
14:52:58
drmeister
I used luabind as the template when I developed clbind - I think I also stole code.
14:53:19
drmeister
The developers of luabind were fine with us taking their documentation and modifying it to describe clbind.
14:54:31
drmeister
Understood. The clbind-doc is the closest thing. The luabind documentation is more like my springboard to fill in the gaps of the clbind-doc.
14:55:20
Bike
this is like, the main feature. if we want users we should document it so if somebody goes "i want to use opencv" or whatever we can just point them at it rather than walking through the whole thing based on oral tradition
15:01:15
Bike
i know it's yet more to do, but we should probably act like clasp is actually useful rather than perpetually coming soon. then we can fake it til we make it
15:05:31
drmeister
The build system has been a major barrier for me for releasing it. I also kinda wanted to get a minimum set of features that Common Lisp users expect (CFFI, threading, Slime, quicklisp) and MPS. We've pretty much ticked off all the boxes.
15:06:32
Bike
sure, there's lots of problems. but even if we don't release we should adopt a kind of release mindset, i think.
15:12:13
frgo
Nope. Too many changes in clasp and I am unable to follow at that pace with changes. Or I just don't understand.
15:12:23
drmeister
Those may still be broken - I don't recall the fleeting feeling of satisfaction that comes with fixing something related to callbacks.
15:14:40
drmeister
I have an important DOD meeting in just over a week and then I can push for a release.
15:26:05
Shinmera
It's so that the kids in this channel are saved from being exposed to swear words when they work on clasp.
15:28:17
Shinmera
Ahahaha, it's all me https://www.google.ch/search?q=site%3Airclog.tymoon.eu+clasp+"fucking"&oq=site%3Airclog.tymoon.eu+clasp+"fucking"
15:28:49
Bike
https://www.google.ch/search?q=site%3Airclog.tymoon.eu+clasp+"fucking"&oq=site%3Airclog.tymoon.eu+clasp+"fucking"