freenode/#lisp - IRC Chatlog
Search
22:07:58
pjb
francogrex: Toaster from assembler (from scratch): https://www.youtube.com/watch?v=R3Qn98bE880 Toaster from sophisticated tools: https://www.youtube.com/watch?v=crc8n7D4kYg
22:08:16
verisimilitude
You'd need a loading program that's what's actually called. This could also contain the version and help information. This would be written in, say, sh.
22:09:14
verisimilitude
Closing, all the machine code would do is READ and one of the first things that the loading program would do is feed in the rest of the CL package before locking it. Then you have a working implementation. One thing this would permit is customizing how most of the CL package is compiled, if that's desired.
22:10:03
pjb
verisimilitude: if you're building your assembler using stuff you didn't build yourself, you're fucked. See Reflection on Trusting Trust. https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf
22:10:47
verisimilitude
Clearly, I'm eventually going to need to use an iteration of the tool to write a version of itself in machine code and then manually inspect that.
22:14:03
verisimilitude
I eventually want a 6502 version of this tool I can run on a C64, which is trustworthy enough. But that's then tangential.
22:14:18
verisimilitude
I'm glad you found it interesting, francogrex; did you refer to what I wrote here or the article I linked to?
22:14:47
francogrex
ok guys. I will save this to read it again later. verisimilitude yes i saved for reading it. thanks
22:15:34
verisimilitude
It's no issue. So, you haven't read the article yet; in that case, I anticipate your thoughts concerning that.
22:17:18
francogrex
not yet. I will and I come back here hoping to discuss it with you. I will try to understand :)
22:30:13
fiddlerwoaroof
as it turns out, most lisps do use assembler, they just don't use an external assembler...
22:31:02
fiddlerwoaroof
all the vops in sbcl, for example, bottom out in an sexp-syntax assembly language
22:42:25
pjb
verisimilitude: That said, while as an individual you might not want to invest the resources to make your own toaster, as a country we should consider making our own processors and other components. The case Huawei shows that's important. (and who would trust Intel apart from the US government and companies?)
4:09:49
beach
Writing a Common Lisp system in assembler? It would have been funny if it were a joke, but I fear it wasn't.
4:11:16
no-defun-allowed
Some people wonder why there are no (large) Common Lisp systems written in C, or if they are educated enough, why ECL is the only Common Lisp in C.
4:12:08
no-defun-allowed
I think people expect other languages to be bootstrappable from a C compiler, but most languages are implemented in themselves once they're developed enough.
4:14:52
diip
I am quite the opposite, I wonder why most CL implementations must have C in them and don't seem to be able to interact with the hardware directly, I know of clozure and SBCL, but even clozure has some C in it.
4:15:21
beach
The other is a conviction that higher level languages must somehow be written in lower-level languages.
4:15:31
no-defun-allowed
So, I suppose I will tell them to compile ECL or CLISP and use that to bootstrap SBCL.
4:16:33
no-defun-allowed
I imagine in some places of SBCL and other implementations, it would be terrible for the programmer to accidentally cons or do any mutations to the Lisp image's state for some reason.
4:17:41
no-defun-allowed
I could also imagine an internal special form or declaration that refuses to compile any consing code. It'd be like assembler with nested parens.
4:19:37
rme
been hacking CL for RavenPack since july and working on Opusmodus and CCL in my copious spare time
4:20:09
diip
beach: I meant to do anything hardware related, you seem to have to go through the CFFI and generally end up using C rather than straight to the hardware itself (or assembly)
4:21:19
djeis[m]
I mean, strictly speaking the CFFI is no more CL than the hardware access something like Genera or Mezzano provides.
4:23:00
diip
I am still very much learning though but there is little to no documentation on hardware interaction with CL, it seems implementation specific and generally not advertised by the implementations themselves
4:24:01
beach
diip: It not only SEEMS implementation specific. It IS, because there is nothing like that defined in the standard, just like with most other languages.
4:24:10
djeis[m]
diip: It is entirely implementation specific, CFFI just abstracts over the differences in many implementations API's for C.
4:25:08
no-defun-allowed
When it's provided, I think it'd be hidden quite well or regulated so the Lisp image doesn't destroy itself.
4:26:14
diip
beach: nothing like "hardware interaction" defined in other languages. some languages (C inline ) do
4:26:23
rme
RavenPack would like it if I moved to Spain, but I don't know Spanish and the kids are too old now to just go to Spanish public schools and pick up the language by immersion.
4:27:09
diip
no-defun-allowed: I think this image part is where I get lost, how would using assembly affect the lisp image?
4:27:56
djeis[m]
It's more that direct hardware access gives you the tools to royally shoot yourself in the foot.
4:27:59
no-defun-allowed
Assembly has very little protection, and can be used to harm a Lisp system.
4:28:33
no-defun-allowed
diip: One simple experiment is to write a function with (declaim (optimize (speed 3) (safety 0))) and some type declarations, and then funcall it with different types.
4:29:02
djeis[m]
I think it's just that there aren't a lot of people that want to do that sort of thing, so the interface isn't well documented for the vast majority of implementations.
4:29:28
no-defun-allowed
You will quickly encounter strange values, and objects which the Lisp system doesn't understand. Often, the garbage objects will cause the garbage collector and printer to error.
4:29:32
djeis[m]
Telling SBCL to generate arbitrary ASM is actually relatively (relatively) straightforward when you've done it a couple times.
4:30:56
rme
in CCL, we implement certain functions in asm, e.g., https://github.com/Clozure/ccl/blob/master/level-0/X86/x86-pred.lisp#L33 but as you say, it is extremely easy to mess up.
4:31:02
djeis[m]
And the interface for it is only meant to be used by people with knowledge of the compiler internals, so the docs for it aren't written to be user-facing and are also relatively sparse.
4:34:00
diip
djeis[m]: I agree, but that assumes that the lisp compiler make use of all the optimisations of the chip (which for x86_64 I find hard to believe)
4:35:27
djeis[m]
Are you allowing for SSE? because SBCL can make use of those, although admittedly I haven't seen it generate them on its own.
4:35:32
fiddlerwoaroof
Part of the problem is that oftentimes the "below-C" level is an implementation detail
4:35:52
fiddlerwoaroof
e.g. on OSX, a lot of the stable interfaces are defined in terms of C or Objective-C
4:36:25
fiddlerwoaroof
If you work below that level, you have a lot more maintenance work to do every time Apple releases an update.
4:36:42
no-defun-allowed
diip: You'll need to write your own function which requires specific types. One good, simple function just adds two fixnums.
4:36:49
beach
I personally think that if your goal is to turn Common Lisp into C or assembler, you might as well use C or assembler in the first place.
4:37:17
fiddlerwoaroof
Well, I would generally like to write CL applications that "fit in" with the rest of the system.
4:38:47
djeis[m]
If you just want to hand optimize a particular function for use cases involving very specific types and you know how SBCL has laid those types out in memory, go for it and write an SBCL VOP and teach the compiler the hand written assembly for your problem. But that sort of thing is well beyond diminishing returns for a lot of use cases- the compiler is very clever, especially given the type annotations you'd need for that
4:39:47
fiddlerwoaroof
Also, from my fiddling with the objective-c runtime and the various APIs exposed on a Mac, my general impression is that the technologies underlying OSX are much nicer than the equivalent layer on Linux
4:39:55
diip
beach: don't like C and I do sometimes use assembler, it is just a bit tedious. but I am not trying to turn CL into anything else, I just wondered why.
4:40:14
fiddlerwoaroof
They're quite inspired by Smalltalk and so they feel more "lispy" when you integrate with them.
4:41:14
fiddlerwoaroof
diip: if you're on sbcl, you might also look at pvk.ca and/or figure out how to use sbcl's built-in assembler
4:41:49
beach
diip: I used properly, Common Lisp is a language that is much safer than assembler, C or C++. That is one of its major strengths to me. By "turning Common Lisp into C", I mean making Common Lisp as unsafe as C, which is taking away one of its main advantages.
4:43:40
diip
fiddlerwoaroof: I am thinking of that but like I mention the documentation is a bit evasive, I have only heard of rumours online which point to read the source code eventually
4:44:21
djeis[m]
There are a couple blogposts on that site which walk through the process of writing and interacting with VOPs.
4:45:26
djeis[m]
Past that yea, you basically have to read the source. There are some manuals for the CMUCL compiler and the architecture/terminology is pretty similar to SBCL, so those are helpful.
4:47:13
diip
beach: I would like to do some unsafe stuff from time to time even if just to find out why it is unsafe, I like assembler for that I just which I could experiment within the CL environment.
4:48:20
djeis[m]
Did you look at this one in particular: https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-code-breadboard/ ?
4:50:23
djeis[m]
I'd suggest going back to it every now and then as you learn more of the higher level terminology and such, it actually covers a lot.
4:51:22
djeis[m]
There are SBCL-specific calls unrelated to ASM that you're kinda just expected to know about in order to follow along.
4:52:27
djeis[m]
I'm not honestly sure how similar they are at the line-by-line code level at this point.
4:56:38
djeis[m]
I'm happy to answer particular questions to the best of my own knowledge, and that's also a good way for us both to arrive at right answers as someone more knowledgeable jumps in to correct me.
5:06:05
no-defun-allowed
If all my threads are waiting on mutexes (with bt:make-lock mutexes) for some reason, is it likely that would appear as high CPU usage?
5:07:56
no-defun-allowed
My parallel program sometimes stops and I can't tell why. I think there are mutexes halting everything.
5:11:17
no-defun-allowed
Well, with these pauses there's little benefit to parallelism. It just appears to go very, very slowly after a while.
5:12:11
no-defun-allowed
Maybe it is for a while. The one lock I can think of that's important is usually free though.
5:13:14
no-defun-allowed
It could also be garbage collecting, actually, but the pauses are very, very long for a GC.
5:14:26
Jachy
For deadlock to occur you need to have mutual exclusion, lock & wait (once a thread has a resource, it won't let it go until it's acquired all the others and done its work), no preemption, and circular wait (T1 needs R1 and R2, has R1, waiting on R2. T2 also needs R1 and R2, has R2, waiting on R1.)
5:14:40
Jachy
If you can rule out any of those then you're ok as far as deadlock is concerned, though might have other problems.
5:15:33
no-defun-allowed
Yeah, I think there wouldn't be any circular waits since the threads will only ever wait on two independent resources, the statistics output and the database.
7:15:04
no-defun-allowed
So....I think the issue with my parallel program was that the cache was being overwritten by many differenct processes working on different parts of the computation, so I would need to make the cache larger to accomodate.
9:26:21
no-defun-allowed
what about the #.(tagbody :loop (unwind-protect (loop) (go :loop))) license?