Search
Friday, 9th of November 2018, 10:50:49 UTC
12:23:31
stassats
thinking about w^x, i don't think we can do it without either stopping the world or each new code object occupying a separate page
14:26:45
stassats`
the latest substitute-single-use-lvar is caused by it working on a yet uninlined local function
14:26:57
stassats`
so it doesn't see the where the value is actually going when inlined
14:27:58
pfdietz
Minimizing that was annpying, as I had to do it by hand.
14:29:18
stassats`
i might just give up and only transform (let ((x ...)) x) to ..., but i wanted it to be general
14:29:38
ebrasca
How is going port of sbcl for ppc64?
14:30:37
ebrasca
stassats`: I don't undestand what do you mean with "nohow".
14:35:38
sjl_
nohow : How is ... going? :: nowhere : Where is ... going?
14:37:37
ebrasca
Can sbcl run on ppc64le?
14:39:28
pfdietz
I think there's a lack of developer bandwidth outside x86-64 (and 32?) and ARM64.
14:40:17
stassats
only ppc64be, via ppc32
14:42:17
ebrasca
How I can integrate fat32 suport to sbcl? ( It currently work in Mezzano OS )
14:42:46
ebrasca
Here link of my fat32 https://github.com/ebrasca/Mezzano/blob/master/file/fat32.lisp
14:43:08
stassats
fat32 is not something sbcl needs
14:45:50
ebrasca
Is it better for my fat32 to ignore other common lisps?
14:47:10
stassats
other common lisps are not trying to be operating systems
14:54:15
stassats
ok, i'm giving up on smart substitute-single-use-lvar for returns and exits, even though i now can handle anything, except for inlining, since it can produce arbitrary code
14:54:25
stassats
whether the thing itself is inlined, or something inside of it is inlined
14:54:58
stassats
but, i gather a bit of tests and a better understanding, it's really not an important optimization
14:55:17
ebrasca
Can sbcl compile in parallel?
14:58:15
stassats
but i still get (defun foo (z x) (position z (the simple-bit-vector x))) optimized, which was the target
14:59:30
stassats
pfdietz: i accidentally managed to make a crashing test care, so you just got a bit unlucky
15:03:56
stassats
i could probably predicate it on is-everything-inlined, but that's not something already available or worth it
15:04:00
pfdietz
Wondering why ignore-errors is defining those functions (looking at the macroexpansion).
15:04:28
pfdietz
I assume one or both FLET functions are inlined.
15:04:41
pfdietz
The lambda in handler-case is also?
15:05:08
stassats
no idea why the protected form gets its own function
15:05:39
stassats
handler-bind won't get inlined, it'll go to a dynamic-extent list
15:05:51
stassats
it's turned into a flet cause we can't dx lambdas
15:06:11
pfdietz
But why functions at all?
15:07:42
stassats
well, the handlers need to have dynamic-extent and be called non-locally, where would you inline that?
15:08:10
stassats
the protected form? no idea, but it's not like anybody is experiencing slow downs due to excessive compiler work on handler-bind
15:09:10
pfdietz
Each is called in just one place, yes?
15:10:26
pfdietz
I am looking at (macroexpand '(ignore-errors (foo)))
15:10:41
pfdietz
There are two functions defined in the outer FLET form. Each is called in just one place.
15:11:54
stassats
my previous message is applicable to that
15:14:02
pfdietz
I guess the declaration (DECLARE (OPTIMIZE (SB-C::CHECK-TAG-EXISTENCE 0))) is relevant; may not want it to apply to the user code.
15:14:27
pfdietz
So that user code gets put into flet functions, which are out of the declaration's scope.
16:46:14
stassats
the new wxallowed detection works
16:48:36
stassats
is it really newsworthy? now, if it actually worked with w^x
16:49:06
joshe
I didn't think a NEWS entry was worthwhile, it wasn't even a build fix
16:49:38
stassats
default w^x would probably be not a good idea, but optionally it's somewhat trivially doable
16:50:00
stassats
always allocate on new pages, the GC will coalesce them laer
16:50:10
Xof
we get flak on reddit for a trend decrease in the number of NEWS items
16:50:40
Xof
I realise that this might not be the best reason for an extravagantly detailed NEWS
16:50:49
stassats
well, right now lisp is getting flak on reddit for PRINTING IN UPPERCASE
16:51:10
Xof
slightly more importantly, having this kind of thing in NEWS sends the signal that yes, we do care (a little bit) about things being broken
16:52:19
stassats
if we're testing for w^x, why not for noexec too?
16:52:21
pfdietz
More NEWS entries, especially if they boost my ego.
16:52:36
Xof
at least to the extent of telling people that things are about to be broken
16:52:41
joshe
/home isn't noexec in the default install
16:52:54
stassats
well, not limited to openbsd
16:53:22
joshe
and I'm not sure adding detection for an openbsd feature added two years ago really sends a message that someone really cares ;)
16:53:53
joshe
but sure, I can add an entry
16:54:00
pfdietz
Thinking "fix committed" is not enough state; should indicate if the bug had been present at the last release. Short term bug fixes of things that were never released do not belong in the release notes.
16:54:19
stassats
noexec fails really early, so, no need
16:54:32
stassats
make-config.sh: 263: make-config.sh: ./generate-version.sh: Permission denied
16:56:10
stassats
i have some other stuff to work on, otherwise i would've tried my hand at w^x
16:57:14
joshe
I tried to follow what was going on but quickly got lost
17:00:23
stassats
i think the simplest thing to do is just to inflate the code object size at creation time to be a multiple of pagesize, then reprotect when gcing
17:00:40
stassats
and probably reconstituting sanctify-for-execution on x86oids
17:01:35
stassats
it does sound so simple i might actually try doing it
17:07:32
pkhuong
foreign callbacks will become even heftier
17:08:03
stassats
well, it's optional, and openbsd doesn't mind performance degradation for performance reasons
17:09:40
joshe
pkhuong: become heftier as in use more virtual address space?
20:36:51
asarch
What went wrong?: http://paste.scsys.co.uk/582188
20:38:19
stassats
scratch that, that's not a usual sb-concurrency test failure
20:40:41
joshe
make: *** [../asdf-module.mk:41: test] Illegal instruction (core dumped)
20:41:18
stassats
seen that reported, but it builds fine here
20:41:39
stassats
i unscrewed my virtualbox and installed 6.4
20:42:03
joshe
the sb-concurrency tests frequently fail on openbsd but I don't recall how
20:43:13
stassats
the frequently fail on risc
20:43:22
stassats
but not by doing illegal things
20:43:50
joshe
there are a few timing problems in various tests caused by openbsd's course scheduler granularity
20:54:57
stassats
it needs to be reproduced by me or joshe
20:55:23
stassats
or you need to send the core dump and the binary
20:55:51
stassats
/cores? or your `pwd`?
20:56:57
asarch
Ok, then tar vcf core-dump.tar sbcl-1.4.13/
20:57:14
asarch
And then gzip --verbose --best core-dump.tar
20:57:22
asarch
How could I send you? By email?
20:57:31
stassats
i would only need the dump and the src/runtime/sbcl file
20:58:32
asarch
find . -type d -name cores <- Finds nothing
20:58:45
asarch
find . -type f -name sbcl: ./src/runtime/sbcl
20:58:59
stassats
well, it's not named "cores"
20:59:07
pkhuong
asarch: you're looking for a file, probably named "core" or "core.$PID"
20:59:53
asarch
I found /output: after-xc.core, cold-sbcl.core and sbcl.core
21:00:04
asarch
./contrib/sb-concurrency/sbcl.core
21:00:19
pkhuong
asarch: you want the actual core dumped by the OS when the process crashed.
21:00:57
asarch
http://paste.scsys.co.uk/582189
21:01:06
asarch
That is all the core I found
21:01:14
stassats
./contrib/sb-concurrency/sbcl.core is the one
21:04:23
asarch
The command line I used was: ./make.sh --fancy --prefix=$HOME/bin/sbcl
21:04:50
stassats
oh, i did not build with threds
21:04:58
stassats
just hold on, don't send me anything yet
21:05:14
asarch
From dmesg: http://paste.scsys.co.uk/582190
21:05:20
asarch
Do you want the full dmesg output?
21:05:51
asarch
ACTION waits for new instructions...
21:07:13
stassats
that's an atom, but nothing about it is out of the ordinary
21:09:13
stassats
sb-concurrency tests are kinda slow on openbsd, maybe i need to adjust the cpu number detector, but no sigills yet
21:10:05
stassats
asarch: ok, i can't reproduce it now either, so, i'll need that core dump after all
21:12:33
asarch
Ok, how could I send it to you?
21:14:24
asarch
Ok. Warte bitte because I am from Mexico...
21:14:30
joshe
I can reproduce on real hardware
21:14:47
stassats
joshe: can you tell me which instruction it is then?
21:15:08
stassats
asarch: they speak german in mexico?
21:16:15
stassats
i've been thinking the other day, why don't we catch sigill
21:16:47
joshe
huh, gdb thinks it's in futex()
21:17:00
joshe
0x00000002f137f510 <futex+0>: mov $0x53,%eax
21:17:00
joshe
0x00000002f137f515 <futex+5>: mov %rcx,%r10
21:17:01
joshe
0x00000002f137f518 <futex+8>: syscall
21:17:01
joshe
0x00000002f137f51a <futex+10>: retq
21:17:59
joshe
oh nevermind, $pc is futex+10 which is into the trapsled
21:18:26
joshe
I don't think sbcl uses them directly, but via the pthread_* functions, sure
21:18:43
stassats
do you have a backtrace?
21:18:44
joshe
http://man.openbsd.org/futex
21:19:19
joshe
bad stack pointer, maybe:
21:19:19
joshe
Cannot access memory at address 0x2f9c379d8
21:19:26
stassats
if openbsd does have futex then sbcl should use them, because the non-futex path is really horrible
21:19:51
pkhuong
joshe: is 6.2 old enough to assume no one runs older releases?
21:20:26
joshe
6.2 isn't supported by openbsd anymore, but some people may run it
21:20:30
stassats
pkhuong: it could be make-config.sh detected, i don't think openbsd is big on binary compatibility across versions anyway
21:20:49
joshe
yea, libc is already bumped with every 6-month release
21:21:08
joshe
the libc major version, that is
21:21:16
stassats
but i'd rather be improving the mutexes on darwin, not openbsd
21:21:20
stassats
(cause that's what i use)
21:22:04
pkhuong
stassats: I honestly think the old futex emulation code was the right approach... too bad reentrancy was so hard.
21:23:31
joshe
https://gist.githubusercontent.com/jre/90bce0519cc928859282c36c7ba65494/raw/aba75d89ead5f9861371f42e35e96e51749f4379/gistfile1.txt
21:24:49
pkhuong
joshe: can you print the instruction bytes?
21:26:28
joshe
0x2f137f510 <futex>: 0xb8 0x53 0x00 0x00 0x00 0x49 0x89 0xca
21:26:28
joshe
0x2f137f518 <futex+8>: 0x0f 0x05 0xc3 0xcc 0xcc 0xcc 0xcc 0xcc
21:27:22
joshe
http://ref.x86asm.net/coder32.html#xC3
21:28:05
joshe
I suppose that should be coder64, but the opcode is the same either way
21:32:00
stassats
or what was the option
21:32:29
pkhuong
joshe: any other thread in that core?
21:32:47
stassats
oh yeah, it's sb-concurrency
21:33:28
joshe
oh, 12 other threads in _thread_sys_nanosleep
21:35:20
joshe
I updated that paste with disassemble /r and info threads
21:35:59
stassats
you didn't link the paste, though
21:36:23
joshe
my bad https://gist.github.com/jre/90bce0519cc928859282c36c7ba65494
21:38:32
stassats
it's all in os code, so, what gives?
21:38:36
asarch
Maybe at the spam folder...
21:41:55
stassats
ok, sigill does land in ldb
21:42:41
stassats
so, a sigill in a foreign thread would not be caught
21:43:26
stassats
i only have 1 core in my vm, let's increase that
21:46:29
asarch
ACTION whispers: "Damn Hotmail!"
21:47:15
stassats
no failure with two cores, but it appears to be even slower
21:52:27
stassats
but openbsd still thinks there's one core
21:52:57
joshe
I see what you mean about being used to the old host-2 output
21:53:07
joshe
I keep thinking it's about to dump the cold core
21:53:41
stassats
when you stare at the same thing for over a decade you get used to it
22:01:41
asarch
Gotcha!: "Blocked for security reasons!"
22:02:21
stassats
can't be sending illegal instructions
22:07:15
joshe
I wonder if the kernel would send SIGILL for other reasons
22:08:04
joshe
oh, I think that's how something with an invalid stack pointer is killed
22:08:35
stassats
why is it surfacing with multiple threads only?
22:08:36
joshe
outside of a region with a special mmap flag
22:08:49
stassats
why doesn't it happen to me?
22:09:46
stassats
do i need an smp kernel to get multiple cores or something?
22:10:37
stassats
i had only one core during installation
22:10:49
asarch_
This is the link from Dropbox: https://www.dropbox.com/s/wcwohmr0km2sdpv/debug.tar.gz?dl=0
22:11:00
joshe
oh, yes if you added cores then you need to change kernels
22:11:32
joshe
mv /bsd /bsd.sp && mv /bsd.mp /bsd
22:12:17
joshe
which is all the installer would do on the next upgrade anyway
22:12:44
stassats
ok, now i need to find bsd.mp
22:12:57
joshe
oh right, it wasn't installed
22:13:35
stassats
i can just download it
22:15:16
joshe
ftp http://cdn.openbsd.org/pub/OpenBSD/$(uname -r)/$(machine)/{bsd.mp,SHA256.sig}
22:15:18
joshe
signify -C -p /etc/signify/openbsd-$(uname -r | tr -d .)-base.pub -x SHA25.sig bsd.mp
22:22:09
joshe
if I'm reading this right, you get killed with an uncatchable SIGILL if the kernel can't write to your stack while delivering a signal
22:22:54
joshe
so where'd the bad RSP value come from?
22:23:00
stassats
that shouldn't be the case, it can write to the stack
22:24:03
joshe
gdb shows no memory mapped where RSP points in the core dump I have
22:25:07
stassats
1 is the main thread, we move the stack without informing the os
22:25:10
joshe
also, you can pkg_add gdb as root to get a less-old gdb installed as 'egdb'
22:31:54
stassats
and it's non deterministic
22:38:55
stassats
well, $rsp is kinda not where the control stack is supposed to be, but it may be just sigaltstack
22:41:16
stassats
but it has no trouble receiving signals
22:45:11
stassats
well, if $rsp is not even dump
22:45:41
stassats
i actually see some output in the console
22:46:20
stassats
trap [sbcl]47135/509372 type 6: sp 220f3fdf8 not inside 220d50000-220f40000 |
22:46:20
stassats
trap [sbcl]39758/313918 type 6: sp 244f77178 not inside 244d88000-244f78000
22:49:38
stassats
(< #X244d88000 #x244f77178 #x244f78000) => T
Friday, 9th of November 2018, 22:50:49 UTC