Search
Sunday, 9th of February 2020, 18:45:13 UTC
18:47:58
drmeister
Bike: Where did generated-encodings.lsp come from?
18:48:21
Bike
um, maybe kpoeck added it?
18:49:06
Bike
https://github.com/clasp-developers/clasp/pull/862 yes
18:54:11
drmeister
I'm wondering how it was generated. It's slow to compile because it builds code to build hashtables.
18:59:47
kpoeck
did generate the code from ecl
18:59:50
drmeister
Hey kpoeck. How did you generate it?
19:03:10
kpoeck
with https://github.com/clasp-developers/clasp/blob/dev/src/lisp/kernel/lsp/encodings.lsp#L150
19:03:58
drmeister
It looks like Clasp doesn't compile long lists very well. That might be a file that we can rearrange to compiler better if we have the original generator.
19:04:34
kpoeck
the generator is in the source I just pasted
19:05:31
kpoeck
Less than 100 lines, easy to change
19:06:44
kpoeck
If you tell me what to avoid, I can change the generator
19:07:06
drmeister
What do you think - could we use the generator instead of the product?
19:07:26
Bike
meaning generating the encodings during every startup?
19:07:52
kpoeck
I'd need a translation drmeister <-> mere mortals
19:07:57
drmeister
Would that be expensive?
19:08:49
drmeister
kpoeck: I mean you have a generator that generates a bunch of code that clasp then compiles.
19:09:22
kpoeck
remember it generates from ecl
19:10:24
kpoeck
If the list "mappings" is too long, I could do in smaller chunks
19:11:36
drmeister
Smaller chunks might be faster to compile.
19:12:00
drmeister
But I was thinking of running whatever code generates the load-time-value list of list of lists.
19:14:52
Bike
could we do a vector instead?
19:15:58
drmeister
Hmm, do we compile vectors better?
19:16:52
Bike
maybe? we don't need to worry about circularity and stuff quite as much
19:17:15
kpoeck
Could generates vectors, no prob
19:17:18
Bike
i'm not sure what to do about compiling lists any faster cos we're already technically doing it wrong
19:17:38
drmeister
How are we doing it wrong?
19:18:34
Bike
https://github.com/clasp-developers/clasp/issues/737
19:23:38
drmeister
kpoeck: If you are interested in trying this - try it as a vector if integers
19:24:11
drmeister
Say #(0 0 1 1 2 2 3 3 4 4 ...) -> (0 (code-char 0)) ... (1 (code-char 1))
19:24:20
kpoeck
ok, will do (but first measure how long it compiles to have a baseline
19:24:25
drmeister
If you don't have time - I'll give it a try in an hour or so.
19:24:27
Bike
althrough wait, how is compiling literals even a problem now? don't we just generate a huge bytevector and a call to the literal VM thingie?
19:24:49
Bike
i mean, we do our own processing, but it shouldn't be llvm time
19:24:56
Bike
is it the load-time-values?
19:25:03
drmeister
I just noticed a long delay at the end of compiling cclasp and checked and it was taking a long time compiling generated-encodings.lsp .
19:25:13
drmeister
That's why I brought it up.
19:25:31
drmeister
Other slow compilation files like inline.lisp and fli.lsp I move up earlier.
19:25:38
drmeister
I could do this with generated-encodings.lsp
19:26:05
Bike
ok i see, the function does (let ((mappings (load-time-value ...))) ...)
19:26:12
drmeister
It's just that the generator generates the worst kind of large data generating source file that it could.
19:26:21
Bike
and then... builds a hash table outside of the ltv list... that's kind of weird too
19:27:05
Bike
i forget, can we dump hash tables directly?
19:27:22
drmeister
Yes - I think we can.
19:27:48
Bike
we could just dump a literal hash table then
19:28:05
Bike
i'm not s ure how this function is used exactly, but basically it takes a constant list and builds a hash table out of it when called
19:28:20
Bike
actually, why is it even (load-time-value (list ...))? can we not just use a quoted list?
19:28:32
Bike
because it checks code-char at runtime... hrm...
19:28:50
Bike
some more abstract thought may improve this
19:32:24
kpoeck
for the encodings to work, I need the hash-table
19:33:07
kpoeck
I can drop the load-time-value
19:34:31
Bike
well what i mean is like, is this function going to be called more than once for the same encoding? the hash tables are effectively constant, i would imagine
19:35:16
kpoeck
hash tables are constant, could precalculate
19:36:05
Bike
i don't know how generate-encoding-hashtable is called or anything see
19:38:55
Bike
where is that called from?
19:40:15
Bike
github search is not cooperating.
19:40:47
kpoeck
might be from c++, let me check
19:41:09
kpoeck
For sure it is call from asdf
19:41:29
kpoeck
modules/asdf/ext/asdf-encodings/encodings.lisp
19:42:34
kpoeck
format = eval::funcall(ext::_sym_make_encoding, format);
19:44:37
kpoeck
https://github.com/clasp-developers/clasp/blob/dev/src/core/lispStream.cc#L3286
19:46:43
Bike
so we're building a new hash table every time parse_external_format is called?
19:49:00
kpoeck
can obviously be improved by caching since the tables are constant
19:50:11
kpoeck
like creating a variable with a alist of enconding-names and their hash-tables
19:50:24
kpoeck
encoding-name and hash-tables
19:53:40
kpoeck
(time (compile-file "/Users/karstenpoeck/lisp/compiler/clasp-karsten/src/lisp/kernel/lsp/generated-encodings.lsp")) -> 39.741 secs
19:54:49
kpoeck
on my 8 year old machine
20:15:15
Bike
kpoeck: any way we can do those defstruct/etc tests without digging through macroexpansions? i mean that's less a test of behavior and more a test that we haven't changed the code lately
20:30:17
kpoeck
If there is another way to check that an accessor is redefined?
20:31:26
kpoeck
fboundp will not check that (perhaps I could fmakunbound instead)
20:33:12
kpoeck
or check for a redefinition warning
20:33:54
Bike
should be possible to catch the warning. i think it's stable, since slime relies on it
20:34:32
kpoeck
ok, than I change the test accordingly
20:35:00
Bike
should be... cmp:redefined-function-warning, i think
20:35:36
Bike
might not work outside of compile-file. hm hm
2:20:11
drmeister
When we switch to faso mode - .o files become .faso files and .fasl files become .fasp files. Is this a good idea?
2:20:46
drmeister
An alternative is to keep the .o and .fasl extensions but internally they both get the .faso/.fasp file format.
2:20:52
drmeister
This would work better with ASDF.
2:21:10
drmeister
Alternatively, we change ASDF
2:23:54
drmeister
No - I' changes to ASDF
2:24:37
drmeister
I'm not sure if it requires changes to ASDF.
4:03:10
beach
Good morning everyone!
Monday, 10th of February 2020, 6:45:13 UTC