freenode/#clasp - IRC Chatlog
Search
20:47:53
balrog
drmeister, Bike: the merge-dev-cst branch is pulling SICL from upstream, not Bike's fork
20:50:31
Bike
the point of this merge was that it looks like it's going to take even longer to get cst working, so in the meantime we should at least unsplit the code
20:52:27
drmeister
Yes - but I believe in you - I think you can get cst working sooner rather than later.
20:54:15
Bike
what i thought we were doing is we'd do this merge, i'd spend a day or two getting it working with generate-ast again, and then it would be dev.
20:55:07
drmeister
Ok, what about plan (B) you spend some days getting inlining working and then we merge merge-dev-cst into dev?
20:55:29
Bike
i spent the last several weeks trying to get inlining working. it's not working. that's why i suggested this in the first place.
20:56:27
drmeister
We have come a long way over the last days - haven't we? Fixed lots of bugs in the inlining.
20:57:17
Bike
i don't know how many there will be, and in the meantime we're in this situation of having all our code continuing to diverge.
20:57:39
drmeister
And if we make merge-dev-cst --> dev then you still need to branch off of dev to work on inlining - right?
20:58:58
Bike
i think it should just be using generate-ast calls instead of cst-to-ast calls in translate.lisp in a few places.
20:59:29
Bike
i don't know. i don't want to do more big dumb merges and i honestly don't know how much longer it will take to get inlining working effectively.
21:08:58
Bike
unrelated topic. we know clasp spends alot of time garbage collecting, right? like in the flame graphs it takes a fair amount of time. but we don't have much information about what allocations are being done, i don't think?
21:14:59
drmeister
I added code to keep track of how many of what kind of objects you are allocation. It's like TIME - but it dumps what kind and how many objects were allocated. This would be useful now wouldn't it?
21:15:05
balrog
for the time being I'm going to see about simplifying the build instructions on Linux platforms
21:15:34
Bike
of course, since closure cells are just conses it wouldn't be helpful without a bit more info.
21:15:56
Bike
my home system is arch, and ubuntu is what we use for the docker image and also it's popular
21:16:45
drmeister
Bike: What do you mean by "since closure cells are just conses it wouldn't be helpful without a bit more info."
21:18:36
Bike
just counting cc_makeCell calls would be informative... but yeah, that might be a good idea anyway. we're burning twice as much memory for no reason.
21:19:43
Bike
anyway i think i can implement a pass to mark closure cells as stack allocatable without going through kildall.
21:20:04
drmeister
However, cons cells have no header - so they are 16 bytes. General objects have a header - and a vtable - so we should create a new pool of just closure cells.
21:20:48
Bike
and we should special case multiple-value-prog1-function and so on in the same way call-with-variable-bound is, probably, until we can do things more robustly.
21:21:58
drmeister
I'm going to start using the property lists that are stored in a slot in 'chem:matter' objects more.
21:23:11
drmeister
Things like 'stereochemical-type' An atom can have undefined stereochemistry, be achiral, prochiral or chiral.
21:24:06
drmeister
The property lists are copied when atoms are converted into structures that are like atom templates - things that can create atoms.
21:25:02
drmeister
I have this class called a ConstitutionAtom_O - it is a factory object that creates atoms.
21:25:32
drmeister
I just realized that I need to copy the stereochemical-type from Atom_O to ConstitutionAtom_O and later back again.
21:26:47
drmeister
A group of molecules that represent the same constitutional isomer have the same atoms and bonding - but can differ in stereochemistry.
21:27:57
drmeister
I create molecules with atoms all the time - but they are single instances. Cando needed something to generate molecules, residues and atoms from template objects. I call the thing that creates atoms a constitution-atom.
21:29:04
drmeister
The stereochemistry-type is a property of the atom that it will create. A ConstitutionAtom needs to know that the atom that it will create is achiral, prochiral or chiral. It doesn't keep track of what the stereochemistry is if the atom does have stereochemistry.
21:31:24
drmeister
I then figure out the stereochemical-type of every atom in every molecule - I can do that for intact molecules. I can't do it with just residues.
21:31:46
drmeister
Because a chiral center may be at the interface between two residues and residues are thus insufficient.
21:33:14
drmeister
Then I generate ConstitutionAtoms_O objects for each residue in each molecule. ConstitutionAtoms_O should probably be called ConstitutionResidue_O - it's the 'constitution' equivalent of a residue.
21:33:47
drmeister
The ConstitutionAtoms_O object contains a bunch of ConstitutionAtom_O objects that store information that can later be used to build Atom_O objects.
21:35:01
drmeister
But I haven't been copying the stereochemistry-type information from Atom_O to ConstitutionAtom_O so that I can use it later to generate an Atom_O with the same information as the original. I don't want to store everything in ConstitutionAtom_O - just enough info to recreate the Atom_O object in a stereochemical undefined way.
21:35:52
drmeister
The property list doesn't have any special constraints on what it can store - it can store any kind of information.
21:36:29
drmeister
https://github.com/drmeister/cando/blob/dev/include/cando/chem/constitutionAtoms.h#L75
21:37:31
drmeister
Why is element and atom name not in the property list? What goes in the property list and what doesn't?
21:38:12
drmeister
I want fundamental properties as slots in the ConstitutionAtom_O class and things that the programmer wants to add without recompiling everything in the property list.
22:08:57
drmeister
When rebuilding cando after adding a C++ class - it's important to clear the quicklisp AND slime caches.
22:09:21
drmeister
I had a hell of a time last night because slime was crashing all the time - I had to empty the cache and rebuild it.
22:14:34
drmeister
Do we have a map of class names to stamp values? I should know the answer to this... thinking...
22:16:49
drmeister
If we did I could hash it and include the hash in the results of (lisp-implementation-version)
22:52:55
balrog
will keep y'all updated on arch linux progress. testing on my slow dual-atom box that also powers IRC bouncer and other things, and llvm5 has to compile first :)
23:14:31
attila_lendvai
drmeister: ping. the build seems to be broken on my debian stable, even without my last two commits.
23:15:10
attila_lendvai
drmeister: I have these in wscript.config: USE_LLD = True, LLVM5_ORC_NOTIFIER_PATCH = False
23:17:58
attila_lendvai
drmeister: I've sent you a message with the entire build log. it's probably some llvm version issue... "can't create dynamic relocation R_X86_64_PC32 against symbol"
23:21:54
attila_lendvai
there's this, too, maybe this sheds some light to someone who better understands the C++ landscape: https://bugzilla.mozilla.org/show_bug.cgi?id=1348021
23:29:18
drmeister
attila_lendvai: Someone mentioned recently that llvm has removed the default code model.
23:30:14
attila_lendvai
drmeister: if it doesn't happen in some of your linux installs, then feel free to forget it for now.
23:30:35
drmeister
balrog: You mentioned something about llvm CodeModel changes - what is the story there?
23:31:09
attila_lendvai
on the apache issue the problem was due to some non-included header file, or somesuch -- if that rings a bell
23:35:10
attila_lendvai
AFAIK, nothing changed on my computer besides a few security updates and the clasp git repo, so it's most probably something new in clasp.
23:35:37
balrog
drmeister: https://github.com/llvm-mirror/llvm/commit/9aafb854cc7cb8df8338c50cb411a54ce1e09796 see the commit message
23:36:47
attila_lendvai
someone who understands C++ may get enlightened by reading the comment here: https://reviews.freebsd.org/D12064
23:37:30
attila_lendvai
i.e. this: "libunwind.so has symbols with ELF protected visibility which various test applications attempt to preempt. Both GNU gold and LLVM's lld fail to link this case, with errors like"
23:38:54
attila_lendvai
"On ELF, protected visibility indicates that the symbol will be placed in the dynamic symbol table, but that references within the defining module will bind to the local symbol. That is, the symbol cannot be overridden by another module."
2:21:00
drmeister
We should avoid putting DEFMETHOD and DEFUN and other tags in header files. The scraper gets inconsistent information because different .sif files contain different information.
2:21:36
drmeister
I am now flushing all of the sif files when I change tags in header files so that scraping works properly.
2:52:31
drmeister
balrog: There is a Common Lisp program in clasp/src/scraper that reads every C++ source file of clasp and runs the C++ preprocessor on it. The C++ preprocessed output has metadata injected into it that the scraper extracts and builds tables that describe functions, classes, methods, enums with source information that are all exposed to Clasp Common Lisp.
2:53:10
balrog
I'm assuming this is also used on C++ libraries that are going to interface with Clasp?
2:54:21
drmeister
If you want to expose it to Clasp Common Lisp all that you do is: CL_LISPIFY(core:add-name); CLDEFUN int add2(int x, int y) { return x + y; };
2:56:23
drmeister
It's not used by C++ libraries because you would have to edit them. There is a different syntax for external C++ libraries.
2:57:48
drmeister
All of the files in the build/boehm/generated/ directory are generated by the scraper.
2:58:28
drmeister
There are several variants - and they may be in the header file rather than the .cc file
2:58:49
drmeister
Although I should move as many as possible into the .cc files because in the header files they cause problems with consistency.
2:59:26
balrog
ahh, it was probably CL_LISPIFY_NAME()'s logic confusing me at first (I eventually figured that out)
3:00:23
drmeister
CL_LISPIFY_NAME - right. When the code is not being scraped - these are CPP macros that disappear to nothing.
3:00:55
drmeister
When the scraper is operating they expand to additional text and __FILE__ and __LINE__ entries that are parsed out by the scraper.
3:02:22
drmeister
CL_LISPIFY_NAME(...) is a kludge that I developed to deal with years of not consistently naming things.
3:02:41
drmeister
It's got a complicated algorithm that converts C++ style names to Common Lisp style names
3:02:47
balrog
so like -- where does module-get-function-list come from? I see a declaration for getFunctionList() in llvmoExpose.h, but it's not clear where the metadata for the scraper is located.
3:03:33
balrog
(it makes sense that that's where it's coming from, though, that being classname Module, instance method name getFunctionList)
3:07:39
drmeister
Right - I have to apologize for that. I have not been consistent with naming C++ functions
3:08:27
drmeister
The plan is to write a source-to-source transformation program at some point and make all the names consistent.
3:15:35
drmeister
We can generate a list of every function and what we want to rename it to and then let it loose on the entire Clasp+Cando source code.
4:02:00
drmeister
Bike: With the function descriptions that are compiled into the Module's I can provide the function name in errors like: Too few arguments given 3 required 4
4:02:51
balrog
I still don't know why the whole thing is blowing up, though. I'm guessing the cst work is intended to make that less of a problem?