Using some designs unearthed from building the higher level model, I have brough that design down to the lower level to speed things up. It only took 8 years, but finally, I think I have mastered mult-threading. No more deadlocks, and no more need for waitgroups for everything.
I think it should be pretty much bullet proof at the moment (until the next failure). Also added external index lookups from main index for external docs. found this bug after refining lookups in schemas.
now things are robust, we can move things around a little to prepare for the next set of incoming models. The extraction and builder functions have all been moved to the low packakge, and out of the v3 package.