Plugins Dependency Bootstrapper

[TBD]

TBD submitted a new resource:

Dependency Bootstrapper - automate installation of scene dependencies

Bootstrap your scenes' dependencies
Intended for scene creators, this plugin will supercharge dependency installation for users when creators include it with their scenes.
That doesn't mean it's not useful to users for scenes that weren't build with it in mind.

What is it and how does it work?
The plugin comes with a proxy scene, which downloads missing dependencies and redirects to the target scene once the download completes.
All the end user needs to do is...

Read more about this resource...

 

[TBD]

TBD updated Dependency Bootstrapper with a new update entry:

CC-BY-SA

now free, enjoy

Read the rest of this update entry...

 

[AlienAxeMan]

I must be missing something. What's the benefit here as opposed to using "Scan Hub For Missing Referenced Packages" to acquire all the referenced packages for all local resources?

I used to use the Package Manager, which is really slow to open and then also to Scan Hub For Missing Packages. Once I discovered the HUB has a similar scanning feature that is way faster, then that seems like a very efficient process. I simply download all the new stuff I want each day after starting up VAM, then run that scan from the HUB UI once, hit the button to download all, and every dependency that's available on the HUB is quickly downloaded.

It seems like perhaps a unique aspect here is the ability for a creator to set the "required" dependencies. I'm still not seeing much of a net win there though, as missing dependencies will still clutter up the error window and such. So, I really must still be failing to grasp something here...

 

[TBD]

As long as you're ok with downloading each and every dependency, the native "scan for missing" feature is totally sufficient. However, if you want to exclude some, there's no way to do that.

For example, you might want to keep only the latest versions of resources and delete the old ones. There might also be some you don't want to download altogether. Personally, there are many clothing items, plugins and assets, which I don't use. My AddonPackages would probably triple in size if I downloaded all of them.

You're right that you'll have to live with the error messages of missing dependencies. I don't think that's a big issue, though. In the package manager, you can disable error messages for individual vars if you want to.
I haven't looked at whether it's feasible to do the same programmatically, yet. It might be.

 

[b00marrows]

This is a great idea for usability, putting it upfront when loading a scene if a user is missing something.

But..
Is this, or can this be made smarter in the sense of "chained dependencies"?
Too often i find scenes with massive chained together downloading of dependencies, when it turns out it just needed a model of a candle 6 downloads in...

 

[TBD]

This is a great idea for usability, putting it upfront when loading a scene if a user is missing something.

But..
Is this, or can this be made smarter in the sense of "chained dependencies"?
Too often i find scenes with massive chained together downloading of dependencies, when it turns out it just needed a model of a candle 6 downloads in...

To be blunt, that's really the scene authors' responsibility. The Dependency Bootstrapper can be configured in a way to only require the candle in your example and make everything else optional. However, this doesn't alleviate the fact that the package's dependencies aren't set up correctly.

I plan on adding features to aid with var packaging. As a start, I could show some info on where and why a given dependency is being used in a scene.

 

[b00marrows]

To be blunt, that's really the scene authors' responsibility. The Dependency Bootstrapper can be configured in a way to only require the candle in your example and make everything else optional. However, this doesn't alleviate the fact that the package's dependencies aren't set up correctly.

I plan on adding features to aid with var packaging. As a start, I could show some info on where and why a given dependency is being used in a scene.

Oh no I agree, tho i will also note that the games default way of doing it is... sub par from what little i know of it.
I have attempted, on occasion to "drill into" the meta files to fix things. Not something that goes well all that often.

BUT i think the dependencies are listen properly inside the meta.inf file.
As in, the dependency chain is listed inside the meta.json, each level indented.
What i suspect is, if if was "read backwards" and we tried to pull from the bottom first we might be able to "skip" the chains in some cases.

For example, instead of:
(Need Z) W has X has Y has Z W -> X -> Y -> Z
Check from the "bottom" of the chain to see if Z is available independently, if not take a step up each time.

I'm no expert and may be talking out my ass. Would this not be possible?

 

[TBD]

Oh no I agree, tho i will also note that the games default way of doing it is... sub par from what little i know of it.
I have attempted, on occasion to "drill into" the meta files to fix things. Not something that goes well all that often.

BUT i think the dependencies are listen properly inside the meta.inf file.
As in, the dependency chain is listed inside the meta.json, each level indented.
What i suspect is, if if was "read backwards" and we tried to pull from the bottom first we might be able to "skip" the chains in some cases.

For example, instead of:
(Need Z) W has X has Y has Z W -> X -> Y -> Z
Check from the "bottom" of the chain to see if Z is available independently, if not take a step up each time.

I'm no expert and may be talking out my ass. Would this not be possible?

The problem is what's referenced in the scene json and/or any presets included in the var. The meta file just lists everything and shouldn't be tempered with.
Most commonly, large dependency trees happen if scene authors reference other scenes instead of e.g. assets or use appearances, which have dependencies themselves. References to other scenes should be avoided at all costs.

 

[b00marrows]

The problem is what's referenced in the scene json and/or any presets included in the var. The meta file just lists everything and shouldn't be tempered with.
Most commonly, large dependency trees happen if scene authors reference other scenes instead of e.g. assets or use appearances, which have dependencies themselves. References to other scenes should be avoided at all costs.

Yeah its an understandably annoying and understandable issue that not really fixable outside of the scene creator.

Whats the potential of limitimg depedancies in a more logical way? like... how to explain...

Often when a scene needs a dependency[1] it only needs a small part of that dependency[1], but often none of dependancy[1] own dependency[2]
Does that make sense?
Would there be a way to make the game ignore the missing, not needed depenancies? like stop the log spamming for 1.

 

[TBD]

Yeah its an understandably annoying and understandable issue that not really fixable outside of the scene creator.

Whats the potential of limitimg depedancies in a more logical way? like... how to explain...

Often when a scene needs a dependency[1] it only needs a small part of that dependency[1], but often none of dependancy[1] own dependency[2]
Does that make sense?
Would there be a way to make the game ignore the missing, not needed depenancies? like stop the log spamming for 1.

A real world example for a var like this would be a plugin/asset/appearance that comes with a demo scene, which has additional dependencies not needed by the resource itself.
Sadly, there's nothing you can do with/to vam to cherry pick the parts you want or need. In this example, the only thing you can do is to mute the error log for this specific var.
The clean way to solve it is to build modular vars. If the author released separate vars for the plugin and the demo scene, anyone referencing it wouldn't have to pull in all of the demo scene's dependencies.

The dependency bootstrap plugin can get around it, to some extend, by marking those 2nd level dependencies as optional. Thereby, the end user can opt to skip downloading them.

 

[TBD]

TBD updated Dependency Bootstrapper with a new update entry:

small fix

Fixed a crash when processing scenes with circular references.
(e.g. Zinigo's The Summoning, which has a trigger to reload the same scene)

Read the rest of this update entry...

 

[mightkc]

Would it be possible to add a way to have dependencies searched for from disk and a from a configurable path?

 

[TBD]

Would it be possible to add a way to have dependencies searched for from disk and a from a configurable path?

Certainly, could you elaborate on the use case you have in mind, please? A local repository to pull packages from instead of the hub to make dependencies volatile and loaded on demand?

 

[mightkc]

Certainly, could you elaborate on the use case you have in mind, please? A local repository to pull packages from instead of the hub to make dependencies volatile and loaded on demand?

Yeah, I have a lot of VARs and don't like them all in the addonpackages of my VAM install. So I store them at different locations on my device or local network. When on network, I might network map the network path so that in VAM, it looks like a drive letter path.

I would want to drop some arbitrarily selected VAR from my collection and place it in the addonpackage or use a tool to then pull in from that external path (and subdirectories) all dependencies needed (including the recursive dependencies those sub dependencies require).

 

[TBD]

Yeah, I have a lot of VARs and don't like them all in the addonpackages of my VAM install. So I store them at different locations on my device or local network. When on network, I might network map the network path so that in VAM, it looks like a drive letter path.

I would want to drop some arbitrarily selected VAR from my collection and place it in the addonpackage or use a tool to then pull in from that external path (and subdirectories) all dependencies needed (including the recursive dependencies those sub dependencies require).

It's possible with some limitations. Plugins cannot read from directories outside of vam and they cannot delete files in the AddonPackages folder.
This means you'd need a symlink to your storage location inside the vam directory. Not being able to delete vars mean I'd have to overwrite them with zero size files, causing a bunch of noise in vam's error log during start up. It should still result in quicker start times but it isn't pretty.

That being said, those limitations don't apply to external tools, obviously. You could try VaM Thin Twin, which aims to do exactly that. I haven't tested it myself and you'd have to get the paid version, though.

 

[mightkc]

Thanks for having given it some thought! And the suggestion!