`Fog::Core` Provider/Service registration

[tokengeek]

This is work in progress specification for adding core functionality to register a provider and its services within Fog::Core making the register available to third party applications.

This is an expansion of #16

Goals:

• fog references no providers in code only dependencies. For any provider that is required, they register themselves and are available through the current API.
• fog-provider can contain all the code for a provider.
• Applications can require fog and "new experimental provider" and NEP is available via current API
• To avoid clashes with existing code and modules, they will exist in Fog::Core
• Configuration/Credentials will vary per provider/service but their passing should be standardised in the registration calls.
• Configuration/Credentials should not depend on .fog since it should be usable programmatically from third party applications.
• Providers may offer different versions of a service that we either need to support or remove. Registering 2 compute versions verses a specific version for a provider.

This is arising from fog/fog-rackspace#10 where the need for this functionality meant quite a disruptive change of code occurred.

/cc @geemus @plribeiro3000 @mdarby

[tokengeek]

Also to improve loading, there should be checks possible against each provider/service that is registered checking for dependencies or configuration.

Fog::Core::ProviderRegister.providers # => [<Fog::Example name: :example>, ...]
provider = Fog::Core::ProviderRegister[:example]
provider.dependencies_met? # => true
provider.configured? # => false

So for each registered provider, we can avoid loading the bulk of anything based on criteria we've been dealing with for a while.

Namely you can require fog-(native hypervisor) and it will return false if you do not have hypervisor-ruby gem and hypervisor install.

When testing, providers can decide if they are configured and the tests run only those who are.

[plribeiro3000]

I think this is a good starting point.

What if we start using autoload. Shouldn't it be enough to improve loading?
I've already started doing this refactoring on the providers i got out of the main gem.

[tokengeek]

I always get pointed back to https://www.ruby-forum.com/topic/3036681 when autoload is mentioned. I've not heard any updates so I always think of it as deprecated.

Really what I was planning for those checks was allowing questions to be asked to solve a few problems.

So in test_helper we can use the Fog::Core version for do this change... (fog/fog@00ebbd3)

That is kind of secondary though.

First we need to register providers and services (with versions) and expose something that returns the correct Class when Fog::Service[:provider] or Fog::Service.new(:provider ...) is used.

The additional tricky part is that Fog::Service needs to be registered somewhere by someone. For the main, agreed upon services they can be predeclared in core, but for some providers they just declared whatever they like.

Still haven't decided the best way to avoid polluting the Fog namespace with services connected with one provider.

Going to have a spike on this today and see if I can get some code ready.

[plribeiro3000]

👍 Thanks for the link.

I'm gonna have to start refactoring my projects asap. 😅

[tokengeek]

I dug out my old code and moved some bits around. Some of the original names would have clashed with files after the split.

I've written tests (fog/fog#3326) to check the current (as at v1.25.0) interface to the Fog::Bin stuff.

So I hope to get those tests working against my version of fog-core which has a Fog::Core::ProviderRegistry class inside.

I have a rough fog-brightbox that registers itself as a subclass of Fog::Core::ServiceProvider that allows a few of the checks I mentioned above.

I need to look at moving stuff from the bin to this new class. I'll try to pull together enough code to push later.

[geemus]

@tokengeek Sounds good, just let me know if you need anything. Thanks!

[tokengeek]

So a quick update on this. I have a simple registry in Fog::Core, a new class for providers to inherit from and a rough prototype of fog-brightbox.

I'm erring towards backward compatibility since there is a simple method there anyway that builds a hash as the current "registry" but then parts delegate to the "bin" stuff.

I am still trying to get a few issues resolved before I push the code because I keep finding other issues the more I dig in. In adding tests I discovered fog/fog-storm_on_demand#1 for example.

The main one is the use of Fog.credentials in the #available? checks. I tackled some of this earlier (fog/fog#1390) but the "bin" behaviour is still checking the global.

I'm trying to solve it in the least disruptive manner. I find the most disruptive being forcing an update to every provider in order to make it work. So I'm trying to allow configuration in fog for those providers who have not been extracted.

I don't want to rush this because of the coordination involved across many projects. When I have a stable prototype across all three layers I'll try to get it pushed.

I think I'm going to ticket fog 2.0 and I can probably do with help from people triaging issues into "next release", a "1.99" release with lots of deprecation warnings switched on and the actual "2.0"

Pretty much we should feature freeze fog 1.x

I can then try and get some of this stuff done in the next weeks whilst I (may) have some free time / holiday.