diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..261eeb9 --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.md b/README.md new file mode 100644 index 0000000..97bfbd7 --- /dev/null +++ b/README.md @@ -0,0 +1,213 @@ +## Guide for CLIAPI Plugin Developers: +**cliapi** ( pronounced _calliope_ ): A Python framework for creating Unix, "getopt()" style CLI scripts +composed from an arbitrary set of APIs. + +...at its core, **cliapi** is _a data driven, state machine for dynamically generating CLIs_. + +### cliapi features: +- entire CLI is generated by applying python decorators to API calls together with a dictionary which +specifies a _backing-API-store_. +This decorator is applied to each API that is supported by a "plugin provider". +For each cloud provider (eg. Azure or GCE) a supporting "plugin" module is added to the +`cliapi/providers` directory (see **`cliapi directory layout`** below) +Presumably the plugin developer will rarely need to change the cliapi framework +itself which involves a little python meta-programming (1 decorator + 1 dictionary +overriding class). + +- consistent command line behavior is enforced by the cliapi framework across all plugin providers. +Several common commands are automatically inherited by all plugin providers: +(this allows a level of "discoverablility" to the CLI across all plugin providers) + - "**--list-providers**" Lists all the available plugin providers. + - "**--provider=**" specify a particular provider plugin for all CLI commands. + - "**--all**" Lists all data returned by all APIs supported by a single provider. + - "**--list-apis**" Lists all the APIs available by a particular plugin provider. + - "**--item=**" Display an item from API by it's right-most JSON path specifier. + For example: + - "**--item=location**" will return the value for **meta_data.compute.location** + - "**--item=privateIpAddress**" will return a list of all keys ending in "**privateIpAddress**" + to get a particular element from a list the item should be referenced using it's unique right-most JSON path name, **i.e**: + ``` + root # azuremetadata --item=interface[0].macAddress + "000D3A3AE8A5" + ``` +- other behaviors enforced by the cliapi framework: + - flexible provider handling based on the way cliapi is invoked: + - if invoked using "cliapi" name then API provider defaults to first valid plugin imported. + - if invoked using _non-cliapi_ name then API provider defaults to the plugin provider with same _non-cliapi_ name. + - the default provider can be over ridden using "--provider=" config option. + - error handling and help is also consistent across all plugin providers. + - multiple queries in same command will return a JSON list in "option order" by default + - single query will return a single JSON element. + +- multiple data queries within the same CLI will result in "at most" a single API call. + +### Key cliapi plugin developer concepts: + +An example cliapi plugin provider for Azuremeta API + SUSE extensions can be found here: +https://github.com/edlane/cliapi/blob/master/cliapi/providers/azure.py + +- **provider**: an associated set of APIs accessed from the CLI in a particular context or cloud + environment e.g. "azure", "gce", "ec2", ... but can essentially be any mix of apis + +- **api**: an API for interfacing to remote or local services. If the interface can be implemented as +a python function with an (_*args, **kwargs_) style calling convention AND it returns a JSON serializable +object, THEN it can easily become a configurable CLI query. With the cliapi decorator, both required and +optional parameters are expressible through the CLI. Help is also handled by the cliapi framework. + +- **fetchers**: a dictionary which maps from a particular API name to the actual python function which +provides the _backing store_ for the contents of the top-level API dictionary. + +### cliapi directory layout: +``` +├── cliapi root of cliapi package +│   ├── __init__.py +│   ├── cliapi_lib.py framework meta classes and decorators +│   ├── cliapi.py main() and the CLI generation code +│   └── what_cloud.py module (useful for detecting which cloud plugins are valid) +│   ├── providers directory for plugin providers +│   │   ├── __init__.py +│   │   ├── azuremetadata.py plugin for Azure/SUSE APIs +│   │   ├── ...your plugin goes here +│   │   ├── ... ...additional plugins are automatically discovered by cliapi +│   │   └── test.py ..."because it's not a framework without at least 2 plugins" +├── design.md design considerations for cliapi +├── LICENSE provisional license (Apache2 is mutable into any other license) +├── README.md this document +└── setup.py Python installation script +``` + +### Examples: + +**example #1** - Common help AND plugin provider help +``` +ed-sle12sp3byos:/home/lane/cliapi # azuremetadata --help +usage: ./azuremetadata --item=[API display item #1]... [API config option#1]... [API required options]... [Common CLI options]... + +***[ azuremetadata ]*** provider Display options: + cloud-service + privateIpAddress + publicIpAddress + address + prefix + macAddress + osType + version + placementGroupId + vmId + sku + platformUpdateDomain + subscriptionId + offer + name + tags + resourceGroupName + vmSize + publisher + platformFaultDomain + location + tag + +***[ azuremetadata ]*** provider API config options: + --api_version= default='2017-08-01' + +Common CLI options: + --list-apis list all available APIs for specified provider + --item= value to return from an API using right-most path name specifier + --list-providers list all available providers + --help help for this CLI command + --all output all API results for a specific provider + --provider= specify name of provider module + +``` +--- +**example #2** - a single display item +``` +ed-sle12sp3byos:/home/lane/cliapi # azuremetadata --item=privateIpAddress +"172.16.3.8" +``` +--- +**example #3** - all values returned by azuremetadata APIs +``` +ed-sle12sp3byos:/home/lane/cliapi # ./azuremetadata --all +{ + "meta_data": { + "network": { + "interface": [ + { + "macAddress": "000D3A3AE8A5", + "ipv4": { + "subnet": [ + { + "address": "172.16.3.0", + "prefix": "24" + } + ], + "ipAddress": [ + { + "privateIpAddress": "172.16.3.8", + "publicIpAddress": "40.112.253.198" + } + ] + }, + "ipv6": { + "ipAddress": [] + } + } + ] + }, + "compute": { + "offer": "SLES-BYOS", + "placementGroupId": "", + "publisher": "SUSE", + "sku": "12-SP3", + "osType": "Linux", + "platformFaultDomain": "0", + "platformUpdateDomain": "0", + "tags": "", + "vmSize": "Standard_B1ms", + "location": "westus", + "name": "ed-sle12sp3byos", + "subscriptionId": "ce73a2b0-d2e7-4ff6-b987-b32d6908de4e", + "version": "2018.02.21", + "vmId": "ce01dc32-6d0a-40bd-9534-a3509f768a53", + "resourceGroupName": "ed_lane" + } + }, + "cloud-service": "__ed-sle12sp3byosService.cloudapp.net", + "tag": "391e4f53-b82d-5af5-8f58-dc1035e46e5e" +} +``` +--- +**example #4** - list of valid plugin providers +``` +ed-sle12sp3byos:/home/lane/cliapi # azuremetadata --list-providers +[ + "test", + "azuremetadata" +] +``` +--- +**example #5** - list of APIs supported by plugin provider +``` +ed-sle12sp3byos:/home/lane/cliapi # azuremetadata --list-apis +[ + "tag", + "cloud-service", + "meta_data" +] +``` +--- +**example #6** - return several display items in single call +``` +ed-sle12sp3byos:/home/lane/cliapi # ./azuremetadata --item=privateIpAddress --item=publisher --item=vmSize --item=location --item=name --item=offer --item=interface[0].macAddress +[ + "172.16.3.8", + "SUSE", + "Standard_B1ms", + "westus", + "ed-sle12sp3byos", + "SLES-BYOS", + "000D3A3AE8A5" +] +``` + diff --git a/cliapi/__init__.py b/cliapi/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/cliapi/cliapi.py b/cliapi/cliapi.py new file mode 100755 index 0000000..5679b68 --- /dev/null +++ b/cliapi/cliapi.py @@ -0,0 +1,287 @@ +#!/usr/bin/python3 + +# Guide for cliapi developers found here: +# https://github.com/edlane/cliapi/blob/master/README.md + +import os + +if 'PYCHARM_DEBUG_ME' in os.environ: + import pydevd + + # we want debugging enabled up top because much of the cliapi framework + # "import-meta-magic" happens before __main__() has ever been called... + # + # Remote debugging session enabled in cloud environments + # by establishing a reverse ssh tunnel with bash: + # + # local_host> ssh -R 8282:localhost:8282 lane@remote_host + # + # Then connect back to your local pycharm debugger with: + # + # remote_host> PYCHARM_DEBUG_ME= cliapi [various options]... + # + pydevd.settrace('localhost', port=8282, stdoutToServer=True, + stderrToServer=True) + +import sys +import importlib +import pkgutil +import getopt +import json +from collections import defaultdict + +import cliapi.providers as providers +from cliapi.cliapi_lib import Provider + +prefix = providers.__name__ + '.' + +# this will scan the "providers" directory for all candidate provider modules... +all_providers = [mod for _, mod, _ in pkgutil.iter_modules(providers.__path__, prefix)] +valid_providers = {} + +# import all valid providers found in the "providers" directory... +for pro in all_providers: + module_name = pro.split('.')[-1] + try: + vp = importlib.import_module(pro) + valid_providers[module_name] = vp + except Exception as e: + print("failure to import {}, {}".format(pro, e)) + pass + +# Help for Common CLI options... +common_help = { + 'help': 'help for this CLI command', + 'provider=': 'specify name of provider module', + 'list-providers': 'list all available providers', + # TODO: does "xml-out" really need to be implemented in a modern Devops world??? + # 'xml-out': 'output in xml format', + 'list-apis': 'list all available APIs for specified provider', + 'all': 'output all API results for a specific provider', + 'item=': 'value to return from an API using right-most path name specifier' +} + + +def _cli_parse(argv): + cli_tree = dict() + for arg in argv: + kv = arg.split('=') + if len(kv) > 1: + cli_tree[kv[0]] = kv[1] + else: + cli_tree[kv[0]] = None + return cli_tree + + +def _print_help(provider): + global common_help + # Help for Common CLI options... + + if provider == None: + vpp = Provider() + # no provider supplied so at least print out the "common" help... + vpp.help = common_help + else: + vpp = valid_providers[provider].provider + required = list() + for fetcher in vpp.fetchers.items(): + # create a list of required options for fast lookup... + required += fetcher[Provider.FUNC_PARMS][Provider.FUNC_KWARGS][Provider.FUNC_REQUIRED] + indent2_format = ' --{:<15} {:<15}' + print( + "usage: {} --item=[API display item #1]... [API config option#1]... [API required options]... [Common CLI options]...".format( + sys.argv[0])) + print("\n***[ {} ]*** provider Display options:".format(provider)) + try: + _fetch_all_apis(vpp) + except AssertionError as e: + print(e.args[0], 'required options must be provided to list all displayable API options') + + all_apis = _scan_apis(vpp) + _print_api_options(all_apis) + print("\n***[ {} ]*** provider API config options:".format(provider)) + for k, v in vpp.options.items(): + # list API configuration options for this provider... + if v.startswith('$'): + opt = v[1:] + help = vpp.help.get(opt, '') + if help != '': + # help message is provided by plugin provider... + help = ''.join((', ', help)) + if v in required: + # if this option is required, then say so... + help = ''.join(('-REQUIRED-', help)) + elif opt in vpp.template: + # otherwise display the default... + help = ''.join(('default=\'', vpp.template[opt], '\'', help)) + + print (indent2_format.format(opt + '=', help)) + print("\nCommon CLI options:") + for key in common_help.keys(): + # list of common CLI options supported by this module, across all providers + print (indent2_format.format(key, common_help.get(key, ''))) + + +def _fetch_all_apis(vpp): + # return a composite dictionary of all API calls... + for fetch in vpp.fetchers: + # prefetch all the APIs... + # must populate the vpp dictionary by walking the top-level API... + try: + vpp[fetch] + except Exception as e: + raise e + # return all the API results from dictionary... + return vpp + + +def _get_value_by_endswith(api_list, endswith): + for path in api_list: + if path[0].endswith(endswith): + return path[1] + + +def _scan_apis(vpp, path='', path_list=[]): + leaf = None + if isinstance(vpp, dict): + for element in vpp.keys(): + leaf = _scan_apis(vpp[element], path + '.' + element, path_list) + if isinstance(leaf, tuple): + path_list.append((list(leaf[1]))) + elif isinstance(vpp, list): + index = 0 + for element in vpp: + leaf = _scan_apis(element, path + '[' + str(index) + ']', path_list) + index += 1 + if isinstance(leaf, tuple): + path_list.append((list(leaf[1]))) + else: + # print(path, vpp ) + return path_list, (path, vpp) + + return path_list + + +def _print_api_options(path_list): + for element in path_list: + split_element = element[0].split('.') + element_len = len(split_element) + print(2 * element_len * ' ', split_element[-1]) + + +def main(): + global common_help + # must do our own parsing here since we don't know the + # valid options until we establish the plugin provider + # -- getopt() does not allow this usage... + ct = _cli_parse(sys.argv[1:]) + + if '--list-providers' in ct: + print(json.dumps(list(valid_providers.keys()), indent=2)) + exit(0) + + cmd_dict = defaultdict(list) + _, cli_name = os.path.split(sys.argv[0]) + if '--provider' in ct: + # using supplied provider... + provider = ct['--provider'] + elif cli_name == 'cliapi': + try: + # use the first valid provider in list... + provider = list(valid_providers.keys())[0] + except Exception as e: + # error -- print help and exit + print('No valid plugin providers found') + _print_help(None) + exit(-1) + else: + # the name of calling program is used to specify the provider... + if cli_name in list(valid_providers.keys()): + provider = cli_name + else: + print('No plugin provider for {} was found'.format(cli_name)) + _print_help(None) + exit(-1) + + vpp = valid_providers[provider].provider + all_opts = [] + options = [] + for k, v in vpp.options.items(): # process args... + if v.startswith('$'): + options.append(v[1:] + '=') + all_opts += options + + # add "common" CLI options to the plugin's cli_options... + all_opts += common_help.keys() + + try: + # now we have enough info to use getopt() for cli parsing... + optlist, arg = getopt.getopt(sys.argv[1:], '', all_opts) + except getopt.GetoptError as e: + # error -- print help and exit + print(e.msg) + cmd_dict['help'] = None + else: + # build a dictionary of the actual supplied CLI options... + for opt in optlist: + if opt[0] == '--item': + # allow for many "items" in defaultdict as a list... + # skipping the leading '--' + cmd_dict[opt[0][2:]].append(opt[1]) + else: + cmd_dict[opt[0][2:]] = opt[1] + + vpp.template.update(cmd_dict) + + # use defaults if option NOT supplied in CLI... + for default in options: + ds = default.split('=') + try: + if not cmd_dict.get(ds[0]): + # override with default... + cmd_dict[ds[0]] = ds[1] + except: + # no option to override or none specified -- ignore this... + pass + + if 'help' in cmd_dict: + _print_help(provider) + exit(0) + + if 'list-apis' in cmd_dict: + print(json.dumps(list(vpp.fetchers.keys()), indent=2)) + exit(0) + + if 'all' in cmd_dict: + data = _fetch_all_apis(vpp) + + else: + data = [] + for key, value in cmd_dict.items(): + if key == 'item': + try: + _fetch_all_apis(vpp) + except AssertionError as e: + _print_help(provider) + exit(-1) + all_apis = _scan_apis(vpp) + for item in cmd_dict['item']: + # ensure that last name is a complete item by prepending '.' ... + endswith = '.' + item + item = _get_value_by_endswith(all_apis, endswith) + data.append(item) + continue + else: + # Not an item? + # then ignore option, continue processing... + continue + + if len(data) == 1: + # a single value was returned so no list is returned... + data = data[0] + + print(json.dumps(data, indent=2)) + + +if __name__ == '__main__': + main() diff --git a/cliapi/cliapi_lib.py b/cliapi/cliapi_lib.py new file mode 100644 index 0000000..c094a3d --- /dev/null +++ b/cliapi/cliapi_lib.py @@ -0,0 +1,135 @@ +import inspect +import importlib +from string import Template + + +class Provider(dict): + # The Provider class overrides Python's dictionary with + # an API-backing-store. + # When a KeyError is encountered during lookup, + # a fetcher, or "API call", is prepared with the appropriate CLI + # options substituted as python function arguments. + + # offsets needed by framework to access the values in the + # "fetcher" dictionary... + FUNC_FULL_NAME = 0 + FUNC_PARMS = 1 + FUNC_ARGS = 0 + FUNC_KWARGS = 1 + FUNC_CALLABLE = 2 + FUNC_NAME = 3 + FUNC_REQUIRED = 0 + + def __init__(self): + self.fetchers = {} + self.options = {} + self.help = {} + self.template = {} + + return super().__init__({}) + + def __getitem__(self, item): + + try: + result = super().__getitem__(item) + except KeyError: + # this dictionary item does not currently exist + # so fetch it using the API which resolves to + # the top level dictionary with that API key name... + fetcher = self.fetchers[item] + fetcher_function = fetcher[Provider.FUNC_FULL_NAME].split('.') + + # setup for calling the appropriate plugin API... + provider = importlib.import_module('.'.join(fetcher_function[0:3])) + + arg_list = list() + kwarg_dict = dict() + try: + # replace API args and kwargs with CLI provided options... + for arg in fetcher[Provider.FUNC_PARMS][Provider.FUNC_ARGS]: + # build the *args tuple... + arg_list.append(Template(arg).substitute(self.template)) + for key, value in fetcher[Provider.FUNC_PARMS][Provider.FUNC_KWARGS].items(): + # build the **kwargs dictionary... + kwarg_dict[key] = Template(value).substitute(self.template) + function = getattr(provider, fetcher_function[Provider.FUNC_NAME]) + # fault-in the API values... + super().__setitem__(item, function(*arg_list, **kwarg_dict)) + except Exception as e: + raise AssertionError('required option {}, missing'.format(e)) + # rerun the dictionary lookup and return results... + result = self.get(item) + return result + + +def cliapi_compile(prov, api_alias=None, options={}, help={}): + # Compile each API supported by the plugin provider as a python + # function and assemble into the various dictionaries of + # the "Provider" class. + # These dictionaries are later used by the "cliapi engine" + # in "cliapi.main()" to create a Unix, getopt()-style CLI... + + def assemble_it(func): + # REMEMBER: All this work happens at "import time"... + if api_alias is None: + alias = func.__name__ + else: + alias = api_alias + # assemble more plugin options... + prov.options.update(options) + # assemble more help... + prov.help.update(help) + + # Here is some meta-magic... + # use introspection to extract calling requirements + # for this particular "API fetcher" and save in fetchers + # dictionary for later when it is actually called... + argspec = inspect.getargspec(func) + if argspec.defaults is None: + default_count = 0 + else: + default_count = len(argspec.defaults) + last_arg = len(argspec.args) - default_count + func_args = [] + func_kwargs = {} + for i, arg in enumerate(argspec.args): + if i < last_arg: + # substitute CLI options... + arg = prov.options.get(arg, arg) + func_args.append(arg) + else: + default = argspec.defaults[i - last_arg] + # substitute CLI options... + op_default = prov.options.get(arg, default) + if op_default.startswith('$'): + # skipping the leading '$'... + prov.template[op_default[1:]] = default + func_kwargs[arg] = op_default + + # Register this function as an "API fetcher"... + prov.fetchers.update({alias: + (func.__module__ + '.' + + func.__name__, + (func_args, func_kwargs), + func) + }) + + def do_it(*args, **kwargs): + # REMEMBER: All this work happens at call time... + # TODO: this is a hack + # at a "Universal Python Function Wrapper". + # There is probably a better way to do this but it + # eludes me at the moment... + if len(args) == 0: + a = () + else: + a = args + if len(kwargs) == 0: + b = {} + else: + b = kwargs + return func(*a, **b) + + return do_it + + return assemble_it diff --git a/cliapi/providers/azuremetadata.py b/cliapi/providers/azuremetadata.py new file mode 100644 index 0000000..0013ce1 --- /dev/null +++ b/cliapi/providers/azuremetadata.py @@ -0,0 +1,48 @@ +from cliapi.cliapi_lib import Provider, cliapi_compile +from cliapi.what_cloud import determine_provider + +import json +import os +import uuid +import urllib.request, urllib.error, urllib.parse +import xml.etree.ElementTree as ET + +try: + # ----- raises an error on import if not running in a "proper" cloud... + cp = determine_provider() + if cp != 'azure': + raise Exception("not a valid provider") +except Exception as e: + raise e + +provider = Provider() + +options = dict(api_version='$api_version') + + +@cliapi_compile(provider, api_alias='meta_data', options=options) +def get_meta_data_azure(api_version='2017-08-01'): + HEADERS = {'Metadata': 'true'} + IP = '169.254.169.254' + url = 'http://{0}/metadata/instance?api-version={1}'.format(IP, api_version) + req = urllib.request.Request(url, headers=HEADERS) + content = urllib.request.urlopen(req).read() + result = json.loads(content.decode('utf-8')) + return result + + +@cliapi_compile(provider, api_alias='cloud-service') +def get_cloud_service(): + # tree = ET.parse('/home/lane/Downloads/SharedConfig.xml') + tree = ET.parse('/var/lib/waagent/SharedConfig.xml') + root = tree.getroot() + cloud_service = root.find('Deployment').find('Service').get('name') + '.cloudapp.net' + return cloud_service + + +@cliapi_compile(provider, api_alias='tag') +def read_billing_guid(device='/dev/sda'): + fd = os.open(device, os.O_RDONLY) + os.lseek(fd, 65536, os.SEEK_SET) + uuid_string = str(uuid.UUID(bytes_le=os.read(fd, 16))) + return uuid_string diff --git a/cliapi/providers/test.py b/cliapi/providers/test.py new file mode 100644 index 0000000..8265719 --- /dev/null +++ b/cliapi/providers/test.py @@ -0,0 +1,43 @@ + +from cliapi.cliapi_lib import Provider, cliapi_compile + +provider = Provider() + + +help = { + 'smurf': 'what is smurf name?', + 'dweebville': 'where is dweebville?', +} + +# options = dict(arg1='hello', key1='$dufus', key2='$dweebville',) +options = dict(arg1='$smurf', key1='$dufus', key2='$dweebville',) + +@cliapi_compile(provider, api_alias='test', help=help, options=options) +def foo(arg1, key1='bar', key2='baz'): + return [arg1, key1, key2] + + +@cliapi_compile(provider, api_alias='meta_data') +def get_meta_data_mock(): + return {"compute": {"location": "westus", + "name": "ed-sle12sp3byos", "offer": "SLES-BYOS", + "osType": "Linux", "placementGroupId": "", + "platformFaultDomain": "0", + "platformUpdateDomain": "0", "publisher": "SUSE", + "resourceGroupName": "ed_lane", "sku": "12-SP3", + "subscriptionId": "ce73a2b0-d2e7-4ff6-b987-b32d6908de4e", + "tags": "", "version": "2018.02.21", "vmId": "ce01dc32-6d0a-40bd-9534-a3509f768a53", + "vmSize": "Standard_B1ms"}, + "network": {"interface": [{"ipv4": {"ipAddress": + [{"privateIpAddress": "172.16.3.8", + "publicIpAddress": "104.42.34.116"}], + "subnet": [{"address": "172.16.3.0", "prefix": "24"}]}, + "ipv6": {"ipAddress": []}, + "macAddress": "000D3A3AE8A5"}]}} + + +options = dict(api_version='$api_version') + +@cliapi_compile(provider, api_alias='some_stuff', options=options) +def get_stuff(api_version='2017-08-01'): + return api_version diff --git a/cliapi/what_cloud.py b/cliapi/what_cloud.py new file mode 100644 index 0000000..6ae194a --- /dev/null +++ b/cliapi/what_cloud.py @@ -0,0 +1,15 @@ +from os import popen + + +def determine_provider(): + result = popen('/usr/sbin/dmidecode -t system').read() + output = result.lower() + if 'amazon' in output: + provider = 'ec2' + elif 'microsoft' in output: + provider = 'azure' + elif 'google' in output: + provider = 'gce' + else: + raise Exception('Provider not found.') + return provider diff --git a/setup.py b/setup.py new file mode 100644 index 0000000..90020c0 --- /dev/null +++ b/setup.py @@ -0,0 +1,200 @@ +"""A setuptools based setup module. + +See: +https://packaging.python.org/en/latest/distributing.html +https://github.com/pypa/sampleproject +""" + +# Always prefer setuptools over distutils +from setuptools import setup, find_packages +from os import path +# io.open is needed for projects that support Python 2.7 +# It ensures open() defaults to text mode with universal newlines, +# and accepts an argument to specify the text encoding +# Python 3 only projects can skip this import +from io import open + +here = path.abspath(path.dirname(__file__)) + +# Get the long description from the README file +with open(path.join(here, 'README.md'), encoding='utf-8') as f: + long_description = f.read() + +# Arguments marked as "Required" below must be included for upload to PyPI. +# Fields marked as "Optional" may be commented out. + +setup( + # This is the name of your project. The first time you publish this + # package, this name will be registered for you. It will determine how + # users can install this project, e.g.: + # + # $ pip install sampleproject + # + # And where it will live on PyPI: https://pypi.org/project/sampleproject/ + # + # There are some restrictions on what makes a valid project name + # specification here: + # https://packaging.python.org/specifications/core-metadata/#name + name='cliapi', # Required + + # Versions should comply with PEP 440: + # https://www.python.org/dev/peps/pep-0440/ + # + # For a discussion on single-sourcing the version across setup.py and the + # project code, see + # https://packaging.python.org/en/latest/single_source_version.html + version='0.1', # Required + + # This is a one-line description or tagline of what your project does. This + # corresponds to the "Summary" metadata field: + # https://packaging.python.org/specifications/core-metadata/#summary + description='a dynamic getopts()-style CLI generator', # Required + + # This is an optional longer description of your project that represents + # the body of text which users will see when they visit PyPI. + # + # Often, this is the same as your README, so you can just read it in from + # that file directly (as we have already done above) + # + # This field corresponds to the "Description" metadata field: + # https://packaging.python.org/specifications/core-metadata/#description-optional + long_description=long_description, # Optional + + # Denotes that our long_description is in Markdown; valid values are + # text/plain, text/x-rst, and text/markdown + # + # Optional if long_description is written in reStructuredText (rst) but + # required for plain-text or Markdown; if unspecified, "applications should + # attempt to render [the long_description] as text/x-rst; charset=UTF-8 and + # fall back to text/plain if it is not valid rst" (see link below) + # + # This field corresponds to the "Description-Content-Type" metadata field: + # https://packaging.python.org/specifications/core-metadata/#description-content-type-optional + long_description_content_type='text/markdown', # Optional (see note above) + + # This should be a valid link to your project's main homepage. + # + # This field corresponds to the "Home-Page" metadata field: + # https://packaging.python.org/specifications/core-metadata/#home-page-optional + url='https://github.com/edlane/cliapi', # Optional + + # This should be your name or the name of the organization which owns the + # project. + author='ed lane', # Optional + + # This should be a valid email address corresponding to the author listed + # above. + author_email='ed.lane@suse.com', # Optional + + # Classifiers help users find your project by categorizing it. + # + # For a list of valid classifiers, see https://pypi.org/classifiers/ + classifiers=[ # Optional + # How mature is this project? Common values are + # 3 - Alpha + # 4 - Beta + # 5 - Production/Stable + 'Development Status :: 3 - Alpha', + + # Indicate who your project is intended for + 'Intended Audience :: Developers', + 'Topic :: Software Development :: CLI coding tools', + + # Pick your license as you wish + 'License :: OSI Approved :: Apache 2.0', + + # Specify the Python versions you support here. In particular, ensure + # that you indicate whether you support Python 2, Python 3 or both. + 'Programming Language :: Python :: 2', + 'Programming Language :: Python :: 2.7', + 'Programming Language :: Python :: 3', + 'Programming Language :: Python :: 3.4', + 'Programming Language :: Python :: 3.5', + 'Programming Language :: Python :: 3.6', + ], + + # This field adds keywords for your project which will appear on the + # project page. What does your project relate to? + # + # Note that this is a string of words separated by whitespace, not a list. + keywords='CLI coding tools', # Optional + + # You can just specify package directories manually here if your project is + # simple. Or you can use find_packages(). + # + # Alternatively, if you just want to distribute a single Python file, use + # the `py_modules` argument instead as follows, which will expect a file + # called `my_module.py` to exist: + # + # py_modules=["cliapi.py", "cliapi_lib.py"], + # py_modules=["my_module"], + # + packages=find_packages(exclude=['contrib', 'docs', 'tests']), # Required + + # This field lists other packages that your project depends on to run. + # Any package you put here will be installed by pip when your project is + # installed, so they must be valid existing projects. + # + # For an analysis of "install_requires" vs pip's requirements files see: + # https://packaging.python.org/en/latest/requirements.html + # install_requires=['peppercorn'], # Optional + # install_requires=['cliapi'], # Optional + + # List additional groups of dependencies here (e.g. development + # dependencies). Users will be able to install these using the "extras" + # syntax, for example: + # + # $ pip install sampleproject[dev] + # + # Similar to `install_requires` above, these must be valid existing + # projects. + # extras_require={ # Optional + # 'dev': ['check-manifest'], + # 'test': ['coverage'], + # }, + + # If there are data files included in your packages that need to be + # installed, specify them here. + # + # If using Python 2.6 or earlier, then these have to be included in + # MANIFEST.in as well. + # package_data={ # Optional + # 'sample': ['package_data.dat'], + # }, + + # Although 'package_data' is the preferred approach, in some case you may + # need to place data files outside of your packages. See: + # http://docs.python.org/3.4/distutils/setupscript.html#installing-additional-files + # + # In this case, 'data_file' will be installed into '/my_data' + # data_files=[('my_data', ['data/data_file'])], # Optional + + # To provide executable scripts, use entry points in preference to the + # "scripts" keyword. Entry points provide cross-platform support and allow + # `pip` to create the appropriate form of executable for the target + # platform. + # + # For example, the following would provide a command called `sample` which + # executes the function `main` from this package when invoked: + entry_points={ # Optional + 'console_scripts': [ + 'cliapi=cliapi.cliapi:main', + ], + }, + + # List additional URLs that are relevant to your project as a dict. + # + # This field corresponds to the "Project-URL" metadata fields: + # https://packaging.python.org/specifications/core-metadata/#project-url-multiple-use + # + # Examples listed include a pattern for specifying where the package tracks + # issues, where the source is hosted, where to say thanks to the package + # maintainers, and where to support the project financially. The key is + # what's used to render the link text on PyPI. + # project_urls={ # Optional + # 'Bug Reports': 'https://github.com/pypa/sampleproject/issues', + # 'Funding': 'https://donate.pypi.org', + # 'Say Thanks!': 'http://saythanks.io/to/example', + # 'Source': 'https://github.com/pypa/sampleproject/', + # }, +)