CM python package API

Submodules

cmind.artifact module

class cmind.artifact.Artifact(cmind, path)[source]

Bases: object

CM artifact class

Initialize CM artifact class

Parameters:

  • cmind (CM class)

  • path (str) – path to a CM artifact

Returns:

(python class) with the following vars –

  • path (str): path to CM artifact

  • original_meta (dict): original meta description of this artifact without inheritance

  • meta (dict): meta description of this artifact after inheritance

load(ignore_inheritance=False, base_recursion=0)[source]

Load CM artifact

Parameters:

  • ignore_inheritance (bool) – if True ignore artifact meta description inheritance

  • base_recursion (int) – internal to avoid infinite recursion during inheritance

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

update(meta, append_lists=True, replace=False, tags=[])[source]

Update CM artifact

Parameters:

  • meta (dict) – new meta description

  • replace (bool) – if True, replace original meta description instead of merging

  • append_lists (bool) – if True and replace is False, append lists when merging meta descriptions instead of substituting

  • tags (list) – replace tags in meta

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.automation module

class cmind.automation.Automation(cmind, automation_file)[source]

Bases: object

CM automation class

Initialize CM automation class

Parameters:

  • cmind (CM class)

  • automation_file (str) – path to the CM automation Python module

Returns:

(python class) with the following vars –

  • automation_file_path (str): full path to the CM automation Python module

  • path (str): directory with the CM automation

add(i)[source]

Add CM artifact

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • meta (dict) – meta description of this artifact

  • (tags) (string or list) – tags to be added to meta

  • (new_tags) (string or list) – new tags to be added to meta (the same as tags)

  • (yaml) (bool) – if True, record YAML instead of JSON

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): final meta of the artifact

  • path (str): path to the created artifact

copy(i)[source]

Copy CM artifact(s) either to a new artifact or to a new repository

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • parsed_artifacts (list) – prepared in CM CLI or CM access function - 1st entry has a new artifact name and repository: [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • new_tags (string) – add new tags (separated by comma) to new artifacts

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • list (list): list of copied CM artifacts

delete(i)[source]

Delete CM artifact

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • (tags) (string or list) – optional tags to find artifacts

  • (force) (bool) – force deleting CM artifacts without asking a user

  • (f) (bool) – the same as “force”

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • deleted_list (list): a list of deleted CM artifacts

info(i)[source]

Get info about artifacts

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • parsed_artifacts (list) – prepared in CM CLI or CM access function - 1st entry has a new artifact name and repository: [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • (uid) (bool) – if True, copy CID in UID::UID format to clipboard

  • (md) (bool) – if True, copy CID in [](UID::UID) format to clipboard

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • list (list): list of CM artifacts

load(i)[source]

Load CM artifact

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • (tags) (string or list) – optional tags to find artifacts

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • path (str): path to a loaded CM artifact

  • original_meta (dict): meta description of the loaded CM artifact before inheritance

  • meta (dict): meta description of the loaded CM artifact after inheritance

  • artifact (CM artifact object)

move(i)[source]

Rename CM artifacts and/or move them to another CM repository.

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • parsed_artifacts (list) – prepared in CM CLI or CM access function - 1st entry has a new artifact name and repository: [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • list (list): list of renamed/moved CM artifacts

search(i)[source]

Find CM artifacts (slow - we plan to accelerate it in the future using different indexing mechanisms)

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • ignore_inheritance (bool) – if ‘True’, ignore artifact meta description inheritance and just load meta to match UID/alias

  • (skip_index_search) (bool) – If True, skip indexing

  • (force_index_add) (bool) – If True, force index add (for reindexing)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • list (list): list of CM artifact objects

test(i)[source]

Test CM and print various info

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

update(i)[source]

Update CM artifact.

Note: not thread safe - we expect one pipeline running on a system

can make it thread safe when needed (similar to CK)

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

  • parsed_automation (list) – prepared in CM CLI or CM access function [ (automation alias, automation UID) ] or [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]

  • parsed_artifact (list) – prepared in CM CLI or CM access function [ (artifact alias, artifact UID) ] or [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]

  • (search_tags) (string or list) – optional tags to find artifacts

  • (meta) (dict) – new meta description to be merged or to replace the original meta description of a CM artifact

  • (tags) (string or list) – tags to be added to meta

  • (replace) (bool) – if True, replace original meta description (False by default)

  • (force) (bool) – if True, force updates if more than 1 artifact found (False by default)

  • (replace_lists) (bool) – if True, replace lists in meta description rather than appending them (False by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • list (list): list of updated CM artifacts

version(i)[source]

Print CM version

Parameters:

  • (CM input dict)

  • (out) (str) – if ‘con’, output to console

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.cli module

cmind.cli.docker_script(argv=None)[source]

Shortcut to “cm docker script …”

CM command line format:

Parameters:

argv (list | string) – command line arguments

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • Output from a CM automation action

cmind.cli.gui_script(argv=None)[source]

Shortcut to “cm gui script …”

CM command line format:

Parameters:

argv (list | string) – command line arguments

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • Output from a CM automation action

cmind.cli.parse(cmd)[source]

Parse CM command line into CM input dictionary.

Parameters:

cmd (str | list) – arguments as a string or list

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • cm_input (dict): CM unified input to the CM access function

cmind.cli.run(argv=None)[source]

Run CM automation actions from the command line.

CM command line format:

cm {action} {automation} (artifacts) (–flags) (@input.yaml) (@input.json)

Parameters:

argv (list | string) – command line arguments

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • Output from a CM automation action

cmind.cli.run_experiment(argv=None)[source]

Shortcut to “cm run experiment …”

CM command line format:

Parameters:

argv (list | string) – command line arguments

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • Output from a CM automation action

cmind.cli.run_script(argv=None)[source]

Shortcut to “cm run script …”

CM command line format:

Parameters:

argv (list | string) – command line arguments

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • Output from a CM automation action

cmind.config module

class cmind.config.Config(config_file=None)[source]

Bases: object

CM configuration class

Initialize CM configuration class

Parameters:

(config_file) (str) – If not None, merge this file with the internal configuration

Returns:

(python class) with the following vars –

  • cfg (dict): internal CM configuration

cmind.core module

class cmind.core.CM(repos_path='', debug=False)[source]

Bases: object

Main CM class

Initialize CM configuration class

Parameters:

  • (repos_path) (str)

  • (debug) (bool) – instead of returning a dictionary with “return”>0.

Returns:

(python class) with the following vars –

  • cfg (dict): internal CM configuration

  • debug (bool): debug state

  • repos_path (str): path to CM repositories and other information

  • path_to_cmind (str): path to the used CM toolkit

  • path_to_cmind_core_module (str): path to CM core module

  • path_to_cmind_repo (str): path to the “internal” CM repo

  • repos (list): list of initialized CM repository objects

  • common_automation (Automation class): initialized common automation with actions when a given automation is not found or doesn’t exist

  • output (str): value of –out during the first access to CM (to print errors, etc)

access(i, out=None)[source]

Access CM automation actions in a unified way similar to micro-services.

i (dict | str | argv): unified CM input

Parameters:

  • (action) (str) – automation action

  • (automation (CM object) – CM automation in format (alias | UID | alias,UID) or (repo alias | repo UID | repo alias,UID):(alias | UID | alias,UID)

  • (artifact) (CM object) – CM artifact

  • (artifacts) (list of CM objects) – extra CM artifacts

  • (common) (bool) – if True, use common automation action from Automation class

  • (help) (bool) – if True, print CM automation action API

  • (ignore_inheritance) (bool) – if True, ignore inheritance when searching for artifacts and automations

  • (out) (str) – if ‘con’, tell automations and CM to output extra information to console

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • Output from a CM automation action

error(r)[source]

If r[‘return’]>0: print CM error and raise error if in debugging mode

Parameters:

r (dict) – output from CM function with “return” and “error”

Returns:

(dict) – r

halt(r)[source]

If r[‘return’]>0: print CM error and raise error if in debugging mode or halt with “return” code

Parameters:

r (dict) – output from CM function with “return” and “error”

Returns:

(dict) – r

log(s)[source]
Parameters:

s (string) – log string

Returns:

None

cmind.core.access(i)[source]

Automatically initialize CM and run automations without the need to initialize and customize CM class. Useful for Python automation scripts.

See CM.access function for more details.

cmind.core.error(i)[source]

Automatically initialize CM and print error if needed without the need to initialize and customize CM class. Useful for Python automation scripts.

See CM.error function for more details.

cmind.core.get_version()[source]

Get CM version.

Args:

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • version (str): CM version

cmind.core.halt(i)[source]

Automatically initialize CM, print error and halt if needed without the need to initialize and customize CM class. Useful for Python automation scripts.

See CM.halt function for more details.

cmind.core.parse_cm_object_and_check_current_dir(cmind, artifact)[source]

Internal function: parses CM object and check if there is ‘.’ to detect CM repository in a current directory.

cmind.core.print_action_help(automation, common_automation, print_automation, action, original_action)[source]
cmind.core.print_db_actions(automation, equivalent_actions, automation_name)[source]

Internal function: prints CM database actions.

cmind.index module

class cmind.index.Index(path, cfg, meta={})[source]

Bases: object

CM index class

Initialize CM index class

Parameters:

  • path (str) – path to directory with repos and index file

  • cfg (dict) – CM configuration

  • (meta) (dict) – CM index meta

Returns:

(python class) with the following vars –

  • path (str): path to CM repositories and index file

  • meta (dict): CM index meta

add(meta, path, repo, update=False, delete=False)[source]

Add to CM index

Parameters:

  • meta (dict) – artifact meta

  • path (str) – artifact path

  • repo (tuple of str) – (alias, uid) of repo

  • update (bool) – force update

  • delete (bool) – delete instead of adding

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

find(automation_name, artifact_obj, artifact_repo, pruned_repos, and_tags, no_tags)[source]

Find in CM index

Parameters:

  • automation_name (tuple of str) – (alias, uid) of automation

  • artifact_obj (tuple of str) – (alias, uid) of artifact obj

  • artifact_repo (tuple of str of None) – if not None, use pruned_repos

  • pruned_repos (list of Repo classes) – pruned repos

  • and_tags (list of str) – AND tags

  • no_tags (list of str) – NO tags

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

load()[source]

Load or initialize CM index

Parameters:

None

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

save()[source]

Save CM index TBD: it’s not thread safe and we should provide better handling if CM is used in parallel

we need to reload

Parameters:

None

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.net module

cmind.net.request(i)[source]

Web request to cKnowledge.org server to get release notes about components (for CK compatibility)

Parameters:

CM input passed to cKnowledge.org API

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • version (str): CM version

  • string (str): returned string from the server

  • dict (dict): JSON string converted to dict (if possible)

cmind.repo module

class cmind.repo.Repo(path, cfg)[source]

Bases: object

CM repository class

Initialize CM repository class

Parameters:

  • path (str) – path to a CM repository

  • cfg (dict) – CM configuration

Returns:

(python class) with the following vars –

  • path (str): path to a CM repository

  • path_prefix (str): use extra directory inside CM repository to keep CM artifacts

  • path_with_prefix (str): path to a CM repository if directory prefix is used

  • meta (dict): CM repository meta description

load()[source]

Load CM repository

Parameters:

None

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.repos module

class cmind.repos.Repos(path, cfg, path_to_internal_repo='')[source]

Bases: object

CM repositories class

Initialize CM repositories class

Parameters:

  • path (str) – path to directory with repos.json where all paths of registered CM repositories are stored

  • cfg (dict) – CM configuration

  • (path_to_internal_repo) (str) – path to the internal CM repository (useful during development/debugging)

Returns:

(python class) with the following vars –

  • path (str): path to CM repositories and index file

  • full_path_to_repos (str): path to CM repositories

  • path_to_internal_repo (str): path to the internal CM repository

  • full_path_to_repo_paths (str): full path to repos.json

  • lst (list): list of initialized CM repository classes

delete(lst, remove_all=False, console=False, force=False)[source]

Delete CM repository or repositories with or without content

Parameters:

  • lst (list of CM repository classes) – list of CM repositories

  • remove_all (bool) – if True, remove the content and unregister CM repository

  • console (bool) – if True, output some info to console

  • force (bool) – if True, do not ask questions

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

init(alias, uid, path='', console=False, desc='', prefix='', only_register=False)[source]

Init CM repository in a given path

Parameters:

  • alias (str) – CM repository alias

  • uid (str) – CM repository UID

  • (path) (str) – local path to a given repository (otherwise use $HOME/CM/repos)

  • (desc) (str) – optional repository description

  • (prefix) (str) – sub-directory to be used inside this CM repository to store artifacts

  • (only_register) (bool) – if True, only register path in the CM index of repositories but do not recreate cmr.yaml

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): meta of the CM repository

  • path_to_repo (str): path to repository

  • path_to_repo_desc (str): path to repository description

  • path_to_repo_with_prefix (str): path to repository with prefix (== path_to_repo if prefix == “”)

  • (warnings) (list of str): warnings to install more CM repositories

load(init=False)[source]

Load or initialize repos.json file with repositories

Parameters:

init (bool) – if False do not init individual CM repositories

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

process(repo_path, mode='add')[source]

Add or delete CM repository

Parameters:

  • repo_path (str) – path to CM repository

  • (mode) (str) – “add” (default) or “delete”

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • (warnings) (list of str): warnings to install more CM repositories

pull(alias, url='', branch='', checkout='', console=False, desc='', prefix='', depth=None, path_to_repo=None, checkout_only=False, skip_zip_parent_dir=False)[source]

Clone or pull CM repository

Parameters:

  • alias (str) – CM repository alias

  • (url) (str) – Git repository URL

  • (branch) (str) – Git repository branch

  • (checkout) (str) – Git repository checkout

  • (checkout_only) (bool) – only checkout existing repo

  • (depth) (int) – Git repository depth

  • (console) (bool) – if True, print some info to console

  • (desc) (str) – optional repository description

  • (prefix) (str) – sub-directory to be used inside this CM repository to store artifacts

  • (path_to_repo) (str) – force path to repo (useful to pull imported repos with non-standard path)

  • (checkout_only) (bool) – only checkout Git repository but don’t pull

  • (skip_zip_parent_dir) (bool) – skip parent dir in CM ZIP repo (useful when downloading CM repo archives from GitHub)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • (meta) (dict): meta of the CM repository

  • (warnings) (list of str): warnings to install more CM repositories

cmind.utils module

cmind.utils.assemble_cm_object(alias, uid)[source]

Assemble CM object string from alias and uid strings

Parameters:

  • alias (str) – CM artifact alias

  • uid (str) – CM artifact UID

Returns:

(str) CM object (alias,uid)

cmind.utils.assemble_cm_object1(cm_dict)[source]

Assemble CM object string from dictionary with ‘alias’ and ‘uid’ keys

Parameters:

cm_dict (dict) – dictionary with CM artifact keys (‘alias’, ‘uid’)

Returns:

(str) CM object (alias,uid)

cmind.utils.assemble_cm_object2(cm_obj)[source]

Assemble CM object string from tuple

Parameters:

cm_obj (tuple) – CM object tuple (alias, uid)

Returns:

(str) CM object (alias,uid)

cmind.utils.call_internal_module(module_self, path_to_current_module, module_name, module_func, i)[source]

Call CM function from internal submodule

Parameters:

  • path_to_current_module (obj) – must be __file__

  • module_name (str) – module name

  • module_func (str) – module function

  • i (dict) – CM input. Note that i[‘self_module’] = self from calling module will be added to the input

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.check_and_create_dir(path)[source]

Create directories if path doesn’t exist. (from the CK framework).

Parameters:

path (str) – path

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.check_if_none_false_no_off(env, key)[source]

Universal check if str(env.get(key, ‘’)).lower() in [‘false’, ‘no’, ‘off’]:

Parameters:

  • env (dict) – dictionary

  • key (str) – key

Returns:

True if str(env.get(key, ‘’)).lower() in [‘false’, ‘no’, ‘off’]

cmind.utils.check_if_true_yes_on(env, key)[source]

Universal check if str(env.get(key, ‘’)).lower() in [‘true’, ‘yes’, ‘on’]:

Parameters:

  • env (dict) – dictionary

  • key (str) – key

Returns:

True if str(env.get(key, ‘’)).lower() in [‘true’, ‘yes’, ‘on’]

cmind.utils.compare_versions(version1, version2)[source]

Compare versions

Parameters:

  • version1 (str) – version 1

  • version2 (str) – version 2

Returns:

comparison (int) –

1 - version 1 > version 2

0 - version 1 == version 2

-1 - version 1 < version 2

cmind.utils.convert_env_to_dict(s)[source]

Create sub-input from the input using list of keys

Parameters:

s (str) – string with env

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

(dict): dictionary

cmind.utils.convert_tags_to_list(i)[source]

Convert string tags to list

Parameters:

  • (CM input dict)

  • (tags) (str)

Returns:

tags_list (list) – list of tags

cmind.utils.copy_to_clipboard(i)[source]

Copy string to a clipboard

Parameters:

  • string (str) – string to copy to a clipboard

  • (add_quotes) (bool) – add quotes to the string in a clipboard

  • (skip_fail) (bool) – if True, do not fail

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.debug_here(module_path, host='localhost', port=5678, text='', env={}, env_debug_uid='')[source]
cmind.utils.dump_safe_json(i)[source]

Dump safe JSON

Parameters:

(CM input dict) – input to dump to console

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): only “safe” values left (that can be serialized)

cmind.utils.filter_tags(tags)[source]

Filter tags (remove starting with -)

Parameters:

tags (list) – list of tags

Returns:

(list) list of filtered tags

cmind.utils.find_api(file_name, func)[source]

Find automation action API in a Python module

Parameters:

  • file_name (str) – Python module

  • func (str) – automation action name

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • api (str): API

cmind.utils.find_file_in_current_directory_or_above(file_names, path_to_start=None, reverse=False, path_to_stop=None)[source]

Find file(s) in the current directory or above.

Parameters:

  • file_names (list) – files to find

  • (path_to_start) (str) – path to start; use current directory if None

  • (reverse) (bool) – if True search recursively in current directory and below.

  • (path_to_stop) (str) – path to stop search (usually path to found repo)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • found (bool): True if found

  • path_to_file (str): full path to found file

  • path (str): path where found file is

cmind.utils.find_file_in_dir_and_above(filename, path='')[source]

Find file in the current directory or above

Parameters:

  • filename (str)

  • path (str)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • path (str): path where file is found

  • path_to_file (str): path to file

cmind.utils.gen_tmp_file(i)[source]

Generate temporary files

Parameters:

(CM input dict) – (suffix) (str): temp file suffix (prefix) (str): temp file prefix (remove_dir) (bool): if True, remove dir

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • file_name (str): temp file name

cmind.utils.gen_uid()[source]

Generate CM UID.

Parameters:

None

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • uid (str): CM UID

cmind.utils.get_current_date_time(i)[source]

Get current date and time.

Parameters:

(CM input dict) –

  • (timezone) (str): timezone in pytz format: “Europe/Paris”

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • array (dict); dict with date and time

    • date_year (str)

    • date_month (str)

    • date_day (str)

    • time_hour (str)

    • time_minute (str)

    • time_second (str)

  • iso_datetime (str): date and time in ISO format

cmind.utils.get_list_from_cli(i, key)[source]

Get list from CM command line.

Parameters:

  • i (dict) – CM input dict

  • key (str) – key to get

Returns:

tags (list) – list of tags for a given key

cmind.utils.is_cm_uid(obj)[source]

Check if a string is a valid CM UID.

Parameters:

obj (str) – CM alias or UID

Returns:

(bool) – True if valid CM UID 16 hex characters

cmind.utils.is_file_json_or_yaml(file_name)[source]

Is file JSON or YAML?

Parameters:

  • (CM input dict)

  • file_name (str) – path to file without extension

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • is_file (bool): if True, there is a file with YAML and/or JSON extension

cmind.utils.list_all_files(i)[source]

List all files recursively in a given directory. (from the CK framework).

Parameters:

  • (CM input dict)

  • path (str) – top level path

  • (file_name) (str) – search for a specific file name

  • (pattern) (str) – return only files with this pattern

  • (path_ext) (str) – path extension (needed for recursion)

  • (limit) (str) – limit number of files (if directories with a large number of files)

  • (number) (int) – current number of files

  • (all) (str) – if ‘yes’ do not ignore special directories (like .cm)

  • (ignore_names) (list) – list of names to ignore

  • (ignore_symb_dirs) (str) – if ‘yes’, ignore symbolically linked dirs (to avoid recursion such as in LLVM)

  • (add_path) (str)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • list (dict): dictionary of all files:

    {“file_with_full_path”:{“size”:.., “path”:..}

  • number (int): (internal) total number of files in a current directory (needed for recursion)

cmind.utils.load_bin(file_name)[source]

Load binary file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • bin (bytes): file content

cmind.utils.load_json(file_name, check_if_exists=False, encoding='utf8')[source]

Load JSON file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name

  • (check_if_exists) (bool) – if True, fail if doesn’t exist

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): meta from the file

cmind.utils.load_json_or_yaml(file_name, check_if_exists=False, encoding='utf8')[source]

Attempt to load file as JSON or YAML.

Parameters:

  • (CM input dict)

  • file_name (str) – file name that has either JSON or YAML extension

  • (check_if_exists) (bool) – if True, fail if doesn’t exist

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): merged meta of found file(s)

cmind.utils.load_python_module(i)[source]

Load python module

Parameters:

(CM input dict) – path (str): path to a python module name (str): Python module name (without .py)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.load_txt(file_name, encoding='utf8', remove_after_read=False, check_if_exists=False, split=False, match_text='', fail_if_no_match='', debug=False)[source]

Load text file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name

  • (encoding) (str) – file encoding (‘utf8’ by default)

  • (remove_after_read) (bool) – if True, remove file after read (False by default)

  • (check_if_exists) (bool) – If True, check if file exists and return CM error instead of raising error (False by default)

  • (split) (bool) – If True, split string into list (False by default)

  • (match_text) (str) – Regular expression to match text (useful for version detection)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • string (str): string from file

cmind.utils.load_yaml(file_name, check_if_exists=False, encoding='utf8')[source]

Load YAML file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name

  • (check_if_exists) (bool) – if True, fail if doesn’t exist

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): meta from the file

cmind.utils.load_yaml_and_json(file_name_without_ext, check_if_exists=False, encoding='utf8')[source]

Load YAML file if exists then JSON file if exists and merge with the first one

Parameters:

  • (CM input dict)

  • file_name_without_ext (str) – file name without extension (to check yaml and then json)

  • (check_if_exists) (bool) – if True, fail if doesn’t exist

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): merged meta of found file(s)

cmind.utils.match_objects(uid, alias, uid2, alias2, more_strict=False)[source]

Check if 2 CM objects match. Used to search CM artifacts in CM repositories.

Examples

281d5c3e3f69d8e7,* == 281d5c3e3f69d8e7,*

281d5c3e3f69d8e7,os == ,os

,os == 281d5c3e3f69d8e7,os

,* != 281d5c3e3f69d8e7,*

os

281d5c3e3f69d8e7

os,281d5c3e3f69d8e7

281d5c3e3f69d8e7,os

Parameters:
  • uid (str) – artifact UID

    • alias (str) – atifact alias that can’t have wildcards [real CM object]

    • uid2 (str) – atifact 2 UID

    • alias2 (str)

(more_strict) (bool): if True, then ,os != 281d5c3e3f69d8e7, [needed to check automation]

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • match (bool): True if 2 CM objects match (uid and/or alias)

cmind.utils.merge_dicts(i)[source]

Merge intelligently dict1 with dict2 key by key in contrast with dict1.update(dict2)

It can merge sub-dictionaries and lists instead of substituting them

Parameters:

  • (CM input dict)

  • dict1 (dict) – merge this dict with dict2 (will be directly modified!)

  • dict2 (dict) – dict to be merged

  • (append_lists) (bool) – if True, append lists instead of creating the new ones

  • (append_unique) (bool) – if True, append lists when value doesn’t exists in the original one

  • (ignore_keys) (list) – ignore keys

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • dict1 (dict): dict1 passed through the function

cmind.utils.parse_cm_object(obj, max_length=2)[source]

Parse CM object in string and return tuple of CM objects.

Examples

CM sub-object = UID | alias | alias,UID | UID,alias

repo CM sub-object | CM sub-object

cm os

cm 281d5c3e3f69d8e7

cm os,281d5c3e3f69d8e7

cm 281d5c3e3f69d8e7,os

cm octoml@mlops,os

cm octoml@mlops,dbfa91645e429380:os,281d5c3e3f69d8e7

cm dbfa91645e429380:281d5c3e3f69d8e7

Parameters:

  • obj (str) – CM object

  • (max_length) (int)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • cm_object (list): first element: CM alias | UID

    (second element: CM repo | UID)

cmind.utils.process_meta_for_inheritance(i)[source]

Check CM meta description for inheritance and update it if needed (“_base”:”automation::artifact”).

Parameters:

  • (CM input dict)

  • automation (str) – CM automation

  • meta (dict) – original meta

  • cmind (obj) – initialized CM to search for base artifacts

  • (base_recursion) (int) – track recursion during inheritance

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): final meta description wihhout _base

cmind.utils.rm_read_only(f, p, e)[source]

Internal aux function to remove files and dirs even if read only particularly on Windows

cmind.utils.safe_load_json(path, file_name='', encoding='utf8')[source]

Load JSON file if exists, otherwise return empty dict

Parameters:

  • (CM input dict)

  • file_name (str) – file name

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

  • meta (dict): meta from the file

cmind.utils.save_json(file_name, meta={}, indent=2, sort_keys=True, encoding='utf8')[source]

Save meta to JSON file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name

  • meta (dict) – meta to save

  • (indent) (int) – 2 by default

  • (sort_keys) (bool) – if True, sort keys

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.save_json_or_yaml(file_name, meta, sort_keys=False, encoding='utf8')[source]

Save meta to either JSON or YAML file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name that has either JSON or YAML extension

  • meta (dict) – meta to save

  • (sort_keys) (bool) – if True, sort keys

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.save_txt(file_name, string='', encoding='utf8')[source]

Save string to text file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name

  • string (str) – string to save

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.save_yaml(file_name, meta={}, sort_keys=True, encoding='utf8')[source]

Save meta to YAML file.

Parameters:

  • (CM input dict)

  • file_name (str) – file name

  • meta (dict) – meta to save

  • (sort_keys) (bool) – if True, sort keys

  • (encoding) (str) – file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.sub_input(i, keys, reverse=False)[source]

Create sub-input from the input using list of keys

Parameters:

  • i (dict) – input

  • keys (list) – keys not to add if reverse if True

  • reverse (bool) – either add or skip keys

Returns:

(dict) – sub-input

cmind.utils.tags_matched(tags, and_tags, no_tags)[source]

Check if AND tags and NO tags match tags

Parameters:

  • tags (list of str) – full list of tags

  • and_tags (list of str) – list of AND tags

  • no_tags (list of str) – list of NO tags

Returns:

True if tags matched

cmind.utils.update_dict_if_empty(d, key, value)[source]

Update dictionary if “key” is empty

Parameters:

  • d (dict) – dict to check

  • key (str)

  • value – if d[key] is empty, set to this value

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

cmind.utils.update_yaml(file_name, meta={}, encoding='utf8')[source]

Update yaml file directly (unsafe - only first keys)

Parameters:

(CM input dict) – file_name (str): YAML file name meta (dict): keys to update (encoding) (str): file encoding (‘utf8’ by default)

Returns:

(CM return dict) –

  • return (int): return code == 0 if no error and >0 if error

  • (error) (str): error string if return>0

Module contents