News: we have successfully completed the prototyping phase of the Collective Knowledge technology to make it easier to reproduce AI&ML and deploy it in production with the help of portable CK workflows, reusable artifacts and MLOps as described in this white paper and the CK presentation. We are now preparing the second phase of this project to make CK simpler to use, more stable and more user friendly - don't hesitate to get in touch with the CK author to know more!
Developing novel applications based on deep tech (ML, AI, HPC, quantum, IoT) and deploying them in production is a very painful, ad-hoc, time consuming and expensive process due to continuously evolving software, hardware, models, data sets and research techniques.
After struggling with these problems for many years, we started the Collective Knowledge project (CK) to decompose complex systems and research projects into reusable, portable, customizable and non-virtualized CK components with the unified automation actions, Python APIs, CLI and JSON meta descriptions.
Our idea is to gradually abstract all existing artifacts (software, hardware, models, data sets, results) and use the DevOps methodology to connect such components together into functional CK solutions. Such solutions can automatically adapt to evolving models, data sets and bare-metal platforms with the help of customizable program workflows, a list of all dependencies (models, data sets, frameworks), and a portable meta package manager.
CK is basically our intermediate language to connect researchers and practitioners to collaboratively design, benchmark, optimize and validate innovative computational systems. It then makes it possible to find the most efficient system configutations on a Pareto frontier (trading off speed, accuracy, energy, size and different costs) using an open repository of knowledge with live SOTA scoreboards and reproducible papers.
We use CK to complement related reproducibility initiatives including MLPerf, PapersWithCode, ACM artifact review and badging, and artfact evaluation. See the CK use cases from our partners and try our MLPerf automation demo on your platform.

Even though the CK technology is used in production for more than 5 years, it is still a proof-of-concept prototype requiring further improvements and standardization. Depending on the available resources, we plan to develop a new, backward-compatible and more user-friendly version - please get in touch if you are interested to know more!

Several presentations about CK (General Motors, ACM and FOSDEM)

CK attempts to bring DevOps principles to computational research while abstracting, unifying and connecting popular tools and services instead of substituting them

This is an ongoing community project and there is still a lot to be done. Don't hesitate to get in touch if you have any suggestions or encounter any issues! Thank you!