Collective Knowledge framework (CK)
helps to organize any software project as a database of reusable components
(algorithms, datasets, models, frameworks, scripts, experimental results, papers, etc)
with common automation actions and extensible meta descriptions
based on FAIR principles (findability, accessibility, interoperability and reusability).
The goal is to make it easier for researchers, practitioners and students to reproduce, compare
and build upon techniques from published papers shared in the common CK format,
adopt them in production and reuse best R&D practices.
See how the CK technology helps to automate benchmarking, optimization and design space exploration
of ML Systems and accelerate AI, ML and System innovation:
ACM tech talk,
and some real-world use cases
from MLPerf, General Motors, Arm, IBM, Amazon, Qualcomm, DELL, the Raspberry Pi foundation and ACM.
Presentations about CK (ACM, General Motors and FOSDEM)
Developing innovative technology (AI, ML, quantum, IoT),
and deploying it in the real world is a very painful, ad-hoc, time consuming and expensive process
due to continuously evolving software, hardware, models, data sets and research techniques.
After struggling with these problems for many years, we started developing the
Collective Knowledge framework (CK)
to decompose complex research projects into reusable automation actions
and components with unified APIs, CLI and JSON meta descriptions.
These CK components help to abstract software, hardware, models, data sets, scripts, results
and can be connected into portable CK workflows
while applying powerful DevOps principles.
Such workflows use a portable meta package manager
with a list all dependencies
to automatically adapt to a given platform (detect hardware, software and environment, install missing packages and build/optimize code).
We have spent the last few years testing CK with our
great academic and industrial partners
as a playground to implement and share reusable automation actions
and components typical in AI, ML and systems R&D
while agreeing on common APIs and JSON meta description.
We used such components to assemble portable workflows
from reproduced research papers
during the artifact evaluation process
that we have helped to arrange at different ML&systems conferences including ASPLOS, CGO, PPoPP and MLSys.
We then demonstrated that it was possible to use such portable workflows
to automate design space exploration of AI/ML/SW/HW stacks,
automate MLPerf benchmark submissions,
and simplify deployment of ML Systems in production
in the most efficient way (speed, accuracy, energy, costs)
across diverse platforms from data centers to edge devices.
CK framework is basically a common playground that connects researchers and practitioners
to learn how to collaboratively design, benchmark, optimize and validate innovative computational technology
including self-optimizing and bio-inspired computing systems.
Our mission is to organize and systematize all our AI, ML and systems knowledge
in the form of portable workflow, automation actions and reusable artifacts
using our open
with reproducible papers
and live SOTA scoreboards for crowd-benchmarking.
We continue using CK to support related initiatives including MLPerf, PapersWithCode,
ACM artifact review and badging,
and artfact evaluation.
Even though the
is used in production
for more than 5 years, it is still a proof-of-concept prototype requiring further improvements
You can check the following resources to know more about our long-term vision:
Collective Tuning Initiative,
We are preparing the second phase of this project - please get in touch if you are interested to know more!
This is an ongoing community project and there is still a lot to be done.
Don't hesitate to
get in touch
if you have any suggestions or encounter any issues!