Join open MLCommons task force on automation and reproducibility to participate in the collaborative development of the Collective Knowledge v3 playground (MLCommons CK) powered by the Collective Mind automation language (MLCommons CM) and Modular Inference Library (MIL) to automate benchmarking, optimization, design space exploration and deployment of Pareto-efficient AI/ML applications across any software and hardware from the cloud to the edge!

Android demo of real AI apps automatically assembled from CK components

[ Project overview, Reddit disccusion ]

You can download and install the latest version of our simple Android demo to participate in collaborative benchmarking, optimization, autotuning and co-design of AI-based applications: APK Version 2.12.

Just make sure that your Android device is configured to allow installation from sources other than the Play Store before downloading and installing our CK-based Android apps. You can check it as follows:

Go to "Settings" of your Android device

Tap on "Lock screen and security" or "Security" (depending on the version of your Android device)

Check the box next to "Unknown sources" (Allow installation of apps from sources other than the Play Store). Note that you can always revert it back to original setting after installing our apps.

You can see all aggregated results from these experiments (speed, accuracy, costs) across diverse mobile phones in the public repository (note that we gradually move all CK SOTA scoreboards here). You can also check the collective training set with image misclassifications.

Our applicaiton downloads slightly modified AI pipelines assembled from cKnowledge components (frameworks, models, datasets) and runs them across diverse platforms and environments provided by volunteers similar to SETI@home to collaboratively find the most efficient AI solutions in terms of speed, accuracy, costs and other metrics or automatically detect software and hardware bugs (see paper 1, paper 2 and paper 3).

We use this open-source technology is used in the open and reproducible ACM ReQuEST tournaments to co-design Pareto-efficient deep learning stacks (speed, accuracy, costs) and in the open project with Raspberry Pi foundation to crowdsource compiler autotuning and machine learning!

You can check out another Android application to help crowdsource benchmarking and autotuning of popular compilers (LLVM, GCC): You can now download, install and run two CK-based Android applications to

Crowdsource LLVM and GCC benchmarking and autotuning (V2.5)

Authors

This application was originally developed by Grigori Fursin with a user-friendly GUI developed by our partners. See this white paper for our future plans.