Join open MLCommons task force on automation and reproducibility to participate in the collaborative development of the Collective Knowledge v3 playground (MLCommons CK) powered by the Collective Mind automation language (MLCommons CM) and Modular Inference Library (MIL) to automate benchmarking, optimization, design space exploration and deployment of Pareto-efficient AI/ML applications across any software and hardware from the cloud to the edge!

CK-based Android applications to crowdsource AI/SW/HW co-design experiments

[ Project overview, Reddit disccusion ]

You can download and install two CK-based applications for your Android device to participate in collaborative benchmarking, optimization, autotuning and co-design of efficient model/SW/HW stacks for emerging workloads. You can see all aggregated results from these experiments (speed, accuracy, costs) across diverse mobile phones in the public repository (note that we gradually move all CK SOTA scoreboards here). We use this open-source technology in open and reproducible ACM ReQuEST tournaments to co-design Pareto-efficient deep learning stacks (speed, accuracy, costs) and in a joint project with Raspberry Pi foundation to crowdsource compiler autotuning and machine learning (see interactive article).

To our surprise, our open science applications were removed from Google Play in August 2018 after 3 years of successful participation by thousands of users because it apparently violates recent Google's terms of use: it's not allowed to download extra code or data from external sources. However, the whole point of our applications and our novel crowd-tuning concept is to download slightly differently modified code and models to different users to collaboratively find the most efficient solutions across diverse platforms and environments provided by volunteers similar to SETI@home in terms of speed, accuracy, costs and other metrics or automatically detect software and hardware bugs (see paper 1, paper 2, paper 3 and paper 4).

While we are discussing with Google how to move our applications back to Google Play Store, you can download and install them here. Just make sure that your Android device is configured to allow installation from sources other than the Play Store before downloading and installing our CK-based Android apps. You can check it as follows:

• Go to "Settings" of your Android device
• Tap on "Lock screen and security" or "Security" (depending on the version of your Android device)
• Check the box next to "Unknown sources" (Allow installation of apps from sources other than the Play Store). Note that you can always revert it back to original setting after installing our apps.

You can now download, install and run two CK-based Android applications to

• Crowdsource DNN benchmarking, optimization and SW/HW co-design (V2.12)
  • View results
  • See sources
• Crowdsource LLVM and GCC benchmarking and autotuning (V2.5)
  • View GCC results
  • View LLVM results
  • See open sources
  • View interactive report (Raspberry Pi foundation)

Authors

This application was originally developed by Grigori Fursin with a user-friendly GUI developed by our partners. See this white paper for our future plans.