This is the second post in a series about tiny machine learning (TinyML) at the deep IoT edge. Read our earlier introduction to TinyMl as-a-Service, to learn how it ranks in respect to traditional cloud-based machine learning or the embedded systems domain.
TinyML is an emerging concept (and community) to run ML inference on Ultra Low-Power (ULP ~1mW) microcontrollers. TinyML as a Service will democratize TinyML, allowing manufacturers to start their AI business with TinyML running on microcontrollers.
In this article, we introduce the challenges behind the applicability of ML concepts within the IoT embedded world. Furthermore, we emphasize how these challenges are not simply due to the constraints added by the limited capabilities of embedded devices but are also evident where the computation capabilities of ML-based IoT deployments are empowered by additional resources confined at the network edge.
To summarize the nature of these challenges, we can say:
Below, we take a closer look at each of these challenges.
Edge computing promises higher performing service provisioning, both from a computational and a connectivity point of view.
Edge nodes support the latency requirements of mission critical communications thanks to their proximity to the end-devices, and enhanced hardware and software capabilities allow execution of increasingly complex and resource-demanding services in the edge nodes. There is growing attention, investments and R&D to make execution of ML tasks at the network edge easier. In fact, there are already several ML-dedicated "edge" hardware examples (e.g. Edge TPU by Google, Jetson Nano by Nvidia, Movidius by Intel) which confirm this.
Therefore, the question we are asking is: what are the issues that the edge computing paradigm has not been able to completely solve yet? And how can these issues undermine the applicability of ML concepts in IoT and edge computing scenarios?
We intend to focus on and analyze five areas in particular: (Note: Some areas we describe below may have solutions through other emerging types of edge computing but are not yet commonly available).
Figure 1
The web and the embedded worlds feature very heterogeneous characteristics. Figure 1 (above) depicts how this high heterogeneity is characterized, by comparing qualitatively and quantitively the capacities of the two paradigms both from a hardware and software perspective. Web services can rely on powerful underlying CPU architectures with high memory and storage capabilities. From a software perspective, web technologies can be designed to choose and benefit from a multitude of sophisticated operating systems (OS) and complex software tools.
On the other hand, embedded systems can rely on the limited capacity of microcontroller units (MCUs) and CPUs that are much less powerful when compared with general-purpose and consumer CPUs. The same applies with memory and storage capabilities, where 500KB of SRAM and a few MBs of FLASH memory can already be considered a high resource. There have been several attempts to bring the flexibility of Linux-based systems in the embedded scenario (e.g. Yocto Project), but nevertheless most of 32bit MCU-based devices owns the capacity for running real-time operating systems and no more complex distribution.
In simple terms, when Linux can run, system deployment is made easier since software portability becomes straightforward. Furthermore, an even higher cross-platform software portability is also made possible thanks to the wide support and usage of lightweight virtualization technologies such as containers. With almost no effort, developers can basically ship the same software functionalities between entities operating under Linux distributions, as happens in the case of cloud and edge.
The impossibility of running Linux and container-based virtualization in MCUs represents one of the most limiting issue and bigger challenge for current deployments. In fact, it appears clear how in typical "cloud-edge-embedded devices" scenarios, cloud and edge services are developed and deployed with hardware and software technologies, which are fundamentally different and easier to be managed if compared to embedded technologies.
TinyML as-a-Service tries to tackle this issue by taking advantage of alternative (and lightweight) software solutions.
Figure 2
In the previous section, we considered on a high-level how the technological differences between web and embedded domains can implicitly and significantly affect the execution of ML tasks on IoT devices. Here, we analyze how a big technological gap exists also in the availability of ML-dedicated hardware and software web, edge, and embedded entities.
From a hardware perspective, during most of computing history there have been only a few types of processor, mostly available for general use. Recently, the relentless growth of artificial intelligence (AI) has led to the optimization of ML tasks for existing chip designs such as graphics processing units (GPUs), as well as the design of new dedicated hardware forms such as application specific integrated circuits (ASICs), which embed chips designed exclusively for the execution of specific ML operations. The common thread that connects all these new devices is their usage at the edge. In fact, these credit-card sized devices are designed with the idea of operating at the network edge.
At the beginning of this article we mentioned a few examples of this new family of devices (Edge TPU, Jetson Nano, Movidius). We foresee that in the near future even more big and small chip and hardware manufacturers will increasingly invest resources into the design and production of ML-dedicated hardware. However, it appears clear how, at least so far, there has not been the same effort in the embedded world.
Such a lack of hardware availability undermines somehow a homogeneous and seamless ML "cloud-to-embedded" deployments. In many scenarios, the software can help compensate for hardware deficiencies. However, the same boundaries that we find in the hardware sphere apply for the development of software tools. Today, in the web domain, there are hundreds of ML-oriented application software. Such availability is registering a constant growth thanks also to the possibility given by the different open source initiatives that allow passionate developers all over the world to merge efforts. The result is more effective, refined, and niche applications. However, the portability of these applications into embedded devices is not so straightforward. The usage of high-level programming languages (e.g., Python), as well as the large sizes of the software runtime (intended as both runtime system and runtime program lifecycle phase) are just some of the reasons why the software portability is painful if not impossible.
The main rationale behind the TinyML as-a-Service approach is precisely the one to break the existing wall between cloud/edge and embedded entities. However, to expect exactly the same ML experience in the embedded domain as we have in the web and enterprise world would be unrealistic. It is still an irrefutable fact that size matters. The execution of ML inference is the only operation that we reasonably foresee to be executed in an IoT device. We are happy to leave all the other cumbersome ML tasks, such as data processing and training, to the more equipped and resourceful side of the scenario depicted in Figure 2.
In the next article, we will go through the different features which characterize TinyML as-a-Service and share the technological approach underlying the TinyML as-a-Service concept.
In the meantime, if you have not read it yet, we recommend reading our earlier introduction to TinyMl as-a-Service.
The IoT world needs a complete ML experience. TinyML as-a-service can be one possible solution for making this enhanced experience possible, as well as expanding potential technology opportunities. Stay tuned!
Continued here:
TinyML as a Service and machine learning at the edge - Ericsson
- Microsoft reveals how it caught mutating Monero mining malware with machine learning - The Next Web [Last Updated On: December 1st, 2019] [Originally Added On: December 1st, 2019]
- The role of machine learning in IT service management - ITProPortal [Last Updated On: December 1st, 2019] [Originally Added On: December 1st, 2019]
- Workday talks machine learning and the future of human capital management - ZDNet [Last Updated On: December 1st, 2019] [Originally Added On: December 1st, 2019]
- Verification In The Era Of Autonomous Driving, Artificial Intelligence And Machine Learning - SemiEngineering [Last Updated On: December 1st, 2019] [Originally Added On: December 1st, 2019]
- Synthesis-planning program relies on human insight and machine learning - Chemical & Engineering News [Last Updated On: December 1st, 2019] [Originally Added On: December 1st, 2019]
- Here's why machine learning is critical to success for banks of the future - Tech Wire Asia [Last Updated On: December 1st, 2019] [Originally Added On: December 1st, 2019]
- The 10 Hottest AI And Machine Learning Startups Of 2019 - CRN: The Biggest Tech News For Partners And The IT Channel [Last Updated On: December 1st, 2019] [Originally Added On: December 1st, 2019]
- Onica Showcases Advanced Internet of Things, Artificial Intelligence, and Machine Learning Capabilities at AWS re:Invent 2019 - PR Web [Last Updated On: December 3rd, 2019] [Originally Added On: December 3rd, 2019]
- Machine Learning Answers: If Caterpillar Stock Drops 10% A Week, Whats The Chance Itll Recoup Its Losses In A Month? - Forbes [Last Updated On: December 3rd, 2019] [Originally Added On: December 3rd, 2019]
- Amazons new AI keyboard is confusing everyone - The Verge [Last Updated On: December 5th, 2019] [Originally Added On: December 5th, 2019]
- Exploring the Present and Future Impact of Robotics and Machine Learning on the Healthcare Industry - Robotics and Automation News [Last Updated On: December 5th, 2019] [Originally Added On: December 5th, 2019]
- 3 questions to ask before investing in machine learning for pop health - Healthcare IT News [Last Updated On: December 5th, 2019] [Originally Added On: December 5th, 2019]
- Amazon Wants to Teach You Machine Learning Through Music? - Dice Insights [Last Updated On: December 5th, 2019] [Originally Added On: December 5th, 2019]
- Measuring Employee Engagement with A.I. and Machine Learning - Dice Insights [Last Updated On: December 6th, 2019] [Originally Added On: December 6th, 2019]
- The NFL And Amazon Want To Transform Player Health Through Machine Learning - Forbes [Last Updated On: December 11th, 2019] [Originally Added On: December 11th, 2019]
- Scientists are using machine learning algos to draw maps of 10 billion cells from the human body to fight cancer - The Register [Last Updated On: December 11th, 2019] [Originally Added On: December 11th, 2019]
- Appearance of proteins used to predict function with machine learning - Drug Target Review [Last Updated On: December 11th, 2019] [Originally Added On: December 11th, 2019]
- Google is using machine learning to make alarm tones based on the time and weather - The Verge [Last Updated On: December 11th, 2019] [Originally Added On: December 11th, 2019]
- 10 Machine Learning Techniques and their Definitions - AiThority [Last Updated On: December 11th, 2019] [Originally Added On: December 11th, 2019]
- Taking UX and finance security to the next level with IBM's machine learning - The Paypers [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Government invests 49m in data analytics, machine learning and AI Ireland, news for Ireland, FDI,Ireland,Technology, - Business World [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Machine Learning Answers: If Nvidia Stock Drops 10% A Week, Whats The Chance Itll Recoup Its Losses In A Month? - Forbes [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Bing: To Use Machine Learning; You Have To Be Okay With It Not Being Perfect - Search Engine Roundtable [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- IQVIA on the adoption of AI and machine learning - OutSourcing-Pharma.com [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Schneider Electric Wins 'AI/ Machine Learning Innovation' and 'Edge Project of the Year' at the 2019 SDC Awards - PRNewswire [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Industry Call to Define Universal Open Standards for Machine Learning Operations and Governance - MarTech Series [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Qualitest Acquires AI and Machine Learning Company AlgoTrace to Expand Its Offering - PRNewswire [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Automation And Machine Learning: Transforming The Office Of The CFO - Forbes [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- Machine learning results: pay attention to what you don't see - STAT [Last Updated On: December 12th, 2019] [Originally Added On: December 12th, 2019]
- The challenge in Deep Learning is to sustain the current pace of innovation, explains Ivan Vasilev, machine learning engineer - Packt Hub [Last Updated On: December 15th, 2019] [Originally Added On: December 15th, 2019]
- Israelis develop 'self-healing' cars powered by machine learning and AI - The Jerusalem Post [Last Updated On: December 15th, 2019] [Originally Added On: December 15th, 2019]
- Theres No Such Thing As The Machine Learning Platform - Forbes [Last Updated On: December 15th, 2019] [Originally Added On: December 15th, 2019]
- Global Contextual Advertising Markets, 2019-2025: Advances in AI and Machine Learning to Boost Prospects for Real-Time Contextual Targeting -... [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Machine Learning Answers: If Twitter Stock Drops 10% A Week, Whats The Chance Itll Recoup Its Losses In A Month? - Forbes [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Tech connection: To reach patients, pharma adds AI, machine learning and more to its digital toolbox - FiercePharma [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Machine Learning Answers: If Seagate Stock Drops 10% A Week, Whats The Chance Itll Recoup Its Losses In A Month? - Forbes [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- MJ or LeBron Who's the G.O.A.T.? Machine Learning and AI Might Give Us an Answer - Built In Chicago [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Amazon Releases A New Tool To Improve Machine Learning Processes - Forbes [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- AI and machine learning platforms will start to challenge conventional thinking - CRN.in [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- What is Deep Learning? Everything you need to know - TechRadar [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Machine Learning Answers: If BlackBerry Stock Drops 10% A Week, Whats The Chance Itll Recoup Its Losses In A Month? - Forbes [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- QStride to be acquired by India-based blockchain, analytics, machine learning consultancy - Staffing Industry Analysts [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Dotscience Forms Partnerships to Strengthen Machine Learning - Database Trends and Applications [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- The Machines Are Learning, and So Are the Students - The New York Times [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Kubernetes and containers are the perfect fit for machine learning - JAXenter [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Data science and machine learning: what to learn in 2020 - Packt Hub [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- What is Machine Learning? A definition - Expert System [Last Updated On: December 20th, 2019] [Originally Added On: December 20th, 2019]
- Want to dive into the lucrative world of deep learning? Take this $29 class. - Mashable [Last Updated On: December 24th, 2019] [Originally Added On: December 24th, 2019]
- Another free web course to gain machine-learning skills (thanks, Finland), NIST probes 'racist' face-recog and more - The Register [Last Updated On: December 24th, 2019] [Originally Added On: December 24th, 2019]
- Machine Learning in 2019 Was About Balancing Privacy and Progress - ITPro Today [Last Updated On: December 24th, 2019] [Originally Added On: December 24th, 2019]
- Ten Predictions for AI and Machine Learning in 2020 - Database Trends and Applications [Last Updated On: December 25th, 2019] [Originally Added On: December 25th, 2019]
- The Value of Machine-Driven Initiatives for K12 Schools - EdTech Magazine: Focus on Higher Education [Last Updated On: December 25th, 2019] [Originally Added On: December 25th, 2019]
- CMSWire's Top 10 AI and Machine Learning Articles of 2019 - CMSWire [Last Updated On: December 25th, 2019] [Originally Added On: December 25th, 2019]
- Machine Learning Market Accounted for US$ 1,289.5 Mn in 2016 and is expected to grow at a CAGR of 49.7% during the forecast period 2017 2025 - The... [Last Updated On: December 27th, 2019] [Originally Added On: December 27th, 2019]
- Are We Overly Infatuated With Deep Learning? - Forbes [Last Updated On: December 27th, 2019] [Originally Added On: December 27th, 2019]
- Can machine learning take over the role of investors? - TechHQ [Last Updated On: December 27th, 2019] [Originally Added On: December 27th, 2019]
- Dr. Max Welling on Federated Learning and Bayesian Thinking - Synced [Last Updated On: December 28th, 2019] [Originally Added On: December 28th, 2019]
- 2010 2019: The rise of deep learning - The Next Web [Last Updated On: January 4th, 2020] [Originally Added On: January 4th, 2020]
- Machine Learning Answers: Sprint Stock Is Down 15% Over The Last Quarter, What Are The Chances It'll Rebound? - Trefis [Last Updated On: January 4th, 2020] [Originally Added On: January 4th, 2020]
- Sports Organizations Using Machine Learning Technology to Drive Sponsorship Revenues - Sports Illustrated [Last Updated On: January 4th, 2020] [Originally Added On: January 4th, 2020]
- What is deep learning and why is it in demand? - Express Computer [Last Updated On: January 4th, 2020] [Originally Added On: January 4th, 2020]
- Byrider to Partner With PointPredictive as Machine Learning AI Partner to Prevent Fraud - CloudWedge [Last Updated On: January 4th, 2020] [Originally Added On: January 4th, 2020]
- Stare into the mind of God with this algorithmic beetle generator - SB Nation [Last Updated On: January 5th, 2020] [Originally Added On: January 5th, 2020]
- US announces AI software export restrictions - The Verge [Last Updated On: January 5th, 2020] [Originally Added On: January 5th, 2020]
- How AI And Machine Learning Can Make Forecasting Intelligent - Demand Gen Report [Last Updated On: January 5th, 2020] [Originally Added On: January 5th, 2020]
- Fighting the Risks Associated with Transparency of AI Models - EnterpriseTalk [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- NXP Debuts i.MX Applications Processor with Dedicated Neural Processing Unit for Advanced Machine Learning at the Edge - GlobeNewswire [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Cerner Expands Collaboration with Amazon Web as its Preferred Machine Learning Provider - Story of Future [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Can We Do Deep Learning Without Multiplications? - Analytics India Magazine [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Machine learning is innately conservative and wants you to either act like everyone else, or never change - Boing Boing [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Pear Therapeutics Expands Pipeline with Machine Learning, Digital Therapeutic and Digital Biomarker Technologies - Business Wire [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- FLIR Systems and ANSYS to Speed Thermal Camera Machine Learning for Safer Cars - Business Wire [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- SiFive and CEVA Partner to Bring Machine Learning Processors to Mainstream Markets - PRNewswire [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Tiny Machine Learning On The Attiny85 - Hackaday [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Finally, a good use for AI: Machine-learning tool guesstimates how well your code will run on a CPU core - The Register [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- AI, machine learning, and other frothy tech subjects remained overhyped in 2019 - Boing Boing [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Chemists are training machine learning algorithms used by Facebook and Google to find new molecules - News@Northeastern [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- AI and machine learning trends to look toward in 2020 - Healthcare IT News [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- What Is Machine Learning? | How It Works, Techniques ... [Last Updated On: January 7th, 2020] [Originally Added On: January 7th, 2020]
- Achieving Paperless Operations and Document Automation with AI and ML - ReadWrite [Last Updated On: January 8th, 2020] [Originally Added On: January 8th, 2020]