Artificial Intelligence AI Solutions on Edge Devices
Artificial Intelligence (AI) Solutions on Edge Devices
Artificial Intelligence (AI) Solutions, particularly those based on DeepLearning in the areas of Computer Vision, are done in a cloud-basedenvironment requiring heavy computing capacity.Inference is a relatively lower compute-intensive task than training, wherelatency is of greater importance for providing real-time results on a model.Most inference is still performed in the cloud or on a server, but as thediversity of AI applications grows, the centralized training and inferenceparadigm is coming into question.Artificial Intelligence (AI) & Computer Vision Solutions on Edge DevicesIt is possible, and becoming easier, to run AI and Machine Learning withanalytics at the Edge today, depending on the size and scale of the Edge siteand the particular system being used. While Edge site computing systems aremuch smaller than those found in central data centers, they have matured, andnow successfully run many workloads due to an immense growth in the processingpower of today’s x86 commodity servers. It’s quite amazing how many workloadscan now run successfully at the Edge.
Some Applications of AI deployed on the Edge
AI is powering a lot of visual and audio intelligence and enables newinteresting and valuable use cases. Some examples include: * Security and home camera: Smart detection for when important activities are happening and not requiring 24/7 video streaming (for example, detect a person rather than a smart vacuum cleaner robot). * Virtual assistant (smart speaker, phone, etc.): Personalization for natural and intuitive conversations and visual interfaces.Face Recognition Deep Learning Models on Raspberry Pi 3+ & Intel Movidus NCS 2 * Phones: Naturally, the smartphone is the pervasive platform for AI. Your phone will detect your context, such as if you are in the car. You can also apply machine learning to smartphones for a better user experience, such as improved power management for better battery life, enhanced photography, and on-device malware detection. And many other examples. * Smart transportation: On-device AI is beneficial, for example, for sending less data to the cloud in order to know how many seats are available on a bus. * Industrial IoT: Automating the factory of the future will require lots of AI, from visual inspection for defects and intricate robotic control for assembly. * Drones/robots: Self navigation in unknown environment as well as coordination with other drones/robots. * Auto: Machine learning for passenger safety, scene understanding, sensor fusion, path planning, etc. The huge, real benefit of autonomous driving is saving lives and time.
IoT and IT/OT convergence
The internet of things, especially when paired with edge computing, enablesthe IT portion of IT/OT convergence. As mentioned, OT devices are nottraditionally networked technology. IoT devices, by definition, are networkedcomputing devices with the ability to collect, transfer and analyze data.Traditional OT devices, sensors for example, can collect data, but themselvescannot transmit the data over a large network or perform any sort of in-depthanalysis on that data.Newer smart sensors, however, would be able to collect the data from thesource, like a factory floor, and transmit it to an IoT hub or gateway; whichwould then transfer that information to an analytics application or anenterprise resource planning (ERP) software platform, to be integrated into anorganization’s unified system of business operations. When networked, an OTdevice functions as an IoT device. In the factory floor example, a sensor cancollect operational data on materials or machines in the factory and send itover a wireless network to the back-end system application to be interpretedand trigger an action — maintenance on factory equipment, for example.The addition of edge computing capabilities to industrial internet of things(IIoT) devices allows for processing of real-time data closer to the source.Instead of sending the data over a network to a centralized location forprocessing, the IIoT devices can analyze time-sensitive manufacturing processdata and return insights quickly for direct monitoring of industrialconditions, before it becomes obsolete. This is important because IIoT and OTdevices are often part of a distributed network architecture, makingtransmission to a central processing location difficult or impossible. Thesedevices are also often responsible for critical industrial systems, that, ifshut down or interrupted, would incur severe consequences.The addition of IT technology to OT allows organizations to make better use ofdata that’s generated by OT through IoT devices and edge computing.
IoT device characteristics
As an IoT landscape develops, new devices and platforms will be introduced.There are key characteristics that are common to IoT devices that provide abasis of comparison when selecting hardware and software to configure a newIoT network or to develop and expand an existing one.IoT devices may be characterized by capabilities: * Data acquisition and control * Data processing and storage * Connectivity * Power management
Data processing and storage
IoT devices require specific data processing and storage capability. Thishelps achieve data aggregation, transmission, and analysis. Some IoT devicesmay process data directly, while others transmit this data to other devices,gateway devices, or cloud applications for further aggregation and analysis.Edge analytics performs data analysis at the edges of a network rather than ina centralized location. Data can be analyzed in realtime on the devicesthemselves, or it can be analytized on a nearby gateway device (like a router)connected to the IoT devices, in lieu of transmitting large volumes of dataupstream to a cloud server or data center for further analysis. Processingdata at the edge aggregates and filters the data as it is collected, with onlythe most salient data sent upstream. Edge analytics reduces upstreamprocessing and storage requirements alleviates network load.The processing power and storage used by an IoT application depends onprocessing required by the services or apps that consume the data. Availablememory and processor specifications, clock speed, and number of cores, all ofwhich subsequently determine the devices rate of data processing. The capacityof the non-volatile flash, which is used to persist data until transmittedupstream, determines how much data can be stored on the device. Devicesperforming edge analytics require substantially more processing capabilitiesthan devices that perform only basic data processing like validating,normalizing, scaling, or converting readings.
Types of off-the-shelf hardware for prototyping your IoT project
Developing IoT applications is more accessible with the growing availabilityof low-cost, commercially available off-the-shelf hardware development boards,platforms, and prototyping kits. Modular hardware designs provide greatflexibility. With a greater selection of components, designers may substitutenew sensors with different specifications. Alternatively, you canindependently upgrade the networking, data processing, or storage modules of adevice for evolving requirements.Many commercial off-the-shelf hardware devices, including microcontrollers andsingle-board computers, are designed around System-on-a-Chip (SoC) integratedcircuits. SoCs bundle capabilities such as data processing, storage, andnetworking onto a single chip. This configuration means that you sacrificesome flexibility for the sake of convenience, but, fortunately, there are ahuge number of commodity devices available with a range of configurations tochoose from. For example, Table 1 lists the technical specifications for aselection of microcontrollers that can be used for prototyping IoT projectsand provides a comparison of three popular Single-Board-Computers (SBCs).
IoT hardware requirements for deploying your IoT project
IoT devices are highly specialized. They are designed to operate within veryspecific environments. The hardware for IoT projects vary widely. While youmay prototype with generic off-the-shelf hardware, you eventually can movetoward the design and development of custom PCBs and components, tailored tothe requirement of your IoT solutions. As part of this process, you will needto consider these kinds of hardware requirements: * Security requirements * Ease of development * Data acquisition, processing and storage requirements * Connectivity requirements * Power requirements * Physical device design * Cost requirements
What is edge computing?
Edge computing is the computational processing of sensor data away from thecentralized nodes and close to the logical edge of the network, towardindividual sources of data. It may be referred to as a distributed IT networkarchitecture that enables mobile computing for data produced locally. Insteadof sending the data to cloud data centers, edge computing decentralizesprocessing power to ensure real-time processing without latency while reducingbandwidth and storage requirements on the network.The concept dates back to the 1990s, when Akamai solved the challenge of Webtraffic congestion by introducing Content Delivery Network (CDN) solutions.The technology involved network nodes storing static cached media informationat locations closer to end-users.Today, edge computing takes this concept further, introducing computationalcapabilities into nodes at the network edge to process information and deliverservices.
Examples of edge computing
Edge computing offers a range of value propositions for smart IoT applicationsand use cases across a variety of industries. Some of the most popular usecases that will depend on edge computing to deliver improved performance,security and productivity for enterprises include:
Benefits of edge computing
Edge computing addresses vital infrastructure challenges — such as bandwidthlimitations, excess latency and network congestion — but there are severalpotential additional benefits to edge computing that can make the approachappealing in other situations.Autonomy. Edge computing is useful where connectivity is unreliable orbandwidth is restricted because of the site’s environmental characteristics.Examples include oil rigs, ships at sea, remote farms or other remotelocations, such as a rainforest or desert. Edge computing does the computework on site — sometimes on the edge device itself — such as water qualitysensors on water purifiers in remote villages, and can save data to transmitto a central point only when connectivity is available. By processing datalocally, the amount of data to be sent can be vastly reduced, requiring farless bandwidth or connectivity time than might otherwise be necessary.Edge devices encompass a broad range of device types, including sensors,actuators and other endpoints, as well as IoT gateways.Data sovereignty. Moving huge amounts of data isn’t just a technical problem.Data’s journey across national and regional boundaries can pose additionalproblems for data security, privacy and other legal issues. Edge computing canbe used to keep data close to its source and within the bounds of prevailingdata sovereignty laws, such as the European Union’s GDPR, which defines howdata should be stored, processed and exposed. This can allow raw data to beprocessed locally, obscuring or securing any sensitive data before sendinganything to the cloud or primary data center, which can be in otherjurisdictions.Research shows that the move toward edge computing will only increase over thenext couple of years.Edge security. Finally, edge computing offers an additional opportunity toimplement and ensure data security. Although cloud providers have IoT servicesand specialize in complex analysis, enterprises remain concerned about thesafety and security of data once it leaves the edge and travels back to thecloud or data center. By implementing computing at the edge, any datatraversing the network back to the cloud or data center can be secured throughencryption, and the edge deployment itself can be hardened against hackers andother malicious activities — even when security on IoT devices remainslimited.
Challenges of edge computing
Although edge computing has the potential to provide compelling benefitsacross a multitude of use cases, the technology is far from foolproof. Beyondthe traditional problems of network limitations, there are several keyconsiderations that can affect the adoption of edge computing: * Limited capability. Part of the allure that cloud computing brings to edge — or fog — computing is the variety and scale of the resources and services. Deploying an infrastructure at the edge can be effective, but the scope and purpose of the edge deployment must be clearly defined — even an extensive edge computing deployment serves a specific purpose at a pre-determined scale using limited resources and few services. * Connectivity. Edge computing overcomes typical network limitations, but even the most forgiving edge deployment will require some minimum level of connectivity. It’s critical to design an edge deployment that accommodates poor or erratic connectivity and consider what happens at the edge when connectivity is lost. Autonomy, AI and graceful failure planning in the wake of connectivity problems are essential to successful edge computing. * Security. IoT devices are notoriously insecure, so it’s vital to design an edge computing deployment that will emphasize proper device management, such as policy-driven configuration enforcement, as well as security in the computing and storage resources — including factors such as software patching and updates — with special attention to encryption in the data at rest and in flight. IoT services from major cloud providers include secure communications, but this isn’t automatic when building an edge site from scratch. * Data lifecycles. The perennial problem with today’s data glut is that so much of that data is unnecessary. Consider a medical monitoring device — it’s just the problem data that’s critical, and there’s little point in keeping days of normal patient data. Most of the data involved in real-time analytics is short-term data that isn’t kept over the long term. A business must decide which data to keep and what to discard once analyses are performed. And the data that is retained must be protected in accordance with business and regulatory policies.
Edge computing, IoT and 5G possibilities
Edge computing continues to evolve, using new technologies and practices toenhance its capabilities and performance. Perhaps the most noteworthy trend isedge availability, and edge services are expected to become availableworldwide by 2028. Where edge computing is often situation-specific today, thetechnology is expected to become more ubiquitous and shift the way that theinternet is used, bringing more abstraction and potential use cases for edgetechnology.This can be seen in the proliferation of compute, storage and networkappliance products specifically designed for edge computing. More multivendorpartnerships will enable better product interoperability and flexibility atthe edge. An example includes a partnership between AWS and Verizon to bringbetter connectivity to the edge.Wireless communication technologies, such as 5G and Wi-Fi 6, will also affectedge deployments and utilization in the coming years, enabling virtualizationand automation capabilities that have yet to be explored, such as bettervehicle autonomy and workload migrations to the edge, while making wirelessnetworks more flexible and cost-effective.This diagram shows in detail about how 5G provides significant advancementsfor edge computing and core networks over 4G and LTE capabilities.Edge computing gained notice with the rise of IoT and the sudden glut of datasuch devices produce. But with IoT technologies still in relative infancy, theevolution of IoT devices will also have an impact on the future development ofedge computing. One example of such future alternatives is the development ofmicro modular data centers (MMDCs). The MMDC is basically a data center in abox, putting a complete data center within a small mobile system that can bedeployed closer to data — such as across a city or a region — to getcomputing much closer to data without putting the edge at the data proper.This was last updated in October 2020