Real-time automatic number plate recognition for city surveillance is at its best with Edge computing by ANX
Introduction
![Dr. Seva Panda, Director and Chief Technology Officer, Konnet Vian](http://www.eletimes.com/wp-content/uploads/2016/11/Dr.-Seva-Panda_Konnet-Vian-249x300.jpg)
With growing urban population and its supporting transport services, there is an urgent need to improve traffic management and secure the transport systems. Automation in transport has been used successfully in signaling systems and has helped in managing urban traffic to a great extent. In recent times, it has been possible to use CCTV for effective surveillance.
Advances in video analytics and image processing along with availability of high definition video capture devices, has increased the possibilities of reliable automation in surveillance.
Automatic Number Plate Recognition (ANPR) is one such application that has become an essential commodity for managing traffic and enforcing rules. It can also be used effectively for security.
Indian cities have a myriad of transport vehicles from bicycles, motorized two wheelers, three wheelers, cars, heavy vehicles, etc. With some exceptions for heavy vehicles, all these vehicles ply on the same city roads at the same time at varying speeds without maintaining any lane discipline.
![Prakash Mohapatra, Product Manager, Toradex](http://www.eletimes.com/wp-content/uploads/2016/11/Prakash-Mohapatra-Product-Manager-Toradex-238x300.jpg)
As a first step in building any solution to managing the situation, we need to identify the vehicles and then create solutions with this data. All motorized vehicles have a unique registration/license number that is displayed on the front and back of the vehicles. The ANPR application detects and recognizes this vehicle number automatically when viewed by the CCTV camera.
In this paper, we present a distributed computing architecture solution with ViAn ANPR_SIM solution using our ANX device for city surveillance.
Edge Computing for ANPR
ANX is a computing device that is used to compute the video analytic algorithms for video streaming from a camera. The ANX device is located at outdoor sites near the camera. This reduces the computing requirement at the central server, as well the network bandwidth required for streaming HD video. Therefore, even though the number of cameras scale-up for better city surveillance, by virtue of ANPR computing being distributed, the central server sizing does not scale-up much.
ViAn ANPR_SIM Solution
![Figure 1 Network Architecture](http://www.eletimes.com/wp-content/uploads/2016/11/zz-300x191.png)
Our solution consists of hardware as well as software components to provide a complete end-to-end solution. The network diagram shown below presents the connectivity diagram used in our City Surveillance solution.
The ANX device interfaces with the video IP/analog camera and performs required video analytic functions to detect number plates and convert the characters in the image to text. The date, time, location, and required vehicle information is pushed to the central SIM (Security Information Management) Server. This ensures real-time event notifications with optimal computing and network usage.
ANPR Software application
![Figure 2 ANX Application modules](http://www.eletimes.com/wp-content/uploads/2016/11/xx-300x145.png)
This is an automation solution with rules specified for vehicle motion and text recognition, along with appropriate installation design for camera position, lighting, and camera lens. To give an idea of the complexity of the ANPR algorithms performed on the ANX, the video analytic modules are presented below.
The rules for direction of vehicle motion, region, and size of number plates are saved in the database during configuration of the solution.
Once the application starts, each image frame from the video source is analyzed for number plate detection. The number plate is detected using edge
![Figure 3 Number Plate Detection Flow Chart](http://www.eletimes.com/wp-content/uploads/2016/11/Untitled-1-178x300.jpg)
analysis of the image. The detected region is then analyzed for shearing (i.e. rotation) and corrected by de-shearing. The corrected number plate image is then processed for text recognition.
Figure 3shows the data flowto recognize each character and then a decision making model checks if the standard number plate format is found. Once the number plate is identified, it is tracked and the direction of motion is identified. If the direction of the motion is as required, the number plate is validated and the data is pushed to SIM_Server.
Besides the video analytic functions, which are computer intensive, there is a need to decode and encode images/video for communication to the server.
Advantages
- Optimization on network bandwidth because of edge computing
- Server processing hardware optimized
- ANX is a low-power device (12 watts)
- Reliability in ANPR performance due to HD video availability at required frame rate
- High performance- to-cost ratio of ANX
- Easy hardware up gradation and maintenance
The Embedded Platform
On basis of Figure 2 that shows the real-time processing on ANX device, it is evident that this device needs a platform that offers high performance and advanced graphics capabilities. The ANX device is based on NVIDIA Tegra 3 processor, which is capable of handling real-time processing along with supreme graphics capabilities.
There are three options of embedded development once the processor is finalized:
Option 1: Full-custom design using Tegra 3 processor
Option 2: Use off-the-shelf Single Board Computer (SBC) based on Tegra 3 processor
Option 3: Use off-the-shelf System on Module (SOM) based on Tegra 3 processor
General parameters for Industrial product development
Apart from the processor and other technical requirements, there are few major parameters or requirements that are considered while building any industrial product.
- Time-to-market – Reduce development time, so that time-to-market can be minimized
- Development cost – Most industrial products have low sales volume, so it is critical to limit development cost to achieve healthy ROI (return on investment)
- Component obsolescence – Mitigate component obsolescence issues for essential design components such as processor and memory, as industrial products have a lifecycle of around 7-10 years
- Flexibility – The platform should be flexible to allow designers to build the product as per their requirements in terms of I/Os, configuration, and size
- Scalability – It should be easy to update the platform based on customer requirements and latest technologies. The platform should be scalable to handle these updates without significant investment in redesigns
The above three options of embedded development will be weighed against these five requirements.
Option 1: Full-custom development
Full-custom development involves building the hardware and software from scratch.
Time to market
Designing the hardware and software from scratch takes a long time. Significant time is spent on fixing bugs and maturing the software. This stretches the time to market.
Development cost
Embedded product development involves high NRE cost. A diverse engineering team is needed to build the product from scratch which increases the project cost. Full-custom designs are preferable with sales volume exceeding 70-80K per year. Cost wise, off-the-shelf platforms are more ideal for products with low sales volume.
Component obsolescence
Once a critical design component (processor, Flash, RAM) reaches its End of Life (EOL), then the platform has to be tested and validated with a new replacement component. In case an ideal replacement is not available, then the product’s life is shortened.
Flexibility
A full-custom development offers complete flexibility to designers to build the product based on their cost, I/O, and size requirements.
Scalability
Upgrading the performance of a platform developed from scratch requires redesign and this leads to a significant investment in time and money.
Time to Market | « |
Development cost | « |
Component Obsolescence | «« |
Customization | ««««« |
Scalability | « |
Figure 4 Weightage of Full-custom Development
Option 2: Single Board Computer (SBC)
SBC offers a ready-to-use embedded platform (firmware and hardware including SoC, memory, power requirements, I/Os such as USB, CAN, HDMI, Ethernet, display, etc.) on a single printed circuit board (PCB) for building any end-product. OEMs select SBCs best suited for their requirements and then develop the application software.
Time to market
The development time is reduced as the hardware and low-level software is a part of the SBC. Only application-level software has to be developed. So, timetomarket is accelerated.
Development cost
As the hardware and majority of software is available off-the-shelf, the development cost comes down. NRE cost and resource cost are reduced.
Component Obsolescence
OEMs do not have to worry about component obsolescence as the SBC vendor ensures the availability of the SBC over extended time period.
Flexibility
SBCs come in fixed sizes and offer standard I/Os, so, it cannot be customized based on the product requirements. Although peripherals can be added with interface boards, these will increase the size of the platform, which is not suited for portable end-products.
Scalability
The processing section and I/O section are integrated on a single PCB, so performance enhancement based on latest technologies and customer demands is not possible. A new SBC has to be used for upgrading the product.
Time to Market | ««««« |
Development cost | «««« |
Component Obsolescence | ««««« |
Customization | «« |
Scalability | « |
Figure 5 Weightage for SBC
Option 3: System on Module (SOM)
![Figure 6 System on Module Details](http://www.eletimes.com/wp-content/uploads/2016/11/cc-300x212.jpg)
A Computer on Module (COM) or System on Module (SOM) is a cost-effective, reliable, and off-the-shelf computing solution that consists of the essential design commodities such as processor, memory, power circuitry, operating system, and Board Support Packages (BSPs). The SOM is connected with a carrier board, which houses the I/Os (USB, CAN, Ethernet, UART, etc.), by means of some connectors such as SODIMM. OEMs can design the carrier board based on their I/O needs, configuration, and size requirements. Using SOMs, system designers can only focus on the application-specific software and hardware development. Product development is limited to carrier board design, application-development, and integration.
Time to market
The scope of development is reduced to application-specific hardware and software development, so the time to market is lesser than that of full-custom development.
Development cost
![Figure 7 System on Module with Carrier Board](http://www.eletimes.com/wp-content/uploads/2016/11/vv-300x176.jpg)
Resource cost and NRE cost comes down as the application-agnostic hardware and software is available with the SOM. So, the development cost reduces.
Component Obsolescence
OEMs do not have to worry about obsolescence of critical components as the SOM vendors ensure that the SOM is available for the entire end-product life.
Flexibility
OEMs can design the carrier board as per their I/O, configuration, and size requirements, so SOM offer more flexibility in comparison to that of SBC.
Scalability
Most SOM vendors offer pin-compatible SOMs, so upgrading a platform based on latest technologies and customer requirements is as easy as plug-and-play. The same carrier board can house multiple SOMs. Product variants with different memory sizes and performance, can also be launched easily without significant investment in development time and cost.
Time to Market | ««« |
Development cost | ««« |
Component Obsolescence | ««««« |
Customization | ««« |
Scalability | «««« |
Figure 8 Weightage of SOM
From the above tables, it is evident that SOM offers the most optimal platform to build industrial products. Konnet ViAN developed the ANX device on Colibri T30:a NVIDIA Tegra 3 based SOM by Toradex.