Nvidia Jetson Xavier NX review - Redefining GPU accelerated machine learning - Android

The Xavier NX development kit allows you to use Nvidia's machine learning module for product development and as a desktop.

Nvidia launched the Jetson Xavier NX embedded System-on-Module (SoM) at the end of last year. It is pin-compatible with the Jetson Nano SoM and includes a CPU, a GPU, PMICs, DRAM, and flash storage. However, it was missing an important accessory, its own development kit. Since a SoM is an embedded board with just a row of connector pins, it is hard to use out-of-the-box. A development board connects all the pins on the module to ports like HDMI, Ethernet, and USB. A Jetson module combined with a development board looks similar to a Raspberry Pi or other Single Board Computers (SBC). But don’t be fooled this is no low-end, low-performance device.

Like the Jetson Nano, the Jetson Xavier NX developer kit is a machine learning platform; unlike the Jetson Nano, it isn’t an entry-level device. The Xavier is designed for applications that need some serious AI processing power.

Onboard of the SoM you get a hexa-core CPU using Nvidia’s custom Carmel ARM-based cores, a 384-core Volta-based GPU, and 8GB of LPDDR4x RAM @51.2 GB/s. The development board adds HDMI, DisplayPort, Gigabit Ethernet, 4x USB 3.1 ports, Wi-Fi, Bluetooth, 2x camera connectors, 40 GPIO pins, and an M.2 slot for an SSD!

The 8GB of RAM and support for M.2 NVMe makes this a significant upgrade to the Jetson Nano, but the real upgrade is in the processing power. Compared to the Jetson Nano, the Xavier NX is anywhere between two to seven times faster, depending on the application.

This is due to the improved CPU, hexa-core Nvidia Carmel (ARM v8.2 64-bit with 6 MB L2 + 4 MB L3 caches) upgraded from quad-core Cortex-A57; better GPU, 384-core Voltra compared to 128-core Maxwell; plus the inclusion of 48 tensor cores and two Deep Learning Accelerator (DLA) engines.

Read more: Artificial Intelligence vs Machine Learning: what’s the difference?

Nvidia’s Jetson modules are primarily designed for embedded applications, meaning the SoM will be embedded into a specific product. Anything from robots, drones, machine vision systems, high-resolution sensor arrays, video analytics, and autonomous machines can benefit from the machine learning performance, small form factor, and lower power requirements of the Xavier NX.

Nvidia’s primary aim is to sell the SoMs to device manufacturers. However, the development kit is essential for product design and development, and for anyone who wants to try advanced machine learning at home.

Performance and form factor are essential for embedded projects, but so is power usage. The Jetson Xavier NX delivers up to 21 Trillions Operations per Second (TOPS) while using up to 15 watts of power. When needed the board can be set into a 10W mode. Both power modes can be tweaked depending on how much CPU performance you need compared to the GPU performance. For example, you could run just two CPU cores at 1.9GHz and the GPU at 1.1GHz or alternatively you could use four CPU cores @1.2GHz and clock the GPU at 800Mhz. The level of control is exceptional.

Tell me about the GPU

When you think of Nvidia you probably think about graphics cards and GPUs, and rightly so. While Graphic Processing Units are great for 3D gaming, it also turns out that they are good at running machine learning algorithms. Nvidia has a whole software eco-system based around its CUDA parallel computing and programming model. The CUDA toolkit gives you everything you need to develop GPU-accelerated applications and includes GPU-accelerated libraries, a compiler, development tools, and the CUDA runtime.

I was able to build Doom 3 for the Xavier NX and run it at 4K!

The Jetson Xavier NX has a 384 core GPU based on the Volta architecture. Each generation of GPU from Nvidia is based on a new microarchitecture design. This central design is then used to create different GPUs (with different core counts, and so on) for that generation. The Volta architecture is aimed at the datacenter and at AI applications. It can be found in PC graphic cards like the Nvidia Titan V.

 

The potential for fast and smooth 3D games, like those based on the various 3D engines released under open source from ID software, is good. I was able to build Doom 3 for the Xavier NX and run it at 4K! At Ultra High Quality the board managed 41 fps. Not bad for 15 watts!

Nvidia has a universal software offering that covers all of its Jetson boards, including the Jetson Nano and the Jetson Xavier NX, called JetPack. It is based on Ubuntu Linux and comes pre-installed with the CUDA toolkit and other relevant GPU accelerated development packages like TensorRT and DeepStream. There is also a large collection of CUDA demos from smoke particle simulations to Mandelbrot rendering with a healthy dose of Gaussian blurs, jpeg encoding, and fog simulations along the way.

Read more: Jetson Nano review: Is it AI for the masses?

Make my machine learn

Having a good GPU for CUDA based computations and for gaming is nice, but the real power of the Jetson Nano is when you start using it for machine learning (or AI as the marketing people like to call it). Jetson Xavier NX supports all the popular AI frameworks including TensorFlow, PyTorch, MxNet, Keras, and Caffe.

All of Nvidia’s Jetson boards come with excellent documentation and example projects. Because they all use the same ecosystem and software (JetPack etc) then the examples work equally as well on the Jetson Nano or on the Jetson Xavier NX. A great place to start is the Hello AI World example.  It is simple to download and compile, and in just a few minutes, you will have an AI demo up and running for image classification, object detection, and semantic segmentation, all using pre-trained models.

I fished out a picture of a Jellyfish (pun intended) from my visit to the Monterey Bay Aquarium in 2018 and asked the image classifier to label it.

Why pre-trained? The hardest part about machine learning is getting to the point where you can present data to a model and get a result. Before that the model needs training, and training AI models is not a trivial effort. To help, Nvidia provides pre-trained models as well as a Transfer Learning ToolKit (TLT) which allows developers to take the pre-trained models and retrain them with their own data.

The Hello AI World demo gives you a set of tools to play around with including an image classifier, and an object detection program. These tools can either process photos or use a live camera feed. I fished out a picture of a Jellyfish (pun intended) from my visit to the Monterey Bay Aquarium in 2018 and asked the image classifier to label it.