How to prepare a custom ROS environment on the NVIDIA Jetson board

7 July 2023

7 mins read

Mariusz Szczepanik

Lead C++ Software Engineer

Due to its capabilities, NVIDIA Jetson board is an ideal platform for installing a ROS (Robot Operating System) environment. However, setting up ROS on the board can present challenges, such as missing system dependencies, slow package building, and the need for hardware optimisation. In this article, I will guide you through a tutorial on preparing a custom ROS environment on the NVIDIA Jetson board, addressing these issues and providing step-by-step instructions for installation.

Why NVIDIA Jetson for ROS?

Jetson board is a series of embedded computing platforms developed by NVIDIA. These boards are specifically designed for running AI and deep learning applications in resource-constrained environments. Jetson boards integrate powerful GPUs, multicore CPUs, and dedicated AI accelerators to provide high-performance computing capabilities in a compact form factor. They are known for their ability to efficiently process and analyse sensor data, perform computer vision tasks, and execute complex AI algorithms. Thus, they are commonly used in robotics, autonomous vehicles, drones, industrial automation, and other edge computing applications.

What are major ROS environment setup challenges?

Developers may encounter a few common issues when preparing ROS on Jetson boards, such as the following:

ROS has several system dependencies that need to be installed before you can use it. If you encounter compiler errors when building ROS packages, you’re likely missing a system dependency, or there’s a version mismatch. The best way to detect package collision and build errors is to use docker files and create an isolated environment for development and easy maintenance.
Building packages directly on the Jetson board is quite a slow process, and, in most cases, it requires an internet connection to download dependent packages. This can be solved by creating images on a much more powerful host machine and flashing them directly on an SD card which then will be used on the Jetson board.
To improve hardware performance on a Jetson board, we should optimise software applications to run more efficiently. This may involve tweaking code, using more efficient packages, or reducing the workload on the board by avoiding the installation of unnecessary functionalities, such as Linux Graphical User Interface.

ROS environment installation on Jetson board in three steps

In this article, I will focus on a quite common setup: ROS1 Melodic release (Ubuntu 18.04) inside the NVIDIA Jetson Nano board. We will use Linux4Tegra, a dedicated operating system on the ARM64 Jetson environment that contains scripts we could reuse to generate IMG files.

Step 1. Create ROS docker files

First, we must prepare an ARM64 build environment on our host machine. You will probably use standard AMD64 architecture (for example, ubuntu 22.04); to do so, you need to install the QEMU tool. The main level where we store our custom setup will be a docker file (let’s call it Dockerfile.eg), where we define the L4T system version, and install the required ROS packages, for example:

# Install base packages
RUN apt-get update \
    && apt-get install -y \
    lsb-core \
    cmake \
    git \
    ninja-build \
    curl

RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
RUN curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc |  apt-key add -

# Install ROS packages - please add the next one's on the end if needed
RUN apt-get update \
    && apt-get install -y \
    ros-melodic-ros-base \
    ros-melodic-navigation \
    ros-melodic-robot-localization \
    ros-melodic-tf2-ros \
    ros-melodic-rviz \
    python-catkin-pkg \
    python-catkin-tools \
    python-rosdep \
    python-rosinstall \
    python-rosinstall-generator

We can inject and rebuild our repositories to receive a fully plug-and-play operated environment. In the code below the 9th line is very useful to avoid errors built on required package dependencies and preinstall them before catkin_make macro starts.

# Install and build your git repositories
RUN mkdir -p /catkin_ws/src \
    && cd /catkin_ws/src \
    && git clone https://github.com/<myRepo>.git \
    && git clone https://github.com/<myRepo1>.git \
    && git clone https://github.com/<myRepo2>.git \
    && cd /catkin_ws/ \
    && . /opt/ros/melodic/setup.sh \
    && rosdep install --from-paths src -y --ignore-src \
    catkin_make_isolated --use-ninja --cmake-args  -DCMAKE_BUILD_TYPE=Release

We can add resource files or directories to the docker image if needed. Additionally, we can create our system service, for example, roscoreinit.service to activate roscore on every L4T system start:

[Unit]
Description=Init roscore
After=remote-fs.target
After=syslog.target

[Service]
ExecStart=/root/init_roscore.sh
Restart=on-abort

[Install]
WantedBy=multi-user.target

Dedicated script init_roscore.sh:

#!/bin/bash
source /catkin_ws/devel_isolated/setup.bash
roscore

Add activation of roscoreinit.service and init_roscore.sh inside dockerfile:

COPY roscoreinit.service /lib/systemd/system/roscoreinit.service
COPY init_roscore.sh /root/init_roscore.shRUN systemctl enable roscoreinit

Step 2. Build image

To build docker and export it to a file, we call the below commands:

docker build --build-arg --network=host . -f Dockerfile.eg -t myImage
docker export "$(docker create --name nano-rootfs --platform linux/arm64 myImage)" -o myImage.tar

Next, we need to decompress the root file system (Rootfs) from the previous step, and, using the Linux4Tegra board support package, we will create a Jetson Nano image which can be flashed directly on the SD card:

# BSP 32.7.2 more details could be found here: https://developer.nvidia.com/embedded/linux-tegra-r3272
L4T_PACKAGE=https://developer.nvidia.com/embedded/l4t/r32_release_v7.2/t210/jetson-210_linux_r32.7.2_aarch64.tbz2
L4T_PACKAGE_BUILD_DIR=l4t_bsp
wget --no-check-certificate -qO- $L4TPackage | tar -jxpf - -C $L4T_PACKAGE_BUILD_DIR
ROOTFS_L4T_DIR=/tmp/l4t/rootfs
mkdir -p $ROOTFS_L4T_DIR
tar --same-owner -xf myImage.tar -C $ROOTFS_L4T_DIR

# Here we could modify Linux kernel, optional step described in the next section

# Generate IMG file
ROOTFS_DIR=$ROOTFS_L4T_DIR $L4T_PACKAGE_BUILD_DIR/Linux_for_Tegra/tools/jetson-disk-image-creator.sh \
-o myImage.img -b jetson-nano -r 300

Output file myImage.img is the final image that can be flashed directly on the SD card, for example, by balenaEtcher tooling.

Step 3, optional. Rebuild the Linux kernel from source

Rebuilding the kernel from the source allows you to select only the features you need. This can result in a leaner, faster, and more stable system performance. Sometimes we need to significantly change the Linux kernel configuration to support the requested devices. For sensor synchronisation purposes, we could enable various functionalities such as Pulse per second (PPS) or Precision Time Protocol (PTP), so ROS nodes could work in a stable environment. Then we need to find a compatible cross-compiler GCC version and kernel sources. In our scenario, we check the NVIDIA webpage below which describes L4T 32.7.2 version:

https://developer.nvidia.com/embedded/linux-tegra-r3272

After rebuilding the new kernel, you need to install device tree blobs in L4T package build mentioned in the Build Image section:

cp <build_from_sources_kernel>/arch/arm64/boot/Image $L4T_PACKAGE_BUILD_DIR/Linux_for_Tegra/kernel
cp -r <build_from_sources_kernel>/arch/arm64/boot/dts/* $L4T_PACKAGE_BUILD_DIR/Linux_for_Tegra/kernel/dtb

If your NVIDIA Jetson Nano has an embedded multi-media card, then you also need to synchronise the new kernel with eMMC when the bootloader will start for the first time. In Jetson Nano board without eMMC, the kernel will be initialised from an external SD card or USB storage where the L4T image is installed directly.

Overall, rebuilding the Linux kernel from the source can provide greater flexibility and control over the system, making it a worthwhile endeavour for advanced developers.

Summary

Developing and deploying the ROS environment on the NVIDIA Jetson board can easily package your application and its dependencies into a container image.

When building a Jetson image, Docker provides a standardised, reproducible environment that ensures consistency across different builds. This is particularly important when working with embedded systems like the Jetson, where hardware compatibility and optimisation are critical. Docker also allows you to easily share your Jetson image with others, reducing the time and effort required for setup and configuration. It also makes it easier to build a continuous integration side.

In summary, the use of Docker in the creation of a Jetson image offers a standardised, reproducible environment that ensures consistency across builds and simplifies sharing and collaboration.