Lesson 06: TensorFlow, PyTorch and OpenCV

TensorFlow™ is an open-source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code.

Prerequisites and Dependencies:

Before you install TensorFlow for Jetson, ensure you:

1. Install JetPack on your Jetson device.
2. Install system packages required by TensorFlow:
   

   sudo apt-get update
   sudo apt-get install libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev liblapack-dev libblas-dev gfortran

3. Install and upgrade pip3.
   

   
   sudo apt-get install python3-pip
   sudo python3 -m pip install --upgrade pip
   sudo pip3 install -U testresources setuptools==65.5.0
   

4. Install the Python package dependencies.
   

   $ sudo pip3 install -U numpy==1.22 future==0.18.2 mock==3.0.5 keras_preprocessing==1.1.2 keras_applications==1.0.8 gast==0.4.0 protobuf pybind11 cython pkgconfig packaging h5py==3.7.0

Installing TensorFlow

Install TensorFlow using pip3. This command will install the latest version of TensorFlow compatible with JetPack 6.3.2.

sudo pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v60 tensorflow==2.16.1+nv24.07

Installing Multiple TensorFlow Versions

If you want to have multiple versions of TensorFlow available at the same time, this can be accomplished using virtual environments. See below.

1. Set up the Virtual Environment:
   First, install the virtualenv package and create a new Python 3 virtual environment:

   sudo apt-get install virtualenv
   python3 -m virtualenv -p python3 
   

2. Activate the Virtual Environment:
   Next, activate the virtual environment:
   

   source <chosen_venv_name>/bin/activate

   Install the desired version of TensorFlow and its dependencies:

   
   pip3 install -U numpy grpcio absl-py py-cpuinfo psutil portpicker six mock requests gast h5py astor termcolor protobuf keras-applications keras-preprocessing wrapt google-pasta setuptools testresources
   pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v60 tensorflow==$TF_VERSION+nv$NV_VERSION
   

3. Deactivate the Virtual Environment:
   Finally, deactivate the virtual environment:
   

   deactivate

4. Run a Specific Version of TensorFlow:
   After the virtual environment has been set up, activate it to access the specific version of TensorFlow. Make sure to deactivate the environment after use:
   

   
   source /bin/activate
   <Run the desired TensorFlow scripts>
   deactivate
   

5. Upgrading TensorFlow:
   To upgrade to a more recent release of TensorFlow, if one is available, run the install command with the ‘upgrade’ flag:
   

   $ sudo pip3 install --upgrade --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v60 tensorflow==2.16+nv24.07

Verifying The Installation

To verify that TensorFlow has been successfully installed on your device, you’ll need to launch a Python prompt and import TensorFlow.

From the terminal, run:

python3

Import TensorFlow:

import tensorflow as tf
print("TensorFlow version:", tf.__version__)

If TensorFlow was installed correctly, this command should execute and display the TensorFlow version.

Uninstalling

TensorFlow can easily be uninstalled using the pip3 uninstall command, as below:

sudo pip3 uninstall -y tensorflow

PyTorch for JetPack is an optimized tensor library for deep learning that uses GPUs and CPUs. It features automatic differentiation with a tape-based system at both functional and neural network levels, offering flexibility and speed as a deep learning framework. It also provides accelerated NumPy-like functionality. NVIDIA provides Python pip wheel installers for PyTorch, with GPU acceleration and cuDNN support. These packages are meant to be installed on top of the specified JetPack version.

Prerequisites and Installation for PyTorch on Jetson

Prerequisites:

• Install required system packages:
  

  
  sudo apt-get -y update
  sudo apt-get -y install python3-pip libopenblas-dev
  

• Install PyTorch:
  • Export the PyTorch wheel URL:
    

    
    export TORCH_INSTALL=https://developer.download.nvidia.cn/compute/redist/jp/v60/pytorch/torch-2.4.0a0+3bcc3cddb5.nv24.07.16234504-cp310-cp310-linux_aarch64.whl
    

  • Install PyTorch:
    

    
    python3 -m pip install --upgrade pip
    python3 -m pip install numpy==1.26.1
    python3 -m pip install --no-cache $TORCH_INSTALL
    

• The released version of the PyTorch wheels is shown in the compatibility matrix.

Installing Multiple PyTorch Versions

To use multiple versions of PyTorch simultaneously, utilize virtual environments. See below for instructions.

• Setup Virtual Environment:
  First, install the virtualenv package and create a new Python 3 virtual environment:

  sudo apt-get install virtualenv
  python3 -m virtualenv -p python3 <chosen_venv_name>
  

• Activate Virtual Environment:
  

  source <chosen_venv_name>/bin/activate

• Install PyTorch:

  pip3 install --no-cache https://developer.download.nvidia.com/compute/redist/jp/v60/pytorch/torch-2.4.0a0+3bcc3cddb5.nv24.07.16234504-cp310-cp310-linux_aarch64.whl
  

• Deactivate the Virtual Environment:
  

  deactivate

• Run a Specific Version of PyTorch:
  After setting up the virtual environment, activate it to access the desired PyTorch version. Deactivate the environment after use:

  source <venv_name</bin/activate
  # Run PyTorch scripts
  
  deactivate
  

Verifying the Installation

To confirm PyTorch is installed on Jetson:

• Open a terminal and run:
  

  
  export LD_LIBRARY_PATH=/usr/lib/llvm-8/lib:$LD_LIBRARY_PATH
  python3

• Import PyTorch in Python:

  import torch

  If PyTorch was installed correctly, this command should execute without error.

Uninstalling

PyTorch can easily be uninstalled using the pip3 uninstall command, as below:

sudo pip3 uninstall -y torch

The OpenCV version included in the Jetpack image does not support CUDA. To enable CUDA support in OpenCV, you must build it from the source.

#!/bin/bash
set -e
install_opencv () {
  # Check if the file /proc/device-tree/model exists
  if [ -e "/proc/device-tree/model" ]; then
      # Read the model information from /proc/device-tree/model and remove null bytes
      model=$(tr -d '\0' < /proc/device-tree/model)
      # Check if the model information contains "Jetson Nano Orion"
      echo ""
      if [[ $model == *"Orin"* ]]; then
          echo "Detecting a Jetson Nano Orin."
	  # Use always "-j 4"
          NO_JOB=4
          ARCH=8.7
          PTX="sm_87"
      elif [[ $model == *"Jetson Nano"* ]]; then
          echo "Detecting a regular Jetson Nano."
          ARCH=5.3
          PTX="sm_53"
	  # Use "-j 4" only swap space is larger than 5.5GB
	  FREE_MEM="$(free -m | awk '/^Swap/ {print $2}')"
	  if [[ "FREE_MEM" -gt "5500" ]]; then
	    NO_JOB=4
	  else
	    echo "Due to limited swap, make only uses 1 core"
	    NO_JOB=1
	  fi
      else
          echo "Unable to determine the Jetson Nano model."
          exit 1
      fi
      echo ""
  else
      echo "Error: /proc/device-tree/model not found. Are you sure this is a Jetson Nano?"
      exit 1
  fi
  
  echo "Installing OpenCV 4.9.0 on your Nano"
  echo "It will take 3.5 hours !"
  
  # reveal the CUDA location
  cd ~
  sudo sh -c "echo '/usr/local/cuda/lib64' >> /etc/ld.so.conf.d/nvidia-tegra.conf"
  sudo ldconfig
  
  # install the Jetson Nano dependencies first
  if [[ $model == *"Jetson Nano"* ]]; then
    sudo apt-get install -y build-essential git unzip pkg-config zlib1g-dev
    sudo apt-get install -y python3-dev python3-numpy
    sudo apt-get install -y python-dev python-numpy
    sudo apt-get install -y gstreamer1.0-tools libgstreamer-plugins-base1.0-dev
    sudo apt-get install -y libgstreamer-plugins-good1.0-dev
    sudo apt-get install -y libtbb2 libgtk-3-dev v4l2ucp libxine2-dev
  fi
  
  if [ -f /etc/os-release ]; then
      # Source the /etc/os-release file to get variables
      . /etc/os-release
      # Extract the major version number from VERSION_ID
      VERSION_MAJOR=$(echo "$VERSION_ID" | cut -d'.' -f1)
      # Check if the extracted major version is 22 or earlier
      if [ "$VERSION_MAJOR" = "22" ]; then
          sudo apt-get install -y libswresample-dev libdc1394-dev
      else
	  sudo apt-get install -y libavresample-dev libdc1394-22-dev
      fi
  else
      sudo apt-get install -y libavresample-dev libdc1394-22-dev
  fi

  # install the common dependencies
  sudo apt-get install -y cmake
  sudo apt-get install -y libjpeg-dev libjpeg8-dev libjpeg-turbo8-dev
  sudo apt-get install -y libpng-dev libtiff-dev libglew-dev
  sudo apt-get install -y libavcodec-dev libavformat-dev libswscale-dev
  sudo apt-get install -y libgtk2.0-dev libgtk-3-dev libcanberra-gtk*
  sudo apt-get install -y python3-pip
  sudo apt-get install -y libxvidcore-dev libx264-dev
  sudo apt-get install -y libtbb-dev libxine2-dev
  sudo apt-get install -y libv4l-dev v4l-utils qv4l2
  sudo apt-get install -y libtesseract-dev libpostproc-dev
  sudo apt-get install -y libvorbis-dev
  sudo apt-get install -y libfaac-dev libmp3lame-dev libtheora-dev
  sudo apt-get install -y libopencore-amrnb-dev libopencore-amrwb-dev
  sudo apt-get install -y libopenblas-dev libatlas-base-dev libblas-dev
  sudo apt-get install -y liblapack-dev liblapacke-dev libeigen3-dev gfortran
  sudo apt-get install -y libhdf5-dev libprotobuf-dev protobuf-compiler
  sudo apt-get install -y libgoogle-glog-dev libgflags-dev
 
  # remove old versions or previous builds
  cd ~ 
  sudo rm -rf opencv*
  # download the latest version
  git clone --depth=1 https://github.com/opencv/opencv.git
  git clone --depth=1 https://github.com/opencv/opencv_contrib.git
  
  # set install dir
  cd ~/opencv
  mkdir build
  cd build
  
  # run cmake
  cmake -D CMAKE_BUILD_TYPE=RELEASE \
  -D CMAKE_INSTALL_PREFIX=/usr \
  -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
  -D EIGEN_INCLUDE_PATH=/usr/include/eigen3 \
  -D WITH_OPENCL=OFF \
  -D CUDA_ARCH_BIN=${ARCH} \
  -D CUDA_ARCH_PTX=${PTX} \
  -D WITH_CUDA=ON \
  -D WITH_CUDNN=ON \
  -D WITH_CUBLAS=ON \
  -D ENABLE_FAST_MATH=ON \
  -D CUDA_FAST_MATH=ON \
  -D OPENCV_DNN_CUDA=ON \
  -D ENABLE_NEON=ON \
  -D WITH_QT=OFF \
  -D WITH_OPENMP=ON \
  -D BUILD_TIFF=ON \
  -D WITH_FFMPEG=ON \
  -D WITH_GSTREAMER=ON \
  -D WITH_TBB=ON \
  -D BUILD_TBB=ON \
  -D BUILD_TESTS=OFF \
  -D WITH_EIGEN=ON \
  -D WITH_V4L=ON \
  -D WITH_LIBV4L=ON \
  -D WITH_PROTOBUF=ON \
  -D OPENCV_ENABLE_NONFREE=ON \
  -D INSTALL_C_EXAMPLES=OFF \
  -D INSTALL_PYTHON_EXAMPLES=OFF \
  -D PYTHON3_PACKAGES_PATH=/usr/lib/python3/dist-packages \
  -D OPENCV_GENERATE_PKGCONFIG=ON \
  -D BUILD_EXAMPLES=OFF \
  -D CMAKE_CXX_FLAGS="-march=native -mtune=native" \
  -D CMAKE_C_FLAGS="-march=native -mtune=native" ..
 
  make -j ${NO_JOB} 
  
  directory="/usr/include/opencv4/opencv2"
  if [ -d "$directory" ]; then
    # Directory exists, so delete it
    sudo rm -rf "$directory"
  fi
  
  sudo make install
  sudo ldconfig
  
  # cleaning (frees 320 MB)
  make clean
  sudo apt-get update
  
  echo "Congratulations!"
  echo "You've successfully installed OpenCV 4.9.0 on your Nano"
}

cd ~

if [ -d ~/opencv/build ]; then
  echo " "
  echo "You have a directory ~/opencv/build on your disk."
  echo "Continuing the installation will replace this folder."
  echo " "
  
  printf "Do you wish to continue (Y/n)?"
  read answer

  if [ "$answer" != "${answer#[Nn]}" ] ;then 
      echo "Leaving without installing OpenCV"
  else
      install_opencv
  fi
else
    install_opencv
fi