Build RealSense for macOS Monterey (Intel + Apple Silicon)

RealSense + macOS ≠ ❤

Intel RealSense is a state-of-the-art family of depth cameras. Depth cameras allow your apps to “see” the world in 3D. How? Combining a traditional RGB video camera with an Infrared emitter, RealSense understands both the color and the physical location of each point! Intel has managed to manufacture high-precision yet affordable cameras for various applications (e.g., robotics, scanning, body-tracking, and more). I own more than ten of these cuties, and I particularly love the D455 model.

To their honor, Intel has open-sourced the official RealSense SDK, allowing developers to tinker and experiment with the C++ code.

Problem 1: Apple Silicon and Intel chips

I love the RealSense SDK because it’s cross-platform: it supports Ubuntu, Windows, Android, and macOS. However, since Apple transitioned to its new Apple Silicon chipsets, RealSense has found it hard to keep up.

Today, most Mac developers need to develop universal binaries: single binary files that run on both Intel (x86) and Apple Silicon (ARM) processors. The RealSense installation via brew is not such a binary. In this guide, we are going to develop purely universal RealSense binaries.

Problem 2: macOS 12 Monterey

A second major issue is the lack of support for the latest version of Apple’s desktop operating system: macOS Monterey (version 12). According to Intel, RealSense officially supports macOS High Sierra only – an operating system released in 2017!

In this tutorial, we are going to build the RealSense SDK to support macOS Big Sur (version 11) and Monterey (version 12). At the end of the article, I’m also including a link to download the prebuilt binaries.

Build RealSense for macOS

You are now aware of the problems we are going to solve. So, without further ado, let’s get to the build process. Many thanks to the RealSense developers who commented on GitHub and pointed us in the right direction.

Prerequisites

To build RealSense, you need to update your macOS computer to the latest version, which is macOS Monterey 12.3.1. Before starting, install the following software tools:

• CMake 3.22
• XCode 13.3
• Homebrew 3.4.6

With the developer tools in place, install the OpenSSL library and the XCode command-line tools.

xcode-select --install
brew install openssl

1) Build libusb

RealSense cameras are (literally) powered by a USB-3 port. As a result, the SDK relies on the popular libusb library to manage the serial bus connection. libusb comes pre-installed on your Mac; however, it’s either an x86 or ARM binary. In our scenario, we need universal binaries, so we have to build libusb from source. To do so:

• Clone the libusb repository.
• Go to the XCode folder.
• Open the libusb.xcodeproj file.

In XCode, go to Product → Scheme and select libusb from the drop-down list. Then, select Product → Scheme → Edit Scheme… Under the Run panel in the Info tab, set Build Configuration to Release.

Then, select the libusb project and the libusb target and navigate to the General tab. Set the Deployment Target value to 11. That means we are building libusb with Big Sur support, too.

Next, go to the Build Settings tab and set the Architectures value to Standard Architectures (Apple Silicon, Intel) – $(ARCH_STANDARD).

Finally, go to Product → Build or just click Cmd + B to build libusb.

When XCode finishes, open the build folder in Finder (Product → Show Build Folder in Finder).

Open Products → Release and launch a Terminal in that folder. You should see a file named libusb-1.0.0.dylib. Stay with me here, as we are going to remove the symbolic links of that binary. This way, we’ll be able to reference it from other binaries located next to it – instead of searching for it in a specific location.

install_name_tool -id @loader_path/libusb-1.0.0.dylib libusb-1.0.0.dylib

You may see a couple of warnings informing you that you are changing the signature of the file. That’s OK. It’s exactly what we want to do 🙂

2) Build librealsense

Moving on, it’s time to build the RealSense SDK for macOS. First, we’ll clone the GitHub repository and switch to the latest RealSense release (currently version 2.50.0). Next, open your Terminal to a location of your choice and type the following commands:

git clone https://github.com/IntelRealSense/librealsense.git
git checkout tags/v2.50.0

IF USING VERSION 2.49.0: Navigate to the root folder of the repository, open the file src → proc → color-formats-converter.cpp, and append the following in line 21: || defined(__aarch64__).

While in the librealsense repository folder, we’ll create a build folder and reset XCode:

mkdir build && cd build
sudo xcode-select --reset

Now, we’ll use CMake to create an XCode project:

cmake .. -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64" \
-DCMAKE_THREAD_LIBS_INIT="-lpthread" \
-DCMAKE_BUILD_TYPE=RELEASE \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_EXAMPLES=false \
-DBUILD_WITH_OPENMP=false \
-DHWM_OVER_XU=false \
-DOPENSSL_ROOT_DIR=/opt/homebrew/opt/openssl \
-G Xcode

Notice what we did there? First, we set the target architecture to universal by specifying both the arm64 and x86_64 variants. We also linked the thread library to avoid common CMake errors.

Hit Return and wait for XCode to build the project. Upon completion, the build folder of the RealSense repository should contain a file named librealsense2.xcodeproj. Open that file using XCode.

Just like we did for libusb earlier, go to Product → Scheme and select realsense2 from the drop-down list. Then, select Product → Scheme → Edit Scheme… Under the Run panel in the Info tab, set Build Configuration to Release.

Don’t forget backward compatibility! In your Targets section, select realsense2. Under the General tab, change the Deployment Target to 11 to support Big Sur.

The tricky part begins. Move to the Build Settings tab and find Header Search Paths. That is the section we provide the external C/C++ header files our project relies on. We’ll instruct the compiler to search for our custom version of libusb instead of the built-in one. Find the default libusb location and replace it with the repository’s header folder location. In my case, it’s:

/Users/vangos/Documents/GitHub/libusb/libusb

Then, go to the Other Linker Flags section and select Release. That’s where we link our binary with the external dependencies. Find the default libusb location and replace it with the absolute path to the custom build folder. In my case, it is:

/Users/vangos/Library/Developer/Xcode/DerivedData/libusb-xyz/Build/Products/Release/libusb-1.0.0.dylib

Ready? Click Cmd + B to build the project. You may grab a cup of coffee since the process takes quite some time. When XCode finishes building the library, you should see the RealSense target under the Products folder.

As for the last step, we need to remove the symbolic links to have the RealSense binary next to our primary executable. Reveal librealsense2.2.50.0.dylib in Finder and open a Terminal there. Type:

install_name_tool -id @loader_path/librealsense2.2.50.0.dylib librealsense2.2.50.0.dylib

This is it! You can now create your C++ program and link it to the RealSense binaries. The binary even works with Unity apps. Hooray!

If you’ve reached that far, I assume you know how to develop, compile, and run apps on macOS. Teaching you how to do that is outside the scope of this tutorial, but let my team know if you need help.

3) Run as admin [Monterey]

Starting macOS Monterey, RealSense applications need elevated privileges to run properly. As a result, you need to run the app as an admin.

If your app is a C++ executable, simply run:

sudo ./MyProgram

If you developed a GUI app, you need to right-click your .app file and select Show Package Contents. That’s because GUI apps are nothing but specifically-structured folders. So, go to Contents → MacOS and run your main executable:

sudo ./MyApp

You can run your RealSense app without admin privileges if using Big Sur.

Enable Root User

If you are tired of typing “sudo” all the time, you can enable the root user on your Mac (or, God mode). Here’s how to do that:

Remember to disable the root user after completing your task.

Results

Your RealSense apps should run perfectly fine on macOS if you’ve followed the above steps. Notice how smoother the 60 or 90 frame rates seem in the M1 chipsets! Here’s the result on my M1 Mac Mini:

Precompiled portable binaries

Tired of spending all that time compiling third-party libraries? Not familiar with C++ and CMake? Worry not! Our team has put together the precompiled portable C++ binaries so that you can use them in your apps right away. The RealSense Viewer is included, too. Click the button below to download.

Summary

In this guide, we built a single universal RealSense binary to support macOS Big Sure and Monterey on both Apple Silicon and Intel chipsets.

If you liked this post, do me a favor and share it with your friends and colleagues. ‘Til the next time, keep coding!