Skip to content

README.md

Latest commit

 

History

History
77 lines (53 loc) · 4.69 KB

README.md

File metadata and controls

77 lines (53 loc) · 4.69 KB

SENSEable-Spot-AR-2021

Spot - HoloLens 2 - Unity

Includes the spot-sdk (python version)

Set up Mixed Reality Unity project with HoloLens 2 support

  1. Pick Unity version. I went with 2021.3.15 (whatever 2021.3 LTS version should work) - https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/unity-development-overview?tabs=arr%2CD365%2Chl2 Originally went with 2020, but ran into later issues with building.
  2. Following their installation recommendations, but include the C++ v143 in Universal Windows Platform dropdown on the right side of the visual studio installer. They recommend C++ v142, but this led to build errors (#using failed on 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\lib\x86\store\references\platform.winmd').
  3. Use MRTK 2. MRTK 3 has less long term support and I know there were some Microsoft AR layoffs.
  4. Use MRTK import tool. Make sure your packages you select have their pre-reqs also manually added. If you select one, it should tell you what it requires.
  5. Project Settings > Player > Other settings API compatibility
  6. Follow the online guide for setting up the rest fo the project

Three methodologies for Spot-Unity

  1. Run Python INSIDE Unity using some version of IronPython/PythonNet Having trouble getting PythonNet running in Unity, but someone HAS done it (see https://github.com/shiena/Unity-PythonNet). Concerned about potential compatibility/performance issues down the line.
  2. Create own C# SDK with gRPC or extend off Spot.Net Having trouble getting Spot.Net working, and concerned about having to extend off its limited selection.
  3. Communicate using ROS in Unity Seems to be the standard that's emerging. Used by Holo-spok and Holospot.

Methodology 1 Notes

Should explore more, but likely a better way to do it. Was unable to get a quick project working in about an hour, but with some leg work it should be possible.

Methodology 2 Notes

This was a mess. Didn't work. Maybe explore later.

  1. Download Spot.net repo - https://github.com/Sharks-Interactive/Spot.NET/tree/prod
  2. Install NuGetForUnity into project (easiest way to get NuGet projects into Unity - https://github.com/GlitchEnzo/NuGetForUnity
  3. Import Spot.Net using NuGetForUnity
  4. Copy-paste files from repo 'Spot.NET\spot-sdk\bin\Debug\net6.0' to Unity 'Assets\Packages\Spot.NET.0.1.0\lib'
  5. Open NuGet preferences in Unity. Add New Source. Delete the 'source_path' data to make the bar blank. This will tell NuGetForUnity to look in the base Assets folder for projects as well.
  6. Open NuGetForUnity and make sure 'Show All Versions' in the top left is checked yes.
  7. Import Grpc.Core.API using NuGetForUnity
  8. Import Grpc.Core using NuGetForUnity
  9. Download Google.Protobuf (v3.18.0) from - https://www.nuget.org/packages/Google.Protobuf/3.18.0. Place this file in the base Assets folder.
  10. Import Google.Protobuf (v3.18.0) using NuGetForUnity. You may need to scroll down far.

Methodology 3 Notes

General Workflow:

  1. Get ROS running on Spot. Does not come with native support, but Boston Dynamics AI Institute maintains this wrapper (https://support.bostondynamics.com/s/article/About-ROS-with-Spot) (https://github.com/bdaiinstitute/spot_ros2)

RANDOM NOTES

FIRST, get working with Python. Basic tests https://dev.bostondynamics.com/docs/python/quickstart#system-requirements

Documentation written for Linux? Windows conversion tips: 'export' Linux command = 'setx' in Windows

Python 3.8.10 - Latest version that's Spot acceptable

Meantime, getting Unity-HoloLens 2 project going Select Unity version - I used 2021 LTS Install the tools - https://learn.microsoft.com/en-us/windows/mixed-reality/develop/install-the-tools

  • Windows 10 SDK version 10.0.19041.0
  • USB Device Connectivity (required to deploy/debug to HoloLens over USB)
  • C++ (v143) Universal Windows Platform tools (required when using Unity) - IMPORTANT: Using v142 (their recommended) led to build errors for me (#using failed on 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\lib\x86\store\references\platform.winmd')

https://learn.microsoft.com/en-us/training/paths/beginner-hololens-2-tutorials/ is mild-ly hidden

#using failed on 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\lib\x86\store\references\platform.winmd'

6/25/23

Spot.Net library is difficult to get into Unity. Finally figuring out how to run it in visual studio, but the import process continues to be challenging. Will need to do more research on getting DLLs to run in Unity

6/26/23

Last week, FAQ page on BD website mentioning ROS and Spot was taken down. I noticed they put up a new one with "official" support (I think). This seems to be the way.