Positional Tracking on Gear VR ! Using ARcore = 6 DOF Standalone Tracking! (Unity 2017 Tutorial)


hey everyone Fuseman coming at you today
I hope you’re as excited as I am to be playing around with mobile VR and
positional now this technology has given kind of exclusive to PC VR so
I’m really excited to have positional tracking on the gear VR now one thing to
keep in mind is that this is currently only working on the Samsung S 8 and the
new gear VR using AR core and if you want a tutorial that has an in-depth
explanation on how AR core works I recommend checking out our previous
tutorial on shooting stars where we go into all the components and how they’re
used within a our core so without out of the way let’s just
kind of dive into unity to see how we can quickly set up positional tracking
to work with in gear VR awesome so we’re here in unity if you want to follow
along this project is on github and a link in the description and also if
you’re looking for the AR core shooting star tutorial that’s also linked in the
description below so let’s get started here
if we go look at the project we’re on the demo one scene and find that within
low-poly environment and then demos and this is the scene that you saw
originally from the teaser clip and you can see here it’s kind of already set
for AR core you have the particles here which is this point cloud and then we
have air core device setup these were copied from the example scene and
specifically here we can we just moved just those two game objects into the
scene and everything else was from demo 1 so now moving on to the inspector you
can see here we have two components on our AR core device and we need to tweak
these a little bit in order to get it to work with the gear VR as opposed to
working on just say a flat screen on your
phone so the first one here and this is where we make most of our changes is the
tracked pose driver now this is basically in part of unities XR package and it
basically allows us to add positional order additional tracking to pretty much
any object within our XR settings so first thing here is we can choose a
device might be a controller could be a remote the next one here is the post
source and this is just gonna be set to a color camera both of those stay the
same from what is usually used for a our core but these last three are what is
actually different so first up we have the tracking type we can either have
rotation or rotation position or just position we’re gonna use only position
the reason for this is that gear VR if we’re just using the native built in
unity controls automatically provides us with rotational tracking so if we have
the rotational tracking from AR core that’ll basically duplicate our
rotations which we don’t really want to have happen so let’s just use the
position from AR core and the rotations from the gear VR next up here we have
the update type in the example scene is set to just before render but really by
default if you just add a track pose driver
it defaults to update and before render basically what this is doing is it
samples the positional tracking from the camera at before the actual rendering
happens as well as during your update frame so you basically get two sample
points per frame which allows your positional tracking to be a lot better
and be a little more accurate it does come a performance cost but if you’re
looking for very high accurate 30-plus frames per second positional tracking
this is where you want to look and last but not least is the relative transform
this is basically saying based on wherever a our course starts moving
relative to that which makes complete sense because we never know where we’re
going to initialize so that’s it for the track pose driver the next one here is
the session component and this is specific to a our core first up we have
the default such an config this is just a file that
you can manipulate and go ahead and click on it it basically it’s like do
you want a our background enable do you want plane funding do you want point
clouds if you disable some of these then you’re gonna save a little bit on
performance for our specific purposes we’re going to keep them all enabled if
you want positional tracking then you definitely have to make sure that a our
background is enabled otherwise you’re not gonna get it so it’s going back to
some component the other one here that’s really important is this first-person
camera you’ll see that it’s set to none and that’s for a good reason so if we
actually go ahead and double click on this to open it up and just look for a
camera this first-person camera here this is what is used by ARcore to set
the background render mode what that means is because ARcore is an AR
application it wants to render at the real world camera and so it takes
wherever your virtual camera is and will start if you have a camera reference
it’ll start adding frames from your real world camera into it however if you set
it to none it’s not gonna do that so if you just want your skybox to render or
anything else in the background to render you need to make sure this is set
to none actually if you take a look here you’ll see that if you do assign a
camera that we’ve get a real world view within the gear VR it does appear a
little distorted I’ll be honest but if you do want to see the world around you
with a kind of pass-through effect that’s what that’s gonna look like so
that pretty much does it for all the setup we need to do within our own scene
the only other thing you don’t need to do for the gear VR within your project
is making sure that you have an au cig file you can see here I have some of my
own OSIG files for different devices if if you want to build this you’ll need
to make sure you have a no osig file you can go the oculus website to actually it
and get that set up and with that that’s everything we need to do for the project
let’s just go quickly to our build settings make sure you have Android
setup because this is a our core in other settings you’ll just need to make
sure you have package ID and all this good stuff they
traditionally need to do for any Android app definitely making sure that you have
the minimum at 7.0 because you need nougat to actually run this and the
other thing that’s really important is the XR settings right here so you can
see we have both ticked the virtual reality support for oculus because we’re
running the gear VR and we also have a our core supported enabled so it’s
interesting that a lot of things are starting to move in this is XR settings
you can see there’s probably some Authority of stuff coming and probably a
lot more so really cool to see this being a part of actually unity this is
probably a good point also this is only as of unity 2017 point to previous
versions did not have this so that’s something to keep in mind so that’s
really all you need to do to set up before I just tell you to go and build
and run it at the top level here you’ll see that we have a sample position so
this is just if you want a bare-bones test then you can try that out it’s just
really a bunch of cubes in here you can add whatever you want again really it’s
only these two that are really what’s enabling a our core to work and then we
have the FPS script so again at the intro video you might have seen like a
little FPS reader at the top what I was calculating here was basically how many
frames per second was a our core updating compared to rendering so this
is called in the update frame and you can see here I left a little note that
the max FPS that updates gonna run at 60fps that’s just a limitation of the
gear VR and the frame the display retroreflection at 60fps so when we
calculate our AR fps we are really only calculating at most 60 frames per second
the AR FPS could be a little higher it can be a little lower it can be a little
lower and we’ll see it but if it’s higher we won’t be able to see it
because update is only refreshing at most 60 frames per second so I’ll be
able to see what at most it’s refreshing at again these are kind of approximate
values I’m not really like let’s calculate the number of frames each second and
report that value we’re just kind of saying like how much
time is happening between each frame and then updating the just based on that
updating how much we think what happened over 60 seconds so it’s not perfect but
it does give you a rough estimate of how this is working so you can take a look
here again if you want to learn more about air four and kind of what this
frame is anything like that definitely go check out the other tutorials that I
mentioned so just go ahead and close that pretty much you’re pretty much set
to go just go ahead and go to your belt settings make sure whatever you seen you
want it should default if you’re just grabbing this off of github go ahead if
you have your phone to touch go ahead build and run and then check it out I
hope you all enjoyed this tutorial it’s actually surprisingly easy now to get
positional tracking working on mobile which is really really cool again this
only works with the s8 and the new gear VR but hopefully a our core will be
opening up to more phone soon because it’s currently in beta if you liked this
video make sure to share it with a friend because that helps us out a ton
and if you haven’t go check out our a our core shooting stars tutorial where
we give a more in-depth explanation of how arcore works until next time this has been fuseman and I’m signing out

53 thoughts on “Positional Tracking on Gear VR ! Using ARcore = 6 DOF Standalone Tracking! (Unity 2017 Tutorial)

  1. Are you guys planning to work with Microsoft MR -based devices? By Acer/Lenovo, etc. Really curious to see, especially in context of compatibility with VRTK and SteamVR.

  2. Will you support S7 when ARcore comes to the s7? How is the tracking, would you say it's on par with Oculus, vive, or psvr?

  3. I used the Unity tutorial from the other day to get this working on Daydream too so this can be done on a Pixel for those who are curious https://gfycat.com/gifs/detail/AcceptableInformalArchaeopteryx Obviously I had to use a headset with the camera exposed so it doesn't use Google's headset, but it does work with the controller.

  4. Love the tutorial, got it work on my pixel XL however I'm having problems with low frame rate when adding more than a few cubes into my scene for testing

  5. How was the latency moving around with the headset on? Do you think you can play a simulation comfortably? Thanks! Awesome Video!

  6. HI, in my project in "Session Component" i use "First Person Camera" but phone S8 camera add much Zoom+. Did you have this problem?

  7. Do I have to open this in Unity and compile the program myself, or did you provide an .apk? If you didn't, is it possible for you to provide one?

  8. I hope someone will take an arcore program and split the screen (for vr headset) so you can have an AR headset for price of Google cardboard

  9. I published a version for the new ARCore preview 2. Check it out here: https://github.com/ChristophGeske/ARCoreInsideOutTrackingS7
    Interestingly the preview 2 apk runs smoothly on the Galaxy S7 without overheating despite not being listed as a supported device.

  10. Great tutorial, but after updating to the latest ARCore I'm getting this error: Assets/FPS.cs(21,20): error CS0117: `GoogleARCore.Frame' does not contain a definition for `Timestamp'

  11. this is insane – well done, sir. Thanks for connecting those dots – I don't think GearVR owners realize many of them have a 6DoF headset:)

  12. I had an existing GearVR project and added the ARCoreDevice and PointCloud game objects from the Demo1 project from your github, changed the ARCoreDevice settings as per the video, set my API to Android 7, running on an S8 (for which I already have a working ARCore unity project) but every time the app starts, (just after the unity logo) I get an App has stopped unexpectedly message. Have I forgotten something? Is there something else I need to import other than the ARCoreDevice prefab from the GoogleARCore folder and the pointcloud from the LowPoly Environment Pack?

  13. Great video !
    But i am facing some issue "Unable to convert classes into dex format." when build apk.
    Please help if you have any idea

  14. When i set "position only " rotation doesnt work. Screen isnt moving when i rotate head. Without arcore it works(Pure gear vr tracking). Any idea how to fix it. S7

  15. hey I know its good and all, but can you show us how can we use it with Opentrack so it can be used with Riftcat

  16. Hi. does it work with any other phone (2K resolution, octa core, Android 7, etc…) similar to S8 and any other headset than GearVR ?

  17. When I build this to a s9 it opens then freezes to a gray screen after the made in unity splash screen. Anyone else getting this?

  18. Theres no GearVR camera rig or any oculus assets I dont understand how this works. Are we supposed to install that ourselvs or do we use what we already have????

Leave a Reply

Your email address will not be published. Required fields are marked *