Syncing Vuforia’s AR Camera with Another Camera in Unity 3D

by / Tuesday, 30 July 2013 / Published in Blog, Technical

Here at KwazyLABS, we love the Qualcomm’s Vuforia package for Unity3D. It’s a technology that fits our products perfectly, and it is relatively easy to develop with.

In particular, we love combining an AR camera with a UI camera: this enables us to provide the WOW effect of objects jumping out of the real world and flying into the mobile device. The AR camera is used to detect real-world images, and attach virtual 3D objects to them. The UI camera is then used to move those virtual objects from the coordinates space of the real world, where they were initially positioned, into the coordinates space of the app’s UI – and… viola! we get the look-and-feel of “collecting” stuff into the device.

But switching between two cameras does not come without a price. While playing with Unity objects and cameras, I was trying to move objects from the AR camera’s children-tree into the UI camera’s children-tree. This resulted in a noticeable jump cut – for some reason, the two cameras did not show the same view of the world.

I made sure that the position, rotation, and scale of both cameras will be the same, and also the specific camera parameters “Field of View” and “Clipping planes”. I separated the cameras depths (set the AR camera’s depth to 1 and the UI’s to 2), and then set the UI camera’s “Clear Flags” to “Depth only”. But still – it just didn’t work – in the editor’s player window everything looked fine, but on the phone, the two cameras didn’t show the same view of the world.

It took me some time to discover that the problem was in the “projection matrix”. It seems that Vuforia’s AR camera uses its own specific version of projection matrix, which defines how the “world” will be shown on screen. A simple script solved the issue, just by applying the AR camera’s matrix to the UI camera:

private bool sync = false;
void Update()
    if (!sync)
        uiCamera.projectionMatrix = arCamera.projectionMatrix;
        sync = true;

Only after I applied this change – the two cameras got synced. Now I could use the AR camera to detect a target image in the real world, attach a 3D object to that image, then move the object to be the child of the UI camera without experiencing any jump cuts. This makes the object’s new coordinates relate to the app’s UI, therefore I can move it around – to the center of the screen, or anywhere else I want in the app’s UI layout.


10 Responses to “Syncing Vuforia’s AR Camera with Another Camera in Unity 3D”

  1. Oscar Zamme says : Reply

    Can you please elaborate more on the line “if (!sync)”.
    What is sync and what is the boolean meaning of !sync, having blittered the line “sync = true”.

    • Yair says : Reply

      Sorry for the late respond.
      sync is a simple flag that make sure we will sync the cameras only once. The sync should be done in the “Start” function, but for some reason it didn’t work.

  2. revolutive says : Reply

    Can you please share some kind of example of how to achieve the change between cameras? I’m a newbie in Unity and i’m trying to make an app that will allow the user to jump from the AR camera to the GUI camera.. thanks for your words!

  3. Max says : Reply

    Thank you so much. I’ve been dealing with this for a long time. I was convinced that Vuforia was doing “something else” to the camera, but i didn’t know i had access to that.

  4. sangram says : Reply

    i will be informative to know that you if you have tried to make vuforia camera work with the google cardboard vr sdk .

  5. Ajit says : Reply

    Hi Yasir,

    I got both the camera’s working fine and in sync. The camera’s are in sync.
    I am using Unity3D 4.6.5p4 and Vuforia 3.0.9 .
    I am getting a black screen when installed on mobile . In editor it works fine. I am not able to understand why its getting a black screen. The thing is I know camera is active (I have QR recognition working and its reading QR despite the black screen)
    As suggested in Forums I have tried the android screen delay fix and also set the quality.antialising=0 . It still renders a black screen !

  6. Ajit says : Reply

    Yes Yasir , It does work well without the second camera. and Yes , also I have tried on different devices .

Leave a Reply