Auki::ConjureKit::Manna::Manna

Functionalities of the Manna module are enabled by instantiating a [Manna](/#) object while passing your instance of ConjureKit

Inherits from ConjureKitModule

Public Functions

	Name
	Manna(IConjureKit conjureKit, float distanceToSizeRatio =10f) Constructor - initializes and registers Manna as a ConjureKit module.
void	ProcessVideoFrameTexture(Texture texture, Matrix4x4 projectionMatrix, Matrix4x4 worldToCameraMatrix) Pass a video frame to Manna so that Manna can scan it for QrCodes. (See code samples.) See the overloaded version of this function for finer-grained control on the coordinate transform pipeline. In this version of the function, textureToProjectedCoordinatesMatrix defaults to the matrix computed by DefaultTextureToNDCMatrices.ARFoundationCameraBackgroundDefaultMatrix().
void	ProcessVideoFrameTexture(Texture texture, Matrix4x4 textureToProjectedCoordinatesMatrix, Matrix4x4 projectionMatrix, Matrix4x4 worldToCameraMatrix) Pass a video frame together with a custom texture-to-screen-coordinate matrix to Manna.
float	OwnDeviceScreenPixelsPerInch() Returns the number of screen pixels per physical inch, to the best of Manna's knowledge of the current device. The implicit guarantee is that dividing Screen.width by this value gives the physical width of the current device screen, in inches, and likewise for Screen.height.
long	MillisecondsSinceLastOnCalibrationSuccess() Returns the number of milliseconds since the last time Manna executed its the OnCalibrationSuccess callback in the current session; is reset after (not right before) that callback; is -1 if OnCalibrationSuccess has not yet been called.
void	SetOnBeforeCalibration(Action< Lighthouse, Action< bool >> onBeforeCalibration) Set a callback that is called after a calibration frame has been decided and allows for the developer to do some logic before the calibration and decide if it should continue. If the action is called with true, the calibration will continue, else it will not.
void	SetStaticLighthousePoseSelector(Action< StaticLighthouseData, Action< LighthousePose >> poseSelector) Set a callback that is called when a static lighthouse has been scanned. The callback should invoke the provided Action with a chosen LighthousePose from the array. To defer without selecting any lighthouse, invoke the poseSelector Action with null. Note that scanning will not resume until the poseSelector Action has been invoked.
Rect	LighthouseRectInMetersFromScreenTopLeft() Returns a Rect whose .x .y is the visual bottom left of the lighthouse, in Unity coordinates in the frame of reference of the screen's top left corner.
Rect	LighthouseBackgroundRectInMetersFromScreenTopLeft() Returns a Rect whose .x .y is the visual bottom left of the lighthouse background frame, in Unity coordinates in the frame of reference of the screen's top left corner.
void	SetLighthouseVisible(bool visible) Display/hide the on-screen instant calibration QR code.
void	SetLighthousePosition(Vector2 positionInIMGUICoordinates) Sets the position of the center on-screen instant calibration QR code, in pixels from the top left corner of the screen. The position values are truncated such as to prevent the code from being displayed too close to the screen's edge. For example, SetLighthousePosition(Screen.width, Screen.height) will result in a QR code that is inset from the bottom right of the screen.
override void	_Init(Action onComplete, Action< string > onFailed)
override void	_Update()
void	_StoreLastCalibrationFrameData(Action< Texture2D, Texture2D, long, long, string > onCalibrationFrameDataReady =null) (Internal) Stores debug data from last calibration.
void	_StoreVideoFeedDataFrameData(Action< Texture2D, long, string > onVideoFeedFrameDataReady =null) (Internal) Stores debug data from video feed.
void	LeftMultiplyTransformByMatrix(Transform transform, Matrix4x4 matrix) Utility function. See usage notes in the Manna class overview.
void	LeftMultiplyTransformByMatrix(GameObject gameObject, Matrix4x4 matrix) Utility function. See usage notes in the Manna class overview.

Public Properties

	Name
bool	VibrateOnReportPose Cause the device offering calibration to vibrate when it receives a request from a device trying to calibrate. This vibration will occur before the device trying to calibrate sets its state to Calibrated, but after that other's 'VibrateOnReadPose' haptic feedback, if that flag is set.
bool	VibrateOnReadPose Cause a device to vibrate whenever it scans a lighthouse (QR code) at sufficiently close range. This vibration will occur before the device succeeds in joining an associated session, and before the 'VibrateOnReportPose' haptic feedback of the device offering calibration, if any, if that flag is set.
bool	VibrateOnCalibrate Cause the device to vibrate each time its successfully recalibrates from scanning a lighthouse in its environment; this haptic feedback will occur after the haptic feedback from 'VibrateOnReadPose', if the former flag is set.
Action< Lighthouse, Pose, bool >	OnLighthouseTracked Called when a lighthouse is being tracked even if its not close enough the boolean passed indicates weather its deemed to be close enough or not.
Action< Matrix4x4 >	OnCalibrationSuccess This callback is called at the same frame that the pose of the ARSessionOrigin is adjusted to accommodate for joining a pre-existing session via QR code calibration. See the detailed usage notes above in the Manna class overview.
Action< CalibrationFailureData >	OnCalibrationFailed Called if calibration fails.
Action< bool >	OnSetImageTracker Called when image tracker is changed. Will return true if Manna is using the ARFoundation image tracker or false if it's using the native one. Deprecated: this event is no longer triggered/used.
long	CalibrationCooldownMs Minimum cooldown time in milliseconds between calibrations. This prevents calibrations from being performed too close in time from each other.

Detailed Description

class Auki::ConjureKit::Manna::Manna;

Functionalities of the Manna module are enabled by instantiating a [Manna](/#) object while passing your instance of ConjureKit

Use the SetLighthouseVisible method of the Manna class to hide and show the calibrating QR code.

The on-screen position and appearance of the QR code is not adjustable in this release of the Manna module.

When a participant running an application with the same app_key scans a code, that participant is automatically taken out of their current session and brought into the session of the participant displaying the code. This triggers ConjureKit.

By examining the pose of the displayed QR code relative to itself the joining device also computes a 4x4 matrix M mapping its current world coordinates to the world coordinates of the new session. This matrix is applied to its ARSessionOrigin transform, effectively changing the (physical) pose of the joining participant's Unity scene origin to match the (physical) pose of the scene origin in the session being joined. Note that any visible assets will change (physical) position alongside the Unity scene origin during this step unless corrective action is taken.

Visible assets can be kept in place during the coordinate change by setting Manna's .OnCalibrationSuccess callback. This callback takes the before-mentioned matrix M as argument. One should left-multiply the transforms of any first-level game objects that belong to the old coordinate system to keep them in physical place. The Manna class provides a convenience static method for this, LeftMultiplyTransformByMatrix. E.g.:

using Auki.ConjureKit;
using Auki.ConjureKit.Manna;

public class Main : MonoBehaviour
{
    _gameObject1 = ...;
    _gameObject2 = ...;
    _conjureKit = new ConjureKit(app_key, app_secret);
    _manna = new Manna(_conjureKit);
    _manna.OnCalibrationSuccess += (matrix) =>
    {
        Manna.LeftMultiplyTransformByMatrix(_gameObject1.transform, matrix);
        Manna.LeftMultiplyTransformByMatrix(_gameObject2.transform, matrix);
    }
} 

LeftMultiplyTransformByMatrix is overloaded to directly accept GameObjects for its first argument, as well.

Note that in the above example, any possible children of _gameObject1 or _gameObject2 need not have their transforms left-multiplied by matrix since the transform has already been applied at the parent level. (Indeed, applications may wish to group all of their visible assets inside of a single subtree of the scene, such as to apply Manna.LeftMultiplyTransformByMatrix(..., matrix); only once, at the root of that subtree. See also the next note, however.)

Note that there is a window of time between when ConjureKit. Children of ARSessionOrigin should not be left-multiplied, either.

Public Functions Documentation

function Manna

Manna(
    IConjureKit conjureKit,
    float distanceToSizeRatio =10f
)

Constructor - initializes and registers Manna as a ConjureKit module.

Parameters:

• conjureKit ConjureKit instance
• distanceToSizeRatio (internal) optional max calibration distance to QR code size ratio

function ProcessVideoFrameTexture

void ProcessVideoFrameTexture(
    Texture texture,
    Matrix4x4 projectionMatrix,
    Matrix4x4 worldToCameraMatrix
)

Pass a video frame to Manna so that Manna can scan it for QrCodes. (See code samples.) See the overloaded version of this function for finer-grained control on the coordinate transform pipeline. In this version of the function, textureToProjectedCoordinatesMatrix defaults to the matrix computed by DefaultTextureToNDCMatrices.ARFoundationCameraBackgroundDefaultMatrix().

Parameters:

• texture The texture.
• projectionMatrix The coordinate transform from camera space to normalized device coordinates.
• worldToCameraMatrix The coordinate transform from camera space to world space.

function ProcessVideoFrameTexture

void ProcessVideoFrameTexture(
    Texture texture,
    Matrix4x4 textureToProjectedCoordinatesMatrix,
    Matrix4x4 projectionMatrix,
    Matrix4x4 worldToCameraMatrix
)

Pass a video frame together with a custom texture-to-screen-coordinate matrix to Manna.

Parameters:

• texture The texture.
• textureToProjectedCoordinatesMatrix The matrix mapping texture coordinates, expressed in the form of 4-dimensional coordinate vectors (x, y, 0, 1) with 0 ≤ x ≤ texture.width, 0 ≤ y ≤ texture.height, to screen coordinates. The matrix textureToProjectedCoordinatesMatrix^-1 * projectionMatrix should map from camera space to texture coordinates modulo division by the fourth coordinate of the result, and keeping the first two coordinates; namely, the texture coordinates corresponding to a point (x, y, z) in camera space should be obtained by running unscaled = textureToProjectedCoordinatesMatrix^-1 * projectionMatrix * (x, y, z, 1), scaled = unscaled / unscaled.w, and keeping the first two coordinates of scaled. Conversely, z * projectionMatrix^-1 * textureToProjectedCoordinatesMatrix * (x, y, 0, 1) should be the camera space point at depth z that projects to the point whose coordinates are (x, y) in texture space. Some standard values of the textureToProjectedCoordinatesMatrix are computed by the DefaultTextureToNDCMatrices static class.
• projectionMatrix The coordinate transform from camera space to screen coordinates, respecting the constraints described above.
• worldToCameraMatrix The coordinate transform from camera space to world space.

function OwnDeviceScreenPixelsPerInch

float OwnDeviceScreenPixelsPerInch()

Returns the number of screen pixels per physical inch, to the best of Manna's knowledge of the current device. The implicit guarantee is that dividing Screen.width by this value gives the physical width of the current device screen, in inches, and likewise for Screen.height.

function MillisecondsSinceLastOnCalibrationSuccess

long MillisecondsSinceLastOnCalibrationSuccess()

Returns the number of milliseconds since the last time Manna executed its the OnCalibrationSuccess callback in the current session; is reset after (not right before) that callback; is -1 if OnCalibrationSuccess has not yet been called.

Return: A number of milliseconds

function SetOnBeforeCalibration

void SetOnBeforeCalibration(
    Action< Lighthouse, Action< bool >> onBeforeCalibration
)

Set a callback that is called after a calibration frame has been decided and allows for the developer to do some logic before the calibration and decide if it should continue. If the action is called with true, the calibration will continue, else it will not.

Parameters:

• onBeforeCalibration Callback to invoke with true (calibrate) or false (don't calibrate)

function SetStaticLighthousePoseSelector

void SetStaticLighthousePoseSelector(
    Action< StaticLighthouseData, Action< LighthousePose >> poseSelector
)

Set a callback that is called when a static lighthouse has been scanned. The callback should invoke the provided Action with a chosen LighthousePose from the array. To defer without selecting any lighthouse, invoke the poseSelector Action with null. Note that scanning will not resume until the poseSelector Action has been invoked.

Parameters:

• poseSelector Callback when a static lighthouse has been scanned

function LighthouseRectInMetersFromScreenTopLeft

Rect LighthouseRectInMetersFromScreenTopLeft()

Returns a Rect whose .x .y is the visual bottom left of the lighthouse, in Unity coordinates in the frame of reference of the screen's top left corner.

function LighthouseBackgroundRectInMetersFromScreenTopLeft

Rect LighthouseBackgroundRectInMetersFromScreenTopLeft()

Returns a Rect whose .x .y is the visual bottom left of the lighthouse background frame, in Unity coordinates in the frame of reference of the screen's top left corner.

function SetLighthouseVisible

void SetLighthouseVisible(
    bool visible
)

Display/hide the on-screen instant calibration QR code.

Parameters:

• visible true to display the QR code, false to hide it

function SetLighthousePosition

void SetLighthousePosition(
    Vector2 positionInIMGUICoordinates
)

Sets the position of the center on-screen instant calibration QR code, in pixels from the top left corner of the screen. The position values are truncated such as to prevent the code from being displayed too close to the screen's edge. For example, SetLighthousePosition(Screen.width, Screen.height) will result in a QR code that is inset from the bottom right of the screen.

Parameters:

• positionInIMGUICoordinates The IMGUI (x, y) coordinates of the center of the QR code before the position is truncated to fit inside the screen

function _Init

override void _Init(
    Action onComplete,
    Action< string > onFailed
)

function _Update

override void _Update()

function _StoreLastCalibrationFrameData

void _StoreLastCalibrationFrameData(
    Action< Texture2D, Texture2D, long, long, string > onCalibrationFrameDataReady =null
)

(Internal) Stores debug data from last calibration.

Parameters:

• onCalibrationFrameDataReady Callback that gives calibration video frame, video frame with detected QR corners, timestamp in ms, framestamp, and JSON-encoded coordinates environment

function _StoreVideoFeedDataFrameData

void _StoreVideoFeedDataFrameData(
    Action< Texture2D, long, string > onVideoFeedFrameDataReady =null
)

(Internal) Stores debug data from video feed.

Parameters:

• onVideoFeedFrameDataReady Callback that gives last video frame processed, timestamp in ms, and JSON-encoded coordinates environment

function LeftMultiplyTransformByMatrix

static void LeftMultiplyTransformByMatrix(
    Transform transform,
    Matrix4x4 matrix
)

Utility function. See usage notes in the Manna class overview.

Parameters:

• transform A Unity Transform object
• matrix A Unity Matrix4x4 object

function LeftMultiplyTransformByMatrix

static void LeftMultiplyTransformByMatrix(
    GameObject gameObject,
    Matrix4x4 matrix
)

Utility function. See usage notes in the Manna class overview.

Parameters:

• gameObject A Unity GameObject
• matrix A Unity Matrix4x4 object

Public Property Documentation

property VibrateOnReportPose

bool VibrateOnReportPose;

Cause the device offering calibration to vibrate when it receives a request from a device trying to calibrate. This vibration will occur before the device trying to calibrate sets its state to Calibrated, but after that other's 'VibrateOnReadPose' haptic feedback, if that flag is set.

property VibrateOnReadPose

bool VibrateOnReadPose;

Cause a device to vibrate whenever it scans a lighthouse (QR code) at sufficiently close range. This vibration will occur before the device succeeds in joining an associated session, and before the 'VibrateOnReportPose' haptic feedback of the device offering calibration, if any, if that flag is set.

property VibrateOnCalibrate

bool VibrateOnCalibrate;

Cause the device to vibrate each time its successfully recalibrates from scanning a lighthouse in its environment; this haptic feedback will occur after the haptic feedback from 'VibrateOnReadPose', if the former flag is set.

property OnLighthouseTracked

Action< Lighthouse, Pose, bool > OnLighthouseTracked;

Called when a lighthouse is being tracked even if its not close enough the boolean passed indicates weather its deemed to be close enough or not.

property OnCalibrationSuccess

Action< Matrix4x4 > OnCalibrationSuccess;

This callback is called at the same frame that the pose of the ARSessionOrigin is adjusted to accommodate for joining a pre-existing session via QR code calibration. See the detailed usage notes above in the Manna class overview.

Return: Unity Matrix4x4 specifying the change of coordinates from old to new world coordinates

property OnCalibrationFailed

Action< CalibrationFailureData > OnCalibrationFailed;

Called if calibration fails.

Return: CalibrationFailureData containing Lighthouse, its pose and a failure reason

property OnSetImageTracker

Action< bool > OnSetImageTracker;

Called when image tracker is changed. Will return true if Manna is using the ARFoundation image tracker or false if it's using the native one. Deprecated: this event is no longer triggered/used.

property CalibrationCooldownMs

long CalibrationCooldownMs;

Minimum cooldown time in milliseconds between calibrations. This prevents calibrations from being performed too close in time from each other.