Release details, including backlog items, can be found on the GitHub milestone pages. The complete set of open issues can also be found on GitHub. Mixed Reality Toolkit (MRTK) roadmap. The Mixed Reality Toolkit is built to be cross MR/AR/VR/XR platform by design. The toolkit currently supports Unity 2019.4.x and Unity 2018.4.x.

• Mrtk Github
• Mrtk Github Unity

• MRTK Examples Hub is a Unity scene that makes it easy to experience multiple scenes. It uses MRTK's Scene System to load & unload the scenes. MRTKExamplesHub.unity is the container scene that has shared components including MixedRealityToolkit and MixedRealityPlayspace. MRTKExamplesHubMainMenu.unity scene has the cube buttons.
• ️ WebXR supported (AR) ️ WebXR supported (VR) WebXR not supported: 👌 Hand tracking enabled: 👎 Hand tracking not supported on this device.
• MRTK gitignore for MRTK 2.3. GitHub Gist: instantly share code, notes, and snippets.

Caution

See this page on the new docs website.We've moved so we can provide you with a better docs experience. We will no longer be maintaing documentation on Github.Check out the new site to get started with MRTK in Unity!

MRTK Standard shading system utilizes a single, flexible shader that can achieve visuals similar to Unity's Standard Shader, implement Fluent Design System principles, and remain performant on mixed reality devices.

Example scenes

You can find the shader material examples in the MaterialGallery scene underMRTK/Examples/Demos/StandardShader/Scenes/. A &bittersweet& lesson on copyright. All materials in this scene are using the MRTK/Standard shader.

You can find a comparison scene to compare and test the MRTK/Standard shader against the Unity/Standard shader example in the StandardMaterialComparison scene under MRTK/Examples/Demos/StandardShader/Scenes/.

Architecture

The MRTK/Standard shading system is an 'uber shader' that uses Unity's shader program variant feature to auto-generate optimal shader code based on material properties. When a user selects material properties in the material inspector, they only incur performance cost for features they have enabled.

Material inspector

A custom material inspector exists for the MRTK/Standard shader called MixedRealityStandardShaderGUI.cs. The inspector automatically enables/disables shader features, based on user selection and aides in setting up render state. For more information about each feature please hover over each property in the Unity Editor for a tooltip.

The first portion of the inspector controls the material's render state. Rendering Mode determines when and how a material will be rendered. The aim of the MRTK/Standard shader is to mirror the rendering modes found in the Unity/Standard shader. The MRTK/Standard shader also includes an Additive rendering mode and Custom rendering mode for complete user control.

Performance

One of the primary advantages to using the MRTK Standard shader over the Unity standard shader is performance. The MRTK Standard Shader is extensible to only utilize the features enabled. However, the MRTK Standard shader has also been written to deliver comparable aesthetic results as the Unity Standard shader, but at a much lower cost. One simple way to compare shader performance is via the number of operations that needs to be performed on the GPU. Of course, the magnitude of calculations may fluctuate by features enabled and other rendering configurations. But, in general, the MRTK Standard shader performs significantly less computation than the Unity Standard shader.

Unity Standard shader statistics example

MRTK Standard shader statistics example

Note

These results can be generated by selecting and viewing a shader asset in the Unity inspector, then clicking the Compile and show code button.

Lighting

The MRTK/Standard uses a simple approximation for lighting. Because this shader does not calculate for physical correctness and energy conservation, it renders quickly and efficiently. Blinn-Phong is the primary lighting technique which is blended with Fresnel and image-based lighting to approximate physically-based lighting. The shader supports the following lighting techniques:

Directional light

The shader will respect the direction, color, and intensity of the first Unity Directional Light in the scene (if enabled). Dynamic point lights, spot lights, or any other Unity light will not be considered in real time lighting.

Spherical harmonics

The shader will use Light Probes to approximate lights in the scene using Spherical Harmonics, if enabled. Spherical harmonics calculations are performed per vertex to reduce calculation cost.

Lightmapping

For static lighting, the shader will respect lightmaps built by Unity's Lightmapping system. Simply mark the renderer as static (or lightmap static) to use lightmaps.

Hover light

• See Hover Light

Proximity light

• See Proximity Light

Lightweight Scriptable Render Pipeline support

The MRTK contains an upgrade path to allow developers to utilize Unity's Lightweight Scriptable Render Pipeline (LWRP) with MRTK shaders. Tested in Unity 2019.1.1f1 and Lightweight RP 5.7.2 package. or instructions on getting started with the LWRP, see this page.

To perform the MRTK upgrade, select: Mixed Reality Toolkit -> Utilities -> Upgrade MRTK Standard Shader for Lightweight Render Pipeline

After the upgrade occurs, the MRTK/Standard shader will be altered and any magenta (shader error) materials should be fixed. To verify the upgrade successfully occurred, check the console for: Upgraded Assets/MixedRealityToolkit/StandardAssets/Shaders/MixedRealityStandard.shader for use with the Lightweight Render Pipeline.

UGUI support

The MRTK Standard shading system works with Unity's built in UI system. On Unity UI components, the unity_ObjectToWorld matrix is not the transformation matrix of the local transform the Graphic component lives on but that of its parent Canvas. Many MRTK/Standard shader effects require object scale to be known. To solve this issue, the ScaleMeshEffect.cs will store scaling information into UV channel attributes during UI mesh construction.

Note, when using a Unity Image component, it is recommended to specify 'None (Sprite)' for the Source Image to prevent Unity UI from generating extra vertices.

A Canvas within the MRTK will prompt for the addition of a ScaleMeshEffect.cs when one is required:

Texture combiner

To improve parity with the Unity Standard shader per pixel metallic, smoothness, emissive, and occlusion values can all be controlled via channel packing. For example:

When you use channel packing, you only have to sample and load one texture into memory instead of four separate ones. When you write your texture maps in a program like Substance or Photoshop, you can hand pack them like the following:

Or, you can use the MRTK Texture Combiner Tool. To open the tool, select: Mixed Reality Toolkit -> Utilities -> Texture Combiner which will open the below window:

This window can be automatically filled out by selecting a Unity Standard shader and clicking 'Autopopulate from Standard Material.' Or, you can manually specify a texture (or constant value) per red, green, blue, or alpha channel. The texture combination is GPU accelerated and does not require the input texture to be CPU accessible.

Additional feature documentation

Below are extra details on a handful of feature details available with the MRTK/Standard shader. Rpgflash game heaven.

Primitive clipping

• See Clipping Primitive

Mesh outlines

Many mesh outline techniques are done using a post processing technique. Post processing provides great quality outlines, but can be prohibitively expensive on many Mixed Reality devices. You can find a scene that demonstrates usage of mesh outlines in the OutlineExamples scene under MRTK/Examples/Demos/StandardShader/Scenes/.

Learn to fly 2gaming potatoes. MeshOutline.cs and MeshOutlineHierarchy.cs can be used to render an outline around a mesh renderer. Enabling this component introduces an additional render pass of the object being outlined, but is designed to run performantly on mobile Mixed Reality devices and does not utilize any post processes. Limitations of this effect include it not working well on objects which are not watertight (or required to be two sided) and depth sorting issues can occur on overlapping objects.

The outline behaviors are designed to be used in conjunction with the MRTK/Standard shader. Outline materials are usually a solid unlit color, but can be configured to achieve a wide array of effects. The default configuration of a outline material is as follows:

1. Depth Write - should be disabled for outline materials to make sure the outline does not prevent other objects from rendering.
2. Vertex Extrusion - needs to be enabled to render the outline.
3. Use Smooth Normals - this setting is optional for some meshes. Extrusion occurs by moving a vertex along a vertex normal, on some meshes extruding along the default normals will cause discontinuities in the outline. To fix these discontinuities, you can check this box to use another set of smoothed normals which get generated by MeshSmoother.cs

MeshSmoother.cs is a component which can be used to automatically generate smoothed normals on a mesh. This method groups vertices in a mesh that share the same location in space then averages the normals of those vertices. This process creates a copy of the underlying mesh and should be used only when required.

1. Smooth normals generated via MeshSmoother.cs.
2. Default normals used, notice the artifacts around the cube corners.

Stencil testing

Built in configurable stencil test support to achieve a wide array of effects. Such as portals:

Instanced color support

Mrtk Github

Instanced color support to give thousands of GPU instanced meshes unique material properties:

Mrtk Github Unity

Triplanar mapping

Triplanar mapping is a technique to programmatically texture a mesh. Often used in terrain, meshes without UVs, or difficult to unwrap shapes. This implementation supports world or local space projection, the specification of blending smoothness, and normal map support. Note, each texture used requires 3 texture samples, so use sparingly in performance critical situations.

Vertex extrusion

Vertex extrusion in world space. Useful for visualizing extruded bounding volumes or transitions in/out meshes.

Miscellaneous

A checkbox to control albedo optimizations. As an optimization albedo operations are disabled when no albedo texture is specified. This is useful for controlling remote texture loading.

Simply check this box:

Per pixel clipping textures, local edge based anti aliasing, and normal map scaling are supported.

See also