# data[].attributes.attachment_caption
Create an AR Jenga Game with Reality Composer

# data[].attributes.attachment_link.title
Create an AR Jenga Game with Reality Composer

# data[].attributes.body
Check out my marble

# data[].attributes.body
Capture your #workspace  What industries / #business can take advantage of #roomplan ? And often with devices they already have! Here I’m using brilliant

@Polycam3D

Low entry tech - high impact (ceiling)  Reach out for training if you need it

@Apple

@Usingtechbetter

#enterprise

# data[].attributes.attachment_link.title
How to Capture Your Space Like a Pro with RoomPlan

# data[].attributes.attachment_caption
Angry Bird style game in Reality Composer

# data[].attributes.headline
Reality Composer: Angry Birds Style Game

# data[].attributes.subject
Reality Composer: Angry Birds Style Game

# data[].attributes.attachment_link.title
Angry Bird style game in Reality Composer

# data[].attributes.body
Using Scaniverse App, capture the outside world in 3D and analyse with accurate time measurements.

# data[].attributes.attachment_link.title
Commonwealth Games Augmented Reality Special Tutorial

# data[].attributes.attachment_link.title
Learn How to Create AR User Interactions for Your Business

# data[].attributes.body
Perfect for Matariki and Indigenous Astronomy.

# data[].attributes.attachment_link.title
Creating AR family star clusters with Keynote and Reality Composer

# data[].attributes.attachment_link.title
How to Create an AR Scene using Reality Composer

# data[].attributes.attachment_link.title
How to Scan Objects for Augmented Reality

# data[].attributes.headline
9 second AR shopping effect on the brain

# data[].attributes.subject
9 second AR shopping effect on the brain

# data[].attributes.body
With tap apply force behaviour

# data[].attributes.body
USDZ file attached

# data[].attributes.headline
AR for the Australian Curriculum (Primary)

# data[].attributes.subject
AR for the Australian Curriculum (Primary)

# data[].attributes.body
Interesting study

# data[].attributes.attachment_caption
Featured Free Downloads - A 3D model collection by Sketchfab

# data[].attributes.body
Ensure you have logged in

# data[].attributes.attachment_link.title
Featured Free Downloads - A 3D model collection by Sketchfab

# data[].attributes.attachment_link.description
Featured Free Downloads - A 3D model collection by Sketchfab

# data[].attributes.headline
Rakugaki AR for early learners: PDF print out shapes

# data[].attributes.subject
Rakugaki AR for early learners: PDF print out shapes

# data[].attributes.body
Great for portals

# data[].attributes.body
Tutorial for teachers - with Narration - masking objects and adding photos from the camera roll.

# data[].attributes.body
Capture your space using the Measure App and then design your flooring. This is a Keynote template with place holders.

# data[].attributes.headline
Below are Interactive AR Scenes and 3D files based on the Lorax

# data[].attributes.subject
Below are Interactive AR Scenes and 3D files based on the Lorax

# data[].attributes.body
Download the Truffula 3D tree as USDZ below.

# data[].attributes.body
Ensure you export as obj before opening in Procreate.

# data[].attributes.attachment_link.readable_text
Overview

The aim of this app is to let you interact with a range of parametric shape generation algorithms, using the adjacent sliders to adjust parameters and see their effects in real-time. As a starting point, use the examples menu as a basis and the 'Randomize' button () to explore different shapes.

The shape parameter sliders are specific to each algorithm whereas the shell parameters allow you to stretch and offset the generated surface into shell-like forms. It should be fast enough on most modern computers and tablets, but you can turn off real-time updating if you need to.

Background

This is another HTML5 version of one of my Java applets. This one grew out of some experiments using spherical harmonics as a way of doing real-time diffuse lighting and shading simulations. As I tend to only really understand stuff when I can see it, I did a quick viewer in Processing a while ago to help make sure I was getting all the calculations correct.

Some of the visualisations and shapes started to look reasonably interesting, so I polished it up a bit and then did this WebGL version. As the basic infrastructure was already there, it seemed logical to add in some other shape types as well.

Spherical Harmonics are really interesting as they can be used to reduce what is usually an inordinately expensive integration of the diffuse lighting environment into a series of much faster dot products. This is done by approximating the diffuse lighting environment, given as a complex luminous distribution over an intergrating sphere, with a matrix of Spherical Harmonics coefficients.

Using this instead of a standard diffuse surface reflection model can make very detailed and realistic diffuse lighting effects possible in close to real-time. Robin Green explains it better and in much more detail in Spherical Harmonic Lighting: The Gritty Details.

Development

This viewer was probably most inspired by Paul Bourke's page on Spherical Harmonics, as well as the many other supershapes viewers written by others. As a lot of my visualisations of building performance involve overlaying analysis data on coloured grids within a 3D model, I figured it was worth spending some time on the generation and animation of them as a way of developing and testing my own grid libraries.

Interesting Features

I guess the most interesting features of this app are as follows:

Animating the shape transitions entirely on the GPU using interpolating shaders,

Developing SVG-based data sliders that scale and interact reasonably well across desktop and mobile,

Doing a fair bit of work with the parameters of each shape so that the random generation is a bit smarter,

Getting my silhouette calculations to work fast enough in Javascript, and

Implementing an app-wide Undo/Redo system.

© Dr. Andrew J. Marsh, 2014.
View Manipulation

You can interactively adjust the 3D view of the model using a mouse, pen or stylus, or by touch on a tablet or phone. You can also use the items in the 3D View Settings popup.

NOTE: You can use the Shift and Ctrl/Meta keys to adjust the increment of each scroll event or key press.

Keyboard Modifiers

The Shift and Ctrl/Meta keys are used pretty extensively to modify interactive data entry. This applies to all increment buttons, scroll wheel motion, slider controls and input elements.

Shift Key:

Increases input to larger values, usually by ten times or significantly larger increments such as 1 month for dates and 1 hour for times.

Ctrl or Meta Key:

Decreases input to smaller values, usually by one tenth or the smallest reasonable increment such as 1 day for dates or 1 minute for times.

NOTE: You can use the scroll wheel to edit a data value when hovering over any slider, numeric input or even table rows that indicate their editibility.

# data[].attributes.attachment_link.description
Overview The aim of this app is to let you interact with a range of parametric shape generation algorithms, using the adjacent sliders to adjust parameters and see their effects in real-time. As a starting point, use the examples menu as a basis and the 'Randomize' button () to explore different shapes.

# data[].attributes.body
Move around the bin to face in direction a den then tap push pull to collect containers.

# data[].attributes.body
Don't forget to activate animation in Reality Composer.

# data[].attributes.body
Watch these quick how to videos

# data[].attributes.attachment_link.description
Basic tutorials for teachers and students to become content creators with Reality Composer. Augmented Reality App for iPad and Mac.

# data[].attributes.headline
Instructions for embed usdz into websites

# data[].attributes.subject
Instructions for embed usdz into websites

# data[].attributes.body
Editable Keynote slide for students to place their ideas as part of the idea phase.

# data[].attributes.attachment_caption
Using Augmented Reality for Dementia Diagnostics and Therapeutics - Altoida

# data[].attributes.attachment_link.readable_text
Healthcare is currently undergoing a digital transformation, generating a whole new approach to healthcare. Big data, mobile devices, wearable devices, and innovative artificial intelligence are changing the game, opening up a whole new paradigm in medicine.

This digital transformation will change the way we diagnose, monitor, and even treat symptoms of a breadth of diseases, including neurological diseases. For patients, this means being able to take their health into their own hands using new digital technologies, and for national healthcare systems, this means being able to utilize resources more efficiently while simultaneously improving patient engagement and patient outcomes.

Virtual and augmented reality for dementia are two of the newest additions to this digital revolution and have applications spanning the diagnostic and therapeutic spectrum.

Let's take a closer look at how virtual and augmented reality are making their way into the neurological disease space.

Applications of Virtual and Augmented Reality for Dementia

According to Allied Market Research, the market for virtual and augmented reality in healthcare is expected to reach $2.4 billion by 2026. The growing demand for new, innovative diagnostic and therapeutic techniques, alongside increased awareness surrounding applications of virtual and augmented realities technologies, is quickly bringing immersive technologies into the neurological disease space.

Highly personalized, patient-oriented medicine is now within reach and includes augmented reality for supporting dementia patients with everyday tasks and managing behavioral symptoms of dementia, psychological need-based virtual reality experiences, and artificial intelligence to diagnose Alzheimer's disease.

Neurocognitive Training Through Augmented Reality

Neurocognitive deterioration in patients with dementia, particularly those with dementia caused by Alzheimer's disease, is known to negatively impact patients' abilities to complete complex Activities of Daily Living (ADLs), such as shopping for groceries, navigating routines, and preparing drinks and meals. The ADL capacity of a dementia patient greatly influences their ability to live and function independently.

In an effort to find an avenue for patients to practice and potentially improve upon these activities to remain independent, researchers have begun investigating using virtual and augmented reality for dementia patients.

In a 2019 study, researchers aimed to analyze the feasibility and usability of a head-mounted Microsoft HoloLens to support patients with Alzheimer's disease in executing the ADL of tea making. While executing this task, patients received three-dimensional dynamic holographs of the sub-steps required to complete the task.

Although they did not find that the guidance and support provided by augmented reality reduced errors or improved success rates, the overall acceptability of this application was high. Researchers concluded that the biggest impediment was the bulky hardware, and overall, the paradigm of augmented support is generally working, but the implementation itself needs improvement (e.g. through an improved user interface).

Virtual Reality for Mild Cognitive Impairment and Dementia

Virtual reality-based interventions are also making their mark in research and clinical trials. Over the last few years, researchers have investigated the potential of virtual reality to improve balance and reduce fall risk, reduce aggressive behaviors, and improve overall interactions and behaviors with caregivers.

Balance, gait, and fall risk: A 2019 pilot study aimed to assess the feasibility, patient experience, and effects of a novel sensor-based balance training program using an interactive virtual user interface for patients with Mild Cognitive Impairment (MCI). The training included weight shifting and virtual obstacle crossing. When measuring post-intervention effects of the patients with MCI, they found that sway and fear of falling were reduced, indicating this method is beneficial for improving postural control.

Behavioral and psychological symptoms of dementia (BPSD): A 2021 study aimed to develop a virtual reality intervention program for dementia patients in nursing homes to reduce BPSD and assess the program's feasibility. Surveys and questionnaires were utilized to identify specific activities that alleviated BPSD among patients. Patients used arm motions in their virtual environment to choose the virtual location to visit and interact with. The study demonstrated the feasibility, safety, and patient satisfaction of the program and warrants further research to determine if the program can be utilized to reduce BPSD in dementia patients.

Altoida: Applying Augmented Reality for Alzheimer's Diagnosis and Monitoring

At Altoida, we are dedicated to bringing precision medicine approaches to neurological disease diagnosis and monitoring. We are building the world's-first Precision Neurology platform and app-based medical device -backed by 11 years of clinical validation-to accelerate and improve drug development, neurological disease research, and patient care.

By completing a series of augmented reality and motor activities designed to simulate complex ADLs on a smartphone or tablet, Altoida's device extracts and provides robust measurements of brain function across 13 neurocognitive domains. Our device measures and analyzes nearly 800 multimodal cognitive and functional digital biomark e rs. Through the collection of highly granular data from integrated smartphone or tablet sensors, Altoida's device produces comprehensive neurocognitive domain scores.

This method, along with our innovative artificial intelligence, will pioneer fully digital predictive neurological disease diagnosis. After our Breakthrough Device designation by the FDA, Altoida's device will provide patients with a predictive score that will enable a highly accurate prediction of whether a patient aged 55 and older will or will not convert from Mild Cognitive Impairment to Alzheimer's disease.

When this score is taken in conjunction with our neurocognitive domain scores, providers will be able to make specific, personalized conclusions about how neurological diseases are uniquely affecting their patients. This, in turn, will enable a personalized precision approach to treatment and care plan deve l opment for neurological disease patients.

To learn more about augmented reality for dementia or about Altoida's Precision Neurology platform and medical device, contact us today.

# data[].attributes.attachment_link.title
Using Augmented Reality for Dementia Diagnostics and Therapeutics - Altoida

# data[].attributes.attachment_link.description
Healthcare is currently undergoing a digital transformation, generating a whole new approach to healthcare. Big data, mobile devices, wearable devices, and innovative artificial intelligence are changing the game, opening up a whole new paradigm in medicine. This digital transformation will change the way we diagnose, monitor, and even treat symptoms of a breadth of diseases, including neurological diseases.

# data[].attributes.body
Place the Truffula trees in Augmented Reality. Tap on the trees to knock them down.

# data[].attributes.headline
Sketchfab - resource for downloading 3D models

# data[].attributes.subject
Sketchfab - resource for downloading 3D models

# data[].attributes.body
Usdz file - tap to create another 1m ruler on top. To be used for informal and formal measurements.

# data[].attributes.body
Single Glass Dome

# data[].attributes.body
Tap to view

# data[].attributes.headline
Creating a shark simulation with Sketchfab and RC

# data[].attributes.subject
Creating a shark simulation with Sketchfab and RC

# data[].attributes.body
Made a quick tutorial on how to create custom animations from @Sketchfab models into #RealityComposer Watch it here: https://youtu.be/B97NegTm5VI

73A - Great Hammerhead Shark" (https://skfb.ly/6VoZC) DigitalLife3D is licensed under CCA @AppleEDU @Usingtechbetter #AugmentedReality

# data[].attributes.attachment_link.title
Creating animations and simulations with Reality Composer and Sketchfab

# data[].attributes.headline
Creating 3d models using Super Shape Generator

# data[].attributes.subject
Creating 3d models using Super Shape Generator

# data[].attributes.body
Take a look at this tutorials for iPad using FREE web based tool, procreate and Reality Composer.

# data[].attributes.attachment_link.title
Shape Generator and creating AR sculptures

# data[].attributes.attachment_caption
Augmented Reality and Virtual Reality in Medical Devices

# data[].attributes.attachment_link.readable_text
Augmented Reality and Virtual Reality (AR/VR) have the potential to transform health care, delivering altogether new types of treatments and diagnostics, and changing how and where care is delivered. Central to their potential in diagnosis and treatment is their ability to deliver both standard and entirely new types of content in highly immersive and realistic ways, remotely, and tailored to a variety of clinical contexts. Physicians, patients, and caregivers can enlist AR/VR to help them prepare for, or perform, certain treatments or procedures.
On this page:
What Is Augmented Reality and Virtual Reality?

Augmented Reality (AR) is a real-world augmented experience with overlaying or mixing simulated digital imagery with the real world as seen through a camera or display, such as a smartphone or head-mounted or heads-up display (HUD). Digital imagery may be able to interact with real surroundings (often controlled by users). This is sometimes referred to as mixed or merged reality.

Virtual Reality (VR) is a virtual world immersive experience that may require a headset to completely replace a user's surrounding view with a simulated, immersive, and interactive virtual environment.

The term extended reality, or augmented reality and virtual reality, is often enlisted to encompass the two approaches, but for the purposes of this web page, we will use the terms augmented reality and virtual reality.

Some real-world examples of augmented reality and virtual reality applications already being used to treat patients include:

An AR system that overlays medical images onto a patient during an operation to help guide the surgeon's technique.

A VR system that is used to treat post-traumatic stress disorder in army veterans.

A VR rehabilitation therapy that simulates real-life situations to improve physical functions for patients who have experienced a physical disability associated with a stroke or other medical condition.

How Are Augmented Reality and Virtual Reality Impacting Medical Devices?

AR/VR can deliver some types of clinical services-including some normally delivered only in clinics and hospitals-to patients in their homes or other non-clinical settings. This could enable patients, including the socioeconomically vulnerable and underserved communities, the elderly or disabled, to access needed health care services when accessing them in person would otherwise be difficult, and this could make it easier, and more likely, for patients to complete treatment and monitoring regimens.

There are a number of treatment domains in which augmented reality and virtual reality are used to treat patients. Among those domains are:

Pediatric diagnostics and treatments

Pain management

Mental health

Neurological disorders

Surgery planning

Intraoperative procedures

Ophthalmic diagnostics

Telemedicine

Virtual Care

Post operative and other rehabilitation therapies

Relative to current therapies, an AR/VR device may introduce new benefits and novel approaches. Such benefits may be identified across a wide variety of patients, or it may be more beneficial for some patients than others. A device may demonstrate particular benefit for underserved populations (for example, patients with more limited access to medical care) or vulnerable populations (for example, children, patients with mental health or cognitive impairment). However, an AR/VR device may have risks as well, including risks related to the usability of the device (such as neck pain from the weight of the headset), and risks related to the content and images that the AR/VR technologies provide (such as low contrast images, display errors such as location or depth of anatomy, information overload, dizziness, fatigue, or effects on vision).

Examples of probable benefits:

Increase access to necessary health care when accessing in person would be difficult

Improve health care professional's ability to prepare for certain treatments

Fulfill unmet medical needs

Mitigate preoperative anxiety

Make procedures less invasive

Accelerate diagnoses

Allow for self-directed care

Examples of probable risks:

Cybersickness

Head and neck strain

Cybersecurity risks

Privacy risks

Distraction in the operating room

Effects of AR/VR on populations

Unknown side effects and/or risks (particularly in pediatrics or other vulnerable populations)

Potentially worsening disparities in diagnostics and treatment

List of Medical Devices that Incorporate Augmented Reality and Virtual Reality

The FDA has reviewed and authorized for marketing a growing number of devices through 510(k) clearance, granted De Novo request, or Premarket Approval with AR/VR across many different fields of medicine-and expects this trend to continue.

The FDA is providing this list of medical devices that incorporate AR/VR marketed in the United States as a resource to the public about these devices and the FDA's work in this area.

Contents of this list:

The FDA assembled this list by searching FDA's publicly facing information. The list is not meant to be an exhaustive or comprehensive resource of medical devices that incorporate Augmented Reality and Virtual Reality. Rather, it is a list of medical devices that incorporate A/R and V/R across medical disciplines based on information provided in the summary descriptions of their marketing authorization document.

Updates to this list: The FDA plans to update this list on a periodic basis.

Augmented Reality and Virtual Reality News and Updates

FDA Resources

Contact Us

If you have questions about augmented reality, virtual reality, or other digital health topics, please visit Ask a Question About Digital Health Regulatory Policies.

# data[].attributes.attachment_link.title
Augmented Reality and Virtual Reality in Medical Devices

# data[].attributes.attachment_link.description
Augmented Reality and Virtual Reality are transforming health care, delivering new types of treatments and diagnostics, and changing how/where care is delivered

# data[].attributes.body
You will need to login into Sketchfab to download his USDZ animations and models.

# data[].attributes.attachment_caption
Reality 20Composer 20Book 20of 20Spells 20updated 20with 20AR 20and 20Fonts

# data[].attributes.headline
Book of Spells: Updated Version with AR and fonts

# data[].attributes.subject
Book of Spells: Updated Version with AR and fonts

# data[].attributes.body
Usdz file of region

# data[].attributes.body
For layering 2d files

# data[].attributes.body
Double Glass Dome

# data[].attributes.headline
Keynote sound hunt template with image placeholders

# data[].attributes.subject
Keynote sound hunt template with image placeholders

# data[].attributes.attachment_caption
Augmented Reality - System Capabilities - iOS - Human Interface Guidelines - Apple Developer

# data[].attributes.attachment_link.readable_text
Augmented reality (or AR) lets you deliver immersive, engaging experiences that seamlessly blend virtual objects with the real world.

Using the device's camera to present the physical world onscreen live, your app superimposes three-dimensional virtual objects, creating the illusion that these objects actually exist. Depending on the experiences your app offers, people can reorient the device to explore the objects from different angles, interact with objects using gestures and movement, and even join other people in multiuser AR experiences. For developer guidance, see ARKit.

Offer AR features only on capable devices. If your app's primary purpose is AR, make your app available only to devices that support ARKit. If your app includes features that require specific AR capabilities, or if AR features are optional in your app, don't show people an error if they try to use these features on a device that doesn't support them; instead, simply avoid offering the feature on an unsupported device. For developer guidance, see Verifying device support and user permission.

Creating an engaging, comfortable experience

Let people use the entire display. Devote as much of the screen as possible to displaying the physical world and your app's virtual objects. Avoid cluttering the screen with controls and information that diminish the immersive experience.

Strive for convincing illusions when placing realistic objects. Design detailed 3D assets with lifelike textures to create objects that appear to inhabit the physical environment in which you place them. Using information from ARKit, you can scale objects properly and position them on detected real-world surfaces, reflect environmental lighting conditions and simulate camera grain, cast top-down diffuse object shadows on real-world surfaces, and update visuals as the camera's position changes. To help avoid breaking the illusion you create, make sure your app updates scenes 60 times per second so objects don't appear to jump or flicker.

Consider how virtual objects with reflective surfaces show the environment. Reflections in ARKit are approximations based on the environment captured by the camera. To help maintain the illusion that an AR experience is real, prefer small or coarse reflective surfaces that downplay the effect of these approximations.

Use audio and haptics to enhance the immersive experience. A sound effect or bump sensation is a great way to confirm that a virtual object has made contact with a physical surface or other virtual object. Background music can also help envelop people in the virtual world. For guidance, see Playing audio and Playing haptics.

Minimize text in the environment. Display only the information that people need for your app experience.

If additional information or controls are necessary, consider displaying them in screen space. Content in screen space appears fixed to a consistent location either in the virtual world or, less commonly, on the device screen. It's typically easy for people to find and view content in screen space because it remains stationary while the underlying AR environment moves with the device.

Consider using indirect controls when you need to provide persistent controls. Indirect controls are not part of the virtual environment - instead, they are 2D controls displayed in screen space. If people need access to persistent controls in your app, consider placing the controls so that people don't have to adjust how they're holding the device to reach them. Also, consider using translucency in an indirect control to help avoid blocking the underlying scene. For example, the Measure app uses screen space to display a mix of translucent and opaque controls that people use to measure objects in the real world.

Anticipate that people will use your app in a wide variety of real-world environments. People may open your app in a place where there isn't much room to move around or there aren't any large, flat surfaces. Clearly communicate your app's requirements and expectations to people up front to help them understand how their physical environment can affect their AR experience. You might also consider offering different sets of features for use in different environments.

Be mindful of people's comfort. Holding a device at a certain distance or angle for a prolonged period can be fatiguing. To help avoid causing fatigue, consider placing objects at a distance that reduces the need to move the device closer to the object; in a game, consider keeping levels short and intermixed with brief periods of downtime.

If your app encourages people to move, introduce motion gradually. For example, you might not want to make people dodge a virtual projectile as soon as they enter your AR game. Give people time to adapt to the AR experience in your app and then progressively encourage movement.

Be mindful of people's safety. When people are immersed in an AR experience, they're not necessarily aware of their physical surroundings, so making rapid, sweeping, or expansive motions might be dangerous. Consider ways of making your app safe to operate; for example, a game could avoid encouraging large or sudden movements.

Using coaching to get people started

Before people can enjoy an AR experience in your app, they need to move their device in ways that lets ARKit evaluate the surroundings and detect surfaces. In iOS 13 and later, you can use the built-in coaching view to show people what to do and provide feedback during the initialization process. You can also use the coaching view to help people reinitialize AR - a process known as relocalization - after an AR experience is interrupted by, for example, people switching briefly to a different app. For guidance on relocalization, see Handling interruptions; for developer guidance, see ARCoachingOverlayView.

Hide unnecessary app UI while people are using a coaching view. By default, the coaching view appears automatically when initialization or relocalization starts, so you should be prepared to hide unrelated UI to help people focus on the coaching view's instructions.

If necessary, offer a custom coaching experience. Although you can configure the system-provided coaching view to help people provide specific information - such as the detection of a horizontal or vertical plane - you might need additional information or want to use a different visual style. If you want to design a custom coaching experience, use the system-provided coaching view for reference.

Helping people place objects

Show people when to locate a surface and place an object. You can use the system-provided coaching view to help people find a horizontal or vertical flat surface on which to place an object. After ARKit detects a surface, your app can display a custom visual indicator to show when object placement is possible. You can help people understand how the placed object will look in the environment by aligning your indicator with the plane of the detected surface.

When people place an object, immediately integrate that object into the AR environment. Although surface detection quickly and progressively refines accuracy, it's best to avoid waiting for more accurate data before placing an object. Use the information available to respond instantly when people place an object; then, when surface detection completes, subtly refine the object's position if necessary. For example, if people place an object beyond the bounds of the detected surface, gently nudge the object back onto the surface. For developer guidance on refining an object's position, see ARTrackedRaycast.

Consider guiding people toward offscreen virtual objects. Sometimes, it can be difficult for people to locate an object that's positioned offscreen. When this is the case, you can help people find such objects by offering visual or audible cues. For example, if an object is offscreen to the left, you could display an indicator along the left edge of the screen that guides people to point the camera in that direction.

Avoid trying to precisely align objects with the edges of detected surfaces. In AR, surface boundaries are approximations that may change as people's surroundings are further analyzed.

Incorporate plane classification information to inform object placement. For example, only let people place a virtual piece of furniture on a plane that's classified as "floor," or require a plane to be classified as "table" in order to place a virtual game board.

Designing intuitive, delightful object interactions

Let people use direct manipulation to interact with objects when possible. It's more immersive and intuitive when people can interact with onscreen 3D objects by touching them directly, than by using indirect controls in screen space. However, in situations where people are moving around as they use your app, indirect controls can work better.

Let people directly interact with virtual objects using standard, familiar gestures. For example, consider supporting a single-finger drag gesture for moving objects, and a two-finger rotation gesture for spinning objects. For guidance, see Touchscreen gestures.

In general, keep interactions simple. Touch gestures are inherently two-dimensional, but an AR experience involves the three dimensions of the real world. Consider the following approaches to simplifying user interactions with virtual objects.

Respond to gestures within reasonable proximity of interactive virtual objects. It can be difficult for people to be precise when aiming to touch specific points on objects that are small, thin, or placed at a distance. When your app detects a gesture near an interactive object, it's usually best to assume that people want to affect that object.

Support user-initiated object scaling when it makes sense in your app. For example, if your app lets people explore an imaginary environment, it probably makes sense to support object scaling because your app doesn't need to represent the real world. On the other hand, if your app helps shoppers decide on furniture to buy, letting people scale a chair object doesn't help them visualize how the chair will look in a room.

TIP Regardless of the purpose of your app, don't use scaling as a way to adjust the distance of an object. If you enlarge a distant object in an effort to make it appear closer, the result is a larger object that still looks far away.

Be wary of potentially conflicting gestures. A two-finger pinch gesture, for example, is similar to a two-finger rotation gesture. If you implement two similar gestures like this, be sure to test your app and make sure they're interpreted properly.

Strive for virtual object movement that's consistent with the physics of your app's AR environment. People don't necessarily expect an object to move smoothly over a rough or uneven surface, but they do expect objects to remain visible during movement. Aim to keep moving objects attached to real-world surfaces and avoid causing objects to jump or vanish and reappear as people resize, rotate, or move them.

Explore even more engaging methods of interaction. Gestures aren't the only way for people to interact with virtual objects in AR. Your app can use other factors, like motion and proximity, to bring content to life. A game character, for example, could turn its head to look at a person as they walk toward it.

Designing a great multiuser experience

When multiple people share your app's AR experience, each participant maps the environment independently and ARKit automatically merges the maps. For developer guidance, see isCollaborationEnabled.

Consider enabling people occlusion. If your app supports placing virtual objects behind people who appear in the device's camera feed, enhance the illusion of reality by letting the people occlude the objects. For developer guidance, see Occluding virtual content with people.

When possible, let new participants enter a multiuser AR experience. Unless your app requires all participants to join before the experience begins, consider using implicit map merging to let new people quickly join an ongoing AR experience. For developer guidance, see isCollaborationEnabled.

Reacting to real-world objects

You can enhance an AR experience by using known images and objects in the real-world environment to trigger the appearance of virtual content. For example, an app that recognizes theater posters for a sci-fi film could cause virtual space ships to emerge from the posters and fly around the environment. Another example is an app for an art museum that presents a virtual tour guide when it recognizes a sculpture. To enable experiences like these, your app provides a set of 2D reference images or 3D reference objects, and ARKit indicates when and where it detects any of these items in the current environment. For developer guidance, see Recognizing images in an AR experience.

When a detected image first disappears, consider delaying the removal of virtual objects that are attached to it. ARKit doesn't track changes to the position or orientation of each detected image. To help prevent virtual objects from flickering, consider waiting up to one second before fading them out or removing them.

Limit the number of reference images in use at one time. Image detection performance works best when ARKit looks for 100 or fewer distinct images in the real-world environment. If you need more than 100 reference images, you can change the set of active reference images based on context. For example, a museum guide app could ask permission to use location services to determine the part of the museum a person is in, and then look only for images displayed in that area.

Limit the number of reference images requiring an accurate position. Updating the position of a reference image requires more resources. Use a tracked image when the image may move in the environment or when an attached animation or virtual object is small compared to the size of the image.

Communicating with people

If you must display instructional text, use approachable terminology. AR is an advanced concept that may be intimidating to some people. To help make it approachable, avoid using technical terms like ARKit, world detection, and tracking. Instead, use friendly, conversational terms that most people will understand.

Unable to find a surface. Try moving to the side or repositioning your phone.

Unable to find a plane. Adjust tracking.

Tap a location to place the [name of object to be placed].

Tap a plane to anchor an object.

Try turning on more lights and moving around.

Insufficient features.

Try moving your phone more slowly.

Excessive motion detected.

In a three-dimensional context, prefer 3D hints. For example, placing a 3D rotation indicator around an object is more intuitive than displaying text-based instructions in a 2D overlay. Avoid displaying textual overlay hints in a 3D context unless people aren't responding to contextual hints.

Make important text readable. Use screen space to display text used for critical labels, annotations, and instructions. If you need to display text in 3D space, make sure the text faces people and that you use the same type size regardless of the distance between the text and the labeled object.

If necessary, provide a way to get more information. Design a visual indicator that fits with your app experience to show people that they can tap for more information.

Handling interruptions

ARKit can't track device position and orientation during an interruption, such as when people briefly switch to another app or accept a phone call. After an interruption ends, previously placed virtual objects are likely to appear in the wrong real-world positions. When you enable relocalization, ARKit attempts to restore those virtual objects to their original real-world positions using new information. For developer guidance, see Managing session lifecycle and tracking quality.

Consider using the system-provided coaching view to help people relocalize. During relocalization, ARKit attempts to reconcile its previous state with new observations of the current environment. To enable these observations, you can use the coaching view to help people return the device to its previous position and orientation.

Consider hiding previously placed virtual objects during relocalization. To avoid flickering or other unpleasant visual effects during relocalization, it can be best to hide virtual objects and redisplay them in their new positions.

Minimize interruptions if your app supports both AR and non-AR experiences. One way to avoid interruptions is by embedding a non-AR experience within an AR experience so that people can handle the task without exiting and re-entering AR. For example, if your app helps people decide on a piece of furniture to purchase by placing the item in a room, you might let them change the upholstery without leaving the AR experience.

Allow people to cancel relocalization. If people don't position and orient their device near where it was before an interruption, relocalization continues indefinitely without success. If coaching people to resume their session isn't successful, consider providing a reset button or other way to restart the AR experience.

Indicate when the front-facing camera is unable to track a face for more than about half a second. Use a visual indicator to indicate that the camera can no longer track the person's face. If you need to provide text instructions in this situation, keep them to a minimum.

Suggesting ways to resolve problems

Let people reset the experience if it doesn't meet their expectations. Don't force people to wait for conditions to improve or struggle with object placement. Give them a way to start over again and see if they have better results.

Suggest possible fixes if problems occur. Analysis of the real-world environment and surface detection can fail or take too long for a variety of reasons - insufficient light, an overly reflective surface, a surface without enough detail, or too much camera motion. If your app is notified of these problems, use straightforward, friendly language to offer suggestions for resolving them.

Insufficient features detected.

Try turning on more lights and moving around.

Excessive motion detected.

Try moving your phone slower.

Surface detection takes too long.

Try moving around, turning on more lights, and making sure your phone is pointed at a sufficiently textured surface.

AR icon

Apps can display an AR icon in controls that launch ARKit-based experiences. You can download this icon in Resources.

Use the AR glyph as intended. The glyph should be used strictly for initiating an ARKit-based experience. Never alter the glyph (other than adjusting its size and color), use it for other purposes, or use it in conjunction with AR experiences not created using ARKit.

Maintain minimum clear space. The minimum amount of clear space required around an AR glyph is 10% of the glyph's height. Don't let other elements infringe on this space or occlude the glyph in any way.

AR badges

Apps that include collections of products or other objects can use badging to identify specific items that can be viewed in AR using ARKit. For example, a department store app might use a badge to mark furniture that people can preview in their home before making a purchase.

Use the AR badges as intended and don't alter them. You can download AR badges, available in collapsed and expanded form, in Resources. Use these images exclusively to identify products or other objects that can be viewed in AR using ARKit. Never alter the badges, change their color, use them for other purposes, or use them in conjunction with AR experiences not created with ARKit.

Prefer the AR badge to the glyph-only badge. In general, use the glyph-only badge for constrained spaces that can't accommodate the AR badge. Both badges work well at their default size.

Use badging only when your app contains a mixture of objects that can be viewed in AR and objects that cannot. If all objects in your app can be viewed in AR, then badging is redundant.

Keep badge placement consistent and clear. A badge looks best when displayed in one corner of an object's photo. Always place it in the same corner and make sure it's large enough to be seen clearly (but not so large that it occludes important detail in the photo).

Maintain minimum clear space. The minimum amount of clear space required around an AR badge is 10% of the badge's height. Other elements shouldn't infringe on this space and occlude the badge in any way.

Resources

Related

Developer documentation

Videos

# data[].attributes.attachment_link.title
Augmented Reality - Technologies - Human Interface Guidelines - Design - Apple Developer

# data[].attributes.attachment_link.description
Augmented reality (or AR) lets you deliver immersive, engaging experiences that seamlessly blend virtual objects with the real world. Using the device's camera to present the physical world onscreen live, your app superimposes three-dimensional virtual objects, creating the illusion that these objects actually exist.

# data[].attributes.body
To view as USDZ or bring into Reality Composer add add multimedia.

# data[].attributes.wish_content.attachment_props.text
iPad not only allows you to view #AR for #business, #enterprise, #training BUT also offers free App #realitycomposer to create AR with no coding skills required +easy deployment. Is your business using iPads to their potential? @Usingtechbetter @apple #highered #healthsafety

# data[].attributes.wish_content.attachment_props.original_text
iPad not only allows you to view #AR for #business, #enterprise, #training BUT also offers free App #realitycomposer to create AR with no coding skills required +easy deployment. Is your business using iPads to their potential? @Usingtechbetter @apple #highered #healthsafety https://t.co/T7spsWqnyd

# data[].attributes.attachment_link.embed_code
iPad not only allows you to view #AR for #business, #enterprise, #training BUT also offers free App #realitycomposer to create AR with no coding skills required +easy deployment. Is your business using iPads to their potential? @Usingtechbetter @apple #highered #healthsafety pic.twitter.com/T7spsWqnyd
— Paul Hamilton (@PaulHamilton8) May 19, 2022

# data[].attributes.attachment_link.metadata.attachment_props_source.data.text
iPad not only allows you to view #AR for #business, #enterprise, #training BUT also offers free App #realitycomposer to create AR with no coding skills required +easy deployment. Is your business using iPads to their potential? @Usingtechbetter @apple #highered #healthsafety https://t.co/T7spsWqnyd

# data[].attributes.attachment_link.metadata.attachment_props.original_text
iPad not only allows you to view #AR for #business, #enterprise, #training BUT also offers free App #realitycomposer to create AR with no coding skills required +easy deployment. Is your business using iPads to their potential? @Usingtechbetter @apple #highered #healthsafety https://t.co/T7spsWqnyd

# data[].attributes.attachment_link.metadata.attachment_props.text
iPad not only allows you to view #AR for #business, #enterprise, #training BUT also offers free App #realitycomposer to create AR with no coding skills required +easy deployment. Is your business using iPads to their potential? @Usingtechbetter @apple #highered #healthsafety

# data[].attributes.headline
Adding Images and Keynote drawings to Reality Composer

# data[].attributes.subject
Adding Images and Keynote drawings to Reality Composer

# data[].attributes.body
Tutorial showing how to add images and Keynote drawings to Reality Composer.