U.S. Patent No. 10,902,681: Method and system for displaying a virtual object
Issued January 26, 2021, to Sony Interactive Entertainment Inc.
Filed/Priority to June 22, 2018
Overview:
U.S. Patent No. 10,902,681 (the ‘681 patent) relates to displaying a virtual object in an augmented reality (AR) environment. The ‘681 patent details a system with a display device for displaying a virtual object with a camera that captures video of a user’s environment, the area captured on video being larger than the area of the environment displayed by the display device. The system includes a motion predictor which predicts the motion of the video camera, a scene processor which identifies and processes the area of the environment predicted to be displayed at the display device, and an image generator which generates the virtual object for display. The system also has two video cameras, which detect at least partially different areas of the environment. The ‘681 patent also has the image generator generate the virtual object as if exposed to the lighting conditions in the environment and the scene processor detects at least one physical object in the environment to reference for positioning of the virtual object. The ‘681 patent also includes that the image generator may generate an animation for the virtual object corresponding to an interaction between the virtual object and a physical one. The system can also use machine learning to train determinations of relationships between content of captured video image and the motion of the display device. Ultimately, the ‘681 patent could be used for effective implementation of a more convincing augmented reality environment.
Abstract:
A system for displaying a virtual object includes a display device for displaying an environment; a video camera, where the video camera is operable to capture a video image of an area of a user’s environment, the size of the area in the video image being larger than the size of the area of the environment viewable at the display device; a motion predictor operable to predict a motion of the camera; a scene processor operable to identify and process a portion of the captured video image that corresponds to an area of the environment that is expected to be made viewable at the display device, as a result of the predicted motion; an image generator configured to receive an input from the scene processor, and in response thereto, generate a virtual object for display at the display device; and where the image generator is configured to output the virtual object for display in response to a detection of the predicted motion.
Illustrative Claim:
- A system for displaying a virtual object, the system comprising: a display device for displaying an environment; a video camera; wherein the video camera is operable to capture a video image of an area of the environment of a user, a size of the area of the environment in the captured video image being larger than a size of the area of the environment viewable at the display device; a motion predictor operable to predict a motion of the video camera; a scene processor operable to identify and process a portion of the captured video image that corresponds to the area of the environment that is expected to be made viewable at the display device, as a result of the predicted motion of the video camera; and an image generator configured to receive an input from the scene processor, and in response thereto, generate the virtual object for display at the display device, wherein the image generator is configured to output the virtual object for display in response to a detection of the predicted motion of the video camera, wherein the system comprises two video cameras, the two video cameras being configured to capture images of at least partially different areas of the environment, wherein the area of the environment viewable at the display device is less than the area of the environment captured by the two video cameras, and wherein the scene processor is configured to process video image data for an area outside the area of the environment viewable at the display device.