U.S. Patent No. 10,532,284: Camera based safety mechanisms for users of head mounted displays

Issued January 14, 2020 to Sony Interactive Entertainment Inc.
Priority Date: September 30, 2013

Summary:
U.S. Patent No. 10,532,284 (the ’284 Patent) relates to warning the user of a head mounted display (“HMD”) that they are nearing the edge of the field of view of a camera, or that they are approaching a physical obstacle. The ’284 Patent details a method of warning the user of the HMD through audio, messages displayed on the HMD, or moving the HMD’s display away from the user’s eyes. In one embodiment, the real world obstacle may be rendered to the HMD’s display to alert the user to its location in their real-world environment. A user’s movements and location may be determined through image and depth sensors on the HMD and/or in a user’s room. In some embodiments, the intensity of the warning may increase as a user gets closer to the boundary of the interaction space or a real-world obstacle. 

Abstract:
Methods and systems for warning a user of a head mounted display (HMD) during game play of a video game. A game is executed causing interactive scenes of the game to be transmitted for rendering on a display portion of the HMD. Coordinates of the HMD are determined in a three-dimensional space of a real-world environment in which the user wearing the HMD is present, to identify a current position of the user. A movement of the HMD is determined during execution of the game by identifying a change in one or more coordinates of the HMD in the three-dimensional space. When it is determined that the user is approaching a boundary of an interaction space in a real-world environment, a warning signal is conveyed to the HMD to indicate proximity of the user to the boundary of the interaction space.

Illustrative Claim:
1. A method for warning a user of a head mounted display (HMD), the method comprising: capturing images of a physical environment in which the user wearing the HMD is present, using one or more sensors disposed on the HMD; processing the images to generate a three-dimensional map of the physical environment, the three-dimensional map identifying depth information of real-world objects identified in an area of the physical environment; determining a location of the HMD by tracking movement of the user wearing the HMD in the physical environment as the user is interacting with content presented on a display portion of the HMD, wherein the tracking includes computing relative distance between the HMD worn by the user and one or more of the real-world objects in the physical environment using depth information captured in the three-dimensional map and from inputs obtained from inertial sensors disposed in the HMD; and generating a warning to the user of the HMD upon detecting the user approaching an edge of the area of the physical environment, based on the tracking of the movement of the user, wherein operations of the method are performed by a processor of the HMD.