U.S. Patent No. 10,977,872: Graphical Style Modification for Video Games Using Machine Learning

Issued April 13, 2021, to Sony Interactive Entertainment Inc.
Filed/Priority to October 31, 2018


U.S. Patent No. 10,977,872 (the ‘872 patent) relates to modifying and enhancing graphics of a video stream via machine learning to accommodate users’ disabilities or preferences. The ‘872 patent details a system which adapts video frames in a buffered video stream of preexisting media. The video frames are treated as a source image to be altered by a style adapted from a target image, and the target image provides a desired style which a user chooses. The system can be implemented in hardware, software, or a combination of the two and uses machine learning to apply the modifications to media, so it potentially could offer a wide range of modifications. The style adapted video stream is generated by applying the style from the target image to the source image while the video buffers, so the user receives the media with their selected style change as the video stream plays, ideally without creating synchronization problems. This manifests as an On-Demand Accessibility system which disabled and nondisabled players can use to customize already existing media to their needs or preferences, and can be used over a variety of media such as video games and movies.

The system described in the ‘872 patent presently relates to Graphical Style Modification but also mentions several more potential functionalities providing support for a range of disabilities and preferences such as: Acoustic Effect Annotation, Color Accommodation, and Scene Annotation. This potentially could make games previously inaccessible to disabled players accessible, and lessen the workload of game developers who want to accommodate a wider audience.



Graphical style modification may be implemented using machine learning. A color accommodation module receives an image frame from a host system and generates a color-adapted version of the image frame. A Graphical Style Modification module receives a first image frame from a host system and applies a style adapted from a second image frame to the first image frame to generate a style adapted first image frame.


Illustrative Claim:

  1. A system for enhancing Audio Visual content, the system comprising: a processor; a memory coupled to the processor; non-transitory instructions embedded in memory for a method of Graphical Style Modification comprising receiving a source image frame from a host system; applying a style adapted from a target image frame to the source image frame to generate a style adapted source image frame, wherein the source image frame is part of a buffered video stream; and generating a style adapted video stream wherein generating the style adapted video steam includes applying the style adapted from the target image frame to each image frame in the buffered video stream for a duration of the buffered video stream that is less than or equal to the time it takes for a Graphical Style Modification neural network to apply the style adapted from the target image frame to the source image frame.