Animation Frames Patents Spotlight
Earlier this week we covered the tight inbetweening patent, U.S. Patent No. 8,451,277 (the ‘277 patent). This patent involved the computer-generation of inbetween frames in animation, inbetweens being the often-tedious frames in-between key frames. Key frame approaches are common in both 2D and 3D animation and are widely considered tedious and time consuming, so inventions like the ‘277 patent are useful in expediting animation. It is important to know such inventions cannot capture artistic principles of animation and cannot animate without an animator at work; such inventions are only tools to aid animators.
Patent Eligibility of Animation Tools
Patents similar to the ‘277 patent, U.S. Patent Nos. 6,307,576 (the ‘576 patent) and 6,611,278 (the ‘278 patent) were at issue in the United States Court of Appeals for the Federal Circuit in 2016, in the lawsuit, McRO, Inc. v. Bandai Namco Games America Inc. The ‘576 and ‘278 patents, both titled, Method for Automatically Animating Lip Synchronization and Facial Expression of Animated Characters, relate to 3D animation of faces talking. The aim of these patents was mainly to automate a 3D animator’s tasks in determining and setting key frames, designated by the mouth shape a character makes for particular sounds. This was achieved by preassigned morph weights associated with certain phonetic sounds (“phonemes”).
The original method for 3D talking animation was much more laborious than even other types of animation and involved the animators having to create a transcription of points in time when a character would pronounce certain sounds. This could easily consume hours of work for seconds of results. McRO, Inc.’s (“McRO”) patents sought to lessen the workload of animators by having rapid, integrated lip synchronization through determining where and when to set keyframes and doing so. This is accomplished via rules applied to the timed transcript which assigns the morph weight presets automatically. The program also considers context of the phonemes for slight variations in similar sounds.
After registering the ‘576 and ‘278 patents, McRO, Inc. sued several video game developers for infringing these animation patents with the methods of automating lip synchronizing the developers were using. The defendant game developers challenged the ‘576 and ‘278 as invalid, and the district court agreed with the defendants that the patents were directed to unpatentable subject matter. The district court followed the two-step Alice test, created in Alice Corp. v. CLS Bank Int’l. The first step under Alice is to determine if the claims are directed to an abstract idea (or a natural phenomenon or law of nature), and if they find it is abstract, the second step is to see if the claims elevate the idea to patent eligible subject matter. Despite finding that under both steps of Alice, the ‘576 and ‘278 patents were patent eligible, the court still ruled it was too broad. McRO then appealed to the United States Court of Appeals for the Federal Circuit, arguing that its patents were not invalid.
On appeal, McRO argued that the claims were not directed towards an abstract idea because the claims generated a tangible product. McRO also argued the claimed process was technological because it created the character animation without an artist’s constant interventions. In the alternative, McRO argued that if directed to an abstract idea, the patents effected an improvement to the technology by employing specific types of rules and were limited in breadth by the focus on phonemes in the patents. The defendants argued that the patents were unpatentable because they were algorithms which could be performed solely with a pencil and paper and that they only claim to make a preexisting process faster.
The Federal Circuit found that the claims were clearly limited in scope by the focus on phonemes and utilized the text of 35 U.S.C. § 101 in its analysis. 101 defines patent eligible subject matter as, “any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof,” and the general understanding that laws of nature, natural phenomena, and abstract ideas are not patentable. Under Alice, if the claims are not directed at an abstract idea, the inquiry ends at step one, and here the court found that the patents were not directed at an abstract idea. Because the ‘576 and ‘278 patents improved on prior art through use of rules, the court also found that even step two of Alice was satisfied.
Thus, the court found that despite the claims were technically improving on an existing technique, they were directed at patent eligible subject matter because there was no evidence this was the only process of animating lip syncing. The defendants also conceded an animator’s process is driven by subjective determinations rather than consistent mathematical algorithms or rules, and the court found these patents were directed to perform a distinct process to automate something previously done by humans. Ultimately the patents were a specific set of rules which produced a desired product. Thus, the court held that the claims of the ‘576 and ‘278 patents were patent eligible.
The ‘277 patent covered earlier this week is a similar invention and is likely patent eligible because, like the ‘576 and the ‘278 patents, it dictates a set of rules which create a desired product. It is also likely limited by the fact it is within a computer-assisted animation system and is saved in a way that can be edited by animators.
Ultimately patent eligibility of animation tools cannot be found if they are directed broadly to an abstract idea in animation. A patent for 3D animation itself is patent-ineligible, but a patent for a method of improving a particular process in 3D animation may be patent eligible.
Frames Per Second
It is important to know that animation for film and animation for video games are often very different, and operate under different philosophies. Both films and animations are most commonly comprised of 24 frames per second (“fps”), while video games run at 60 fps. In animation for film, more frames do not equate to better animation. In games, higher fps can equate to better game feel for a player because more frames mean less input lag. This is mainly applicable in first person shooters, but it is notably not true in all games. Even if a game is rendered in 60 fps, some animations may not take place in 60 fps. For example, the game Cuphead, which is classically hand animated, runs in the classic animation 24 fps, but the game itself runs at 60 fps. Similarly, the more modern installments in the Super Smash Bros. series intentionally create frames of lag on moves and animation (which is common in fighting games), meaning that there are not nearly 60 unique frames per second.
Animators for film have artistic intent behind choosing a framerate and changing the framerate after-the-fact undermines that intent. Artists work in different numbers of frames, often referred to as ones, twos, or threes, to represent how many frames images hold for. “Ones” in 24 fps would mean 24 drawings, “twos” would mean 12 drawings in 24 fps. An animator working alone and traditionally will often work in phases, doing keyframes, inbetweens, cleanup, and color (each animator may have their own process, this is just a generalization of the broad steps). Animators choose these to show different speeds and styles of movement in their characters, and animation is not bound by the same rules as reality.
Animators use the 12 principles of animation to squash the rules that bind real bodies, and ease viewers into a fun experience with more dynamic movements. The 12 principles of animation are: squash and stretch, anticipation, staging, straight ahead action and pose-to-pose, follow through and overlapping action, ease in and out, arcs, secondary action, timing, exaggeration, solid drawing, and appeal. The video by the animator Noodle, mentioned last week, discussed the trend of using tools like the ‘277 patent to add more frames to already existing animations, which is not the intended use of such tools. He goes through how some of these principles cannot be achieved by AI alone, and how applying extra frames after the fact harms the artistic intent. Animation legend, James Baxter, (who has worked on Who Framed Roger Rabbit?, Beauty and the Beast, and Adventure Time for a guest episode, to scratch the surface) even commented on the video agreeing that after-the-fact interpolation harms artistic intent.
Our eyes are not necessarily made to view 60 fps, before photography no one was even sure if a horse ever has all of its legs off of the ground while galloping.
So pieces of movement in animation that are implied between frames, or even are choppy, are intentional, and do not necessarily mean that a viewer would gain something by having more frames in the “empty” space.
The fps is not what makes an animation good, but fps can make gameplay feel good. It’s important to remember tools that help animation cannot replace an animator or artistic intent, and extra frames in animation after-the-fact do not make an animation better. Support tools, like that described in the ‘277 patent, may be patent eligible because they narrow in on a particular process of expediting elements of animation. After years of development since the 2014 Alice decision, we know that patentability can turn on whether the inventions reflect a technical solution to a technical problem. One useful inquiry is whether the statement of the invention (e.g., in the claims) could be said to capture broader concepts of animation rather than a defined technical improvement. Technical solutions to technical problems are patent eligible; broad concepts merely applied on a computer are not.
Thank you for reading! Check back here for future posts concerning game-relevant IP!