"Secrets of Movies and Games: Why 24 frames per second has become the art standard."

 

Scientific explanation - why movies look fake at 60 fps, but in games, the higher the frequency, the better

shazoo.ru

8 min

August 12, 2020

View Original

If you have been using the Internet for a long time, love movies and games, then there is a good chance that at least once you have come across discussions or heated debates regarding frame rate. Not only in regards to video games, but also movies.

Some argue that 24-30 frames per second is enough, because they can no longer distinguish above. Others claim that the more frames, the better - whether it be a game or a movie.

It is a biological fact that the human eye sees the world at a rate higher than 24 fps. And even higher than 240 fps. Because our organs of vision are not a camera with a mechanical or electronic shutter that opens and closes at a certain frequency. If we perceive the world without some hard frequency, then fair questions arise:

• Why do movies at 48 or 60 frames per second look like they were shot on a smartphone camera?
• Why do films at 24 or 30 frames per second look nicer, more cinematic?
• Why do we prefer 60 fps and above in games rather than 24 or 30 frames?

Let's understand it from a scientific point of view. Attention, there will be serious simplifications in the text, but with links to more detailed information.

The eyes are an optical instrument, not a camera.

You may not be aware, but your eyes are in constant motion. Like a hummingbird, they make many micro movements. This is necessary to form a detailed picture of the environment, since in the maximum “resolution” and "focus" we can only see an area the size of a coin at arm's length.

This part of the retina is called the “fovea,” which is responsible for the clarity and color of what we see, thanks to a high concentration of light-sensitive cones. Our brain "glues" information from this fossa, creating a fairly detailed image of the world. Detailed description -Wiki.

According to research, the physical limit of visual acuity is 6 arcseconds when looking at two parallel lines next to each other.

However, there is the so-called Rayleigh criterion, which sets the limits of angular resolution for any optical instrument - from the human eye to a camera or video camera. If you use the appropriate formula, then under optimal conditions the eye of an ordinary person has an acuity of about 25 arcseconds.

Moreover, the light-sensitive cones themselves are 30 to 60 arcseconds wide, which is 5 to 10 times larger than the minimum line spacing that can be hypothetically distinguished.

However, the eye is not a camera. If you compare the retina to anything, the processor is the best because this part of the eye performs a number of processing functions. Just look at the structure of cones.

Cone structure

Cones are highly specialized light-sensitive receptors that have evolved over millions of years to collect the maximum available information. It's not just a camera sensor that registers a pixel - cones "prefer" when light hits them directly. This property is calledStiles-Crawford effect.

The shape of the top of the cone resembles the conical bottom of the flask, with the Stiles-Crawford effect being associated with the shape. Because if the receptor can reject excess light, then more details can be seen. It's possible that the shape also allows refracted light to be ignored so that the picture doesn't appear washed out.

Note that the diameter of the tip of the cone is approximately 1/3 of the base. Thus, if we take the width of 30-60 arcseconds and divide by 3, then we get the actual acuity of the cone. More or less.

In other words, it turns out that there should be gaps in the image. After all, "sensors" will not be able to determine the distance because their width is the same size.

Constant movement

However, unlike camera sensors, our retina is not fixed. There is a phenomenon called eye tremor - when the muscles vibrate slightly, with an average frequency of 83.68 Hz. The frames arefrom 70 to 103 Hz.

Thanks to these movements, light can fall on different cones. Using temporal sampling and post-processing, the brain can generate a picture of a much larger image from a single receptor fixed in place. Considering that our eyes are also filled with “jelly”, which already changes shape when we move, then why not use the extra information for something useful.

Recognition areas

The sensitive field of a sensory neuron is divided into two parts - central and circumferential, which looks something like this:

Thanks to this division, it is possible to recognize the boundaries of objects with high efficiency. If you simulate a picture, it turns out something like this:

Thus, if vibrations are present, the sensitive cells will register light as they cross boundaries. The result is an image with a resolution at least twice as high.

Similar methods for generating high-quality images are also used in various technological systems. The simplest example is creating a panorama using a smartphone camera. It is enough to turn on the function, draw along a given line, and you get a panorama that cannot be achieved through standard shooting.

How does all this relate to frame rate?

Suppose that if everything we see is constantly changing and “noisy,” then the brain effectively registers information. The brain is capable of supersampling (increasing resolution) and receiving twice as much data. And indeed it is. Moreover, to obtain the best results, the signal must be "noisy" - this phenomenon is known asStochastic resonance.

Moreover, assuming that oscillations with a frequency of 83.68 Hz allow us to double the resolution, what happens if we show a person a moving picture (movie or game) with a frequency less than half the frequency of the oscillations?

It turns out that we no longer receive a signal that changes quickly enough for supersampling. As a result, much of the perceived movement and detail is lost.

What happens if the signal is updated at a rate higher than half the oscillation frequency? As the eye moves, it will register more details, using this information to create a detailed picture of the world. It will be even better if you add "grain" (preferably via temporal anti-aliasing) to fill in the gaps.

Half of 83.68 Hz is approximately 41 Hz. Thus, to obtain high-quality resolution from a picture, it must be “noisy” (similar to film grain) and refresh at rates above 41 Hz. Example - the film "The Hobbit" at 48 fps, or "Gemini" at 60 fps. The same goes for video games.

What will happen to the frequency of 24 or 30 frames per second, since this is below the limit? The eyes will analyze the image twice and will not be able to collect additional information due to the vibrations. The movie or game will turn out to be more “fabulous”, not so detailed. Limited resolution of the format itself.

There are theories that this may be due to motion blur, but in the case of film the effect should not play a big role.

What does all this mean for cinema?

At a refresh rate of 48-60 fps, our eyes see more detail than at 24-30 fps, both in terms of motion and detail. However, we will receive more than 2 times more information, because in addition to surrounding information, the brain also registers movements. Therefore, action scenes with sudden frame changes, a higher frame rate will have better results among the audience.

However, the audience will register more details from the scene than at 24-30 fps. This creates the staging effect. We see not an image, but a scene as a whole, which is hardly possible in reality.

As a visual demonstration, you can conduct an experiment right now. To do this, you need to open video recording on your smartphone and select the frequency in the settings - 60 fps. Look at the screen and move the camera in front of you, it turns out much smoother than if you just move your head.

As a result, to achieve cinematic quality, you need to shoot at a frequency below 41 Hz, but above the frequency where movement becomes choppy - from 16 Hz.

Why did the old TV series look fake?

This was due to broadcast technologies of the last century in NTSC regions, when video was shown at a frequency of 59.99 Hz, but in an interlaced manner. That is, in 1/60 of a second we saw only half of the “lines” Pictures. But the point is that the overall frequency was higher than the fluctuations, which created a soap opera effect.

What does all this mean for video games?

Unlike movies, especially those shot on film at a resolution that is phenomenal even by today's standards, video games have limited resolution. Most of us play at 1080p or 1440p, only in recent years have 4K panels become more affordable. Under such conditions, we are able to distinguish individual pixels and they are distributed in the form of a grid.

Therefore, the issue of resolution and frequency will remain a compromise for some time. Even on new generation consoles you will have to find a balance. However, even 38-43 frames per second, with good grain, timing and anti-aliasing, you can achieve better results. Otherwise, our brain will subconsciously register the pixel grid rather than the content.

Because of this, action games with a lot of movement prioritize frequency, while more static games like strategy games should prioritize resolution. This partly explains the use of dynamic resolution by some developers - when there is no action in scenes, the picture can be rendered in high resolution, when the action increases, the resolution is reduced in favor of a stable frequency.

But if you have the opportunity to play at 60+ fps with 4K resolution, then it will only be more enjoyable.

In addition, the higher the frequency, the faster you can react to what is happening. Although above 144 Hz the positive effect begins to decline and decent results are possible for professional gamers. But this is connected not only with reflexes, but also with the games themselves, since the higher the frequency, the lower the input lag.

Conclusion

This is not all that can be said about frequency and resolution. In particular, the problem of motion sickness and headaches when watching certain videos or playing games deserves attention, but we will talk about this separately.

**Title:**

"Secrets of Movies and Games: Why 24 frames per second has become the art standard."

**Subtitle:**

"A scientific look at visual perception: Why movies and games choose certain frame rates"

**Blog Keywords:**

technoblog, cinematography, games, frame rate, resolution, pixel mosaic, visual perception, Stochastic resonance, eye resolution, retinal cones, eye movement, refresh rate, dynamic resolution, anti-aliasing, image mosaic, visual illusions.

**25 hashtags:**

#Cinema Magic #GameDesign #Frame Rate #Visual Perception #Eye Science #Cinema Art #Game Graphics #TechnoBlog #Cinema Facts #Cinema Theory #Frame Rate24fps #GamerWorld #Video Direction #Screen Resolution #Mosaic of Pixels #Gaming Illusions #Science Approach #Vs Eye Camera #Anti-Aliasing #Gaming Experiments #Mosaic Eye #Technical Aesthetics #Secrets of Cinema #Motion Perception

In the meantime, you can conduct your own experiments with frequency and resolution in various games on PC.

Illustration in the header: Rashed AlAkroka

Просмотры: