This page intentionally left blank. ⬇️, ➡️, or spacebar 🛰 to start slidedeck.
📼
📼 📼 📼
📼 📼 📼 📼 📼
📼 📼 📼 📼 Video
📼 📼 📼 📼 📼
📼 📼 📼
📼
What is Video?
- Streams
- Container
- Codecs
- Colorspaces
- Color bars
- Compression
- Chroma subsampling
- Television display
- Pixels
- Bit depth / Color depth
- Pixel width
- Resolutions
- Frame rates
- Aspect ratios
- Interlacing
- Timecodes
- 3D and beyond
Streams 🌊🌊🌊
Video is made up of streams, wrapped in a container, and some of those streams are encoded with a codec
Streams
What is inside the video file can be thought of in terms of streams:
- General/metadata
- Video
- Audio
- Subtitles/text
- Chapters
The extent and availability of the above is contingent on the parameters of the container in use.
Colorspaces
How is color organized, translated, conveyed, and presented?
- RGB
- YUV
- YCbCr
- YCoCg
- XYZ (CIE 1931 color space)
- and more
RGB
❤️ 💚 💙
Red, Green, Blue, like most modern computer monitors.
YUV
- Y = luma (brightness)
- U 💙 and V ❤️ = chrominance (color)
U and V are red-difference and blue-difference
Sometimes people say YUV when they actually mean something else:
YCbCr / YPbPr
- Y = luma (brightness)
- Cb and Cr = chrominance (color)
Cb and Cr are blue-difference and red-difference
YCoCg
- Y = luma (brightness)
- Co and Cg = chrominance (color)
Co and Cg re orange-difference and green-difference
Color bars
Let's look at the SMPTE color bars:
What does this do? Yeah, it checks your colors and makes sure they look good! But how does it do that?
Color bars
Top part: Different colors (white, primary, and secondary colors) set at 75% saturation (Why 75%? Because 100% would be too much and may cause clipping).
Middle row: "Reverse bars"
Bottom left: Subcarrier frequency at -/ phase, white, subcarrier frequency at +Q phase
Bottom right: "PLUGE" element.
Most of this isn't really useful anymore, not useful digitally, although the bottom-left is especially not-useful.
Color bars
Reverse bars?
- If display is set to "blue-only", reverse bars should be the identical color as above bars. Saturation can be checked this way.
Subcarrier frequency?
- the wavelength frequency at which the video is set to stream
PLUGE?
- "Picture Line-Up Generating Equipment"
- Reference black, one slightly more negative, and one slightly more positive
Compression
- Uncompressed: Data stream with no attempts to decrease size
- Compressed: Data is smushed in some way
- Losslessly compressed: Data is smushed in some way, but can be un-smushed
- Lossily Compressed: Data is smushed and this cannot be undone
Chroma subsampling
A thing we do because human eyes can't really see color that well. Chroma subsampling is "the practice of encoding images by implementing less resolution for chroma information than for luma information."
Some common ones for preservation (because we want the best):
- 4:4:4 (not actually subsampling)
- 4:2:2 (this is broadcast standard)
- 4:2:0 (also common in production)
Remember that in video, chroma is in both the Cb and Cr channels so each get subsampled.
Chroma subsampling, explained
What do these numbers mean?
x:y:z
x = width (horizontal) of sample size
y = number of chroma samples in the first row
z = number of chroma samples in the second row
Chroma subsampling, explained
⬛️ = checked pixel
⬜️ = ignored pixel
4:4:4 is full chroma and not subsampling at all because everything is being sampled.
⬛️ ⬛️ ⬛️ ⬛️
⬛️ ⬛️ ⬛️ ⬛️4:2:2 is checking every other pixel horizontally and vertically
⬛️ ⬜️ ⬛️ ⬜️
⬛️ ⬜️ ⬛️ ⬜️4:2:0 is checking every other pixel horizontally but not vertically
⬛️ ⬜️ ⬛️ ⬜️
⬜️ ⬜️ ⬜️ ⬜️
This is, of course, a simplification. Some algorithms (JPG?) don't just sample the first pixel, but sample both together and take the average results. So it's smarter than as basically outlined above.
Television display systems
Good news, there are only three systems:
- NTSC
- PAL
- SECAM
Bad news, they are completely incompatible with each other.
NTSC
National Television Standards Committee
Developed in the USA in 1954.
- Lines: 525
- Frame rate: 29.97 Hz
- Picture resolutions: 720x480; 704x480; 352x480; 352x240
PAL
Phase Alternating Line
Developed in 1967 by the United Kingdom & Germany.
- Lines: 625
- Frame rate: 25 Hz
- Picture resolution: 720x576; 704x576; 352x576; 352x288
SECAM
Séquentiel couleur à mémoire ("Sequential colour with memory")
Developed in France in 1967.
SECAM uses the same bandwidth and resolution (720x576) as PAL but transmits color differently.
Display resolutions
By Jedi787plus - https://en.wikipedia.org/wiki/File:Vector_Video_Standards4.svg, GFDL, https://commons.wikimedia.org/w/index.php?curid=37694717
Display resolutions
But of course it's even more complicated than that.
Pixels
Bit depth / Color depth
Pixel bit depth refers to color depth for each pixel, the amount of color information stored (as bits) in each and every pixel on screen.
For perspective, 1-bit will have just two colors. Each pixel will have to answer a yes/no question for "Am I like this color?" and there is only one color to ask about. 2-bit allows for four colors.
8 bit = 256 colors
10 bit = 1024 colors
16 bit = 4,096 colors
32 bit = 4,294,967,296 colors
Bit depth / Color depth
256 colors isn't very much. But video comes in as three components -- YCbCr or RGB, so it is running the bit depth over one pixel three times.
So when it's per channel, it means 8bit is 256x256x256
10bit is 1024x1024x1024.
Both of these numbers equate to way more colors than humans are capable of perceiving.
Pixel Aspect Ratio
Wait -- pixels can be different SIZES?
Ahem... Pixel Aspect Ratio
That's right, just when you think you were started to understand how video works...
- PAR
The horizontal to vertical ratio of each, rectangular, physical pixel - SAR
The horizontal to vertical ratio of solely the number of pixels in each direction - DAR
The combination (which occurs by multiplication) of both the pixel aspect ratio and storage aspect ratio giving the aspect ratio as experienced by the viewer.
Frame rates
How fast do things go?
- Pre-video rates
- 24
- 25
- 29.97
- 48
- and more
Aspect ratios
- 4:3 (1.33:1) SD
- 16:9 (1.77:1) HD
- 21:9 (2.35:1) Movies
- 19:10 (1.9:1) IMAX
- and more
Practice: Beyonce game
Interlacing
Interlacing is for optimizing perceived motion in lossy video.
Interlacing
"Interlacing issues can be seen during movement, where squiggly lines appear in places of motion. The concept of interlacing involves each frame containing 50% of the line information required for a full picture, and having even and odd frames play back half of the information quickly enough would result in a full-looking image. This was done because it was faster to send video signals, like in the case of television, when this kind of technique is used. Now it just lingers around making everything look like garbage." -- Format-ion: Video playback errors in Beyoncé’s latest music video
TFF, BFF
When interlacing, every other line is skipped. Which gets skipped depends on if the Scan Order is set to TFF or BFF.
TFF = Top Field First (TFF)
BFF = Bottom Field First (BFF)
De-interlacing
MBAFF Macroblock-Level Adaptive Frame/Field PAFF Picture Adaptive Frame Field
- PAFF allows to decide, on a frame basis, whether to encode each frame as a frame or as two fields. When encoded as two fields, it's as if you would have done a separatefields() on the frame.
- MBAFF allows to decide, on a macroblock basis, whether to encode as field or frame. Actually, it works on vertical pairs of macroblocks, so on 16x32 areas. You can easily put two frame macroblocks ( on under the other ), or two field ones ( one with the top lines, the other with the bottom lines ).
Timecodes
Timecodes assign a number to each frame.
BCD: Binary Coded Decimal (HH:MM:SS:FF)
BITC: Burnt-In Time Code
LTC: Linear Timecode
VITC: Vertical Interval Time Code
3D and beyond?
- 3D
- 360 video
- Virtual reality
- all of the above???
Additional Resources
- A/V Artifact Atlas
- Chroma Sampling: An Investigation
- Charles Poynton / Chroma subsampling notation
- Computerphile - Compression
- Dave Rodriguez: Introduction to AV Codecs & Containers
- Digital Video Introduction
- Ethan Gates: Color is Math
- Lurker's Guide to Video Systems
- PAR, SAR, and DAR: Making Sense of Standard Definition (SD) video pixels
- Sustaining Consistent Video Presentation
- 411 on 422
- What is bit rate?
- Video transcoding
Additional Resources, analog
- Dead Media Research Lab
- EIA Video Art Resource Glossary
- LabGuy's World: The History of Video Tape Recorders before Betamax and VHS
- Vector Hack Festival