It's a common misconseption that camera sensors (and our eyes too for that matter) have "red", "green", and "blue" filters as you say. It's not like that. It's a filter alright, but it can let in light in a range of wavelengths at varying filtration rate. It's quite common that what's referred to as the red filter also lets in some light in the deep blue since our eyes work that way too (look up human cone cell responsivity). Debayering is part of the color science, but the most important thing is to guess what color each particular combination of site activations should be. Where each value is the "red", "green", and "blue" sites respectively, the expected site activations for pure green could be [10%, 40%, 8%]. This depends greatly on the characteristics of the sensor array, but from these values, you can make a pretty good guess at what color made it. (bonus: dogs only have 2 cone types. You can imagine how much less information that is, but how it's still sufficient to distinguish a lot of colors). To think of it another way. Our bodies are extremely good at these kinds of input to interpretation problems, not just our eyes. There's a guy who built a vest which have lots of points that give the skin a small electric shock all over his torso. He programmed it to do sentiment analysis on tweets he was shown and it would give distinct outputs depending on the sentiment of the tweet he read. The actual outputs were completely arbitrary and had no real meaning except that they were different whenever the sentiment was different, and vice versa. This worked incredibly well and he was able to guess if a tweet was negative or positive without reading it. He used it to follow hashtags and just know the general sentiment around the topic in real time. Just like our eyes interpret a set of inputs, this guy essentially turned his torso into a capable sensor by just feeding it correlating inputs.
Please correct me if I'm wrong, but aren't the color filters on a sensor designed around the sensor's native color space, rather than the response of the human eye? I know that the response of our cones to different frequencies isn't as simple as "red, green, and blue" because each type of cone responds to a wide range of frequencies, with lots of overlap between them. I covered this in my latest video on color spaces. But, if my understanding is correct, every camera sensor has a native color space which is defined by its color filters, and is usually different than the response of human eyes. So for example if my camera's sensor natively operated in Rec.709 (which I realize, most don't) then if I were to look at only the "red" pixels prior to debayering I would basically be seeing the red channel of a Rec.709 image, correct? As I understand it, there's no such thing as an absolute "red," "green," or "blue," but we can define RGB color primaries using CIE 1931, and it's those primaries that a camera sensor is filtering for. I always appreciate an opportunity to learn more, though, if I am wrong :)
I don't think it's like that with camera sensors. These are great questions and i wish i knew the answer for sure. I can only speculate, but here's my educated guess. I have several reasons to believe it's like i describe. Firstly, it's probably much easier to develop software to guess the color from sensor readout than to design hardware that responds just right to every wavelength. I'm not versed in silicon chip material science, but i think it's quite fair to assume. I study cs with specialization in machine learning, and i think this task is quite trivial software wise. Tuning and comparing with real life is probably where the bulk of the work is. I'm not sure of this one, but I believe it's actually not possible to have the color sensors map directly to primary colors. There are many ways to combine wavelengths at different intensities to make the same color. Seems like there would inevitably be gabs in the color space if you don't use high CRI lighting. It seems 3 sensors with varying responsivity is enough to distinguish all colors as our brain can do evidently. Lastly, just looking at the responsivity graph of some cameras who have made it public, it kinda looks like it's not mapping to primaries, though I'd need a spectrometer and know the camera's internal color space to know for sure. I definitely need more insight into this. I guess there's only one way to know for sure, and that's to ask someone who works on this stuff.
@@Omaryllo I think we may be in agreement without realizing it! I agree that the camera sensor can't perfectly distinguish every frequency, and instead processes the raw input from the sensor to create color. And I also agree that the native color space of a camera sensor is likely not a standard one like sRGB (that was only used as an example in my previous comment.) The camera has some native color space which the system converts to whatever gamut has been chosen for the final output. And whatever that native color space is, I think that its three primaries at least *roughly* correspond to Red, Green, and Blue, even if they aren't actually the *same* Red, Green, and Blue used in something like sRGB. My point is that I think the main point of the video still stands, even if it is a bit of a simplification. The video does gloss over the intricacies of how color spaces work (though I cover that in depth in my latest video) and it also fails to mention that the camera is likely using different color primaries from the final output. The main purpose of the video was to highlight how the camera can't actually capture full RGB data at every pixel, and instead has to extrapolate from incomplete information. I'll accept the critique, though, and consider updating this video at some point in the future with a more comprehensive explanation :)
About the camera sensor thing, I can't say that's true for all cameras, especially newer ones, since my experiment used quite an old one, but by looking at some sunlight dispersed through a diffraction grating, I was able to verify that the filters used in my camera had actually very little overlap between frequencies. In other words, what looked to my eyes like a whole bunch of very smoothly spaced out colors, looked to the camera like a block of blue, a block of green, and then a red one, with thin slivers of cyan and yellow separating them
I work in a television station. Many people don't know what you're saying, not even my colleagues. Even say in a video that colors don't exist, only in our brains! I subscribed. I wish you all the best!
That was a cool video, gained a like and a sub. I started today a 'computer photography and AI in mobile devices' course at my university. The first class was a broad introduction about lenses, cameras, sensors, processing etc. Your video helped me visualize and understand some concepts way better. It is fascinating.
Well. Yes and no. Our eyes can truly see colours. Three of them + white. Cameras can only “see” white natively, that’s why they use bayer filters. But our eyes do not have any fillers like that. Each cone cell natively only reacts to a specific span of colour wavelengths. So saying that our eyes can’t see any colour is incorrect. However. Every other colour except red, green, blue and white are basically imagined by our brains by combining information from different cone cells. So. For example. Our eyes truly cannot see colours like cyan, yellow and magenta.
Great Video! I use 3 cameras, a Nikon, a Sony and a Canon. It's always interesting to see how they reproduce color for the same image that is subtly different.
such a cool explanation, loved it when you are actually applying it to the video on running. This is very creative and information practically. Thanks for saving a day to understand this
sh***t i spend months trying to understand how it work, you make it easy and clear in few munites, you are the best and you are so young, i guess you would change the image technology some day, keep going.
Can you make a video about additive and subtractive color mixing? I am confused on what happens especially with hue and saturation. Also, why are basic colors of subtractive color mixing not red, green and blue, but cyan, yellow and magenta?
FYI: the colours you see in a final camera image have practically nothing to do with the demosaic algorithm. What determines the colour is first the CFA dyes and the bandpass colour filter (blocks UV and IR) in front of them. Next would be the colorimetry math that calibrates scene colours to XYZ (and from there to any defined colour space). Finally, what you see is a "graded" image, and colour there goes through a creative process. Even with camera defaults, there's still a creative process involved. Looking at the shot you chose from Phil Holland, he was also looking at flares into the lens, for which the colour will be impacted by the formulation of anti-reflecting coatings in the optical path. Where demosaic algorithm will alter colour is on the very small scale. Say you have some fine scene detail - the colour of an individual pixel on an edge will be determined by demosaic, but the average colour of the scene object (which is what you will be seeing in the finished image) will not to any real extent.
All bayers are still a B&W sensor with a colour filter slapped on top. It is a compromise, and not many manufactures want to really admit that. This is one reason pixel shift exists and is a selling point as manufactures know the bayer filter is not the best at capturing colour. Foveon is a different story.
@@imac3355 Yes, Foveon is a different story. It's a different type of compromise because silicon depth is not a good colour filter, so you're replacing the lack of co-sited colours with poor and noisy (the noise is due to trying very hard to extract accurate colour) colours.
Basically in debayering you need to interpolated not to extrapolate. The main difference between different cameras\sensors is not the debayering algorithm but the different color filters, compare the spectral response of different sensor from different manufacturers.
Why a don't understand is that it's supposed (what I learned) to be sub pixel green, red and blue that code either in 8 or 10bit color information for the most common camera. So how does it come out black and white ?
Amazing explanation. If there are de-bayering algorithms is there such a thing as a RE-bayering algorithm? that is, a software that can create a bayer array out from a jpeg? the loss of resolution wouldn't matter to me.
Partly! Theoretically any color can be used for chroma keying, but green and blue are by far the most common since they tend not to clash with human skintones. Green has been an especially popular choice since the switch to digital cameras because bayer-pattern sensors record much more information about the green channel than any other
This as you call it intelligent algorithms actually just convert one type of lack of data to another type of lack of data. And what people consider as 4k is actually far from the true 4k. People just got used to it and call it 4k. In reality if you downscale 4k image to 1080 you lose just 25% of information, which is barely noticeable.
Hmmm? Different colors at different positions on an array... imagine if instead it was different colours at different points in time too close together to distinguish.
Good video, but the title is horribly misleading. Of course cameras see color. You just described how they do. In fact it's done in a way to mimic how our eyes see color. Perhaps it's more of a philosophical argument, but one could say color doesn't even exist in the way you experience it. It's only how our minds differentiate different wavelengths of light for us. It's actually so abstract an experience that saying cameras don't see color because of the process involved is wildly disingenuous. But I know, clickbait is necessary.
Hello, I just saw this video from a Chinese video site, (a Chinese netizen reposted this video, but I can't get any profit), many people think this video is very good, so do I, so I came to youtu, specializing in Follow you, (using translator)ʚ😊ɞ
It's a common misconseption that camera sensors (and our eyes too for that matter) have "red", "green", and "blue" filters as you say. It's not like that. It's a filter alright, but it can let in light in a range of wavelengths at varying filtration rate. It's quite common that what's referred to as the red filter also lets in some light in the deep blue since our eyes work that way too (look up human cone cell responsivity). Debayering is part of the color science, but the most important thing is to guess what color each particular combination of site activations should be. Where each value is the "red", "green", and "blue" sites respectively, the expected site activations for pure green could be [10%, 40%, 8%]. This depends greatly on the characteristics of the sensor array, but from these values, you can make a pretty good guess at what color made it. (bonus: dogs only have 2 cone types. You can imagine how much less information that is, but how it's still sufficient to distinguish a lot of colors).
To think of it another way. Our bodies are extremely good at these kinds of input to interpretation problems, not just our eyes. There's a guy who built a vest which have lots of points that give the skin a small electric shock all over his torso. He programmed it to do sentiment analysis on tweets he was shown and it would give distinct outputs depending on the sentiment of the tweet he read. The actual outputs were completely arbitrary and had no real meaning except that they were different whenever the sentiment was different, and vice versa. This worked incredibly well and he was able to guess if a tweet was negative or positive without reading it. He used it to follow hashtags and just know the general sentiment around the topic in real time. Just like our eyes interpret a set of inputs, this guy essentially turned his torso into a capable sensor by just feeding it correlating inputs.
Please correct me if I'm wrong, but aren't the color filters on a sensor designed around the sensor's native color space, rather than the response of the human eye?
I know that the response of our cones to different frequencies isn't as simple as "red, green, and blue" because each type of cone responds to a wide range of frequencies, with lots of overlap between them. I covered this in my latest video on color spaces.
But, if my understanding is correct, every camera sensor has a native color space which is defined by its color filters, and is usually different than the response of human eyes. So for example if my camera's sensor natively operated in Rec.709 (which I realize, most don't) then if I were to look at only the "red" pixels prior to debayering I would basically be seeing the red channel of a Rec.709 image, correct?
As I understand it, there's no such thing as an absolute "red," "green," or "blue," but we can define RGB color primaries using CIE 1931, and it's those primaries that a camera sensor is filtering for.
I always appreciate an opportunity to learn more, though, if I am wrong :)
I don't think it's like that with camera sensors. These are great questions and i wish i knew the answer for sure. I can only speculate, but here's my educated guess.
I have several reasons to believe it's like i describe. Firstly, it's probably much easier to develop software to guess the color from sensor readout than to design hardware that responds just right to every wavelength. I'm not versed in silicon chip material science, but i think it's quite fair to assume. I study cs with specialization in machine learning, and i think this task is quite trivial software wise. Tuning and comparing with real life is probably where the bulk of the work is.
I'm not sure of this one, but I believe it's actually not possible to have the color sensors map directly to primary colors. There are many ways to combine wavelengths at different intensities to make the same color. Seems like there would inevitably be gabs in the color space if you don't use high CRI lighting. It seems 3 sensors with varying responsivity is enough to distinguish all colors as our brain can do evidently.
Lastly, just looking at the responsivity graph of some cameras who have made it public, it kinda looks like it's not mapping to primaries, though I'd need a spectrometer and know the camera's internal color space to know for sure.
I definitely need more insight into this. I guess there's only one way to know for sure, and that's to ask someone who works on this stuff.
@@Omaryllo I think we may be in agreement without realizing it! I agree that the camera sensor can't perfectly distinguish every frequency, and instead processes the raw input from the sensor to create color. And I also agree that the native color space of a camera sensor is likely not a standard one like sRGB (that was only used as an example in my previous comment.) The camera has some native color space which the system converts to whatever gamut has been chosen for the final output. And whatever that native color space is, I think that its three primaries at least *roughly* correspond to Red, Green, and Blue, even if they aren't actually the *same* Red, Green, and Blue used in something like sRGB.
My point is that I think the main point of the video still stands, even if it is a bit of a simplification. The video does gloss over the intricacies of how color spaces work (though I cover that in depth in my latest video) and it also fails to mention that the camera is likely using different color primaries from the final output.
The main purpose of the video was to highlight how the camera can't actually capture full RGB data at every pixel, and instead has to extrapolate from incomplete information. I'll accept the critique, though, and consider updating this video at some point in the future with a more comprehensive explanation :)
About the camera sensor thing, I can't say that's true for all cameras, especially newer ones, since my experiment used quite an old one, but by looking at some sunlight dispersed through a diffraction grating, I was able to verify that the filters used in my camera had actually very little overlap between frequencies.
In other words, what looked to my eyes like a whole bunch of very smoothly spaced out colors, looked to the camera like a block of blue, a block of green, and then a red one, with thin slivers of cyan and yellow separating them
@@VideoTechExplained In professional cameras, the ratio of these can be controlled with the "User matrix" function.
Never thought I'd live to see the 3-chip cameras of my youth described like archaeological artifacts.
Your way of making this video was so creative! I love it!
I was actually in other videos looking for pixel size effect on camera sensors, ended up accidently in this genius channel.
Your channel will go far one day. Keep at it my dude. You have what it takes.
You sir, are a genius when it comes to explaining complex science in simple words!
I work in a television station. Many people don't know what you're saying, not even my colleagues. Even say in a video that colors don't exist, only in our brains! I subscribed. I wish you all the best!
great explanation of this complex material. thank you.
That was a cool video, gained a like and a sub. I started today a 'computer photography and AI in mobile devices' course at my university. The first class was a broad introduction about lenses, cameras, sensors, processing etc. Your video helped me visualize and understand some concepts way better. It is fascinating.
Amazing video ! You are a genius and a great teacher ! Thank you so much !!!
awesome video, super underrated channel you got here!
Good job explaining a complex subject in layman terminology!
honestly you captured what my professor taught me in a whole semester, but much better.
In fairness, human eyes can't see colors either. We just have three kinds of cone cells that react differently to different wavelengths.
Our brain imagines colors.
Well. Yes and no.
Our eyes can truly see colours. Three of them + white.
Cameras can only “see” white natively, that’s why they use bayer filters.
But our eyes do not have any fillers like that. Each cone cell natively only reacts to a specific span of colour wavelengths.
So saying that our eyes can’t see any colour is incorrect.
However. Every other colour except red, green, blue and white are basically imagined by our brains by combining information from different cone cells.
So. For example. Our eyes truly cannot see colours like cyan, yellow and magenta.
Great Video! I use 3 cameras, a Nikon, a Sony and a Canon. It's always interesting to see how they reproduce color for the same image that is subtly different.
such a cool explanation, loved it when you are actually applying it to the video on running. This is very creative and information practically. Thanks for saving a day to understand this
Yet another terrific video! Really well done.
this guy is too underrated, great video as always!
sh***t i spend months trying to understand how it work, you make it easy and clear in few munites, you are the best and you are so young, i guess you would change the image technology some day, keep going.
Beautiful Explaination, on point, sufficient and untangling the wonders of technology for a mere photographer
what a great explanation, thank you!
Best kolour explonation so far!
Props to you mister!
Thank you for explaining so well in a video so well though. You deserve so much more views and subscribers!
Dude, you totally knocked it out of the park. Great explanation.
Incredibly intuitive explanation, thank you so much
I did enjoy it actually .... Nice ... Keep making Videos about cameras and videography please and more often..
pretty good content. very well explained
best explaination💪🏽💪🏽.. the video editing is on point. and it conveys the message well
its the best tech channel on youtube if u ask me
Great explanation and editing!
Very cool demonstration! Wonderful!
This is so well explained! Thank you :)
Really nice content and a nice way to structure it :)
Incredible bro, I liked it ! So much information! Thank you very much!
Can you make a video about additive and subtractive color mixing?
I am confused on what happens especially with hue and saturation. Also, why are basic colors of subtractive color mixing not red, green and blue, but cyan, yellow and magenta?
Great explanation with a great edit.
Excellent explanation!
FYI: the colours you see in a final camera image have practically nothing to do with the demosaic algorithm. What determines the colour is first the CFA dyes and the bandpass colour filter (blocks UV and IR) in front of them. Next would be the colorimetry math that calibrates scene colours to XYZ (and from there to any defined colour space). Finally, what you see is a "graded" image, and colour there goes through a creative process. Even with camera defaults, there's still a creative process involved. Looking at the shot you chose from Phil Holland, he was also looking at flares into the lens, for which the colour will be impacted by the formulation of anti-reflecting coatings in the optical path.
Where demosaic algorithm will alter colour is on the very small scale. Say you have some fine scene detail - the colour of an individual pixel on an edge will be determined by demosaic, but the average colour of the scene object (which is what you will be seeing in the finished image) will not to any real extent.
All bayers are still a B&W sensor with a colour filter slapped on top. It is a compromise, and not many manufactures want to really admit that. This is one reason pixel shift exists and is a selling point as manufactures know the bayer filter is not the best at capturing colour. Foveon is a different story.
@@imac3355 Yes, Foveon is a different story. It's a different type of compromise because silicon depth is not a good colour filter, so you're replacing the lack of co-sited colours with poor and noisy (the noise is due to trying very hard to extract accurate colour) colours.
loved it, thanks for the lesson
What a great video! Very clear explanation)
It would be great if you and technology connections did something together!
What a great explanation!! Thanks man
Excellent explanation! Thank you very much!
Really great video man. Thanks!
Basically in debayering you need to interpolated not to extrapolate. The main difference between different cameras\sensors is not the debayering algorithm but the different color filters, compare the spectral response of different sensor from different manufacturers.
Excellent video! 👍
MAAAAAAN THIS VIDEO WAS TOOOOO HELPFUL!!!!!
with this one video u got my sub
Damn ! that some great informations ! thanks a lot for sharing ! Greatly explained !
Not enough time for Fujifilm’s X-Trans?
jesus dude, you are crazy underrated, this video was amazing, good job!
With the possible comeback of analog computing, could we improve image capture?
Great Explanation....
Great explanation
Suggested title: how color camera sensors work
Very helpful. Thank you!
your're a legend! love your content
Well explained!
You are simply great.
Thanks for this!!!
Why a don't understand is that it's supposed (what I learned) to be sub pixel green, red and blue that code either in 8 or 10bit color information for the most common camera. So how does it come out black and white ?
Just make an addendum with info about Fuji X Trans and Sigma Foveon and other kind of filter systems
really very informative video this is 😍😊
0:41 What about Sigma's Foveon sensors?
Great video!
Nicely explained.
Perfect explenation!!!
This dude is a legend
great video!!
Amazing video thank you!
lovely video
Amazing explanation. If there are de-bayering algorithms is there such a thing as a RE-bayering algorithm? that is, a software that can create a bayer array out from a jpeg? the loss of resolution wouldn't matter to me.
Is this the reason for a “green screen” instead of some other color?
Partly! Theoretically any color can be used for chroma keying, but green and blue are by far the most common since they tend not to clash with human skintones. Green has been an especially popular choice since the switch to digital cameras because bayer-pattern sensors record much more information about the green channel than any other
Great video
thank you soo much, !!!!!!
This as you call it intelligent algorithms actually just convert one type of lack of data to another type of lack of data. And what people consider as 4k is actually far from the true 4k. People just got used to it and call it 4k. In reality if you downscale 4k image to 1080 you lose just 25% of information, which is barely noticeable.
Hmmm? Different colors at different positions on an array... imagine if instead it was different colours at different points in time too close together to distinguish.
Mind blown!
Can u talk about the relationship between green and why we use it as a Chroma key
🐐
I guess skynet might not be about to take over.
My camera shoots on film, it sees color just fine.
wow
genius
Good video, but the title is horribly misleading. Of course cameras see color. You just described how they do. In fact it's done in a way to mimic how our eyes see color. Perhaps it's more of a philosophical argument, but one could say color doesn't even exist in the way you experience it. It's only how our minds differentiate different wavelengths of light for us. It's actually so abstract an experience that saying cameras don't see color because of the process involved is wildly disingenuous. But I know, clickbait is necessary.
the CMOS photodiode truly cannot see color. It depends on filters to do so. It's the same reason CMOS can be used in infrared and UV astrophotography.
@@kevinclass2010 Title of video "Cameras Can't Actually See Color", not CMOS photodiodes.
oh my god, colors aren't real ! like birds !
Just do a simple naive debearing algo, take the simple average.
That's why digital cameras are bad at representing color. They only guess the color.
who else came here from TikTok? ;)
Iron your shirt? I can't hear what you're saying over the wrinkles. :P
Hello, I just saw this video from a Chinese video site, (a Chinese netizen reposted this video, but I can't get any profit), many people think this video is very good, so do I, so I came to youtu, specializing in Follow you, (using translator)ʚ😊ɞ
Hats off dude.......
really good video