I'd love to have confocal laser microscope but I need it so little that I cannot justify even the cost of DIY project to get one. Just like I would want to have accurate spectrophotometer, too.
❤ Thanks! Don't tell anyone but I only release videos because I like color grading footage and pretending I'm Michael Bay. All the science junk is an elaborate cover story 😇
Since you're here, and your most recent video is weeks old and has tens of thousands of comments :P @Stuff Made Here: Idea for you: A holster for cops that when the firearm is drawn it automatically calls for backup and EMS. Beau of the Fifth Column did a 9 minute video showing some of the benefits of it. Either try to make one yourself, or help put out the message that this would be good for cops and the people they point firearms at.
Very cool. In the 1990s I worked with a startup developing a commercial confocal laser review station in the semiconductor industry. We scanned the laser in x-y and had a piezo stage for the z-axis. It also required a proprietary frame grabber board that was synced to the laser scanner and z-stage. My work was on the microcontrollers, so I only knew the general optics design. We experienced what we called the pit-particle issue where what we knew to be a solid above the surface (a sub micron calibrated latex sphere) could sometimes show up in the 3D image as a hole.That seems similar to what you were sing on the one image. Our optics engineering team spent a lot of time resolving that, but I don’t recall how they attacked it. Great video. Thanks.
That seems like a problem that could be "solved" (mitigated maybe) by analyzing the curve. If the peak is out of reach there should be a steady climb before, which should be detectable with a proper numerical analysis. Then I would replace this value with a max value. It would at least make it obvious in the image where the 'z cropping' happens, instead of random valleys.
If I may suggest - please consider putting the transimpedance amplifier _very_ close to the photodiode, preferably mount the diode directly piggybacked on the opamp IC (directly soldered, no sockets) onto the DIP8 of the opamp. The capacitance of the long coax between the PD and the TIA limits the bandwidth, in your current setup.
Cheers for the tips! This project really taught me that my EE skills are terrible and I need to spend some time learning the fundamentals. Just barely got the amp working and it really is awful in every way :) Will keep that in mind for the next one, I forsee myself needing a decent photodiode amplifier in the future :)
What's this, I see laser but nothing getting burnt, evaporated or even slightly charred? 🤨 JK that's really cool and I am looking forward to that thing back there in the summer!
If you're still working on this, consider replacing your photodiode and aperture with a linear CCD array. You can use the reflected beam width as feedback for the next Z position to find the focal height in 2-3 iterations instead of scanning.
You could try to have the beamsplitter at the Brewster angle of you laser source and mirror material, could avoid the laser dump, when the optical path is geometrically redesigned.
@@BreakingTaps There are two more reasons for the poor SNR: (1) the laser beam is highly aberrated due to poor alignment quality, and (2) some reflected light will be fed into the laser cavity, which actually affects the output beam intensity. The reflected beam is definitely something you want to avoid.
If the laser is polarized, it might be an option to filter the returning light in that direction to exclude specular reflection from those shiny surfaces. That reduces the return signal very much, but scans for scattering and the surface geometry matters much less, which may otherwise deflect a focused beam in a different direction. It has the same effect as coating the surface matte.
Really cool project! Another way to do this is by using a standard cd or dvd laser head. You cannot scan very large areas, but with very high resolution.
Thanks, and cheers for stopping by! Will take a closer look at those DVD units, are there open source designs/plans/software to control them? I'm not sure I have the skills to hack one myself :)
@@BreakingTaps I think that styropyro has some tutorials on youtube, and they mostly revolve around an lm317 regulator. Also I had an idea for the optimisation which might be interesting: set a grid over the area you're scanning, and go to each point sequentially and move the z axis to find the height of the point. After the first scan look for areas where there is a large change in height and then go back to get more resolution in those areas. This should result in being able to start with a coarser grid which should make the scan a bit faster, although fine detail may be lost in some areas. This was really cool and I don't need a confocal microscope but now I'd really like to try at some point
@@BreakingTaps They are actually not that difficult to control. But, one potential problem is that you need a special 2x2 photodiode to focus properly. Let me try to explain: imagine a 2x2 grid of photodiodes, with them being oriented at 45 degrees. The optics in there are special so that if you are too far away, the beam becomes elliptical from top to bottom, so the top and bottom photodiodes get more light than the left and right one, but if you're too too far away, the left and right ones get more light than the top and bottom ones. And if you're perfectly in focus, the laser for forms a perfect circle and all photodiodes get the same amount of light. I know Hamamatsu offers photodiodes that can be used for that, but they'd probably cost you a kidney, and the chip that's currently in there doesn't expose the focus data to you. If you want pictures for what I'm talking about, wikipedia has images in the article about the CD player. But driving the coils themselves is relatively easy, their movement is proportional to the voltage across them. Actually, a laser unit should be a pretty good confocal stage if I'm not completely mistaken, since you have the exact same thing happening that's happening here, and it's all one unit.
@@BreakingTaps dude there are absolutely tons of wonderfully documented open source projects utilizing the high end opto-mechanics in a standard DVD/Blu-ray optical pickup. a bit of googling, there are a few things on hackaday and elsewhere. Really incredible documented projects out there. One of my favorites will show up in Google image results… You will see a nice looking finished product with all of the circuitboards in a purple color. They got some really good results with that, and I think A set a fresh eyes like yours could really push the project further. Billions of dollars of R&D have gone into developing the optical pickups in some of the higher end units. They absolutely engineered those things to within an inch of their life 😂 I’ve got dozens of them, so if you need some parts just let me know and I’ll ship you a box no charge. Also have a bunch of really high-end hamamatsu optical sensors and components from a bunch of equipment I’ve repaired or salvaged. So I may just have something you need. I’ll gladly chip in and help however I can
I think one of the reasons why you're getting artifacts is because of how metals reflect light : it's mostly specular reflection(it's directional, in your case parasitic) and not diffusive (unidirectional, what you're detecting). metal oxides, however, tend to build up in tiny crevices of metal surfaces and they are dielectrics, so they create tiny diffusive areas in otherwise specular metal which might look like noisy height changes.
This is a bugaboo when working on PCB's under a microscope generally. They are SHINY, and what you see is not a nice illuminated surface but a reflection of the light source. That's why we use ring lights to work on PCBs, at least that eliminates reflections from the surface plane, although not from solder fillets.
I think you are right about reflections. And to add something else. I think some reflections are courses by lights other than the laser. Like the image with the "EC". The left, white part of the image could have a lot of room light. Then the lights were turned of for a little while. And then a small light in the room is turned on what causes the reflections on top of the characters. But I could be wrong because the scans took multiple days and then I would expect to see more day night effects.
@@Daniel-zw6gz What type of temperature fluctuation is seen in this workspace over 24 hours? The PCB has many components, each with a different CTE. Also conformal coatings can confuse the sense of sharp focus.
I was thinking about reflectivity, too, after a recent experience trying to use photographs to create 3D models using Meshroom. Reflections make it basically impossible for the algorithms to figure out what's going on in the images. Anything reflective has to be dusted with a matte powder to get it to work.
Jesus christ! this is a 2Msub channel level production quality! how you manage! I mean, i loved all your videos but you really outdid yourself with this one!
This is really well done! I watched an OpenFlexure video and this was suggested. Really shows that a lot of time went into the project and into the presentation. Wishing you the best of success.
Sounds like something Hank Green (SciShow, Crash Course, Journey to the Microcosmos, etc etc etc) could be interested in... but no idea how to get his attention.
Cool project. Try to modulate ur laser beam with chopper or so and demodulate ur photodiode signal at the same frequency. This will remove much of noise.
Good point! I was about to suggest some physical wavelength filtering as well, just to remove signals from ambient light, but some decent modulation will remove a lot of the ambient (DC in any case)
Hey, awesome video! I see there already was some comments about reflective surfaces here - and that is indeed the probable cause for the inverted shield structure. When the beam hits the slightly higher point of the shields edge it is scattered into the room and not back into the lense making the signal weaker. This would explain the smaller values in some of the curves peaks too. Alexander Sannikov mentioned in their comment that the metal oxides being diffuseve cause noise in the signal. Indeed if the samples were to be prepared beforehand (like they do for electron microscopes) to be compleately matt with a coating of a white matt substance the laser would behave more uniformly along the entire surface.
Fascinating. Great job and good explanation of the optical path. I wonder if it would be faster to run the scan like an atomic force microscope, riding the surface by tracking the amplitude of the photodiode. Move a step in x, then move the z up or down to maximize the amplitude, repeat. Advance y each time you hit the end of x. The assumption is the surface is often flat, or at least has a gentle slope, so instead of scanning the whole z stack for an x,y point, just search around the last height (z) each time you move to a neighboring point.
I would think you could then attempt to recognise the trend from previous points in the direction you're currently moving and predict which way you need to scan in z.
Great video, 20 minutes fly by, just the right level of detail. From my experience stacking microscope shots and focus failures, it will be that 'specular highlights' (very shiny surfaces) break your algorithm. You'll be assuming that the thing you are scanning has unchanging topology (which is true), but accidentally assuming that when you move the microscope stage that the pattern of light reflected would not change (untrue), because unless the surface is perfectly matte, the 'brightness' does not 1:1 correlate with 'in focus', it also correlates with 'shiny thing pointing at lucky angle to send light back to sender'. Imagine you were scanning a microscopic disco ball, if you strapped a torch to just above a camera lens and moved it closer and further, you'd see that different panels would suddenly become very bright, and others very dark. It confuses normal cameras with their contrast based focus detection (and even sometimes phase detection). If you want to test what I'm talking about, scan some glitter or a retroreflective tape. I would spray the item to be scanned with a misting of 'airplane glue' smelling hairspray in the giant cans from 50cm away. It will 'matte' your surfaces sufficiently. If it kills the retroreflection of tape, it will be scannable by your algorithm.
Very cool, i worked on a similar project a few years back using DVD optical pickup heads and its internal optical path, which is indeed a confocal arrangement with an electromagnetic vertical adjustment which provides a high degree of precision.. my failure was in the X-Y stage.. might have to revisit with openflexure but it was something that worked as an arduino shield.
I was about to comment on your complaints about the slowness of the system given the insane amount of points that you have on the z-scans (eg. 13:10), but then I heard the explanation at 21:01... Anyway, it seems that you have quite a lot of resolution, a quick and easy way of trading it for a bit more speed or range would be adjusting the gearing of the steppers. Also those 250 ms of "network response time" are weird and likely worth profiling.
YOU GIVE ME HOPE. I'm in awe of this video. My intellect is only barely sufficient to grasp what the components do. It's true that I feel bitter in myself for not being smart enough to do something like this. But I am profoundly happy that you are. It gives me hope for humanity's future that there are people as brilliant as you in the world. It really does. I wish you all the very best
In a previous life, I learned a lot about CT scanners. The basics are simple: take a big hubless wheel. Put an X-ray source on the wheel, pointed at the centre of rotation. Put a detector on the other side of the wheel. spin the wheel, gather signal strength data, do *LOTS* of computation, get a detailed image, called a slice. Throw in a motorized bed to move the patient in and out of the machine, and you can scan a volume by gathering *many* slices. OK, so at some point, Toshiba had a bright idea: they used an X-ray source that produced a line of light, not a point, and opposite it, they put a line of sensors. That way, they got multiple slices in a single pass. By about a decade ago, they had such density that they could image an entire human heart in a single rotation, and since they were doing about 3RPM IIRC, that meant that they could make CT scan videos of a heart as it pumped! Anyhow, it seems to me that you could do the same: miniaturise the laser+sensor system and put more in side by side. I haven't looked into the optics of it, but if you could use a bar diode and a linear ccd, you might be able build a system that can scan the entire Y-axis at once. A 64hour scan would come down to 8 hours. I also wonder if there are better scanning patterns. eg: diagonals. Start at [MaxX, Miny,MaxZ] and command a move to [MinX,MinY,minZ], then move to [MinX,MinY,MinZ+1], then scan to [MaxX-1,MinY,MaxZ], etc.
I dont know if it was already mentioned in the comments, but. I have some experience in 3d scanning, basically extracting height information from a series of images, and the general recommendation for an object not to bee too white, or too black in albedo and what is sometimes more important not to be too shiny. From your samples you can clearely see that varnished surface of a PCB net you the worst result, and edges of the shiny metallic coin tricked sensor into wrong depth, even though the xy shape was correct. Try taking pictures of embossed areas of paper money and see if it will make a real difference. Amazing video though!
12:15 you can possibly increase the scanning speed dramatically by making a feedback system, like autofocus, which tracks the surface or, equivalently, the height at which the intensity is maximum, which is the maximum of the curve so when x-y changes, you need only change the z enough to stay at the maximum
Great idea. And if you can implement it with phase detection similar to SLR cameras, you don't need to even guess which way you need to move to get into focus. The end result would be pretty much like optical LP needle. That said, laser scanner LP players do exist. Maybe one could take some ideas about those. I think the tech is old enough that the patents have already expired but are publicly available.
I really can't stand how at such exhuming scientific and video quality you only have 25k subscribers. Luckily you have already managed to capture the attention of major scientific channels.
funny video ! It reminded me one of my internship : I had to build an interferometer to mesure optical elements with 10 nm accuracy for the Z dimension. But instead of laser I used a white light source. Thanks for the video, it brought back some great memories :)
I have a personal ambition to build an STM. I will consider myself successful when I can see atomic dislocations in a metallic crystal. So far I have designs but no working prototypes. Just an occasional image before a tip crash. I have no real need for one of these but my purpose of designing one is one of learning. At the same time, I end up creating supporting hardware that has other uses. Videos like this one are inspirational. I work in the semiconductor industry and need to worry about particle contamination at the sub-micron level. The difference between what I do at home and at work is about 4 orders of magnitude in cost.
Nice project. Very good details on what this microscope is and how it works, very good. About the optimization "issue" you commented, I see it as a simple thing to run an arduino (or any other microcontroller) driving the steppers directly and sampling the data. I am not sure how much ADC resolution is required to get nice data, but many uC have 10~12bit ADCs and sample at > 20kS/s, which is way over the stepper frequency. A simple loop will take stepper to "home" and do step; take sample; print sample; next. repeat. You do not need to "verify" the position, just let the actuator (stepper) do it job. There is for sure a time need for the the actuator to stop moving (settle position), but that can be measured by looking at the samples, and dynamically determine how long to wait for. Also running the scan at constant speed, will help to vibration and "inertia problems", ie accelerations. I am not familiar at all with the OpenFlexure device, so maybe I do not see how it would not work like I described.
Congrats on making a convocal! I build microscopes for biological research in my phd for living. Some tips: it seems there are is lot of aberration in the spot at the pinhole, resulting in bad contrast. Use a mirror (in focus) and try tilting and moving the second lens to get a bettet spot. You probably want a kinematic mount for that lens to make the alignment easier.
I feel like a faster way to scan this would be to use a PID to try and keep focus by affecting the Z height of the stage. You could start by moving to (0,0) and finding the optimal z height with the highest signal intensity, and then begin a slow raster scan of the subject. Provided the x-y movement was slow enough that the PID had ample time to try and keep the focus optimal, you could just directly save your Z height as the output, essentially eliminating a whole dimension from your problem
first thing that comes to mind is reflectivity would greatly affect the results and create ALLOT of noise.. because after all, your relying on the reflection of the laser to plot the data.. so if the surface of what your measuring has a higher reflection than that of other points, then it will throw it off, because your measuring the reflection at a specific Z point, as opposed to the actual surface height. this is why electron microscopes work so well, because an electron cares less about reflectivity of light upon the surface that its measuring.
Two thoughts: 1. How about using a nozzle as a pinhole. 0.1mm nozzles are pretty cheap, and you could always drill out the back if you need more room for the beam to expand. They come with a convenient screw thread for mounting and fine adjustment along the beam path! 2. RobRenz has a video on an autocollimator mirror system with some interesting differential screws for the precise positioning of mirrors. Been trying to figure out if it is possible to make a 3D printed laser autocollimator...
Wow only just saw this video. For faster scanning you could use an x-y galvo stage and f-theta lens like used in a laser marker or SLM 3D printer, that way you can raster scan an area with a fixed subject vs. having to move the subject around. You could effectively steer your beam around as fast as you could read the photodiode and your reading electronics would become the main bottleneck.
I think the biggest issue is the range of reflectivity. I had this issue with 3d scanning using photogrammetry. Your photos will be greatly affected anywhere there is solder on that PCB, or other curved highly reflective areas. It might be worth trying to air brush a flat color on everything to make the reflective properties uniform, which should normalize your readings and greatly increase the resolution and accuracy of your photos. If air brushing adds too much material to the sample, there might be some gaseous coating options like 2D boron nitride that would only add 1 atom to the height. Good work though. Your channel will do great :)
I've worked with confocals, and the easiest way to get the pin hole and X/Y/Z stage is to use an optical fibre for the pinhole then move the fibre tip. Also we used photomultiplier tubes which give you much better sensitivity.
Ah, that's clever! I assume you have to use a single mode fiber, or would a larger multi-mode work? That would certainly cut down on the complexity of the optics and alignment!
It'd be cool to do this with multiple wavelengths. IR vs. visible topologies for example. Another thing worth looking at might be to use a mild diffuser and concentric photodiodes instead of a single diode or a camera. Because those contours are slopey and fairly linear, you could get instantaneous readings (if I understand the principle at work here lol) and if you set up the diode array intelligently, you possibly reduce the problem with finding the wrong maximum. Another option is reducing the number of measurements with active scanning by processing the samples in real time; if you combine this with the multi-diode thing, maybe the differences between the diodes can get you closer faster. Of course, I get that there's a lot more mechanical complexity involved when you have four or more pinholes, and need chambers for those holes.
Man! your videos are awesome. We get months worth of education in science and technology in every one of them. Your detailed way of explaining is incredible. Thanks
did you actualy try imaging a clean, flat surface? It seems arbitrary but you might get information about your noise (and if you try different materials, there might even be different noise values due to different reflectances of these materials).
To be able to program your "Quantum Google Home according to your @Feyman 3D-Printing Recycling unit would be a very cool translational unit for Artistic expression in the home that has never ending customizability"
As mentioned in the end of the video, scan in z axis. If remove the pin hole and change the photodiode to a camera. A single trip scan result 100 bitmap image, let’s say 100steps in a scan trip. By calculate the coefficient of the 100 image, the height image can calculated. That change 10000s single data captures to cpu extensive calculation. In the setup above, a modern cpu with GB memory is needed, and computing time may longer than scan time.
Not sure if anyone mentioned this before but you can make the raspberry pi camera into a photodiode by integrating all pixels. This kind of avoids all the extra electronics for amplification. You can also increase signal by binning pixels which will just increase signal strength at the cost of reduced resolution.
Yep, that's the approach I took at first. It was still too slow :( To overcome shot noise on the sensor you have to leave the pinhole light fairly concentrated on the sensor, so there is only a small region to bin together and find average intensity. Still needs 1-3s of exposure at max gain and ISO.
Since you already have the raspberry pi camera, you could probably reduce the issue with the local maxima: Take the expanded beam, and take only the top half of the spot (so before the beam splitter). Instead of the pinhole and the photodiode, just use the raspberry pi camera. Out of focus away from the lens will look like a half circle on the top half of the sensor, out of focus near the lens will look like a half circle on the top half of the sensor. With the right amount of calibration, you could even do away with the entire Z-positioning, and just use the spot size as a distance indicator. This has the benefit of being fully independent of the reflectivity of the target (as long as enough gets reflected for the camera, but you could also change the photodiode brightness to compensate.
I am working regularly with a confocal laser scanning microscope since the beginning of this year and I had no clue this technology existed before and now YT randomly suggests me your video! what coincidence. And yes you are right about them costing a fortune. they are obscenely expensive... on the other hand our particular microscope takes seconds for a scan, so maybe the price tag is reasonable...
While I am only marginally interested in the topic of metrology and microscopes, I will now be looking into the openflexure project, because I am very much intrigued by micromanipulation.
20:57 Hilarious to me, that I know nothing about microscopy or any kind of precision metrology, but when you were talking about the images you took, and the issues they had, I came up with this exact same solution.
So, when laser scanning Total Internal Reflection (TIR) optics they have to be painted black in order for the laser to see them properly. Some of your data looks like the shinyness of the items affected your data by messing with exposure rates or something. You might want to try a matte black or something to improve clarity. Though it might affect details, it might be a worthwhile experiment since you're looking for relative depth and not a "color picture" Very cool microscope by the way, I don't even know why yet, but I want one!
Makes sense! Might give that a shot, would be an easy test to see how much the specular reflections affect things. And I hear ya about wanting instruments even without a need... that accurately sums up most of my shop experience :)
Love the project! Miss doing this stuff. Man if it takes a week to image a surface you wonder how much temperature change over that time frame would impact the sample. Its a small area you're sampling but the whole object would chmage temp. For efficiency buildings can drop temp overnight. You're measuring such small features I wonder if it might be enough to cause some of those anomalies in the data plots. It's always a challenge coming up with the right algorithms to filter the way you want but not kill good data or pick the wrong point.
Really cool work. Couple of questions/comments. Did you calibrate your sensor data with a dark measurement? What's the spot size achieved? We have built a confocal microscope at our lab. we do line scans to get faster results (easy if you know the acceleration and speed of the stage). Limiting factor is of course how fast you sample the data. We limit 'stop and measure' at each point to situations where we need more precise measurement or small area scans. We also have a dynamic profile measurement mode where we can measure periodically moving samples. Also I see that you have a structuring laser. You could make custom precise pinholes using black hard paper. You could over complicate your optics to get a smaller spot size. Or use white light as your source which result in different focal point for each wavelength and use a spectrometer to get the profile.
Punching my ticket, this channel is gonna be in the hundreds of thousands some day. Damn good explanation and camera work, + the start skit. Maybe even pushing a million like TOT...
I’d say that when it scans a rounded surface, some of the light is reflected away at an angle which would show as a lower intensity in the data. That would explain why the top of the rounded surface kind of showed but the sides didn’t.
I used to use a KLA Tencor Confocal Review Station when I worked for a certain large semiconductor company. Normally we just used the optical microscope. I thought it was a really neat tool but we couldn't save pictures from it to our database. The only way we could save them is to print them which was expensive and slow.
If you use a ccd sensor without the pinhole, you could scan for the smallest radius, so intensity doesn't matter. that way you can scan both reflectivity and height at the same time, instead of trying to correlate both
Really interesting; for pinhole making, you might try an acupuncture needle - sharp and round - works well, and of course various gauges are available.
I would suggest trying something call white light confocal instead of intensity only pinhole confocal, but you need a spectrometer which can also be quite easily made as theres little need for narrow FWHM, a commercial version will be chrocodile from precitec, it gives depth info and you only need to scan in XY, plus with different chromatic abberation lens, you can get different scan range, something like 100um to few mm dynamic range
Hehe, on the todo list actually! I discovered the Nanovea version of the same thing (I think they call it "chromatic confocal" instead, just to be confusing i guess) while working on this project, but was too invested in the regular confocal to quit at that point. I agree, it looks great! Should be so much faster, and it's such a clever mechanism using/abusing chromatic aberration. I think the optics should be pretty straightforward, right? Just a bunch of regular lenses stacked up to increase the chromatic aberration at each lens?
@@BreakingTaps yes, the optics are very straight forward, calibration is a bit tricky, but doable in a home shop i think, we bodged together a chrocodile taped to a benchtop cnc router and did some mapping that way, it worked alright as at high speed weight will average out imperfections, didnt even need filtering. one thing chrocodile can do is 4KHz readout, with a tiny DSP, i will think with a FPGA demo board it will be equally good.
I think that the problems you had with the pcbs were due to the fact that the green surface absorb most of the power from your blue laser; that, and the lake of flux that seems to cover almost everything would blur out almost all the details. You also had problems with the screw because the sharp edges and tilted faces shine the light away from the collection path. I think your system would work best for measuring stacked flat surfaces, something like mems or microchips.
You should go for backup holders for the mirrors with play to the mirrors but holding them in case of the magnets not Holding any more. For example when carrying around your machine or a heavy magnetic pulse hitting your building.
Have you tried pointing the laser at the photo receptor for the expected duration of a scan? That’ll quantify the noise that’s outside of the microscope stage.
Very cool! My knowledge of optical scanning goes no further than end user where I push button and machine goes brrt, so it's great to see the community come together with advice. I'm very excited to see an upgraded model!
Thank you for the video and sharing your journey. Inquisitive as I am about so much, the lessons you share of in these challenges including the good the bad and the ugly enlighten and inspire. Your success is a journey filled with experiences won and lost, always best shared. Thanks again well done.
Speed: I'd modify your algorithm to not do the whole z stack. You can just do a few measurements above and below any known adjacent maximum. It's kind of like doing delta method/Taylor approximation with verification--if you cannot verify a maximum then you expand the z range. Sure, there might be fake maxima, but that should correct on the next pixel over. Also, consider the reflectivity of your scanned surface: I'm not surprised that you had problems scanning the metallic surfaces.
Maybe you could use a camera and a sheet of LEDs with different colors to create a normal map? The normal map could then be used to define the expected signal, and reduce the number of samples needed to take a picture.
My guess is that for some images the normal of the surface matters very much (e.g. the screw). Thus you don't get the light back as it's reflected in a different direction
Yeah I think you're probably right, between being highly reflective and also at a very bad angle, probably very little light probably made it to the sensor.
@ 17:39, your issue is with the zener diodes encased in glass, the solder joint is fairly clear but the round glass is scattering the light and there goes the resolution
Laser scanning confocal microscopes are known to miscalculate shiny metal surfaces, when they reflect (multiple times) back into the diode directly. maybe thats why your results were hard to evaluate. Try with coarse/dull surfaces -> should be getting better results than coins/smd 's. :)
I have suspicions as to why the top of the shield did not scan properly. I have had huge issues with this scanning shiny objects. Because of the Fresnel effect, the more oblique you get to a surface or edge, you can get a double reading, because all objects become a mirror at the right angle. (The mirage effect). So I often try to scan expensive cars that the owners will not let you use dulling spray on, so i get very similar artifacts. When I scan tiny objects, I have tried dulling spray, but the paint dots are too large, so I started to make my own using pigment that I grind very finely, and spray with a pasche AB turbo airbrush. The spray is so fine, you are able to keep the fidelity of a tiny object and get the dulling you need to cancel the Fresnel effect. I hope that actually helps you. You are a very smart guy and I feel pompous telling you what to do differently. Good luck.. This was an incredibly interesting video to me.
Super cool project may I suggest that you replace the photo diode with a photo multiplier tube yes these require a high Voltage supply of good quality but the resulting gain factor, low noise and bandwidth with a output in the volt range so little or no additional amplification needed, hard to beat with any other tech. PM tubes are abundant and low cost on ebay HV usable power supplies cost more but can often be found there too.
I'm floored. That was awesome. I finally have a project to pursue that will force me to crack the seals of signal processing, python, and optics. Thanks so much!
This is amazing! It's fantastic that you can achieve this level of detail using 3d printed components. The explanation was very useful and covers a topic which isn't well covered elsewhere. Thank you.
You may also consider a diode sensor array for faster imaging. I would imagine more optical points of interest pulsed at the correct intervals to avoid interference would significantly decrease the scanning time.
This would not work in the way you suggested. The array would sample your Airy spot and not the sample space. Instead this idea is used today to generate super resolution.
You could also get a really quick profile using a camera in reflected brightfield. Image a focus series in Z, and then do an Extended Depth of Focus in FIJI (ImageJ). This will return a height map identical to the one you are getting with the confocal. Spiral scanning and photomultiplier tubes will also greatly increase your speed. Finally, a Teensy running in DMA mode can do dual channel ADC at 1.1 MHz per channel which can then be dumped from RAM to an SD card for rapid storage and later processing.
Great project, going to try to repeat it myself) One question, why scanning the whole Z axis? When you have height-to-intencity curve (which is looking somewhat parabolish), just a single measurement makes you to choose a height from one of the two points on that curve. So you could make another measurement in any other point to distinguish them. Moreover this method should work even when your scanning range does not cover all the heights of a specimen, you can calculate the height out of the range.
at college they talked about improving scanning electron microscopes by using a DLP mirror array to be able to scan so much faster. May be worth investigating if you're interested, but I have no idea the technical challenges that this would require to overcome.
Very cool project indeed and well done. It also makes me appreciate what a commercial confocal scanner can do. The one we have at work scans @ 4kHz and gives a distance value every sample (without requiring z-motion) and the optical device is way smaller. But yeah, it does cost a buck or two more ;-)
I have absolutely zero need for a "confocal laser microscope" but your channel is so incredibly well done I can't help but watch.
I'd love to have confocal laser microscope but I need it so little that I cannot justify even the cost of DIY project to get one. Just like I would want to have accurate spectrophotometer, too.
I've never heard of one but now I need one
It's one of those impulse buys frfr
Dude your videos look soooo good. Also, sweet microscope :)
*Videos shot with a confocal laser microscope*
❤ Thanks! Don't tell anyone but I only release videos because I like color grading footage and pretending I'm Michael Bay. All the science junk is an elaborate cover story 😇
@@BreakingTaps where's the explosions then? bay movies are 50% explosions minimum!
@Herr Gerd Soon™ hahaha
Since you're here, and your most recent video is weeks old and has tens of thousands of comments :P
@Stuff Made Here: Idea for you: A holster for cops that when the firearm is drawn it automatically calls for backup and EMS. Beau of the Fifth Column did a 9 minute video showing some of the benefits of it.
Either try to make one yourself, or help put out the message that this would be good for cops and the people they point firearms at.
Very cool. In the 1990s I worked with a startup developing a commercial confocal laser review station in the semiconductor industry. We scanned the laser in x-y and had a piezo stage for the z-axis. It also required a proprietary frame grabber board that was synced to the laser scanner and z-stage. My work was on the microcontrollers, so I only knew the general optics design. We experienced what we called the pit-particle issue where what we knew to be a solid above the surface (a sub micron calibrated latex sphere) could sometimes show up in the 3D image as a hole.That seems similar to what you were sing on the one image. Our optics engineering team spent a lot of time resolving that, but I don’t recall how they attacked it. Great video. Thanks.
That seems like a problem that could be "solved" (mitigated maybe) by analyzing the curve. If the peak is out of reach there should be a steady climb before, which should be detectable with a proper numerical analysis. Then I would replace this value with a max value. It would at least make it obvious in the image where the 'z cropping' happens, instead of random valleys.
If I may suggest - please consider putting the transimpedance amplifier _very_ close to the photodiode, preferably mount the diode directly piggybacked on the opamp IC (directly soldered, no sockets) onto the DIP8 of the opamp. The capacitance of the long coax between the PD and the TIA limits the bandwidth, in your current setup.
Yep. And also using a smaller diode will give better speed (thx lower capacitance) and lower noise.
Cheers for the tips! This project really taught me that my EE skills are terrible and I need to spend some time learning the fundamentals. Just barely got the amp working and it really is awful in every way :) Will keep that in mind for the next one, I forsee myself needing a decent photodiode amplifier in the future :)
What's this, I see laser but nothing getting burnt, evaporated or even slightly charred? 🤨
JK that's really cool and I am looking forward to that thing back there in the summer!
Nice to see you here. Guten Tag!
okay, break time's over. Go work on OSMU some more.
Just _really tiny_ charring on the sample 😉
So fun to always see my most loved youtubers on the same channels I subscribed to 😄😄😄
Well we all like Science 😄
why are you browsing UA-cam videos? Don't you have a CNC mill to be recording for us?
If you're still working on this, consider replacing your photodiode and aperture with a linear CCD array. You can use the reflected beam width as feedback for the next Z position to find the focal height in 2-3 iterations instead of scanning.
You could try to have the beamsplitter at the Brewster angle of you laser source and mirror material, could avoid the laser dump, when the optical path is geometrically redesigned.
Woah! That's new to me, just did some reading. Very cool! I had no idea polarized light had that property.
@@BreakingTaps brewster's angle is the kind of thing when you see it happen, the coffee goes out your nose
even if you're not drinking coffee
@@GeorgeTsiros This comment caused coffee to come out my nose. And judging by my neighbor's shouts, I know from where the coffee was teleported.
@@BreakingTaps There are two more reasons for the poor SNR: (1) the laser beam is highly aberrated due to poor alignment quality, and (2) some reflected light will be fed into the laser cavity, which actually affects the output beam intensity. The reflected beam is definitely something you want to avoid.
If the laser is polarized, it might be an option to filter the returning light in that direction to exclude specular reflection from those shiny surfaces. That reduces the return signal very much, but scans for scattering and the surface geometry matters much less, which may otherwise deflect a focused beam in a different direction. It has the same effect as coating the surface matte.
Really cool project! Another way to do this is by using a standard cd or dvd laser head. You cannot scan very large areas, but with very high resolution.
Thanks, and cheers for stopping by! Will take a closer look at those DVD units, are there open source designs/plans/software to control them? I'm not sure I have the skills to hack one myself :)
@@BreakingTaps I think that styropyro has some tutorials on youtube, and they mostly revolve around an lm317 regulator. Also I had an idea for the optimisation which might be interesting: set a grid over the area you're scanning, and go to each point sequentially and move the z axis to find the height of the point. After the first scan look for areas where there is a large change in height and then go back to get more resolution in those areas. This should result in being able to start with a coarser grid which should make the scan a bit faster, although fine detail may be lost in some areas. This was really cool and I don't need a confocal microscope but now I'd really like to try at some point
@@BreakingTaps They are actually not that difficult to control. But, one potential problem is that you need a special 2x2 photodiode to focus properly.
Let me try to explain: imagine a 2x2 grid of photodiodes, with them being oriented at 45 degrees. The optics in there are special so that if you are too far away, the beam becomes elliptical from top to bottom, so the top and bottom photodiodes get more light than the left and right one, but if you're too too far away, the left and right ones get more light than the top and bottom ones. And if you're perfectly in focus, the laser for forms a perfect circle and all photodiodes get the same amount of light. I know Hamamatsu offers photodiodes that can be used for that, but they'd probably cost you a kidney, and the chip that's currently in there doesn't expose the focus data to you. If you want pictures for what I'm talking about, wikipedia has images in the article about the CD player.
But driving the coils themselves is relatively easy, their movement is proportional to the voltage across them.
Actually, a laser unit should be a pretty good confocal stage if I'm not completely mistaken, since you have the exact same thing happening that's happening here, and it's all one unit.
@@BreakingTaps dude there are absolutely tons of wonderfully documented open source projects utilizing the high end opto-mechanics in a standard DVD/Blu-ray optical pickup.
a bit of googling, there are a few things on hackaday and elsewhere. Really incredible documented projects out there.
One of my favorites will show up in Google image results… You will see a nice looking finished product with all of the circuitboards in a purple color. They got some really good results with that, and I think A set a fresh eyes like yours could really push the project further.
Billions of dollars of R&D have gone into developing the optical pickups in some of the higher end units. They absolutely engineered those things to within an inch of their life 😂
I’ve got dozens of them, so if you need some parts just let me know and I’ll ship you a box no charge. Also have a bunch of really high-end hamamatsu optical sensors and components from a bunch of equipment I’ve repaired or salvaged. So I may just have something you need. I’ll gladly chip in and help however I can
I think one of the reasons why you're getting artifacts is because of how metals reflect light : it's mostly specular reflection(it's directional, in your case parasitic) and not diffusive (unidirectional, what you're detecting). metal oxides, however, tend to build up in tiny crevices of metal surfaces and they are dielectrics, so they create tiny diffusive areas in otherwise specular metal which might look like noisy height changes.
This is a bugaboo when working on PCB's under a microscope generally. They are SHINY, and what you see is not a nice illuminated surface but a reflection of the light source. That's why we use ring lights to work on PCBs, at least that eliminates reflections from the surface plane, although not from solder fillets.
I think you are right about reflections. And to add something else. I think some reflections are courses by lights other than the laser. Like the image with the "EC". The left, white part of the image could have a lot of room light. Then the lights were turned of for a little while. And then a small light in the room is turned on what causes the reflections on top of the characters.
But I could be wrong because the scans took multiple days and then I would expect to see more day night effects.
@@Daniel-zw6gz What type of temperature fluctuation is seen in this workspace over 24 hours? The PCB has many components, each with a different CTE. Also conformal coatings can confuse the sense of sharp focus.
Egads! Sorry, this is not where I should have put the question!
I was thinking about reflectivity, too, after a recent experience trying to use photographs to create 3D models using Meshroom. Reflections make it basically impossible for the algorithms to figure out what's going on in the images. Anything reflective has to be dusted with a matte powder to get it to work.
Dude your video quality and experimental setups keep on getting better! Awesome stuff, keep it up!
Thanks! Really appreciate it :)
Jesus christ! this is a 2Msub channel level production quality! how you manage!
I mean, i loved all your videos but you really outdid yourself with this one!
This is really well done! I watched an OpenFlexure video and this was suggested. Really shows that a lot of time went into the project and into the presentation. Wishing you the best of success.
Thanks Kent! Good to see you, hope things are well!
There should be a tech/science youtuber collab, you'd fit in nicely!
Sounds like something Hank Green (SciShow, Crash Course, Journey to the Microcosmos, etc etc etc) could be interested in... but no idea how to get his attention.
It's just a matter of time until this channel explodes. Keep on!
Cool project. Try to modulate ur laser beam with chopper or so and demodulate ur photodiode signal at the same frequency. This will remove much of noise.
Good point! I was about to suggest some physical wavelength filtering as well, just to remove signals from ambient light, but some decent modulation will remove a lot of the ambient (DC in any case)
This channel just keeps getting better!
Hey, awesome video! I see there already was some comments about reflective surfaces here - and that is indeed the probable cause for the inverted shield structure. When the beam hits the slightly higher point of the shields edge it is scattered into the room and not back into the lense making the signal weaker. This would explain the smaller values in some of the curves peaks too. Alexander Sannikov mentioned in their comment that the metal oxides being diffuseve cause noise in the signal. Indeed if the samples were to be prepared beforehand (like they do for electron microscopes) to be compleately matt with a coating of a white matt substance the laser would behave more uniformly along the entire surface.
Fascinating. Great job and good explanation of the optical path. I wonder if it would be faster to run the scan like an atomic force microscope, riding the surface by tracking the amplitude of the photodiode. Move a step in x, then move the z up or down to maximize the amplitude, repeat. Advance y each time you hit the end of x. The assumption is the surface is often flat, or at least has a gentle slope, so instead of scanning the whole z stack for an x,y point, just search around the last height (z) each time you move to a neighboring point.
I would think you could then attempt to recognise the trend from previous points in the direction you're currently moving and predict which way you need to scan in z.
This man is absolutely incredible. I wish I was able to retain and use learned information as well as he does. He just is a wealth of knowledge.
You retain information by using it. Memorization is a myth and not real learning.
Great video, 20 minutes fly by, just the right level of detail. From my experience stacking microscope shots and focus failures, it will be that 'specular highlights' (very shiny surfaces) break your algorithm. You'll be assuming that the thing you are scanning has unchanging topology (which is true), but accidentally assuming that when you move the microscope stage that the pattern of light reflected would not change (untrue), because unless the surface is perfectly matte, the 'brightness' does not 1:1 correlate with 'in focus', it also correlates with 'shiny thing pointing at lucky angle to send light back to sender'. Imagine you were scanning a microscopic disco ball, if you strapped a torch to just above a camera lens and moved it closer and further, you'd see that different panels would suddenly become very bright, and others very dark. It confuses normal cameras with their contrast based focus detection (and even sometimes phase detection). If you want to test what I'm talking about, scan some glitter or a retroreflective tape.
I would spray the item to be scanned with a misting of 'airplane glue' smelling hairspray in the giant cans from 50cm away. It will 'matte' your surfaces sufficiently. If it kills the retroreflection of tape, it will be scannable by your algorithm.
That project deserves a subscription right out of the box ! Thanks for producing such an interesting content.
Very cool, i worked on a similar project a few years back using DVD optical pickup heads and its internal optical path, which is indeed a confocal arrangement with an electromagnetic vertical adjustment which provides a high degree of precision.. my failure was in the X-Y stage.. might have to revisit with openflexure but it was something that worked as an arduino shield.
You've come such a long way in your video production. Hilarious intro and great quality, plus intriguing topics!
I was about to comment on your complaints about the slowness of the system given the insane amount of points that you have on the z-scans (eg. 13:10), but then I heard the explanation at 21:01...
Anyway, it seems that you have quite a lot of resolution, a quick and easy way of trading it for a bit more speed or range would be adjusting the gearing of the steppers.
Also those 250 ms of "network response time" are weird and likely worth profiling.
YOU GIVE ME HOPE.
I'm in awe of this video. My intellect is only barely sufficient to grasp what the components do.
It's true that I feel bitter in myself for not being smart enough to do something like this. But I am profoundly happy that you are. It gives me hope for humanity's future that there are people as brilliant as you in the world. It really does. I wish you all the very best
In a previous life, I learned a lot about CT scanners. The basics are simple: take a big hubless wheel. Put an X-ray source on the wheel, pointed at the centre of rotation. Put a detector on the other side of the wheel. spin the wheel, gather signal strength data, do *LOTS* of computation, get a detailed image, called a slice. Throw in a motorized bed to move the patient in and out of the machine, and you can scan a volume by gathering *many* slices.
OK, so at some point, Toshiba had a bright idea: they used an X-ray source that produced a line of light, not a point, and opposite it, they put a line of sensors. That way, they got multiple slices in a single pass. By about a decade ago, they had such density that they could image an entire human heart in a single rotation, and since they were doing about 3RPM IIRC, that meant that they could make CT scan videos of a heart as it pumped!
Anyhow, it seems to me that you could do the same: miniaturise the laser+sensor system and put more in side by side. I haven't looked into the optics of it, but if you could use a bar diode and a linear ccd, you might be able build a system that can scan the entire Y-axis at once. A 64hour scan would come down to 8 hours.
I also wonder if there are better scanning patterns. eg: diagonals. Start at [MaxX, Miny,MaxZ] and command a move to [MinX,MinY,minZ], then move to [MinX,MinY,MinZ+1], then scan to [MaxX-1,MinY,MaxZ], etc.
I dont know if it was already mentioned in the comments, but. I have some experience in 3d scanning, basically extracting height information from a series of images, and the general recommendation for an object not to bee too white, or too black in albedo and what is sometimes more important not to be too shiny. From your samples you can clearely see that varnished surface of a PCB net you the worst result, and edges of the shiny metallic coin tricked sensor into wrong depth, even though the xy shape was correct. Try taking pictures of embossed areas of paper money and see if it will make a real difference. Amazing video though!
Dude the cinematography in this is NUTS
12:15 you can possibly increase the scanning speed dramatically by making a feedback system, like autofocus, which tracks the surface or, equivalently, the height at which the intensity is maximum, which is the maximum of the curve so when x-y changes, you need only change the z enough to stay at the maximum
Great idea. And if you can implement it with phase detection similar to SLR cameras, you don't need to even guess which way you need to move to get into focus. The end result would be pretty much like optical LP needle.
That said, laser scanner LP players do exist. Maybe one could take some ideas about those. I think the tech is old enough that the patents have already expired but are publicly available.
I really can't stand how at such exhuming scientific and video quality you only have 25k subscribers. Luckily you have already managed to capture the attention of major scientific channels.
funny video ! It reminded me one of my internship : I had to build an interferometer to mesure optical elements with 10 nm accuracy for the Z dimension. But instead of laser I used a white light source.
Thanks for the video, it brought back some great memories :)
In the near future, these microscopes will be shipped into kids boxes like your toy microscopes in their fancy plastic briefcases.
I have a personal ambition to build an STM. I will consider myself successful when I can see atomic dislocations in a metallic crystal. So far I have designs but no working prototypes. Just an occasional image before a tip crash. I have no real need for one of these but my purpose of designing one is one of learning. At the same time, I end up creating supporting hardware that has other uses. Videos like this one are inspirational. I work in the semiconductor industry and need to worry about particle contamination at the sub-micron level. The difference between what I do at home and at work is about 4 orders of magnitude in cost.
Nice project. Very good details on what this microscope is and how it works, very good.
About the optimization "issue" you commented, I see it as a simple thing to run an arduino (or any other microcontroller) driving the steppers directly and sampling the data. I am not sure how much ADC resolution is required to get nice data, but many uC have 10~12bit ADCs and sample at > 20kS/s, which is way over the stepper frequency. A simple loop will take stepper to "home" and do step; take sample; print sample; next. repeat. You do not need to "verify" the position, just let the actuator (stepper) do it job. There is for sure a time need for the the actuator to stop moving (settle position), but that can be measured by looking at the samples, and dynamically determine how long to wait for. Also running the scan at constant speed, will help to vibration and "inertia problems", ie accelerations.
I am not familiar at all with the OpenFlexure device, so maybe I do not see how it would not work like I described.
Congrats on making a convocal! I build microscopes for biological research in my phd for living. Some tips: it seems there are is lot of aberration in the spot at the pinhole, resulting in bad contrast. Use a mirror (in focus) and try tilting and moving the second lens to get a bettet spot. You probably want a kinematic mount for that lens to make the alignment easier.
I feel like a faster way to scan this would be to use a PID to try and keep focus by affecting the Z height of the stage. You could start by moving to (0,0) and finding the optimal z height with the highest signal intensity, and then begin a slow raster scan of the subject. Provided the x-y movement was slow enough that the PID had ample time to try and keep the focus optimal, you could just directly save your Z height as the output, essentially eliminating a whole dimension from your problem
first thing that comes to mind is reflectivity would greatly affect the results and create ALLOT of noise.. because after all, your relying on the reflection of the laser to plot the data.. so if the surface of what your measuring has a higher reflection than that of other points, then it will throw it off, because your measuring the reflection at a specific Z point, as opposed to the actual surface height. this is why electron microscopes work so well, because an electron cares less about reflectivity of light upon the surface that its measuring.
Two thoughts: 1. How about using a nozzle as a pinhole. 0.1mm nozzles are pretty cheap, and you could always drill out the back if you need more room for the beam to expand. They come with a convenient screw thread for mounting and fine adjustment along the beam path! 2. RobRenz has a video on an autocollimator mirror system with some interesting differential screws for the precise positioning of mirrors. Been trying to figure out if it is possible to make a 3D printed laser autocollimator...
Wow only just saw this video. For faster scanning you could use an x-y galvo stage and f-theta lens like used in a laser marker or SLM 3D printer, that way you can raster scan an area with a fixed subject vs. having to move the subject around. You could effectively steer your beam around as fast as you could read the photodiode and your reading electronics would become the main bottleneck.
Your channel is *insanely* underrated. This is sooo cool.
I think the biggest issue is the range of reflectivity. I had this issue with 3d scanning using photogrammetry. Your photos will be greatly affected anywhere there is solder on that PCB, or other curved highly reflective areas. It might be worth trying to air brush a flat color on everything to make the reflective properties uniform, which should normalize your readings and greatly increase the resolution and accuracy of your photos. If air brushing adds too much material to the sample, there might be some gaseous coating options like 2D boron nitride that would only add 1 atom to the height. Good work though. Your channel will do great :)
The correct term is "debossed".
deboss: the design is recessed into the surface.
emboss: the design is raised from the surface.
Holy cow. Hidden gem of a channel. Proud to say I was here before he hits 10 million subs!
I've worked with confocals, and the easiest way to get the pin hole and X/Y/Z stage is to use an optical fibre for the pinhole then move the fibre tip. Also we used photomultiplier tubes which give you much better sensitivity.
Ah, that's clever! I assume you have to use a single mode fiber, or would a larger multi-mode work? That would certainly cut down on the complexity of the optics and alignment!
It'd be cool to do this with multiple wavelengths. IR vs. visible topologies for example. Another thing worth looking at might be to use a mild diffuser and concentric photodiodes instead of a single diode or a camera. Because those contours are slopey and fairly linear, you could get instantaneous readings (if I understand the principle at work here lol) and if you set up the diode array intelligently, you possibly reduce the problem with finding the wrong maximum.
Another option is reducing the number of measurements with active scanning by processing the samples in real time; if you combine this with the multi-diode thing, maybe the differences between the diodes can get you closer faster.
Of course, I get that there's a lot more mechanical complexity involved when you have four or more pinholes, and need chambers for those holes.
It's amazing to think in 14 years we went from first advanced computer in your pocket to garage microscopy and nanolithaography.
Man! your videos are awesome. We get months worth of education in science and technology in every one of them. Your detailed way of explaining is incredible. Thanks
This is really impressive! It's great to see people really pushing the limits of DIY science.
coming from the field of microcopy im impressed at what you were able to make on the basis of these rather simple devices! keep going!
did you actualy try imaging a clean, flat surface? It seems arbitrary but you might get information about your noise (and if you try different materials, there might even be different noise values due to different reflectances of these materials).
Yunno, I didnt... but that's a great idea. Could probably use that to help calibrate/clean up the images too.
This is great for a "Google Home" - like device that is ready for the Quantum era of technological application.
To be able to program your "Quantum Google Home according to your @Feyman 3D-Printing Recycling unit would be a very cool translational unit for Artistic expression in the home that has never ending customizability"
i.e "Input = Photon Chemical Identification , Output= Laser Projector Configuration "
Absolutely love the jank to performance ratio of this project!
You are living my dream, my friend. I’ve always had this fantasy living in my head of making my own custom microchips.
As mentioned in the end of the video, scan in z axis. If remove the pin hole and change the photodiode to a camera. A single trip scan result 100 bitmap image, let’s say 100steps in a scan trip. By calculate the coefficient of the 100 image, the height image can calculated. That change 10000s single data captures to cpu extensive calculation.
In the setup above, a modern cpu with GB memory is needed, and computing time may longer than scan time.
Not sure if anyone mentioned this before but you can make the raspberry pi camera into a photodiode by integrating all pixels. This kind of avoids all the extra electronics for amplification. You can also increase signal by binning pixels which will just increase signal strength at the cost of reduced resolution.
Yep, that's the approach I took at first. It was still too slow :( To overcome shot noise on the sensor you have to leave the pinhole light fairly concentrated on the sensor, so there is only a small region to bin together and find average intensity. Still needs 1-3s of exposure at max gain and ISO.
Since you already have the raspberry pi camera, you could probably reduce the issue with the local maxima:
Take the expanded beam, and take only the top half of the spot (so before the beam splitter). Instead of the pinhole and the photodiode, just use the raspberry pi camera. Out of focus away from the lens will look like a half circle on the top half of the sensor, out of focus near the lens will look like a half circle on the top half of the sensor. With the right amount of calibration, you could even do away with the entire Z-positioning, and just use the spot size as a distance indicator. This has the benefit of being fully independent of the reflectivity of the target (as long as enough gets reflected for the camera, but you could also change the photodiode brightness to compensate.
I am working regularly with a confocal laser scanning microscope since the beginning of this year and I had no clue this technology existed before and now YT randomly suggests me your video! what coincidence.
And yes you are right about them costing a fortune. they are obscenely expensive... on the other hand our particular microscope takes seconds for a scan, so maybe the price tag is reasonable...
While I am only marginally interested in the topic of metrology and microscopes, I will now be looking into the openflexure project, because I am very much intrigued by micromanipulation.
20:57 Hilarious to me, that I know nothing about microscopy or any kind of precision metrology, but when you were talking about the images you took, and the issues they had, I came up with this exact same solution.
OMG the kafka-esque deconvolution line XD funniest shit I've heard all week.
So, when laser scanning Total Internal Reflection (TIR) optics they have to be painted black in order for the laser to see them properly. Some of your data looks like the shinyness of the items affected your data by messing with exposure rates or something. You might want to try a matte black or something to improve clarity. Though it might affect details, it might be a worthwhile experiment since you're looking for relative depth and not a "color picture"
Very cool microscope by the way, I don't even know why yet, but I want one!
Makes sense! Might give that a shot, would be an easy test to see how much the specular reflections affect things. And I hear ya about wanting instruments even without a need... that accurately sums up most of my shop experience :)
I audibly shouted "SIIIIICK" when you showed the topological renderings. Instant subscription.
Love the project! Miss doing this stuff. Man if it takes a week to image a surface you wonder how much temperature change over that time frame would impact the sample. Its a small area you're sampling but the whole object would chmage temp. For efficiency buildings can drop temp overnight. You're measuring such small features I wonder if it might be enough to cause some of those anomalies in the data plots. It's always a challenge coming up with the right algorithms to filter the way you want but not kill good data or pick the wrong point.
Really cool work. Couple of questions/comments.
Did you calibrate your sensor data with a dark measurement?
What's the spot size achieved?
We have built a confocal microscope at our lab. we do line scans to get faster results (easy if you know the acceleration and speed of the stage). Limiting factor is of course how fast you sample the data. We limit 'stop and measure' at each point to situations where we need more precise measurement or small area scans. We also have a dynamic profile measurement mode where we can measure periodically moving samples.
Also I see that you have a structuring laser. You could make custom precise pinholes using black hard paper.
You could over complicate your optics to get a smaller spot size. Or use white light as your source which result in different focal point for each wavelength and use a spectrometer to get the profile.
Punching my ticket, this channel is gonna be in the hundreds of thousands some day. Damn good explanation and camera work, + the start skit. Maybe even pushing a million like TOT...
Wow!
Really interesting project. Hope to see more of this soon. Thank you for sharing!
I’d say that when it scans a rounded surface, some of the light is reflected away at an angle which would show as a lower intensity in the data. That would explain why the top of the rounded surface kind of showed but the sides didn’t.
Love the project, great video, glad it showed up in my feed somehow!
2 minutes in, this is what it's all about on so many levels. So impressed already!
I used to use a KLA Tencor Confocal Review Station when I worked for a certain large semiconductor company. Normally we just used the optical microscope. I thought it was a really neat tool but we couldn't save pictures from it to our database. The only way we could save them is to print them which was expensive and slow.
If you use a ccd sensor without the pinhole, you could scan for the smallest radius, so intensity doesn't matter. that way you can scan both reflectivity and height at the same time, instead of trying to correlate both
Really interesting; for pinhole making, you might try an acupuncture needle - sharp and round - works well, and of course various gauges are available.
I would suggest trying something call white light confocal instead of intensity only pinhole confocal, but you need a spectrometer which can also be quite easily made as theres little need for narrow FWHM, a commercial version will be chrocodile from precitec, it gives depth info and you only need to scan in XY, plus with different chromatic abberation lens, you can get different scan range, something like 100um to few mm dynamic range
Hehe, on the todo list actually! I discovered the Nanovea version of the same thing (I think they call it "chromatic confocal" instead, just to be confusing i guess) while working on this project, but was too invested in the regular confocal to quit at that point. I agree, it looks great! Should be so much faster, and it's such a clever mechanism using/abusing chromatic aberration. I think the optics should be pretty straightforward, right? Just a bunch of regular lenses stacked up to increase the chromatic aberration at each lens?
@@BreakingTaps yes, the optics are very straight forward, calibration is a bit tricky, but doable in a home shop i think, we bodged together a chrocodile taped to a benchtop cnc router and did some mapping that way, it worked alright as at high speed weight will average out imperfections, didnt even need filtering. one thing chrocodile can do is 4KHz readout, with a tiny DSP, i will think with a FPGA demo board it will be equally good.
I think that the problems you had with the pcbs were due to the fact that the green surface absorb most of the power from your blue laser; that, and the lake of flux that seems to cover almost everything would blur out almost all the details. You also had problems with the screw because the sharp edges and tilted faces shine the light away from the collection path. I think your system would work best for measuring stacked flat surfaces, something like mems or microchips.
i have learned more about building optical set-ups from this video than in all of my 3 years in an electro-optics phd program
You should go for backup holders for the mirrors with play to the mirrors but holding them in case of the magnets not Holding any more. For example when carrying around your machine or a heavy magnetic pulse hitting your building.
Have you tried pointing the laser at the photo receptor for the expected duration of a scan? That’ll quantify the noise that’s outside of the microscope stage.
The top of the shield is convex, so it spreads the laser beam out, which might translate into lower intensity after pinhole. That's my theory.
Very cool! My knowledge of optical scanning goes no further than end user where I push button and machine goes brrt, so it's great to see the community come together with advice.
I'm very excited to see an upgraded model!
Many, many congratulations on an incredible, private lab achievement.
Thank you for the video and sharing your journey. Inquisitive as I am about so much, the lessons you share of in these challenges including the good the bad and the ugly enlighten and inspire. Your success is a journey filled with experiences won and lost, always best shared. Thanks again well done.
Speed: I'd modify your algorithm to not do the whole z stack. You can just do a few measurements above and below any known adjacent maximum. It's kind of like doing delta method/Taylor approximation with verification--if you cannot verify a maximum then you expand the z range. Sure, there might be fake maxima, but that should correct on the next pixel over. Also, consider the reflectivity of your scanned surface: I'm not surprised that you had problems scanning the metallic surfaces.
Maybe you could use a camera and a sheet of LEDs with different colors to create a normal map? The normal map could then be used to define the expected signal, and reduce the number of samples needed to take a picture.
My guess is that for some images the normal of the surface matters very much (e.g. the screw). Thus you don't get the light back as it's reflected in a different direction
Yeah I think you're probably right, between being highly reflective and also at a very bad angle, probably very little light probably made it to the sensor.
@ 17:39, your issue is with the zener diodes encased in glass, the solder joint is fairly clear but the round glass is scattering the light and there goes the resolution
Laser scanning confocal microscopes are known to miscalculate shiny metal surfaces, when they reflect (multiple times) back into the diode directly. maybe thats why your results were hard to evaluate. Try with coarse/dull surfaces -> should be getting better results than coins/smd 's. :)
I have suspicions as to why the top of the shield did not scan properly. I have had huge issues with this scanning shiny objects. Because of the Fresnel effect, the more oblique you get to a surface or edge, you can get a double reading, because all objects become a mirror at the right angle. (The mirage effect). So I often try to scan expensive cars that the owners will not let you use dulling spray on, so i get very similar artifacts. When I scan tiny objects, I have tried dulling spray, but the paint dots are too large, so I started to make my own using pigment that I grind very finely, and spray with a pasche AB turbo airbrush. The spray is so fine, you are able to keep the fidelity of a tiny object and get the dulling you need to cancel the Fresnel effect. I hope that actually helps you. You are a very smart guy and I feel pompous telling you what to do differently. Good luck.. This was an incredibly interesting video to me.
0:18 , no but I have woken up in a study daze and tried to compute the integral of the alarm clock numbers. Then I went back to sleep. hahaha
Super cool project may I suggest that you replace the photo diode with a photo multiplier tube yes these require a high Voltage
supply of good quality but the resulting gain factor, low noise and bandwidth with a output in the volt range so little or no additional
amplification needed, hard to beat with any other tech. PM tubes are abundant and low cost on ebay HV usable power supplies cost
more but can often be found there too.
I'm floored. That was awesome. I finally have a project to pursue that will force me to crack the seals of signal processing, python, and optics. Thanks so much!
Goodluck! Happy to answer questions if you have any!
I would try to use a photomultiplier detector with the smaller aperture, this should result in both speed and precision
This is amazing! It's fantastic that you can achieve this level of detail using 3d printed components. The explanation was very useful and covers a topic which isn't well covered elsewhere. Thank you.
I know you aren't, but it sounds like you're making up words as you go. What an amazing project!
You may also consider a diode sensor array for faster imaging. I would imagine more optical points of interest pulsed at the correct intervals to avoid interference would significantly decrease the scanning time.
This would not work in the way you suggested. The array would sample your Airy spot and not the sample space. Instead this idea is used today to generate super resolution.
your light setup was so good :)
You could also get a really quick profile using a camera in reflected brightfield. Image a focus series in Z, and then do an Extended Depth of Focus in FIJI (ImageJ). This will return a height map identical to the one you are getting with the confocal. Spiral scanning and photomultiplier tubes will also greatly increase your speed. Finally, a Teensy running in DMA mode can do dual channel ADC at 1.1 MHz per channel which can then be dumped from RAM to an SD card for rapid storage and later processing.
This man is quite literally a nanotechnology engineer
Great project, going to try to repeat it myself) One question, why scanning the whole Z axis? When you have height-to-intencity curve (which is looking somewhat parabolish), just a single measurement makes you to choose a height from one of the two points on that curve. So you could make another measurement in any other point to distinguish them. Moreover this method should work even when your scanning range does not cover all the heights of a specimen, you can calculate the height out of the range.
at college they talked about improving scanning electron microscopes by using a DLP mirror array to be able to scan so much faster. May be worth investigating if you're interested, but I have no idea the technical challenges that this would require to overcome.
Very cool project indeed and well done. It also makes me appreciate what a commercial confocal scanner can do. The one we have at work scans @ 4kHz and gives a distance value every sample (without requiring z-motion) and the optical device is way smaller. But yeah, it does cost a buck or two more ;-)
Agreed! Commercial machines really are remarkable pieces of equipment. And they have all the optics for filtering too, which mine totally ignored! :)
Raster along a radius of a spinning wafer and you have a good way to detect particle defects when processing semiconductors I think