Bilateral prediction and intersection calculation autofocus method for automated microscopy

Autofocusing of laser lithography through the crosshair projection method

In laser direct writing lithography, there is not any image information from the sample surface, which makes it difficult to find the position of the focal plane. To overcome the problem, an autofocusing through the crosshair projection method is proposed in this work. The crosshair on the reticle is inserted into the lighting path and imaged onto the sample surface. The addition of the crosshair projection increases the image information from the sample surface, meeting the requirement for the image information in focusing and improving the focusing environment. Furthermore, this work presents what we believe to be a new division of the focusing curve based on the range of the perpendicular feature extracted from the crosshair projection during the focusing process. The perpendicular feature can be extracted from the crosshair projection in the focusing zone but not in the flat zone. Compared with the traditional division, this new division enables the use of the perpendicular feature to directly determine the zone of the current sample position and to find the focusing zone during the focusing process. This can completely filter out the interference of local fluctuations in the flat zone, greatly facilitating the sample focusing. The autofocusing process was designed based on this division, and experiments were carried out accordingly. The focusing accuracy is about 0.15 µm, which is in the range of the depth of focus of the optical system. The results show that the proposed method provides a good solution to achieve accurate focusing based on the crosshair projection image from the sample surface in laser lithography.

High-precision microscopic autofocus with a single natural image

In industrial microscopic detection, learning-based autofocus methods have empowered operators to acquire high-quality images quickly. However, there are two parts of errors in Learning-based methods: the fitting error of the network model and the making error of the prior dataset, which limits the potential for further improvements in focusing accuracy. In this paper, a high-precision autofocus pipeline was introduced, which predicts the defocus distance from a single natural image. A new method for making datasets was proposed, which overcomes the limitations of the sharpness metric itself and improves the overall accuracy of the dataset. Furthermore, a lightweight regression network was built, namely Natural-image Defocus Prediction Model (NDPM), to improve the focusing accuracy. A realistic dataset of sufficient size was made to train all models. The experiment shows NDPM has better focusing performance compared with other models, with a mean focusing error of 0.422µm.

Autofocus methods based on laser illumination

Autofocusing system plays an important role in microscopic measurement. However, natural-image-based autofocus methods encounter difficulties in improving focusing accuracy and robustness due to the diversity of detection objects. In this paper, a high-precision autofocus method with laser illumination was proposed, termed laser split-image autofocus (LSA), which actively endows the detection scene with image features. The common non-learning-based and learning-based methods for LSA were quantitatively analyzed and evaluated. Furthermore, a lightweight comparative framework model for LSA, termed split-image comparison model (SCM), was proposed to further improve the focusing accuracy and robustness, and a realistic split-image dataset of sufficient size was made to train all models. The experiment showed LSA has better focusing performance than natural-image-based method. In addition, SCM has a great improvement in accuracy and robustness compared with previous learning and non-learning methods, with a mean focusing error of 0.317µm in complex scenes. Therefore, SCM is more suitable for industrial measurement.

An Auto-Focus Method of Microscope for the Surface Structure of Transparent Materials under Transmission Illumination

This paper is concerned with auto-focus of microscopes for the surface structure of transparent materials under transmission illumination, where two distinct focus states appear in the focusing process and the focus position is located between the two states with the local minimum of sharpness. Please note that most existing results are derived for one focus state with the global maximum value of sharpness, they cannot provide a feasible solution to this particular problem. In this paper, an auto-focus method is developed for such a specific situation with two focus states. Firstly, a focus state recognition model, which is essentially an image classification model based on a deep convolution neural network, is established to identify the focus states of the microscopy system. Then, an endpoint search algorithm which is an evolutionary algorithm based on differential evolution is designed to obtain the positions of the two endpoints of the region where the real focus position is located, by updating the parameters according to the focus states. At last, a region search algorithm is devised to locate the focus position. The experimental results show that our method can achieve auto-focus rapidly and accurately for such a specific situation with two focus states.

Improved autofocus method for human red blood cell images

This paper presents an improved autofocus method for human red blood cell images in a microscope. The products of the sum modulus difference and the real-valued fast Fourier transform function are multiplied to obtain an improved sharpness evaluation using the properties of a Gaussian function. It is superior to traditional evaluations in terms of unimodality, steepness, and sensitivity. A new quantitative criterion is proposed to represent the ability of sharpness evaluation against noise. An adaptive focus window with great robustness is proposed that can reduce the computation cost and adverse effects of the background. The better performances of the proposed algorithms are all proved by experiment results, and they can help to find the quasi-focus position more quickly and accurately.

Autofocus on moving object in scanning electron microscope

The sharpness of the images coming from a Scanning Electron Microscope (SEM) is a very important property for many computer vision applications at micro- and nanoscale. It represents how much object details are distinctive in the images: the object may be perceived sharp or blurred. Image sharpness highly depends on the value of focal distance, or working distance in the case of the SEM. Autofocus is the technique allowing to automatically adjust the working distance to maximize the sharpness. Most of the existing algorithms allows working only with a static object which is enough for the tasks of visualization, manual microanalysis or microcharacterization. These applications work with a low frame rate, less than 1 Hz, that guarantees a low level of noise. However, static autofocus can not be used for samples performing continuous 3D motion, which is the case of robotic applications where it is required to carry out a continuous 3D position measurement, e.g., nano-assembly or nanomanipulation. Moreover, in addition to constantly keeping object in focus while it is moving, it is required to perform the operation at high frame rate. The approach offering both these possibilities is presented in this paper and is referred as dynamic autofocus. The presented solution is based on stochastic optimization techniques. It allows tracking the maximum of the sharpness of the images without sweep and without training. It works under uncertainty conditions: presence of noise in images, unknown maximal sharpness and unknown 3D motion of the specimen. The experiments, that were performed with noisy images at high frame rate (5 Hz), were conducted on a Carl Zeiss Auriga 60 FE-SEM. They prove the robustness of the algorithm with respect to the variation of optimization parameters, object speed and magnification. Moreover, it is invariant to the object structure and its variation in time.

Combining gradient ascent search and support vector machines for effective autofocus of a field emission-scanning electron microscope

Autofocus is an important issue in electron microscopy, particularly at high magnification. It consists in searching for sharp image of a specimen, that is corresponding to the peak of focus. The paper presents a machine learning solution to this issue. From seven focus measures, support vector machines fitting is used to compute the peak with an initial guess obtained from a gradient ascent search, that is search in the direction of higher gradient of focus. The solution is implemented on a Carl Zeiss Auriga FE-SEM with a three benchmark specimen and magnification ranging from x300 to x160 000. Based on regularized nonlinear least squares optimization, the solution overtakes the literature nonregularized search and Fibonacci search methods: accuracy improvement ranges from 1.25 to 8 times, fidelity improvement ranges from 1.6 to 28 times, and speed improvement ranges from 1.5 to 4 times. Moreover, the solution is practical by requiring only an off-line easy automatic train with cross-validation of the support vector machines.

A novel autofocusing method using the angle of Hilbert space for microscopy

Autofocusing technology is indispensable for routine use of microscopes on a large scale in biological field. The autofocusing method using the angle of Hilbert space is brought forward to measure whether the image is focused or not. The angle of Hillbert space can be used to evaluate accurately the similarity degree of two images. The experiment results show that the autofocusing method can decrease the computational cost and get accuracy for real-time biological and biomedical images with noise robustness. The focus curves are smooth and possess the unimodality, the monotonicity and the symmetry. Compared with other classic and optimum focus method, the Hilbert method demonstrates its robustness to noise and can improve the focus speed. The experiments showed that the proposed method can increase the overall performance of an autofocus system and has strong applicability in various autofocusing algorithms.