巴西专利BR102012033723B1 method for restoring blurred barcode images

专利PDF首页>>巴西专利

专利附录

专利说明

权利要求

类似技术

同族专利

引用文献

法律状态

优先权

专利摘要:
METHOD FOR RESTORING IMAGES OF FOCUSED BAR CODES. The present invention relates to the field of human interaction with mobile devices, more specifically the interaction made by the automatic recognition of barcode images captured by the integrated camera in portable devices, particularly mobile phones, smartphones, personal digital assistants (PDA) , portable digital cameras, among others. More specifically, it refers to a method for scanning bar codes using portable devices with digital cameras. The present invention provides a method for restoring barcode images damaged by the blur effect due to the lack of focus of images captured with low-cost cameras with fixed focus. The proposed method treats the image of bar codes in order to enable the correct decoding of its content.
公开号:BR102012033723B1
申请号:R102012033723-1
申请日:2012-12-28
公开日:2020-12-01
发明作者:Eduardo Telmo Fonseca Santos；Eduardo Manuel De Freitas Jorge；Gustavo DE ALMEIDA NEVES；Geovane Dos Santos Anunciação；Luciano Rebouças De Oliveira
申请人:Samsung Eletrônica da Amazônia Ltda.；
IPC主号:

专利说明:

Field of the Invention
[0001] The present invention is related to the field of human interaction with portable devices, more specifically to the interaction made through the automatic recognition of barcode images captured through the camera integrated in portable devices, in particular, cell phones, smartphones, assistants personal digital devices or PDAs (Personal Digital Assistants), portable digital cameras, among others.
[0002] The present patent application refers to a method for scanning bar codes by means of portable devices equipped with digital cameras. The present invention presents a method capable of restoring barcode images impaired by the blur effect due to the lack of focus of images captured with low-cost cameras with fixed focus. The proposed method treats the image of bar codes in order to enable the correct decoding of its content. Background of the Invention
[0003] Currently, barcode technology is part of several sectors in the commercial, industrial and financial areas. We can find bar codes on the identification of products in a supermarket, on books in a library, on machines in an industry, on bank slips, on the equipment of an organization, on the badges of employees of a company, etc.
[0004] In general, the original bar codes represented information across the widths and spaces of parallel bars arranged horizontally. This type of barcode has only one dimension and is known as one-dimensional or linear barcode. As an example of linear bar codes we have EAN-13, code 128, code 39, among others.
[0005] More recently, bar codes capable of storing information vertically and horizontally have emerged, being called 2D or two-dimensional bar codes. They can store a lot of information. As an example, the QR Code, DataMatrix, PDF417, Code 49, among others, can be mentioned.
[0006] In order to read the bar code, an equipment based on optical reading is used, which is known as a scanner or bar code reader.
[0007] US patent document US20110068173A1 entitled: System And Method For Recognizing Deformed Linear Barcodes From A Stream Of Varied-Focus Video Frame, published on March 24, 2011, proposes a bar code recognition system. This system reads several frames filmed by the camera in the preview mode, at least one of these frames must have a bar code. The said system of document US20110068173A1 tries to estimate the level of blur caused by the lack of focus in the video frames. From this process, one also tries to estimate the identity of the barcode. The method taught by the document US20110068173A1 finds the region that has the bar code. Another operation performed by the document US20110068173A1 is the creation of a geometric barcode model with the deformities found in the barcode, which, in turn, is used in the restoration of the barcode. These deformities can be caused by the curvature of the packaging, the level of inclination of the camera, the ambient light, among other factors. However, the patent cited does not provide details on the creation of this geometric model. The document US20110068173A1 also estimates the level of defocusing, in which if the level of defocusing is high, the probability of success in decoding the barcode of this frame will be low, and this, in turn, is discarded. This estimate is made by calculating the differences between the filmed frames and through this, we try to obtain the defocus radius present in the bar code, where if it is less than the predefined defocus threshold, then it is carried out reading the bar code. The document US20110068173A1 does not present a method capable of restoring the blur present in the barcode image, it only checks if the image has an ideal focus; otherwise, it is discarded. With this, only the bar codes present in the clear images will be decoded.
[0008] The international patent application WO2011011051A2, entitled: Image-Based Barcode Reader, published on January 27, 2011, presents a method that performs the decoding of bar codes present in images from deformable models, where from the parameterization of these attempts is made using the maximum similarity to obtain the digits of the bar code, without binarizing the image with the bar code and using all the pixel information in gray scale. For the decoding to be carried out successfully, it is necessary that the location of the bar code is reasonably accurate, where the location tolerance is equal to twice the width of the thinnest bar, at either end. Since the sizes of labels on which the bar codes are printed do not have a pattern, this method can limit the decoding of bar codes to situations where the widths of the bars do not match the location tolerance shown. Thus, the solution presented by the international patent application WO2011011051A2 is not adaptive with regard to the decoding of bar codes with varied geometries. In order to perform barcode decoding, the method proposed by said patent application WO2011011051A2 attempts to obtain the digits by comparing the pixels of the barcode image with predetermined models, where they are scaled and translated to resemble the digits under analysis . The decoded digit will be the one with the greatest likelihood. This process can be slower than the use of filtering, since the process of obtaining the digit corresponds to the model is computationally expensive.
[0009] The patent document US20110007967A1, entitled: Method and Arrangement for Retrieving Information Comprised in a Barcode, published on January 13, 2011 presents a method that returns the information composed in a bar code. This method analyzes the first image to check if it has a bar code, checking its quality and the region where the bar code is located. If the first image has a bar code, then a second image is acquired, where the quality of this image is higher than that of the first image and the second region with the bar code overlaps at least part of the first region. Then, the barcode is decoded from the second image. Said document US20110007967A1 tries to locate the barcode in a first image with low quality. To find the barcode, a search is made for rectangular regions in the image using a sliding window. Then, the signatures associated with the barcode are extracted and the SVM (Support Vector Machine) method is used to assess whether or not that window has a barcode. If it is found, then a high quality image is obtained. This is achieved using quality improvement techniques, such as autofocus, flash, lighting techniques, among others. Said document US20110007967A1 does not present any method to restore the damaged image and does not propose an adaptive method for obtaining the information represented by the bar code.
[0010] The Brazilian patent document PI0603273A, entitled: “System and method that uses software to perform the decoding of bar codes directly on mobile communication devices equipped with digital cameras”, published on January 15, 2008, describes a method for decoding bar codes from images captured by digital cameras from mobile communication devices. This sends the result of the decoding to a server through a network connection, which in turn will provide information about the product associated with the number represented by the decoded bar code. If there is no network connection, then it is possible to store the decoding result for future reference. The decoding is carried out in real time from the analysis of the intercepted frames in the preview mode, as can also be carried out by capturing a high resolution image by pressing a button. After obtaining the image with the bar code, the image processing, recognition and decoding process begins. The Brazilian document PI0603273A proposes methods to binarize the image, correct lighting and remove noise, but it does not present details about their operation. This document also does not present a solution for restoring blurred images and does not have adaptive methods for converting the image into the information represented by the bar code.
[0011] The US document US007237721B2, entitled: Image Processing for Pattern Detection, published on July 3, 2007 describes a method that detects patterns, such as bar codes, in an image. This method locates the start and end points of the patterns in a region of an image, tries to calculate the PSF (Point Spread Function (PSF)) which is the degree to which an optical system blurs (spreads) a point PSF is the inverse of the Fourier transform of the optical transfer function in the frequency domain) from the detected start and end points, and restores the image region using the obtained PSF. The method proposed in the referred document US007237721B2 assumes that the PSF is Gaussian, which is not always correct and that it can limit the restoration of the bar code. In addition, the method performs focusing with 1D PSFs along scan lines, however the PSF to restore an image should be 2D, which limits the power of retrieving the information in the barcode. Another limitation of the proposed method is the use of a PSF with parameters determined for the image, however the parameters of the PSF should vary throughout the image to perform a reconstruction closer to the ideal.
[0012] The North American document US2006020039A1, entitled Camera-Based Barcode Recognition, published on September 14, 2006, presents a system for barcode identification using digital cameras. Said document US2006020039A1 uses methods that decrease noise and increase the contrast between the bars, but it does not describe in detail how these operations are performed. A method is provided to locate the bar code in any position and orientation, however, for this, it is necessary that the bar code is framed in the center of the image. The localization method presented by the document US2006020039A1 is inefficient because it has a high computational cost and fails to identify the barcode in degraded images. To binarize the image, the document US2006020039A1 uses non-adaptive methods, making the captured image in ideal lighting conditions so that the binarization is efficient. The document also does not propose a method for restoring blurred images.
[0013] The North American document US20050011957A1, entitled: System and Method for Decoding and Analyzing Barcodes Using a Mobile Device, published on January 20, 2005, proposes a system and method for decoding bar codes. This document presents a system and method that tries to improve the image captured by a camera integrated with the mobile device; then we try to obtain the information represented by the bar code present in this image. The information obtained is sent to a server through a wireless connection, where a query is made about the content corresponding to the decoded number. This content, in turn, is transmitted to the mobile device. US20050011957A1 presents a block diagram that describes a process used to correct and decode the barcode image, but it does not detail the procedures presented in this block diagram. The procedure described in the referred document obtains the frames filmed by the camera integrated with the mobile device and from these it makes numerous attempts to decode with different parameters, until the barcode is successfully decoded or until all parameters are used. US20050011957A1 mentions methods for removing distortions that damage the barcode image and, in turn, hinder its decoding. These methods are used for: perspective correction; shear removal; correction of problems associated with lighting; focus adjustment; among others. However, details on how these methods work are not presented, making it impossible to evaluate them. US20050011957A1 does not describe the step to obtain the cut-off threshold between white and black, which is used in the process of converting pixels to bars. This document does not mention any method for previously locating or identifying the barcode before decoding, it does not detail the process of obtaining the bars and does not present a method for restoring blurred images.
[0014] The article “Robust Recognition of 1-D Barcodes Using Camera Phones” by Steffen Wachenfeld et al., Published in 2008 in the IEEE, describes an algorithm for decoding bar codes using cell phone cameras. The algorithm described by the article cited in this paragraph uses image processing techniques, such as: Hough transforms; Wavelets to locate barcode patterns and morphological operations. However, this article does not describe any method to restore barcode images damaged by the blur effect, it only details the procedures used to locate the barcode, scan line binarization and identify the digits represented by it. In turn, the aforementioned procedures proved to be ineffective in decoding bar codes present in damaged images, as the proposed procedures are not adaptive enough, making decoding less effective.
[0015] The article “A Robust Method for Blind Deconvolution of Barcode Signals and Nonuniform Illumination Estimation”, by L. Dumas et al., 17th IEEE International Conference on Image Processing (ICIP), September 2010, Hong Kong, available through link http://www.math.uvsq.fr/~dumas/Proc8.pdf, proposes a method that restores the barcode image, removing noise and blur using blind deconvolution. The method proposed by that article tries to remove distortions caused by non-uniform lighting. This method is based on a genetic algorithm that combines discrete and continuous optimization applied to data strongly affected by the presence of noise and the blur effect. The article describes a blind deconvolution method based on a genetic algorithm whose target solution is a binary image, that is, the parameters of the PSF, lighting and regularization are also automatically estimated in order to minimize a cost function. It is worth mentioning that the uncertainty in the speed of genetics can limit its applications in portable devices.
[0016] The article “Deblurring and Restoration in Barcode Signal Processing”, by Todd Wittman, published in 2004 at the Siam Conference on Imaging Processing 2004, available through the link http://www.math.ucla.edu/~wittman/thesis / bars poster.pdf, presents a blur removal technique based on minimizing the total variation to restore barcode images. The approach adopted can also be classified as blind deconvolution, as it estimates the parameters of the PSF and the barcode together. The minimization of a cost function is performed using classic optimization methods, such as Newton and “Steeper descent”, which although more efficient than genetic algorithms, suffer from the limitations of local methods and also imply a computational cost that may limit its applicability to portable devices. Another limitation of the method is to assume that PSF is a Gaussian 1D function, which is not always true.
[0017] The article “Digital Processing of Barcode Images Using Mobile Phones” by Tiago Henrique Tudisco dos Santos and Almir Olivette Artero, published in 2010 at the VI Computer Vision Workshop, in July 2010, held in Presidente Prudente, São Paulo, proposes the use of mobile phones integrated with digital cameras for decoding EAN-13 bar codes. That article describes a method for locating and decoding the bar code. From the Sobel operator for edge detection, it is possible to locate the bar code and the direction of the edges from the gradients calculated after its execution. This article does not present any method to correct problems related to lack of focus, problems associated with lighting, presence of noise, among others. The binarization described in this article can be ineffective, as it has a fixed threshold and therefore the results of this can be impaired under different lighting conditions.
[0018] The article “Locating and Decoding EAN-13 Barcodes from Images Captured by Digital Cameras” by Douglas Chai and Florian Hock, from 2006, proposes a vision-based technique for locating and decoding bar codes. The article cited in this paragraph does not present any technique for treating image distortions, which emphasized the procedure for locating the bar code.
[0019] The aforementioned documents present methods that perform barcode decoding and restore damaged images due to the presence of noise, problems related to focal length, problems associated with ambient lighting, low contrast, distortions caused by perspective, among others. Due to these problems, it becomes complex to perform barcode decoding from images captured by low-cost cameras, because, due to the inferior quality of these cameras, the captured images easily present the mentioned problems. Most of the documents mentioned above are able to perform barcode decoding under certain circumstances. However, for the decoding to be carried out successfully, it is necessary that the image with the bar code is above an acceptable threshold. Some of the referred documents mention the need to carry out a previous treatment on the captured image for the decoding to be carried out successfully. However, these works do not detail the procedures necessary to restore the image, they only add a kind of black box indicating the need to improve the quality of the captured image. One of the solutions of the state of the art is to estimate the level of defocus, where decoding will only be performed if it is less than a pre-established threshold.
[0020] In contrast, the present invention presents technical details of a solution capable of restoring images damaged by the effect of the blur and presents a robust method for obtaining the numbers represented by the bar code present in the captured image.
[0021] The solutions materialized by the present invention make it possible to decode bar codes present in images captured by low-cost cameras and which have fixed focus. The present invention has a camera calibration procedure that creates a base of PSF’s. These reflect precisely the incidence of the blur effect at a given point, from a certain distance. The PSF's in this database are used in the deconvolution operation that removes the blur effect and restores the image with the barcode. The present invention has adaptive methods for obtaining the bars, in order to provide greater strength under various adverse conditions. The decoding of the bar code performs a validation and combination procedure between different results, which, in turn, increases the probability of success in obtaining the numbers represented by the bar code.
[0022] The present invention uses image processing and computer vision techniques to extract the sequence of numbers that is contained in a barcode image. These techniques make it possible to segment the barcode image, remove the blurring effect, make the image adaptive binarization, convert the barcode into digits and validate it.
[0023] The decoding method of the present invention was developed to be executed in a portable device, with which the image of a bar code is captured. In the case of devices that have a camera without autofocus, the acquired image needs to go through pre-processing to remove the blur due to lack of focus, compensate for the wear and tear caused by external interference and make corrections in order to improve its quality and carry out the correct barcode decoding and validation.
[0024] The present invention provides the decoding of bar codes, restoring blurred images that have bar codes and decoding them in real time by means of portable devices that have a low-cost digital camera. The system attempts to obtain the ideal focused image and perform bar code decoding in the EAN-13 standard, a standard that is commonly used for product identification.
[0025] This invention aims to provide a low cost, efficient and effective solution, which allows consumers to consult product prices wherever they are.
[0026] The present invention also allows the user to use cameras with fixed focus to perform the barcode reading. The user must fit the barcode within the limits of a previously positioned sight. Brief Description of the Figures
[0027] The objectives and advantages of the present invention will become more evident from the following detailed description of an embodiment example of the invention and attached drawings as a non-limiting example, in which:
[0028] Figure 1A shows an example of a blurred photo.
[0029] Figure 1B shows an example of a focused photo.
[0030] Figure 1C shows a generic model of the portable device on which the method will work.
[0031] Figure 2 illustrates the interface that will be presented to the user.
[0032] Figure 3 shows the blurred barcode and the result of its restoration.
[0033] Figure 4 shows a focused pixel and the result of this after the convolution with the PSF.
[0034] Figure 5 illustrates the points to be used in the extraction of PSF’s.
[0035] Figure 6 shows the result of the capture of the points by the target camera.
[0036] Figure 7 illustrates the framing of the barcode in the camera's scope.
[0037] Figure 8 is the block diagram of the PSF's extraction process.
[0038] Figure 9 shows the process of capturing the image of points by the target camera.
[0039] Figure 10 is the block diagram of the PSF’s segmentation process after the capture by the target camera.
[0040] Figure 11 illustrates the process for segmenting PSF’s.
[0041] Figure 12 is the block diagram of the barcode restoration and decoding process.
[0042] Figure 13 illustrates the convolution process in the frequency domain.
[0043] Figure 14 shows the image of a barcode after mirroring.
[0044] Figure 15 illustrates the conversion of PSF to OTF.
[0045] Figure 16 illustrates the conversion of the image that is in the space domain to the frequency domain.
[0046] Figure 17 shows the process the result of the process of restoring the blurred image.
[0047] Figure 18 shows the process of decoding the barcode after image restoration.
[0048] Figure 19 illustrates the processing carried out on the horizontal slice.
[0049] Figure 20 shows the process of converting pixels to bars.
[0050] Figure 21 shows the structure of the bar code in the EAN-13 standard.
[0051] Figure 22 describes the process of translating bars to numbers.
[0052] Figure 23 illustrates the process of obtaining the final result.
[0053] The system of the present invention is realized from the methods developed to restore barcode images impaired by the blur effect and then tries to decode them from a portable device equipped with a low-cost digital camera. With this, it is possible to develop a method capable of reading bar codes that uses images captured by a camera with fixed focus, whose quality is impaired due to a possible incorrect focal length. Problems Solved by the Invention
[0054] Despite technological advances, portable devices still have hardware with relatively limited resources to execute algorithms that perform advanced calculations. Therefore, the present invention applied processing techniques in order to decrease the computational cost, enabling a reduction in decoding time and minimizing the processor use of the portable device, in addition to saving memory, thus optimizing the use of resources increasing the efficiency of the process described in this invention. These techniques also provided an increase in the robustness of the barcode reader, since it was possible to increase the number of frames analyzed for the same time.
[0055] In portable devices with cameras without autofocus, certain images captured with the camera may not be able to clearly scan small objects, as is the case with some fonts, where, as the camera approaches, they remain blurred and unreadable. Some packages of shelf products have bar codes so small that when the camera is above a certain distance, it is not possible to distinguish their bars due to the insufficient amount of pixels, and when the camera is too close the object may be out of focus. This problem is overcome by using the focusing process described in this patent, thus obtaining an image with the focus corrected by image processing.
[0056] For cases in which the captured image of the bar code is not in proper conditions, some of its information could be lost. This type of problem can be caused by the ambient lighting conditions, which can cause light blurs, excessive shadows, low sharpness, among others. Also very common are situations in which the bar code is already damaged due to several factors, among which the following stand out: poor print quality; aged surface; damaged container or packaging; among others. In order to improve the effectiveness of the present invention, methods have been created to correct the aforementioned possible damages. These methods are based on stacking techniques, that is, the average of several adjacent lines in the barcode image is calculated, thus improving the signal / noise ratio of the image.
[0057] In order to improve the effectiveness of the method of the present invention for reading bar codes, even under unfavorable conditions, a set of methods are applied on the image. The barcode is divided into vertical slices, each of which is processed separately. To obtain the values of the bars, vertical sub-regions are defined and, in turn, the black and white threshold is calculated for each sub-region. A set of weights that adjust the threshold to the level of illumination present in the image are applied to the threshold. With this, the effects caused by the poor image quality and / or external factors are eliminated, allowing the reading of the bar code.
[0058] The product packaging has different formats. For example, images of packaging with a curved surface distort the size of the bars, thus making it difficult to decode the bar code. To solve this problem, the present invention uses efficient and effective algorithms to estimate the size of the bars, considering a threshold based on the specific characteristics of the dimensions of the bars used in EAN-13. Technical / Functional Advantages of the Invention
[0059] Decoding of unfocused bar codes: The use of methods to deconvolve the blurred image using PSF or OTF makes it possible to estimate an image closer to the ideal one for decoding the bar code. In this way, the blur effect caused by spreading the pixels of the barcode image is removed, increasing the hit rate of the method.
[0060] Versatility for adaptation in different cameras: A bank of PSF’s can be estimated for each different type of camera, thus allowing the same focusing process to be used with different cameras. This gives the method versatility, since it can be used without major changes in different portable devices.
[0061] Possibility of using devices with low-cost cameras: Image correction makes it possible to use devices with cameras without autofocus or with other optical limitations, since deconvolution using PSF or OTF aims to minimize the blur effect caused by the optical arrangement /camera. The process described in this patent acts as a digital signal processing algorithm that simulates the effects of corrective lenses applied to the captured image. Thus, a more adequate image is obtained for decoding bar codes, even with some limitations in capturing the image, thus also enabling the use of low-cost cameras.
[0062] Low computational cost: Seeking to improve the barcode reading process, programming practices were used that optimize the execution of computational instructions, such as using fixed point instead of floating point, using bit manipulation to perform some operations, use as few instructions as possible to scan or copy the pixels of an image, among others.
[0063] Efficient and effective barcode reading: Reading is efficient because it minimizes the use of computational resources, and it is effective because it succeeds in most scanning attempts, being possible to decode bar codes with different sizes, any positioning, any inclination, high perspective and in environments with different lighting.
[0064] Real-time processing: There is no need to capture a photo to perform decoding. The system performs the entire procedure in real time by analyzing the frames obtained in the preview mode. With this it was possible to increase the robustness of the algorithm, since at each moment a new frame is processed until the reading is carried out successfully. In addition, depending on the resolution supported in the preview mode by the camera of the portable device, it is possible to further increase the efficiency of the barcode reader.
[0065] Elimination of useless regions: regions outside the limits found are excluded from reading, avoiding the processing of regions that do not belong to the barcode. This has improved the efficiency of the reader, since fewer parts of the image will be analyzed, thus reducing processing time. The efficiency is increased, because elements that negatively influence the definition of the cut limits of the barcode region are eliminated, causing only the bars, which are the relevant information, to remain in the processing, and with this increasing the probability of correctness in the decoding.
[0066] Barcode identification: In order to prevent the decoder from running on images that do not have a barcode, a pre-analysis is carried out within the limits of the barcode. At this moment, the contrast level will be checked, where, for cases in which this contrast is very low, the image is immediately discarded, preventing the reader from carrying out an unsuccessful decoding attempt, and with this, avoiding the waste of time by computing something that would hardly be decoded. This increases efficiency in the sense that, after discarding the image, the process will be restarted and, therefore, a new frame that may have a better quality and greater chances of decoding will be provided.
[0067] Horizontal barcode slicing: the barcode is sliced horizontally and the reading is performed on each of these slices individually. With this, a better efficiency was obtained, since the number of samples was increased, which led to an increase in the percentage of correct answers. If a slice is unsuccessful, another slice is selected until the code is decoded or until all slices have been selected. The number of slices is adjusted according to the execution of the reading, making it possible to obtain slices of different positions and different sizes during the process, thus increasing the probability of success in decoding.
[0068] Adaptive binarization method: phase where the definition of the code bars occurs. In this step it is necessary to determine a cut-off threshold to separate the white level from the black level in the gray barcode image. For this, methods were created that calculate the threshold in a flexible and adjustable way. This threshold adjusts to the ambient lighting. This is because the slice to be decoded is divided into vertical regions, and for each of these sub-regions a threshold is obtained according to the intensity of its pixels. In addition, weights are assigned to this threshold that adapt the threshold to the ambient lighting. These weights were defined through exhaustive tests in which it was possible to extract the proper weight for each type of lighting. During the reading process, the number of vertical slices is adjusted, computing new thresholds for black and white.
[0069] Barcode reading on non-flat surfaces: The product packaging has different formats. Products can be found with packaging with rounded, oval surfaces, or even packages that are malleable plastic bags whose surface does not have a specific shape. In order to increase the robustness of the process, an adaptive method was created that allows the decoding of bar codes on surfaces with different formats. Knowing that depending on the surface the bar is on, the dimensions of the bars can increase or decrease according to the perspective caused by the packaging format. Because of this, a reference bar size is estimated to perform decoding in each region of the image.
[0070] Barcode reading with perspective: in order to increase the effectiveness of the reader, techniques have been developed that make it possible to decode bar codes in perspective. This was possible due to the horizontal slicing that selects slices of the code that are in the same perspective favoring the decoding of bar codes with high perspective.
[0071] Adaptive method for obtaining the bars: in order to increase the level of correctness of the decoding, an adaptive method was created to convert the pixels in the bars. This method attempts to compensate for problems related to the leakage of pixels in its vicinity by checking for bars that have become too thin or too wide.
[0072] Result validation: in order to show only correct values, the present invention has validation methods that check the code guard bars and the check digit. Right after defining the bars, it is verified that the guard bars are correct. After all digits have been decoded, the check digit is checked. If the check is correct, the decoded value is returned and the result is displayed.
[0073] Use of HARDWARE device with high mobility: the present invention can be performed in a small and light portable device that can be carried to different places without taking up much space. This resulted in a portable and lightweight device that performs barcode reading in real time, providing convenience to the user, as he will not need to travel to, for example, consult product prices.
[0074] In this way, a preferred embodiment of the present invention is obtained with methods capable of restoring images blurred by the blur effect and performing the decoding of bar codes present in degraded images, with execution in real time from an integrated system hardware and software.
[0075] Figures 1A and 1B illustrate images of blurred and focused photos, respectively.
[0076] The hardware necessary to materialize the system of the present invention is a portable device, where its main characteristics are illustrated in figure 1C. CPU 110 (Central Processing Unit) is responsible for executing the instructions necessary for managing the camera, in addition to making computational calculations for image processing with the barcode. Camera 111 is responsible for capturing images. The storage medium 112 serves to store information and data, including decoded codes, products consulted, among others. The hardware is also composed of information input devices 113 and audio components 114. Through some connectivity component 115 it is possible to communicate between the application and a remote database that has information about the products. The display medium 116 is used to view the frames captured by the device's camera.
[0077] The system of the present invention starts at the moment the user activates it in a portable device in which he was previously installed.
[0078] Figure 2 shows the system in execution. After the initialization, the digital camera will be executed and the preview mode will start, displaying the frames filmed on the display screen 210 of the device at a certain display rate (eg: 30 FPS). The described materialization of the system has two status bars, where the upper 211 indicates the status of the barcode reader, and the lower 212 shows the partial results of the decoding.
[0079] The main objective of the present invention is to insert the functionality of a one-dimensional barcode reader into a portable device equipped with a digital camera with fixed focus. In turn, making it possible to create a low-cost solution that is capable of performing the functions of barcode reader equipment, commonly found in commercial establishments and that are used by customers to make inquiries about the value of the product.
[0080] The solution presented by the present invention has the principle of performing the deconvolution process to remove the effect caused by the defocus, which deforms the image causing a degradation in the original image.
[0081] Figure 3 shows the result of restoring the image of a bar code. The barcode affected by the blur effect 310 is blurred, and its bars no longer represent the original information. After deconvolution, the bar code appears clearer 311 and a good part of the original information is recovered.
[0082] The restoration of the image is closely linked to the PSF. The PSF causes the pixels of a focused image to be spread over the neighboring region, generating the effect of a blurred image. This effect is illustrated by figure 4 which shows the pixel representation 410 and its spread after the convolution 411 with the PSF.
[0083] To make it possible to restore the image, it is necessary to first estimate the PSF that caused the pixels to scatter and then perform the reverse process. For this reason, a method was developed capable of extracting static PSF's to be used in deconvolution. This method is based on capturing images with a black background and white dots that represent 510 pixels. Then, this image is captured with the target camera. The result is illustrated in figure 6, whose pixels were scattered 610 due to the incorrect focal length. In addition, 611 noise was also added to the captured image. This spread 610 consists of the PSF, which must be segmented for use in the method of the present invention.
[0084] The PSF that caused the defocus can be changed according to the distance from the camera to the target object. In order to reduce the number of PSF's to be used, a 710 crosshair was created to frame the bar code, causing the camera to be approximately the same distance from the code to the filmed or photographed bar code.
[0085] Figure 7 exemplifies how the code should be framed in the scope.
[0086] To restore the image and perform decoding, first, it is necessary to perform the camera calibration to obtain the PFS’s. The steps to perform the calibration are shown in figure 8. The calibration consists of a process of capturing images of points from previously fixed distances, and, in turn, segmenting the points of the captured images.
[0087] As mentioned earlier, to extract the PSF’s, it is first necessary to create an 810 image with a black background and white dots representing the pixels. Before capturing the images, it is necessary to first stipulate the distances 811 at which the portable device will remain. This distance should reflect the positioning in which the user fits the bar code in the crosshair, as it is measured according to the width of the bar codes. As the bar codes in the EAN-13 standard do not have a standard size, then it is necessary to take pictures at different distances.
[0088] Figure 9 illustrates the capture of the points for two different barcode sizes, where the camera 910 is positioned framing an image of white dots 911 that has been resized according to the size of a given barcode. To capture this image, it is then necessary to frame it in the 912 sight, simulating the user when framing the barcode. Then, another image of white dots 914 is created, which is scaled according to the width of another barcode. In turn, the camera must be repositioned 913 to frame the image. This creates a base of PSF’s for different sizes of bar codes.
[0089] The capture of these images must be made from the target camera 812. Different camera models may have different focal lengths, making it necessary to obtain the PSF's database for the particular camera.
[0090] After capturing the images, it is necessary to segment the PFS’s. To do this, the image is binarized first, separating the image background from the 1010 white pixels. Then, a scan is performed over the binary image, looking for areas with high levels of intensity 1011. Then, the white points, which are now scattered in their vicinity, are cut and separated into individual 1012 images. This image with the PSF is added to an initial 1013 database for further analysis.
[0091] Figure 11 shows the process of extracting the PSF's, where the captured image 1110 is binary 1111 creating a mask of the regions with the likely positions of the PSF's. Then, only the values of PFS’s 1112 are copied, excluding the background of the image. Then, cut out each of the PSF’s 1113 present in the images to store them separately on an initial basis. This initial base is analyzed to select the PSF’s that best restore the 814 images. This check is done through a method that tries to decode various blurred barcodes by applying the PSF’s from the initial base to restore the image. PSF’s whose decryptions are successfully performed and whose decoded numbers are exactly the same as the numbers encoded by bar codes will be selected.
[0092] According to the present invention, the deconvolution performed to remove the blur is applied in the frequency domain. For this reason, to avoid unnecessary processing, the PSF that is in the space domain is converted into OTF, used to make convolution and deconvolution in the frequency domain. Then, the PSF’s are first normalized from a previously established range and the conversion to OTF is carried out. This process will be presented in detail below.
[0093] After calibrating the camera and obtaining the respective OTF's for image restoration, the process for reading the bar code itself is carried out.
[0094] Figure 12 illustrates the steps performed on the portable device at the moment when the barcode reading starts.
[0095] The step “captures the image with the barcode” 1210 happens in real time and is the moment when a frame filmed by the camera is selected and sent for decoding. In the system that was developed as one of the possible embodiments for the present invention, a button was created that is used to initialize the capture of the frames. At this point, the user must frame the bar code in the crosshairs as shown in figure 2. The image capture will be performed until the bar code is successfully decoded, or until the time limit is ended.
[0096] After capturing the frame, the “restore image” 1211 step begins. In this step, an OTF is selected in the database and the deconvolution is performed. To describe the image restoration process, it is necessary to describe mathematically how pixel scattering occurs and to deduce it inversely until reaching the results of the present invention.
[0097] The blur effect can be modeled mathematically as a convolution between the pixels and the PSF. This operation is presented with details below:

[0098] Equation (1) presents the mathematical model that describes how pixel scattering occurs, and the symbol “*” is used to define the convolution operation between psf (x, y) and i (x, y) at the point (x, y), where i (x, y) is the ideal image.
[0099] The result of this convolution will be the blurred image, represented by b (x, y). So, to remove the blur effect, it is necessary to perform the reverse process, where, from the blurred image, the original image is obtained.

[0100] Equation (2) shows how image restoration using convolution would be, where i (x, y) is the ideal focused image, psf-1 (x, y) represents the inverse PSF and b (x, y) represents the blurred image. Then, to remove the blur effect from the convolution operation, it is necessary to estimate the inverse PSF, and by applying it by convolution with the blurred image, it is possible to remove the scattering shown in the pixels.
[0101] It is known that the convolution between two functions in the space domain becomes the product between the transforms of these two functions in the frequency domain. Therefore, starting from this property, the deconvolution in the frequency domain is given by the division between the transforms of two functions. That is, the division between the transformed of the blurred image and the PSF will result in the transformation of an ideal focused image.
[0102] The image transform in the frequency domain is obtained through a discrete Fourier transform. In turn, the inverse Fourier transform is applied to return to the space domain.

[0103] The discrete Fourier transform is given by equation (3), where u = (0, 1, 2, ..., M-1) and v = (0.1, 2, ..., N-1 ).

[0104] Equation (4) corresponds to the inverse Fourier transform, for x = (0, 1, 2, ..., M-1) and y = (0, 1, 2, ... N-1), where u = 1 / Mx and v = 1 / Ny.
[0105] The method of the present invention uses the Fast Fourier Transform (FFT) algorithm to calculate the discrete Fourier transform, since it is possible to reduce the complexity of N2 to N log2 N operations and in turn reduce the computational cost incurred in calculating the transforms.
[0106] Equations (5) to (8) present a mathematical model used to cause the blur effect from the frequency domain.

[0107] Equation (5) calculates image transformation from the space domain (or spatial domain) to the frequency domain, using the fast Fourier transform, represented by FFT, where i (x, y) represents the ideal focused image in the space domain and I (u, v) is the respective image in the frequency domain.
[0108] Equation (6) performs the same calculations as equation (5), where psf (x, y) represents the point spreading function in the space domain and PSF (u, v) represents the same in the frequency domain .
[0109] Equation (7) adds the blur effect by multiplying the PSF (u, v) by I (u, v). Finally, the image for the space domain is transformed again using the Fourier inverse fast transform algorithm, as shown in equation (8) through FFT-1, returning, in turn, the blurred image to the domain of space b (x, y).
[0110] The process described in the previous paragraph is illustrated in figure 13. The barcode 1310 and PSF 1311 are in the space domain. Their Fourier 1312 transforms are then calculated, obtaining the images in the frequency domain. The magnitude of the spectra of barcode 1313 and PSF 1314 transforms originating from the center are shown and represent the images in the frequency domain. The transforms are multiplied (convolution operation in the frequency domain) 1315 and the inverse Fourier transform 1317 is applied to the result, obtaining in turn the blurred image in the space domain 1318.
[0111] From the understanding of this information, a process was developed that performs a set of operations, where from the blurred image one tries to obtain the ideal focused image. These operations are based on the deconvolution of the image in the frequency domain, dividing the transform of the blurred image by the respective transform of the scattering function.
[0112] Equation (9) mathematically describes the deconvolution operation described in the previous paragraph.

[0113] The methods for image processing in order to obtain the ideal focused image:

[0114] To decrease complexity and increase the speed of transform calculations, the Fast Fourier Transform algorithm used works with vectors or matrices whose dimensions are power of 2. Equation (10) is used to obtain the appropriate dimensions for the calculation of the fast Fourier transform, where col is the number of columns, lin the number of lines in the image and ceil is a function used to round up a value. Thus, the matrix used to calculate the transform will have dimensions equal to NxN, where N is equal to the largest base 2 exponent rounded up between the powers of the number of columns and the number of lines.
[0115] After obtaining the appropriate dimensions for calculating the transform, a matrix with these dimensions is created and the image pixels are copied to this matrix, which can result in excess space due to the difference between the dimensions. In general, the excess space is filled with zeros, but to minimize the edge effect, often caused in the image by the Fourier transform, the image is copied to the center of the new matrix 1410 and mirrored upwards 1411 and down 1412.
[0116] Figure 14 illustrates an input image for the fast Fourier transform.
[0117] The present invention has a base of PSF’s that are extracted for the target camera. To decrease processing, PSF’s are stored in the frequency domain in the form of an OTF. In this way, it is possible to eliminate the step of converting PSF to OTF on the portable device, thus reducing the computational cost during image restoration.
[0118] Figure 15 illustrates the process of converting PSF to OTF. PSF 1510 is copied to a new image 1511 with suitable dimensions 1512 for calculating the transform. Then, circular displacement 1513 is carried out with a displacement value equal to half the dimension of the PSF multiplied by -1, repositioning the pixels of the new image as shown in 1514. Next, the Fast Fourier transform 1515 algorithm is executed, transforming the matrix of the space domain to the frequency domain, where the result will be the OTF, whose spectrum is presented by 1516. This OTF is stored and shipped in the portable device to be used in the restoration of the image during the reading of the bars.
[0119] The process performed to calculate the image transform in the frequency domain is shown in figure 16. The image of the barcode 1610 is copied to the center of an image with the appropriate dimensions to calculate the transform 1611, then it is carried out mirroring 1612 thereof. Then, the transform 1613 of the new image is calculated, whose spectrum result is presented by 1614.

[0120] In equations 11 to 13, the symbol x represents conventional arithmetic multiplication. After obtaining the PSF transform and the blurred image transform, the deconvolution operation is then performed, which can be mathematically modeled as equation (11), where this is given as the division between OTF (u, v) and B (u, v). Starting from the properties of complex numbers, the numerator and denominator are multiplied by the OTF conjugate (u, v), as shown in equation (12). Equation (13) changes the denominator to | OTF (u, v) | 2, since a complex number multiplied by its conjugate is equal to its squared norm. In equation (11), the problem of division by zero or very small numbers can occur. To eliminate this problem, it is necessary to determine a number that after dividing the transforms does not result in inconsistent values. The denominator of equation (13) is a real number, which facilitates comparisons to obtain this number.

[0121] Equation (14) calculates the limit number for the denominator, which is represented by. This value will be the highest value of the numerator multiplied by the root of, where the value of is a small number that tends to zero.

[0122] The final mathematical representation for the deconvolution operation used by the present invention to remove the blur effect is given by equation (16), where the denominator D (u, v) is obtained from equation (15). This equation will return, if | OTF (u, v) | 2 is less than, otherwise it will return | OTF (u, v) | 2. Through this process, division by zero or very small numbers is avoided. The result of this equation, represented by I (u, v) is the transformed image restored.
[0123] After deconvolution in the frequency domain, the ideal focused image of the frequency domain is converted back to the space domain, using the Fast Fourier Inverse Transform algorithm, thus obtaining the ideal image focused on the domain of space.
[0124] Figure 17 shows the result of the image restoration. The 1710 barcode image is prepared for the 1710 deconvolution operation in the frequency domain. This operation results in an image transform without the blur effect. In 1712, the spectrum from the center of the transform resulting from the deconvolution operation is shown. Using the algorithm of the fast inverse Fourier transform 1713, the resulting transform is converted to the space domain, transforming it into the focused image 1714. Then, only the center region 1715 with the restored bar code 1716 is cut out This in turn is used as an input parameter for the barcode decoding algorithm.
[0125] After restoring the image, the step “Decodes barcode” 1212 starts, which receives the restored barcode image and attempts to decode it. This process is divided into sub-steps that perform image processing in order to return the numbers represented by the bar code present in the image. This process is detailed through the flowchart shown in figure 18 and is described below.
[0126] The barcode 1810 present in the restored image is sliced horizontally 1811 and each of these slices is processed individually. The number of horizontal slices is one of the inputs of the barcode reader, and this is adaptive, that is, depending on the execution of the barcode reading, the number of horizontal slices can be changed seeking to increase the number of decoded digits . Each slice will return a vector with the decoded digits, the size of this vector being equal to the number of barcode digits in the EAN-13 standard. Horizontal slicing is illustrated in 1812.

[0127] Equation (17) calculates the height of the horizontal slice, where floor is a function that rounds down. The slice size is represented by Hs, which is a percentage, defined by, of H, which represents the height of the barcode image. The number of horizontal slices is calculated using equation (18) represented by Ns. The displacement between the lines of the barcode image, represented by s, is calculated from equation (19).
[0128] The barcode in the EAN-13 standard has only one dimension, that is, the information is arranged in a line, therefore, it is necessary to convert the horizontal slice, which is a two-dimensional matrix, into a vector. This process is done through vertical stacking 1813 that transforms a two-dimensional matrix into a vector 1814, with the values of the vector corresponding to the weighted average of the columns of the first matrix.

[0129] Vertical stacking is calculated as shown by equation (20), converting the two-dimensional matrix represented by i (x, y) into the vector s (x). This vector stores the weighted averages of the columns between s and s + Hs, where s represents the vertical displacement and Hs represents the number of lines of the horizontal slices.
[0130] The next process is the 1815 binarization of the vector resulting from the vertical stacking. This process will convert the values of this vector, which are between the range of 0 to 255, in just two values, black “0” and white “1” 1816. For this, it is necessary to define a threshold that separates the range of values that match black from values that match white. Due to adverse conditions, such as the presence of noise in the image and the lighting levels of the external environment, the ranges of values of the vector may vary affecting the result of binarization. Because of this, the vector is divided into regions and a threshold is calculated for each of these regions. The threshold is a weighted average of the region multiplied by a weight belonging to a previously defined set of weights. These weights are used to adapt the threshold to different lighting levels. If the vector value is less than the threshold, then black “0” is assigned; otherwise, white “1” is assigned, resulting in another vector whose values are between 0 and 1.

[0131] In equations 21 to 24, the vector binary method is presented. Equation (21) represents the set of weights used in calculating the thresholds. In equation (22) the size of the regions in which the vector will be divided is calculated, where W represents the size of the vector, Rn the number of regions to be used and Wr corresponds to the size of the regions. The Rn value is adaptive and can be changed to improve the decoding result. The threshold is calculated by equation (23), where xr represents the beginning of the region r, wi represents a weight i in the set of weights and lr represents the threshold calculated for region r. Finally, equation (24) calculates the binary vector, where b (x) will be white “1” for s (x) greater than or equal to lr or black “0” for s (x) less than lr.
[0132] The procedure described in the previous paragraph is detailed in figure 19. The horizontal slice 1910 is stacked vertically 1911 converting it into a vector. Then, the weight 1912 to be applied in the calculation of the threshold is obtained, the vector is divided into 1913 regions and the thresholds are calculated for the 1914 binarization. After this, it is possible to identify the bars of the code and calculate the digits of these 1915. The digits obtained are stored 1916 for later use. This process is repeated until all the weights in the weight set are used 1917.
[0133] The step converts pixels to bars 1817 receives the binary vector with the information in pixels and converts it to a vector with the bar code representation in the EAN-13 standard, which is formed by four different bar levels: “thin ”; “Average: 2 thin bars”; “Wide: 3 thin bars”; “Very wide: 4 thin bars”. With this, a method was created that tries to estimate how many pixels each bar has. This method tries to find the number of pixels in a thin bar and from there define the size of the other bars.
[0134] The binary vector 2010 is converted into two vectors, one with the sizes of the bars in pixels and a second vector containing the colors of the bars (black “0” or white “1”) 2011. These vectors are used in the calculations performed when converting pixels to bars. To obtain the correct sizes of each bar, it is first necessary to find the beginning of the bar code. It is known that the bar code in the EAN-13 standard starts with 3 thin bars interspersed between black and white. The beginning of the barcode is found when a black bar "0", a white bar "1" and a black bar "0" are found, whose biggest difference between the sizes of the bars is less than a threshold previously stipulated.
[0135] Figure 21 shows the structure of a bar code in the EAN-13 standard. In this standard, the first three bars 2110 are used to identify the beginning of the bar code and the last three bars 2114 identify the end of the bar code. The five bars of the 2112 centers divide the barcode into two groups, where the first group 2111 represents the initial seven numbers of the barcode, the first digit 2115 being defined by the parity between the bars of the first group. The second group 2113 represents the last six digits of the barcode. Each digit of the barcode is represented by four bars.
[0136] Thus, in order to read the barcode, first find the bars that represent the beginning of the barcode. This procedure uses the vector with the sizes of the bars in pixels and the vector with the colors of the bars, illustrated in 2011. It is known that the beginning of the bar code in the EAN-13 standard is represented by three thin bars with black colors, white and black, respectively 2110. Therefore, an adaptive method was created that tries to locate this pattern of bars from the biggest difference between the sizes of three continuous bars. If this difference is less than the threshold, used to represent the shortest distance between the sizes of the bars, and the colors of the bars coincide with the colors of the bars of the pattern that represents the beginning, then it means that the beginning of the bar code was located.

[0137] Equation (25) calculates the difference between the bars, where v is the vector with the size of the bars in pixels and x represents the displacement by the vector. If the value returned by M (v, x) is less than the threshold of the smallest difference, then the first group of the bar code starts at position x + 3 of the vector v. Therefore, the translation of the bars into the bar code digits starts from the position x + 3 of the vector with the size of the bars in pixels and with the vector containing the colors of the bars.
[0138] To carry out the translation of the digits, the vectors are traversed every four bars until completing the six digits of the first group 2110. Then, the five bars of the center 2112 are discarded, and the translation of the last six is carried out digits of the second group 2113. In the translation of the first digits, the parities of these, which represent the first digit of the barcode 2115, are also calculated.
[0139] Figure 22 shows an example of the process of converting bars to digits. In 2210, the bars representing the number four in the second group are presented. In this process, a vector with the size of the bars in pixels 2211 and a second vector are used to identify the colors of the bars. From these, an adaptive method was created that aims to convert pixels in just four levels of bar: "1" to thin bar; "2" for medium bar; “3” for wide bar; “4” for very wide bar.
[0140] It is known that each digit of the bar code is represented by seven thin bars. Therefore, the threshold used to define the bars is the size of the thin bar 2212. The threshold or size of the thin bar is obtained by adding the size of the bars in pixels divided by seven 2213.
[0141] Even after the image restoration process, it is possible to find noise in the image. These, in turn, affect the final result of the barcode decoding. To work around this problem, the method that converts pixels to bars tries to adjust the threshold at the time of translation. This adjustment is made by applying a set of weights to the threshold 2214, generating different thresholds 2215.
[0142] The conversion of pixels to bars is done by dividing the size of the bars in pixels by the thresholds obtained after applying the weights 2216. The result will be a matrix whose lines represent the results of the different thresholds and the columns represent the sizes of bars 2217.
[0143] Due to the presence of noise, there may be cases in which the process for obtaining the sizes of the bars is impaired, causing some bars to have more pixels, and in turn, others to have fewer pixels. That is, some bars will be very thin and others extremely wide. To solve this problem, a method has been developed that tries to restore the size of the bar. This method tries to identify the bars that became very thin, looking for values whose result of dividing the sizes of the bars in pixels by the thresholds was less than 0.5. Then 0.5 is added to the bar that became very thin and 0.5 is subtracted from the neighboring bars. This process is described in 2218 and an example of its result is presented in 2219.
[0144] The following procedure is to convert the resulting values to whole numbers. For this, the values 2220 are rounded up, leaving only the entire part 2221.
[0145] Then, the values obtained in the bars are converted, which are represented by a combination of 0’s (white bars) and 1’s (black bars). This combination has seven elements, each of which represents a thin bar. A list of these combinations 2223 is then obtained and then the bars are decoded 2224 in search of the number represented by them. In case of success, a list with the decoded numbers is obtained. If one or more digits are not decoded, a value of - 1 is returned for each of these respectively 2225.
[0146] The result of the process described in the previous paragraph is a list with the probable numbers 2225 that can be represented by the decoded bars. Then, from this list, an attempt is made to estimate the number most likely to be represented by the decoded bars. This is done by calculating the dominant digit 2226, which is the number other than -1 most frequently in the list of decoded numbers, as shown in 2227. This concludes the barcode digit translation step. 1819.
[0147] The binary process 1815, conversion of pixels to bars 1817, and translation of the digits of the bar code 1819 generates a vector with thirteen positions that contains the possible values of the numbers represented by the bar code. These operations are performed on each of the horizontal slices, generating, in turn, a vector of results for each of the slices. These vectors are used to construct an 1820 digit histogram, which is used to obtain the numbers represented by the bar code.
[0148] Through the digit histogram that is generated from the decoding of the horizontal slices, an attempt is made to reach an optimal result.
[0149] Figure 23 illustrates the process of obtaining the digit histogram and generating the final result.
[0150] Each horizontal slice of code 2310 is decoded separately, returning a sequence of decoded digits for each slice 2311. In case of failure to decode a digit, the value -1 (minus one) is assigned in its position.
[0151] To obtain the final value, a 2312 digit histogram is calculated. The frequency of the values in this histogram is affected by the number of numbers returned successfully during decoding.

[0152] The process for obtaining the digit histogram is presented in equations 26 to 28. Equation (26) returns 0 (zero) if the digit is equal to -1 (minus one), otherwise it returns 1 (one), where ui represents a vector with the sequence of digits returned when decoding a slice whose index is represented by i. Equation (27) calculates the increment of the histogram, where it is equal to the number of ui elements other than -1 (minus one). Equation (28) increments a value in the histogram, represented by H, where y is the digit to be incremented and x is the position of this digit in the barcode. The histogram will have its value updated only if the ui value in the x position is different from -1 (minus one).

[0153] After calculating the digit histogram, the numbers represented by bar code 1821, 2313 are then obtained, which will be those with the highest frequency in the digit histogram. This process is presented by equation (29), where y represents the barcode numbers and x its position. The final result of the decoding is represented in equation (29) by R, being stored in a vector with 13 (thirteen) positions.
[0154] The next process is the validation of the 1822 result, from which the sequence of numbers returned in the decoding is analyzed in order to verify whether this is a valid sequence. For this, it is firstly checked whether all digits have been decoded correctly. If all numbers have been successfully decoded, then a check digit is calculated. If the check digit is correct, then the decoded string is valid.
[0155] In case of error, a new frame is captured and the process is carried out again until the barcode is successfully decoded or until the time limit for reading is over.
[0156] When the decoding of a bar code is successful, then the “display the result” 1213 step begins. In this step the decoded code is displayed on the display screen of the portable device. From this result, it is possible to carry out research on the points of sale that supply the product identified by the decoded bar code, as well as to consult its price.
[0157] Although a preferred embodiment of the present invention is shown and described, those skilled in the art have understood that various modifications can be made without departing from the scope and spirit of the present invention, as defined in the appended claims.
[0158] It is expressly provided that all combinations of elements that perform the same function in substantially the same way, to achieve the same results, are within the scope of the present invention.

权利要求:
Claims (18)
[0001]
1. Method for restoring an image of an unfocused bar code characterized by the fact that it comprises: - capturing the image; - restore the image by selecting an optical transfer function (OTF) in a database and deconvolution the image; - decode the barcode on the restored image in order to return numbers represented by the barcode; and - display the returned numbers, in which, in the deconvolution of the barcode, a cut-off threshold is determined, which adjusts to the ambient lighting, to separate a white level from a black level in the barcode image in shades of green. Grey.
[0002]
2. Method, according to claim 1, characterized by the fact that the image comprises black and white dots that represent the pixels.
[0003]
3. Method, according to claim 1, characterized by the fact that pixels are scattered due to the incorrect focal length, and the scattering comprises a Point Scattering Function (PSF), which must be segmented for use.
[0004]
4. Method, according to claim 3, characterized by the fact that the deconvolution of the blurred image using the PSF or OTF estimates an image closer to the ideal one for decoding the bar code, by removing the blur effect caused by spreading the pixels of the barcode image, and increasing the method's hit rate.
[0005]
5. Method, according to claim 3, characterized by the fact that it additionally comprises a camera calibration procedure that creates a database of PSF's, reflecting the incidence of the blur effect at a certain point, from a certain point distance, the PSF's of this database being used in deconvolution which removes the blur effect and restores the image with the barcode.
[0006]
6. Method, according to claim 5, characterized by the fact that it additionally comprises extracting static PSF's for use in deconvolution.
[0007]
7. Method, according to claim 5, characterized by the fact that the PSF's database comprises data relating to the camera in particular, depending on the models of different cameras having different focal lengths.
[0008]
8. Method, according to claim 3, characterized by the fact that a bank of PSF’s is estimated.
[0009]
9. Method, according to claim 1, characterized by the fact that it also comprises: - digitizing the captured image creating a mask of regions with the probable positions of Point Scattering Functions (PSF’s); - copy only the PFS'S values, excluding a background image; - cut each of the PSF's present in the image to store them separately in an initial database, the referred database being analyzed to select the initial PSF's that best restore the image; - check several barcodes by decoding using the PSF's from the database to restore the image; - where the PSF’s whose decoding is successful are PSF’s whose numbers are exactly the same as the numbers encoded by the selected bar codes.
[0010]
10. Method, according to claim 1, characterized by the fact that a size of a bar code is estimated considering a threshold based on the specific characteristics of dimensions of the bar code used in EAN-13.
[0011]
11. Method, according to claim 1, characterized by the fact that the restoration of the image allows the use of devices with cameras without autofocus.
[0012]
12. Method, according to claim 1, characterized by the fact that it additionally comprises performing real-time processing and analyzing frames obtained in a preview mode.
[0013]
13. Method, according to claim 1, characterized by the fact that regions that do not belong to the barcode are excluded from processing.
[0014]
14. Method, according to claim 1, characterized by the fact that the bar code is sliced horizontally and the reading is performed on each of these slices individually.
[0015]
15. Method, according to claim 14, characterized by the fact that the number of slices is adjusted according to the execution, causing slices of different positions and sizes to be obtained, thus increasing the probability of success in decoding.
[0016]
16. Method, according to claim 14, characterized by the fact that the slice to be decoded is divided into vertical regions, and for each slice a threshold is obtained according to the intensity of a pixel.
[0017]
17. Method, according to claim 1, characterized by the fact that the barcode decoding is performed on images captured by low-cost cameras that have a fixed focus.
[0018]
18. Method, according to claim 1, characterized by the fact that it additionally comprises mirroring the barcode image.

类似技术:

公开号 | 公开日 | 专利标题

BR102012033723B1|2020-12-01|method for restoring blurred barcode images

BRPI1100823B1|2022-01-11|AUTOMATIC SYSTEM AND METHOD FOR TRACKING AND DECODING BARCODE USING PORTABLE DEVICES EQUIPPED WITH DIGITAL CAMERAS

KR102113911B1|2020-05-21|Feature extraction and matching and template update for biometric authentication

KR102082301B1|2020-02-27|Method, apparatus and computer-readable recording medium for converting document image captured by camera to the scanned document image

EP2415015B1|2015-08-12|Barcode processing

US20140247470A1|2014-09-04|Camera-based document imaging

US10140495B2|2018-11-27|Deconvolution of digital images

US8457403B2|2013-06-04|Method of detecting and correcting digital images of books in the book spine area

US11194984B2|2021-12-07|Localization of machine-readable indicia in digital capture systems

Sörös et al.2013|Blur-resistant joint 1D and 2D barcode localization for smartphones

US8811751B1|2014-08-19|Method and system for correcting projective distortions with elimination steps on multiple levels

JP2011045078A|2011-03-03|Adaptive deblurring for camera-based document image processing

Dubská et al.2016|Real-time precise detection of regular grids and matrix codes

US8897600B1|2014-11-25|Method and system for determining vanishing point candidates for projective correction

Chu et al.2011|Stabilization and extraction of 2D barcodes for camera phones

Liu et al.2013|Two-dimensional bar code out-of-focus deblurring via the increment constrained least squares filter

CN105469026A|2016-04-06|Horizontal and vertical line detection and removal for document images

Munoz-Mejias et al.2011|A low-complexity pre-processing system for restoring low-quality QR code images

BR112013011943B1|2020-12-29|method for identifying a two-dimensional bar code in digital image data of the bar code, non-transitory computer readable medium and apparatus configured to identify a two-dimensional bar code in digital image data of the bar code

JP6542230B2|2019-07-10|Method and system for correcting projected distortion

Kim et al.2013|Dewarping book page spreads captured with a mobile phone camera

Liu et al.2008|Bar code recognition in complex scenes by camera phones

Liyanage2007|Efficient decoding of blurred, pitched, and scratched barcode images

Wasule et al.2016|An effective approach to recover corrupted or mobile captured 2D barcode images with improved accuracy

Dinesh et al.2013|Classification and decoding of barcodes: an image processing approach

同族专利:

公开号 | 公开日

US20130170765A1|2013-07-04|

BR102012033723A2|2015-05-12|

BR102012033723A8|2018-06-05|

US8867857B2|2014-10-21|

引用文献:

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

US6152371A|1998-08-12|2000-11-28|Welch Allyn, Inc.|Method and apparatus for decoding bar code symbols|

US7156311B2|2003-07-16|2007-01-02|Scanbuy, Inc.|System and method for decoding and analyzing barcodes using a mobile device|

WO2005069216A1|2004-01-15|2005-07-28|Matsushita Electric Industrial Co., Ltd.|Measuring method for optical transfer function, image restoring method, and digital imaging device|

US7172124B2|2005-03-10|2007-02-06|Microsoft Corporation|Camera-based barcode recognition|

US7237721B2|2005-05-24|2007-07-03|Nokia Corporation|Image processing for pattern detection|

WO2008020449A2|2006-08-16|2008-02-21|Xceed Imaging Ltd.|Method and system for imaging with a zoom|

US8199246B2|2007-05-30|2012-06-12|Fujifilm Corporation|Image capturing apparatus, image capturing method, and computer readable media|

US20090152358A1|2007-12-14|2009-06-18|Epshteyn Alan J|System and Method for a Barcode Reading Functionality Using an Image Capturing Device|

EP2093697B1|2008-02-25|2017-08-23|Telefonaktiebolaget LM Ericsson |Method and arrangement for retrieving information comprised in a barcode|

EP2382583B1|2008-12-26|2016-09-21|Datalogic ADC, Inc.|Systems and methods for imaging|

WO2010114449A1|2009-03-30|2010-10-07|Telefonaktiebolaget Lm Ericsson |Barcode processing|

US9098764B2|2009-07-20|2015-08-04|The Regents Of The University Of California|Image-based barcode reader|

US8851378B2|2009-09-24|2014-10-07|Ebay Inc.|System and method for recognizing deformed linear barcodes from a stream of varied-focus video frames|

KR101055853B1|2011-01-04|2011-08-09|올라웍스|Method, terminal and computer-readable recording medium for deblurring barcode image|US8948513B2|2009-01-27|2015-02-03|Apple Inc.|Blurring based content recognizer|

US9098764B2|2009-07-20|2015-08-04|The Regents Of The University Of California|Image-based barcode reader|

US8606032B2|2009-12-23|2013-12-10|Sony Corporation|Image processing method, device and program to process a moving image|

US8905314B2|2010-09-30|2014-12-09|Apple Inc.|Barcode recognition using data-driven classifier|

CN102270292B|2011-05-24|2015-09-09|惠州Tcl移动通信有限公司|Mobile terminal and coding/decoding method thereof|

US9124797B2|2011-06-28|2015-09-01|Microsoft Technology Licensing, Llc|Image enhancement via lens simulation|

WO2013052241A1|2011-10-03|2013-04-11|Nikon Corporation|Motion blur estimation and restoration using light trails|

US9137526B2|2012-05-07|2015-09-15|Microsoft Technology Licensing, Llc|Image enhancement via calibrated lens simulation|

US20140231449A1|2013-02-19|2014-08-21|Gojo Industries, Inc.|Refill container labeling|

US9418400B2|2013-06-18|2016-08-16|Nvidia Corporation|Method and system for rendering simulated depth-of-field visual effect|

US10909845B2|2013-07-01|2021-02-02|Conduent Business Services, Llc|System and method for enhancing images and video frames|

JP6318520B2|2013-09-27|2018-05-09|株式会社リコー|Imaging apparatus, imaging system, and imaging method|

WO2016010560A1|2014-07-18|2016-01-21|Hewlett-Packard Development Company, L. P.|Frequency domain range determination for a periodic or quasi-periodic target|

US10248825B2|2014-08-29|2019-04-02|Hewlett-Packard Development Company, L.P.|Image processing|

US10198648B1|2015-04-10|2019-02-05|Digimarc Corporation|Decoding 1D-barcodes in digital capture systems|

US10331928B2|2015-11-06|2019-06-25|International Business Machines Corporation|Low-computation barcode detector for egocentric product recognition|

US20170200247A1|2016-01-08|2017-07-13|Confirm, Inc.|Systems and methods for authentication of physical features on identification documents|

CN108536707A|2017-03-03|2018-09-14|绍兴读图网络科技有限公司|Pictorial information safe processing method and system|

US10185857B2|2017-06-14|2019-01-22|Canon Information And Imaging Solutions, Inc.|Devices, systems, and methods for reading barcodes|

法律状态:
2015-05-12| B03A| Publication of a patent application or of a certificate of addition of invention [chapter 3.1 patent gazette]|

2018-06-05| B03H| Publication of an application: rectification [chapter 3.8 patent gazette]|

2018-12-04| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|

2020-02-04| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

2020-07-21| B09A| Decision: intention to grant [chapter 9.1 patent gazette]|

2020-12-01| B16A| Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]|Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 28/12/2012, OBSERVADAS AS CONDICOES LEGAIS. |

优先权:

申请号 | 申请日 | 专利标题

US13/338,988|2011-12-28|

US13/338,988|US8867857B2|2011-12-28|2011-12-28|Method for restoration of blurred barcode images|

[返回顶部]