专利摘要:
An image enhancement method includes constructing an input histogram corresponding to an input image received at a focal plane array, the input histogram representing a distribution distribution. pixel intensities corresponding to the input image and executing an analytic operation on the input histogram to produce a modified cumulative distribution, wherein the analytic operation is dependent on a camera temperature. The input image is transformed using the modified cumulative distribution to produce an improved output image corresponding to the input image, wherein at least a portion of the input image is improved in the output image. In addition to or instead of the nonlinear operation, the class-grouping edges of the input histogram can be adjusted based on at least one of a camera temperature and a sensitivity rate for construct an adjusted cumulative distribution.
公开号:BE1024436B1
申请号:E2017/5411
申请日:2017-06-09
公开日:2018-02-27
发明作者:Jonathan Nazemi
申请人:Sensors Unlimited Inc.;
IPC主号:
专利说明:

(30) Priority data:
06/10/2016 US 15178888 (73) Holder (s):
SENSORS UNLIMITED INC. NJ 08540, PRINCETON United States (72) Inventor (s):
NAZEMI Jonathan 18901 DOYLESTOWN United States (54) IMAGE ENHANCEMENT (57) The invention relates to an image improvement method comprising the construction of an input histogram corresponding to an input image received at the level a focal plane array, the input histogram representing a distribution of pixel intensities corresponding to the input image and performing an analytical operation on the input histogram to produce a cumulative distribution modified, in which the analytical operation depends on a camera temperature. The input image is transformed using the modified cumulative distribution to produce an improved output image corresponding to the input image, in which at least part of the input image is improved in the output image. In addition to or in place of the non-linear operation, the grouping edges by class of the input histogram can be adjusted based on at least one of a camera temperature and a sensitivity rate for build an adjusted cumulative distribution.
BELGIAN INVENTION PATENT
FPS Economy, SMEs, Middle Classes & Energy
Publication number: 1024436 Deposit number: BE2017 / 5411
Intellectual Property Office International Classification: G06T 5/00 G06T 5/40 Date of issue: 02/27/2018
The Minister of the Economy,
Having regard to the Paris Convention of March 20, 1883 for the Protection of Industrial Property;
Considering the law of March 28, 1984 on patents for invention, article 22, for patent applications introduced before September 22, 2014;
Given Title 1 “Patents for invention” of Book XI of the Code of Economic Law, article XI.24, for patent applications introduced from September 22, 2014;
Having regard to the Royal Decree of 2 December 1986 relating to the request, the issue and the maintenance in force of invention patents, article 28;
Considering the patent application received by the Intellectual Property Office on 09/06/2017.
Whereas for patent applications falling within the scope of Title 1, Book XI of the Code of Economic Law (hereinafter CDE), in accordance with article XI. 19, §4, paragraph 2, of the CDE, if the patent application has been the subject of a search report mentioning a lack of unity of invention within the meaning of the §ler of article XI.19 cited above and in the event that the applicant does not limit or file a divisional application in accordance with the results of the search report, the granted patent will be limited to the claims for which the search report has been drawn up.
Stopped :
First article. - It is issued to
SENSORS UNLIMITED INC., 330 Carter Road, Suite 100, NJ 08540 PRINCETON United States;
represented by
OFFICE KIRKPATRICK S.A., Avenue Wolfers 32, 1310, LA HULPE;
a 20-year Belgian invention patent, subject to payment of the annual fees referred to in article XI.48, §1 of the Code of Economic Law, for: IMPROVEMENT OF IMAGES.
INVENTOR (S):
NAZEMI Jonathan, 520 Portsmouth Ct „18901, DOYLESTOWN;
PRIORITY (S):
06/10/2016 US 15178888;
DIVISION:
divided from the basic application: filing date of the basic application:
Article 2. - This patent is granted without prior examination of the patentability of the invention, without guarantee of the merit of the invention or of the accuracy of the description thereof and at the risk and peril of the applicant (s) ( s).
Brussels, 02/27/2018, By special delegation:
BE2017 / 5411
IMPROVED IMAGES
STATEMENT CONCERNING FEDERAL GOVERNMENT RESEARCH OR DEVELOPMENT
The present invention has been produced with the support of the government under contracts numbers W15P7T-10D-D413 / 0024 (P8650) and N00014-14-C-0061 awarded by the Naval Research Bureau and CACI-NVESD. The government has certain rights in this invention.
BACKGROUND OF THE INVENTION
1. Field of the invention
The present invention relates to image enhancement, and more particularly to image enhancements based on histogram manipulation and the like.
2. Description of the related technique
It is often desirable to improve an image acquired by a camera or a sensor to reveal additional detail in the image. For example, when images are taken in bright light, details in the shadows and in bright areas are often lost. Likewise, details are often lost in dark areas of dark images. As such, it is desirable to reveal such details without compromising overall image quality.
A number of image enhancement methods have been developed in the past. Examples of these methods include stretching contrasts,
BE2017 / 5411 histogram equalization, histogram projection, and plateau equalization. In a stretch of contrasts, pixel intensity distributions are extended ("stretched") to cover all possible values. According to this method, maximum and minimum pixel intensity values are measured (or estimated), and offset and gain correction values are applied to the pixel intensity values corresponding to each pixel in the image, from so that all intensity values are covered. In histogram equalization, the pixel intensity distribution is attenuated or flattened. According to this method, an input histogram is constructed, which includes the total number of pixels corresponding to each intensity level inside the image. Then, a cumulative histogram is constructed and normalized, and each pixel in the image is transformed using the normalized cumulative histogram as a conversion table (LUT). In a histogram projection, a histogram is constructed, made up of covered and uncovered levels. A cumulative histogram is then constructed and normalized, and each pixel in the image is transformed using the normalized cumulative histogram as a conversion table. In plateau equalization, an input histogram is constructed and clipped at a specified level or "plateau". Next, a cumulative histogram is constructed and normalized, and each pixel in the image is transformed using the normalized cumulative histogram as a conversion table.
The aforementioned image enhancement methods can have a number of disadvantages. By
BE2017 / 5411 For example, conventional image enhancement methods may not adequately display detail in specific regions of an image, such as shadows or bright areas. On the contrary, these methods tend to uniformly improve the whole of an image, which can cause distortion of other regions of the image in favor of the improvement of a desired region of the image. In addition, conventional image enhancement methods can hardly be adapted, without manual adjustment, to enhance images in different environments (for example, dynamically altering enhancement characteristics in response to changes in the camera sensitivity).
Such conventional methods and systems have generally been considered satisfactory for their intended purposes. However, there is still a need in the art for improved image enhancement. The present invention relates to a solution to this need.
SUMMARY OF THE INVENTION
A method for improving an image comprises the construction of an input histogram corresponding to an input image received at a focal plane network, the input histogram representing a distribution of pixel intensities corresponding to the input image. From the input histogram, a cumulative distribution is calculated. The cumulative distribution is subject to modifications aimed at adjusting an image contrast by redistributing the pixel values. Modifications may include the application of analytical operations which may depend on a camera temperature and / or a
BE2017 / 5411 sensitivity rate. The input image is transformed using the modified cumulative distribution to produce an improved output image corresponding to the input image, at least a portion of the input image being enhanced in the output image.
Executing an analytical operation may include calculating a power of the cumulative distribution to produce a resulting LUT conversion table. The execution of an analytical operation can include the multiplication of the resulting cumulative distribution by a ramp function expressed as follows
RAMP_NORM = (X + ENH_RAMP) / (1 + ENH_RAMP) and
RAMP_LUT = MAX_BIN * RAMP_NORM, with
X = ([0: MAX_BIN] - OFFSET (T)) / (CLIP (T) - OFFSET (T)),
in which if X <0 then X = 0, if X > 1 then X = 1.
where OFFSET (T) is here the minimum input histogram level depending on the temperature, CLIP (T) is the maximum input histogram level dependent on the temperature, ENH_RAMP is a slope adjustment, RAMP_NORM is the normalized ramp function, RAMP_LUT is the conversion table for the ramp calculation, and MAX_BIN is the maximum value for grouping by class of the cumulative LUT.
Executing an analytical operation may include receiving camera temperature data and using the camera temperature,
BE2017 / 5411 the analytical operation depending on a camera temperature. The input image can be received from the focal plane network, the focal plane network possibly varying in temperature.
A system may include an imaging device including a focal plane array for capturing an input image. The system may also include an image enhancement module operatively connected to the imaging device to implement any of the methods described herein for image enhancement. The imaging device may include a temperature sensor operatively connected to the module to provide the camera temperature. The system may include any suitable output device for outputting the transformed or improved image.
These and other features of the systems and methods of the present invention will become more apparent to those skilled in the art upon reading the following detailed description of the preferred embodiments associated with the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
In order for those skilled in the art to which the present invention is intended to readily understand how to easily realize and implement the devices and methods of the present invention, preferred embodiments thereof will be described in detail in the this document below, with reference to certain figures, among which:
Figure 1 is a schematic view of an illustrative embodiment of a system constructed in accordance with the present invention, showing a
BE2017 / 5411 imaging device and a module for enhancing images from the imaging device; and Figure 2 is a schematic view of an illustrative histogram in accordance with the present invention, schematically showing a temperature-dependent adjustment of the grouping edges by class and a non-linear operation for image enhancement.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Reference will now be made to the drawings in which the reference numbers define similar features or structural aspects of the present invention. By way of explanation and illustration, but not limitation, a partial view of an embodiment of a system according to the invention is shown in FIG. 1 and is generally designated by the reference character 100. Other embodiments systems according to the invention, or aspects thereof, are provided in Figure 2, as described below. The systems and methods described in this document can be used in image enhancement in uncontrolled or only partially controlled temperature imaging systems, for example, for infrared imaging. Reference is made to U.S. Patent No. 8,218,868 which is cited herein for reference in its entirety.
A system 100 includes an imaging device having optical elements 102 optically coupled to a focal plane array (FPA) 104 to capture an input image. An image enhancement module 106 is connected
BE2017 / 5411 functionally to the focal plane network 104 of the imaging device to implement any of the methods described in this document for improving images. A temperature sensor 108 is operatively connected to the module 106 to provide the camera temperature. According to this example, a sensor 108 is shown mounted on an FPA 104 to detect the temperature of the FPA 104, however those skilled in the art will easily understand that the sensor 108 can be mounted elsewhere to determine an FPA or a temperature indirectly. An output device 110 is operatively connected to the module 106 for outputting the transformed or improved images, and may include any suitable type of output device such as a display or storage device.
Referring now to Figure 2, a method of enhancing an image includes constructing an input histogram 112 corresponding to an input image received at the FPA 104 in Figure 1. The histogram input represents a pixel intensity distribution corresponding to the input image. The grouping edges by class 114 and 116 of the input histogram are adjusted based on at least one of a camera temperature and a sensitivity rate to construct a cumulative distribution (a). The adjustment of the grouping edges by class 114 and 116 is shown diagrammatically in FIG. 2 by double arrows. The input image is transformed using the cumulative distribution (b) to produce an improved output image corresponding to the input image, at least part of the input image
BE2017 / 5411 being improved in the output image. This improved image can be in the form of an output from an output device 110.
Adjusting the grouping edges by class 114 and 116 may include receiving camera temperature data, for example, from sensor 104, and using the camera temperature to adjust the grouping edges by class 114 and 116. The method may include performing automatic gain selection and / or automatic sensitivity selection over the focal plane array 104 as a function of camera temperature using the adjusted class clustering edges 114 and 116. The input image can be received from the focal plane network 104, the focal plane network 104 possibly varying in temperature, that is to say that the focal plane network is not temperature controlled or is only partially temperature controlled. Thus, image enhancement works in systems without thermoelectric cooling or the like.
Executing an analytical operation may include calculating a power of the cumulative distribution to produce a resulting LUT. The execution of an analytical operation can include the multiplication of the resulting cumulative distribution by a ramp function expressed as follows
RAMP_NORM = (X + ENH_RAMP) / (1 + ENH_RAMP) and
RAMP_LUT = MAX_BIN * RAMP_NORM, with
X = ([0: MAX_BIN] - OFFSET (T)) / (CLIP (T) - OFFSET (T)),
BE2017 / 5411 in which if X <0 then X = 0, if X> 1 then X = 1.
where OFFSET (T) is here the minimum input histogram level depending on the temperature, CLIP (T) is the maximum input histogram level dependent on the temperature, ENH_RAMP is a slope adjustment, RAMP_NORM is the normalized ramp function, RAMP_LUT is the conversion table for the ramp calculation, MAX_BIN is the maximum value for grouping by class of the cumulative LUT.
Executing an analytical operation can include receiving camera temperature data, for example, a module 106 can receive temperature data from a sensor 108, and using camera temperature, analytical operation depending on a camera temperature. As described previously with regard to the adjustment of the grouping edges by class 114 and 116, the input image can be received from the focal plane network 104, the focal plane network 104 possibly varying in temperature.
The method may include both an analytical operation and the adjustment of grouping edges by class as described above. For example, the grouping edges by class 114 and 116 of the input histogram 112 can be adjusted based on at least one of a camera temperature and a sensitivity rate to construct an adjusted cumulative distribution ( a), the execution of an analytical operation on the input histogram comprising the execution of
BE2017 / 5411 the analytical operation on the adjusted histogram (a) to obtain the modified cumulative distribution (b).
Potential advantages of the systems and methods described herein include uncontrolled or only partially controlled temperature imaging with image enhancements and allowing automatic gain and sensitivity control which was previously only possible for temperature-controlled imaging systems.
The methods and systems of the present invention, as described above and shown in the drawings, allow image enhancement with superior properties including image enhancement within uncontrolled temperature imaging systems or only partially controlled. Although the apparatuses and methods of the present invention have been presented and described with reference to the preferred embodiments, those skilled in the art will readily understand that changes and / or modifications may be made thereto. without departing from the scope of the present invention.
BE2017 / 5411
权利要求:
Claims (11)
[1]
1. Method for improving an image comprising:
constructing an input histogram corresponding to an input image received at a focal plane array, the input histogram representing a distribution of pixel intensities corresponding to the input image; adjusting grouping edges by class of the input histogram base on at least one of a camera temperature and a sensitivity rate to construct a modified cumulative distribution; and transforming the input image using the modified cumulative distribution to produce an image of
improved output corre spondant to the image < the entrance, in which at less a part of .1 'i mage input is improved d .years 1 ' picture Release. 2. Process is law ί. the claim 1, in which
adjusting the grouping edges by class involves receiving camera temperature data and using the camera temperature to adjust the grouping edges by class.
[2]
3, The method according to claim 1, further comprising receiving the input image from the focal plane network in which the focal plane network can vary in temperature.
[3]
4, Method for improving an image comprising:
construction of an input histogram corresponding to an input image received at a focal plane network,
BE2017 / 5411
1 / input histogram representing a distribution of pixel intensities corresponding to the input image; performing an analytical operation on the input histogram to produce a modified histogram, in which the analytical operation is dependent on a camera temperature;
transforming the input image using a cumulative distribution of the modified histogram to produce an improved output image corresponding to the input image, in which at least part of the input image is improved in the output image, the execution of an analytical operation comprising the calculation of a power of the input histogram to produce a resulting histogram and the multiplication of the resulting histogram by a ramp function expressed as follows
RAMP_NORM = (X + ENH_RAMP) / (1 + ENH_RAMP) and
20 RAMP__LUT - MAX__BIN ^ RAMP__NORM, with
X - ([0: MAX BIN] - OFFSET (T)) / (CLIP (T) - OFFSET (T)) in which if X <0 then X = 0,
25 if X> 1 then X = 1;
where OFFSET (T) is the minimum temperature dependent input histogram level, CLIP (T) is the maximum temperature dependent input histogram level, ENH RAMP is a slope adjustment, RAMP “ NORM is the normalized ramp function, RAMP LUT is the conversion table for the ramp calculation, MAX BIN is
BE2017 / 5411 the maximum value for grouping by class of the
Cumulative LUT.
[4]
5. The method of claim 4, wherein performing an analytical operation includes receiving camera temperature data and using the camera temperature, wherein the non-linear operation is dependent on a temperature. camera.
[5]
The method of claim 4, further comprising receiving the input image from the focal plane array in which the focal plane array can vary in temperature.
[6]
7. The method of claim 4, further comprising: adjusting the grouping edges by class of the input histogram base on at least one of a camera temperature and a sensitivity rate for
Constructing an adjusted histogram, wherein performing an analytical operation on the input histogram includes performing the analytical operation on the adjusted histogram to produce the modified histogram.
25
[7]
The method of claim 7, wherein adjusting the grouping edges by class includes receiving camera temperature data and using the camera temperature to adjust the data grouping edges by class.
[8]
The method of claim 4, further comprising receiving the input image from the planar network.
BE2017 / 5411 focal in which the local plane network can vary in temperature.
r_;
-j
BE2017 / 5411
BE2017 / 5411
BE2017 / 5411
ABBREVIATION OF THE INVENTION
AME LIORATION OF IMAGE S
The invention relates to a method for improving an image comprising the construction of an input histogram corresponding to an input image received at a focal plane network, the input histogram
[9]
10 showing a distribution of pixel intensities corresponding to the input image and performing an analytical operation on the input histogram to produce a modified cumulative distribution, in which the analytical operation depends on a camera temperature
[10]
The input image is transformed using the modified cumulative distribution to produce an improved output image corresponding to the input image, in which at least part of the input image is enhanced in the output image . In addition or instead
[11]
20 of the non-linear operation, the grouping edges by class of the input histogram can be adjusted based on at least one of a camera temperature and a sensitivity rate to construct an adjusted cumulative distribution .
Uumero de ia defrsatxte £ £ urt> pSÎSÎÎiCÎ ftcertt Office üftîq «K-urccp & en ôrî> patents
80 11 483 BE 201705411
类似技术:
公开号 | 公开日 | 专利标题
BE1024436B1|2018-02-27|IMAGE ENHANCEMENT
Snyder et al.1993|Image recovery from data acquired with a charge-coupled-device camera
US20050053309A1|2005-03-10|Image processors and methods of image processing
Hope et al.2016|High-resolution speckle imaging through strong atmospheric turbulence
Tyson1986|Low-light-level charge-coupled device imaging in astronomy
Sun et al.2019|Improving the performance of computational ghost imaging by using a quadrant detector and digital micro-scanning
Ashiba et al.2018|Enhancement of infrared images based on efficient histogram processing
Negrete-Regagnon1996|Practical aspects of image recovery by means of the bispectrum
Meza et al.2015|Simultaneous digital super-resolution and nonuniformity correction for infrared imaging systems
WO2020053837A1|2020-03-19|Photographic underexposure correction using a neural network
Sadot et al.1995|Target acquisition modeling for contrast-limited imaging: effects of atmospheric blur and image restoration
Arsenault et al.1981|Transformation of film-grain noise into signal-independent additive Gaussian noise
Vollmerhausen et al.2017|Extending the range performance of diffraction limited imagers
Konnik et al.2010|Extension of the possibilities of a commercial digital camera in detecting spatial intensity distribution of laser radiation
Du Bosq et al.2017|Performance assessment of a compressive sensing single-pixel imaging system
Jara et al.2020|Multiplanar full-field blur correction method for infrared microscopy imaging
Konnik et al.2009|The use of a consumer grade photo camera in optical–digital correlator for pattern recognition and input scene restoration
Rousset et al.2014|Comparison of EMCCD post-processing methods for photon counting flux ranges
Short et al.2019|Holistic approach to high-performance long-wave infrared system design
Wu et al.2020|Automatic on-orbit adjusting TDI stages of intelligent space camera based on sun zenith angles
Bourree2014|Performance of PHOTONIS'low light level CMOS imaging sensor for long range observation
Dietz et al.2015|ISO-less?
Vítek et al.2017|Radiometric calibration of wide-field camera system with an application in astronomy
Fei et al.2021|Short-wave infrared real-time high dynamic range imaging and display based on correlated double sampling
Bürker et al.2014|Exposure control for HDR video
同族专利:
公开号 | 公开日
BE1024436A1|2018-02-19|
US20180276804A1|2018-09-27|
TW201743285A|2017-12-16|
US10007974B2|2018-06-26|
US10346961B2|2019-07-09|
TWI720205B|2021-03-01|
US20170358066A1|2017-12-14|
引用文献:
公开号 | 申请日 | 公开日 | 申请人 | 专利标题

US4868651A|1988-05-17|1989-09-19|S&S Inficon, Inc.|Digital radiography with image brightness and contrast normalization|
US5063607A|1988-10-24|1991-11-05|Hughes Aircraft Company|Image contrast enhancement technique|
JPH0773308A|1993-09-03|1995-03-17|Matsushita Electric Ind Co Ltd|Digital image processor|
GB2357649A|1999-12-22|2001-06-27|Nokia Mobile Phones Ltd|Image enhancement using inverse histogram based pixel mapping|
US6792141B2|2001-04-19|2004-09-14|Raytheon Company|Infrared detection system and method with histogram based manual level and gain control with local gain clipping|
US20090231436A1|2001-04-19|2009-09-17|Faltesek Anthony E|Method and apparatus for tracking with identification|
SE519547C2|2001-06-18|2003-03-11|Flir Systems Ab|Method and apparatus for providing an infrared image.|
JP4250595B2|2004-02-16|2009-04-08|キヤノン株式会社|Signal processing method and signal processing circuit|
US8285041B2|2004-09-14|2012-10-09|Olympus Corporation|Image processing apparatus, image recording apparatus, and image processing method|
US7340162B2|2005-02-23|2008-03-04|Flir Systems, Inc.|Infrared camera systems and methods|
US8077995B1|2005-02-23|2011-12-13|Flir Systems, Inc.|Infrared camera systems and methods using environmental information|
US7796168B1|2006-06-09|2010-09-14|Flir Systems, Inc.|Methods and systems for detection and mitigation of image-flash in infrared cameras|
US7936921B2|2007-01-04|2011-05-03|Freescale Semiconductor, Inc.|Efficient fixed-point real-time thresholding for signal processing|
US8218868B2|2007-06-13|2012-07-10|Sensors Unlimited, Inc.|Method and apparatus for enhancing images|
KR101361788B1|2008-02-25|2014-02-21|알제이에스 테크놀로지, 인코포레이티드|System and method for a high dynamic range image sensor sensitive array|
US9491376B2|2009-02-23|2016-11-08|Flir Systems, Inc.|Flat field correction for infrared cameras|
US10051210B2|2011-06-10|2018-08-14|Flir Systems, Inc.|Infrared detector array with selectable pixel binning systems and methods|
US20100245608A1|2009-03-25|2010-09-30|Itt Manufacturing Enterprises, Inc.|Adaptive method and system for extracting a bright image from a thermal image|
US9819880B2|2009-06-03|2017-11-14|Flir Systems, Inc.|Systems and methods of suppressing sky regions in images|
US8569684B2|2009-11-06|2013-10-29|Steven J. Olson|Infrared sensor control architecture|
JP2011166522A|2010-02-10|2011-08-25|Sony Corp|Image processing device, image processing method and program|
US8649578B2|2010-03-08|2014-02-11|Agfa Healthcare Inc.|Method and system for defining a breast window|
US8954322B2|2011-07-25|2015-02-10|Via Telecom Co., Ltd.|Acoustic shock protection device and method thereof|
TWM458747U|2013-03-27|2013-08-01|Regulus Technologies Co Ltd|Image processing module|
US10089787B2|2013-12-26|2018-10-02|Flir Systems Ab|Systems and methods for displaying infrared images|
KR102301231B1|2014-07-31|2021-09-13|삼성전자주식회사|Method and device for providing image|
US9930324B2|2014-08-05|2018-03-27|Seek Thermal, Inc.|Time based offset correction for imaging systems|
WO2016089823A1|2014-12-02|2016-06-09|Seek Thermal, Inc.|Image adjustment based on locally flat scenes|
EP3142355B1|2015-09-08|2017-10-25|Axis AB|Method and apparatus for enhancing local contrast in a thermal image|
US10007974B2|2016-06-10|2018-06-26|Sensors Unlimited, Inc.|Enhancing images|US10007974B2|2016-06-10|2018-06-26|Sensors Unlimited, Inc.|Enhancing images|
KR20180040286A|2016-10-12|2018-04-20|삼성전자주식회사|Display apparatus and method of controlling thereof|
US10565685B2|2017-03-30|2020-02-18|Disney Enterprises, Inc.|Denoising binned-depth images|
法律状态:
2018-03-14| FG| Patent granted|Effective date: 20180227 |
优先权:
申请号 | 申请日 | 专利标题
US15/178,888|US10007974B2|2016-06-10|2016-06-10|Enhancing images|
US15178888|2016-06-10|
[返回顶部]