• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The present invention relates to a sensor unit 20 comprising a camera 24, one or more preprocessing means 26 for performing image pre-processing, object detection and extraction of data features to generate a reduced data set 80 containing extracted data features 192. The sensor unit 20 may further comprise sensor communication means (s). 22 to transmit a reduced data set 80. The present invention relates to a sensor unit 20 which can be implemented in etvision system 300.
    公开号:DK201800090U1
    申请号:DK201800090U
    申请日:2017-12-13
    公开日:2019-07-15
    发明作者:Eld Ibsen Per；Geltzer Dinesen Palle；Stankovic Boris
    申请人:Ubiqisense Aps；
    IPC主号:
    专利说明:

    Vision sensor system for detecting, classifying and tracking objects
    Area of production
    The present invention relates to a sensor unit comprising a camera, one or more pre-processing means for performing image pre-processing, object detection and extraction of data features to generate a reduced data set containing extracted data features. The sensor unit may further comprise sensor communication means (s) for transmitting a reduced data set. The present invention relates to a sensor unit which can be implemented in a vision system.
    Background to the creation
    With the Internet and Things (IoT) and with an ever-increasing number of sensors, a rapid increase in load on the required bandwidth is generally expected.
    In particular, the increasing use of computer vision technology may cause large bandwidth needs depending on the image quality, frame rate and image processing.
    Vision technology is already used today for many different purposes and in a wide range of technologies. Machine operation, including tasks such as control and feedback, is a frequently used purpose. Monitoring and detection are two more, among other things. frequently used purposes, and the purposes and uses continue to grow continuously.
    The use of sensors and vision technology in IoT raises concerns about data confidentiality in the public, as well as with the private consumer. Data confidentiality can be referred to here as confidentiality.
    Today, there are computer vision sensors, such as cameras with built-in processing, where all image processing is performed in the sensor. Such sensors are generally used where bandwidth capacity is limited. The sensors can, in addition to having low bandwidth requirements, provide confidentiality as there is no need to transmit the
    GB 2018 00090 U1 single images from a sensor for further processing. However, such sensors may require rather expensive processors to be able to perform complete analysis on the images at the required frame rate. And the requirements of the processor may also include other tasks, such as machine learning for object detection, object classification, and object recognition.
    A vision system that uses computer vision sensors with embedded processors can be manufactured as low bandwidth systems with integrated confidentiality, as no single images should be sent from each computer vision sensor to a main or cloud server. However, such systems can be costly due to the requirements of the processor.
    A different approach is to create a vision system where all image processing is performed on a main or cloud server and simple and inexpensive sensors with limited processor capacity can be used. However, the use of such systems may require a high bandwidth transmission from each sensor to the server as the individual images are sent directly to the server. Furthermore, in such systems, confidentiality is not guaranteed solely by the method and / or equipment used, as opposed to systems that use sensors with built-in processing.
    Purpose of production
    The object of the invention is to solve one or more of the aforementioned shortcomings in the prior art.
    Description of the production
    One of the objects of the invention can be achieved by a method of object detection comprising method steps performed in a sensor unit and a method step in a gateway processor. The process steps performed in the sensor unit may be to acquire an image from a camera and perform image preprocessing on the acquired image and to generate a pre-processed image. A further process step, performed in the sensor unit, may be the object detection performance using a computer vision detection algorithm. Yet another method of writing, performed in the sensor unit, may be to extract data feature on the pretreated image using a computer vision algorithm to extract data feature to generate a reduced data set
    DK 2018 00090 U1 comprising extracted data features. Another further step in the sensor unit may be to send a reduced data set to a gateway processor.
    The procedure step, performed in the gateway processor, may be to receive a reduced dataset on the gateway processor.
    Computer vision detection algorithm will be referred to as detection algorithm the following. Computer vision algorithm for extraction of data features will be referred to as vision algorithm.
    The method allows for distributed image processing based on extracted imagery, in which only a reduced data set is transmitted between the individual devices which may be the sensor unit and the gateway processor. This may be advantageous over obtaining a method which requires a reduced network bandwidth to transmit only the reduced data set.
    Extraction of data features can be performed using information from the object detection performed. The reduced data set may comprise only information from the extracted datasets or comprise a combination of the extracted datasets and the object detection performed. A data set, comprising information from the object detection performed, can be sent to the gateway processor as a separate data set.
    A further effect of the method may be that the method steps performed in the sensor unit extract only the most necessary or crucial data from the acquired image to be included in the reduced data set. The reduced data set can later include is used, for example, to determine object classification and / or recognition. However, the reduced data set to be transmitted between the sensor unit and the gateway processor cannot be used to reconstruct the original image acquired through the camera. A further advantage of the method may be to ensure data confidentiality due to the nature of the reduced data set.
    DK 2018 00090 U1
    The reduced data set may include sufficient information that can be used for motion and object detection, including body and face detection, face recognition, object classification, item count, etc., as this is a non-exhaustive list.
    A further object of the production can be achieved by the method wherein the method step, performed in the sensor unit, of performing image preprocessing on the acquired image comprises the process steps of obtaining one or more sub-images in a complete image, the complete image being the acquired image, and generating one or more pre-processed images of one or more sub-frames.
    This embodiment may be referred to as image splitting.
    This further embodiment may have a further effect of the object detection and extraction of data features being performed on a single image only, thus generating only a reduced data set of the images, thus obtaining a distribution of these tasks over time by sequentially using only a partial image of the complete image. image and by performing the object detection and data set reduction in the sub-image. This may provide a first processing of the frames, the first processing being a low computationally demanding processing, to obtain a reduced data set comprising only data which makes the reduced data set sufficient to be understood by a machine learning model - causing the transmitted data to be meaningless to humans.
    The confidentiality of the extraction of data features on the device can thus be maintained by performing a low computationally demanding processing of the frames.
    In a further embodiment, the method may comprise an additional method step performed in the gateway processor, which method step performs object recognition and / or object classification by loading the reduced data set into a machine learning model which executes a machine learning algorithm adapted to execute object recognition based on it.
    This embodiment can be referred to as machine learning.
    DK 2018 00090 U1
    The machine learning model can be pre-trained using multiple training images. The machine learning model can be trained continuously based on the continuously performed item recognition and / or item classification in the reduced dataset and / or through further use of training images.
    A further effect of this further embodiment is that the object recognition and / or object classification can be performed on a device other than the device which acquires the image, using only extracted data features, thereby obtaining object recognition and / or object classification while securing data confidentiality. the nature of the reduced data set. Hereby, object recognition and / or object classification in the reduced data set is performed solely with comprehensive data sufficient to be understood by the machine learning model, while said data is meaningless to humans.
    The object recognition can only be performed using extracted data features or in combination with the information obtained through the object detection performed.
    The item recognition can be performed on any suitable item that can be identified as separate items based on one or more data extracted features of the separate item within a type of item. The extracted datasets may be based on high contrast areas, areas with changing colors, sizes and / or patterns on the object. The extracted data features may be based on object features with high contrasts in the object's distinctiveness or with high contrast to the background.
    An example might be the recognition of faces where the extracted features are the contrasts found in different areas of the face or in the object features. The distinctive features, such as the nose, eyes or eyebrows, just to mention a few distinctive features in an incomplete list. The separate item may be Mr. X, and the type of objects may be human in general or a small group, e.g. employees of company Y.
    DK 2018 00090 U1
    Another example may be the recognition of separate license plates, where the distinctive features may be the numbers of the license plates, and the object type is the license plates of cars.
    Another example could be for the recognition of persons using body movement detection. The examples are for illustrative purposes only and exemplary purposes only, and the object recognition is not limited to the examples mentioned under any circumstances.
    In a further embodiment, the method may comprise additional method steps, performed in the sensor unit and / or in the gateway processor, of acquiring a pixel object height of a detected object in the reduced data set and comparing the pixel object height to one or more tabulated physical object heights and one or more tabulated camera parameters. , to estimate the distance between one or more detected objects and the camera, which is the object-camera distance.
    This embodiment can be referred to as distance estimation.
    One effect of this further embodiment may be to locate separate objects using a single image. This has the advantage that a 3D location of the objects can be achieved using a single camera. Furthermore, by combining object detection into multiple objects in an image, it may be possible to determine several properties in the separate objects. An example, which is non-limiting and purely illustrative, could be the detection of a human, where the detection of a face and the detection of a body can be compared in height to evaluate whether the person sits or stands. Furthermore, the use of the object-camera distance in a sequence of images over time may be advantageous in achieving motion tracking of the object.
    The present invention relates to various aspects. One aspect is the aforementioned process, other aspects may include entities, algorithms, systems and / or additional process steps of the process, each process step producing one or more of the effects and benefits that are
    DK 2018 00090 U1 described in connection with the aspect already mentioned, that is, the method and embodiments thereof described above. Each aspect may have embodiments similar to the embodiments described in connection with the second mentioned aspect.
    An object of the invention may be achieved by a sensor unit comprising means (s) adapted to perform the method steps of the method, performed in the sensor unit, comprising a camera adapted to obtain an image. In addition, the sensor unit comprises pre-processing means (s) adapted to perform image pre-processing, perform object detection, and perform extraction data extraction to generate a reduced data set comprising extracted data embossments. Further, the sensor unit comprises camera communication means (s) adapted to transmit a reduced data set.
    The camera in the sensor unit can be integrated into an embedded electronic device. This can be termed a 'system on a chip' (Soc). The embedded electronic device may include a CPU, GPU, FPGA, ASIC and similar types of systems. The system can be integrated with custom vision processors.
    One effect of this embodiment is that only a reduced data set can be sent from the sensor unit. The reduced data set may include only the most necessary or crucial data from the acquired image. As mentioned earlier, the embodiment of the sensor unit may be advantageous to obtain a built-in data confidentiality of the data set to be transmitted due to the nature of the reduced data set.
    Furthermore, the reduced data set is advantageous over reduced data transmission and thus lower bandwidth requirements on one or more sensor communication means.
    A further effect of this embodiment is that the object detection and data set reduction can be distributed over time by sequentially using only full-image sub-images. This can provide a first one low
    DK 2018 00090 U1 computationally demanding processing of the images to obtain the reduced data set.
    The sensor unit embodiment further allows a device to be used in a distributed network-based architecture, with the sensor unit acting as a separate node, where the image acquisition and the first distinctive extraction are performed. Furthermore, due to the nature of the reduced data set and the distance estimation embodiment, the sensor unit can be used to achieve 3D localization and motion detection.
    An object of the invention can be achieved by using a computer vision algorithm for extraction of data features comprising instructions for causing the sensor unit to perform the step of the process of performing data feature extraction in a pretreated image to generate a reduced data set comprising extracted data features.
    The image preprocessing can be done using standard computer vision filters such as HOG, LBP, DoG, PCA, SIFT, HCD, SURF, ORB or similar filters.
    The extraction of data features can be based on one or more methods using spatial filtering and / or one or more complex neural networks. The extraction of data features can be performed using one or more algorithms, such as Sobel, LBP, HAAR, HOG or similar algorithms.
    Machine learning can be implemented using SVM, KNN, rCNN or similar methods.
    The examples should not be construed as a limiting feature and should be understood solely as examples.
    One effect of this embodiment is that the datasets are extracted using a method for extracting datasets which must not be shared and / or sent with the reduced dataset. Since the method used to extract data features is unknown at the time of dispatch and / or on the receiving unit and / or
    DK 2018 00090 U1 processes the reduced data set for dispatch, it is impossible to reconstruct the original image.
    This means that data confidentiality can be ensured solely due to the nature of the reduced data set.
    In a further embodiment, the computer vision algorithm for extraction of data features can be further adapted to perform object tracking of one or more detected objects in one or more pre-processed images in one or more subsequent acquired images.
    One effect of this embodiment is that the object detection and / or extraction of data features to achieve a reduction of the data set can be distributed in time by using only a partial image of the complete image. In one sub-image, object recognition and / or extraction of data features can be performed to reduce the data set. In the following complete image, another sub-image can be analyzed. The following sub-images are thus not the currently analyzed sub-images. A tracking algorithm can be used to track the movement of detected features and objects in a frame until the specific frame is re-analyzed.
    An example could be that the complete image is divided into four sub-frames. The complete image can be defined at image edges, and the part images can be defined at the part edges. The four frames can overlap with their frames.
    For each complete image, only one subpage can be analyzed according to the method of obtaining the reduced data set, while the other three subpages are processed to a lesser extent. In the next complete frame, a new frame can be analyzed so that all four frames have been analyzed in a sequence of four frames. By the fifth complete image, the sequence begins again.
    In the case where a sub-image, e.g. screen 1, analyzed in accordance with the method of obtaining a reduced data set, and where one or more objects are detected, the detected objects can only be traced in the following three
    GB 2018 00090 U1 images when using tracer tracking. Thus, a smaller degree of processing is performed in the sub-image. In the event that no objects are detected, e.g. in image 1, no processing will be performed in the subsequent three images 1.
    One effect of this embodiment may be that through the use of a constant frame rate, detected objects are obtained to keep track of detected objects, even when in motion. One advantage may be a limited and constant need for processing power.
    An object of the invention can be achieved by using a computer vision detection algorithm comprising instructions for causing the sensor unit to perform the step of the method of performing object detection in a pretreated image by extracting object features from the pretreated image.
    One effect of this embodiment may be that the object detection can be performed independently of the extraction of data features. This may be advantageous over the specific use of the object detection. Information from the object detection performed can subsequently be used in the extraction of data imprints. Alternatively, the information from the object detection performed can simply be combined with the extracted data features of the reduced dataset. As yet another alternative, the data set comprising information from the object detection performed can be sent and / or used independently of the reduced data set comprising the extracted data features.
    An object of the invention can be achieved with a gateway processor comprising means (s) adapted to perform the process steps of the method performed in the gateway processor comprising gateway communication means (s).
    One effect of the gateway processor may be the possibility that a device can be used in a distributed network-based architecture, with the gateway processor acting as a separate server node. This server node can be a simple server node that distributes the data to other devices or systems, or the gateway processor may act as an intelligent server node which performs additional analyzes in the reduced data set.
    DK 2018 00090 U1
    An object of the invention can be achieved by using a machine learning algorithm comprising instructions for causing the gateway processor to perform the process step in the method of performing object recognition in a reduced data set comprising extracted datasets.
    One effect of this embodiment may be to use the gateway processor as an intelligent server node to perform additional analyzes in the received reduced dataset. It may be advantageous to achieve a more detailed object detection and / or recognition. Furthermore, this analysis can result in data that can be used for land use, which can be referred to as usage data.
    Machine learning can be implemented using SVM, KNN, rCNN or similar methods.
    An object of the invention can be achieved with an object detection vision system comprising a gateway processor and one or more sensor units with sensor communication means (s) adapted to transmit reduced data sets to the gateway processor. The vision system may be adapted to perform the process steps of the described method.
    The Vision system allows for a distributed network-based architecture, where the sensor unit acts as a separate node, where image acquisition and first feature extraction is performed, and where the gateway processor acts as a server node where further analysis can be performed, or a server node, where the received dataset can be further distributed for further analysis. One effect is thus to obtain a system that extracts crucial data sets from raw image data and can be used to determine object classification and recognition, but where the transmitted data set cannot be used to reconstruct the original image obtained from the camera.
    In accordance with the aforementioned embodiments of the method, the information from the method for extracting data features is not shared between the individual units covered by the system, thus making it impossible to
    DK 2018 00090 U1 reconstruction of the original image, since the method used for extracting data features is not outside the sensor unit.
    The benefits of such a vision system can be a system with a distributed network-based architecture with a reduced network bandwidth with built-in data confidentiality, due to the nature of the reduced data set.
    The additional effects of the vision system may include one or more of the aforementioned advantages of the method, devices, and / or algorithms, such as an initially reduced processing power, which may be advantageous over using standard sensor units in the system. A further effect may be a vision system that uses open source platforms for the algorithms and thereby achieves the building of a distributed network-based architecture using common and freely available programming platforms.
    The Vision system's network-based architecture may be based on LAN, WLAN, Bluetooth or similar types of connections.
    ELEMENT
    In a further effect, the method of using the vision system, comprising two or more sensor units, may comprise additional process steps performed in the gateway processor of monitoring the operating status of the sensor unit and distributing data from a first sensor unit to at least a second sensor unit. The first sensor unit performs at least the steps of:
    - acquire an image from that camera and
    - performing image preprocessing on the acquired image and generating a preprocessed image,
    The second sensor unit performs one or more of the process steps of:
    - performing object detection in the pretreatment image using a computer vision detection algorithm,
    performing extraction of data imprints in the pretreated image using a computer vision algorithm for extraction of data imprints to generate a reduced data set comprising extracted data imprints, or
    - send the reduced data set to a gateway processor.
    DK 2018 00090 U1
    This element can be referred to as distributed edge computing.
    One effect of this approach may be to obtain an intelligent system of connected sensor units which can be used to distribute the computational load by performing the extraction of distinctive and other non-time sensitive tasks. The gateway processor keeps track of which sensor units are active and which sensor units are in hibernation, and uses the processing power of the separate sensor unit by requesting the active sensor units to distribute data for processing in the sensor units that are in hibernation, thereby distributing the computational load on the active sensor units. This can be advantageous in terms of utilizing the distributed processing power of the system's image processing sensor units. However, using this approach can reduce the level of data confidentiality of the data transmitted between the system's sensor units.
    Distributed edge computing is an embodiment that can reduce the level of confidentiality in the use of the vision system. If this embodiment does not allow for a sufficient level of confidentiality, in this embodiment, an improved level of confidentiality should be provided otherwise.
    In a further embodiment, the method, using the vision system comprising two or more sensor units, may comprise additional process steps of:
    - estimating a first object-camera distance to a detected object in a first pretreated image,
    estimating a second object-camera distance to a detected object in a second pretreated image, wherein the first pretreated image captures a first scene, and the second pretreated image captures a second scene which overlaps or partially overlaps the first scene, and
    - using a first and second object-camera spacing to validate that the detected object in the first and second preprocessed images is the same object.
    DK 2018 00090 U1
    This element can be referred to as duplication.
    This embodiment may have the effect of alleviating the appearance of duplicates of objects when the reduced data sets are further analyzed after being sent from the sensor units. This is advantageous in relation to an increased system quality, and thus an increased system reliability.
    The use of multiple sensor units can provide a vision system that covers a large area. This can be achieved because the separate scenes captured by each sensor unit can only overlap in small areas, thus covering a large area. For this reason, an increased number of sensor units may thus have the advantage of increasing the covered area.
    Alternatively, the use of multiple sensor units may allow for a vision system that depicts a scene from multiple directions and / or angles. One can thus achieve the image of one or more objects from multiple angles and / or with an increased level of detail. For this reason, an increased number of sensor units may have the advantage of increasing the level of detail of improved object detection.
    In general, use of multiple cameras can have the advantage that the vision system achieves increased robustness. Increasing the number of sensor units can thus further increase the robustness.
    In a further embodiment, the method may comprise further process steps of:
    - extracting object data from one or more reduced datasets loaded into a machine learning model which performs a machine learning algorithm to perform object detection and / or object recognition in the reduced dataset, and
    - send object data to a management server.
    This element can be referred to as usage data.
    This embodiment may have the effect of retrieving the recovered data from the use of the depicted locations. That kind of data can be advantageous compared to
    GB 2018 00090 U1 land utilization, monitoring of activities - which can be used to optimize the use of land - or adaptations of the related activities.
    For example, the embodiment may be used for land use of an office building, where the usage data may be the number of cars in the parking lot or the number of people using the meeting facilities and offices. The usage data can be used to adjust the need for cleaning, the need for heating or cooling of the offices or meeting facilities. Area utilization can also be used to rent out parking facilities, if there is generally a surplus at certain times, or to lend meeting facilities for external use.
    In a further embodiment, the vision system may comprise means adapted to carry out the further process steps of the method, described in the elements (edge computing, duplication and usage data), and further comprising a management server configured with a management system adapted to receive object data from the gateway processor.
    This embodiment has the effect that the vision system can be expanded to include the functions of edge computing, duplication, and obtain usage data by using only an additional management server and custom algorithms. This embodiment of the vision system may have the additional effects and benefits already described above in connection with the features of edge computing, duplicating, and obtaining usage data.
    In a further embodiment of the vision system, one or more sensor units may be adapted to be operated in a fixed position.
    One effect of this embodiment is that the vision system should not use any moving parts. Moving parts are often subject to higher wear and tear and thus require regular maintenance. This can advantageously be avoided by using sensor units in fixed positions.
    Description of figures
    Figure 1 shows an embodiment of the object detection method.
    DK 2018 00090 U1
    Figure 2 shows another embodiment of the object detection method.
    Figure 3 shows one embodiment of the process step of the image pre-processing.
    Figure 4 shows an embodiment of object tracking.
    Figure 5 shows an embodiment of the vision system.
    Detailed description of the production
    No. Reference 10 vision system 20 sensor unit 22 sensor communication means 24 camera 26 preprocessing means 28 sensor device parameter 30 gateway processor 32 gateway communication agency 40 administration server 42 object data 60 acquired image 62 complete picture 64 partial view 66 scene 67 first scene 68 second scene 70 pre-processed image 71 first pre-processed image 72 second pre-processed image 80 reduced data set 90 detected object 92 pixel object height 94 physical object height 96 object-camera distance 97 first object-camera distance
    DK 2018 00090 U1
    98 other object-camera distance 100 course of action 110 get 112 perform 114 send 116 receive 118 achieve 120 generate 122 load 124 compare 126 estimate 130 pretreat 140 genstandsdetektion 142 object character 150 object recognition 160 subject tracking 170 object management 180 subject classification 190 extraction of data features 192 extracted data features 194 detected object data 210 computer vision-detection algorithm 220 computer vision algorithm for extraction of data features 230 spatial filters 232 complex neural network 240 machine learning algorithm 242 machine learning model 270 management system 300 vision system 322 first sensor unit 324 other sensor unit
    DK 2018 00090 U1
    Figure 1 shows an embodiment of the method for 100 object detection 140. The method 100 comprises a number of process steps. In the context of some of the steps, intermediates appear. Process 100 is shown by a dotted line that encloses the process steps of the process. The process steps of the process are also shown in dotted lines. The intermediates are shown in full lines, as are the units in which the process steps are performed. The units include a sensor unit 20 comprising a camera 24 and a gateway processor 30 comprising gateway communication means (s) 32. The camera 24 provides 110 an image 60. A method step of the method of performing 112 image preprocessing 130 is performed in the acquired image 60 and thus is obtained. a pre-processed image 70. The image pre-processing is performed using pre-processing means (s) 26.
    The pre-processed image 70 is used to perform 112 object detection 140. The object detection 140 is performed using a computer vision detection algorithm 210. In another method step of the method of performing 112 extraction of data feature 190, a reduced data set 80 is generated. The extraction of data embossing 190 is performed using a computer vision algorithm to extract data embossing 220. The pretreated image 70, information from the performed object detection 140 and the detected object data 194 is used in the computer vision algorithm to extract data embossing 220 to generating a reduced data set 80 which includes extracted data features 192. The reduced data set 80 is sent 114 from the sensor unit 20 to the gateway processor 30 using one or more sensor communication means 22. In the gateway processor 30, the reduced data set 80 is received using one or more gateway processor means 32.
    Figure 1 further illustrates an embodiment in which gateway processor 30 is configured with a machine learning model 242 configured to execute a machine learning algorithm 240 comprising instructions for causing gateway processor 30 to perform the method step of the process of performing object recognition. The reduced data set 80, extracted data specification 192, and the detected object data 194 are input 122 into machine learning model 242.
    DK 2018 00090 U1
    Figure 2 shows another embodiment of the method 100 for object detection 140. Aspects of the previous figure 1 may also concern the details described in this embodiment. The difference between the two embodiments relates to the use of the detected object data 194. In this embodiment, the detected object data 194 is not loaded into the computer vision algorithm for extracting data features 220 and is not included in the reduced data set 80. Instead, the detected object data is 194. sent 114 to gateway processor 30 for further processing or analysis. In gateway processor 30, reduced data set 80 and detected object data 194 are received using one or more gateway processor means 32.
    Figure 2 further illustrates an embodiment in which gateway processor 30 is configured with a machine learning model 242 configured to execute a machine learning algorithm 240 comprising instructions for causing gateway processor 30 to execute the method step of the object recognition method. The reduced data set 80, the extracted data character 192 and the detected object data 194 are loaded 122 into the machine learning model 242.
    An embodiment of the process step of the image preprocessing process 130 is shown in Figure 3. The process step of the process is performed 112 in the acquired image 60. The process step of the process is shown in dotted lines. The intermediates are shown in full lines. In the preprocessing 130, the acquired image 60 is received as a complete image 62. One or more sub-images 64 is obtained 118 within the complete image 62. The complete image 62 is thus divided into a number of sub-images 64. In this embodiment, the complete image 62 is divided in four frames 64. The frames can be defined by frame edges. The frames can be generated so that the edges of the different frames overlap. One or more of the sub-images 64 can be further pre-processed to generate 120 a pre-processed image 70.
    An embodiment of object tracking is shown in Figure 4. The object tracking can also be referred to as distinctive tracking, as the object tracking can be performed by tracking object features 142. In this embodiment, the detected object 90 to be tracked is a face. Figure 4A shows an acquired image 60 in which three faces are shown.
    DK 2018 00090 U1
    Figure 4B shows an acquired image 60 or a complete image 62 comprising a frame 64. The frame may be one of several frames included in the complete frame. The sub-image 64 is pretreated so that a pre-processed image 70 is obtained and wherein the detected object 90 is a face. The face can be detected as a face or as a collection of distinctive features, such as eyes, nose, mouth, etc. As far as object tracking is concerned, object specification 142 may be used. The object features of the illustrated embodiment are shown with X's and are selected here as the corners of the mouth, two points in the forehead and on the cheeks. Using object characteristics, rather than the face, as the objects to be traced has the effect that when the face is turned away, e.g. 90 degrees, some of the object features are still visible in the image, whereas the face to be detected is no longer fully visible. This can be advantageous over improved object detection, even when the object is turned away or is partially hidden by another object which partially obscures the object to be tracked.
    The object tracing can thus be performed by tracing object features 142. The object tracing can be performed by performing only a minor degree of analysis in subsequent sub-images, where it is only the object features that are traced, and the sub-image is not analyzed for new objects. In the following complete images, other sub-images can be analyzed successively.
    Utilizing the item features to track can help in further application of the approach and vision system. The distinctive features can reveal a person's mood by estimating the distance from the eyes to the corners of the mouth, a change in the size of the eye, the change in the position of the shoulders, to name a few useful features.
    An embodiment of the use of vision system 300 is shown in Figure 5. Seven sensor units 20 are located in a room and depict different scenes. The embodiment shown is a meeting which takes place in the room where the seven persons x1-x7 participate. The seven participants are placed around a table.
    This embodiment illustrates the use of multiple sensor units. The figure shows how one or more persons can be imaged by several sensor units, each imaged
    GB 2018 00090 U1 a scene that is different from the scenes of the other sensor units. Person x4 is shown to be depicted by five sensor units. In the case where x4 is positioned in the direction towards the table, he is depicted from behind, from the side, frontal and partially frontal. This embodiment may show the element of the description described as remediation of duplicates.
    This embodiment may have the effect of alleviating the appearance of duplicates of objects when the reduced data sets are further analyzed after being sent from the sensor units, thereby increasing the quality and robustness of the vision system 300.
    The embodiment of Figure 5 further shows a vision system comprising a gateway server 30 and a management server 40, wherein the sensor unit 80 sends reduced data sets 80 to the gateway server 30 and the object data 42 is sent from the gateway server 30 to the management server 40.
    Figure 5 further illustrates an embodiment in which gateway processor 30 is configured with a machine learning model 242 configured to execute a machine learning algorithm 240 comprising instructions for causing gateway processor 30 to perform the process step of performing object recognition.
    权利要求:
    Claims (6)
    [1]
    REQUIREMENTS
    A sensor unit (20) comprising:
    - a camera (24) for obtaining an image (60),
    preprocessing means (26) for performing image preprocessing (130), performing object detection (140), and extracting data embossing (190) to generate a reduced data set (80) comprising extracted data embossing (192),
    sensor communication means (22) for transmitting a reduced data set (80),
    - means adapted to obtain an image (60) from said camera (24),
    means adapted to perform image pre-processing (130) on the acquired image (60) and to generate a pre-processed image (70),
    means adapted to perform object detection (140) in said pretreatment image (70),
    means adapted to transmit the reduced data set (80) to a gateway processor (30),
    - means adapted for performing extraction data (190) on detected objects (90) in said pretreated image (70)
    [2]
    A sensor unit (20) according to claim 1 which is new in that the means (s) adapted to perform (112) image preprocessing (130) on the acquired image (60) are further adapted to obtain (118) a or more sub-frames (64) in a complete image (62), wherein the complete image (62) is the acquired image (60) and to generate (120) one or more pre-processed images (70) of one or more sub-images (64).
    [3]
    A sensor unit (20) according to any one of claims 1 or 2 which is new in comprising means further adapted to provide (110) a pixel object height (92) of a detected object (90) in the reduced data set ( 80), and to compare (124) the pixel height (92) with one or more tabulated physical object heights (94) and one or more tabulated camera parameters (28), to estimate (126) the distance between the one or more localized objects (90) ) and the camera (24), which distance is the object-camera distance (96).
    [4]
    A sensor unit (20) according to any one of claims 1 to 3 which is new in that it is adapted for use in a fixed position.
    DK 2018 00090 U1
    [5]
    A sensor unit (20) according to any one of claims 1-4 which is new in that the means adapted to carry out object detection (140) in the pretreatment image (70) is adapted to perform the object detection (140) using a computer vision detection algorithm.
    [6]
    A sensor unit (20) according to any one of claims 1-5 which is new in that the means adapted to transmit the reduced data set (80) to a gateway processor (30) is adapted to use a computer vision algorithm for extracting data features (220) to generate a reduced data set (80) comprising extracted data features (192).
    类似技术:
    公开号 | 公开日 | 专利标题
    Ahmad et al.2013|Image-based face detection and recognition:" state of the art"
    US9750420B1|2017-09-05|Facial feature selection for heart rate detection
    Araujo et al.2013|Towards skeleton biometric identification using the microsoft kinect sensor
    Barnich et al.2009|Frontal-view gait recognition by intra-and inter-frame rectangle size distribution
    US20160328621A1|2016-11-10|Facial spoofing detection in image based biometrics
    CN101999900A|2011-04-06|Living body detecting method and system applied to human face recognition
    Piatkowska et al.2013|Asynchronous stereo vision for event-driven dynamic stereo sensor using an adaptive cooperative approach
    Le et al.2017|UHDB31: A dataset for better understanding face recognition across pose and illumination variation
    CN109359514B|2020-08-04|DeskVR-oriented gesture tracking and recognition combined strategy method
    JP2017059945A|2017-03-23|Device and method for image analysis
    KR20130129529A|2013-11-29|Server for analysing video
    Liu et al.2011|Human gait recognition for multiple views
    Özbudak et al.2010|Effects of the facial and racial features on gender classification
    DK201800090Y4|2019-08-01|Vision sensor system for detecting, classifying and tracking objects
    Choi et al.2013|Comparing strategies for 3D face recognition from a 3D sensor
    Arai et al.2012|Gait recognition method based on wavelet transformation and its evaluation with chinese academy of sciences | gait database as a human gait recognition dataset
    Ferrara et al.2014|On the use of the Kinect sensor for human identification in smart environments
    Sulong et al.2014|HUMAN ACTIVITIES RECOGNITION VIA FEATURES EXTRACTION FROM SKELETON.
    Alashkar et al.2014|A 3D dynamic database for unconstrained face recognition
    Dharejo et al.2017|PCA Based Improved Face Recognition System.
    Cheng et al.2020|DTFA-Net: dynamic and texture features fusion attention network for face antispoofing
    Jelsovka et al.2012|2d-3d face recognition using shapes of facial curves based on modified cca method
    Layher et al.2011|Robust stereoscopic head pose estimation in human-computer interaction and a unified evaluation framework
    KR20170040692A|2017-04-13|Method for extracting information of facial movement based on Action Unit
    WO2019114901A1|2019-06-20|Vision system for object detection, recognition, classification and tracking and the method thereof
    同族专利:
    公开号 | 公开日
    DK201800090Y4|2019-08-01|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    法律状态:
    2019-07-15| UAT| Utility model published|Effective date: 20190313 |
    2019-08-01| UME| Utility model registered|Effective date: 20190801 |
    优先权:
    申请号 | 申请日 | 专利标题
    DKBA201800090U|DK201800090Y4|2017-12-13|2017-12-13|Vision sensor system for detecting, classifying and tracking objects|DKBA201800090U| DK201800090Y4|2017-12-13|2017-12-13|Vision sensor system for detecting, classifying and tracking objects|
    [返回顶部]