• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Embodiments of the invention provide a system for integrating message content into a target processing device (2), said target data processing device (2) being configured to process an input data having a data structure. predefined data, the system comprising a mail server (11) configured to receive a message from a mail client device (14) that executes a mail application (15), the message including the message content. wherein the system further comprises an orchestrator device (18) configured to integrate at least a portion of the message content into a target data processing device (2), the orchestrator device (18) being otherwise configured to: - receive the portion of the message content of the mail server (1); and - transmitting a determined file of the message content portion to a file processing device (19), the file processing device being configured to transform each received file (190) into a description file (191) comprising a set of predefined keys, at least some of the keys being associated with one or more values, the orchestrator device (18) being configured to determine an input file (190) having the predefined data structure from the description file (191) ) and for transmitting the determined input file (182) to the target data processing device (2) for processing the input file determined by the target processing device.
    公开号:FR3069075A1
    申请号:FR1756717
    申请日:2017-07-13
    公开日:2019-01-18
    发明作者:Eduardo Rafael Lopez Ruiz；Nicolas Guillon；Paul Krion；Jurgen Oesterle；Martin Stammler；Martin Kuhn；Sebastian Bildner；Thomas Stark
    申请人:Amadeus SAS；
    IPC主号:
    专利说明:

    SYSTEM AND METHOD FOR INTEGRATING CONTENT
    MESSAGE IN A TARGET DATA PROCESSING DEVICE
    CONTEXT
    The invention relates generally to messaging systems, and in particular to methods, systems, and computer program products for integrating message content into a target data processing device.
    In conventional messaging systems, a user may need to enter unstructured data about a common item, such as digital or electronic invoices for fees, to a target data processor relating to a dedicated application area , such as an expense reporting system; the target data processing device then processes the data entered by the user and if the entered data and the receipts are in conformity, the expense report system generates an expense report for one or more receipts supplied by a user for processing .
    In some cases, the user can connect directly to the target data processing device via a dedicated interface to manually fill in a form for each operation or transaction (eg a business trip), in which he must enter the information contained in receipts or invoices.
    This type of interaction is tedious, time-consuming and a source of errors (typographical errors, etc.) for the user. In addition, they can generate erroneous information which forces the user to check the form or even the data entered again. In addition, when these interactions are manual, they are also limited in terms of the quantity or wealth of information collected.
    Therefore, there is a need for improved systems, methods, and computer program products for integrating content into a target data processing device.
    ABSTRACT
    In order to address these and other problems, a system for integrating message content into a target processing device is provided, the target data processing device being configured to process input data having a predefined data structure , the system comprising a messaging server configured to receive a message from a messaging client device that runs a messaging application, the message comprising the content of the message. The system also comprises an orchestrator device configured to integrate at least part of the content of the message into a target data processing device, the orchestrator device being further configured to:
    - receive part of the message content from the messaging server; and
    transmitting a determined file from the message content part to a file processing device, the file processing device being configured to transform each file received into a description file comprising a set of predefined keys, at least some keys being associated with one or more values, the orchestrator device being configured to determine an input file having the predefined data structure from the description file and to transmit the determined input file to the target data processing device .
    In one embodiment, the orchestrator device is connected to the messaging server according to a first communication protocol, and / or to the messaging application according to a second protocol, and / or to the target data processing device according to a third protocol Communication.
    The messaging application may include an application interface and an application extension configured to generate an input file rendering determined from the description file provided by the file processing device in a dedicated area of the application interface.
    The file processing device can be configured to map one or more data elements of said part of the message content at least in the form of keys from a predefined set of keys, the file processing device being configured to generate the file for describing the part of the message content comprising said set of predefined keys, said one or more values associated with the keys of the description file being determined from the data elements which are mapped there.
    The file processing device can also be configured to determine sets of positioning data from the file received from the orchestrator device, each set of positioning data identifying the position of a data element of the file mapping a key of the file. 'predefined set of keys, each set of positioning data being included in the description file associated with the key mapped to the data element.
    Each set of positioning data can include the positioning coordinates in a given frame of reference.
    In one embodiment, the application interface can be a graphical application interface, the application extension being configured to render the description file in a dedicated area of the application interface.
    The file processing device can also be configured to determine a notation for each value associated with a given key from the predefined set of keys and to include the notation determined for the value associated with the key given in the description file.
    The application extension can also be configured to display an image of the part of the message content in the dedicated zone and to generate the display of one or more selectable highlighting elements for each data element of the content part of the message by mapping a key, each highlight element for a given data element being displayed on a position in the displayed image which is determined from all the positioning data identifying the position of the data item.
    The orchestrating device can be configured to use a message identifier associated with the message for each exchange between the orchestrating device and the messaging server, and / or the application extension, and / or the file processing device and / or the target device.
    Furthermore, a method is provided for integrating message content into a target processing device, the target data processing device being configured to process an input data having a predefined data structure, the method comprising receiving a message a messaging client device that runs a messaging application, the message including the content of the message. The method may include integrating at least part of the message content into said target data processing device, the method being further configured to determine a determined file from the message content part and to transform the file into a description file comprising a set of predefined keys, at least some of the keys being associated with one or more values, the method comprising deriving an input file having the predefined data structure from said description file and transmitting said determined input file to the target data processor for processing the input file determined by the target processor.
    A computer program product is also provided, which includes:
    a non-transient computer-readable storage medium; and instructions stored on the non-transient, computer-readable storage medium which, when executed by a processor, causes the processor to integrate message content into a target data processor, the data processor target being configured to process the input data having a predefined data structure, the processor being also led to:
    - receive a message from a messaging client device that is running a messaging application, the message including message content;
    - integrate at least part of the message content into the target data processing device, the processor being also required to determine a determined file from the message content part and to transform the file into a description file comprising a set of predefined keys, at least some of the keys being associated with one or more values,
    - determine an input file having the predefined data structure from the description file and transmit the determined file to the target data processing device for processing the input file determined by the target processing device.
    BRIEF DESCRIPTION OF THE DRAWINGS
    The drawings, which form an integral part of this specification, illustrate various embodiments of the invention and, with the general description of the invention above and the detailed description of the embodiments given below, serve to explain the embodiments of the invention.
    - Figure 1 is a schematic view of an example of an operating environment including a system for integrating at least one file in a target data processing device;
    - Figure 2 illustrates an example of application interface according to an embodiment;
    - Figure 3 schematically illustrates an example of an electronic message structure;
    - Figure 4 is a schematic view of a file processing device, according to certain embodiments;
    - Figure 5 is a schematic view of a data extraction engine of the file processing device, according to certain embodiments;
    - Figure 6 shows an example of a view of the application interface, according to certain embodiments;
    - Figure 7 is a flowchart describing the method of integrating at least part of message content in a target data processing device, according to certain embodiments;
    - Figure 8 is a flowchart describing the initialization process implemented by the application extension according to certain embodiments;
    - Figure 9 is a flowchart describing the process of integrating an attached file into a target processing device, according to certain embodiments;
    - Figure 10 is a flowchart describing the process implemented by the file processing device, according to certain embodiments;
    - Figure 11 is a view of an example description file in Json format;
    - Figure 12 is a flowchart describing the process of refining a description file in certain embodiments; and
    - Figure 13 is a diagram of a computer device or system.
    DETAILED DESCRIPTION
    Figure 1 is a schematic view of an exemplary operating environment including a system 100 for integrating at least one file into a target data processing device 2. The system 100 may include a mail server 11 configured to receive an electronic message 12 (such as an "email", an instant message also known as a chat or online chat stream) from a messaging client device 14 that runs a messaging application 15. The electronic message may include message content in the form of message data included in the body of the message, and / or one or more files attached to the message (also known as "attachments" or "attachments"). The content of the message can also relate to one or more operations (also known as "transactions"). The application can for example be Microsoft Outlook and the messaging server 11 can for example be Microsoft Exchange Server (Microsoft Outlook and Microsoft Exchange Server are registered trademarks of the company Microsoft).
    The target data processing device 2 can be configured to process an input data file having a predefined data structure and relating to target operations, in accordance with a dedicated process.
    In an example of application of the invention, the target data processing device 2 can be an expense processing device or a tool (also designated as expense reporting device) used by a company or an entity to process the expense reports incurred by employees of the company or entity, or by an individual to manage their personal expenses. In a business context, such a cost processing device 2 can be configured to receive a description file which identifies a cost transaction (eg travel costs) in order to generate a cost report making it possible to reimburse the user if the data entered relating to costs is in accordance with predefined rules. All Each of the costs that can be associated with a statement of costs representing all the costs incurred by a company or on behalf of a company for a given transaction. The expense processing device 2 can be located inside (e.g. a desktop software application preinstalled on the user's device) or outside of each client device 14, and / or distributed between multiple computers (e.g. as a server-client software application, such as a web application). In such an application example, the attached files may include attachments corresponding to receipts relating to one or more costs, the attachments being for example a photo or a scanned copy of a receipt, an electronic invoice in PDF format sent by a hotel, restaurant or taxi company that matches the service billed in the receipt.
    The messaging server 11 can be configured to receive electronic messages, buffer with received messages and send messages to a destination device.
    Each client device 14 (also referred to as a "user device") can be a personal computing device, a tablet computer, a thin client terminal, a smartphone and / or another computing device. Each client 14 can host web browsers and / or custom application software (eg, a client system), and can include a client user interface.
    Generally, the client device 14 may be any suitable computer system configured to execute the messaging application 15 associated with the application interface 150 via which a user can send or receive electronic messages.
    Each electronic message 12 can be associated with a unique message identifier which uniquely identifies the message.
    The messaging application 15 can include an interface manager 151 for transmitting the application data in the application interface 150.
    The mail server 12 can be configured to store each electronic message in a database with the attached files, if the content of the message includes attachments.
    The term "message attachment" as used herein (also referred to herein as "email attachment", "attachment" or simply "attachment") refers to a file electronic or to a document included in an email, the electronic file being represented by a clickable representation element, such as a thumbnail associated with a file name. Each attached file 121 has a file format, for example a jpeg, gif, pdf, Word, Html format. A file format can be "structured" or "unstructured". Message attachments can include photos or images in different formats. An attachment to a message may otherwise include a file that is in the body of the message, such as an image in an email or other reproduction of the file.
    The system 100 may include an orchestrator device 18 configured to integrate the content of the selected message received by the messaging application 15 into the target data processing device 2. The content of the message can be any content received in any type of messages taken supported by the application such as email content, instant message, feed. Using the example of an email, the content can be included in the body of the email or in an attached file.
    To facilitate understanding of certain embodiments of the invention, the following description will be made with reference to the integration of attached files into a target data processing device, but those skilled in the art will readily understand that in general the The invention applies to message content received by a messaging application 15. However, the terms “attachment” or “attached file” should be interpreted as including any message content included.
    The attached files can be selected or filtered by the orchestrator device 18 according to several criteria relating to the file, such as the name of the file, the extension of the file, the size of the file.
    The messaging application 15 may include an executable application extension 152 (an application extension may also be referred to as a "plug-in", "add-in" or "an extension software component") configured for add a dynamic data integration characteristic 15. In particular, the application extension 152 can be configured to manage a dedicated area of the application interface 150 and to interact with the orchestrator device 18. The extension Application 152 can also be configured to generate a restitution of data received from the orchestrator device 18.
    The interface application 150 and in particular the dedicated area of the application interface managed by the application extension 152 may include different types of graphic elements, such as windows, text entry fields, icons, selected elements, graphic control elements such as drop-down menus or list boxes, activation buttons, etc.
    The system 100 may further include a file processing device 19 (also referred to as a "file processing device") configured to convert or transform an input file 190 in an unstructured format into a description file 191 having the predefined data structure supported by the target data processing device 2 and comprising a predefined set of keys, at least some of these keys being associated with one or more values.
    In an application of the invention for reporting or expense management, all of the keys can include keys which are common to all receipts (also referred to as "mandatory keys") such as:
    - the type of receipt;
    - the date of the receipt;
    - the amount of the receipt;
    - the seller's identification (name of the invoicing company such as the name of a hotel, the name of a taxi company, etc.).
    The set of keys can also include keys which depend on a receipt subtype (e.g. a taxi receipt, a hotel receipt, a restaurant receipt) such as:
    - the route of a taxi (origin / destination) for a taxi receipt;
    - the number of nights for a hotel receipt;
    - additional services for a hotel: breakfast, dinner, etc.
    The orchestrator device 18 can be configured to:
    - receive the selected files stored by the messaging server 11 or coming directly from the messaging client device 14 (the user can for example forward an email comprising a receipt attached to a component of the orchestrator device); and
    - transmit a set of files determined from at least certain files among the files selected to the file processing device 19.
    The orchestrator device 18 can be configured to transmit a description file 182 determined from the description file 191 to the target data processing device 2. In a preferred embodiment, the description file 191 can be previously refined by the device orchestrator 18 in response to input received from the user via the dedicated area of the application interface managed by the application extension 152.
    In particular, the application extension 152 can be configured to generate a rendering of the description file determined from the description file 191 supplied by the file processing device using the interface manager 151.
    The target data processing device 2 can then process the description file 182, received from the orchestrator device 18 and initiate a dedicated process depending on the field of application of the target data processing device.
    In certain embodiments, each file 190 entered in the file processing device 19 by the orchestrator device 18 corresponding to an original attached file 121, can be associated with a user identifier and / or with a context (for example example, travel information, company information, location information for a travel receipt, in the example of implementing an expense report of the invention).
    In an application of the invention for the generation of expense reports or expense management, the system 100 according to embodiments of the invention allows reliable acquisition of receipts relating to expenses without the user being obliged to enter data manually in a form via a direct interface with the expense processing system 2. The expense processing system can therefore acquire, verify and reconcile each receipt with a user account allowing the user to be reimbursed who made the expense. Simplifying, by increasing the reliability and the rate of acquisition of expense data by the expense processing system 2, can improve the reimbursement time for the user.
    Furthermore, according to embodiments of the invention, the client device 14 can integrate different costs relating to one or more expense statements in the expense processing device 2 without the user needing to interface directly with the expense processing device or fill in a dedicated form in the interface of the expense processing device.
    In certain embodiments, the orchestrator device 18 can be connected to the messaging server 11 according to a first communication protocol, to the messaging application 150 according to a second protocol, to the target file processing device 19 according to a third communication protocol. communication and to the target data processing device 2 according to a fourth communication protocol
    In one embodiment, the first, second, third and fourth communication protocols can be the same. Otherwise, at least part of the first, second, third and fourth communication protocol can be different.
    The messaging application extension 152 can be launched automatically when the user launches the messaging application 15. Otherwise, the messaging application extension 152 can be launched dynamically in response to one or more relative conditions. attached files in an email, such as if an email includes at least one attachment, or if the email includes at least one attachment with specific file names, or specific file extensions, or conditions relative to the sender, in relation to the formulation of the subject of an email, or other conditions which can be determined by automatic learning with other emails received previously.
    In another embodiment, the application extension 152 can be launched statically in response to the activation of a dedicated activation element, such as a button, by a click of the user. Such an activation element can be displayed in the application interface 15, for example directly in the body of the email, or in the toolbar.
    Enabling messaging app extension 152 can trigger a selection of a subset of attached files based on one or more conditions, such as conditions regarding the format of the attachment. Activation of the messaging application extension 152 can also cause a thumbnail of each selected file to be displayed in a dedicated area of the application interface.
    In another embodiment, the user can directly select the attached files by a drag-and-drop operation to bring them to the dedicated area by clicking on a selection button associated with each attached file. The application extension can then generate a thumbnail that can be activated by a click for the attachments selected in the dedicated area. In some embodiments, the application extension can also display an entire view of each attached file (eg a receipt) being integrated into the target data processing device 2 in the form of a picture. In some embodiments, the displayed attached file that was initially displayed visually in its entirety can be enlarged or decreased with the zoom by the user in order to improve the view of the receipt for the user.
    In certain embodiments, the file processing device 19 can be part of the orchestrator device 18. However, the following description of certain embodiments of the invention will be made with reference to a separate file processing device 19.
    The messaging server 11 can communicate with one or more client devices 14 via a communication network 60.
    The orchestrator device 18 can be hosted in the same computer system as the file processing device 19 and / or the target data processing device 2, and / or the messaging server 11. Otherwise, the orchestrator device 18, the device file processing unit 19 and / or the target data processing device 2, and / or the messaging server 11 can be hosted in different computer systems and communicate via one or more communication networks.
    Each communication network used to enable communication between two devices of the system 100 may include one or more private and / or public networks (e.g. the Internet) which allow the exchange of data, e.g. from the Internet, a local area network (LAN), a wide area network (WAN), a cellular voice / data network, one or more high speed bus connections and / or other types of networks Communication. Each communication network such as network 100 can use standard communication technologies and / or protocols such as 4 G, Ethernet, 802.11, TCP / IP (Transmission control protocol / Internet protocol), HTTP (Hypertext transport protocol ), FTP (File Transfer Protocol), etc. Data can be exchanged on each network using different data exchange technologies and / or formats such as hypertext markup language (HTML), the JSON model and extended markup language (XML).
    In a conventional operation:
    - in transmission mode, the messaging client device 11 can ask the messaging server 11 to transmit a message 12 to one or more recipients identified in the recipient designation elements 122 corresponding to the recipient client devices, within the same network or another accessible network.
    - in reception mode, the client device 14 can receive a message 12 from another client device directly in "push" mode or indirectly via the reception of a notification informing the server 12 of the reception of a new message, the client device then being configured to "pull" the message from the server 11.
    A security token can be used to retrieve files and validate communication / exchanges.
    Referring to Figure 2, an exemplary application interface 150 is shown with the target processing device 2 envisioned in an expense reporting device, in accordance with an exemplary embodiment. According to this exemplary embodiment, the activation of the messaging application extension 152 can trigger a display in a dedicated interface zone 5 located in a part of the application interface 150. The zone of interface 150 may include the message inbox in a portion 1500 of the interface area including the current email which includes attachments. In the exemplary embodiment illustrated in FIG. 2, the dedicated area 5 can be divided into three parts comprising for example:
    - a first part 50 ("image view part"),
    - a second part 51 (“view of the thumbnails”), and
    - a third part 52 (“view part of the verification form”).
    The first part 50 can be provided to display an image view of each attachment which can be an entire view and which can be moved, enlarged or decreased by the user, during the processing of this attachment by the device orchestrator 18. Part two 51 may include a thumbnail of the relevant attachments (email images). The third part 52 can be provided to display a verification form determined from the description file 191 returned by the file processing device 19. This allows the user to compare the data of the form as it has been extracted. by the file processing system with the corresponding original attached file 121 displayed in part 50.
    A person skilled in the art will readily understand that the form is not limited to the fields of the form indicated in the example in FIG. 2, but can include other fields which can be extracted or deduced (for example the field "country" can be deducted from an “address” or “currency” data element).
    The files displayed in the dedicated area 5 can be processed sequentially by the orchestrator device 18. In this embodiment, the processing of an attached file 121 by the orchestrator device 18 can be triggered by the user. In a more general application of the invention, any content of a message could be treated in a similar manner.
    The image view part 50 may for example include a zoom element 501 to allow the user to zoom or move a displayed attached file. In another embodiment, the application extension 152 can include check boxes in the second part 52 of the dedicated area 5 next to each individual attached file, the user being able to select an attachment for integration into the target data processing device 2 using the check box. In another embodiment, the attached files can be processed automatically according to an arbitrary order or predefined criteria.
    In another embodiment, the selected files (eg receipts) can be processed in parallel by the orchestrator device 18 and / or the file processing device 19. To facilitate understanding of the invention, the following description will be made with reference to a sequential processing of the files selected for illustration.
    In some embodiments, the view part 52 of the form 52 can also include a validation button 520 which can be selected by the user to validate the form.
    FIG. 3 schematically shows an example of an electronic message structure 12 sent from the messaging client device 14 to the messaging server 11.
    As we can see, an electronic message 12 can include raw data 120, recipient designation elements 122 identifying one or more recipients, message attachments 121, a message identifier 123 uniquely identifying the message, attributes 124 representing message attributes such as the message expiration attribute.
    FIG. 4 is a schematic view of a file processing device according to certain embodiments.
    The file processing device 19 can include:
    - a data extraction engine 162 configured to extract characters from the unstructured file 190 (eg a receipt image corresponding for example to a scanned image or to a photo of a receipt) using at least one extraction algorithm such as an optical character reader (OCR) algorithm; the data extraction engine 192 can therefore extract the data from the attached file received as input to an embodiment and provide digital data which can be stored in a memory 193 (extracted data memory);
    - a mapper 194 configured to map at least certain keys of a predefined set of keys 196 to one or more elements of digital data entered, in the unstructured file 190 received as inputs by the file processing device 190; the mapper 194 therefore provides a set of keys, each of which is associated with one or more values corresponding to the data elements mapped to it.
    a description file generator 198 configured to generate a description file 191 from the input file 190, with the description file comprising the set of predefined keys 196, each key being associated with zero, one or more values determined from the data elements of the input file 190 mapped to the key.
    In one embodiment, the key set 196 to be mapped to the attached file data elements can be previously filtered according to the type of files attached or received from the orchestrator device 18, the orchestrator device 18 having previously retrieved the key set of the target processing device 2 according to the type of attached file detected. In a management application / expense report of the invention, considering the attached files of sub-receipts, a receipt can have several sub-types such as the taxi sub-type, the hotel sub-type, the sub-type restaurant, each receipt of a subtype being associated with a predefined set of keys (for example a receipt of taxi subtype can be associated with a set of keys which include the date, the amount, the route [origin / destination], the name of the taxi company, etc.).
    The description file can have any format that makes use of text to transmit data objects which consist of attribute value pairs and possibly of network data types or other serializable values such as JavaScript or JSON object notation.
    As used herein, a "description file" refers to a document that uses text to transmit data objects that consist of value-attribute pairs.
    In one embodiment, the file processing device 19 can also comprise a positioning data determination unit (PDDU) configured to determine all of the positioning data from the input file 190. The assembly positioning data can be determined by the data extraction engine 192 and used by the mapper 194 to identify the position of a data item mapped to a key of the predefined set of keys 196 in an image representing the file attached 190. Each set of positioning data which identifies the position of a data element in the attached file can be mapped to a key in the description file 191 (JSON file for example) associated with the key.
    In one embodiment, each set of positioning data can include positioning coordinates (x, y) in a given reference frame, such as a 2D reference frame (X, Y) defined by the original unstructured file.
    In one embodiment, the file processing device 19 may include a notation determination unit 197 configured to determine a notation for the candidate values mapped to certain keys by the mapper 194. Consequently, for a given key associated with several candidate values, a score can be assigned to each value. The file processing device 19 can also be configured to include the notation determined for each value associated with a key given in the description file 191. Otherwise, the file processing device 19 can present the candidate value in an order of relevance based on rating.
    In certain embodiments, the notation can be determined for the keys which are considered to be mandatory in order to trigger processing by the target data processing device 2. For example, in an application of the invention for the management / expense report, the attached file of a type of receipt is required in order to obtain at least one key date and one key amount.
    Figure 5 shows a schematic representation of the data extraction engine 192 of the file processing device 19 according to an example of application of the invention for management or expense reporting.
    Each attached file received by the file processing device 19 can therefore be a receipt. In certain embodiments, the file processing device 19 can receive all the files attached or otherwise included by a user in an email, process each file and trigger an error if a file does not belong to a type of receipt. In another embodiment, each attached file can be processed in an initial processing phase to verify if it has a receipt format and / or detect the type of receipt and / or recover the keys associated with the type of receipt.
    The data extraction engine 192 may comprise a converter 1921 configured to initially evaluate one or more attributes of the attached file required to apply the OCR and / optimize the performance of the data extraction. The values of these attributes can be checked and / or normalized if their values are not optimal for example:
    - by rotating the image; or
    - by checking and correcting the resolution of the image (dots per inch) in case the resolution indicated by the camera is lower than the true resolution.
    The data extraction engine 192 can also include a file classifier 1922 configured to identify the type of an attached file (eg electronic invoice, receipt, etc.) in order to optimize the extraction of data from from the attached file. These types of documents can provide information about the presentation of the attached file and identify the type of information that should be extracted.
    The 1922 file classifier can be configured to identify the type of file attached (e.g. taxi, hotel, etc.) by comparing the attached file to another known set of similar reference documents (e.g. TAXI text) . A document is considered to be similar to an attached file if it includes a set of attributes that are comparable to the attributes of the attached file. These reference documents can be determined from past reconciliations, set by user input and / or not having been invalidated by a user threshold.
    The 1922 file classifier can be configured to convert the image representing the image of the attached file into a feature vector using transformations and filters to provide standard input. In some embodiments, this can create a much smaller image in terms of file size and / or size, the scale of which can be reduced, with intensified, monochrome outlines and balanced brightness.
    The 1922 file classifier can extrapolate the file type based on the resulting feature vector. In one embodiment, an approach based on machine learning can be used to infer one or more properties of the attached file from the characteristic vector, for example the number of columns, the page ratio, the location of the main body. text, the location of the header and so on.
    In one embodiment, if the input file 190 is received in image format, the data extraction engine 192 may comprise an OCR scanner 1924 configured to perform OCR scanning in order to produce text from the attached file and extract words. Otherwise, any data extraction technique can be used depending on the format of the input file 190 (e.g. HTML and PDF text file).
    The data extraction engine 192 may include a language classifier 1925 configured to use the extracted data (eg extracted by the OCR scanner 1924) to generate trigrams representing groups of three overlapping letters. The repetition and the existence of these trigrams can be used to determine the language (s) with a certain level of confidence (for example the trigram "LAN" and more present in Spanish than in English). The data extraction engine 192 can also use contextual data to improve the reliability of data extraction or to determine certain key values without having to determine them from OCR scanning (e.g. known location information, travel routes, GPS coordinates from photographs, etc.). In embodiments where the attached files include several languages, the predominant language can be selected: a predominance level can be assigned to each language according to the level of use of the language throughout the file. Languages with a level of dominance below a certain threshold can be ignored (for example, if the attached file is a receipt including a restaurant name in French while the receipt is in German, German would be the predominant language ). By identifying the languages of a document, the language classifier 1925 allows the data extraction engine 192 to better identify the text and the language belonging to it, using an extraction algorithm such as an OCR algorithm. This eliminates the need for scanning to detect all possible languages and allows the use of language-specific dictionaries.
    The language classifier 1925 can moreover use the language of localization of the data which can be provided by the algorithms of extraction (algorithm OCR for example) for each detected language by identifying the localization or the region of the attached file in which the detected language is used (e.g. the indication that the top line is in Bulgarian while the bottom line is in English).
    This allows a better interpretation of the contents of the attached file using an extraction algorithm (OCR algorithm).
    In another embodiment, instead of using the 1925 language classifier, the data extraction engine can be configured to translate text into the language supported by the extraction algorithms.
    The data extraction engine 192 may also include an information extractor 1926 using an extraction algorithm to extract the characters in the attached file 121, for example an OCR algorithm. Furthermore, the information extractor 1926 can extract the positioning data sets (e.g. the coordinates) from the attached file 121.
    Those skilled in the art will readily understand that the invention is not limited to the use of OCR information extractors, depending on the type of input file 190 received by the file processing device 19. In particular, in a certain embodiment the file processing device 19 can receive an input file 190 in any format including native text for which there is no need for OCR. The following description of certain embodiments of the invention will be made with reference to the OCR extraction algorithms by way of illustration only, the information extractor 1926 then being designated as the OCR information extractor.
    The OCR 1926 Information Extractor can be configured to serialize OCR data as follows.
    The OCR 1926 Information Extractor can first read the result of the OCR 1924 scan character by character with coordinates and formatting. The OCR information extractor can include a "concatenator" 21 to transform these characters into a target format. The target format can for example be a string which must feed the grammars used for the analysis of the document and / or an index of mapping of the characters to their coordinates and formatting information. The initial "concatenation" can follow the "natural" order of reading the text. However, additional concatenations can be provided to allow recognition of sentences which do not necessarily follow this initial order. For example, in a two-column document presentation, concatenation can first return the text in the left column, then the text in the right column. However, upon receipt of the attachments, the line items can be processed in a column on the left and the prices in the column on the right. Therefore, a concatenation of lines returning lines that cover the entire width of the document can be made.
    Important information about the document layout can be encoded as special characters in the returned string to facilitate interpretation of the string. Markers for "the start of the text box", "line breaks", "paragraph ends", "page breaks", and so on may be included. Therefore, information about the 2-dimensional structure of the document may be made available for grammars and regular expressions when this would not be possible other than for 1-dimensional character streams.
    The OCR 1926 information extractor can also be configured to extract candidate values using grammars and regular expressions. Context-free grammars and regular expressions can then be used to find relevant information in the text. Grammars can for example be based on the Unitex grammar context (http://unitexgramlab.org/). The string can be read to produce an XML result for the matches. This XML result can then be read and transformed into a representation of objects. Using the index provided by the "Concatenator", each object can be assigned positioning data such as coordinate data in a given frame of reference.
    This makes it possible to determine the candidate values for further processing and to enrich the information on the candidate values on the basis of the context. For example, all amounts can be extracted (document substrings which could be amounts, eg "12.00"). If this string is followed by "€" then this is the currency of an amount which can be reported as Euro. If it is preceded by the word "Gesamtbetrag", it can be marked as a good candidate for the total amount. The grammars can be written in such a way that they correspond to the minimum character string constituting the amount, but also the relevant previous or next text, if applicable.
    The OCR 1926 information extractor can also perform plausibility checks and / or validation tests. For example, IBANS have a built-in sum check which can be used to filter out false matches or OCR matches with errors.
    In an application of the invention for an expense report, the OCR 1926 information extractor can also be configured to perform an address extraction by searching for "anchors" which represent the combination of the postal code and the city. (eg "81593 München"). The matches for these anchors allow you to restrict the search area for more complex grammars looking for complete addresses. These anchor mappings can also have additional applications, for example to determine the country / city of the document regardless of the detailed address.
    To map certain specific compulsory keys of the receipt, a certain processing can be implemented by the file processing device 19.
    For example, the OCR 1926 information extractor can still be configured to validate the amount detected (corresponding to an amount key) in an attached file (a receipt) using summation trees. In some cases, the relevant amounts of a given receipt may not be listed alone, but together with other amounts that contribute to the target values. For example, the gross amount may be the sum of the net amounts and the VAT amounts. Combinations of possible values can be added together to result in higher values also found in the document and a set of rules can be applied to try to find relationships between these elements.
    These rules can assign roles to the corresponding amounts, for example: net, gross, total, cash, etc.
    Amount validation can be used in combination with other extraction strategies or in a specific context such as for similar receipts. Amount validation can provide an additional parameter for a confidence estimate. It can be independent of other extraction strategies and can ignore the properties used by these extraction strategies, such as location on the document, font size, etc.
    The amount validation process can start with the complete set of candidate amounts A extracted from an attached file received by the file processing device 19. Then, a sum subset algorithm can be applied producing for each amount a in A all combinations of other amounts in A which add up to the amount a. The combinations can be transformed into summation trees. The value of each node in these summation trees can be the sum of the values of its direct children. Rules can then be applied which take into account the structure of the tree, the values of the amounts and the tags of previous iterations. The rules can be applied to these trees in multiple iterations, allowing access to the results of previous iterations. Some rules may use certain contextual information such as country and tax rate. Rules can assign tags marking the role of candidates. For example, an amount (€ 1.19) is the sum of two other amounts found in the document (€ 1.00 and € 0.19). Knowledge of the country where the receipt was issued (Germany) and of the VAT rate (value added tax) in this country (19%) makes it possible to mark these three amounts as gross, net and amount of VAT, respectively . There may be other amounts in the attachment (eg € 0.70, € 0.20 and € 0.10) that can be added together to obtain the gross amount. We can then assume that these other amounts are the elements of the receipt.
    The resulting tags do not have to be correct, since they are used as one feature among others by the following grading service.
    To score the candidate values of the amounts corresponding to the compulsory amount key of a receipt, the rating determination unit 197 of the file processing device can assign a probability to each amount corresponding to the probability that this amount represents the " total amount ”from the receipt using an amount rating component. The rating component 197 can use properties for each amount extracted based on the previous results. Each amount property can be "true" or "false", depending on certain conditions and validation results. For example, this property can be "gross amount" or "have multiple occurrences in the document" or "have a larger font size". Weights can then be assigned to these values depending on the frequency of said property (or said combination of certain properties) that has been observed for total, VAT or net amounts. Using these weights, the confidence rating can be calculated.
    Since these weights are based on observations in the past, they can be generated using historical data or manually created examples.
    The extraction engine 192 can also be configured to perform a seller identification allowing the seller issuing the receipt corresponding to the current attached file to be identified in order to map the candidate values to the seller identification key. Rather than recognizing the seller directly from the attached file, the extraction engine 192 can be configured to extract information in order to infer the identity of the sellers, for example telephone / fax numbers, company registration, VAT identifiers, URLs and so on. With an appropriate database, such information can be used to infer the identity of the seller. In some embodiments, said one or more candidate seller identities (candidate values for the seller's identity key) which are determined may be weighted according to the type of information used for the inference, it is assumed that the Identity of the seller with the highest rating is the identity of the seller.
    The description file 191 thus generated by the file processing device 19 can be sent directly to the target data processing device 2.
    Otherwise, in a preferred embodiment, the description file 191 can be sent back to the orchestrator device 18 to be refined. The orchestrator device 18 can transmit a description file 181 corresponding to the description file itself 191, or to the description file determined from the description file 191, to the application 15 shown in FIG. 1.
    By way of example only, the following specification will be made with reference to this embodiment where the description file is returned to the orchestrator device 18 for refinement.
    When the description file 181 is returned to the application 15 via the orchestrator device, the application extension 152 can be configured to render the extracted description file 180 in the dedicated area 5 of the application interface 150, for example in the second part 52 using a verification form comprising a set of fields, each field corresponding to one of the keys in the description file 181 and each field having one of the values assigned to the key in the description file 181 or not value if no candidate value was found by the file processing device 19. The value assigned to a field corresponding to a key in the description file 181 may initially be the value which was assigned to the highest notation for the key in the description file. The user can then correct the value manually or using visual tools. This makes it possible to refine description files by interacting with the client device 14 to increase the reliability of the description file 181 compared to the original attached file 121.
    FIG. 6 represents a view of the application interface 150 in response to the reception of the description file 181 from the orchestrator device 18, according to certain embodiments.
    In one embodiment, the application extension 152 can also be configured to generate a display of a selectable element that can be highlighted or of box 53 for each different candidate value found for a key (e.g. the amount key) of each attached file processed 121 (for example) from the description file 181 in order to facilitate the verification of the form by the user. Each highlighted box 53 can be displayed at a position in the file 121 determined from a set of positioning data which identifies the position of the candidate value in the description file 181.
    The highlighted boxes 53 can have different shapes such as rectangles and / or be associated with a code color depending on the rating value assigned to the candidate value in order to highlight the relevance of the candidate value as 'it was evaluated by the file processor 19. For example, a green color code can be used to highlight the values which have been selected to complete the form (highest rating) and the red color can be used to highlight other candidate values. If the user clicks on an item highlighted in red, the corresponding value can be used to update the shape, and the highlighted item can turn green while the highlighted item that was green can turn red.
    Otherwise, different visual events can be used to highlight the relevance of a key candidate value from the description file 181 from the notation assigned to the value. In an exemplary embodiment, a superimposed translation can also be displayed on the image for receipts which are not in the user's native language.
    The user can click on one of the highlight boxes associated with a candidate value for a key in order to select it rather than on the one associated with the highest notation in the description file, or on the contrary to confirm the candidate value associated with the highest rating. This can lead to an update of the verification form displayed in the third part 52 of the dedicated area 5 of the application interface 150 and / or to an update of the notation of the candidate value which may involve a change of color of the boxes. highlighting 53 used for the candidate values determined for the key envisaged as a function of the color code.
    In some implementations, the application extension 152 may include a progress tracker to allow a user to track the processing progress of the selected file (not shown). Said tracing can be stored in the context of the application. If the user interrupts his activity and subsequently returns to the application extension 152, the user can resume his activity or be informed if attachments have been submitted.
    The user can iterate the process for each key and make corrections if necessary. When the user has completed the verification process, the user can select the validation button 520 provided in association with the form to trigger the sending of the form to the orchestrator device 18.
    The orchestrator device 18 can transmit a refined description file 182 (such as a form) to the target data processing device determined from an updated version of the intermediate description file 181 in which the values corrected by the user have been updated. In one embodiment, the refined description file 182 can comprise a single value per key, the value assigned to a key being either the value corrected for this key by the user (manually by direct entry or by selection for example highlighted value) or the value with the highest rating, if the user has not corrected the actively confirmed value.
    In one embodiment, the orchestrator device 18 can moreover transmit a description file 183 to the file processing device 19 determined from an updated version of the intermediate description file 181 or from a signal to inform the device 19 that no update has been performed. The description file 183 transmitted to the file processing device 19 can be the same description file as the description file 182 transmitted to the target data processing device 2 or include additional information. This allows data to be collected for machine learning by the file processor 19 or for the subsequent integration of attached files and in particular for the subsequent mapping and scoring operations performed by the file processor 19 for each subsequent integration of attached files.
    Figure 7 is a flowchart describing the method of integrating at least part of the message content received in a target data processing device 2, according to certain embodiments.
    In step 700, an electronic message 12 is received from a messaging client device 14 executing the messaging application 15, the electronic message 12 comprising message content in the form of one or more attached files 121 to the message 12. The message content (eg files) can relate to a given operation or transaction, such as the same expense for the implementation of an expense report of the invention. The following description of certain embodiments of the process will be made with reference to the message content represented by the attached files of an email, for illustration only.
    Each file 121 has a given file format such as PDF, GIF, JPEG, etc. Each electronic message can be associated with a message identifier 123.
    In step 702, message 12 can be stored in a database with the files.
    In step 705, each attached file 121 of the message 12 can be converted or transformed into a description file 191, such as a JSON file, comprising a set of keys of which at least some of the keys are associated with one or more values , such as a JSON file.
    In step 706, an extracted extracted file 182 determined from the description file 191 can be transmitted to the target data processing device 2.
    FIG. 8 is a flow diagram describing the initialization process carried out by the application extension 152 and the orchestrator device 18, according to certain embodiments.
    In step 800, an email comprising one or more attached files 121 can be received, each attached file having a predefined format (eg a JPEG, GIF, PDF, HTML file).
    In step 802, the application extension 152 can be activated, for example in response to the selection of an activation button in the application interface 150. Otherwise, it can be activated dynamically or automatic on the basis of an email or the attributes of attachments (sender, objects, format files, file size ...).
    In step 804, the messaging application 15 of the client device can connect to the messaging server 11 in order to request the load of the extension 152.
    In step 806, an initialization can be displayed in an open window or an i-frame open in a dedicated area 5 of the application interface 150. The initialization view can for example include a thumbnail for each attached file selected in part 51, a view of a first file attached in part 50 and an initialization form in part 52 of the dedicated area 5. the initialization form in part of the application interface 150, the form d initialization comprising a set of fields such as for example the fields of an application for an expense report: "date", "country", "address", "receipt subtype", "total price", "currency ". In one embodiment, the fields of the initialization form can be generated dynamically as an initialization data function comprising a set of keys extracted from the target data processing device 2 by the orchestrator device 18 and / or from the receipt subtype corresponding to the current attached file 121 (the one displayed in part 50). Otherwise, the initialization form can be a default form defined by the target processing device 2.
    In one embodiment, the orchestrator device 18 can connect at an initial stage of processing to the target data processing device 2 to retrieve a list of the attached file subtypes supported by the target data processing device 2 and / or ongoing transactions relating to the user in order to dynamically adapt the fields of the form. For example, if the target data processing device 2 is an expense reporting tool, the orchestrating device may include retrieving the list of supported receipt subtypes (or required or configured as needed) by the '' expense report tool and / or all of the user's current expense statements.
    In step 808, the orchestrator device 18 can connect to the messaging server 11. In other embodiments, the orchestrator 18 can be automatically connected to the messaging server without requiring the implementation of the specific step 808 .
    In step 809, the messaging server 11 can transmit at least a subset of the attached files 121 to the orchestrating device 18. In one embodiment, the messaging server 11 can only transmit to the orchestrating device files having an unstructured format among the attached files 121. In another embodiment, the messaging server 11 can transmit to the orchestrating device 18 all the attached files 121, the attached files then being filtered by the orchestrating device 18 to filter only a subset of attached files according to predefined criteria, the filtering criteria comprising at least the filtering of the attached files having an unstructured format among the attached files 121.
    In an expense reporting application of the invention, each attached file of a subset may be a receipt (e.g., a photo or the result of a scan) such as a taxi or hotel receipt or a receipt for non-travel related expenses (e.g. computer, dinner with clients, batch of photocopies).
    In step 810, the orchestrator device 18 can convert each attached file into a target format, for example to provide an attached file with the smallest size and / or weights to optimize their display by the application. For example, an attached file in PDF format can be converted to a JPEG file.
    In step 811, the orchestrator device 18 can send each file attached in this target format to the messaging application 15 running on the client device 14.
    The application extension 152 can then display an initialization view in the dedicated area 5 of the application interface 150, the initialization view comprising a display of each attached file received by the orchestrator device 18 in the form of a thumbnail in part 51 of the dedicated area, each thumbnail being an image on which one can click.
    The initialization view can for example include a thumbnail for each attached file selected in part 51, a view of one of the attached files in part 50 and an initialization form in the right part of the dedicated area. The initialization form can include a set of fields such as for example the fields of an expense report application: "date", "country", "receipt subtype", "total price", "currency" . In one embodiment, the fields of the initialization form can be generated dynamically as an initialization data function comprising a set of keys extracted from the target data processing device 2 by the orchestrator device 18 and / or from the receipt subtype corresponding to the current attached file 121 as it is displayed in part 52 of the dedicated area 5. Otherwise, the initialization form can be a default form defined by the processing device target 2. The attached file displayed on the left can be selected at random by the application extension 2 or correspond to the first thumbnail displayed in the middle part of the application interface or correspond to a thumbnail selected by the user in part 51 of the dedicated zone 5 (when he clicks on it).
    The processing of the attached file displayed in part 50 by the orchestrator device 18 can be triggered automatically.
    Figure 9 is a flowchart describing the process of integrating an attached file 121 into the target processing device 2 implemented by the orchestrator device 18 in accordance with certain embodiments.
    In step 900 the processing of the current attached file is requested by the application, for example in response to the selection of a file in the part 51 of the dedicated area or automatically by a selection of a file by the extension 152 or by the orchestrator 18.
    In step 902, for the current attached file, the orchestrator device 18 can request the corresponding attached file from the messaging server 11 if the orchestrator device 18 has not previously stored it.
    In step 904, the orchestrator device 18 receives the corresponding image.
    In step 905, the orchestrator device 18 transfers the attached file to the file processing device 19. In embodiments, rather than transmitting the attached file as it is retrieved from the mail server 11 or to starting from a memory kept by the orchestrator device 18, the orchestrator device 18 may previously apply a preprocessing to recover the image and adapt its format or quality.
    Figure 10 is a flowchart describing the process implemented by the file processing device according to certain embodiments.
    In step 1000, an attached file in a format supported by the file processor is received from the orchestrator 18 (eg, pdf, image format). In embodiments, the attached file 190 can be previously processed to optimize its processing.
    In step 1001, at least one extraction algorithm, OCR for example, is applied to the attached file to extract characters and / or the positioning data from the file 190.
    In steps 1002 to 1005, the data extracted from the attached file are analyzed and structured into a description file conforming to a predefined data structure comprising a set of keys.
    Specifically, step 1002 may include analyzing the extracted data to map each key of a predefined set of keys to one or more data items of the data extracted from the attached file received, each mapped data item representing a value candidate for a key.
    Step 1003 may include calculating a score for each candidate value assigned to a key.
    In embodiments, the mapping step and / or the scoring step can be performed using machine learning.
    In one embodiment, step 1004 can comprise the calculation of a set of positioning data for each candidate value determined for one or more keys, the set of positioning data determined for a candidate value representing the position of the candidate value in the attached file. The positioning data determined for a given data element can include the coordinates in a given repository defined with respect to the non-modifiable file.
    In step 1005, the description file as generated can be sent back to the orchestrator device 18.
    Figure 11 is a view of an excerpt from an example description file 190 in JSON (pseudocode) format.
    As presented, the description file 190 comprises a set of keys, each having one or more candidate values determined by the file processing device 19, each candidate value being associated with a notation representing the level of relevance of the candidate value determined by the file processor. Each key is also associated with a set of positioning data representing the position of the corresponding data element in the original attached file 121.
    Figure 12 is a flowchart describing the process of refining a description file according to embodiments.
    In step 1200, the description file 191 is received by the orchestrator device 18 from the file processing device 19.
    In step 1202, a description file 181 determined from the description file 191 can be transmitted from the orchestrator device 18 to the application 15 of the client device 14 (the description file 181 can be the description file 191 itself, or a transformed version of the description file 191). In some embodiments, the corresponding attached file 121 can be transmitted simultaneously to the application.
    In step 1204, a display of the description file 181 can be generated by the application extension 152 in the form of a verification form, in the part 52 of the dedicated area 5. The corresponding attached file 121 ( e.g. a receipt) can be displayed as an image in part 50 of the dedicated area to allow verification by the user if it was not displayed during the initialization phase. The form includes a set of fields, each field corresponding to a key in the description file. If an initial view of the form was displayed when launching the application extension 152, the fields of the form can be updated from the content of the content description file 181 in step 1204. For each field of the form , the candidate value with the highest notation for the key corresponding to the form field in the description file 181 can be assigned to the value of the field. In some embodiments, the other candidate values can be highlighted or displayed using visual elements in the dedicated area to highlight the relevance of the candidate value according to the rating determined for this value. In one embodiment, the visual elements can be the clickable highlighting boxes 55 which are superimposed directly on the image representing the attached file in the part 50 of the dedicated area 5, the position of the highlighting box being determined from the positioning data set associated with the candidate value in the description file 181.
    In step 1205, the application extension can update the value of a given field of the form in the part 52 of the dedicated zone 5, in response to the selection of another candidate value by the user who uses the visual elements 55, for example by clicking on one of the highlighting boxes in the RO part of the dedicated area 5 in response to a correction made by the user (by text input) directly in the verification form displayed in part 52 of the dedicated area 5. The user can also enter values for the empty fields if no value has been determined for a key. Step 1205 can be iterated for one or more keys until the user validates the form, for example by clicking on a validation button (520). User input can also be subject to the same checks as the checks performed by the file processing device 19 for the extracted data. For example, the IBAN number entered by the user will undergo the same checksum as that performed by the file processor 19.
    In step 1206, the application extension can connect to the messaging server 11 to request the authentication data, such as a password and an identifier, giving access to the target data processing device 2 in the embodiments where these authentication data are stored in the messaging server 11 or in an external database. Otherwise, this authentication data can be requested by the orchestrator device 18 directly.
    In step 1208, the form, possibly with the authentication data, can be submitted to the orchestrator device 18 with the updated field values.
    In step 1210, the description file 191 can be updated by using the corrected values or by selecting the value having the highest notation for each key for which the user has not corrected the values, which allows obtain a refined description file.
    A description file 182 determined from the refined description file can be generated by the orchestrator device 18. The determined description file 182 (also known as "validated description file") can be the refined description file itself or a version transformed from the description file.
    The validated description file 182 can then be transmitted to the target data processing device 2 by the orchestrator device 18 for processing by the target data processing device 2 at step 1212. Step 1214 can also include transmission from the original attached file 121 to the target data processor 2 in the original format or in the format required by the target data processor.
    In one embodiment, a description file 183 determined from the refined description file can also be transmitted to the file processing device 19 in a format supported by the file processing device to allow the collection of data. meta-learning by the file processing device 19 in step 1216 (the description file 183 may include the same content as the description file 182 with the same format or another format, or include additional data). This meta-learning data can be used by the file processing device 19 for the next transformation of attached files. Meta-learning data can be used to determine candidate values. For example, if the receipt is a taxi receipt and one of the keys is the price of the receipt for a given route from place A to place B, the meta-learning data collected may include the average price for this route.
    Such feedback collected from the user can be used by the file processing device 19 to determine the accuracy of the data extractions by comparing the description file returned by the file processing device with the description file refined by the user. This meta-learning data can be stored and used to further improve the system. Using the stored return data, the file processing device 19 can be reformed continuously. For example, in order to perfect the “rating of the amount key” component of the rating unit 197, the returns from the storage of results combined with the intermediate results that are used to generate the amount properties in the rating component of amount, can be collected. The amount scoring component can be restarted and its results can be compared with the feedback data. If there are differences, the weights used in the amount scoring component can be updated.
    In an expense report management application, the target data processing device 2 (management tool / expense report) can therefore manage the receipts to submit a reimbursement request for the user, after checking the validity of the costs (for example if they are associated with an existing trip) and / or its uniqueness (for example, if two users have submitted the same receipt). In one embodiment, this target data processing device 2 can send a return to the user via the orchestrator device to indicate for example that costs corresponding to a processed receipt are being processed and / or a reimbursement request has been triggered and / or the transmitted receipt has triggered a uniqueness validity error.
    It will be noted that even if certain technical aspects of the invention have been described in combination, they can however be used separately in certain applications. In particular while the file processing device has been described in connection with the use of a messaging application, those skilled in the art will readily understand that in certain embodiments the file processing device 19 can be used independently to convert an unstructured input file such as an image or PDF file to a structured description file.
    The embodiments of the invention can be implemented on a computer system comprising one or more networked computers or servers.
    Referring now to Figure 13, the client device 14, the messaging server 11, the orchestrator device 18, the file processing device 19 and the target processing device can be implemented on one or more computer devices or systems, designated collectively as a computer, such as computer 30. The computer 30 may include a processor 32, a memory 34, a mass storage memory device 36, an input / output (I / O) interface 38, and a man-machine interface (HMI) 39. The computer 30 can also be operatively coupled to one or more external resources 42 via the network 6 and / or the I / O interface 38. The external resources can include but not limited to servers, databases, mass storage devices, peripheral devices, cloud network services (cloud), or any other informational resource suitable tick that can be used with computer 30.
    The processor 32 can include one or more selected devices: microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, programmable door networks, programmable logic devices, defined state machines, logic circuits, analog circuits, digital circuits or any other device used to manipulate signals (analog or digital) based on operating instructions stored in memory 34. Memory 34 can include a single device or a plurality of memory devices, in particular but not limited to, read-only memory (ROM), random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM) ), flash memory, cache memory or any other device capable of storing information. The mass storage memory device 36 can include data storage devices such as a hard disk, an optical disk, a magnetic tape drive, a volatile or non-volatile solid state circuit or any other capable device. to store information. A database 44 can reside on the mass storage memory device 36, and can be used to collect and organize the data used by the various systems and modules described herein.
    The processor 32 can operate under the control of an operating system 46 which resides in memory 34. The operating system 46 can manage computer resources so that the computer program code, integrated in the form of one or more application software, such as the application 48 which resides in the memory 34, can have instructions executed by the processor 32. In an alternative embodiment, the processor 32 can directly execute the application 48 ; in this case the operating system 46 can be omitted. One or more data structures 49 can also reside in memory 34, and can be used by processor 32, operating system 46, or application 48 to store or manipulate data.
    The I / O interface 38 can provide a machine interface which couples the processor 32 functionally with other devices and systems, such as the network 6 and / or the external resource 42. The application 48 can thus collaborate with the network 6 and / or with the external resource 42 by communicating via the I / O interface 38 to provide the various elements, functions, applications, processes, modules making up the embodiments of the invention. The application 48 may also include a program code which is executed by one or more external resources 42, or is otherwise based on the functions and / or signals provided by other system or network components external to the computer 30. Indeed, in view of the almost infinite configurations of possible computer hardware and software, those skilled in the art will understand that the embodiments of the invention can include applications located outside of computer 30, distributed to computers. multiple or other external resources 42 or provided by computer resources (hardware and software) which are provided by a service such as a cloud computing service, via the network 6.
    ΗΜΙ 39 (such as the HMI 30 in the implementation of Figure 1 of the client device
    3) can be operatively coupled to processor 32 of computer 30 in a known manner to allow a user of computer 30 to interact directly with computer 30. HMI 39 can include video displays and / or alphanumeric, a touch screen, a loudspeaker and any other visual and audio indicator capable of communicating information to the user. The HMI 39 can also include input devices and controls such as an alphanumeric keyboard, pointing device, keyboards, push buttons, control buttons, microphones, etc., capable of accepting commands or user input, and transmitting the input input to processor 32.
    The database 44 can reside on the mass storage memory device 36, and can be used to collect and organize the data used by the various systems and modules described herein. The database 44 can include data and accommodate the associated data structures that store and organize the data. In particular, the database 44 can be set up with any database organization or structure, in particular but not limited to, a relational database, a hierarchical type database, a networked database, an object-oriented database or combinations thereof. A database management system in the form of application computer software which runs in the form of instructions on processor 32 can be used to access information or data stored in files of the database 44 in response to a query, when a query can be dynamically determined and executed by the operating system 46, the other applications 48, or one or more modules. Although embodiments of the invention may be described herein using relational, hierarchical, network, object-oriented, or other terminology in specific cases, those skilled in the art will understand that the modes of embodiment of the invention may use any suitable database management model, and are not limited to any particular type of database.
    Program code that implements any of the embodiments described herein may be distributed individually or collectively as a program product, in a variety of forms. In particular, the program code can be distributed using computer readable media which may include storage media and computer readable communication media. Computer-readable storage media, being inherently non-transient, can include volatile and non-volatile, removable and non-removable tangible media, implemented in any information storage process or technology, such as computer-readable program instructions, data structures, program modules, or other data. Computer readable storage media may also include RAM, ROM, read only, programmable and erasable memory (EPROM), read only memory, electrically erasable and programmable (EEPROM), flash memory, or any technology solid memory media, a portable compact disc with read-only memory (CD-ROM) or other optical storage, magnetic tapes, magnetic recording tapes or other magnetic storage devices, or any other medium that can be used to store the desired information and able to be read by a computer. Communication media can implement computer-readable instructions, data structures, or other program modules. For example, but not limited to, a communication medium may include wired media such as a wired or direct wired network and wireless media such as acoustic, RF, infrared or other wireless media. Combinations of all of the above have also been included in the range of computer readable media.
    The methods described herein can be implemented by computer program instructions provided by the processor of any type of computer to produce a machine with a processor that executes instructions to implement the functions / actions specified herein . These computer program instructions may also be stored on a computer readable medium which can instruct a computer to operate in a particular way. For this purpose, the computer program instructions can be loaded onto a computer to cause the implementation of a series of operational steps and thereby produce a computer-implemented process, so that the executed instructions provide the processes for implement the functions / actions specified herein.
    In addition, the program code described herein can be identified based on the application or software component in which the program code is implemented in a specific embodiment of the invention. However, it should be noted that any nomenclature of a particular program which follows is used only for convenience; thus the invention cannot be limited to a single use in any specific application identified and / or implied by said nomenclature. It will also be understood that the various applications and characteristics and devices disclosed herein can also be used alone or in any combination. Furthermore, in view of the generally infinite number of means by which computer programs can be organized according to subprograms, procedures, processes, modules, objects, and so on, as well as the various ways of affecting the functionality of a program. among various layers of software that reside in a typical computer system [eg, operating systems, libraries, program application interfaces (APIs), applications, small applications (applets), etc.] , and / or between one or more hardware platforms, it will be noted that the invention is not limited to the specific organization and to the allocation of the program functionalities described herein.
    Although the embodiments of the invention are illustrated by a description of various examples and although these embodiments are described in great detail herein, it is not the intention of the applicant to restrict or restrict limit themselves to these details in any way whatsoever, the scope of the appended claims. Additional advantages and additional modifications will be readily apparent to those skilled in the art. The invention from a wider angle is therefore not limited to specific details, representative processes and apparatus, and to the illustrations shown and described by way of example.
    权利要求:
    Claims (12)
    [1" id="c-fr-0001]
    1. A system for integrating the content of a message into a target processing device (2), said target processing device (2) being configured to process input data having a predefined data structure, the system comprising a messaging server (11) configured to receive a message from a messaging client device (14) which runs a messaging application (15) said message including message content, wherein the system further includes a processing device file (19) and an orchestrator device (18) configured to integrate at least part of the message content in the target data processing device (2), said orchestrator device (18) being further configured to:
    - receive part of the message content from the messaging server (1); and
    transmitting a determined file from said part of the content of the message to a file processing device (19), said file processing device being configured to transform each received file (190) into a description file (191) comprising a set of predefined keys, at least some of the keys being associated with one or more values, the orchestrator device (18) being configured to determine an input file (190) having the predefined data structure from said description file ( 191) and for transmitting the determined input file (182) to the target data processing device (2) for processing the input file determined by the target processing device.
    [2" id="c-fr-0002]
    2. The system according to claim 1, in which the orchestrator device (18) is connected to the messaging server (11) according to a first communication protocol, to the messaging application (150) according to a second protocol, and to the device processing target data (2) according to a third communication protocol.
    [3" id="c-fr-0003]
    3. The system according to claim 2, in which the messaging application (15) comprises an application interface (150) and an application extension (152) configured to generate a reproduction of a determined input file. the description file (191) provided by the file processing device (19) in a dedicated area (5) of the application interface (150).
    [4" id="c-fr-0004]
    4. The system as claimed in any one of the preceding claims, in which the file processing device (19) is configured to map one or more data elements of said part of the message content to at least certain keys of a set predefined keys, the file processing device being configured to generate said description file from said part of the message content, said file description comprising said set of predefined keys, said one or more values associated with said keys of the file description determined from data elements mapped to it.
    [5" id="c-fr-0005]
    The system according to claim 4, wherein the file processing device (19) is further configured to determine sets of positioning data from said file received from the orchestrator device (18), each set of positioning data. identifying the position of a data element of the file by mapping a key from said set of predefined keys, each set of positioning data being included in said description file which is associated with the key mapped to the data element.
    [6" id="c-fr-0006]
    6. The system according to claim 5, in which each set of positioning data comprises positioning coordinates in a given frame of reference.
    [7" id="c-fr-0007]
    7. The system according to any one of the preceding claims 3 to 6, in which said application interface (150) is an application of the graphical interface, said application extension (152) being configured to render the file of description in a dedicated area of said application interface (150).
    [8" id="c-fr-0008]
    8. The system according to any one of the preceding claims 4 to 7, in which the file processing device (19) is also configured to determine a notation for each value associated with a given key of said predefined set of keys, the file processing device being further configured to include the notation determined for the value associated with said key given in the description file.
    [9" id="c-fr-0009]
    The system of claim 8, wherein said application extension (152) is further configured to display an image of the content portion of the message in said dedicated area and to generate the display of one or more selected elements to highlight the elements for each data element of the content part of the message by mapping a key, each highlight element for a given data element being displayed at a position in the displayed image which is determined from the set of positioning data identifying the position of said data element.
    [10" id="c-fr-0010]
    10. The system according to any one of the preceding claims, in which the orchestrator device (18) is configured to use a message identifier associated with the message for each exchange between the orchestrator device (18) and the messaging server (11) , and / or the application extension (152), and / or the file processing device (19) and / or the target data processing device (2).
    [11" id="c-fr-0011]
    11. A method for integrating message content into a target processing device (2), said target data processing device (2) being configured to process input data having a predefined data structure, the method comprising the receiving a message from a messaging client device (14) which runs a messaging application (15), said message comprising the content of the message;
    wherein the method includes integrating at least a portion of message content into said target data processing device (2), said method being further configured to determine a determined file from said portion of message content and to transform said file (190) into a description file (191) comprising a set of predefined keys, at least some of the keys being associated with one or more values, the method comprising deriving an input file (190 ) having said predefined data structure from said description file (191) and transmitting said determined input file (182) to the target data processing device (2) for processing the input file determined by the processing device target.
    [12" id="c-fr-0012]
    12. A computer program product comprising:
    a non-transient computer-readable storage medium; and instructions stored on a non-transient, computer-readable storage medium which, when executed by a processor, causes the processor to integrate message content into a target data processing device (2), said processing device target data processing (2) being configured to process the input data having a predefined data structure, the processor being also required to:
    - receive a message from a messaging client device (14) which runs a messaging application (15), said message comprising message content;
    - integrating at least part of the message content into said target data processing device (2), the processor being moreover led to determine a determined file from said part of message content and to transform said file (190) in a description file (191) comprising a set of predefined keys, at least some of the keys being associated with one or more values,
    - determining an input file (190) having said predefined data structure from said description file (191) and transmitting said determined input file (182) to the target data processing device (2) for processing the input file determined by the target processing device.
    类似技术:
    公开号 | 公开日 | 专利标题
    US10282377B2|2019-05-07|Suggested terms for ambiguous search queries
    US10108726B2|2018-10-23|Scenario-adaptive input method editor
    US11074309B2|2021-07-27|Text-to-media indexes on online social networks
    US20150081686A1|2015-03-19|Default Structured Search Queries on Online Social Networks
    US9218568B2|2015-12-22|Disambiguating data using contextual and historical information
    US20170083599A1|2017-03-23|Incorporation of semantic attributes within social media
    US8838610B2|2014-09-16|Listing tune-up system
    MX2015006040A|2015-08-07|Grammar model for structured search queries.
    FR3069075A1|2019-01-18|SYSTEM AND METHOD FOR INTEGRATING MESSAGE CONTENT IN A TARGET DATA PROCESSING DEVICE
    US10592571B1|2020-03-17|Query modification based on non-textual resource context
    US20090293059A1|2009-11-26|Automatically connecting items of workflow in a computer program
    US10592505B2|2020-03-17|Natural language interfaces for databases using autonomous agents and thesauri
    WO2012028817A1|2012-03-08|Method of gathering data of an event-like nature from electronic forms
    AU2017200341A1|2017-02-09|Suggested terms for ambiguous search queries
    US20200167613A1|2020-05-28|Image analysis enhanced related item decision
    CA3025463A1|2017-11-30|Method of triggering complementary actions implemented by an electronic device cooperating with a peripheral
    US10296913B1|2019-05-21|Integration of heterogenous data using omni-channel ontologies
    US10242033B2|2019-03-26|Extrapolative search techniques
    CN109791545A|2019-05-21|The contextual information of resource for the display including image
    KR102141245B1|2020-09-14|Online contents funding system and method based upon matching with contents creator and investor
    EP3306555A1|2018-04-11|Diversifying media search results on online social networks
    CN110741331A|2020-01-31|System, method and apparatus for image response automated assistant
    KR20210054021A|2021-05-12|Probabilistic item matching and search
    US20210065245A1|2021-03-04|Using machine learning to discern relationships between individuals from digital transactional data
    US20210064921A1|2021-03-04|Systems and Methods for Generating Names Using Machine-Learned Models
    同族专利:
    公开号 | 公开日
    AU2018299826B2|2021-10-07|
    AU2018299826A1|2020-01-30|
    WO2019011804A1|2019-01-17|
    EP3652917A1|2020-05-20|
    CN110999264A|2020-04-10|
    NZ760613A|2021-11-26|
    US20200142862A1|2020-05-07|
    FR3069075B1|2021-02-19|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20060089907A1|2004-10-22|2006-04-27|Klaus Kohlmaier|Invoice verification process|
    EP2634735A1|2012-03-01|2013-09-04|Ricoh Company, Ltd.|Expense report system with receipt image processing|
    WO2014022919A1|2012-08-10|2014-02-13|Transaxy Inc.|System for entering data into a data processing system|
    US20140288981A1|2013-03-20|2014-09-25|Concur Technologies, Inc.|Methods and systems for travel-based interactions|
    CN110138992A|2018-02-08|2019-08-16|精工爱普生株式会社|The production method of receipt processing unit, the storage medium of program and report|
    FR3099605B1|2019-08-02|2021-12-17|Amadeus Sas|DEVICE, SYSTEM AND METHOD FOR PROCESSING IMAGES WHICH INCLUDE AMOUNTS|
    CN112134785B|2020-09-14|2021-11-02|上海纽盾科技股份有限公司|Information processing method, client and system in network security level protection|
    法律状态:
    2019-01-18| PLSC| Publication of the preliminary search report|Effective date: 20190118 |
    2020-07-21| PLFP| Fee payment|Year of fee payment: 4 |
    2021-07-28| PLFP| Fee payment|Year of fee payment: 5 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1756717|2017-07-13|
    FR1756717A|FR3069075B1|2017-07-13|2017-07-13|SYSTEM AND METHOD FOR INTEGRATING MESSAGE CONTENT INTO A TARGET DATA PROCESSING DEVICE|FR1756717A| FR3069075B1|2017-07-13|2017-07-13|SYSTEM AND METHOD FOR INTEGRATING MESSAGE CONTENT INTO A TARGET DATA PROCESSING DEVICE|
    EP18734843.8A| EP3652917A1|2017-07-13|2018-07-06|System and method for integrating message content into a target data processing device|
    AU2018299826A| AU2018299826B2|2017-07-13|2018-07-06|System and method for integrating message content into a target data processing device|
    NZ760613A| NZ760613A|2017-07-13|2018-07-06|System and method for integrating message content into a target data processing device|
    PCT/EP2018/068373| WO2019011804A1|2017-07-13|2018-07-06|System and method for integrating message content into a target data processing device|
    CN201880053503.1A| CN110999264A|2017-07-13|2018-07-06|System and method for integrating message content into a target data processing device|
    US16/629,464| US20200142862A1|2017-07-13|2018-07-06|System and method for integrating message content into a target data processing device|
    [返回顶部]