• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    A system (100) for monitoring network traffic and resource usage in a data center. The system includes a node (101) including input means (102) for capturing network packets. The node (101) includes processing means (103) for running a packet analyzer to analyze the network packets where packets are classified by type into request / response pairs based on deep packet inspection, and the request / response pair types together with timestamps. being saved. The node (101) includes a memory (104) for storing request / response pairs. The system includes means for collecting the request / response pairs. The system includes at least one server (106) running a simple agent to transfer resource usage. The system includes a correlation module to correlate the request / response pair information with resource usage.
    公开号:BE1022604B1
    申请号:E2014/0716
    申请日:2014-09-22
    公开日:2016-06-15
    发明作者:Frederick Ryckbosch；Stijn Polfliet
    申请人:CoScale NV；
    IPC主号:
    专利说明:

    Efficient monitoring of data centers. Field of the invention
    The invention relates to the field of monitoring data centers. More specifically, the invention relates to a system and method for monitoring network traffic and load in a data center.
    BACKGROUND OF THE INVENTION
    The increased information exchange via telecommunication networks has led to an increased load in data centers. There are stringent requirements for data centers in terms of speed when retrieving data, availability of data center services, security and data volumes. These requirements are reflected in the Service Level Agreements.
    Existing monitoring solutions either place the emphasis on the application layer or the platform layer. In particular, monitoring solutions in the application layer, such as Google Analytics and Crazy Egg, find out which part or parts of the application is / are most interesting for end users.
    Monitoring on the platform layer keeps track of server and network load and / or power consumption, see for example Zabbix, Nagios, Hyperic (VMware) etc. US 1822453 B2 (IBM) describes a method and system for managing resources in a data center. This patent monitors the current behavior and resource use of an application, makes a prediction about future application requirements and performance and changes the application environment to meet a certain performance level.
    Two typical monitoring solutions from the prior art are illustrated in FIG. 6 and FIG. 7. FIG. 6 illustrates a monitoring solution from the prior art. A complex agent has been implemented on every server in the data center for monitoring purposes. The problem with this kind of solutions is that they impose an overhead per request, that the execution path is changed and that the monitoring interferes with normal operation in a data center. FIG. 7 illustrates data analysis steps for a state-of-the-art monitoring solution that requires a large storage overhead and computer-intensive processing on a large amount of data.
    That is why there is still room for improvement in data center monitoring systems that identify whether Service Level Agreements are being met and that can identify any bottle necks in a data center and the results of which may be indicative of possible architectural improvements to the data center.
    Summary of the invention
    It is an object of embodiments of the present invention to provide good systems and methods for monitoring the load and efficiency of a data center.
    It is an advantage of embodiments of the present invention that application-level monitoring information and platform-level monitoring information are coupled efficiently. This leads in fact to a holistic monitoring solution for a wide range of people, from technicians, system administrators, business developers, marketers to senior management.
    It is an advantage of embodiments according to the present invention that the size of the packages is reduced. It is an advantage of embodiments of the present invention that only a very limited part of the package information is retained. The packages contain strings, for example a host name, a URL, are processed by the package analyzer and stored as a request type being an integer.
    It is an advantage of embodiments of the present invention that the entire network stream is captured and analyzed so that sampling that can lead to an inaccurate analysis is avoided. In order to have a good or optimal correlation between network traffic and resource use, it is important that the entire network stream (eg incoming and outgoing traffic) is analyzed, since sampling can lead to the overlook of some events, with such events being critical events in terms of network traffic and resource use.
    It is an advantage of an embodiment of the present invention that it allows monitoring within applications. That means that the resource use can be displayed for each request type defined in the embodiment of the present invention. Such request types can correspond to different components in the application, so that a more detailed analysis of the application's source usage becomes possible. This is in contrast to some other tools that usually cannot calculate the resource use of different request types and only display resource monitoring over the entire application. In other words, it is an advantage of embodiments of the present invention that a close-meshed monitoring of performance and information at company level within an application (intra-application level) can be obtained. Through deep packet inspection it is possible to obtain information at the application level and intra-application level. Therefore, different request types and subsequent responses can be identified. Moreover, it is possible to obtain accurate information about the load that the individual requests / responses impose on the data center, which provides useful information for optimizing the application or the way it is processed in the data center.
    It is an advantage of embodiments of the present invention that no complex agent is required on each server. Typically, the complex agent that processes the data and performs the analysis of resource use at the application level is the agent that requires the most efforts from, for example, system administrators in terms of installation, updates and runtime overhead. According to embodiments of the present invention, such complex agents do not have to be installed on each server, but a simple agent can be used that is less demanding, e.g. in terms of installation / updates. The simple agent can be a built-in SNMP agent that comes with the operating system. In this way installation / update and runtime overhead can be reduced. In addition, system administrators are usually reluctant to install additional software on production servers.
    It is an advantage of embodiments of the present invention that it does not improve scalability, energy consumption, data protection and / or security issues.
    It is an advantage of embodiments of the present invention that system administrators and technicians can use the monitoring information to understand how request waiting time (and user satisfaction) is related to the resources required in the data center. Business developers will find the monitoring solution valuable for understanding how application features are related to costs and revenue.
    It is an advantage of embodiments of the present invention that the solutions are scalable for a larger size of the data center, both in terms of the number of servers in the data center and in terms of the amount of network traffic in the data center. In embodiments according to the present invention, it was possible to monitor 260 servers through 1 monitoring server at 50% of its capacity. Existing monitoring systems that use complex agents have an overhead of between 1 and 10% on each server that they monitor. Thus, an embodiment of the present invention can result in a much more efficient system.
    The above object is achieved by a method and device according to the present invention.
    The present invention relates to a system for monitoring network traffic and resource use in a data center, the system comprising a node comprising input means for capturing network packets, processing means for running a packet analyzer to analyze the network packets where packets are classified in request / response pairs per type based on deep packet inspection, and a memory for storing the request / response pairs, means, e.g. one or more processing means or central processing means for collecting the request / answer pairs and for information receive information about resource usage from at least one server, and a correlation module to correlate request / response pair information with resource usage. The correlation module can be contained in one or more processing means.
    The input means can capture substantially all network packets from the network traffic and said substantially all network packets can be processed by the packet analyzer.
    The network packages can first be filtered based on package information and the package analyzer only processes the filtered packages.
    Only the HTTP packets can pass through the filter, filtering on the TCP port data, and the packet analyzer can parse URL and host information from the filtered packets.
    Response times of different requests of the same type can be combined in a waiting time histogram.
    The package analyzer can process the packages in order of arrival.
    The package analyzer can process the packages, classifying them by request types defined by the user.
    Data in the packages can be aggregated by the system.
    The packet analyzer may only completely store 10-100 requests per second per request type to retrieve complete URL and host information.
    The system may furthermore comprise a monitoring unit to display the data center load versus the speed of request / response pairs.
    The system can be implemented as a computer device.
    The system can be implemented as an application.
    The present invention also relates to a method for monitoring network traffic and resource use in a data center, the method comprising the following steps, capturing network packets entering or leaving the data center via a node input means, analyzing the network packets by a packet analyzer running on processing means, the network packets being classified by request / response pairs by type and based on deep packet inspection, obtaining information about resource use of at least one server in a central processing means, correlating the information about request / response pairs with it
    All network packages can be captured in the capture step, while all captured network packages can be analyzed in the analysis step.
    The captured network packages can be filtered based on package information, with the package analyzer only processing the filtered packages. Only the HTTP packets may pass through the filter in the filtering step, filtering on the TCP port data, and the packet analyzer can parse URL and host information from the filtered packets.
    The method may include combining response times of different requests into a wait time histogram.
    In at least some of the steps, the packages can be processed in order of arrival.
    The present invention also relates to a data carrier comprising a set of instructions, when executed on a computer, to monitor network traffic and resource use in a data center according to a method as described above. The data carrier can be, for example, a CD-ROM, a DVD, a flexible disk or floppy disk, a tape, a memory chip, a processor or a computer.
    Specific and preferred aspects of the invention are described in the appended independent and dependent claims. Features of the dependent claims can be combined with features of the independent claims and with features of other dependent claims if appropriate and not only if explicitly stated in the claims.
    These and other aspects of the invention will become clear and further explained with reference to the embodiment (s) described below.
    BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic overview of a system for monitoring network traffic and resource use in a data center according to an embodiment of the present invention. FIG. 2 is a schematic overview of the data components required for the correlation module to operate in accordance with an exemplary embodiment of the present invention. FIG. 3 is a schematic illustration of a network stream captured by input means (the capturing network interface) and post-processing, according to an exemplary embodiment of the present invention. FIG. 4 is a schematic illustration of package processing according to an embodiment of the present invention. FIG. 5 illustrates a simple agent on a server for transferring server system statistics. FIG. 6 illustrates a complex agent that runs on a server according to a solution for monitoring data centers from the prior art. FIG. 7 illustrates processing of a network packet stream according to a solution for monitoring data centers from the prior art. FIG. 8 illustrates a method for monitoring network traffic and resource use in a data center according to an embodiment of the present invention.
    The figures are only schematic and non-limiting. In the figures, the size of some of the elements can be exaggerated and not drawn to scale for illustrative purposes.
    Any references in the claims should not be construed as limiting the scope.
    In the various figures, the same references refer to the same or analogous elements.
    Detailed description of illustrative embodiments
    The present invention will be described with reference to specific embodiments and with reference to certain drawings, but the invention is not limited thereto, but is only limited by the claims. The described figures are only schematic and non-limiting. In the figures, the size of some of the elements can be exaggerated and not drawn to scale for illustrative purposes. The dimensions and the relative dimensions do not correspond to the actual embodiments of the invention.
    Moreover, the terms become first, second, and so forth in the description and not necessarily for describing a sequence, either in time, in space, in importance or in any other way. It is to be understood that the terms used are interchangeable under proper conditions and that the embodiments of the invention described herein are capable of operating in sequences other than those described or illustrated herein.
    In addition, the terms above, below, etc. in the description and claims are used for descriptive purposes and not necessarily to describe relative positions. It is to be understood that the terms used are interchangeable under proper conditions and that the embodiments of the invention described herein are capable of operating in orientations other than those described or illustrated herein.
    It is to be noted that the term "comprising" as used in the claims should not be interpreted as being limited to the means specified thereafter; it does not exclude other elements or steps. It must therefore be interpreted as a specification of the presence of the listed features, units, steps or components referred to, but it does not exclude the presence or addition of one or more other features, units, steps or components or groups thereof. Therefore, the scope of the expression "a device comprising means A and B" should not be limited to devices consisting only of parts A and B. It means that with regard to the present invention, the only relevant parts of the device A and B to be. ,
    References in this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or feature described in connection with the embodiment is included in at least one embodiment of the present invention. Statements of the phrase "in one embodiment" or "in an embodiment" at different places in this specification do not necessarily all refer to the same embodiment, but it is possible. Furthermore, the specific features, structures or characteristics may be combined in any suitable manner in one or more embodiments, as will be apparent to those skilled in the art from this disclosure.
    In a similar manner, it should be noted that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped into a single embodiment, figure, or description thereof to streamline disclosure and understanding of one or more of the various inventive aspects to ease. However, this method of disclosure should not be interpreted as an expression of an intention that the claimed invention requires more features than expressly stated in each claim. As shown in the following claims, the inventive aspects lie in less than all the features of a single preceding disclosed embodiment. Therefore, the claims that follow the detailed description are hereby explicitly included in this detailed description, wherein each claim stands on its own as a separate embodiment of the present invention.
    In addition, since some embodiments described herein include some, but not other, features included in other embodiments, combinations of features of different embodiments are intended to fall within the scope of the invention and form different embodiments, as will be understood. by someone who is trained in this field. For example, in the following claims, any of the claimed embodiments can be used in any combination.
    Numerous specific details are set forth in the description given here. However, it is understood that embodiments of the invention can be worked out without these specific details. In other cases, well-known methods, structures and techniques were not shown in detail in order not to obstruct the understanding of this description.
    When in embodiments of the present invention reference is made to an application, reference is made to a functionality (or a set of functionalities) that is made available to a user and that is referred to as a request for access, reference is made to a network packet received from a user who asks the application to perform an action. When in embodiments of the present invention reference is made to a response, reference is made to a network packet sent to the user with the result of the application action. When in embodiments of the present invention reference is made to a request type, reference is made to a specific functionality or action of the application defined by the fields in a request.
    When in embodiments of the present invention reference is made to deep packet inspection, reference is made to the act of collecting information about a network package by inspecting the entire package and reconstructing its protocols. This relates to the analysis of transport packages where the data of the package is inspected. Such data may include data at the application level to somehow identify aspects of the application used. For example, within an HTTP application, the packets can be subdivided into requests and responses, while the requests and responses can even be further subdivided into types, where a response of a certain type corresponds to a request of the same type. The requests and responses can typically be correlated with application functions or defined criteria of an application.
    When embodiments of the present invention refer to network traffic, reference may be made to an amount and / or type of traffic on a particular network. Monitoring network traffic may include monitoring bandwidth management of a network.
    When in embodiments of the present invention reference is made to resource use, reference is made to the use of resources, e.g. CPU, disk space, memory, network, etc. on a server.
    When in embodiments of the present invention reference is made to a simple agent, reference is made to an agent that periodically reads the kernel statistics about the resource use of the machine.
    When in embodiments of the present invention, reference is made to kernel statistics about the resource use of the machine, but also "hooking" or instrumentation used to collect information about the services running on the server.
    When in some embodiments of the present invention reference is made to a node, reference is made to data equipment, e.g., an active hardware or software device, connected to a network and capable of transmitting, receiving or forwarding information through a data channel. An example of a hardware device is a host computer, for example a router, workstation or server. A specific software or firmware can run on such a hardware device. Alternatively, reference can also be made to a software device in a network, e.g. when a Virtual Machine is being considered.
    In a first aspect, the present invention relates to a system for monitoring network traffic and resource use in a data center, e.g. for a specific application. It is thereby an advantage of embodiments of the present invention that it can be determined for applications whether waiting time problems are caused by problems with network traffic and / or problems with resource use so that the application or the way in which it is processed in the network and / or the data center can be processed. be optimized. According to embodiments of the present invention, the system comprises a node comprising input means for receiving or capturing network packets. The node further comprises processing means to run a packet analyzer to analyze the network packets where packets are classified into request / response pairs by type based on deep packet inspection. Such request / response pairs can be combined with time stamps. They can be stored in a memory. The system also includes central processing means that collect the request / response pairs and receive resource usage information from at least one node. The at least one node can therefore be equipped with a simple agent adapted to provide information about resource use to the central processing means. The central processing means further comprises a correlation module for correlating request / response pair information with resource use. The system according to the embodiments of the present invention can be hardware or software implemented.
    By way of example, wherein embodiments of the present invention are not limited thereto, an exemplary system according to specific embodiments will now be described with reference to FIG. 1 to FIG. 5, showing standard and optional features of such systems.
    According to embodiments of the present invention, the system 100 comprises a node 101. The node comprises input means 102 for capturing network packets. Client requests enter the data center at the task distributor. In embodiments of the present invention, all network traffic is duplicated to the node 101 including the input means 102. In embodiments of the present invention, the node 101 typically comprises a network interface that receives the data sent and received by the task distributor. The task divider can thereby typically be a separate node for which the network interface is mirrored to the network interface at node 101.
    The same node 101 also includes processing means 103 to run a packet analyzer to analyze the network packets wherein packets are classified into request / response pairs by type based on deep packet inspection and wherein the request / response pairs are stored together with time stamps. An example of such a package analyzer is described later with reference to FIG. 4. A time stamp may advantageously be included for the last package of a given request and another time stamp may be included for the first package of the response to the request in question. The delay between the two is generally referred to as the data center response time to that specific request type under the specific load conditions.
    According to embodiments of the present invention, the system 100 also includes means 111, e.g., an element including processing power to receive resource usage information from at least one server. Such a server can be adapted to receive information thereof. The means 111 for receiving information about resource use may be part of central processing means. It can be referred to as an external monitoring system. Such processing means may be the same as processing means 103 for collecting the request / response pairs via the packet analyzer, although the latter is not strictly necessary. The at least one server 106 from which the information is received can typically run a simple agent to transfer resource usage to the element to receive resource usage information. The simple agent can be considered part of the system 100 or the system 100 can be adapted to receive information from the simple agent. FIG. 5 is a schematic illustration of the functionality required on such a server 106 to monitor the server system statistics. The kernel provides an interface to collect statistics from the server system and an extra simple agent is needed to transfer the system statistics to the monitoring system, e.g. the element to receive resource usage information. It is an advantage of embodiments of the present invention that only a simple agent at Server level should be active, the simple agent hardly increasing the load on the server.
    The system also includes a correlation module 112, e.g. running on the node 101, to correlate request / response pair information with resource use. This typically runs in conjunction with the packet analyzer and can typically run independently of the source monitoring system 111.
    To support high request speeds, only very fast package inspection is advantageously used. This makes the traffic analysis scalable for high request speeds. During the analysis, the data is advantageously immediately classified and stored efficiently. A data storage 113 can be provided. FIG. 3 is a schematic illustration of a network stream captured by input means 102 (the capturing network interface). Before being analyzed via deep packet inspection by the packet analyzer, the packets coming from the network stream can be filtered based on parameters in the set based on a set of TCP ports. After filtering, the package analyzer performs a deep packet inspection on the filtered packages, as illustrated in FIG. 3 according to an exemplary embodiment of the present invention. FIG. 4 illustrates an exemplary implementation of the package analyzer running in the processing means 103 according to the present invention. In a first step, packages are divided into request and response packages. If a network package is a request package, it is examined whether the package is the first package of the request (i.e. begins with GET, HEAD, POST, PUT, DELETE, TRACE, OPTIONS, CONNECT, PATCH, etc.). In that case the URL and host are parsed based on the packaged data and the package is classified in a request type. The request type and time stamp are then identified. These can, for example, be stored in the memory 104 contained in the node 101 together with the request. The overview in which the request type and time stamp are stored is indexed on client IP and client port according to an exemplary embodiment of the present invention. In case the request package is not the first package of the request, the time stamp of the request can be changed to the moment the last package of the request was received.
    In case the network package was a response, it is checked whether the request was the first package of the response (i.e. starts with HTTP). If yes, the time stamp of the answer will be updated. Subsequently, the entry is removed from the overview and sent to the element that receives information about resource use or a processing element. The element or a processing element aggregates these results every so often, e.g., every x seconds, where x is a value between 1 second and 300 seconds, preferably between 1 second and 60 seconds. The result is a flow rate and a waiting time per request type. In case the package is not the first package of the answer, the package is ignored. Collecting only limited information allows a large reduction in storage capacity to store such information. In one specific example, embodiments of the present invention are not limited thereto, only the URL and host name are stored in the first step. This usually has a size of less than 200 bytes. Starting from an HTTP page of about 5 kbytes, only a fixed amount of data (e.g., waiting time and bandwidth of about 40 bytes) means stored, regardless of the amount of traffic being e.g. 1 kb / sec or 1 Gb / sec. A memory 104 that provides storage capacity for storing the request / response pairs may be contained in the node 101 previously mentioned.
    In embodiments of the present invention, response times of all requests are measured to quantify the performance observed by the user. Response time data from different requests can be combined in an efficient data structure, namely a waiting time histogram, and such information can be offered to the user. The waiting time histogram is stored / can be stored in the memory 104 in the node 101 or in the central processing means - if it is not part of the node 101.
    The package analyzer processes the packages that enter or leave the data center. It is an advantage of embodiments of the present invention that a high flow rate of network packets can be analyzed while the required information is kept.
    To accomplish this, embodiments of the present invention may generally limit the analysis of network packets by the packet analyzer. An example of this is the analysis of the network packages, taking into account one or more, preferably all, of the following principles: - Packages are processed in order of arrival and they are not reordered or reconstructed. - The bytes or data in a package are only partially parsed to search for the required fields. In one embodiment of the present invention, packets are filtered based on the network service defined by the TCP port number in the TCP header. For example, parsing only on the TCP port number can only track HTTP messages. In yet another, or the same, exemplary embodiment of the present invention, the application header is parsed for required fields. For example, in the case of the HTTP protocol, the application header is parsed for URL and host information. - The URL or host information may be present in a subsequent package in the stream (identified by the TCP port). The stream (identified by the. TCP port) can be highlighted and the following packets in the stream can be analyzed to search for this information.
    Two exceptions may apply. First, the host package arrives before the start of the same request, but since we are on a LAN, it is very unlikely that this will happen.
    Secondly, the host information is spread over 2 packages. Since host names are short compared to package sizes, the chance of this happening is small. In both cases the information will be lost.
    In addition, in order to be able to analyze network packets with a high flow rate, the storage needs to be met by the memory 104 in node 101 are reduced to a minimum by the following steps: - Defining the request types in advance. This can be done by the user at the start of the data center analysis. - Classifying the packages as a request type, immediately upon arrival of the package. This results in a high compression of the amount of data, namely from strings (hostname, URL) to integer. Storing request rates, waiting times and error rates in a configurable interval, advantageously a short interval, for example every 10 seconds. The data can be stored on a disk. This data includes, for example, the waiting time histograms and bandwidths for each request type. Based on this, no storage is required per network package, which reduces the general storage needs.
    In addition, to provide support in analyzing network packets at a high flow rate, the complete URL and host information can optionally be collected through adaptive sampling. For example, if an application receives 10,000 requests per second, storing all requests takes up too much space, with complete information (in addition to storing the waiting time histogram and bandwidth of the other requests). This can be done per request type, so that the user has a complete package example for each request type. - For example, only 10-100 requests per second are completely stored. The current request speed is used to determine the sampling rate.
    According to embodiments of the present invention, both network traffic and resource use for an application are monitored. The input means 102 thereby capture all network packets (monitoring of network traffic). The input means 102 are illustrated in FIG. 3 as the capturing network interface. A simple agent on a server 106, illustrated in FIG. 4, transfers the resource use to the element that receives the information about resource use (monitoring of resource use).
    Embodiments of the present invention help to understand what is happening in the application by correlating the network information with the resource use of all hardware components.
    Said embodiments rely heavily on the network capture by node 101 comprising the input means 102 and the packet analyzer: this includes categorizing each request into a predetermined request type and measuring the waiting time for each request / response pair. In embodiments of the present invention, the entire network stream is captured and analyzed as opposed to other inventions where only a limited set is "sampled". It is an advantage of embodiments of the present invention that the embodiments are scalable for high flow rates. Embodiments of the present invention do indeed emphasize large applications. Therefore, embodiments of the present invention identify through the packet analyzer the request type using as little data and packet reconstruction as quickly as possible. Embodiments of the present invention enable the correlation between network traffic and resource use since a complete analysis of the packet analyzer. This requires a non-sampled, fast and accurate network analysis at the application level. Unlike the prior art, the present invention differs from other network capture instruments since embodiments of the present invention apply deep packet inspection, do not reconstruct TCP streams, and immediately categorize the stream into a predetermined request type. The correlation module relates the network traffic to the use of resources.
    The correlation module, possibly running on processing means, correlates resource usage with the fine-grained request information to determine how many resources each request type uses. Both sets of data are aligned over time and linear regression is used to determine the component of each request type in resource use. The system can therefore also comprise output means for exporting the results obtained. FIG. 2 is a schematic overview of the data components required for the correlation module to operate in accordance with an exemplary embodiment of the present invention.
    The fine-grained request information that gives the number of requests per second for each request type is information that comes from the package analyzer.
    The resource usage is information that comes from the at least one server 106 on which a simple agent runs for transferring resource usage to the central processing means 105.
    Both the fine-grained request information and the resource use are provided with a time stamp so that the correlation module can correlate them. By way of illustration, the graph in FIG. 2 the relationship between the number of requests and the use of sources. In the illustrative graph, only 2 dimensions are shown, but a real correlation shows at least 2 and possibly more dimensions.
    In certain embodiments of the present invention, the system includes an interface to visualize the measurement results. In addition, by automatically aggregating data, the number of data points required to visualize a specific time period is kept within limits. The data can be aggregated with an interval. For example, 10 seconds of data can be 24 hours. If a user is looking for data for a long period, a higher interval can be used in this way (for example 15 minutes instead of 10 seconds), so that the number of data points displayed is limited. Finally, servers are grouped into logical groups to make the data easier to visualize. The system may comprise output means for exporting the results obtained.
    In a second aspect, the present invention relates to a method for monitoring network traffic and resource use in a data center. The method comprising the following steps of capturing network packets entering or leaving the data center through node input means, analyzing the network packets by a packet analyzer running on processing means where the network packets are classified into request / response pairs by type and this based on deep packet inspection, obtaining information about resource use from at least one server in central processing means, and correlating the information about request / response pairs with resource use on the central processing means. By way of illustration, embodiments of the present invention are not limited thereto, an exemplary method is shown in FIG. 8 to illustrate standard and optional features of embodiments of the present invention.
    In a first step 810, the network packets entering or leaving the data center are captured by the input means 102 of node 101. In embodiments according to the present invention, all network packets entering or leaving the data center can be captured by entering the input means 102 instead of the network traffic. sampling.
    In certain embodiments of the present invention, a subsequent step 820 is included. This step 820 filters the incoming packages using a filter based on the contents of the packages. In an exemplary embodiment of the present invention, the filtering is performed based on the header of the package, more specifically based on the overhead of the transport layer. For example, the filtering can be performed on a TCP port that
    In a next step 830, the data of the captured, possibly filtered, packages are analyzed via deep packet inspection. A package type and time stamp can be assigned to the packages. In embodiments according to the present invention, assigning a type to a network package is based on data from the application layer. The types are defined by the user. By distinguishing between time stamps between requests and responses of the same type, the response delay can be calculated.
    In a next step 840, the data (request / response types and time stamps) can be combined in a wait time histogram.
    On the source monitoring side, a first step 850 comprises collecting and sending the server system statistics by a simple agent to the element for receiving information about resource use or to processing means. All this information is collected by the central processing means for further processing in step 850.
    According to embodiments of the present invention, the fine-meshed request information and the resource usage are correlated in step 870. This makes it possible to correlate network traffic with resource usage even up to the message level within the application.
    The above-described embodiments of the system for monitoring network traffic and resource use in a data center may correspond to an implementation of the embodiments of the method for monitoring network traffic and resource use in a data center as a computer-implemented invention in a processor. A configuration of such a processor comprises, for example, at least one programmable computer component coupled to a memory subsystem comprising at least one memory form, e.g. RAM, ROM etc. It is noted that the computer component or computer components may be a computer component for general use or specific use, and may be intended, for example, to be incorporated into an apparatus, e.g., a chip that has other components that the present invention is implemented in digital electronic circuits, or in computer hardware, firmware or software, or in combinations thereof. Each of the method steps may, for example, be a computer-implemented step. In other words, although a processor per se comes from the state of the art, a system containing the instructions to implement aspects of the methods for monitoring network traffic and resource use in a data center for a specific application does not come from the state of the technique.
    The present invention also includes a computer program product that provides the functionality of any of the methods of the present invention when executed on a computer device.
    In another aspect, the invention relates to a data carrier for carrying a computer program for monitoring network traffic and resource use in a data center. Such a data carrier may comprise a computer program product tangibly contained therein and carry machine-readable code to be executed by a programmable processor. The present invention therefore relates to a carrier medium that carries a computer program product which, when executed on computer means, provides instructions for performing any method described above. The term "carrier medium" refers to any medium that plays a role in providing instructions to be executed to a processor. Such a medium can take many forms, including but not limited to non-volatile media and transmission media. Non-volatile media are, for example, optical or magnetic disks, such as a storage device that is part of mass storage. Known forms of computer-readable media are a CD-ROM, a DVD, a flexible disk or floppy disk, a tape, a memory chip or cassette, or any other medium that can be read by a computer. Various forms of computer-readable media may be involved in executing one or more sequences of one or more instructions for a processor to execute. The computer program product can also be
    Transmission media can take the form of sound or light waves as they are generated during radio wave or infrared data communication. Transmission media include coaxial cables, copper wire, and fiber optics, including the cables that form a bus within a computer. Embodiments of the method of the present invention can also be implemented as an application that can be run. Such an application can be represented via a user interface, e.g. a graphical user interface, and can provide the user with an output indicative of network traffic and resource use, with respect to a specific other application being run. The output can include waiting time information.
    The present invention further relates to a data center embedded in a network, the data center comprising a system for monitoring network traffic and resource use of the data center, e.g. with regard to a specific application or components thereof running on the data center and in the network. The system thereby corresponds to a system as described in the first aspect of the present invention.
    权利要求:
    Claims (15)
    [1]
    CONCLUSIONS
    A system (100) for monitoring network traffic and resource use in a data center, the system comprising: - a node (101) comprising input means (102) for capturing network packets, processing means (103) for running a packet analyzer for analyzing the network packets where packets are classified into request / response pairs by type based on deep packet inspection, and a memory (104) to store the request / response pairs, - means to request / answer pairs and to receive resource usage information from at least one server (106), and a correlation module to correlate request / response pairing information with resource usage.
    [2]
    A system (100) according to claim 1, wherein the input means (102) capture substantially all network packets from the network traffic and wherein said substantially all network packets can be processed by the packet analyzer.
    [3]
    A system (100) according to any one of the preceding claims, wherein the network packets are first filtered based on packet information and the packet analyzer only processes the filtered packets.
    [4]
    A system (100) according to any of the preceding claims, wherein only the HTTP packets are passed through the filter, filtering on the TCP port data, and wherein the packet analyzer parses URL and host information of the filtered packets.
    [5]
    A system (100) according to any one of the preceding claims, wherein response times of different requests of the same type are combined in a wait time histogram.
    [6]
    A system (100) according to any one of the preceding claims, wherein the package analyzer processes the packages in order of arrival and / or wherein the package analyzer processes the packages and classifies them according to user-defined request types.
    [7]
    A system (100) according to any one of the preceding claims, wherein data in the requests completely stores per second per request type to retrieve complete URL and host information.
    [8]
    A system (100) according to any one of the preceding claims, wherein the system further comprises a monitoring unit to display the data center load versus the speed of request / response pairs.
    [9]
    A system (100) according to any one of the preceding claims, wherein the system is implemented as a computer device or the system is implemented as an application.
    [10]
    A method (800) for monitoring network traffic and resource use in a data center, the method comprising the steps of: - capturing network packets (810) entering or leaving the data center through input means (102) of a node (101) - analyzing the network packets (830) by a packet analyzer running on processing means (103), the network packets being classified into request / response pairs by type and this based on deep packet inspection, - obtaining information about resource use (850) of at least one server (106), - correlating (870) the request / response pair information with the resource use.
    [11]
    A method (800) according to claim 10, wherein all network packets are captured in the capturing step (810) and all captured network packets are analyzed in the analysis step (830).
    [12]
    A method according to any of claims 10 to 11, wherein the captured network packets are filtered (820) based on packet information and wherein the packet analyzer only processes the filtered packets (830).
    [13]
    A method according to any of claims 10 to 12, wherein only the HTTP packets pass through the filter in step 820, filtering on the TCP port data, and wherein the packet analyzer parses URL and host information from the filtered packets from step (830) and / or wherein the method comprises combining response times of different requests into a wait time histogram.
    [14]
    A method according to any of claims 10 to 13, wherein in at least some of the steps the packages are processed in order of arrival.
    [15]
    A data carrier comprising a set of instructions to monitor, when executed on a computer, network traffic and resource use in a data center according to a method according to any of claims 10 to 14.
    类似技术:
    公开号 | 公开日 | 专利标题
    BE1022604B1|2016-06-15|EFFICIENT MONITORING OF A DATA CENTER
    US9565076B2|2017-02-07|Distributed network traffic data collection and storage
    US9665420B2|2017-05-30|Causal engine and correlation engine based log analyzer
    EP1742416B1|2012-10-17|Method, computer readable medium and system for analyzing and management of application traffic on networks
    US7627669B2|2009-12-01|Automated capturing and characterization of network traffic using feedback
    US20180091394A1|2018-03-29|Filtering network health information based on customer impact
    Han et al.2002|The architecture of NG-MON: A passive network monitoring system for high-speed IP networks
    US10911263B2|2021-02-02|Programmatic interfaces for network health information
    US8005000B1|2011-08-23|Effective measurement/notification of SLA in a service oriented networked environment
    US10601639B2|2020-03-24|Multi cause correlation in wireless protocols
    US20190132377A1|2019-05-02|Dynamic socket qos settings for web service | connections
    WO2017196842A1|2017-11-16|Monitoring network traffic to determine similar content
    US9917747B2|2018-03-13|Problem detection in a distributed digital network through distributed packet analysis
    Kurt et al.2016|A network monitoring system for high speed network traffic
    CN110928934A|2020-03-27|Data processing method and device for business analysis
    CN111258971A|2020-06-09|Application state monitoring alarm system and method based on access log
    CN111124819A|2020-05-08|Method and device for monitoring full link
    CN104754328B|2017-01-25|Distributed video quality diagnosis method
    US20200296189A1|2020-09-17|Packet analysis apparatus, packet analysis method, and storage medium
    JP2018032983A|2018-03-01|Terminal device and communication monitoring method
    Uzun et al.2016|End-to-end internet speed analysis of mobile networks with mapReduce
    CN109284257B|2021-01-05|Log writing method and device, electronic equipment and storage medium
    US11283856B2|2022-03-22|Dynamic socket QoS settings for web service connections
    US20220006854A1|2022-01-06|Microservice manager and optimizer
    Oliveira2021|Near real-time network analysis for the identification of malicious activity
    同族专利:
    公开号 | 公开日
    US20150085695A1|2015-03-26|
    EP2852097A1|2015-03-25|
    EP2852097B1|2016-08-10|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US1822453A|1927-11-02|1931-09-08|Celanese Corp|Printing of fabrics containing organic derivatives of cellulose|
    US7996515B2|2005-06-15|2011-08-09|Bmc Software, Inc.|Network transaction discovery|
    US8656006B2|2006-05-11|2014-02-18|Ca, Inc.|Integrating traffic monitoring data and application runtime data|
    US7912947B2|2008-02-26|2011-03-22|Computer Associates Think, Inc.|Monitoring asynchronous transactions within service oriented architecture|
    US20100106538A1|2008-10-23|2010-04-29|International Business Machines Corporation|Determining disaster recovery service level agreements for data components of an application|
    CN102884828B|2011-01-24|2016-05-25|华为技术有限公司|A kind of method and apparatus that provides business specific resources to use information|
    JP6031597B2|2013-04-26|2016-11-24|株式会社日立製作所|Specific device, specific method, and specific program|US9054992B2|2011-12-27|2015-06-09|Solidfire, Inc.|Quality of service policy sets|
    US9838269B2|2011-12-27|2017-12-05|Netapp, Inc.|Proportional quality of service based on client usage and system metrics|
    US10324754B2|2013-11-07|2019-06-18|International Business Machines Corporation|Managing virtual machine patterns|
    US9584372B2|2014-01-07|2017-02-28|International Business Machines Corporation|Discovering resources of a distributed computing environment|
    US9798728B2|2014-07-24|2017-10-24|Netapp, Inc.|System performing data deduplication using a dense tree data structure|
    US10133511B2|2014-09-12|2018-11-20|Netapp, Inc|Optimized segment cleaning technique|
    US9671960B2|2014-09-12|2017-06-06|Netapp, Inc.|Rate matching technique for balancing segment cleaning and I/O workload|
    US9836229B2|2014-11-18|2017-12-05|Netapp, Inc.|N-way merge technique for updating volume metadata in a storage I/O stack|
    US20160142269A1|2014-11-18|2016-05-19|Cisco Technology, Inc.|Inline Packet Tracing in Data Center Fabric Networks|
    EP3745272A1|2015-02-02|2020-12-02|New Relic, Inc.|An application performance analyzer and corresponding method|
    US9720601B2|2015-02-11|2017-08-01|Netapp, Inc.|Load balancing technique for a storage array|
    US9762460B2|2015-03-24|2017-09-12|Netapp, Inc.|Providing continuous context for operational information of a storage system|
    US9710317B2|2015-03-30|2017-07-18|Netapp, Inc.|Methods to identify, handle and recover from suspect SSDS in a clustered flash array|
    US9740566B2|2015-07-31|2017-08-22|Netapp, Inc.|Snapshot creation workflow|
    US11025514B2|2015-10-30|2021-06-01|Nicira, Inc.|Automatic health check and performance monitoring for applications and protocols using deep packet inspection in a datacenter|
    US10554515B2|2015-12-31|2020-02-04|Bright House Networks, Llc|Customer premises network access device for displaying data usage|
    US10929022B2|2016-04-25|2021-02-23|Netapp. Inc.|Space savings reporting for storage system supporting snapshot and clones|
    US10642763B2|2016-09-20|2020-05-05|Netapp, Inc.|Quality of service policy sets|
    US11202179B2|2019-12-23|2021-12-14|Accenture Global Solutions Limited|Monitoring and analyzing communications across multiple control layers of an operational technology environment|
    法律状态:
    2021-06-28| PD| Change of ownership|Owner name: NEW RELIC, INC.; US Free format text: DETAILS ASSIGNMENT: CHANGE OF OWNER(S), ASSIGNMENT; FORMER OWNER NAME: COSCALE NV Effective date: 20210509 |
    优先权:
    申请号 | 申请日 | 专利标题
    EP13185462.2A|EP2852097B1|2013-09-20|2013-09-20|Efficient data center monitoring|
    EP131854622|2013-09-22|
    [返回顶部]