• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The present invention provides a system, method and apparatus for optimally determining, for a layout of windows on a display wall, the quality of display content to be displayed in each window without exceeding system resources. The display system receives display content from sources, with at least one source providing display content of multiple qualities. A received layout determines a placement of windows =, each window focused on displaying one source. An objective function defines the quality of display content shown in each window. It is optimized in order to determine an optimized layout that does not exceed system resources.
    公开号:BE1022444B1
    申请号:E2013/0861
    申请日:2013-12-19
    公开日:2016-03-31
    发明作者:Pieter Demuytere；Kristof R. Demeyere
    申请人:Barco Nv；
    IPC主号:
    专利说明:

    LAYOUT OPTIMIZATION FOR DISPLAY WALLS TECHNICAL AREA
    The present invention relates generally to display systems (e.g., display walls), and in particular to methods and systems for optimizing a layout of windows displayed on the display system.
    BACKGROUND
    Modern display systems often include display walls assembled from collections of display displays (e.g. monitors, televisions and / or projectors). Display systems are often assigned a layout that matches a specific use of the display wall. The layout describes a placement of windows where the corresponding sources offer display content of different qualities. Each window defines a surface on the display wall in which display content is visualized. Due to the limited system resources of the components of the display system, it is important to manage system resources by selecting the quality of display content in each window of the layout such that the display system is capable of displaying the display content as defined in the layout . For example, with display walls receiving streamed display content from the sources through a network, the display displays are dependent on and limited by the network to receive the display content sources. In addition, each display viewer has limited options and can only receive and assemble a certain amount of display content. A display operator may have to use lower-quality display content due to its limited capabilities.
    With display systems that have been assigned a small layout, including a small number of windows, it is possible to manually determine the quality of display content to be displayed in each window. However, as the number of windows in a layout increases, it becomes very time-consuming and extremely difficult to manually determine the quality of display content to be displayed in each window. It is also possible that a user mistakenly exceeds system resources (with the result that the display system is unable to display the layout) or assigns lower quality display content when the system has resources available to display higher quality display content.
    SUMMARY
    The present invention provides, for a layout of windows on a display wall, a system and method for optimally determining the quality of content to be displayed in each window without exceeding system resources.
    According to one aspect of the invention, for a layout of windows on a display wall of a display system, a controller is provided to optimally determine the quality of display content to be displayed in each window without exceeding system resources. The display system receives display content from at least one source, wherein at least one of the at least one source provides display content of multiple qualities. Display content uses system resources, and high-quality display content uses more system resources than low-quality display content. The controller is arranged to receive the layout. The layout determines the placement of windows on the display wall, with each window focused on displaying display content from one source. The controller is also arranged to set up an objective function that defines the quality of display content displayed in each window. The controller comprises a processor. The processor is arranged to optimize the objective function in order to determine an optimized layout that does not exceed system resources. The controller performs the optimized layout.
    The objective function can be weighted to promote the selection of high-quality display content for windows with larger surfaces.
    In the objective function, the weight given to each window to promote the selection of high-quality display contents for windows with larger areas may be proportional to the area of the window.
    Optimizing the objective function may include maximizing the objective function so that a maximum of system resources is used for the optimized layout without exceeding the system resources.
    The objective function can be a linear objective function which is maximized with the aid of integer programming.
    The objective function can be maximized with the help of binary integer programming.
    Prior to optimizing the objective function, limitations of the objective function can be established and the objective function can be optimized within the limitations.
    The display system can receive display content over a network and one restriction can be that a bandwidth required to receive the entire display content does not exceed a maximum allocated bandwidth of the network.
    During optimization, the objective function cannot assign a quality level to the display content in a given window such that display content is not displayed in the given window.
    Prior to optimizing the objective function, limitations of the objective function can be set, the objective function can be optimized within the limitations, and one limitation can be that each window is assigned a quality level that includes displaying display contents in the window.
    The multiple qualities can be low quality and high quality.
    The multiple qualities can be distinguished by at least one of the following: resolution, compression, color content, encryption or further processing requirements.
    The layout can determine at least one of the following: the number of windows, the position of each window, the shape of each window and the area of each window.
    The layout can be a layout determined by the user.
    Each source can be one of the following: a camera or a computer.
    The controller may include a processor adapted to execute instructions stored in a memory.
    The display system can receive display content from more than one source.
    The controller can calculate the window in the layout and the window area for each window in the layout.
    The controller can further be arranged to receive a proposed layout and the processor can further be arranged to use the proposed layout as a seed point for optimizing the objective function.
    The controller can further be arranged to provide instructions to the at least one source that specifies the quality of display content to be output.
    According to another aspect of the invention, there is provided a display system for optimally determining, for layout of windows on a display wall of the display system, the quality of display content to be displayed in each window without exceeding system resources. The display system comprises a controller as described above, at least one display display, and display wall. Each display viewer is responsible for displaying display contents on a specific area of the display wall. Each display viewer receives display content from at least one of the at least one source. The layout of windows is displayed on the display wall and display content is displayed in each window at the quality level specified in the optimized layout.
    The at least one display viewer can receive display content via a network. The system resources may include at least one of the following: a maximum network bandwidth, a maximum bandwidth of each display viewer to receive display content, a maximum decoding capacity of each display viewer to decode display content, and a maximum coding capacity of each source to code display content.
    The objective function can be limited such that for each display viewer a bandwidth required to receive the entire display content received by each display viewer over the network does not exceed the maximum bandwidth of that display viewer.
    The objective function can be limited such that for each display viewer a decoding capacity required to decode the entire display content received by that display viewer does not exceed the maximum decoding capacity available for that display viewer for decoding display content.
    The objective function can be limited so that for each display viewer, an encoding capacity required to encode all display content received by that display viewer does not exceed the maximum encoding capacity available for each source for encoding display content.
    The system can include at least two display displays.
    Front windows that include multiple display displays, the objective function can be limited to apply the same quality to each area of the window.
    At least one window in the layout can include display content that is displayed by at least two display displays.
    For windows that include multiple display viewers, the objective function can be limited to apply the same quality for each area of the window.
    At least one window in the layout can include display content that is displayed by at least two display displays.
    At least two of the display displays can have at least one of the following: a different screen resolution, a different number of pixels or a different pixel density.
    The display system can receive display content from more than one source.
    The objective function M can be defined as:
    where N is the number of display displays, row is the number of windows in the layout that the ie display display must display, wx is the weight factor given to low quality display content, wy is the weight factor given to high quality display content, where wx <wy, x, j = 1 if the f display viewer should use the low quality display content in the window, yÿ = 1 if the 1st display viewer should use the high quality display content in the window and a, y the surface of the window We is.
    The controller can calculate i / i ^ and a ,, for each window in the layout.
    If adjacent windows w-, j and wk / in the layout receive display content from the same source and window w-, j is on the f display viewer and wki is on the Ze® display viewer, the objective function M can be limited such that x , j = xk and yÿ = yM.
    The objective function M can be limited according to the following restriction:
    where bx is a bandwidth required to receive the low-quality display content, by a bandwidth required to receive the high-quality display content, and P is a maximum bandwidth that a display viewer has available to receive display content.
    The objective function M can be limited according to the following restriction:
    where bx is a bandwidth required to receive low-quality display content, by a bandwidth required to receive high-quality display content, B is a maximum bandwidth of a network available for transferring display content to the display system to wear. If the sources transfer the display contents using unicast, then X and Y can be equal to:
    If the sources transfer the display content using multicast, then X and Y can be equal to:
    where K is the number of sources and% = 1 if the window w-, j is associated with the / c® source and% = 0 if the window w-, j is not associated with the / c® source.
    The objective function M can be limited according to the following restriction:
    where dx is a decoding capacity required to decode the low-quality display content, dy is a decoding capacity that is required to decode high-quality display content, and D is a maximum decoding capacity that a display viewer has available to decode display content.
    The display wall can be an IP display wall and the sources are streamed to the display system via a network.
    Each display display can be one of the following: a projector, a monitor or a television.
    A method can be provided for optimally determining, for a layout of windows on a display wall of a display system, the quality of display content to be displayed in each window without exceeding system resources. The display system can receive display content from the at least one source, wherein at least one of the at least one source provides display content of multiple qualities. Display content can use system resources, and high-quality display content can use more system resources than low-quality display content. The method may include receiving the layout from the display wall. The layout can determine the placement of windows on the display wall and each window can display display content from one source. The method may also include establishing an objective function that defines the quality of the display content displayed in each window of the layout. The method may further include optimizing the objective function to determine an optimized layout that does not exceed system resources and executes the optimized layout.
    The objective function can be weighted to promote the selection of high-quality display content for windows with larger surfaces.
    The features of the present invention will be apparent with reference to the following description and accompanying drawings. In the description and drawings, certain embodiments of the invention have been disclosed in detail as being indicative of some of the ways in which the principles of the invention may be used, but it will be understood that the scope of the invention is not limited thereto.
    Features described and / or illustrated with respect to one embodiment may be applied in the same manner or in a similar manner in one or more other embodiments and / or in combination with or instead of the features of the other embodiments.
    BRIEF DESCRIPTION OF THE DRAWINGS
    Figure 1 is a representation of an exemplary layout displayed on a display system that includes a display wall;
    Figures 2A and 2B are block diagrams showing a display system;
    Figure 3 is a block diagram showing a method for optimally determining the quality of content of a layout to be displayed in each window without exceeding system resources;
    Figure 4 is a block diagram showing the steps for establishing limitations of the objective function based on system resources.
    DETAILED DESCRIPTION
    The present invention provides, for a layout of windows on a display wall, a system and method for optimally determining the quality of content to be displayed in each window without exceeding system resources. Optimally determining the quality of display content to be displayed in each window without exceeding system resources involves the creation of an objective function that defines the quality of display content displayed in each window. The objective function is optimized to determine an optimized layout that does not exceed system resources. In this way, the quality of display content displayed in each window of the layout can be determined such that a maximum of system resources is used without exceeding system resources.
    For smaller layouts with fewer windows, it is possible to manually determine the quality of display content to be displayed in each window. However, as the number of windows in a layout increases, it is very time-consuming and extremely difficult to manually determine the quality of display content to be displayed in each window. For example, when determining the quality of display content, a user must consider coding and decoding capacity of each component in the display system, total network bandwidth and bandwidth of the individual components in the system. Although a user may attempt to make optimum use of system resources, the user may mistakenly exceed system resources or allocate lower quality display content when the system has resources available to display higher quality display content.
    Looking initially at FIG. 1, an exemplary display system is shown which includes a display wall 100. A user-defined layout 110, including multiple windows 112, is displayed on the display wall 100. The display wall 100 allows visualization of a large number of windows 112 spread over the display displays 102, 104, 106, 108. The display wall 100 comprises four display displays 102, 104, 106, 108, arranged tilewise. The display displays 102, 104, 106, 108 are each responsible for displaying content on a certain area of the display wall 100. The display wall 100 is divided into four areas separated by dashed lines and labeled from 1 to 4, with the first display display 102 being responsible for displaying content in area 1, the second display display 104 for displaying content in area 2, the third display display 106 for displaying content in area 3 and the fourth display display 108 for displaying content in area 4. FIG. 2A shows an exemplary display system 10 in accordance with aspects of the present disclosure. The display system 10 receives content from at least one source 14 and includes at least one display display 16 and a controller 12. The at least one display display 16 can receive display content from a single source 14 ("an associated source"). At least one of the at least one source 14 provides display content of multiple qualities. A source 14 that provides display content of multiple qualities may, for example, provide display content at a low quality and a high quality. "Sources" as used herein are believed to provide display content of multiple qualities. However, as one skilled in the art will appreciate, the display system can receive display content from sources that do not provide multi-quality display content. The multiple grades can be distinguished by at least one of resolution, compression, color content, encryption or further processing requirements. The display of display contents by the system uses system resources, the display of high-quality display contents using more system resources than the display of low-quality display contents.
    The at least one source 14 can be one of a web camera, video camera, security camera, networked camera or computer. As one skilled in the art will appreciate, the at least one source 14 may include a source of video content.
    The display content provided by the at least one source 14 may be video content, still images, or any other form of content suitable for display on a display reproducer 16. The at least one source 14 may conveniently provide display content to the display system 10. Zi is shown in FIG. 2A, each source 14 is directly connected to at least one display displayer 16. Alternatively, FIG. 2B, the display system 10 is an IP display system (e.g., an IP display wall) wherein the sources stream display content via a network 20 to the display system 10. The at least one source 14 can moreover be configured to receive instructions from the controller 12 regarding the quality of display content to be provided to the at least one display reproducer 16. For example, in FIG. 2B the controller 12 is connected to the network 20, such that the controller 12 directs the quality of display content provided by the display displays 16 over the network 20. As one skilled in the art will appreciate, the controller 12 can be connected directly or appropriately to the sources 14.
    The display system 10 can receive display content from multiple sources 14. The display viewer 16 responsible for displaying the windows associated with the display content can receive the display content. A source can provide display content to one or more display displays 16 and the display content can be displayed in more than one display window.
    As one skilled in the art will appreciate, the at least one display display 16 may be a projector, television, monitor, or other suitable display. For systems comprising multiple display displays 16, two of the display displays 16 may have at least one of the following: a different screen resolution, a different number of pixels, a different aspect ratio or a different pixel density.
    The controller 12 optimally determines the quality of display content of the layout to be displayed in each window without exceeding system resources. As will be appreciated by those skilled in the art, the controller 12 can have different embodiments. For example, the controller 12 may comprise a processor or other suitable device such as a programmable circuit, integrated circuit, memory and I / O circuits, an application-specific integrated circuit, microcontroller, complex programmable logic device, other programmable circuits or the like . The controller 12 may also include a non-transitory computer-readable medium, such as directly accessible memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), or other suitable medium. Instructions for performing the method described below can be stored in the non-transitory computer-readable medium and executed by the processor. The controller 12 may be a separate physical component or may be combined with the at least one display displayer 16. For example, the controller 12 may be in one of the at least one display displayer 16, as a combination of multiple display displays 16 or in any other suitable manner. are carried out.
    The controller 12 can receive the layout that determines the placement of windows on the display displays 16. The layout can be user defined and can assign display content from one source to each window. The layout can determine at least one of the following: the number of windows, the position of each window, the shape of each window and the area of each window in the layout. Alternatively, the controller can calculate the number of windows, the position of each window, the shape of each window, and the area of each window in the layout. For example, the layout can determine the display viewer to which each window is assigned and the area of each window. Alternatively, the controller can determine the display viewer to which each window is assigned, and / or the area of each window. By way of example, in FIG. 1 the single window ("the split window") in the second display viewer 104 is also included in the first display viewer 102. The window can be defined as two separate windows in the layout, or the controller can determine that the window is included in two display displays and define the window as two separate windows. In FIG. 1, the split window is divided into a first display component and a second display component. The first display component comprises the portion of the split window displayed by the first display 102 and assigned to the first display display 102. The second display component comprises the portion of the split window displayed by the second display 104 and assigned to the second display indicator 104.
    The controller 12 sets up an objective function that defines the quality of display content displayed in each window of the layout, and optimizes the objective function to determine an optimized layout that does not exceed system resources. It is possible to assign quality to windows in the optimized layout (eg low quality or high quality) or to not assign quality to display contents to windows. For example, a window that receives display content from a source 14 that provides low quality or high quality content may be assigned a low quality, high quality, or no quality. For windows that are not assigned quality, display content is not displayed in the window.
    The objective function is optimized not to exceed the system resources. In this way the system resources function as a limitation on the objective function. The system resources can include a variety of different limitations within which the objective function is optimized. For example, the system resources may include at least one of the following: a maximum network bandwidth, a maximum bandwidth of each display reproducer 16 that receives display content, a maximum decoding capacity of each display reproducer 16 that decodes display content, and a maximum coding capacity of each source 14 that encodes display content. The objective function can be limited such that for each display display 16 the bandwidth required to receive the entire display content by the display display 16 over the network does not exceed the maximum bandwidth of the display display 16. Moreover, the objective function can be limited such that for each display reproducer 16 a decoding capacity required to decode the entire display content received by the display reproducer 16 does not exceed the maximum decoding capacity available for the display reproducer 16 to decode display contents. Similarly, another limitation may take into account a maximum encoding capacity available for each source to encode display content. In display systems 10 comprising several (i.e. at least two) display displays 16, the decoding capacity and / or the bandwidth of each display display 16 may be the same for each display display 16 or may be different within the display display 16.
    The objective function is set up to weigh certain features of the optimized layout. For example, the objective function can be weighted to promote the selection of high quality display content for windows with larger surfaces. The weight given to each window to facilitate the selection of high-quality display content may be proportional to the area of each window. As one skilled in the art will appreciate, the objective function can be set up to weight alternative embodiments for different characteristics.
    Optimizing the objective function may include maximizing the objective function such that, for the optimized layout, a maximum of system resources is used without exceeding the system resources. The objective function can be optimized within additional limitations of the system resources, prepared prior to optimizing the objective function. An additional limitation may be, for example, that a quality level is assigned to each window in the layout, such that display content is displayed in the window. For windows that include multiple display viewers, the objective function can also be limited to apply the same quality for the display content in each area of the window.
    The controller 12 can also receive a proposed layout prior to optimizing the objective function. The proposed layout can be used as a seed point for optimization of the layout. The term "seed point", as used herein, refers to the starting point from which layout optimization starts.
    The controller can execute the optimized layout. For example, the controller 12 may execute the optimized layout by instructing the at least one source 14 to provide display content to the associated, at least one display reproducer 16 of the quality defined in the optimized layout. The controller 12 can also output the optimized layout to a memory or database.
    With reference to FIG. 3, a block diagram is shown representing a method for optimally determining the quality of display content of a layout to be displayed in each window, without exceeding system resources. The controller 12 can perform any step of the method.
    A layout is received in process block 150. The layout can define the windows and the surface of each window in the layout. If the windows and window area are not defined, the system, in optional process block 152, can derive the windows and the surface of each window in the layout from the layout.
    An objective function is established in process block 154. The objective function can be a linear objective function that is maximized with the help of integer programming, e.g. binary integer programming. The limitations of the objective function can be, for example, linear or non-linear. For example, where the quality of display content is either low quality or high quality, the objective function M can be defined as described in the following equation:
    In the above equation, N is the number of display displays 16, η is the number of windows in the layout that the 1st display display 16 must show, We is the f window in the 1st display display 16, wx is the weight factor given to low-quality display content , wy is the weight factor given to high quality display content and ay is the surface of the window wy. In the comparison, xy = 1 if the 1st display viewer is to use the low-quality display content in the Wy window. Similarly, yy = 1 if the f display viewer is to use high quality display content in window wy As such, if yy = 1 for a given window wy, high quality display content is displayed in the window. Alternatively, if xy = 1 for a given window wy, low-quality display content is displayed in the window w, j. However, if x, j = O and y-, j = O, no display content is displayed in the wy window.
    As one skilled in the art will appreciate, the objective function Af can be extended to more than two quality levels, with each quality level having a different required bandwidth, coding resources, and decoding resources.
    The objective function Af is arranged in such a way that windows with larger surfaces a, y play a more important role than smaller windows. To give priority to high quality over low quality, the factors wx and Wy are chosen such that wx <wy.
    Maximizing the objective function Af results in a set of variables x ^ and y ^ for each Wÿ window in the layout. Consequently, the following inequality applies to the set of variables x, j and y, f.
    The above inequality expresses that, for a window w #, either low-quality display content, high-quality display content, or, if, for example, the system does not have resources to display display content in that window (ie x // and 3 ^ are both equal to 0), no quality of display content is assigned. If the variables xy and y, j are integers and can only take the values 0 or 1, the model can be a Binary Integer Programming model.
    Then, in process block 156, limitations of the objective function based on the system resources are established. The establishment of the limitations based on the system resources is further described in FIG.
    If desired, additional limitations of the objective function are established in process block 158. For example, layout restrictions can be set. The layout limitations are layout dependent and can be automatically deduced from analyzing the layout. For example, if adjacent windows v / jj and wM in the layout receive display content from the same source and window w-, j is on the f display viewer and wM is on the / c® display viewer, the objective function M is limited such that x # = xwen y, j = y μ. This limitation is based on the assumption that the window w »on the f display viewer and the window wm on the k® display viewer use the same source and should therefore use the same quality level.
    In process block 160, the objective function within the system resources and optionally additional constraints are optimized to define an optimized layout. In the optimized layout, the quality of the display content for the layout windows is indicated. The optimized layout can be displayed on the display wall in process block 162 if desired. FIG. 4 further describes the establishment of restrictions based on system resources. Display blocker limitations are set up in process block 170. For example, just as display displays 16 have limited bandwidth to receive display contents, display displays 16 may also have limited capacity to decode display contents. This limitation can be taken into account by limiting the objective function M according to the following equation:
    In the above equation, dx is a decoding capacity required to decode low-quality display content, dy is a decoding capacity required to decode high-quality display content, and D is a maximum decoding capacity that a display reproducer 16 has to display content. decode. The maximum decoding capacity D can be the same for all display displays 16 in the system 10 or can vary between display displays 16.
    Just as the display displays have limited capacity to decode display content, the sources 14 may have limited capacity to encode display content.
    Network restrictions are established in process block 176. For example, the objective function M can be limited according to the following equation:
    In the above equation, B is a maximum bandwidth of a network that is available for transferring display content to the display displays 16. The variables X and ν are dependent on whether sources 14 provide display content using unicast or multicast. Thus, when determining block 172 prior to determining network constraints, a check is made to determine whether the sources 14 are transmitting the display contents using unicast or multicast. If the display content is transmitted using unicast, the load for the network 20 can be calculated in process block 176 by counting the number of low and high quality streams received by all display displays 16. In this case, X and Y are equal to:
    Conversely, if the sources 14 transfer the display content using multicast, the processing is moved to process block 174. If the display content is sent using multicast, the load on the network 20 can be calculated by taking into account that if for example, one or more display displays 16 requesting a particular source 14 in low quality, which low quality stream need only be counted once. This fact is taken into account by determining s # * for each window in the layout. To accurately count sources 14 that provide display content in multicast, the layout with windows is analyzed and the parameter is derived with the following meaning: if Sÿk = 1, the window Wÿ is associated with the k® source and if% = 0 the window w-, j not associated with k® source. For each window w, j, therefore, exactly one s, jk parameter is 1, since the window Wÿ can only show one source at a time. Using s, ^, Xen / are equal to:
    In the above equation, K is the number of different sources 14 and each source 14 is indexed by k = 1 ... K. Thus, in process block 176, the above formula for X and Y in the multicast scenario counts the number of different sources 14 of low and high quality respectively that will be present on the network 20 when use is made of a specific set of xy and yy variables.
    Another exemplary network constraint on objective function M is described in the following equation:
    In the above equation, bx is a bandwidth required to receive the display content of low quality, is by a bandwidth required to receive the display content of high quality and Peen is a maximum bandwidth that a display reproducer 16 has available to receive display content. In the above comparison, each display reproducer 16 is limited such that the bandwidth of all received display contents is smaller than the maximal bandwidth of the display reproducer 16. The maximal bandwidth P can be the same for each display reproducer or can vary between display displays 16.
    As previously stated, wx expresses the preference of the system for low quality over high quality, while wy expresses the preference for high quality over low quality. The weight factors wx, wy can be assigned such values that wx <wy, i.e. the system prefers high quality over low quality. For example, the following values can be assigned to the weight factors: wx = 1 and wy = 2.
    The greater the wyl wx ratio, the more the system will prefer to assign high quality display content to larger windows. Example values for the weight factors are wx = 1 and wy = 1.2. The actual values for the weight factors wx and wy are configurable and can be adjusted to suit the preferences of the user and the application.
    The method of FIG. 3 can be performed repeatedly to achieve a desired result. For example, in cases where display content in at least one window is not assigned a quality level (i.e., no display content is displayed in at least one window), restrictions can be reconfigured such that display content is displayed in each window of the layout. In the example, the solution space for an optimized layout, in which no display content is displayed in at least one window, can be changed by reconfiguring the bandwidth bx and / or by such that it is possible to determine an optimized solution where display content is displayed in each window displayed. That is, for a given set of limitations, display content may not be displayed in every window, but by the ones shown in FIG. 3 repeatedly to perform the described method and reconfigure the bandwidth bx and / or by at each iteration, it is possible to find bandwidth values for which display content is displayed in each window of the layout.
    The method described above can also be used to represent display content that requires pre-processing before reception by a display reproducer 16 and display content that can be directly received and decoded by a display reproducer 16. For example, if x, y = 1, a pre-processing step is required prior to the reception by the f display viewer of the display contents to be displayed in window w-, j. Alternatively, if y # = 1, no pre-processing step is required prior to receipt by the 1st display operator of the display content to be displayed in window w. This formulation of the model can be used, for example, when scaling down display content by a networked scaler or transcoding the source by a networked transcoder. In these examples, the system can apply a two-phase approach by first calculating the display content that requires pre-processing, followed by enabling the required pre-processing.
    As described above, the network 20 is modeled as a single variable, the maximum bandwidth B that the network can support. If the topology of the network 20 is known, more complex scenarios can be taken into account. A more complex abstraction of the network 20 is, for example, possible if, for example, the topology of the network elements is known, the connectivity of each encoder and decoder is known and the path taken by each source in the network 20 is known. Knowing the topology of the network elements, e.g., switches and routers, may include knowing the connections between elements and the bandwidth available between the elements. Knowing the connectivity of each encoder and decoder may include knowing the sources 14 that transmit display content by each encoder, the network element to which each encoder is connected, each decoder that receives the display content, and the network element to which the decoder is connected. Knowing the path selected by a source 14 may include knowing the path taken by each encoder through the network 20 to reach each receiving decoder. With the help of this information it is possible, for example, to model the limitations on the amount of traffic that can flow between one network element and another, thereby preventing overloading of the links between network elements.
    Sources 14 that provide display content using multicast can be further modeled. For example, the number of simultaneous multicast groups that can be managed by the network 20 can be fixed. By modeling this as an additional constraint, the system 10 can prevent display displays 16 from requesting more multicast groups than the network can handle. This can be modeled in a limitation that is very similar to that of bandwidth.
    For all the windows in a layout, the area a, y can be calculated using an uncomplicated method, such as width multiplied by height of the window. There are also other, more sophisticated ways to calculate the area at / yte. For example, in some cases the pixel density of display displays 16 in a display wall is different. In such a case, the area a, y can be calculated by taking into account the pixel density of each display display 16. For example, the visible area of a window on a high-density LCD panel could be smaller than a window on a a projector cube, although a simple pixel calculation favors the window on the LCD panel. In this case, a user can obtain a better result by giving priority to assigning a higher quality to the window on the projector cube, although the actual number of pixels for that window may be smaller than that of the window on the LCD panel.
    The area a, y of a window w can be determined based on the area of the display content displayed in the window Wy as opposed to the area of the window w itself. For example, if the size of the window area a, y on the display display 16 does not match the aspect ratio of the source 14, a decoder can perform a scaling operation of the display content. Scaling can be performed such that the aspect ratio of the display content is maintained and the display content is displayed in the window using scaling down or scaling up and black edges to match it with the window dimensions. Alternatively, one distance from the display contents can be stretched to fit on the surface of the window. Maintaining the aspect ratio or stretching the display content may be a user preference which is taken into account during the calculation of the area a, y for a window. If the aspect ratio is to be maintained, the area a, y is not calculated as the window area, but instead, the area of the window is calculated to equal the portion of the window that is filled by the scaled source. That is, the area a, y of the window w, j can be calculated based on the portion of the window that is occupied after scaling down or scaling up the display contents to fit on the window.
    权利要求:
    Claims (39)
    [1]
    CONCLUSIONS
    A controller for optimally determining, for a layout of windows on a display wall of a display system, the quality of display content to be displayed in each window without exceeding system resources, the display system receiving display content from at least one source, at least one of which one of the at least one source provides multi-quality display content, displays display content using system resources, and displays high-quality display contents using more system resources than when displaying low-quality display contents, wherein: the controller is arranged to: receive, wherein the layout determines a placement of windows on the display wall, with each window focused on displaying display content from one source; establish an objective function that defines the quality of display content displayed in each window; the controller comprises a processor, the processor being adapted to optimize the objective function in order to define an optimized layout that does not exceed system resources; and the controller executes the optimized layout.
    [2]
    The controller of claim 1, wherein the objective function is weighted to promote the selection of high-quality display contents for windows with larger surfaces.
    [3]
    A controller according to any of claims 1-2, wherein, in the objective function, the weight given to each window to promote the selection of high-quality display content for windows with larger surfaces is proportional to the surface of the window .
    [4]
    Controller according to any of claims 1-3, wherein optimizing the objective function comprises maximizing the objective function, such that a maximum of system resources is used for the optimized layout without exceeding the system resources.
    [5]
    The controller of any one of claims 1-4, wherein the objective function is a linear objective function that is maximized using integer programming.
    [6]
    The controller of any one of claims 1-5, wherein the objective function is maximized using binary integer programming.
    [7]
    A controller according to any of claims 1-6, wherein limitations of the objective function are arranged prior to optimizing the objective function and the objective function is optimized within the limitations.
    [8]
    The controller of claim 7, wherein the display system receives display content via a network, wherein one limitation is that a bandwidth required to receive the entire display content does not exceed a maximum bandwidth of the network.
    [9]
    A controller according to any of claims 1-8, wherein the objective function during optimization may not assign a quality level to the display content in a given window, such that display content is not displayed in the given window.
    [10]
    The controller of claim 9, wherein limitations of the objective function are set prior to optimizing the objective function, the objective function is optimized within the limitations, and one limitation is that each window is assigned a quality level that displays display contents in the window.
    [11]
    Controller according to any of claims 1-10, wherein the multiple qualities are low quality and high quality.
    [12]
    Controller according to any of claims 1-11, wherein the multiple qualities are distinguished by at least one of the following: resolution, compression, color content, encryption or further processing requirements.
    [13]
    A controller according to any of claims 1-12, wherein the layout determines at least one of the following: the number of windows, the position of each window, the shape of each window and the surface of each window.
    [14]
    A controller according to any of claims 1-13, wherein the layout is a layout determined by the user.
    [15]
    The controller of any one of claims 1-14, wherein each source is one of the following: a camera or a computer.
    [16]
    A controller according to any of claims 1-15, wherein the controller comprises a processor adapted to execute instructions stored in a memory.
    [17]
    The controller of any one of claims 1-16, wherein the display system receives display content from more than one source.
    [18]
    The controller of any one of claims 1-17, wherein the controller calculates the windows in the layout and a surface for each window in the layout.
    [19]
    The controller of any one of claims 1-18, wherein: the controller is further configured to receive a proposed layout; and the processor is further adapted to use the proposed layout as a seed point for optimizing the objective function.
    [20]
    The controller of any one of claims 1-19, wherein the controller is further configured to provide instructions to the at least one source that specifies the quality of display content to be output.
    [21]
    A display system for optimally determining, for a layout of windows on a display wall of the display system, the quality of display content to be displayed in each window without exceeding system resources, the display system comprising: the controller according to any of claims 1 to 20 ; at least one display viewer, wherein: each display viewer is responsible for displaying display contents on a particular area of the display wall; each display viewer receives display content from at least one of the at least one source; and the display wall, wherein the layout of windows is displayed on the display wall and display content in each window is displayed at the quality level specified in the optimized layout.
    [22]
    The display system of claim 21, wherein the at least one display viewer receives display content via a network, the system resources comprising at least one of the following: a maximum network bandwidth, a maximum bandwidth of each display viewer receiving display content, a maximum decoding capacity of each display viewer having that display content decodes and a maximum encoding capacity of each source that encodes display content.
    [23]
    The display system of claim 22, wherein the objective function is limited such that for each display viewer a bandwidth required to receive the entire display content received by each display viewer over the network does not exceed the maximum bandwidth of that display viewer.
    [24]
    The display system of any of claims 22-23, wherein the objective function is limited such that for each display viewer a decoding capacity is required to decode the entire display content received by that display viewer, the maximum decoding capacity available for that display viewer to display content. decoding.
    [25]
    The display system of any of claims 22-24, further comprising at least two display displays.
    [26]
    The display system of claim 25, wherein the objective function, for windows comprising multiple display displays, is limited to apply the same quality for each area of the window.
    [27]
    A display system according to any of claims 25-26, wherein at least one window in the layout comprises display content that is displayed by at least two display displays.
    [28]
    A display system according to any of claims 25-27, wherein at least two of the display displays have at least one of the following: a different screen resolution, a different number of pixels or a different pixel density.
    [29]
    The display system of any one of claims 21 to 28, wherein the display system receives display content from more than one source.
    [30]
    The display system of any of claims 21-29, wherein the objective function M is defined as:

    where N is the number of display displays, n is the number of windows in the layout that the 1st display display must display, wx is the weight factor given to low quality display content, wy is the weight factor given to high quality display content, where wx <wy, x, j = 1 if the 1st display viewer should use the low quality display content in window w, j, y-, j = 1 if the f display viewer should use the high quality display content in window wÿ and a y is the surface of the window Wjj.
    [31]
    The display system of claim 30, wherein the controller calculates wy and ay for each window in the layout.
    [32]
    The display system of any of claims 30 to 31 wherein, if adjacent windows wy and wk | In the layout, display content is received from the same source and the window wy is on the ie display viewer and wki is on the ke display viewer, the objective function M is then limited such that xy = xki and yy = yk |
    [33]
    The display system of any of claims 30-32, wherein the objective function M is limited according to the following limitation:

    where bx is a bandwidth required to receive the low-quality display content, is a bandwidth required to receive the high-quality display content, and Peen is a maximum bandwidth that a display viewer has for receiving display content.
    [34]
    The display system of any of claims 30-33, wherein the objective function M is limited according to the following limitation:

    where bx is a bandwidth required to receive low-quality display content, by a bandwidth required to receive high-quality display content, B is a maximum bandwidth of a network available for transferring display content to the display system , if the sources transfer the display contents with the help of unicast, Xen is Vague to:

    if the sources transfer the display contents using multicast, X and Y are equal to:

    where K is the number of sources and sijk = 1 if the window Wy is associated with the ke source and syk = 0 if the window wy is not associated with the ke source.
    [35]
    The display system of any of claims 30-34, wherein the objective function M is limited according to the following limitation:

    where dx is a decoding capacity required to decode the low-quality display content, dy is a decoding capacity that is required to decode high-quality display content, and D is a maximum decoding capacity that a display viewer has available to decode display content.
    [36]
    A display system according to any of claims 21-35, wherein the display wall is an IP display wall and the sources are streamed to the display system via a network.
    [37]
    The display system of any one of claims 21 to 36, wherein each display display is one of the following: a projector, a monitor, or a television.
    [38]
    38. Method for optimally determining, for a layout of windows on a display wall of a display system, the quality of display content to be displayed in each window without exceeding system resources, the display system receiving display content from the at least one source, at least one of the at least one source provides multi-quality display content, displays system content uses system resources, and displays high-quality display contents uses more system resources than displays low-quality display contents, the method comprising: receiving the layout of the display wall, wherein the layout determines a placement of windows on the display wall and each window displays display content from one source; establishing an objective function that defines the quality of display content displayed in each window of the layout; optimizing the objective function to determine an optimized layout that does not exceed system resources; and executing the optimized layout.
    [39]
    The method of claim 38, wherein the objective function is weighted to promote the selection of high quality display content for windows with larger surfaces.
    类似技术:
    公开号 | 公开日 | 专利标题
    BE1022444B1|2016-03-31|LAYOUT OPTIMIZATION FOR DISPLAY WALLS
    US10218776B2|2019-02-26|Distribution of cloud services in a cloud environment
    CN106068495B|2021-02-26|Streaming multiple encodings encoded using different encoding parameters
    Aparicio-Pardo et al.2015|Transcoding live adaptive video streams at a massive scale in the cloud
    KR101595763B1|2016-02-19|Method for mapping media components employing machine learning
    Toni et al.2015|Optimal selection of adaptive streaming representations
    Maia et al.2015|A concise review of the quality of experience assessment for video streaming
    CN105164982B9|2019-03-08|The method and network element of the bandwidth allocation between stream are managed by assigning loss priority
    CA2981983A1|2016-10-13|Method and apparatus for automatic discovery of elements in a system of encoders
    US9444635B2|2016-09-13|Bandwidth allocation for multimedia conferencing
    Zanforlin et al.2014|SSIM-based video admission control and resource allocation algorithms
    JP2020074155A|2020-05-14|Latency reduction in map interface
    CN1527968A|2004-09-08|Method of running a media application and a media system with job control
    CN107846371B|2020-12-29|QoE resource allocation method for multimedia service
    WO2016040906A1|2016-03-17|System and method for controlling dynamic range compression image processing
    JP6758268B2|2020-09-23|Clients, programs and methods to determine delivery profiles based on displayable resolution
    US20210014547A1|2021-01-14|System for distributing an audiovisual content
    CN109218836A|2019-01-15|A kind of method for processing video frequency and its equipment
    KR20090091340A|2009-08-27|Web-based video browsing
    Farhad et al.2016|Dynamic resource provisioning for video transcoding in iaas cloud
    Bohez et al.2018|The crowd as a cameraman: on-stage display of crowdsourced mobile video at large-scale events
    Kantarcı2010|Bandwidth-effective streaming of educational medical videos
    WO2016164910A1|2016-10-13|Method and apparatus for automatic discovery of elements in a system of encoders
    US11128869B1|2021-09-21|Video encoding based on customized bitrate table
    US11055881B2|2021-07-06|System and a method for providing color vision deficiency assistance
    同族专利:
    公开号 | 公开日
    CN103886853B|2018-05-22|
    CN103886853A|2014-06-25|
    US9177402B2|2015-11-03|
    US20140173507A1|2014-06-19|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    JP2002014664A|2000-06-30|2002-01-18|Canon Inc|Picture processing device, picture processing method, and storage medium therefor|
    US20070120763A1|2005-11-23|2007-05-31|Lode De Paepe|Display system for viewing multiple video signals|
    US20080278629A1|2007-05-09|2008-11-13|Matsushita Electric Industrial Co., Ltd.|Image quality adjustment device and image quality adjustment method|
    US20090309808A1|2008-06-17|2009-12-17|Swingler Michael A|Providing a coherent user interface across multiple output devices|
    US20120299967A1|2011-05-23|2012-11-29|Canon Kabushiki Kaisha|Image output apparatus, control method for image output apparatus, and program|
    US5299284A|1990-04-09|1994-03-29|Arizona Board Of Regents, Acting On Behalf Of Arizona State University|Pattern classification using linear programming|
    US7278142B2|2000-08-24|2007-10-02|Veritas Operating Corporation|Dynamic computing environment using remotely allocable resources|
    KR20040068215A|2001-12-10|2004-07-30|코닌클리케 필립스 일렉트로닉스 엔.브이.|Method of and system to set a quality of a media frame|
    US20080313282A1|2002-09-10|2008-12-18|Warila Bruce W|User interface, operating system and architecture|
    US20050010388A1|2003-07-11|2005-01-13|International Business Machines Corporation|Dynamic online multi-parameter optimization system and method for autonomic computing systems|
    US8589140B1|2005-06-10|2013-11-19|Wapp Tech Corp.|System and method for emulating and profiling a frame-based application playing on a mobile device|
    US7873961B2|2005-07-29|2011-01-18|International Business Machines Corporation|Supervisory operating system for running multiple child operating systems simultaneously and optimizing resource usage|
    US20080062304A1|2006-09-07|2008-03-13|Claude Villeneuve|Method and apparatus for displaying at least one video signal on at least one display|
    US8082549B2|2006-11-08|2011-12-20|Board Of Regents, The University Of Texas System|System, method and apparatus for allocating resources by constraint selection|
    US8336055B2|2008-03-12|2012-12-18|International Business Machines Corporation|Determining the status of virtual storage in the first memory within the first operating system and reserving resources for use by augmenting operating system|
    US8195784B2|2008-05-30|2012-06-05|Microsoft Corporation|Linear programming formulation of resources in a data center|
    GB0901351D0|2009-01-28|2009-03-11|Univ Dundee|System and method for arranging items for display|
    US8892693B2|2012-10-02|2014-11-18|Nextbit Systems Inc.|Enabling fragment-based mobile device application streaming|
    US8954611B2|2013-03-21|2015-02-10|Nextbit Systems Inc.|Mechanism for sharing states of applications and devices across different user profiles|US9420237B2|2013-07-09|2016-08-16|Globalfoundries Inc.|Insight-driven augmented auto-coordination of multiple video streams for centralized processors|
    US9817627B2|2014-08-04|2017-11-14|At&T Intellectual Property I, L.P.|Method and apparatus for presentation of media content|
    CN104519321A|2014-12-22|2015-04-15|深圳市科漫达智能管理科技有限公司|Method and device for checking monitoring video|
    CN105916052A|2015-12-15|2016-08-31|乐视致新电子科技（天津）有限公司|Video frame drawing method and device|
    US10284492B2|2016-08-10|2019-05-07|Microsoft Technology Licensing, Llc|Control of casting to a media renderer|
    法律状态:
    优先权:
    申请号 | 申请日 | 专利标题
    US13/720,213|US9177402B2|2012-12-19|2012-12-19|Display wall layout optimization|
    US13720213|2012-12-19|
    [返回顶部]