• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    This invention relates to a predictive system for determining a set of subareas to be used for reasoning a virtual world of a computer graphics application, said subareas belonging to streamable objects to be used in rendering said virtual world, wherein said streamable objects each comprise a variety of sub-regions. The predictive system comprises predictive means capable of receiving from a computer graphics application information about the virtual world and obtaining a variety of predicted sets of sub-regions for rendering a virtual world using streamable objects, each predicted set of the aforementioned diversity is obtained by applying another predictive scheme, - a streaming manager suitable for receiving the predicted set of sub-areas, for comparing the predicted sets of sub-areas with information about sub-areas stored in memory and for performing, in accordance with the result of said comparison, control instructions regarding the set of subareas to be actually used.
    公开号:BE1022664B1
    申请号:E2015/5711
    申请日:2015-11-02
    公开日:2016-06-29
    发明作者:Bart Pieters；Charles-Frederik Hollemeersch；Aljosha Demeulemeester
    申请人:Graphine Nv；
    IPC主号:
    专利说明:

    Field of the Invention The present invention relates generally to the field of computer graphics applications and in particular to applications where a variety of objects are streamed for rendering a virtual world.
    Background of the Invention To achieve visually appealing results, computer graphics applications can render a variety of virtual objects on the screen, with each virtual object preferably having a high level of detail. The application will usually have multiple representations of those virtual objects in memory, for example, the geometry of an object or texture that is projected onto the geometry, each with a set of different levels of detail (LODs), so that the application has the LOD can select a display that leads to the best image quality. For example, a tree at a large virtual distance does not have to be rendered with high fidelity, so that the texture for that tree can have a low resolution. The virtual world rendered by the application can contain many more virtual objects and representations with all different LODs than can fit in the application's memory.
    A streaming module is a module that loads representations with different LODs of the virtual objects from a large but slow storage into a small but fast cache. This is done as soon as the virtual object is needed for processing or rendering. These representations are called streamable objects, for example a texture or a geometric model. To make a more fine-grained streaming possible, the streamable objects with their LODs are divided into sub-areas, so-called tiles, that are streamed by the streaming module. For example, textures can be divided into small, rectangular areas of pixels.
    Streaming tiles from storage to memory can take a considerable amount of time. If the tile from a rotating disk is streamed to the memory, the disk heads must be aligned, data read, tiles decompressed and copied to GPU texture memory. During this period the user may be confronted with a reduced quality of experience since the tile is not available for rendering. For example, the scene can be partially rendered without textures until the texture is available. The scene can be rendered with a tile of lower quality. However, as soon as the requested tile with higher quality is available, the user can see a virtual object whose rendering fidelity suddenly changes. In other words, the user can see extra detail 'appear on the screen' as soon as the tile is loaded.
    It is clearly desirable that the tiles necessary for rendering a scene are available in the cache memory and, preferably, are ready for use in the cache memory before or at the time the application first uses it. If the scene can only be rendered with tiles that are present in the cache, extra detail will not suddenly appear in the rendered results and the user can always enjoy the highest quality of experience.
    By predicting the working set of a virtual scene, i.e. the set of tiles to be used for rendering the scene, the cache can be optimized and the application does not have to wait until the tile is available for rendering. The time between a request for a tile and the moment that a tile is available for the application can be hidden and the user will never experience the abovementioned detailed detail.
    Precise prediction of the toolbox allows better use of bandwidth and computing power since streaming of unused tiles is avoided, but this is accompanied by increased computing complexity. Usually, however, no high-complex predictors (Eng. "Predictors") need to be performed to render a virtual scene, and low-complex predictors can return inaccurate results.
    To solve these problems, there is a need for improved tile prediction that also allows better scaling of the computing load of the predictions on the computer system.
    Summary of the Invention It is an object of embodiments of the present invention to provide a solution that contributes to optimizing the streaming of objects to be used for rendering a virtual world of a computer graphics application.
    The above object is achieved by the solution according to the present invention.
    In a first aspect, the invention relates to a predictive system for determining a set of subareas to be used for rendering a virtual world of a computer graphics application, said subareas belonging to streamable objects to be used with rendering said virtual world, wherein said streamable objects each comprise a variety of subareas. The predictive system includes - predictive means suitable for receiving information from a computer graphics application about the virtual world and for obtaining a variety of predicted sets of subareas for rendering a virtual world using streamable objects, each predicted set of the aforementioned variety is obtained by applying a different predictive scheme, - a streaming manager arranged to receive the predicted sets of subareas, for comparing the predicted sets of subareas with information about subareas stored in the memory and for executing, in accordance with the result of the aforementioned comparison, control instructions regarding the set of subareas that must actually be used.
    The proposed solution does indeed enable optimized streaming when rendering a virtual world from a computer graphics application. The streamable objects to be used for rendering are divided into a number of sub-areas, usually referred to as tiles in the art. By performing two or more predictions of the set of subareas to be used for rendering, the set of subareas to be used for rendering can be more precisely determined, just like the set of subareas that is used for rendering the future virtual scene of a virtual world. Multiple predictors allow a more precise control of a fine-grained streaming system for loading sub-areas into memory and make it possible to load them in time so that the application has the sub-areas from the first use, so that the image quality of the rendered scene is improved. Multiple predictors therefore also allow better scaling of the arithmetic load of the predictive system on the computer system.
    In a preferred embodiment, at least one streamable object is represented by sub-regions corresponding to a variety of detail level versions of the at least one streamable object. Thanks to different versions that correspond to different resolutions, the version that gives the best rendering result in the given circumstances can be selected at any time and a faster fine-grained streaming of subareas of streamable objects becomes possible.
    In one embodiment, the streaming manager is capable of using information about the accuracy of the variety of predictions to optimize the control instructions. This information can be taken into account when choosing the set of subareas that will actually be used for rendering.
    In a preferred embodiment, the streamable object is a texture to be projected onto an object used to render a scene of the virtual world. The subareas, usually referred to as tiles, are parts of the texture image.
    In another embodiment, the streamable object is a geometric model used to render a scene from the virtual world.
    In a preferred embodiment, the predictive means are implemented as a variety of predictive units that are each suitable for obtaining one of the predicted sets of sub-regions.
    In one embodiment, one or more of the predicted sets of subareas are obtained at their own pace. For example, predictions that require less calculation work can be performed more often than predictions with a high calculation load.
    In another embodiment, one of the predicted sets is obtained by determining which streamable objects or which sub-regions lie within a certain virtual distance in a virtual scene.
    Advantageously, at least one of the predictive schemes is arranged for performing a prediction in which a virtual camera position in the virtual world is used differently than a main position of the virtual camera for rendering the application. This makes it possible to predict tile visibility, since a future position of a virtual camera can be used.
    In one embodiment, at least one of the aforementioned predictive schemes includes rendering a scene from a virtual camera position and identifying a unique sub-area identification.
    In another embodiment, the streaming manager is capable of determining the set of subareas that is actually to be used by assigning a weight to the different subareas of the predicted sets. The weight can, for example, take into account the number of times a sub-area occurs in the variety of predicted sets. This makes it possible to prioritize streaming of subareas.
    In a preferred embodiment, the streaming manager is suitable for selecting memory locations to be reused and for forwarding an indication of the selected memory locations in the aforementioned control instructions.
    In a preferred embodiment, the application is a software application for a video game.
    In another aspect, the invention relates to a method for determining a set of subareas to be used for rendering a virtual world of a computer graphics application, said subareas belonging to streamable objects to be used with the rendering of that virtual world, wherein said streamable objects each comprise a variety of subareas. The method comprises - receiving a computer graphics application of information about a virtual world, - obtaining a variety of predicted sets of subareas for rendering that virtual world using streamable objects, each predicted set of the diversity being obtained by applying a different predictive scheme, - comparing the predicted sets of subareas with information about subareas stored in the memory and - executing, in accordance with the result of the comparison, control instructions concerning the set of subareas that are actually must be used.
    In another aspect, the invention relates to a program that can be executed on a programmable device comprising instructions that, when executed, perform the method as described above.
    In yet another aspect, the invention relates to a video game console comprising a predictive system as previously described.
    In order to summarize the invention and the realized advantages over the prior art, certain objects and advantages of the invention have been described above. It goes without saying that all such objectives or advantages are not necessarily achieved in accordance with a specific embodiment of the invention. Thus, for example, persons skilled in the art will recognize that the invention may be embodied or embodied in a manner that achieves or optimizes one advantage or group of benefits as described herein, without necessarily realizing other goals or benefits described or suggested herein. .
    The above and other aspects of the invention will become clear and further explained with reference to the embodiment (s) described below.
    BRIEF DESCRIPTION OF THE DRAWINGS The invention will now be further described, by way of example, with reference to the accompanying drawings, wherein reference numbers refer to the respective elements in the various figures.
    Fig. 1 illustrates an embodiment of a data caching system according to the invention.
    Fig. 2 illustrates texture mapping.
    Fig. 3 illustrates a 2D texture with different MIP levels.
    Fig. 4 illustrates a geometric model with different LOD levels, each of which is divided into tiles.
    Fig. 5 illustrates an embodiment of the proposed predictive system.
    Fig. 6 illustrates a timetable of a predictive system with two predictors.
    Detailed Description of Illustrative Embodiments The present invention will be described with reference to specific embodiments and with reference to certain drawings, but the invention is not limited thereto, but is only limited by the claims.
    In addition, the terms first, second, etc. are used in the description and in the claims to distinguish between similar elements and not necessarily for describing a sequence, either in time, in space, in importance or in any other way. It is to be understood that the terms used are interchangeable under proper conditions and that the embodiments of the invention described herein are capable of operating in sequences other than those described or illustrated herein.
    It is to be noted that the term "comprising" as used in the claims should not be interpreted as being limited to the means specified thereafter; it does not exclude other elements or steps. It must therefore be interpreted as a specification of the presence of the listed features, units, steps or components referred to, but it does not exclude the presence or addition of one or more other features, units, steps or components or groups thereof. Therefore, the scope of the expression "a device comprising means A and B" should not be limited to devices consisting only of parts A and B. It means that with regard to the present invention, the only relevant parts of the device A and B to be.
    References in this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or feature described in connection with the embodiment is included in at least one embodiment of the present invention. Statements of the phrase "in one embodiment" or "in an embodiment" at different places in this specification do not necessarily all refer to the same embodiment, but it is possible. Furthermore, the specific features, structures or characteristics may be combined in any suitable manner in one or more embodiments, as will be apparent to those skilled in the art from this disclosure.
    In a similar manner, it should be noted that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped in a single embodiment, figure, or description thereof to streamline the disclosure and understanding of one or more of the facilitate various inventive aspects. However, this method of disclosure should not be interpreted as an expression of an intention that the claimed invention requires more features than expressly stated in each claim. As shown in the following claims, the inventive aspects lie in less than all the features of a single preceding disclosed embodiment. Therefore, the claims that follow the detailed description are hereby explicitly included in this detailed description, wherein each claim stands on its own as a separate embodiment of the present invention.
    In addition, since some embodiments described herein include some, but not other, features included in other embodiments, combinations of features of different embodiments are intended to fall within the scope of the invention and to form different embodiments, such as will be understood by someone skilled in this field. For example, in the following claims, any of the claimed embodiments can be used in any combination.
    It should be noted that the use of particular terminology in describing certain aspects of the invention does not imply that the terminology herein is redefined to be limited to any specific features of the features or aspects of the invention with which that terminology is associated.
    In the description given here, numerous specific details are set forth. However, it is understood that embodiments of the invention can be worked out without these specific details. In other cases, well-known methods, structures and techniques were not shown in detail in order not to obstruct the understanding of this description.
    The present invention proposes a predictive system for optimizing a streaming system, wherein a variety of subareas of streamable objects is streamed to a memory of a computer graphics application to be used for rendering a virtual world. The proposed predictive system receives information about the virtual world from the computer graphics application and predicts, by means of a set of predictors, each using a different predictive scheme, a variety of predicted sets of subareas used for rendering the aforementioned virtual world using the aforementioned streamable objects. The predictions that thus arise are compared with information about sub-areas stored in the memory of a computer graphics application and a component of the streaming manager sends control instructions regarding the set of sub-areas that should actually be used for rendering the application.
    In most preferred embodiments, a solution is proposed for optimizing a streaming system that streams a variety of sub-areas, optionally with different levels of detail (LODs), to a memory in a rendering application. The subareas that can be streamed independently are referred to as tiles. The proposed approach directs streaming of a streaming module to the streaming cache by performing tile predictions that indicate which tiles the application needs for arithmetic operations such as rendering the current or future virtual scene. The required tile set is called the working set. The system preferably performs multiple predictors. The predictions can each be obtained with their own method / algorithm for predicting the current or future tile tool set, their own accuracy of the results and their own calculation complexity. The predictions can be performed at different frequency intervals to predict the tile tool set repeatedly and accurately.
    Fig. 1 is a high-level block diagram of an embodiment of a predictive system according to the present invention. The proposed system comprises predictive means for performing two or more predictions and a streaming manager and is connected to a computer graphics application, a streaming module and a cache memory. The predictive system can be part of a game system, personal computer, workstation, server system, part of a hardware or part of a software architecture. The application can be a rendering application that renders a 2D or 3D virtual world, or another application that requires processing capabilities for streamable objects. Streaming cache memory includes processor-readable memory to store streamable objects used by the application for rendering. It may be located in system memory, e.g., system RAM, or in GPU accessible memory, on a discrete GPU with specific memory, or in shared system memory accessible to the GPU. In response to requests it receives from the application and the streaming manager, the streaming module loads subareas of streamable objects, possibly with different levels of detail. It can be a specific component in hardware or a software module. The predictive system can be built into hardware (e.g. in a GPU), but can also be implemented as a module that is implemented in software on the system processor (s). A combination of both options is also possible.
    The application may be a video game software application, a general rendering application that renders a virtual scene. It can be an application in the field of virtual reality. It can render or process a single virtual scene or multiple virtual scenes simultaneously, for example for 3D stereoscopic viewing. The computer on which the application is running can be a video game console. Alternatively, the application may be performed on or the computer device may be integrated into, for example, a mobile telephone, a mobile device, an embedded system, a media console, a laptop, a desktop computer, a server and / or a data center. The application can render and / or display a virtual scene, or it can render a virtual scene and use the rendering step output to perform other calculations. The application can render a two-dimensional or a three-dimensional scene. The application can display a rendered scene directly on a screen, can send the results in the form of a video bit stream, or can process the rendered scene to perform other calculations. The application can be a rendering application designed to display and render the virtual world, the application can be designed to edit or create a virtual world, or both display and edit the virtual world. For example, the application may be an editor application for the virtual world of a video game capable of rendering the virtual world in an editor mode for editing the virtual world, and rendering the virtual world in an end user mode rendering the game such as when it would be executed, once production has been completed and, for example, sent to a gamer on a DVD-ROM. In this scenario, the invention can be used to predict the tile tool set and embed descriptions of the tile tool set into the game content that the streaming module should use once the game is run on the end user's device.
    The application pays for the scene with the help of streamable objects (SOs) from the memory. An SO can be a texture that can depict application to geometry. An SO can also be a geometric data, for example a mesh model that is constructed with polygons or points used by the application to render the scene. An SO can also be a more general fact, such as a key frame of an animation. Different detail level (LOD) versions of the same SO may be present. Each LOD version of the SO describes the same object to render, but with a different quality (or possibly a different amount of data for displaying the SO). A variety of LODs may be present in memory to render the scene using the LOD version of the SO selected by the application (e.g., the LOD that gives the best sampling result). For example, a low-accuracy model of a mountain with little geometric data (for example, a small number of polygons) can be used when rendering a mountain from a large virtual distance in the virtual world, while a very accurate model (with many polygons) can be used be close up when rendering the mountain. LODs can be streamed through the streaming module. LODs can be generated by the application and derived from available LODs from an SO. If an SO consists of geometric data, different LODs form different representations of the geometric model, each with a different quality, for example by means of more or fewer polygons for describing the geometric model. If an SO is a texture, LODs can include multiple representations of an image, each with more or fewer pixels for describing the image. If an SO consists of key frames for animation of the model, LODs can include multiple representations of the animation, each with more or fewer key frames for describing the transitions between the models in the animation. A LOD can therefore represent multiple representations of an SO each with a different quantization level. For example, if an SO is a texture, texture images with a smaller color palette can be saved.
    A brief overview of texture mapping is now provided as it can be performed in conjunction with embodiments of the invention. In Fig. 2, a texture sample is applied to an object to obtain a textured object. The texture sample is usually a graphic file that displays a texture that one wants to display on the object. The object as shown in Fig. 2 is a three-dimensional object. However, the object can also be a two-dimensional object. Using a predetermined manner known in the art, the sample is applied to the object to obtain the textured object. The textured object has the same shape as the object, but has the texture pattern on at least its visible surfaces. Sometimes a set of coordinates is set for each corner point of the shape, commonly referred to as UV coordinates. These coordinates determine which area of the texture is projected on the surface of the object. The coordinates usually have values between zero and one. For example, the front of the cube on the figure has four vertices, with UV coordinates [0,0], [0,1], [1,0] and [1,1], which indicate that the full texture on the surface is projected. For example, coordinates [0.0], [0; 0.5], [0.5; 0] and [0.5; 0.5] indicate that the first quarter of the texture is projected onto the surface. Applying the texture to the object to obtain the textured object is commonly referred to as texture mapping.
    For example, an SO can be a texture. A texture can contain any type of two or three-dimensional data, including (but not limited to) color data, normal surface vectors, and mirror reflection data. The texture data may include multi-resolution texture pyramids, MIP (multum-in-parvo) levels. The lowest level (level 0) of a pyramid is the image in full resolution. The next higher level (level 1) of a pyramid is the image in a lower resolution (2x2 pixels or texels at the lowest level, for example, are averaged to form one texel at the next highest level). At the next higher level, the image has an even lower resolution. The closer we get to the top of the pyramid, the more the resolution of the image decreases. Much less data is required to display the image in the lowest resolution than the image in the highest resolution. Each MIP level represents a different level of detail.
    An SO is divided into a variety of subareas, so-called tiles. Each sub-area describes a part of the complete SO. These subareas can be geometric subareas of an SO, e.g. 3D spatial regions of a geometric model or 2D spatial regions of a 2D texture. They may be general subchunks of memory from the memory buffer described by the SO. Since the SO contains different LODs, each LOD can be divided into sub-areas (tiles). A tile with a certain LOD can be displayed by one or more tiles on another LOD. Sub-areas can include a complete MIP level. In other words, a tile corresponds to a single level of the LOD pyramid. Note that tiles do not have to have a uniform size or sub-area layout. The use of tiles makes a more fine-grained streaming possible. There are many different ways available in the technology to divide the SO into subareas. Tiles can contain compressed or uncompressed texture data. The data can be compressed using a variable-length compression format, such as ZIP, or with a vector quantization method, such as DXT1 or ASTC. As those skilled in the art will recognize, other compression schemes are possible. The data can also be uncompressed. The data of a tile can be based on the data of other tiles to display a (part of a) SO. For example, a level 3 tile can contain data that is used to refine the data of a level 4 tile.
    If an SO is a texture, the texture is divided into tiles, i.e., into sub-areas of the texture image. If the texture contains multiple MIP levels, each MIP level can be divided into tiles. Figure 3 illustrates this for a two-dimensional structure. A tile on a certain MIP level can be represented by multiple tiles on a different MIP level. In Fig. 3, the tile of the highest level, i.e., the one with the lowest resolution level, is represented, for example, by four tiles on the next MIP, which has a higher resolution (lower level). Sub-areas can include a complete MIP level. In other words, a tile corresponds to a single level of the pyramid. The tiles do not all have to be the same size. Tiles can be 2D rectangular areas or 3D cubic areas. Two tiles can contain data from two different MIP levels, in other words they contain data with a different level of detail (LOD). Any MIP level, from the texture with the highest level of detail (i.e. the lowest level of the pyramid) to the texture with the lowest resolution (i.e. the highest level) of the texture can be divided into tiles. Tiles can include compressed or uncompressed texture data. The data can be compressed using a variable-length compression format such as JPEG or JPEG XR or with a vector quantization scheme such as DXT1 or ASTC. Other compression techniques are possible. The data of a tile can be based on the data of other tiles to display a (part of a) SO. For example, a texture tile at level 3 may contain data for refining the data of a tile at level 4, for example by containing only high-frequency image data that is added to the lower-resolution image data of the tile at level 4.
    A texture of 1024 by 1024 pixels may contain, for example, 2D color information. It has 11 MlP levels, ie the 1024x1024 image, level 0, a 512x512 level, ie level 1, a 256x256, this is level 2, 128x128, level 3, 64x64, level 4, 32x32, level 5, 16x16, level 6, 8x8, level 7, 4x4, level 8, 2x2, level 9 and a 1x1 level, level 10. This texture can be divided into 2D tiles with a size of 128 by 128 pixels. Level 0, the highest level of detail, contains 8x8, so 64, tiles. Level 1 contains 16 tiles, level 2 contains four tiles and level 3 one tile. All levels that are smaller than the tile size can possibly be omitted. In this example, all levels higher than 3 are omitted. One can therefore also choose to include levels that are smaller than the tile size in tiles. In this example, this means that level 4 with its image of 64 by 64 pixels would be included in a tile of 128 by 128, just like levels 5 to 10.
    If an SO is a geometric model, the model is divided into tiles, i.e. into sub-areas of the model. If the model contains multiple LOD levels, each level is subdivided into tiles. Figure 4 illustrates this for a three-dimensional mesh model that is made up of vertices and triangles. A tile on a certain LOD level can be represented by multiple tiles on a different LOD level. In Fig. 4, the tile at the highest level, i.e., the lowest resolution level, is represented, for example, by two tiles at the next LOD with a higher resolution and a lower level. Sub-areas can include a complete LOD level. Tiles can contain 2D rectangular areas or 3D cubic areas. Two tiles can contain data from two different LOD levels. Any LOD level, from the model with the highest level of detail (i.e. the lowest level of the pyramid) to the model with the lowest resolution (i.e. the highest level) can be divided into tiles. Tiles can contain compressed or uncompressed data. The compressed data can be compressed using a compression format such as ZIP. Other compression techniques are possible.
    For example, a geometric model as shown in Fig. 4 may have four LODs, level 0, level 1, level 2 and level 3. Each LOD represents the same model of a person, but with a different quality. Each LOD is subdivided into subareas, tiles, each containing a number of the vertices that describe the model. Level 2, for example, is divided into two tiles that each contain one half of the model. Level 1 is divided into four tiles, each containing a quarter of the model. Level 0 is divided into six tiles, each containing one sixth of the model.
    The streaming module loads SO tiles in its cache memory in response to requests it receives. Streaming SOs allows the use of a large number of virtual objects with associated SOs, for example, a large number of virtual trees in a virtual world, each with their unique textures. There can be many more virtual objects than would fit in system or GPU memory. These SO tiles are loaded into a fast streaming cache that the application can access. The streaming module can load the SO tiles from a slower storage medium, such as a hard disk. The storage system is usually larger than the cache, which explains why the streaming cache performs a caching function between larger but slower memory and the application. The requests that the streaming module receives are from the application or from other operators, for example, a person who has prepared a list of tiles to be loaded when editing the virtual world. Such operators can help the streaming system to stream the right tiles at the right time.
    Streaming a large number of virtual objects allows better management of bandwidth and computing power. The SOs of these virtual objects are streamed based on need. Streaming an unnecessary SO wastes computer resources, since an SO consumes valuable memory and processing of the SO to move it from storage to memory consumes processing cycles. By going further and streaming SOs on a tile basis, on-demand streaming becomes possible with much finer grain size and therefore better use of bandwidth and computing power. Streaming SO tiles from a virtual object may include reading the SO from a storage device or network and decompressing the read bit stream into a format that is understandable to the rendering system. The streaming module makes a distinction between LODs of different SOs. Multiple LODs of an SO can be requested and streamed to the memory and the set of LODs of an SO loaded in the memory can change constantly.
    For example, suppose that an SO is a geometric model of a virtual mountain. If the application gives a representation of the virtual world of someone standing on a mountain, an SO with the 3D mesh model of the mountain for a low-level version with high level of detail will be requested by the application to the streaming module. If the same application renders a view of the virtual world at a great distance from the aforementioned mountain, an SO with a 3D mesh model for a high-level version with a low level of detail will be requested by the application to the streaming module. In case of multiple mountains, the same SO can be requested and present in the memory with different LODs.
    To enable more fine-grained streaming, SO tiles are streamed through the streaming module. The application asks for tiles instead of complete SOs. Streaming SO tiles allows the use of virtual objects with associated SOs, SOs that are much larger than would fit in a system or GPU memory, for example. It enables better management of bandwidth and computing power. The tiles of these SOs can be streamed based on requirements. Streaming a tile while it is not needed is a waste of computer resources since a tile consumes valuable memory and is used for processing the SO processing cycles. The streaming module makes a distinction between tiles with a different LOD. Multiple tiles with different LODs can be requested and streamed to the memory and the set of LODs loaded in the memory can change constantly.
    Suppose, in another example, an SO is a texture of a 3D model of a virtual mountain. If the application displays the virtual world of someone standing on a mountain, tiles with low level LOD data, i.e. texture data with high detail level for the mesh model of the mountain are requested by the application to the streaming module. Only tiles that contain texture data from the front of the mountain can be requested. If the application renders a view of the virtual world at a great distance from the aforementioned mountain, tiles with high level LOD data, i.e. texture data with a low level of detail, may be requested by the application to the streaming module for the mesh model of the mountain. The tiles with texture data for the back of the mountain that is not visible to the virtual camera may not be streamed. In the case of several mountains, the same tiles can therefore be requested and be present in the memory with different LODs.
    The streaming module can receive its requests directly from an application or from a rendering subsystem or another operator (e.g., a predictive system or another application). The requests may contain information about the tile identification, including possibly LOD if tiles are combined with LOD. For example, the streaming module may receive tile requests for an SO texture based on a two-dimensional index and a LOD index. In another example, texture tiles are requested using a one-dimensional index and a LOD index.
    A brief overview of a texture streaming system is now provided, as it can be used in combination with embodiments of the invention. The streaming system, as shown in Fig. 1, is located between the application and the streaming cache. It receives requests for a tile from the application or another operator. It knows which tiles are present in the cache memory and it knows their position in the memory. When a tile is requested, the streaming system checks whether the tile is in the cache memory. If present, i.e., a cache hit, the streaming system immediately returns the information to the application so that the application can read the tile data from the streaming cache memory. The streaming system can return the tile address in memory and the tile LOD level. It can return the tile itself with its data. If the requested tile is not present in the cache memory, i.e. a cache miss, the streaming system streams the tile to the cache memory. Before streaming for the requested tile is completed, it can return the cache information of the tile for an LOD that is in the cache. The tile with the corresponding LOD can be returned. The answer 'tile not found' can be given. The set of all tiles requested by the streaming module for rendering a scene is called the working set. Below is a brief overview of the reasons for having a streaming cache and a predictive system. Those skilled in the art will recognize that a large number of streaming and tile-based streaming systems have been described in the art and that this invention is not limited to the systems described in this text.
    Streaming tiles from storage to memory can take a considerable amount of time. If the tile from a rotating disk is streamed to the memory, the disk heads must be aligned, data read, tiles decompressed and copied to GPU texture memory. During this period the user may be confronted with a reduced quality of experience. The scene can be rendered partially without textures until the texture is available, or rendering can simply be interrupted until the texture is available.
    One way to improve the user experience if a tile is not present in the cache is to render the scene with a tile with a lower LOD. However, the user may be confronted with a sudden display of additional detail, i.e. the user is confronted with a virtual object whose rendering fidelity suddenly changes as cache misses become cache hits once the streaming module has streamed the tiles to the cache memory. This can present itself as a very observable event when rendering the scene.
    It is clearly desirable that the tiles necessary for rendering a scene are available in the cache memory and, preferably, are ready for use in the cache memory before or at the time the application first uses it. If the scene can only be rendered with cache hits, then extra detail will not suddenly appear in the rendered results and the user can always experience the highest quality. By predicting the working set of a virtual scene, ie the set of tiles to be used for rendering the scene, and / or by predicting the working set of future scenes, the cache can be loaded by the streaming system such that cache misses are minimized. Therefore, the application does not have to wait until the tile is available for rendering, or does not need to render SOs with tiles with different LODs. The time between a request for a tile and the moment that a tile is available for the application can be effectively hidden.
    In addition, prediction allows better management of the streaming cache memory. When all cache slots, being a memory block reserved for a tile in the cache memory, are occupied, certain cache slots are selected to be reused and recycled and are referred to as available for the streaming module for loading new tiles. By predicting the tile tool set, tiles that do not belong to the tool set can be identified, i.e. tiles that are not necessary for rendering the scene. The cache slots for these tiles can be offered to the streaming module for recycling without affecting the rendering results.
    A predictor of the working set, also called predictor for short, determines a working set of a scene. This predicts which tiles will be needed for rendering or processing a virtual scene. This therefore predicts the content or future content of the cache. By predicting the cache, the streaming module can be sent to pre-fill the streaming cache and the tiles may be available in the cache when they are used for the first time.
    There are many ways to determine a scene toolbox. Each method has its own accuracy that is usually related to calculation complexity. More complex algorithms usually require more calculations. In other cases, algorithms have a longer delay, i.e. there is a lot of time between the start of the calculation and the moment when the calculation results are available. For example, performing calculations on a discrete graphics card may cause a greater delay between the initiation of the calculation and the delivery of the result than performing the same calculation on a CPU.
    For a rendering application, it is not always possible to continuously execute highly complex predictive algorithms. The scene tool set can be defined with low-complex algorithms with sufficient accuracy for certain scenes or at specific times. These low-complex algorithms can, in contrast to high-complex algorithms, be executed continuously. By combining the output of low-complex continuous-running predictive algorithms with that of more complex intermittent-running predictive algorithms, visual fidelity can be increased sufficiently to prevent tiles from suddenly being displayed in a way that is striking to the user with additional detail. The core of this invention consists in carrying out preferably a plurality of predictors, each with its own method / algorithm and their own accuracy of the results and their own computational complexity, each of which is performed at different frequency intervals. The aforementioned predictors are then discussed.
    A predictor can be run on the CPU and output its results to the system memory. A predictor can be run on the GPU and output its results to GPU memory or system memory. The following sections contain examples of predictors.
    An SO can be a texture and a predictor can render a scene of the application's virtual camera position on the GPU with a pixel shader that outputs the texture tile used per pixel, including the LOD level (in the case of a texture, the MIP level). The rendered output is then copied from the GPU memory to the system memory where the predictor traverses all rendered pixels and identifies the unique tile identifications (including their LOD level) that are present on the screen. The predictor of the cache executes this unique list of tiles. Such a predictor can best be described as having a very high accuracy and a high computational complexity, but since asynchronous rendering on the GPU usually leads to a delay of a few milliseconds, such a predictor also causes a large delay between the initiation of the algorithm and the time of the results.
    The predictor described above can render the scene from a different virtual camera position than that of the main rendering view of the application. For example, the camera can be placed at a point and in a direction in the virtual world corresponding to a future or past position and direction of the virtual camera. Via a future camera position, the system can, for example, predict the toolbox for a future frame. As a result, the cache system can predict tile visibility.
    The predictor as previously described may render the scene as previously described, but may omit or skip a virtual object responsible for partially blocking the virtual camera during the rendering process. The predictor can do this by either not rendering the virtual object, or not rendering parts of the virtual object, or by rendering the virtual object with a certain level of transparency, which results in the blocked parts becoming partially transparent to the camera and thus become visible again to the predictor.
    An SO can be a geometric model in the virtual scene and a predictor can emit rays from the current and future virtual camera position and have it intersected with substitute geometric models in the virtual world for the actual high-level models that are on the CPU running. When a beam intersects with these placeholder geometric figures, the SO represented by the placeholder is identified and the distance between the intersection point and the camera is used to calculate the LOD level. The process is repeated for a number of rays from the virtual camera. The list of unique SOs with all their tiles is displayed as the result of the predictor. Such a predictor can best be described as having an average accuracy, since it does not distinguish between tiles within the same LOD for an SO. This also has a small delay because all calculations are performed synchronously on the CPU and the results are available almost immediately. The calculation complexity depends on the number of rays that are detected each time the predictor is executed.
    An SO can be a texture and a predictor can request all tile references used in the entire virtual world of the application, a subset of the virtual world or the loaded part of the virtual world, and include these tile references as the working set in a list.
    An SO can be a texture and a predictor can request the application all virtual objects within a certain virtual distance from the current, past and / or future virtual camera position. The predictor can inspect any object, determine the tiles that may be used by the object, and add those tiles to an internal list. The predictor outputs this list as the toolbox.
    The predictor can predict the working set by looking up a number of virtual objects and / or SOs, and (a subset of) their tiles in a data structure. The data structure can be a quad tree, BSP tree, k-d tree, other tree data structure or other data structure including lists, linked lists, hash tables, dictionaries. This can store the data in a spatial way or allow spatial queries to search the virtual objects, SOs or tiles. The data structure can be constructed by the application while it is being executed. The data structure can be generated in advance by the predictor and loaded from the memory, including storage memory.
    The predictor can predict the toolbox or determine a number of tiles to be streamed by looking up a pre-compiled list of tiles, or virtual objects with their tiles, or SOs with their tiles. The predictor can be prompted by the application at certain times to load the tiles from the list. The predictor can be urged to load the tiles from the list when the virtual camera is in the range of a virtual object that displays the list. The application can communicate the list to the predictor. The list can be constructed by the application while it is being executed. The list can be generated in advance by the predictor and loaded from the memory, including storage memory. For example, when an operator of the rendering camera enters a room in the virtual scene, a list of all tiles in that room can be added to the description of the space in memory, the predictor can load the list and use its contents as prediction results.
    The predictor can predict the toolbox by looking up all virtual objects by emitting rays from a virtual camera position and searching the intersections of these rays with virtual objects. Once the virtual objects have been detected that intersect with the rays, i.e. all objects that are visible in the scene, the object tiles are determined and added to an internal list. After processing all objects, this list is then communicated as the toolbox.
    Fig. 5 shows an embodiment of the proposed predictive system. In the proposed embodiment, the predictive system comprises a set of predictors, a toolbox manager and a streaming module manager. As explained earlier, the predictive system, together with the application, controls the streaming system when loading SO tiles in the streaming cache memory. It does this on the basis of information it collects by performing the predictors of the set of predictors. These predictors can request knowledge or information or share the virtual scene that is being rendered or processed by the application. Using this information, the predictive modules predict the tool's set of tiles for the current or future virtual scene. The predictors can be performed independently of the application. The predictors can also use application information about which tiles the application requests from the streaming module.
    In a preferred embodiment, the system comprises a predictor controller for controlling the predictive modules, e.g. activating a predictor. A system with a predictor controller clearly corresponds to a system without such a controller, but instead the predictive system itself contains controller logic, for example logic as described with respect to the predictive system of Fig. 5. In one embodiment, this logic can also be in the application, in the streaming module or in any other component in the system.
    The predictor controller can request its information from the application or get information shared by the application, such as, but not limited to, information about virtual scene rendering, general rendering behavior, etc., to perform its controller function. The controller can also use information about the system resources of the computer system. For example, the predictor controller can request the number of CPU cores and the number of GPUs via the system and set the amount of predictors based on this information. For example, the predictor controller can start an execution of each predictor on a different CPU core or GPU and thus make optimal use of the available computer resources.
    The predictive system executes a set of predictive algorithms at specific times. The predictive algorithms can be executed asynchronously from the application without specific requests from the application to the streaming system. The predictive algorithms can be executed synchronously, whereby the predictors can be executed whenever the application requests a certain tile from the streaming system. The application can stimulate the execution of a predictor.
    Each of the predictive algorithms calculates a proposal from the working set of tiles of the scenes and transmits its results to the working set manager. The toolbox manager manages the various proposals of the scene toolbox and combines them into a single scene toolbox. This toolbox is communicated to the streaming module manager by the toolbox manager. The streaming module manager detects changes in the toolbox and manages the streaming module accordingly. The streaming module manager initiates streaming requests for new tiles and points the streaming module to free cache slots for tiles that are no longer present in the toolbox. Note that the functionality of the toolbox manager and the streaming module manager can be combined without departing from the scope of the invention. Also note that the predictors can be integrated into the application without departing from the scope of the invention.
    The set of predictors comprises a number of predictors. The set of predictors may include predictors for different types of SOs, for example, one predictor for textures and one for point clouds. Each predictor can perform his calculations continuously with a certain frequency or initiate according to a time schedule. The predictors can be performed synchronously or asynchronously to the application. The application can trigger the execution of a predictor. The time schedule can describe the frequency in Hertz at which the predictor is executed. For example, a highly accurate predictor is set to be performed at a frequency of 20 Hz, i.e. every 50 milliseconds. For an application that renders the virtual world at 60 Hz, this corresponds to running the predictor every three rendering frames. Alternatively, the timetable indicates the times at which a predictor is executed by specifying the number of rendering frames between each initiation. For example, the timetable states a number of three frames between each initiation. If the application renders the scene at 30 Hz, it means that the predictor will be executed at 10 Hz. The frequency of the predictor can be synchronized with the rendering frequency. For example, a predictor can perform his calculations in the thread that is responsible for creating rendering commands for the rendering API. This thread can initiate execution every time the thread starts rendering a frame.
    Fig. 6 illustrates an example. When an SO is a structure, the set of predictors can contain two predictors. One of the predictors is a highly accurate, highly complex predictor that renders the scene as described above. The other predictor is a less accurate, low-complex predictor that lists all tiles within a certain virtual distance from the virtual camera. The first predictor is executed every five rendering frames, while the second predictor is executed every rendering frame. The figure shows how the first predictor needs a considerable amount of time to communicate the results to the streaming module manager, namely 2 frames. However, the second predictor communicates its results to the streaming module manager within the same rendering frame. Note that in the example, the first predictive algorithm is not executed for frames 4 and 5. Extra time may be required to make the results available in the system memory. For example, rendering an image on the GPU and reading back the resulting image has a similar delay of 2 to 3 frames. As this example shows, only one predictor is active at certain times (eg Frames 4-5). However, several predictors, two in this example, are used during the lifetime of the application.
    In one embodiment, if a predictor is set to be initiated while a previous instance of that predictor is still running, the predictor is not scheduled to be executed. For example, if a predictor needs 10 ms to be executed and is scheduled to be executed at 120 Hz, the predictor will be executed at an actual frequency of 100 Hz. In another embodiment, if a predictor is set to be initiated when a previous instance of that predictor has been executed, but the results are still being copied to the memory, the new instance can be initiated. For example, the results of a predictive process running on the GPU can be copied asynchronously to system memory while initiating a new instance of the same predictor. In another embodiment, if a predictor is set to be initiated while a previous instance of the same predictor is still running, the predictor can still be initiated to be run again.
    In one embodiment, the initiation of the predictor can be encouraged by the application or another operator. The predictor can be performed continuously or can be performed a certain number of times.
    In one embodiment, all predictors or a subset of predictors can be performed at the same frequency. They can perform their results at different times, since some predictors need more time to calculate than others.
    It is possible that only one subset of all available predictors is executed. This subset can be determined during the term. For example, the number of active predictors is selected in accordance with the number of CPU cores in a system. The subset can be predetermined by the system administrator, for example by means of a user-defined constant in the application. Since predictors can be performed at different frequencies, it is possible that only one predictor or no predictors are performed at a certain point in the lifetime / duration of the application, but that different predictors are active during the lifetime of the application.
    Each predictor executed by the system sends its results to the tile tool set manager. The tile tool set manager combines different tool set proposals that he has received from the set of predictors into a single tool set. This toolbox is communicated to the streaming module manager who updates the cache accordingly.
    A tile toolbox is a list of tile references (including those from different LODs) that must be in the cache system memory for the best rendering results. The tile references can include a tile index and LOD indication.
    The toolset manager has knowledge of the accuracy of each predictor in the set of predictors. Consequently, it can distinguish between inaccurate and accurate tool set descriptions. The toolbox manager can contain a range of valid LOD levels for each predictor. The toolbox manager runs through all tile references in the proposed toolbox and removes tile references with an LOD outside this range. The remaining tile references are added to the refined toolbox. In one embodiment, the toolbox manager has no knowledge of the accuracy of each predictor, but each predictor only outputs results for the tiles that he can predict with high accuracy.
    For example, if the SO is a texture and two predictors are active: a highly accurate predictor renders the scene and a low-accuracy predictor iterates all tiles within a certain virtual range. The toolbox manager can have a range of LOD level [0,1] for the first predictor and a range of LOD levels of [2, 10] for the second predictor. For the results of the first predictor, the toolbox manager rejects all tile references with a LOD level higher than 1. In other words, only the two MIP levels with the highest level of detail remain in the toolbox. For the results of the second predictor, the toolbox manager rejects all tile references with an LOD level lower than 2. Only the two MIP levels with the lowest level of detail (all levels except the two most detailed MIPs) remain in the toolbox. The toolbox manager combines these two tools by merging the two tools. The first predictor is used with others to search for the tiles that are visible on the screen with the highest level of detail. The second predictor is used to search for tiles with lower resolution tiles that will be visible in future frames when the first predictor is not active.
    In one embodiment, the toolbox manager assigns weights to each tile reference in the toolbox proposal represented by a predictor. These weights may be related to the accuracy of the predictor.
    For example, if the SO is a geometric model, the cache system includes two predictors, one performing a beam detection algorithm as previously discussed and the other iterating all SO tiles at a certain distance from the current, past, or future virtual camera position, also as described above. The toolbox receives the results from the first predictor and assigns a weight to each tile reference in the toolbox. For example, the tile description resulting from the second predictor may have a smaller weight than that of the first predictor, so that the toolset manager is instructed to prioritize the output of the second predictor. The toolbox manager can choose to reject tile references based on their weight. Another option is not to discard tiles, but to pass on the weights via the streaming module manager who can then execute his cache replacement strategy based on the weight. A frame counter can also be integrated into the weight, which is increased each time a new frame is rendered by the application. The current time can be recorded. These approaches can add extra weight to the latest results.
    After the working set manager has combined the proposed working sets of the predictors into one working set, the working set that thus arises is communicated to the streaming module manager. The streaming module manager, based on the toolbox that he receives, controls the streaming module when streaming the required tiles that are not available in the cache. When he receives a working set, he runs through all tile references. Tiles that are already in the cache remain in it and new tiles are asked to stream them to the cache. The streaming module manager can access the cache using a number of strategies. This can, for example, use a least-recently-used (LRU) strategy, in which tiles that have been used least recently are replaced in the toolbox. Different combination strategies for work sets can be applied.
    The streaming module manager may designate cache slots that are temporarily unused as available for containing newly streamed tiles. In another embodiment, the streaming module manager receives the weights of the tiles from the toolbox manager, and replaces tiles based on their weight. For example, tile slots with tiles with a smaller weight make way for tiles with a larger weight. In another embodiment, a frame counter can be integrated into the weight which is incremented each time a new frame is rendered by the application. The current time can also be recorded. These two approaches can give extra weight to the latest results and instruct the streaming module manager to prioritize the latest results. Various cache replacement strategies can be applied that are not limited to the examples given here.
    Although the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustrations and descriptions are to be considered as illustrative or exemplary and not restrictive. The foregoing description explains certain embodiments of the invention in detail. It should be noted, however, that no matter how detailed the foregoing has been incorporated into the text, the invention may be practiced in many ways. The invention is not limited to the disclosed embodiments.
    Other variations on the disclosed embodiments may be understood and performed by persons skilled in the art and by practicing the claimed invention, through a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps and the indefinite article "a" does not exclude a plural. A single processor or other unit can perform the functions of different items in the claims. The mere fact that certain measures are listed in mutually different dependent claims does not mean that a combination of those measures cannot be used to benefit. A computer program can be stored / distributed on a suitable medium, such as an optical storage medium or semiconductor medium supplied with or as part of other hardware, but can also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems. Any references in the claims should not be construed as limiting the scope.
    权利要求:
    Claims (19)
    [1]
    Conclusions
    A predictive system for determining a set of subareas to be used for rendering a virtual world from a computer graphics application, wherein said subareas belong to streamable objects to be used in rendering of said virtual world, wherein said streamable objects each comprise a variety of subareas, comprising the predictive system - predictive means adapted to receive information from a computer graphics application about a virtual world and to obtain a number of predicted sets of subareas for rendering a virtual world using streamable objects, wherein each predicted set of the aforementioned variety is obtained by a different predictive scheme, - a streaming manager arranged to receive said predicted sets of subareas, to derive from said predicted sets of subareas of a working set for rendering, and for sending control instructions, based on the aforementioned working set of subareas, concerning the set of subareas that must actually be used.
    [2]
    The predictive system of claim 1, wherein at least one streamable object is represented by sub-regions corresponding to a variety of different detail level versions of the at least one streamable object.
    [3]
    The predictive system according to claim 1 or 2, wherein said streaming manager is adapted to use information about the accuracy of the variety of predictions to optimize said control instructions.
    [4]
    A predictive system according to any one of the preceding claims, comprising a predictor controller adapted to select and control a subset of the plurality of predictive means and / or a time schedule or a frequency at which each predictor of the predictive means works.
    [5]
    The predictive system of claim 4, wherein said subset and / or said time schedule or said frequency at which each predictor works is determined during the execution of the predictive system.
    [6]
    The predictive system according to claim 4 or 5, wherein said subset and / or said time schedule or said stated frequency at which each predictor works is determined taking into account the available system resources.
    [7]
    The predictive system according to claims 4 to 6, wherein said subset and / or said time schedule or said stated frequency at which each predictor works is determined taking into account the arithmetic complexity of said predictors.
    [8]
    The predictive system according to claims 4 to 7, wherein said subset and / or said time schedule or said frequency at which each predictor is working is determined taking into account the accuracy of the results of said number of predicted sets of subregions of subregions and / or the delay with which the predicted sets mentioned become available.
    [9]
    A predictive system according to any one of the preceding claims, wherein at least one of the aforementioned predictive means is capable of acquiring at one pace or frequency one or more predicted sets of subareas of said variety.
    [10]
    A predictive system according to any one of the preceding claims, wherein one of the aforementioned predicted sets is obtained by determining which streamable objects or which sub-regions lie in a virtual scene within a certain virtual distance.
    [11]
    The predictive system of any one of the preceding claims, wherein at least one of said predictive schemes comprises rendering a scene from a virtual camera position and identifying a unique sub-area identification.
    [12]
    A predictive system according to any of the preceding claims, wherein at least one of said predictive schemes is suitable for performing a prediction in which a virtual camera position in the virtual world is used differently than a main position of the virtual camera for rendering said above application.
    [13]
    A predictive system according to any of the preceding claims, wherein at least one of said predictive schemes comprises rendering a scene from a virtual camera position whereby the virtual world is displayed without blocking virtual objects or in which blocking virtual objects are displayed with a level of transparency.
    [14]
    A predictive system according to any one of the preceding claims, wherein said streaming manager is suitable for determining the set of subareas to be used by assigning a weight to the different subareas of said predicted sets.
    [15]
    The predictive system according to claim 14, wherein the weight of said number takes into account the number of hits of a sub-area in the number of predicted sets.
    [16]
    A predictive system according to any one of the preceding claims, wherein said streaming manager is suitable for selecting memory locations to be reused and for forwarding an indication of the selected memory locations in said control instructions.
    [17]
    The predictive system according to any of the preceding claims, wherein the streaming manager is arranged to execute control instructions by refining the predicted sets of subareas of one or more of said predictive means with the output of at least one other predictive means.
    [18]
    A method for determining a set of subareas to be used for rendering a virtual world from a computer graphics application, wherein said subareas belong to streamable objects to be used in rendering of said virtual world, wherein said streamable objects each comprising multiple subareas, comprising the method - receiving information from a computer graphics application about a virtual world, - obtaining a number of predicted sets of subareas for rendering a virtual world using streamable objects, each predicted set of the aforementioned variety is obtained by another predictive scheme, - deriving from aforementioned predicted sets of subareas with information about subareas of a toolbox of subareas to be used in rendering and - the execution of control instructions based on said toolbox ffende the set of subareas to be used.
    [19]
    A program that can be executed on a programmable device comprising instructions that, when executed, perform the method as described in any one of the preceding claims.
    类似技术:
    公开号 | 公开日 | 专利标题
    BE1022664B1|2016-06-29|Predictive system for forwarding textures
    US8970580B2|2015-03-03|Method, apparatus and computer-readable medium rendering three-dimensional | graphics
    Schmalstieg et al.1996|Demand‐Driven Geometry Transmission for Distributed Virtual Environments
    Wonka et al.2001|Instant visibility
    JP2005092863A|2005-04-07|Data processing system
    WO1998000811A1|1998-01-08|Fly-through computer aided design method and apparatus
    KR101536501B1|2015-07-13|Moving image distribution server, moving image reproduction apparatus, control method, recording medium, and moving image distribution system
    US8508550B1|2013-08-13|Selective rendering of objects
    JP6333840B2|2018-05-30|Method for forming shell mesh based on optimized polygons
    CA2837983C|2020-07-07|Partition of a 3d scene into a plurality of zones processed by a computing resource
    US20030011598A1|2003-01-16|Three dimensional rendering including motion sorting
    Beacco et al.2016|A survey of real‐time crowd rendering
    US8223157B1|2012-07-17|Stochastic super sampling or automatic accumulation buffering
    KR101670958B1|2016-11-01|Data processing method and apparatus in heterogeneous multi-core environment
    CN107038017A|2017-08-11|The method and apparatus that rendering data is obtained
    CN106204709A|2016-12-07|For the method and apparatus processing texture
    Millan et al.2006|Impostors and pseudo-instancing for GPU crowd rendering
    Christie et al.2012|Occlusion-free camera control for multiple targets
    KR20100068603A|2010-06-24|Apparatus and method for generating mipmap
    KR20090059800A|2009-06-11|Asterization engine and three-dimension graphics system for rasterizing by order adapted characteristic of polygon
    US10192342B1|2019-01-29|Using stand-in camera to determine grid for rendering an image from a virtual camera
    Hernández et al.2016|A rendering pipeline for real-time crowds
    EP3211601A1|2017-08-30|Rendering the global illumination of a 3d scene
    Hwang et al.2017|Image-based object reconstruction using run-length representation
    Yang et al.2019|Dynamic Load Balancing Algorithm Based on Per-pixel Rendering Cost Estimation for Parallel Ray Tracing on PC Clusters
    同族专利:
    公开号 | 公开日
    EP3216011A1|2017-09-13|
    WO2016071223A1|2016-05-12|
    EP3016075A1|2016-05-04|
    US11195250B2|2021-12-07|
    US10748241B2|2020-08-18|
    US20170316543A1|2017-11-02|
    US20200410633A1|2020-12-31|
    EP3216011B1|2020-07-08|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US6057847A|1996-12-20|2000-05-02|Jenkins; Barry|System and method of image generation and encoding using primitive reprojection|
    KR100291628B1|1999-06-19|2001-05-15|윤덕용|Cache memory for 3D graphic texture and its cache miss penalty reducing method|
    US6518974B2|1999-07-16|2003-02-11|Intel Corporation|Pixel engine|
    US20020075311A1|2000-02-14|2002-06-20|Julian Orbanes|Method for viewing information in virtual space|
    US6873329B2|2002-07-05|2005-03-29|Spatial Data Technologies, Inc.|System and method for caching and rendering images|
    US7848618B2|2005-05-13|2010-12-07|Microsoft Corporation|Unified recording and pause buffer format|
    US7535473B2|2005-09-20|2009-05-19|Erdas, Inc.|Collaborative environments in a graphical information system|
    US7934058B2|2006-12-14|2011-04-26|Microsoft Corporation|Predictive caching of assets to improve level load time on a game console|
    US8698818B2|2008-05-15|2014-04-15|Microsoft Corporation|Software rasterization optimization|
    KR101625058B1|2008-11-10|2016-05-27|엘지전자 주식회사|Method and device for processing a video signal using inter-view prediction|
    CA2751297C|2009-02-12|2014-05-13|Nippon Telegraph And Telephone Corporation|Multi-view image coding method, multi-view image decoding method, multi-view image coding device, multi-view image decoding device, multi-view image coding program, and multi-viewimage decoding program|
    WO2011113444A1|2010-03-15|2011-09-22|Panasonic Corporation|Method and apparatus for trajectory estimation, and method for segmentation|
    US8959313B2|2011-07-26|2015-02-17|International Business Machines Corporation|Using predictive determinism within a streaming environment|
    US9311750B2|2012-06-05|2016-04-12|Apple Inc.|Rotation operations in a mapping application|
    US20140373032A1|2013-06-12|2014-12-18|Microsoft Corporation|Prefetching content for service-connected applications|
    US10181193B2|2014-03-10|2019-01-15|Microsoft Technology Licensing, Llc|Latency reduction in camera-projection systems|
    EP3016075A1|2014-11-03|2016-05-04|Graphine NV|Prediction system for texture streaming|EP3016075A1|2014-11-03|2016-05-04|Graphine NV|Prediction system for texture streaming|
    GB2546286B|2016-01-13|2020-02-12|Sony Interactive Entertainment Inc|Apparatus and method of image rendering|
    US10939038B2|2017-04-24|2021-03-02|Intel Corporation|Object pre-encoding for 360-degree view for optimal quality and latency|
    US10623791B2|2018-06-01|2020-04-14|At&T Intellectual Property I, L.P.|Field of view prediction in live panoramic video streaming|
    US10812774B2|2018-06-06|2020-10-20|At&T Intellectual Property I, L.P.|Methods and devices for adapting the rate of video content streaming|
    US10616621B2|2018-06-29|2020-04-07|At&T Intellectual Property I, L.P.|Methods and devices for determining multipath routing for panoramic video content|
    US11019361B2|2018-08-13|2021-05-25|At&T Intellectual Property I, L.P.|Methods, systems and devices for adjusting panoramic view of a camera for capturing video content|
    US10708494B2|2018-08-13|2020-07-07|At&T Intellectual Property I, L.P.|Methods, systems and devices for adjusting panoramic video content|
    US11166048B2|2018-10-02|2021-11-02|Tencent America LLC|Method and apparatus for video coding|
    US11164293B2|2019-01-30|2021-11-02|National Cheng Kung University|Adaptive enhancement method for image contrast based on level of detail|
    DE102020000306A1|2019-03-21|2020-09-24|Adobe Inc.|Generating a sequence of textures for video transmission|
    法律状态:
    优先权:
    申请号 | 申请日 | 专利标题
    US201462074127P| true| 2014-11-03|2014-11-03|
    EP62/074,127|2014-11-03|
    US62/074,127|2014-11-03|
    EP14196784.4A|EP3016075A1|2014-11-03|2014-12-08|Prediction system for texture streaming|
    [返回顶部]