METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

专利摘要:
A computer implemented method (5400) of overlaying a graphic image in a computing device (5201), the method comprising the steps of: a) providing (5401) a semi-transparent overlay window (5383) containing a first bitmap (bmp_mask) comprising a plurality of semi-transparent pixels and having a transparency level (T) ranging from 5% to 45%, b) configuring (5402) the overlay window (5383) in click-through mode; Repeatedly performing the following steps: c) taking (5403) a screenshot, thereby obtaining a second bitmap (bmp_mix); d) updating the pixel values of the first bitmap (bmp_mask) such that the color values of the pixels of the bitmap (bmp_orig) located below the overlay window (5383) are substantially inverted. A computer program product for performing this method. A computer device (5201) that includes computer executable instructions for performing this method. A computer system (5200) that includes such computer device (5201).
公开号:BE1026516B1
申请号:E20185637
申请日:2018-09-18
公开日:2020-03-09
发明作者:Lambert Jacobs
申请人:Inventrans Bvba；
IPC主号:

专利说明:

METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE
Domain of the invention
The present invention generally relates to the field of graphic overlay in a computer system. More specifically, the present invention relates to a computer implemented method for providing a graphic overlay in a computer device or in a display device. The present invention also relates to a computer device configured to perform such a method, and to a computer program product to perform such a method in a computer device or a display device. The present invention also relates to a portable computing device and a display device.
Background of the invention
FIG. 1 shows a schematic block diagram of a classic computer system 100 comprising a computer device 101 (e.g., a desktop computer), with a keyboard input connected to a keyboard 102, and a mouse input connected to a mouse device 103, and a video output connected to a display or monitor or screen 104. The computer device 101 includes a central processing unit CPU running a multi-tasking operating system O / S with a graphical user interface GUI, and one or more software applications such as, for example, a text viewer or a text editor. Input from the keyboard 102 and mouse device 103 is usually handled by device drivers that receive information from one or more input port (s). Such computer systems and application programs are known in the art.
In the specific example of FIG. 1, the computer device 101 generates a graphic image which is shown on the display 104 as indicated by the rectangular area 109 (shown with rounded corners and slightly offset from the display edges for illustrative purposes). The graphic image 109 of FIG. 1 includes image portions related to a desktop window (visible on the left of the screen 104), and related to an application window showing a text document (visible on the right of the screen 104).
About two or three decades ago, when the Internet was still in its infancy, many people preferred to read (or study) textual information printed on paper rather than directly from a screen, which at the time was usually a cathode ray tube. (CRT, Cathode Ray Tube) screen. At that time, most information, for example technical information, data sheets, product catalogs, manuals, computer magazines, etc. was primarily distributed in printed form and less information was available in electronic form.
BE2018 / 5637
In contrast, today (in 2017) very large amounts of information are provided or made available in electronic form, and many readers prefer to read directly from a screen, rather than first printing the information on paper and then read the paper version. This change is likely due to the advent of LCD screens (stable image, high resolution), the growth and speed of the Internet, the increasing amount of information available electronically, the growing awareness to save trees by reducing printing , the availability of tools to manage documents electronically (eg scan, edit, annotate, search), the increasing size of storage devices (eg hard disk, memory stick), the ability and speed to search electronically, the trend towards paperless office, and the advent of portable devices (such as smartphones, tablets, etc).
The technology has evolved significantly in the past few decades. On the one hand, printing has become easier, and with the advent of laser printers and inkjet printers, printer speed and printer quality have increased significantly (compared to, for example, a matrix printer). On the other hand, display quality has also been significantly increased, and high-resolution LCD panels (eg with 1920 x 1080 pixels or even 3840 x 2160 at 60 Hz frame rate) are ubiquitous. Computers have become faster, with multiple processor cores operating at high speed (eg at 2 GHz), with more RAM (eg at least 4 or at least 8 Gigabytes of RAM) allowing multiple applications to run simultaneously.
For portable devices, especially smartphones, the same trend is visible: high screen resolution (e.g. 1920 x 1080 pixels, or 2560 x 1440 pixels, or 3840 x 2160 or 4096 x 2160 pixels are very common), multiple processor cores that operate at high clock speeds (eg 1.5 GHz or more), large memory size (eg 4 Gigabytes of RAM and 32 Gigabytes flash), etc. But printing from these devices is also very simple, eg using a wireless connection (eg wifi) instead of cables.
So people still have a choice between reading on the screen versus printing and reading on paper. However, this technological advance may make it more difficult for people to read information on a screen.
While a higher image resolution usually means a sharper image, it usually also means that the font size is smaller, and more text can be displayed on the screen. Smaller letters and more information in and of itself do not always make it easier to actually read that information, but rather can increase the difficulty of reading it.
A first problem encountered when reading textual information on a screen is that many people find it difficult to stay focused when it is required to read large amounts of text (e.g. study it) carefully, especially when the text is not contains a lot of formatting (such as bold or underlined characters, titles, different colors, different ones
BE2018 / 5637 fonts, blank lines, etc.). This is the case, for example, when reading or studying published patent documents.
This first problem is recognized in the art, and is addressed, for example, by a tool known as LineReader, which is commercially available on the Internet at the time of writing this document. This tool provides a graphical overlay window 483 with a semi-transparent (partially transparent) line 310 that follows the mouse pointer 399, as schematically illustrated in FIG. 3. In FIG. 3, line 310 is colored gray because patent documents still need to be filed in black and white, but in reality line 310 may be colored red or blue, for example, which is more striking and easier to distinguish than the original (native) mouse pointer or mouse cursor 399 typically provided by an operating system (such as, for example, Mac O / S from Apple Inc. headquartered in Cupertino, or Windows from MicroSoft Corporation, Redmond), especially on high-resolution displays. People can use this tool to underline textual information they are reading on the screen. The line can increase a user's ability to stay focused and focused.
However, as far as known to the inventors, there is no tool available on the market that makes it easier for a user to find, edit, or verify data in a table or worksheet, or locate a particular cell. While there are literally hundreds of millions of people who have used spreadsheets and worksheets on a daily basis for at least two decades, no obvious solution has been found.
Another problem encountered when reading or studying documents on the screen, or generally when spending many hours a day in front of a computer screen, is that many people experience eye fatigue, especially when reading large amounts of text on a bright background (e.g. when reading black text on a white background). Unfortunately, many programs (eg PDF viewers, text editors, etc.) and many web pages provide a bright, white background, probably because these programs are designed to display the information in the way it will be printed (such as the acronym WYSIWYG, which stands for what you see is what you get and what you see is what you get means). Screens that display images with relatively large areas of black text on a white background send relatively large amounts of light energy to the eyes, which is very inconvenient for many people. This inconvenience usually increases as people spend more hours in front of a computer screen, as the screen size increases, the distance to the screen decreases, or combinations of these, and is usually worse in a dimly lit room.
This problem is also recognized in the art, and one of the tools available in the market to address this problem is known as F.Lux, which provides a full-screen semi-transparent graphic overlay, shown schematically in FIG. 5. This
BE2018 / 5637 tool can also be used to dynamically change the so-called color temperature during the day. Relevant to the present invention is that tools such as F.Lux can reduce the brightness of the graphic image generated by the computer device, thereby reducing eye fatigue. However, the tool is not ideal, partly because the overlay does not consider the actual brightness of underlying windows, and / or because a user can no longer take a screenshot (screenshot) without the yellowish color that strongly colors the original images. distorted, and / or because the tool does not allow to underline or highlight textual information.
U.S. Patent No. 6,333,753, issued December 21, 2005, discloses a technique for implementing an on-demand display widget by controlled fading initiated by user contact with a touch sensitive input device. The document describes a semi-transparent Tool Glass, optionally using a graphics accelerator. Fading (fading) and fading out can be accomplished by dynamically changing an alpha transparency value. This document is incorporated herein by reference in its entirety, especially FIG. 8 of which, together with the accompanying description.
U.S. Patent No. 5,798,752, issued August 25, 1998, discloses a technique in which a user can move a semi-transparent overlay window containing a so-called workpiece with the non-dominant hand, and move the mouse pointer with the dominant hand.
Many people read books on an eReader device, but it is difficult to stay focused. Some people use their finger to underline the text they read, but when the device is put aside, the location is lost. There is a need for a more practical way to stay focused.
Many professional users use websites to search for data, eg translators looking for the right terminology. A problem with many free websites is that they show advertisements (advertisements), which can be disruptive. It would be nice to be able to use websites without being distracted by advertisements and stay focused.
Summary of the invention
It is an object of the present invention to provide a computer-implemented method for overlaying a graphic image generated by a computer device with an overlay image.
It is an object of certain embodiments of the present invention to provide an overlay window comprising at least one semi-transparent visible object covering only a minority portion of the underlying graphic image and movable in accordance with movements of an input device connected is with the computer device.
BE2018 / 5637
It is an object of certain embodiments of the present invention to provide such a method that is well suited for assisting a user in reading textual information on a screen.
It is an object of certain embodiments of the present invention to provide such a method that is well suited for assisting a user in extracting information from a table or spreadsheet (worksheet), or when editing a table or spreadsheet .
It is an object of certain embodiments of the present invention to provide such a method wherein the overlay image further comprises at least a second semi-transparent object covering a majority portion of the underlying graphic image.
It is an object of certain embodiments of the present invention to provide such a method with at least a first and a second semi-transparent object, wherein the second semi-transparent object has a higher transparency than the first semi-transparent object.
It is an object of certain embodiments of the present invention to provide such a method in which the transparency of the first movable object is sufficiently low that this object can be easily distinguished from the background, and in which the transparency of the second object is sufficiently high, so that the underlying textual or alphanumeric information retains good readability.
It is an object of certain embodiments of the present invention to provide such a method further comprising at least a third semi-transparent object, which is also movable in accordance with movements of the input device.
It is an object of certain embodiments of the present invention to provide such a method in which a substantially monochrome background color of an underlying image is transformed into a textured pattern background, preferably without significantly compromising the legibility of the original textual information, and / or preferably with an aesthetically more attractive background.
It is also an object of embodiments of the present invention to provide an overlay application that converts clear images to dark images.
It is also an object of the present invention to provide a computer device adapted to perform the said method.
It is also an object of the present invention to provide a computer program product for providing a graphic overlay, which computer program product, when executed on at least one processing unit of said computer device, performs said overlay method.
BE2018 / 5637
It is also an object of the present invention to provide a portable calculator that provides an overlay image.
It is also an object of the present invention to provide a display device that provides an overlay method.
It is also an object of embodiments of the present invention to provide a method of overlay that dynamically adjusts itself depending on visible features of underlying windows.
It is an object of specific embodiments of the present invention to provide an overlay method for darkening the screen that dynamically adjusts itself depending on the brightness of the underlying windows.
It is also an object of embodiments of the present invention to provide an overlay method that allows a screenshot (screen shot) of the underlying windows to be taken (as if the overlay was not there), but without deactivating the overlay application.
It is also an object of specific embodiments of the present invention to provide a method that prevents advertisements from being displayed to the user.
These and other objects are accomplished by a method and a computer device and a computer program product and a computer system and a portable computing device and a display device according to embodiments of the present invention.
In a 1st aspect (nicknamed perforated bitmap), the present invention provides a computer implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing an overlay window; b) providing at least one visible object in said overlay window, the at least one visible object comprising a bitmap having a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are opaque pixels or semitransparent pixels; wherein the first plurality of pixels and the second plurality of pixels are interleaved (alternately positioned); c) configuring the overlay window in click-through mode.
The overlay window can be configured as an opaque window, which means that this window can contain non-transparent pixels or fully transparent pixels, but not semitransparent pixels. (The same visual effect would be obtained with a semi-transparent window with an alpha transparency of 100%).
Alternatively, the overlay window can be configured as a semi-transparent window with an alpha transparency α ranging from 1% to 99% or from 2% to 98% or from 5% to 95%, which
BE2018 / 5637 means that this window can contain fully transparent pixels or semi-transparent pixels, but not opaque pixels.
The bitmap with the fully transparent pixels interspersed with opaque or semi-transparent pixels is referred to herein as a perforated bitmap (by analogy with a perforated plate showing through the apertures).
By graphic image is meant the image formed by the underlying layers of associated underlying applications. The graphic image may comprise or be composed of a layered stack of partial images, at least some of which are provided by an application window associated with a computer application, such as a text editor, a PDF document viewer, a web browser, a spreadsheet (spreadsheet), etc. The bottom image (so-called desktop image) can be generated by an operating system, the optional other images of the stack can be generated by applications such as a text editor, PDF document viewer or editor, a web browser, a spreadsheet, a drawing program, etc.
Preferably, the overlay window extends mainly over the entire graphic area, for example (in a Windows environment) it can extend over the so-called work area, which is the entire desktop area (minus the area occupied). is called a taskbar.
The term fully transparent pixel is well known in the art. In practice, it is usually implemented by assigning a predefined pseudo color value to these pixels.
The bitmap can contain at least one row and / or at least one column containing at least one fully transparent pixel located between two opaque or between two semi-transparent pixels, preferably at least two rows and at least two columns.
The bitmap can contain at least one row and / or at least one column containing at least one opaque or semi-transparent pixel located between two fully transparent pixels, preferably at least two rows, more preferably each row.
The bitmap may contain at least one row and / or at least one column containing a plurality of odd and even numbered sets of pixels, an odd numbered set consisting of only one or at least one fully transparent pixel, and an even numbered set is composed of only one or at least one opaque or semi-transparent pixel.
In one embodiment, the at least one visible object including the bitmap occupies a majority portion of the area of the graphic image to be overlaid, for example at least 75% or at least 80% or at least 85% or at least 90% of the graphic image.
BE2018 / 5637
In simple terms, the semi-transparent object in this embodiment may, for example, form a film layer over the graphic image, which can be used to change the average color and / or the average light intensity of the graphic image.
The at least one visible object can be substantially stationary in the sense that it does not move in accordance with movements of an input device.
In one embodiment, the method further comprises step d) wherein at least a second visible object is provided which is movable in accordance with movements of at least one pointing device, and which occupies only a minority portion of the area of the graphic image to be overlaid.
In one embodiment, the at least one visible object including the bitmap is movable in accordance with pointer movements and occupies only a minority portion of the area of the graphic image to be overlaid, e.g., up to 25%, or up to 20%, or up to 15%, or up to 10% or up to 5%.
In simple terms, the semi-transparent object in this embodiment can be a relatively small object, such as a small line, or a large line (extending the full width of the screen), or a small cross, or a large cross (which extends over the full width and the full height of the screen), which moves in accordance with movements of a pointing device, for example a mouse device, a trackball, a touchpad, etc. Such an object is ideal for marking or underlining textual information in a text document, or alphanumeric information in a worksheet.
In one embodiment, the method further comprises the steps of: repeatedly performing the next step: f) obtaining position information (X, Y) related to a position of a mouse pointer or mouse cursor; or obtaining motion information (dx, dy) related to a movement of the at least one pointing device; and repeatedly performing the following step: g) adjusting a position of the at least one object which is movable and / or of the at least one second object which is movable, using said obtained position information or using said motion information.
For example, configuring the overlay window in click-through mode can be implemented (in an MS Windows environment) using one of the following function calls, or equivalent function calls: SetWindowLong (Handle, GWL_EXSTYLE, WS_EX_TRANSPARENT or WS_EX_LAYERED); SetWindowLong (Handle, GWL_EXSTYLE, GetWindowLong (Handle, GWL_EXSTYLE) or
WS_EX_TRANSPARENT);
As an example, obtaining position information may include querying the operating system and GUI about the position of the native mouse cursor, for example (in an MS Windows environment) using the GetCursorPos () function.
BE2018 / 5637
As an example, obtaining motion information may include: configuring the O / S to send raw input messages, for example, by registering the application with the O / S to receive raw input data, e.g. in the form of input messages known as WM_INPUT messages, for example using the WinAPI function RegisterRawinputDevices (). It is noted that applications do not automatically receive raw input messages.
Step f) is preferably performed each time a new message arrives. Step g) can be based on a timer, for example with a period in the range from 1 ms to 100 ms, preferably in the range from 1 ms to 60 ms. Therefore, step g) does not need to be performed with the same frequency as step f).
In one embodiment, the at least one movable object and / or the at least one second movable object has an elongated shape.
For example, a substantially rectangular shape, for example a rectangle with sharp edges, or a rectangle with truncated edges, or a rectangle with rounded edges, off a rectangle with rounded corners. Such a visible object can easily be generated or modified (size and / or color) on the fly (along the way), and is (by choosing an appropriate height and width) ideal for underlining text, without distracting a user.
In one embodiment, the at least one second movable object comprises a first movable element of an elongated, horizontally oriented shape, and a second movable element of an elongated, vertically oriented shape; and wherein step g) comprises adjusting a position of the first movable element and adjusting a position of the second movable element.
Or expressed in simple terms: the first movable element and the second movable element form a cross. It is an advantage of overlay with a cross that it can clearly indicate a particular cell in a spreadsheet or table.
In one embodiment, the first movable element extends a full width of the overlay window, and the second movable element extends a full height of the overlay window.
It is an advantage of overlaying with a cross that extends across the entire width and height of the screen that not only the desired cell is clearly indicated, but also the corresponding row header (row header) and column header ( column header) of the selected cell is clearly indicated.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a pseudo-random pattern.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern.
BE2018 / 5637
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern according to one of the following options:
i) the regular pattern is a 2x2 pattern, and exactly one out of four pixels is completely transparent;
ii) the regular pattern is a 2x2 pattern and exactly two of the four pixels are completely transparent, the two fully transparent pixels preferably being diagonally opposite;
iii) the regular pattern is a 2x2 pattern and exactly three of the four pixels are completely transparent.
It is an advantage of using a 2x2 pattern that it is small in relation to the entire screen size, and that the human eye tends to integrate the color and intensity of the four pixel values. Depending on how many pixels of the pattern are completely transparent (1 or 2 or 3 out of four), the overall intensity (or darkening) can be adjusted roughly. If the second group of pixels is alpha mixed (English: alpha-blended), both the color of these pixels and the alpha mixing value (English: alpha-blending value) can be used to fine tune the overall intensity (or darkening) .
In one embodiment, the method further comprises the step of: repeatedly adjusting a position of the object containing the bitmap or of the bitmap such that pixels of the underlying graphic image are overlayed (covered) by a fully transparent pixel of the first plurality pixels at a first time, and are overlayed by an opaque or semi-transparent pixel of the second plurality of pixels at a second and optionally third and fourth time.
Preferably, the adjustment is made for each frame (e.g. at 60 Hz), or every two frames (e.g. at 30 Hz) or every 3 frames (e.g. at 20 Hz), or every 4 frames (e.g. at 15 Hz) ). For example, the adjustment can be as simple as sliding the second object or the entire overlay window by 1 pixel to the right or to the left or up or down. This can be seen as a kind of time multiplex.
In an alternative embodiment, a second bitmap is shifted relative to the object's first bitmap, and the method further comprises overlaying the first bitmap at a first time, and overlaying the second bitmap at a second and optionally third and fourth time.
In one embodiment, step b) comprises: providing a bitmap wherein the second group of pixels are extracted or derived from a texture bitmap.
The texture bitmap can be a predefined bitmap or a selectable bitmap, or an adjustment thereof.
BE2018 / 5637
In the context of the present invention, a texture bitmap is preferably a bitmap that does not contain a graphic representation of easily recognizable letters or numbers, but rather contains a regular or irregular or pseudo-random pattern, an example of which is shown in FIG. 27; another example is a picture of a misty sky.
The texture bitmap can be much smaller than the bitmap of the at least one object in the overlay window, in which case it is repeated at least once and / or tiled and / or flipped and / or rotated. Extracting pixel values from a smaller texture bitmap has the advantage of saving memory. Using non-constant values for the second group of pixels can give a paper-like appearance to the user, which is more pleasant to read than a bright white plastic-like appearance.
In one embodiment, the overlay window is configured as a semi-transparent window with an alpha transparency (α) ranging from 1% to 99% or from 2% to 98% or from 5% to 95%, and the method further includes the step of: adjusting pixel values of the second plurality of pixels extracted or derived from said texture bitmap, as a function of an alpha transparency value of the overlay window.
Preferably, the adjustment is made such that a contrast level of the texture remains substantially the same despite variations of the alpha transparency value.
In one embodiment, the pixel values comprise three color components, for example, Red, Green and Blue, and each pixel value is adjusted using a linear expression of the respective color component itself, and optionally limiting the result of that linear expression to the range of 0 to 255.
As an example, the Red component can be adjusted according to the following formula, or an equivalent formula: R: = round ((R-Ravg) * Fcontrast + newR; limited to [0..255], where R is the original red value is of a pixel extracted from the texture bitmap, and Fcontrast is a predefined or selectable contrast factor (a floating-point number) This value preferably changes as a second-order function (eg, as the square) of the transparency level T, where ϊ = 100% -α, newR is a desired or selected mean value for the red components of the overlay image, which may be different from the mean red value of the stored texture bitmap.
It is an advantage if each component value (eg Red) is only calculated as a function of the original component value (in the example Red) and not of the other component values of that pixel (in the example: Blue and Green), because this takes computation time saves, and can be implemented through a one-dimensional lookup table.
In one embodiment, the graphic image is generated as a layered composition of a desktop image and one or more images associated with said one or more applications; and the semi-transparent overlay window is created by an overlay application.
BE2018 / 5637
The at least one processor will typically further run an operating system (O / S) with a graphical user interface (GUI) that provides only a single mouse pointer, and one or more applications selected from the group consisting of: a web browser application, a spreadsheet- application, a PDF document viewer, a PDF document editor, a text viewer and a text editor.
According to a 2nd aspect (nicknamed large cross), the present invention also provides a computer-implemented method for overlaying a graphic image in a computer, the method comprising the steps of: a) providing an overlay window; b) providing at least a first visible object in the overlay window, the first object having an elongated shape, horizontally oriented, and occupying a minority portion of the overlay window; c) providing at least a second visible object in the overlay window, the second object having an elongated shape, vertically oriented and occupying a minority portion of the overlay window; d) configuring the overlay window in click-through mode; repeatedly performing the following step: f) obtaining position information (X, Y) related to a position of a mouse pointer or mouse cursor; or obtaining motion information (dx, dy) related to a movement of the at least one pointing device; and repeatedly performing the following step: g) adjusting a position of the first visible object and adjusting a position of the second visible object based on the obtained position information or using the motion information.
The first and second objects together form a cross. Such an overlay is particularly useful when extracting data from a table or editing a spreadsheet.
In one embodiment, the first object extends at least 80% of a width of the overlay window; and the second object extends at least 80% of a height of the overlay window.
Preferably, the overlay window extends essentially over the entire screen, or over the entire work area of the screen (in the context of a Windows environment, this means the entire area minus the area occupied by the taskbar).
Preferably, the first object has a rectangular shape with a width of at least 80% or at least 85% or at least 90% or at least 95% or equal to 100% of the width of the screen.
Preferably, the second object has a rectangular shape with a height of at least
80% or at least 85% or at least 90% or at least 95% or equal to 100% of the screen height.
BE2018 / 5637
Preferably, the height of the horizontal object and the width of the vertical object are the same.
In one embodiment, the method further comprises step b) of: configuring the overlay window as a semi-transparent overlay window with an alpha transparency in the range of 5% to 95% or from 10% to 90% or from 20% to 80%.
In an alternative embodiment, the method further comprises step b) of: configuring the overlay window as a non-semi-transparent window, also referred to herein as an opaque window.
In one embodiment, the first and second objects comprise a monochrome bitmap of pixels.
In one embodiment, the first and second objects include a bitmap with: a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are opaque or that are semi-transparent pixels to be alpha blended with pixels of windows that are below the overlay window; wherein the first plurality of pixels and the second plurality of pixels are interleaved (alternately positioned).
The first and second plurality of pixels may be arranged in a checkerboard pattern.
According to a 3rd aspect (nicknamed texture bitmap), the present invention also provides a computer-implemented method of overlaying a graphic image in a computing device, the method comprising the steps of: a) providing a semi-transparent overlay window with a alpha transparency in the range of 5% to 95% or from 10% to 90% or from 20% to 80%; b) providing at least one bitmap in the overlay window, wherein the bitmap is at least 50% or at least 60% or at least 70% or at least 80% or at least 90% or about 100% of the area of occupies the overlay window, the bitmap containing a texture bitmap; c) configuring the overlay window in click-through mode.
The use of a texture bitmap transforms a shiny or glossy, eg plastic-like, white background into a matte, eg paper-like background, creating a beautiful aesthetic effect that makes reading a document on a screen more pleasant.
According to a 4th aspect (nicknamed frozen line), the present invention also provides a computer-implemented method of overlaying a graphic image in a computing device, the method comprising the steps of: a) providing an overlay window; b) providing a first visible object in the form of a vertical line; c) providing a second visible object in the form of a horizontal line segment located either on the left or right side of the vertical line; wherein the second visible object is movable in accordance with movements of an indicating device; * Repeatedly performing the following steps: d) Obtaining position information from a
BE2018 / 5637 mouse pointer or mouse cursor; e) testing whether the mouse cursor or pointer is on or above the vertical line, and if the result of the test is true, proceed to step f), otherwise proceed to step h); f) if the overlay window is not already configured in non-click-through mode, configuring the overlay window in non-click-through mode; g) testing whether a mouse button has been pressed, and if the result of this test is true, dragging the vertical line; and go to step d); h) if the overlay window is not already configured in click-through mode, configuring the overlay window in click-through mode; i) adjusting a position of the horizontal line segment based on the position information of the mouse pointer or mouse cursor.
This embodiment provides a line segment that freezes when the mouse cursor is on the right side of the vertical line and moves there along with the mouse cursor when the mouse cursor is on the left side of the vertical line, or vice versa. In addition, this embodiment makes it possible to drag the vertical line in a very intuitive manner. This embodiment is particularly suitable for translators and proofreaders.
In one embodiment, step i) includes: testing whether the mouse pointer or mouse cursor is on the same side of the vertical line as the horizontal line segment, and if the result of this test is true, adjusting a vertical position of the horizontal line segment in consistent with a vertical position of the mouse pointer or mouse cursor, and if the result of this test is false, maintaining a position of the horizontal line.
In one embodiment, the horizontal line segment includes a bitmap with: * a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are opaque pixels or semi-transparent pixels; wherein the first plurality of pixels and the second plurality of pixels are interleaved (alternately positioned).
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a pseudo-random pattern; or are the first plurality of pixels and the second plurality of pixels of the bitmap organized in a regular pattern.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern according to one of the following options: i) the regular pattern is a 2x2 pattern, and exactly one of the four pixels is completely transparent; ii) the regular pattern is a 2x2 pattern and exactly two out of four pixels are completely transparent, the two fully transparent pixels preferably being diagonally opposite; iii) the regular pattern is a 2x2 pattern and exactly three out of four pixels are completely transparent.
According to a 5th aspect (nicknamed auto darkening), the present invention also provides a computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing a semi- transparent overlay window with an alpha transparency value in the range of 1% to 99% or in the range of 5% to 95% or in the range of 10% to 90%, and comprising a first
BE2018 / 5637 bitmap; repeatedly performing the following steps: b) taking a screenshot (screen shot), thereby obtaining a second bitmap; d) determining at least one feature of the second bitmap; e) adjusting the alpha transparency value of the overlay window and / or adjusting one or more pixel values of the first bitmap based on the at least one particular feature.
It is an advantage of this method (where steps b) to e) are performed repeatedly) that the overlay can be dynamically adjusted based on actual features of the underlying image (s), rather than being based solely on the time of day and the geographic location where the calculator is located (as appears to be the case for F.Lux). This is particularly helpful for users who experience eye fatigue.
Thanks to this method, eye fatigue users do not have to manually adjust the overlay parameters (e.g. alpha transparency and / or darkness of the overlay bitmap), but are handled automatically using the actual image content. For example, if the underlying images are clear images (eg with a white background such as in MS Word or MS Paint or MS Powerpoint), more darkening can be used, while in the case that the underlying image (s) is less bright (e.g. certain web pages), less darkening can be used.
The loop can be repeated periodically, for example, based on a timer that has a period in the range of 0.25 s to 10 s, or in the range of 0.5 s to 5 s, for example, equal to about 1 second or about 2 seconds. The smaller this period, the higher the responsiveness of the calculator, but the more computing power is required. The longer this period, the slower the system responds, but the less computing power is required. Those skilled in the art implementing this function can find a suitable compromise. Optionally, this period can be chosen by a user.
The first bitmap not only consists of fully transparent pixels, so the first bitmap contains at least one semi-transparent pixel, typically about 50% of semi-transparent pixels in case the first bitmap is perforated, or typically about 100 % from semi-transparent pixels in case the first bitmap is not perforated.
In one embodiment, the first bitmap is a monochrome bitmap, which can be a perforated monochrome bitmap or an unperforated monochrome bitmap.
It is an advantage that such a bitmap can be easily generated from a single color value (e.g. with three color components R, G, B).
In a specific embodiment, the first bitmap contains only black pixels, or only grayscale pixels with (R, G, B) values equal to (x, x, x), where x is a value less than 93, or less than 65, or less than 49, or less than 33, or less than 17. Such images are ideally suited for darkening the screen without changing the hue.
BE2018 / 5637
In other embodiments, the first bitmap contains a monochrome color bitmap with pixels having (R, G, B) values equal to (x, y, z), each of x, y, z being less than 93 or less than 65 , or less than 49, or less than 33, or less than 17. Such images are well suited for darkening the screen with changes in hue.
In another embodiment, the first bitmap contains a color gradient, for example, a horizontal color gradient, or a vertical color gradient, or a diagonal color gradient.
The color gradient can be based on only two colors, for example a left and right color, an upper and lower color, an upper left color and a lower right color. It is an advantage that such a bitmap can be easily generated from only two color values (each with e.g. three color components R, G, B).
In one embodiment, the characteristic is determined using an under-sampled image, for example, by considering only 1 pixel from every 2x2 = 4 pixels, or by considering only 1 pixel from every 4x4 = 16 pixels, or by only 1 pixel from each 8x8 = 64 pixels, or by considering only 1 pixel out of every 16x16 = 256 pixels. In this way, the calculation time can be considerably reduced, while in most cases the effect is largely the same.
In an embodiment, step c) comprises: determining an average intensity and / or a maximum intensity and / or an intensity histogram of the pixels of the second bitmap; and includes step e): adjusting the alpha transparency level such that the average intensity or maximum intensity or intensity histogram meets a predetermined criterion.
For example, the criterion could be that the average or maximum intensity should be less than a certain threshold. The threshold value can be a predetermined value, or it can be user selectable.
It is an advantage that in this way, the average or maximum intensity can be forced to be less than a certain threshold value.
It is an advantage of this method that only a single value (namely the transparency value) has to be adjusted, which does not require a high CPU load. The target transparency value that meets the predetermined criterion can be calculated all at once, or can be approximated over several iterations. The latter may cause less flicker.
In one embodiment, step c) includes determining average color component values Ravg, Gavg, Bavg and / or color component histograms of the pixels of the second bitmap; and includes step e): adjusting one or more pixel values of the first image (e.g. about 50% of the pixel values in case the bitmap is perforated, or about 100% of the pixel values in case the bitmap is not perforated is) such that the average color component or the color component histogram meets a predetermined criterion.
BE2018 / 5637
The target pixel values can be calculated all at once, or can be approximated gradually. The latter may cause less flicker. It is an advantage of adjusting the pixel values of the first bitmap that there is much more flexibility because, for example, any of the color component values or all component values can be adjusted.
In one embodiment, the method further comprises step d) of determining a brightness value as an indication of a brightness in a room; and the adjustment of step e) further takes into account the brightness value.
In one embodiment, the brightness value can be determined using a light sensor, or using a webcam. It is an advantage of this embodiment that a poorly lit room can be overlayed differently from a bright room, even for the same underlying images.
In an embodiment, step a) further comprises: providing at least one vertical line or bar in the first bitmap for defining at least two areas comprising a first area and a second area; and step c) includes determining at least a first feature of the first area and a second feature of the second area; and comprising step e): adjusting one or more pixel values of the first area such that the first characteristic meets a first predetermined criterion, and adjusting one or more pixel values of the second area such that the second characteristic meets a second criterion.
It is an advantage of this embodiment that it allows the screen to be split into two areas, allowing a first application (e.g. a web browser) to be located in the first area (e.g. on the left side of the vertical line or bar) and allow a second application (eg a word processor) to be located in the second area (eg on the right side of the vertical line). The overlay application will automatically adjust a first feature of the first area (e.g. a first brightness and / or a first color) depending on the graphical information present on the left side of the vertical line, which is mainly based on the visited web page , and the overlay application will adjust a second attribute of the second area, which may be the same attribute or a different attribute as the first attribute, e.g. a second brightness and / or a second color depending on the graphic information present at the right side of the vertical line, which in this example mainly depends on the background color of the word processor). In case a rather dark web page is visited, while the text document has a bright white background color, darker pixels can be assigned on the right side of the first bitmap than on the left side of the first bitmap.
Or in other words: It is an advantage of this embodiment that the overlay bitmap can be customized with several areas, which independently darken
BE2018 / 5637 can be made. This allows for much more advanced screen adjustments, which is especially useful in a multi-tasking environment.
In one embodiment, one of the areas contains completely black pixels with (R, G, B) = (0,0,0), while the other bitmap is monochrome and contains grayscale pixels with (R, G, B) = (x, x, x), where the value of x is dynamically adjusted. The first and / or the second region may be perforated. Since the first and second regions are in the same overlay window, they have the same alpha blend value.
In one embodiment, the alpha blend value is predetermined (e.g., about 50% or about 60% or about 70% or about 80%), and the gray scale of the pixels of the second area is adjusted dynamically, or vice versa, or both alpha mix value if the gray scale level x are both adjusted.
According to a 6th aspect (nicknamed offset screenshot), the present invention also provides a computer-implemented method for overlaying a graphic image in a computing device, comprising the steps of: a) providing a semi-transparent overlay window containing an alpha transparency value α ranges from 1% to 99%, and includes a first bitmap; b) taking a screenshot (screen shot), thereby obtaining a second bitmap; c) calculating a third bitmap to compensate for the effect of the graphic overlay based on the first bitmap and the second bitmap and the alpha transparency value α.
Step b) can be initiated by clicking on a button with a mouse pointer, or by pressing a key on the keyboard.
It is an advantage of this method that it allows a realistic screenshot of the underlying windows to be taken, as if the overlay was not there, and without deactivating the overlay.
The inventors surprisingly realized that it is possible to recalculate the underlying image, because they realized that the recorded image is in fact an alpha-mixed image of the stack of underlying images associated with the underlying applications and the bitmap of the overlay window, taking into account the alpha transparency of the overlay window, both of which are known within the overlay application.
It is an advantage of providing an overlay application with a built-in print screen function, because the overlay application is able to compensate for the alpha blending due to the semi-transparent overlay. This offers the very great advantage to the user that a screenshot of the underlying applications can be taken without temporarily deactivating the overlay application, typically avoiding a flash of light which typically occurs when a darkening overlay is temporarily deactivated. This flash can cause temporary glare and / or eye fatigue.
BE2018 / 5637
In one embodiment, the method further comprises step d) of: storing the third bitmap in a non-volatile memory or in a storage device, optionally after format conversion and / or after image compression.
It is an advantage that the overlay application makes it possible to save the screenshot thus taken in a storage device, for instance by simply clicking the button. In preferred embodiments, the third image is stored as a compressed image file, for example, as a JPEG file. The overlay application may allow the user to choose the folder location and / or the compression ratio. The overlay application can assign an automatically generated file name, for example based on a counter value, or based on time and date. All this is extremely useful for a user.
Preferably, the overlay application also allows the user to choose whether or not to include the entire screen, or just the work area (leaving out the taskbar).
In one embodiment, the method further comprises step e) of: copying the third bitmap to a clipboard of an operating system running on the computer.
This is similar to the print screen button found on many keyboards, but with the important difference that the image is compensated for alpha blending with the overlay bitmap. The bitmap in the clipboard can then be used in the ways known in the art, for example to be edited in a graphic application (eg Paint.Net) or to be pasted in a PowerPoint document, etc.
In one embodiment, step c) includes using the following set of formulas, or an equivalent set of formulas: T = 1-a, and R3 [x, y] = [R2 [x, y] - R1 [x, y] * (1-T)] / T, and G3 [x, y] = [G2 [x, y] - G1 [x, y] * (1-T)] / T, and B3 [x, y] = [B2 [x, y] - B1 [x, y] * (1-T)] / T, where (R1, G1, B1) are color components of a pixel at a location (x, y) of the first bitmap that is overlayed, and where (R2, G2, B2) are color components of a pixel at the corresponding location (x, y) of the second bitmap taken as screenshot, and where (R3, G3, B3) are color components of a pixel at the corresponding location (x, y) of the third bitmap, and where α is the alpha transparency value, and T is a transparency value.
According to a 7th aspect (nicknamed inverted colors), the present invention also provides a computer-implemented method of overlaying a graphic image in a computing device, the method comprising the steps of: a) providing a semi-transparent overlay window which a first bitmap comprising a plurality of semi-transparent pixels and having a transparency level ranging from 5% to 45%, b) configuring the overlay window in click-through mode; Repeatedly performing the following steps: c) taking a screenshot, thereby obtaining a second bitmap; d) updating the pixel values of the first bitmap such that the color values of the pixels of the bitmap are those
BE2018 / 5637 located below the overlay window will be substantially inverted after alpha blending by the overlay window.
In one embodiment, the transparency level T is selected from the group consisting of 4/256, 8/256, 16/256, 32/256 and 64/256.
In one embodiment, the alpha transparency value α is selected from the group consisting of 252/256, 248/256, 240/256, 224/256, and 192/256.
In one embodiment, step d comprises: i) estimating the pixel values of the underlying bitmap based on the pixel values of the second bitmap and based on the pixel values of the first bitmap; ii) inverting the estimated pixel values, thereby obtaining target pixel values; iii) adjusting the pixel values of the first bitmap based on at least the target pixel values.
In one embodiment, step i) comprises estimating the pixel values of the bitmap located under the overlay window based on the following set of formulas, or an equivalent set of formulas:
Runmix = [Rmix - Rmask * (1-T)] / T, limited to the range from 0 to 255 or a partial range thereof; Gunmix = [Gmix - Gmask * (1-T)] / T, limited to the range from 0 to 255 or a partial range thereof; Bunmix = [Bmix - Bmask * (1-T)] / T, limited to the range from 0 to 255 or a subrange thereof; where (Runmix, Gunmix, Bunmix) is the estimated Red, Green, and Blue value of the pixels of the underlying bitmap, and (Rmask, Gmask, Bmask) are the Red, Green, and Blue values of the pixels of the first bitmap that was applied when taking the screenshot of step e), where T is the transparency level.
In one embodiment, step iii) comprises: calculating the pixel values of the first bitmap as a function of only the target pixel values.
In one embodiment, step iii) comprises setting the pixel values of the first bitmap based on the following set of formulas, or an equivalent set of formulas:
Rmask: = (A - B * Runmix), limited to the range from 0 to 255 or part range thereof;
Gmask: = (C - D * Gunmix), limited to the range from 0 to 255 or part range thereof;
_Bmask: = (E - F * Bunmix), limited to the range from 0 to 255 or part range thereof;
where (Rmask, Gmask, Bmask) are the Red, Green and Blue color component of the pixels of the first bitmap (bmp_mask), and where A, B, C, D, E, F are predetermined constants.
In one embodiment, A, C and E are integers ranging from 190 to 255.
In one embodiment, each of the values B, D and F is a value in the range from 0.20 to
2.0.
In one embodiment, each of the values B, D and F is selected from the group consisting of
1/2, 3/4, 5/8, 6/8, 7/8, 9/16, 10/16, 11/16, 12/16, 13/16, 14/16, 15/16, and 1 .
BE2018 / 5637
It is an advantage of this embodiment that the multiplication in the formulas can be performed by shift operations and / or addition operations and / or subtraction operations without having to use multiplication or division. For example, Runmix * 7/8 can be implemented as: Runmix - (Runmix shr 3), which can be performed much faster than an actual multiplication or division operation.
In one embodiment, at least one of the values B, D and F is equal to 1, and at least one of the values B, D and F is different from 1.
In one embodiment, each of the values B, D and F is equal to 1.
In one embodiment, step iii) comprises: setting the pixel values of the first bitmap as a function of the target pixel values and the pixel values of the second bitmap, or as a function of the target pixel values and the estimated pixel values of the underlying bitmap.
In one embodiment, the first bitmap occupies more than 80% of the area of the overlay window.
In one embodiment, the first bitmap occupies less than 95%, or less than 90%, or less than 80%, or less than 60%, or less than 45%, or less than 40% of the area of the overlay window.
In one embodiment, the overlay window further includes a third bitmap comprising a first plurality of fully transparent pixels and a second plurality of semi-transparent pixels interleaved. It is an advantage of this embodiment that the screen can be made substantially darker, with one portion of the screen having inverted colors (e.g. resulting in a black background with bright characters) and another portion of the screen being overlayed with a perforated bitmap with approximately 50% fully transparent pixels (with colors darker but not inverted). This is extremely useful for reducing eye fatigue, while at the same time recognizing the true color of a portion of the screen.
In one embodiment, the overlay window further comprises a fourth bitmap consisting of a plurality of fully transparent pixels.
In one embodiment, the first bitmap takes up almost the entire area of the window, but in the case of a Windows Operating System running on the calculator, preferably not the taskbar.
According to an 8th aspect, the present invention also provides a computer device, comprising: at least one central processing unit, and a first memory connected to the at least one central processing unit which stores computer-executable instructions therein; wherein the computer executable instructions comprise code snippets for performing an overlay method according to any of the 1st to 7th aspects.
BE2018 / 5637
The computer executable instructions may further include computer executable instructions configured to generate a graphic image in response to the operating system and / or one or more applications.
The computer executable instructions may further include an operating system (O / S) with a graphical user interface (GUI) that provides only a single mouse pointer, and one or more applications selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer and a word processor.
By computer device is meant not only a personal computer or a desktop computer or a laptop or a tablet computer, but also portable devices, such as a PDA or a smartphone or an eReader device.
In one embodiment, the computer device further comprises a graphics processing unit (GPU) with alpha mixing functionality, and a second memory connected to the graphics processing unit for storing graphics information therein; and wherein the overlay code fragments are configured to use the graphics processor to perform at least alpha blending, and optionally also time multiplexing.
According to a 9th aspect, the present invention also provides a computer system comprising: a computer device according to the 8th aspect; at least one pointing device connected to the computer device, the pointing device being movable by a user, the computer executable instructions further configured to receive input data indicative of movements of the pointing device; at least one display device connected to an output of the computer device for displaying the graphic image mixed with the overlay image.
The pointing device can be, for example, a mouse device, or a trackball or a touchpad.
Preferably, the display device has a screen resolution of at least 1920 x 1080 pixels, or at least 2560 x 1440 pixels, or at least 3840 x 2160 pixels.
The executable instructions for receiving mouse input data are usually referred to as a mouse device driver.
According to a 10th aspect, the present invention also relates to a computer program product for performing a method according to any of the 1st to 7th aspects on a computer device according to the 8th aspect or a computer system according to the 9th aspect.
Or in other words, according to the 10th aspect, the present invention also provides a computer program product for providing a graphic overlay, the computer program product containing executable instructions which, when executed
BE2018 / 5637 on at least one central processing unit (CPU) of a computer device according to the 8th aspect, or a computer system according to the 9th aspect, cause the computer device to perform a method according to the 1st to 7th aspect.
In an 11th aspect (nicknamed eReader), the present invention also provides a portable computing device comprising: a touch screen; at least one central processing unit, and a first memory connected to the at least one central processing unit which stores computer-executable instructions therein; wherein the computer executable instructions are configured to generate a graphic image containing textual information and to display that graphic image on the touch screen; wherein the computer executable instructions are further configured to generate a line or an elongated object (e.g., a rectangular object with sharp or rounded edges or with sharp or rounded corners) covering said textual information; - where the line or oblong object contains a plurality of semi-transparent pixels with a transparency level of 1% to 99% or from 2% to 98% or from 5% to 95%, or where the line or oblong object has a first plurality contains fully transparent pixels and a second plurality of opaque or semi-transparent pixels interleaved; wherein the computer executable instructions are further configured to detect a contact position on the touch screen, and to adjust a position of the line or elongated object according to the detected position.
The device can be, for example, an eReader or a smartphone device.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a pseudo-random pattern; or are the first plurality of pixels and the second plurality of pixels of the bitmap organized in a regular pattern.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern according to one of the following options: i) the regular pattern is a 2x2 pattern, and exactly one of the four pixels is completely transparent; ii) the regular pattern is a 2x2 pattern and exactly two out of four pixels are completely transparent, the two fully transparent pixels preferably being diagonally opposite; iii) the regular pattern is a 2x2 pattern and exactly three out of four pixels are completely transparent.
In a 12th aspect, the present invention also provides a display device for displaying a graphic image provided by a computer device, the display device comprising: a display panel for generating a visible image; an input port (e.g., a video input port) for receiving said graphic image data from said computer device; an input buffer for storing the graphic image received at the input port, and a texture buffer for storing a texture bitmap, and a frame buffer for storing image data to be displayed on the display panel; a processor adapted to generate the image data stored on it
BE2018 / 5637 the display panel should be shown; the texture bitmap comprising a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are semi-transparent pixels, preferably with a transparency level from 1% to 99% or from 2% to 98% or from 5% to 95 %, wherein the first plurality of pixels and the second plurality of pixels are alternately arranged (interleaved). the processor being adapted to generate the image data to be displayed by copying image data from the input buffer in case the corresponding pixel of the texture bitmap is a fully transparent pixel, and by alpha mixing image data from the input buffer in the case that the corresponding pixel of the texture bitmap is a semi-transparent pixel.
Preferably, the display device is an LCD device or an LCD monitor.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern according to one of the following options: i) the regular pattern is a 2x2 pattern, and exactly one of the four pixels is completely transparent; ii) the regular pattern is a 2x2 pattern and exactly two out of four pixels are completely transparent, the two fully transparent pixels preferably being diagonally opposite; iii) the regular pattern is a 2x2 pattern and exactly three out of four pixels are completely transparent.
In one embodiment, the processor is further adapted to repeatedly adjust a position of the texture bitmap such that pixels of the underlying graphic image are covered (overlayed) by a fully transparent pixel of the first plurality of pixels at a first time, and covered by a semi-transparent pixel of the second plurality of pixels at a second and optionally third and fourth time.
According to a 13th aspect, the present invention also provides a display device for displaying a graphic image provided by a computer device, the display device comprising: a display panel for generating a visible image; an input port for receiving said graphic image data from the computer device; an input buffer for storing the graphic image received at the input port, and a texture buffer for storing a texture bitmap, and a frame buffer for storing image data to be displayed on the display panel; a processor adapted to generate the image data to be displayed on the display panel; the processor being adapted to generate said image data as follows: i) copying computer image data from the input buffer at pixel times for which the sum of the row index and column index is odd, and by alpha mixing computer image data from the input buffer with corresponding pixel data from the texture bitmap at pixel locations for which the sum of the row index and column index is even; and ii) copying computer image data from the input buffer at odd times for pixel locations for which the sum of the row index and the column index is even, and by alpha mixing computer image data
BE2018 / 5637 from the input buffer with corresponding pixel data from the texture bitmap at pixel locations for which the sum of the row index and column index is odd.
Specific and preferred aspects of the invention are included in the appended independent and dependent claims. Features of the dependent claims may be combined with features of the independent claims and features of other dependent claims as appropriate and not merely as expressly stated in the claims.
These and other aspects of the invention will become apparent from and clarify with reference to the embodiments described below.
Brief description of the drawings
FIG. 1 shows a schematic block diagram of a classic computer system.
FIG. 2 is a schematic representation of a so-called Z-order of three windows or image planes that can be used in the computer system of FIG. 1.
FIG. 3 shows a schematic block diagram of a computer system similar to that of FIG. 1, when running an operating system and two applications: a text editor and a specific overlay application, known in the art as LineReader, the latter showing a semi-transparent line across the text editor window, which line can be moved by the user to underline text fragments.
FIG. 4 is a schematic representation of a so-called Z-order of four windows or image planes that can be used in the computer system of FIG. 3.
FIG. 5 shows a schematic block diagram of a computer system similar to that of FIG. 1, when running an operating system and two applications: a text editor and a specific overlay application known in the art as F.Lux, the latter showing a semi-transparent full-screen overlay image over the desktop window and the text editor window.
FIG. 6 is a schematic representation of a so-called Z-order of four windows or image planes that can be used in the computer system of FIG. 5.
FIG. 7 shows a schematic block diagram of an embodiment of a computer system according to the present invention. The computer system of FIG. 7 includes a specific overlay application for implementing a specific overlay method. The overlay application, when executed in the computer system of FIG. 7, shows a semitransparent overlay that includes at least two semi-transparent elements: (i) a horizontal line or bar that extends across the entire width of the screen, and (ii) a vertical line or bar that extends across nearly the entire width of the screen. extends the entire height of the screen. The horizontal and vertical lines are movable in accordance with movements of the input device.
BE2018 / 5637
FIG. 8 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 7.
FIG. 9 shows a schematic block diagram of another computer system according to an embodiment of the present invention. The computer system of FIG. 9 includes a specific overlay application configured to overlay at least two semi-transparent elements: i) a line, and ii) a so-called perforated bitmap. The line is movable in accordance with movements of the input device.
FIG. 10 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 9.
FIG. 11 shows a variant of the computer system of FIG. 9.
FIG. 12 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 11.
FIG. 13 shows a plurality of exemplary screenshots (screenshots). Each row shows a series of five images corresponding to an alpha transparency or alpha mix value α of 100%, 80%, 60%, 40% and 20%, respectively. Each of these images contains a text fragment overlaid (covered) by two lines and / or a monochrome bitmap. Two lines are used for illustrative purposes, to show both highlighting and underlining text in the same drawing.
In FIG. 13 (a), each text fragment is overlaid by two red lines with (R, G, B) = (255,0,0). The rest of the overlay window is completely transparent.
In FIG. 13 (b), each text fragment is overlayed by a bitmap containing light gray pixels with (R, G, B) = (192,192,192).
In FIG. 13 (c), each text fragment is overlayed by a bitmap containing dark gray pixels with (R, G, B) = (64, 64, 64).
In FIG. 13 (d), each text fragment is overlayed by a mere combination of the two red lines as shown in FIG. 13 (a) and through a light gray bitmap as in FIG. 13 (b).
In FIG. 13 (e), each text fragment is overlayed by a mere combination of the two red lines as in FIG. 13 (a) and through a dark gray bitmap as in FIG. 13 (c).
In FIG. 13 (f) to FIG. 13 (i), each text fragment is overlaid by two red lines as in
FIG. 13 (a) and the remaining area is overlaid by a so-called perforated bitmap as illustrated in FIG. 12, where 50% of the pixels are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are light gray pixels with (R, G, B) = (192,192,192), as can be provided by overlay methods according to embodiments of the present invention.
In FIG. 13 (g), each text fragment is overlaid by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as illustrated in FIG. 12, where 50% of the pixels
BE2018 / 5637 are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are dark gray pixels with (R, G, B) = (64, 64, 64), as can be provided by overlay methods according to embodiments of the present invention.
In FIG. 13 (h), each text fragment is overlaid by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as illustrated in FIG. 12, where 50% of the pixels are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are black pixels with (R, G, B) = (0, 0, 0), as can be provided by overlay methods according to embodiments of the present invention.
In FIG. 13 (i), each text fragment is overlaid by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as illustrated in FIG. 12, where 50% of the pixels are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are white pixels with (R, G, B) = (255, 255, 255), as can be provided by overlay methods according to embodiments of the present invention.
FIG. 14 (a) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels, when these pixels are overlayed by a bitmap containing 100% black pixels with (R, G, B) = (0, 0, 0 ), for three different alpha mix values.
FIG. 14 (b) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels, when these pixels are overlayed by a bitmap containing 100% white pixels with (R, G, B) = (255, 255, 255 ), for three different alpha mix values.
FIG. 15 (a) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels, when these pixels are overlayed by a bitmap containing 50% fully transparent pixels and 50% black pixels with (R, G, B ) = (0, 0, 0), arranged in a checkerboard pattern, for three different alpha blending values.
FIG. 15 (b) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels, when these pixels are overlayed by a bitmap containing 50% fully transparent pixels and 50% white pixels with (R, G, B ) = (255, 255, 255), arranged in a checkerboard pattern, for three different alpha mix values.
FIG. 16 to FIG. 19 show four examples of a larger text fragment.
FIG. 16 shows the text fragment that contains black text on a white background as can be displayed by a classic PDF viewer.
BE2018 / 5637
FIG. 17 shows the text fragment of FIG. 16, overlayed by a blue line segment containing pixels with the color (R, G, B) = (0,0,255), and with the transparency level of the overlay window set to 50%.
FIG. 18 shows the text fragment of FIG. 16, overlaid in a manner as described in FIG. 9 and FIG. 10, in particular, using a perforated bitmap with 50% fully transparent pixels, and 50% gray pixels with color value (128, 128, 128) arranged in a checkerboard pattern.
FIG. 19 shows a schematic block diagram of another computer system according to an embodiment of the present invention, the computer system containing a specific overlay application providing an overlay window comprising at least two semi-transparent elements: i) a line, and ii) a so-called perforated texture bitmap . The line is movable in accordance with movements of an input device.
FIG. 20 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 19.
FIG. 21 shows a schematic block diagram of another computer system according to an embodiment of the present invention, wherein the computer system includes a specific overlay application that provides an overlay window that includes at least one semi-transparent object in the form of a so-called perforated texture bitmap.
FIG. 22 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 21.
FIG. 23 shows a schematic block diagram of another computer system according to an embodiment of the present invention, wherein the computer system includes a specific overlay application that provides an overlay window that includes at least one semi-transparent object in the form of a so-called texture bitmap.
FIG. 24 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 23.
FIG. 25 shows the text fragment of FIG. 16, overlaid in a manner as described in FIG. 19 and FIG. 20, using a perforated bitmap with 50% fully transparent pixels, and 50% gray pixels obtained from a texture bitmap, chosen so that the resulting overlayed image has a background with an average color value of about (128, 128, 128) for comparison possible with FIG. 16 (the contrast of the texture is adjustable, and may be slightly exaggerated for illustrative purposes).
FIG. 26 shows the text fragment of FIG. 16, overlaid in a manner as described in FIG. 19 and FIG. 20, with a semi-transparent blue line of color (0,0,255) added and using a perforated bitmap with 50% fully transparent pixels and 50% gray
BE2018 / 5637 pixels organized in a checkerboard pattern, with the gray pixels obtained from a texture bitmap.
FIG. 27 shows the exemplary texture bitmap used to image the FIG. 25 and FIG. 26, before punching, before optional color adjustment, and before optional contrast adjustment.
FIG. 28 schematically illustrates how overlay with a perforated bitmap shifted back and forth over a single pixel can be used for time multiplexing or time averaging.
FIG. 29 (a) to (c) show examples of an image fragment obtained by overlaying a graphic image containing black text on a white background, using a method of the present invention, displayed on a native (native) display device 3840 x 2160 resolution configured at 2560 x 1440 resolution, with 100% text and application scaling factor.
FIG. 29 (a) shows a portion of the image of FIG. 18, viewed from a relatively great distance at which the human eye does not distinguish individual pixels.
FIG. 29 (b) shows the same image zoomed in by a factor of about 350%.
FIG. 29 (c) shows a portion of the same image further zoomed in by a factor of about 200%.
FIG. 30 (a) to FIG. 30 (c) show substantially the same images as those of FIG. 29 (a) to FIG. 29 (c), displayed on the same display device, again configured with a resolution of 2560 x 1440, but configured with 125% scaling for text and applications.
FIG. 30 (a) shows what the resulting image fragment looks like when viewed from a relatively long distance where the human eye does not distinguish individual pixels.
FIG. 30 (b) shows the same image zoomed in by a factor of about 350%.
FIG. 30 (c) shows the same image, further zoomed in by a factor of about 200%.
FIG. 31 shows a flow chart of overlay methods according to embodiments of the present invention.
FIG. 32 shows a flow chart of overlay methods according to embodiments of the present invention.
FIG. 33 shows a flow chart of overlay methods according to embodiments of the present invention.
FIG. 34 shows a simplified high level block diagram of software components and hardware components located in a computer system, which typically cooperate in performing a method according to the present invention.
FIG. 35 shows an exemplary user interface window as can be used by an overlay application according to the present invention.
BE2018 / 5637
FIG. 36 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with a specific overlay application providing an overlay window with a movable object having a so-called perforated bitmap. The line is movable in accordance with movements of the input device.
FIG. 37 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 36.
FIG. 38 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application providing an overlay window with a movable object in the form of a perforated cross movable in accordance with movements of the input device.
FIG. 39 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 38.
FIG. 40 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application providing an overlay window with a movable object in the form of a horizontal line extending across the entire width of the screen, and is movable in accordance with movements of the input device.
FIG. 41 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 40.
FIG. 42 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application providing an overlay window with a vertical line defining a left area and a right area of the screen, and with a horizontal line extending over the width of the left region, which is vertically movable in accordance with movements of the input device when the mouse cursor is in the left region, and freezes when the mouse cursor is in the right region.
FIG. 43 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 42.
FIG. 44 shows a flow chart of another overlay method according to embodiments of the present invention.
FIG. 45 shows a schematic block diagram of a portable device such as, e.g., an eReader or a smartphone, adapted to display textual information overlaid by a semi-transparent line, according to an embodiment of the present invention.
FIG. 46 shows a schematic block diagram of another computer system according to an embodiment of the present invention, which can be viewed as a variant of FIG. 21,
BE2018 / 5637 wherein the computer system provides a graphic overlay image in the form of a so-called perforated texture bitmap overlaid by the display device.
FIG. 47 is a schematic representation of a Z order of three windows or image planes that can be used in the computer system of FIG. 46, and above a fourth semitransparent overlay window provided by the display device.
FIG. 48 shows a schematic block diagram of another computer system according to an embodiment of the present invention, which can be viewed as a variant of FIG. 46, wherein the computer system provides a graphic overlay image in the form of an unperforated texture bitmap, which is overlayed by the display device.
FIG. 49 is a Z-order schematic of three windows or image planes that can be used in the computer system of FIG. 48, and above it a fourth semi-transparent overlay window provided by the display device.
FIG. 50 shows a flow diagram of another computer-implemented method according to an embodiment of the present invention.
FIG. 51 shows a flow diagram of another computer implemented method according to an embodiment of the present invention.
FIG. 52 shows a schematic block diagram of a computer system according to an embodiment of the present invention. The computer system of FIG. 52 (a) includes a computer device that executes an overlay application for inverting colors of at least a portion of the underlying image, which image is generated by the operating system in accordance with the underlying stack of windows, as shown, for example, in FIG. 53.
In FIG. 52 (a), only a portion of the screen is inverted in color.
In FIG. 52 (b) essentially inverts the entire screen of color, e.g., the entire screen area except the task bar.
FIG. 53 is a schematic representation of a so-called Z-order of five windows or image planes that can be used in the exemplary computer system of FIG. 52 (a) and / or FIG. 52 (b), when it executes a classic text editor application and an overlay application according to an embodiment of the present invention.
FIG. 54 is a high-level flow diagram of a computer-implemented method for substantially inverting the colors of at least a portion of the underlying image, according to an embodiment of the present invention, as may be implemented in the overlay application running on the computer device or computer system of FIG. 52 (a) and FIG. 52 (b).
FIG. 55 shows an exemplary user interface window as can be used by an overlay application running on the computer system of FIG. 52 (a) and / or FIG. 52 (b).
BE2018 / 5637
FIG. 56 and FIG. 57 shows a first exemplary image to show the quality of the color inversion available from the overlay application. FIG. 56 shows the original image, FIG. 57 shows the image overlayed in such a way as to cause color inversion.
FIG. 58 and FIG. 59 shows a second exemplary image to show the quality of the color inversion available by the overlay application. FIG. 58 shows the original image, FIG. 59 shows the image overlayed in such a way as to cause color inversion.
Detailed description of illustrative embodiments
The present invention will be described with respect to particular embodiments and with reference to certain drawings, however the invention is not limited thereto but is limited only by the claims. The figures are only schematic and not limitative. In the drawings, the size of certain elements may be exaggerated and not drawn to scale for illustrative purposes. The dimensions and the relative dimensions sometimes do not correspond to the current practical implementation of the invention.
Furthermore, the terms first, second and the like in the description and in the claims are used to distinguish similar elements and are not necessary for describing an order, neither in time, nor spatially, nor in order of precedence or in any other way. It is to be understood that the terms used in this manner are interchangeable under suitable conditions and that the embodiments of the invention described herein are capable of operating in a different order than described or shown herein.
It is to be noted that the term included, as used in the claims, is not to be construed as limited to the means described thereafter; this term does not exclude other elements or steps. It can thus be interpreted as specifying the presence of the referenced features, values, steps or components, but does not exclude the presence or addition of one or more other features, values, steps or components, or groups thereof. Thus, the scope of the term a device comprising means A and B should not be limited to devices consisting only of components A and B. It means that with respect to the present invention, A and B are the only relevant components of the device.
Reference throughout this specification to "one embodiment" or "an embodiment" means that a specific feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, occurrences of the terms "in one embodiment" or "in an embodiment" in various places throughout this specification may not necessarily all refer to the same embodiment, but may do so. Furthermore, the specific features, structures or characteristics can be combined in any suitable manner, as would be apparent
BE2018 / 5637 are for one of ordinary skill in the art based on this disclosure, in one or more embodiments.
Similarly, it should be appreciated that in describing exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure or description thereof for the purpose of streamlining disclosure and aiding in understanding one or several of the various inventive aspects. In any event, this method of disclosure should not be interpreted as reflecting an intention that the invention requires more features than stated explicitly in any claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single prior disclosed embodiment. Thus, the claims following the detailed description are hereby explicitly incorporated into this detailed description, with each standalone claim as a separate embodiment of this invention.
Furthermore, while some embodiments described herein contain some, but not other, features included in other embodiments, combinations of features of different embodiments are intended to be within the scope of the invention, and constitute various embodiments, as would be understood by those skilled in the art . For example, in the following claims, any of the described embodiments can be used in any combination.
Numerous specific details are presented in the description provided here. In any event, it is understood that embodiments of the invention can be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail to keep this description clear.
In this document, the terms alpha mixing or mixing or mixing (of images or bitmaps) are used as synonyms. In the context of graphical overlay, there is a higher image (with a higher Z order) and a lower image (with a lower Z order). Mixing can be done in hardware and / or software, and is based on a parameter called alpha mix value α, which refers to the level of mixing of the two images, where α = 0% means that the original image is not mixed with the overlay image (or that the overlay image is completely transparent), and where α = 100% means that the resulting pixel value is that of the overlay image (or that the overlay image is completely opaque) except for fully transparent pixels. The alpha mixing value can be expressed on a scale from 0 to 255, where α = 255 corresponds to α = 100% and α = 0 corresponds to α = 0%. For example, alpha blending can be used to fade in or fade out a graphic image. In the context of the present invention, a second parameter is also involved in mixing, denoted by the
BE2018 / 5637 term TransparentColorValue. If a pixel of the overlay image has this predefined (false color or pseudo color) value, this pixel is treated as completely transparent. Fully transparent pixels are typically used, for example, when displaying a rectangular image with rounded edges and / or with rounded corners. Alpha blending and fully transparent pixels are both well known in the art.
When reference is made in the present invention to a color of a pixel, it refers not only to the hue (e.g. red or blue or green or yellow), but also its intensity or brightness, unless otherwise specified or clear from the context.
When reference is made in the present invention to a color expressed in R, G, B values, it means a color value that can be represented by a set of color components, for example, by a set of (R, G, B) values, representative of a red , a green and a blue color component value, each with a value in the range of 0 to 255. Pure black is represented by (0, 0, 0) and pure white is represented by (255, 255, 255). The Ren G and B values for a grayscale pixel are equal.
By pixel intensity or brightness or luminance of a pixel represented by a set of (R, G, B) values is meant a value L which is approximately equal to L = R * 0.3 + G * 0.6 + B * 0.1 or approximately equals R * 3/8 + G * 5/8 + B * 1/8, or other formulas known in the art.
Where reference is made in the present invention to a black, gray or white pixel, it is meant a pixel having a color whose R-value and G-value and B-value are equal when the color is expressed in R, G , B values, For example, a black pixel has RGB values (0, 0, 0), a white pixel has RGB values (255, 255, 255), assuming an 8-bit representation was used.
In this document, the term color of a pixel or value of a pixel refers to the collection of color components, for example, a combination of an R, G and B value. A black or white or gray pixel has equal values for its R, G and B components, and can therefore be represented by the color values (x, x, x). For these pixels, the single number x is also referred to as the value of the pixel.
In this document, the expression with color (R, G, B) means the same as with a color that can be expressed in terms of Red, Green and Blue color components, with red = R, green = G, blue = B, each on a scale from 0 to 255.
In this document, the terms display and monitor and screen are used as synonyms, unless explicitly stated or unless the context clearly indicates otherwise. In the context of the present invention, the display is usually a single physical device.
In this document, the terms checkerboard pattern and checkerboard pattern are used as synonyms.
BE2018 / 5637
In this document, the term perforated bitmap means a bitmap containing a first plurality of fully transparent pixels and a second plurality of other pixels. In practice, the first plurality of pixels have a predefined pseudo-color value recognized by a mixing unit (e.g., a software mixer or graphics processor) to treat these pixels as completely transparent. The other pixels of the bitmap can all be the same color (monochrome), or they can be different colors (eg extracted from a texture bitmap or form a color gradient). The first and second plurality of pixels are preferably arranged in a pattern, e.g., in a checkerboard pattern, as illustrated, for example, in FIG. 10 and FIG. 12 and FIG. 22 and FIG. 37.
By average is meant by spatial averaging or time averaging, or both spatial averaging and time averaging.
In this document, the size, width or height of the screen may refer to (i) the size, width and height of the pixel matrix of the screen (not the border around the pixel matrix), or may refer to (ii) the size, width and height of the so-called work area, which is the same area as (i) minus the area occupied by the so-called task bar. The work area is also the area usually occupied by an application when it is maximized.
In this document, the terms line or bar are used as synonyms. They may refer to a (relatively small) line segment or block, or to a rectangular area that extends across almost the full width or full height of the screen, or the full width or full height of the work area of the screen.
In this document, the expressions word processor or text editor should not be interpreted too narrowly, because many so-called word processors (such as eg Microsoft Word) also have drawing options. Likewise, the term spreadsheet or worksheet should not be interpreted too narrowly, because many so-called worksheet applications, eg Microsoft Excel, also have graphical capabilities.
In this document, the expression the overlay window is configured in click-through mode or the overlay window is configured in pass-through mode that the overlay window is configured in such a way that events (events) of input devices (such as eg a mouse, a keyboard, a touchpad, etc.) are sent by the operating system to one or more underlying application windows or to objects thereof, despite the higher Z-order of the overlay application. This applies, among other things, to events caused by a button press (button-press) or a button release (button-release) or a button click (button-click) or a scroll event of a mouse wheel, and to movements. In the context of the present invention, the overlay application typically includes one window configured in click-through mode, but preferably also a second, mainly
BE2018 / 5637 opaque window (optional with fully transparent pixels), with user interface elements.
In this document, the expression the overlay window is configured in non-click-through mode means the overlay window is not configured in click-through mode.
In this document, the term work area refers to a portion of the pixel area of a screen, namely the area that will be occupied by the application 705 when maximized. In a Microsoft Windows environment, the work area means the entire pixel area except the so-called taskbar, which is usually located at the bottom of the screen, but can also be located elsewhere.
In this document, the phrase a bitmap contains semi-transparent pixels usually means that the bitmap is not just full transparent pixels (so it contains at least one color pixel and optionally one or more full transparent pixels), and is in a window that is configured to be semi-transparent with an alpha transparency in the range of 1% to 99%, but other ways of providing a bitmap with semi-transparent pixels can also be used.
The present invention relates to a method of providing a graphic overlay, and to a computer program product adapted to perform such a method when it is performed on a computer system, and to a computer system comprising such a computer program product.
The present invention will be explained in more detail for the example of a Microsoft Windows desktop PC running a version of Microsoft Windows as the O / S (e.g. Windows XP or Windows NT or Windows 10 or Windows 12 or later Windows versions have the same or similar functionality with respect to functions relevant to the present invention, such as alpha blending and overlay), and running one or more applications selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer , a PDF document editor, a text viewer and a text editor; but the invention is not limited thereto, and may also be used on other computing devices, such as, for example, an Apple computer, or on portable devices, such as smartphones, running respective operating systems and applications.
Referring to the figures,
FIG. 1 shows a schematic block diagram of a classic computer system 100 that includes a computer device 101 connected to a keyboard 102 and a mouse 103 as input devices, and to a monitor or display or screen 104 as an output device. The computer device 101 further generally includes storage components (e.g., in the form of
BE2018 / 5637 flash memory or one or more hard drives) that store software (executable machine instructions), in particular, an operating system, device drivers (device drivers) and one or more applications. The computer device 101 further includes at least one central processing unit CPU configured to execute program instructions of a multitasking operating system with a GUI graphical user interface (e.g. MICROSOFT WINDOWS 10 available from Microsoft Corporation of Redmond, Wash, US), and one or more software applications such as a text viewer or text editor (eg Microsoft Word or Notepad (Notepad) or WordPad) and / or a PDF document viewer (eg Adobe Acrobat Reader) and / or a PDF document editor (eg Nuance Power PDF), and / or an internet browser (eg Microsoft Internet Explorer or Mozilla FireFox or Google Chrome), and / or other applications.
Inputs from the keyboard 102 and mouse device 103 are typically handled by device drivers, configured to receive information through respective input ports (e.g., a serial port or parallel port or USB port). Such computer systems and application programs and device drivers are known in the art.
In the specific example of FIG. 1, the computer device 101 generates a graphic image which is shown on the display 104 as indicated by the rectangular area 109 (slightly offset from the display edge shown for illustrative purposes). The graphic image includes image parts related to a so-called desktop window (visible in this example on the left side of the screen 104), and related to an application window (eg MS Word) showing a text document (in this example on the right side of the screen 104 ). In the specific example of FIG. 1, the desktop window includes an image of a car 107 and three icons 108, each icon representing an application, but this is only an example. Also shown is a so-called mouse pointer 199 or mouse cursor which can take various shapes and moves in accordance with movements of the mouse device 103 under control of the operating system.
FIG. 2 is a schematic representation of a possible Z-order of three windows or image planes 281, 282, 283 that can be used in the computer system of FIG. 1, to obtain the combined image 109 shown on the display 104 of FIG. 1. The first window 281 with the lowest Z-order Z1 is called the desktop window, and is usually generated by the operating system. The second window 282 with a second Z-order Z2 higher than Z1 is associated with application 105, (in the example MS Word), and displays alphanumeric or textual information 106. The third window 283 with a Z-order Z3 higher than Z2 is referred to herein as the mouse bitmap 199. In the example, the mouse bitmap 199 is represented by a relatively small window 283, e.g. with a rectangular shape of about 18x22 pixels, or a square shape of 32x32 pixels, comprising a plurality of first pixels P1 showing a white arrow with a black border,
BE2018 / 5637 surrounded or partially surrounded by a plurality of second pixels P2 which are completely transparent pixels. The desktop image 281 and the application image 282 are usually opaque images. The three image planes 281, 282, 283 are combined in known ways, resulting in the combined image shown on display 104. This combined image can be captured, for example, using a well-known print screen function.
FIG. 3 shows a schematic block diagram of a computer system 300 which may have the same hardware configuration as the system 100 of FIG. 1. The system 300 has storage components that store executable instructions from an operating system with a graphical user interface (eg Windows 10) and a text editor application (eg MS Word), and with a specific overlay application known in the art as LineReader (which can be downloaded at http://www.iconico.com/lineReader/ at the time of writing this document). The overlay application shows a semi-transparent line 310 that follows movements of the mouse pointer 399. Line 310 can be used by a user to underline text fragments 306. In fact, LineReader also has a user interface, but that aspect is not relevant to the present invention and is therefore not shown or discussed further.
FIG. 4 is a schematic representation of a possible Z-order of four windows or image planes 481-484 that can be used in the computer system 300 of FIG. 3, to obtain the combined image 309 shown on the display 304 of FIG. 3. As far as the inventors are aware, FIG. 4 is not publicly available in the prior art, but is a likely implementation.
The first window 481 on Z1 is called the desktop window. The second window 482 on Z2 is associated with application 305, (in the example MS Word) and displays alphanumeric or textual information 306. A third window 483 is associated with the LineReader application and has Z order Z3 higher than Z2 of the text editor application 305. The mouse bitmap or mouse image plane 399 is located in window 484 with Z order Z4 higher than Z3.
The desktop image 481 and the application image 482 are usually opaque images. The window 483 includes a small line or bar 310, and is configured in click-through mode and in semi-transparent mode with a configurable alpha blend value that allows to change the transparency level of the line or bar 310. As shown in FIG. 2, the mouse image plane 484 has first pixels P1 showing a white arrow with a black border at least partially surrounded by a plurality of second pixels P2 which are completely transparent pixels. The four image planes 481-484 are combined in known ways, resulting in the combined image 309 shown
BE2018 / 5637 is shown on the display 304. Part of the bar is shown enlarged to show that all pixels P3 of the line 310 have the same color, for example red or blue.
FIG. 5 shows a schematic block diagram of a computer system 500 similar to that of FIG. 1 when running an operating system O / S and two applications: a text editor and a specific overlay application known in the art as F.Lux, (which at the time of writing this document can be downloaded from https : //www.microsoft.com/en-us/store/p/flux/9n9kdphv91jt), which shows a semi-transparent full-screen overlay window, created according to the same website in 2008, which is almost a decade ago.
FIG. 6 is a schematic representation of a possible Z-order of four windows or image planes 681-684 that can be used in the computer system of FIG. 5, to obtain the combined image 509 shown on the display 504 of FIG. 5. As far as the inventors know, FIG. 5 is not publicly available in the prior art, but is a likely implementation.
The first window 681 on Z1 is called the desktop window. The second window 682 on Z2 is associated with application 505 (in the example MS Word) and displays alphanumeric or textual information 506. A third window 683 at height Z3 is associated with the F.Lux application and has Z order Z3 higher than Z2 of the text editor application 505. The mouse bitmap or mouse image plane 599 is located in window 684 with Z order Z4 higher than Z3.
The desktop image 681 and the application image 682 are usually opaque images. As far as known to the inventors of the present invention, the window 683 contains a monochrome bitmap with pixels P4 all of the same color. The window 683 is configured in click-through mode (ie it sends O / S all mouse events to underlying layers, in this example: to the desktop window 681 or to the text editor window 682 depending on the position of the mouse cursor) and is configured in semi -transparent mode with a configurable alpha blend value. As shown in FIG. 2, the mouse image plane 684 has first pixels P1 showing a white arrow with a black border at least partially surrounded by a plurality of second pixels P2 which are completely transparent pixels. The four image planes 681-684 are combined in known ways, resulting in the combined image 509 shown on the display 504.
The computer system of FIG. 3 (with the LineReader application) can assist a user in reading textual information on the screen, but is not ideal for extracting information from tables or editing a worksheet.
BE2018 / 5637
FIG. 7 shows a schematic block diagram of a computer system 700 according to an embodiment of the present invention. The computer system includes a computer device 701, and a keyboard 702 and a pointing or position device 703 (e.g., a mouse device or a touch pad or a joystick) and a display 704 connected to said computer device 701.
The computer device 701 includes a storage device (not shown) or a memory device (not shown) that includes an operating system O / S with a graphical user interface GUI (e.g., Windows XP or Windows NT or Windows 10 or Windows 12 or later versions of Windows), and at least one application selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer, and a text editor.
The computer device 701 further includes an overlay application showing a semi-transparent overlay window 883 that includes at least two semi-transparent elements: i) a horizontal line 721 or beam, and ii) a vertical line 722 or beam. The horizontal line 721 preferably extends over substantially the entire width Wo of the overlay window 883, which is preferably equal to the width Wd of the pixel area or the working area of the screen. The vertical line 722 or beam preferably extends substantially over the entire height Ho of the overlay window 883, which is preferably equal to the height Hd of the pixel area or the working area of the screen, regardless of the actual size of one or more another underlying application window 705 (eg MS Excel). The overlay application, even though it is configured in click-through mode, ensures the horizontal line
721 and the vertical line 722 move in accordance with movements of the input device 703, or more specifically, that the horizontal line 721 moves in accordance with the Y coordinate of the mouse pointer 799 and the vertical line 722 in accordance with the X coordinate of the mouse pointer 799 moves. Preferably, the horizontal line 721 and the vertical line 722 are positioned such that the mouse pointer 799 is at least partially on both the horizontal line 721 and the vertical line 722, for example such that the pointed end of the mouse pointer 799 is in or near the center of the rectangular intersection of the horizontal line 721 and the vertical line
722. In simple terms, this embodiment provides an overlay application with a semi-transparent cross that moves with the mouse pointer 799.
FIG. 8 is a schematic representation of a possible Z-order of four windows or image planes 881-884 that can be used in the computer system 700 of FIG. 7, to obtain the combined image 709 shown on the display 704 of FIG. 7.
The first window 881 at height Z1 is called the desktop window. The second window 882 on Z2, larger than Z1, is associated with application 705 (in the example MS Excel), which typically presents alphanumeric and / or textual information organized in rows and columns.
BE2018 / 5637
A third window 883 is associated with an overlay application according to an embodiment of the present invention, and has a Z order Z3 higher than Z2 of the spreadsheet application 705. The mouse bitmap or mouse image plane 799 provided by the O / S with graphics GUI user interface is located in window 884 on Z4, higher than Z3.
The desktop image 881 and the application image 882 are usually opaque images. Window 883 provides a substantially full-screen overlay, configured in click-through mode and also configured in semi-transparent mode with a configurable alpha blend value α (also called alpha transparency) ranging from 5% to 95%, or from 10% to 90% or from 20% to 80%, or from 30% to 70%, where α = 0% means that the line is completely transparent, and α = 100% means that the line is completely opaque. The value of α should not be too low, because otherwise the cross is almost invisible and difficult to find. The value of α should not be too high, because otherwise the cross is almost opaque, and the information under the cross is obscured. The skilled person and / or the user can find suitable values.
Preferably, the overlay application also has another window (not shown in FIG. 7 and FIG. 8, but see, for example, FIG. 35 for an example) as a user interface. This user interface window is then preferably higher than Z3 of the click-through window 883. Preferably, this user interface window makes it possible to select different features of the cross, such as for example: color, width of the lines forming the cross and transparency level. .
Referring back to FIG. 8, is a majority portion of the semi-transparent window
883 occupied by fully transparent pixels P5. A minority portion of the semi-transparent window 883 is occupied by the movable horizontal line 721 and the movable vertical line 722. The lines 721, 722 can be implemented as purely rectangular areas, with or without a border (eg a black border), and with or without rounded edges, and with or without rounded corners, and with or without other characteristics. Lines 721, 722 can include monochrome pixels, or can include a color gradient. Preferably, the color of both lines is the same, and adjustable or selectable. In preferred embodiments, the height hh of the horizontal line 721, and the width ww of the vertical line 722 are adjustable. The values of hh and ww are preferably the same (which simplifies the user settings), but this is not absolutely required. As shown in FIG. 2, the mouse image plane 884 has first pixels P1 showing a white arrow with a black border at least partially surrounded by a plurality of second pixels P2 which are completely transparent pixels. The stacking of four image planes 881 to 884 is combined in known ways, resulting in the combined image 709 shown on the display 704.
This overlay application is ideal for finding a correct cell, because the horizontal line 721 and the vertical line 722 are above the respective row headings and column42
BE2018 / 5637 headings. This allows a user to immediately see whether the mouse pointer is positioned on or above the correct cell or not, without having to click on any column headings and / or row headings. This overlay application can be seen as an extremely useful add-on (addition) to spreadsheet applications, but can also be used in extracting information from tables presented on web pages. The tool is especially useful on high-resolution screens (with a resolution of at least 1920 x 1080 pixels).
The tool can drastically reduce the cognitive load of a user attempting to find a particular cell located at the intersection of a particular row (with a particular row heading) and a particular column (with a particular column heading). Without the semi-transparent cross overlay tool, many users first temporarily click a cell near where they think the target cell is likely to be, then verify that the correct cell was selected starting from the selected cell (which after clicking is highlighted) and then move his or her eyes horizontally to the row headers trying to stay in the same row as the selected cell, and if the row is not correct, on the cell above or below the selected cell and repeat the process, and then perform a similar check to verify that the selected cell is in the correct column by starting again from the selected cell and moving his or her eyes up to the column headings, and if the column is not correct, select a cell to the left or right of the selected cell, and optionally select the finally selected cell double check. This takes time and is very error prone. It is of course possible to use other modes, e.g. by panning, or by selecting a specific column by clicking on the column heading, or by using freeze panes (a function in Excel to set a specific number of rows and / or columns), but this also takes time, and introduces the risk of inadvertently changing the height of a row, or the width of a column, etc. On the other hand, with the overlay tool shown in FIG. 7, the user can simply move the cross to be positioned above the correct row heading and column heading, and then click to directly select the correct cell. In addition, the tool makes it possible to quickly double-check the selected position by immediately moving his or her eyes to the row headings and column headings, without attempting to remain in the same row or column.
Although no objective evaluation tests have been performed, the overlay tool is expected to help improve performance and / or concentration for people who frequently work with spreadsheets, and / or the risk of extracting or inserting information from / into an incorrect cell drastically is reduced.
The inventors of the present invention believe that it is not obvious or trivial to arrive at this solution, if due account is taken of the fact that
BE2018 / 5637 More than 100 million people use spreadsheets every day for almost two decades. If the solution proposed herein were obvious, it would have been widely available for many years.
Instead of providing a window 883 with fully transparent pixels P5 and two movable objects in the form of a horizontal line 721 and a vertical line 722, the same visual effect can also be implemented in other ways, for example by providing a window plane 883 with a full screen bitmap with pixels of the color of the cross (e.g. red), and by placing on top of this full screen bitmap four other bitmaps, each of which extends from one of the four corners and which have completely transparent pixels, and by adjusting the width and height of each of these four rectangles according to the cursor position 799, or according to movements of the pointer 703, to emulate the cross as the area not overlayed by the four rectangular bitmaps; or in other suitable ways.
The computer system of FIG. 3 (with the LineReader application) can assist in reading textual information, but does not help reduce eye fatigue caused by, for example, a bright white background. The computer system of FIG. 5 (with the F.Lux overlay application) can help reduce eye fatigue, but does not provide a line or bar to underline text. It would be helpful to have a single overlay application that offers both functions at the same time.
As far as known to the inventors, there is currently no computer application tool available on the market that provides an overlay that simultaneously allows (1) to reduce the brightness of a bright (e.g. white) background, and ( 2) showing a semi-transparent line or a semi-transparent cross that follows mouse movements. Initial experiments have shown that a mere combination of the overlay faces shown in FIG. 4 and in FIG. 6 does not yield acceptable results, as can be understood from FIG. 13 (d) to FIG. 13 (e), due to conflicting requirements. Indeed, for the text information of the underlying document to be easily readable (eg by maintaining sufficient contrast), the transparency level of the third window 883 must be sufficiently high (eg T> = 80% or α <= 20% in FIG. 13 (d) to Fig. 13 (e)). At the same time, for the line to be easily recognizable and the text below the line to be readable at the same time, the transparency level of the third window 883 should be low enough (eg T in the range from 30% to 70%). The exact figures can be discussed, but the expert reader will agree (after implementing the method described above on a computer) that no acceptable or attractive combination can be found, or at least such a solution is a serious compromise is.
There is a need for a method to simultaneously reduce the brightness of a bright background and display a semi-transparent line or a semi-transparent cross that
BE2018 / 5637 mouse movements will follow. The combination should be designed in such a way that (1) text information retains good readability, both text information overlaid by the line and text information not overlaid by the line, and (2) the semitransparent line simply from the background can be distinguished, but is still sufficiently transparent to maintain the legibility of the text below the line.
The present invention addresses this need by providing both a high level of transparency (high T, low α) for the (e.g. full-screen) bitmap, and at the same time providing a low level of transparency (low T, high α) for the line , by perforating the bitmap, for example, as illustrated in FIG. 10 and FIG. 29. By a perforated bitmap is meant a bitmap with a first plurality of pixels P6 that are completely transparent (eg having a pseudo color value), and with a second plurality of pixels P7 having one or more true color values. Pixels P7 will be alpha mixed with underlying layers. As can be understood from FIG. 13 (f) to FIG. 13 (i) provides this combination of functional results for almost all transparency levels from 5% to 95% and almost any color or intensity of the pixels P7, but a preferred result is obtained for transparency levels from about 30% to about 70%. Depending on the intended use, a user can configure the transparency level to a more preferred level.
The combination of the two properties semi-transparent line to underline textual information to reduce cognitive load and the semi-transparent bitmap to change the color temperature to reduce eye fatigue, when combined, create a technical challenge related to how a semi - to combine transparent bitmap and a semi-transparent line such that the transparency level of the bitmap is higher than the transparency level of the line. This poses a technical problem related to image overlay, image blending, semi-transparency, and alpha blending, which is a problem rooted in computer technology, and the solution proposed by the present invention (using a perforated bitmap) is also rooted in computer technology. This technology can be used to easily read textual information on a screen with less eye strain.
It is further noted that it is not the overlay application that generates or displays textual or alphanumeric information, but the one or more underlying application (s), eg a text editor, a spreadsheet, a web browser etc. The method of overlay therefore does not present any information.
Although the overlay application can be run on a standard computer with a standard O / S (e.g. Windows 10 from Microsoft Corporation), the solution proposed herein provides a technical effect related to two different levels of transparency in a single semi-transparent window, which effect is anything but standard, and goes far beyond mere
BE2018 / 5637 electrons flowing through a transistor. Furthermore, at least part of the functionality can be implemented in software on the central processing unit or units of the computer system, or at least part of the functionality can be implemented in a graphics processing unit (GPU), if present in the computer, in in particular the alpha blending operation, and / or the time multiplexing as will be further described when discussing FIG. 28.
FIG. 9 shows a schematic block diagram of a computer system 900 according to an embodiment of the present invention. The computer system 900 includes a computer device 901, and a keyboard 902 and a pointing or positioning device 903 (e.g., a mouse or a touch pad or a joystick or a stylus) and a display 904 connected to said computer device 901.
The computer device 901 includes a storage device (not shown) or a memory device (not shown) that includes an operating system O / S with a graphical user interface GUI (e.g. Windows XP or Windows NT or Windows 10), and at least one application selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer and a text editor. Instead of being stored in the computer device itself, the O / S and / or the at least one application can be downloadable from a server, e.g. via a network interface. (this also applies to other embodiments of the present invention).
The computer device 901 further includes an overlay application showing a semi-transparent overlay window 1083 that includes at least two semi-transparent elements: i) a so-called perforated bitmap 930 occupying a majority portion of the overlay window 1083, and ii) a block or a line 910 which occupies only a minority portion of the overlay window 1083. The line 910 is movable in accordance with movements of the input device 903. Instead of being stored in the computer device itself, the overlay application may be downloadable from a server, e.g. via a network interface.
The overlay window 1083 is configured in click-through mode and in semi-transparent mode with a predefined or adjustable alpha blend value α.
The bitmap 930 includes a first plurality of pixels P6 that are completely transparent, and a second plurality of pixels P7 with a color value that will be alpha mixed with the graphic image formed by the stacking of underlying windows (in the example window 1081 and 1082). In preferred embodiments, the first plurality of pixels P6 are alternately positioned (interleaved) with the second plurality of pixels P7, in a checkerboard pattern, as shown in FIG. 10 (see also FIG. 29). So preferably 50% of the pixels are completely transparent, while the other 50% are semi-transparent. The effect of using this perforated bitmap is that the
BE2018 / 5637 spatial mean color of the bitmap has a seemingly higher level of transparency than it would without the perforation. In this manner, the apparent transparency of the bitmap 930 is made higher than that of the line 910, despite both being associated with the same window 1083 which has only a single alpha blend value α.
As was the case for the overlay application of FIG. 7, the overlay application of FIG. 9 usually with a user interface, which is shown in a second, opaque window (not shown in FIG. 10 but see, for example, FIG. 35). In the embodiment of FIG. 10, the user interface will typically allow a user the size (height and / or width) of the line 910, the transparency level α, the color of the pixels P3 of the line 910, the color of the second plurality of pixels P7, can adjust.
The overlay application, even if one of its windows 1083 is configured in click-through mode, causes the line 910 to move in accordance with movements of the input device 903. This can be accomplished, for example, by repeatedly requesting the position of the native mouse pointer 999. to the operating system (eg based on a timer, or based on polling).
Preferably, the line 910 is positioned such that the mouse pointer 999 is near the center of the line, but this is not absolutely required and the mouse pointer may be slightly offset from this position, e.g. positioned above the line or below the line . Alternatively, the mouse pointer may also be positioned at or near the left end of the line, or at or near the right end of the line. In any case, the overlay application provides a semi-transparent line 910 that moves with the mouse pointer 999, both of which are visible.
FIG. 10 is a schematic representation of a possible Z-order of four windows or image planes 1081-1084 that can be used in the computer system 900 of FIG. 9, to obtain the combined image 909 shown on the display 904 of FIG. 9.
The first window 1081 on Z1 is called the desktop window. The second window 1082 on Z2 is associated with application 905 (in the example, a text editor application, eg MS Word), and displays alphanumeric or textual information 906. A third window 1083 is associated with an overlay application according to an embodiment of the present invention and has a Z order Z3 higher than Z2 of the text editor application 905. The mouse bitmap 999 or mouse image plane is located in window 1084 at Z4, higher then Z3.
The desktop image in the first window 1081 and the application image in the second window 1082 are usually opaque images. The window 1083 provides a substantially full-screen overlay configured in click-through mode and also configured in semi-transparent mode with a configurable alpha blend value α. A minority portion of the overlay window 1083 is occupied by pixels P3 with the line color (e.g. red), one
BE2018 / 5637 majority portion is occupied by the perforated bitmap, in the example a bitmap with a first plurality of pixels P6 which are completely transparent pixels, and a second plurality of pixels P7 with a predefined constant color. It is noted that such a perforated bitmap can be efficiently implemented, for example, by first generating a small bitmap of, for example, 64 x 64 pixels (or 100 x 100 pixels or any other suitable size), and then copying or tiling this small bitmap ( tiling) in the substantially full screen bitmap 1083, but other ways can also be used. In an alternative embodiment, this functionality can be implemented as a functionality in GPU chips, thus requiring only minimal loading of the central CPU of the computing device.
Line 910 can be implemented as a pure rectangular area, with or without a border (e.g. a black border), and with or without rounded edges, and with or without rounded corners, and with or without other features (e.g., with a color gradient).
The overlay application of the computer system 900 is ideal for reading large amounts of text on the screen. In particular, the overlay application can be used to change the color (or color temperature) of the underlying graphic image, and / or its brightness, which can help reduce eye fatigue, and provides a movable line 910 that a user can use to highlight or underline text that is displayed in one or more underlying windows.
In certain embodiments, the color temperature can be changed to reduce the amount of blue light. In other embodiments, the color temperature can be changed to increase the amount of blue light.
Although the same alpha blend value α is used for the line 910 as for the bitmap 930, the bitmap 930 is more transparent than the line when viewed from a distance where the eye does not distinguish individual pixels, but averages pixel information.
For the sake of completeness, it is noted that other perforation patterns were tested (for example, where 4 of the 16 pixels are completely transparent, or 12 of the 16 pixels are completely transparent), but the checkerboard pattern with 50% fully transparent pixels may be preferred as it seems to have the best results in terms of least noticeable artifacts.
FIG. 11 shows a variant of the computer system of FIG. 9, wherein the semi-transparent line 1120 of the overlay application extends over substantially the full width of the screen (e.g., over the full width of the pixel area or the full width of the work area of the
BE2018 / 5637 screen). This implies that line 1120 only has to follow vertical movements of the input device 1103, not horizontal movements.
In the example shown, two applications display textual information, for example, a PDF document viewer on the left side of the screen, and a text editor window on the right side of the screen. This can be used, for example, when comparing two documents side-by-side, for example when proofreading a translation, where the source text is in the PDF document and the target text is in the text document. In the example, the window presents
1282 at height Z2 information associated with the text editor application, and presents the window
1283 at height Z3 information associated with the PDF document viewer.
FIG. 12 is a schematic representation of a possible Z-order of four windows or image planes 1281-1284 as may be used in the computer system 1100 of FIG. 11, to obtain the combined image 1109 shown on the display 1104 of FIG. 11. The mouse cursor area is not shown, and the overlay application user interface window (if any) is not shown, so as not to overload the drawings. Preferably, however, the color of the pixels P3 and of the pixels P7, and the transparency level α, and the height of the line 1120 is adjustable via a user interface, for example a user interface similar to the exemplary user interface of FIG. 35 without a slider to select the width of the line.
FIG. 13 shows a plurality of exemplary screenshots (screenshots). Each row shows a series of five images corresponding to an alpha blend value α of 100%, 80%, 60%, 40% and 20%, respectively, corresponding to an overlay window transparency value T of 20%, 40%, 60%, respectively. 80% and 100%, where α = 100% or T = 0% means the overlay image is completely opaque (except for fully transparent pixels), and α = 0% or T = 100% means the overlay image is completely transparent, and values from 1% to 99% or from 2% to 98% or from 5% to 95% mean that the overlay image is semi-transparent, which means that the pixels of the overlay image are proportionally mixed with the underlying image (except for full transparent pixels).
Each of the images shown in FIG. 13 (a) to FIG. 13 (e) contains a text fragment overlaid with two red lines with color (R, G, B) = (255, 0, 0). Two lines are used for illustrative purposes, to show both highlighting and underlining text in the same drawing.
In FIG. 13 (a), each text fragment is overlaid by two red lines with (R, G, B) = (255, 0, 0). The rest of the overlay window is completely transparent. This series of images shows results comparable to what would be obtained by the overlay application of FIG. 3. As understood
BE2018 / 5637, to provide a line well suited for highlighting and / or underlining underlying textual information without obscuring it, the transparency level should preferably be in the range of about 30% to about 75 %. For lower values of T, the text becomes unreadable. For higher values of T, the line is barely visible. (It is noted that a proper evaluation should be performed on a screen rather than on paper, but the paper version allows to understand the principles of the present invention).
In FIG. 13 (b), each text fragment is overlaid (overlayed) with a semi-transparent bitmap containing only light gray pixels with (R, G, B) = (192, 192, 192), and in FIG. 13 (c), each text fragment is overlaid (overlayed) with a semi-transparent bitmap containing only dark gray pixels with (R, G, B) = (64, 64, 64). This series of images shows results similar to what would be obtained by the overlay application of FIG. 5. In order to maintain good legibility of the underlying text, the transparency level T should preferably be at least about 85% (or α <= 15%).
In FIG. 13 (d), each text fragment is overlayed (covered) by a mere combination of the two red lines as shown in FIG. 13 (a) and through a light gray bitmap as in FIG. 13 (b), and in FIG. 13 (e), each text fragment is overlayed (covered) by a mere combination of the two red lines as in FIG. 13 (a) and through a dark gray bitmap as in FIG. 13 (c).
As described above, when the transparency level T is very low (eg <30%), the line is not sufficiently transparent and text is hidden underneath; when the transparency level T is medium (eg from about 30% to about 70% or about 75%), the line is clearly visible and sufficiently transparent, but the text is not easy to read; and if the transparency level T is too high (eg T> 85%), the text is clearly legible, but the line is almost invisible). Neither of these combinations therefore provides a good solution for both the line and the bitmap.
In FIG. 13 (f), each text fragment is overlayed by two red lines as in FIG. 13 (a) and by a so-called perforated bitmap as described above, where 50% of the pixels are fully transparent pixels P6 arranged in a checkerboard pattern, and the other 50% of the pixels are light gray pixels P7, with (R, G, B) = (192, 192, 192). As can be understood from the drawings (but of course this has to be evaluated on a real screen), very acceptable results for both the line and the bitmap are obtained for transparency levels T from about 30% to about 75%.
In FIG. 13 (g), each text fragment is covered (overlayed) by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as described above, where 50% of
BE2018 / 5637 the pixels are fully transparent pixels P6, arranged in a checkerboard pattern, and the other 50% of the pixels are dark gray pixels P7, with (R, G, B) = (64, 64, 64). Again, very acceptable results for both the line and bitmap are obtained for transparency levels T from about 30% to about 75%.
In FIG. 13 (h), each text fragment is overlaid by two red lines as in FIG. 13 (a) and by a so-called perforated bitmap as described above, where 50% of the pixels are fully transparent pixels P6 arranged in a checkerboard pattern, and the other 50% of the pixels are black pixels P7, with (R, G, B) = (0, 0, 0). As can be understood, the text overlaid by the perforated bitmap remains surprisingly very easy to read, despite 50% of the bitmap pixels being pure black, thanks to the perforation.
In FIG. 13 (i), each text fragment is overlaid by two red lines as in FIG. 13 (a) and by a so-called perforated bitmap as described above, where 50% of the pixels are fully transparent pixels P6 arranged in a checkerboard pattern, and the other 50% of the pixels are white pixels P7, with (R, G, B) = (255, 255, 255). As can be understood, the text overlaid by the perforated bitmap remains surprisingly very easy to read, despite 50% of the bitmap pixels being pure white, thanks to the perforation.
As can be understood from all screenshots in FIG. 13 (f) to FIG. 13 (i), it can be understood that the color of the incompletely transparent pixels P7 of the perforated bitmap can be varied over a wide range (from pure black to pure white), and is not critical to the legibility of the underlying textual information.
While the inventors do not wish to be bound by any theory, FIG. 14 and FIG. 15 help to better understand the difference between mixing with a monochrome bitmap versus mixing with a perforated bitmap that contains 50% fully transparent pixels. The vertical axis shows the intensity (I), the horizontal axis is transparency T or alpha transparency α, where α + T = 100%.
FIG. 14 (a) illustrates what happens to a graphic image that contains text information with Original Text Pixels OT (black) with color (0, 0, 0) and Original Background Pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a monochrome bitmap (as in FIG. 6) containing 100% black pixels with color (0, 0, 0), for different values of the transparency T or alpha blend value α.
In general, the result of overlaying or mixing two bitmaps is a color with the following color components:
BE2018 / 5637 (Rm, Gm, Bm) = (R1, G1, B1) x (1-T) + (R2, G2, B2) xT [1] where Rm, Gm, Bm are the color components of pixels of the resulting image (the mixed image), T is the transparency level, (R1, G1, B1) is the color of a pixel of the first (overlay) bitmap, and (R2, G2, B2) is the color of a pixel of the second (underlying) bitmap.
When mixing with a black monochrome bitmap, then (R1, G1, B1) = (0, 0, 0).
If T = 100%, originally white background pixels OB (255, 255, 255) and originally black text pixels OT (0, 0, 0) remain unchanged after mixing.
If T = 50%, then originally white background pixels OB (255, 255, 255) become gray pixels MB (127, 127, 127) aside from rounding errors, and originally black text pixels remain black MT (0, 0, 0).
If T = 0%, then originally white background pixels OB (255, 255, 255) and originally black text pixels OT (0, 0, 0) both turn black (0, 0, 0).
This is more or less visualized by the series of FIG. 13 (c), except that in FIG. 13 (c) has the monochrome bitmap pixels P4 with color (64, 64, 64) instead of (0, 0, 0).
FIG. 14 (b) illustrates what happens to a graphic image containing text information with Original Text Pixels OT (black) with color (0, 0, 0) and Original Background Pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a monochrome bitmap (as in FIG. 6) containing 100% white pixels with color (255, 255, 255), for different values of the transparency T or alpha blend value α.
The same mixing formula [1] applies, but now (R1, G1, B1) = (255, 255, 255).
If T = 100%, white pixels (255, 255, 255) and black pixels (0, 0, 0) remain unchanged after mixing.
If T = 50%, white pixels (255, 255, 255) remain white (255, 255, 255), and black pixels (0,
0, 0) become gray pixels (127, 127, 127), except for rounding errors.
If T = 0%, then white pixels (255, 255, 255) and black pixels (0, 0, 0) both become (255, 255, 255).
This is more or less visualized by the series of FIG. 13 (b), except that in FIG. 13 (b) has the monochrome bitmap pixels P4 with color (192, 192, 192) instead of (255, 255, 255).
FIG. 15 (a) illustrates what happens to a graphic image containing text information with Original Text Pixels OT (black) with color (0, 0, 0) and Original Background Pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a perforated bitmap (as in FIG. 10 or FIG. 12) containing 50% fully transparent pixels P6, and 50% black pixels P7 with color (0, 0, 0), for different values of the transparency T (or alpha mixing value α).
BE2018 / 5637
In general, the result of overlaying or mixing with a perforated bitmap with 50% fully transparent pixels P6 is:
(Rm, Gm, Bm) = (R1, G1, B1) x (1-T) + (R2, G2, B2) xT [2] if the overlay pixel is not completely transparent, which is the case for pixels P7, (Rm, Gm, Bm) = (R2, G2, B2) [3] if the overlay pixel is completely transparent, which is the case for pixels P6, so average (if the number of pixels P6, P7 is 50% each):
(Rm, Gm, Bm) = 50% * [(R1, G1, B1) x (1-T) + (R2, G2, B2) xT] + 50% * (R2, G2, B2) [4] where Rm, Gm, Bm are the color components of pixels of the resulting image (the mixed image), T is the transparency level, (R1, G1, B1) is the color of a pixel of the first (overlay) bitmap, and (R2, G2, B2) is the color of a pixel of the second (underlying) bitmap.
Specifically, when mixing with a perforated black bitmap with 50% fully transparent pixels P6 and 50% black pixels P7, then (R1, G1, B1) = (0, 0, 0).
If T = 100%, white pixels (255, 255, 255) and black pixels (0, 0, 0) remain unchanged.
If T = 50%, white pixels (255, 255, 255) become on average gray pixels (191,191,191), and black pixels remain black (0, 0, 0).
If T = 0%, white pixels (255, 255, 255) on average become gray pixels (127,127,127), and black pixels (0, 0, 0) remain black (0, 0, 0).
This is more or less visualized by the series of FIG. 13 (h), if the two lines are omitted.
FIG. 15 (b) illustrates what happens to a graphic image containing text information with Original Text Pixels OT (black) with color (0, 0, 0) and Original Background Pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a perforated bitmap (as in FIG. 10) containing 50% fully transparent pixels P6, and 50% white pixels with color (255, 255, 255), for different values of transparency T or alpha blend value α .
The same formulas [2] to [4] apply.
When mixing with a perforated white bitmap with 50% fully transparent pixels P6 and 50% white pixels P7, then (R1, G1, B1) = (255, 255, 255).
If T = 100%, white pixels (255, 255, 255) and black pixels (0, 0, 0) remain unchanged.
If T = 50%, white pixels (255, 255, 255) remain white, and black pixels become average (64, 64, 64).
If T = 0%, white pixels (255, 255, 255) remain white, and black pixels become gray pixels (127,127,127) on average.
BE2018 / 5637
This is more or less visualized by the series of FIG. 13 (i), if the two lines are omitted.
So FIG. 14 can help to understand what happens to black text on a white background when overlaid by a monochrome (non-perforated) bitmap, as is the case for the movable object, and FIG. 15 can help you understand what happens to black text on a white background when overlaid by a perforated bitmap.
The following section explains what happens to black text on a white background when overlaid by a red line. The same formula [1] applies, but now it holds that (R1, G1, B1) is (255,0,0).
FIRST EXAMPLE:
As a first example, the colors of the image fragment of FIG. 13 (e) for T = 80% calculated.
The original image includes: black text (0, 0, 0) on a white background (255, 255, 255).
The overlay window includes a (non-perforated) monochrome bitmap with black pixels (0, 0, 0), and a line with red pixels (255,0,0), and mixing is done with T set to 80%. Using formula [1], it can be calculated that:
- white background pixels overlaid by the black overlay bitmap become: (204, 204, 204),
- black text pixels overlaid by the black overlay bitmap become: (0, 0, 0),
- white background pixels overlaid by the red line are:
80% * (255, 255, 255) +20% * (255, 0, 0) = (255, 204, 204), which is very difficult to distinguish from (204, 204, 204),
-the black text pixels overlaid by the red line become: 80% * (0, 0, 0) +20% * (255, 0, 0) = (51, 0, 0).
SECOND EXAMPLE:
As a second example, the colors of the image fragment of FIG. 13 (h) before
T = 60% calculated.
The original image includes: black text (0, 0, 0) on a white background (255, 255, 255).
The overlay includes a perforated bitmap with 50% fully transparent pixels P6 and 50% black pixels P7 (0, 0, 0), and a line with red pixels (255, 0, 0), and mixing is done with T set to 60 %. Using formulas [1] to [5], it can be calculated that:
BE2018 / 5637
-white background pixels overlayed by the perforated bitmap average (204, 204, 204), namely (255, 255, 255) for pixels are overlayed by fully transparent pixels P6, and (153, 153, 153) for the pixels overlayed by a black pixel P7;
- black text pixels overlaid by the black overlay bitmap become: (0, 0, 0),
- white background pixels overlaid by the red line are:
60% * (255, 255, 255) +40% * (255, 0, 0) = (255, 153, 153), which is easy to distinguish from (204, 204, 204),
-the black text pixels overlaid by the red line are:
60% * (0, 0, 0) + 40% * (255, 0, 0) = (102, 0, 0).
Comparison of the first and second examples (or the sample of FIG. 13 (e) for T = 80% and the sample of FIG. 13 (h) for T = 60%) shows that:
- in both cases, black text on a white background (0, 0, 0) becomes an average background color of (204, 204, 204);
- in the case of a non-perforated bitmap, text is overlayed by the red line (51, 0, 0) on (255, 204, 204), which text is easy to read, but the line is very difficult to distinguish from the background of (204, 204, 204), and therefore not very helpful for underlining, while in the case of the perforated bitmap, text is overlaid by the red line (102, 0, 0) on (255, 153, 153), which text is still very legible, but moreover the line can be easily distinguished from the average background of (204, 204, 204).
Tests have shown that (for the settings of FIG. 13 (h), which means: a perforated bitmap with 50% fully transparent pixels and 50% pure black pixels, arranged in a checkerboard pattern), very good results are also obtained for many other line colors besides pure red, for example a black line with color (0, 0, 0), a blue line with color (0, 0, 255), a dark red line with color (128, 0, 0), a dark blue line with color (0, 0, 128), a dark green line with color (0, 50, 0), a dark teal line with color (0, 51, 102), a green line with color (0, 128, 0 ), etc., in fact for most relatively dark lines, but also for most relatively bright lines, e.g. a yellow line with color (255, 255, 0), a bright green line with color (0, 255, 0 ), a turquoise line with color (0, 255, 255), a light gray line of (192, 192, 192) etc. Preferably, the line color can be selected by the user via a user interface, which can be allows the user to select from a variety of suitable colors that yield good results.
While the examples in FIG. 13 are provided for an overlay bitmap (perforated or unperforated) with black, dark gray, light gray or white pixels, the invention is not limited thereto, and the invention also works when these pixels are not grayscale pixels, but are e.g. bluish pixels or yellowish pixels ( e.g. with color (81, 69, 44) or (83, 69, 40) or
BE2018 / 5637 (86, 69, 35) etc.) or reddish pixels, or any other color. In this way, the original white background color can be transformed into a bluish or yellowish or reddish, ... background, which may be more pleasant to read. In particular, yellowish pixels tend to provide a paper-like appearance, especially in combination with a texture bitmap (see below), and bluish pixels tend to provide a sky-like or day-like appearance, etc., both for the perforated bitmap and the non-perforated bitmap. This technique can also be used, for example, to reduce the blue light content emitted from a display, which some sources believe may affect user sleep.
FIG. 16 to FIG. 18 show three examples of a slightly larger text fragment, allowing a better understanding of how a reader would view the textual information on a screen.
FIG. 16 is a screenshot (screen shot) of a text fragment extracted from US6333753B1, presented by a PDF viewer called Nuance PDF Reader, with the document scaled to 200% in the PDF viewer, displayed on a display with a native resolution of 3840 x 2160 pixels, configured with a resolution of 2560 x 1440 pixels, and where the text and applications are scaled to 100% in Windows screen settings. FIG. 17 to FIG. 18 is a screenshot of the same text fragment, but overlayed by an overlay application according to the present invention.
FIG. 16 shows an exemplary text fragment that contains black text on a white background. In fact, many applications do not display textual information as pure black pixels, but also use grayscale pixels, especially near the edges of the alphanumeric characters. This aspect is known, but is not particularly relevant to the present invention.
FIG. 17 shows the text fragment of FIG. 16, overlayed by an overlay window containing a blue line segment containing pixels of the color (R, G, B) = (0, 0, 255), and with the transparency level T of the overlay window set to 50%. Transparency level T only affects the transparency of the line, not the fully transparent overlay pixels. Such an image can probably also be obtained with the LineReader application, as discussed in FIG. 3 and FIG. 4.
FIG. 18 shows the text fragment of FIG. 16, overlayed by a method of the present invention, as described in FIG. 9 and FIG. 10. The alpha blend value is 50%, a minority portion of the overlay image is occupied by a blue line segment with color value before mixing (R, G, B) = (0, 0.25), a majority portion of the overlay window is occupied by a perforated bitmap, this bitmap having 50% pure transparent pixels P6 and 50% gray pixels P7 with color value before mixing
BE2018 / 5637 (R, G, B) = (128, 128, 128), arranged in a checkerboard pattern. As can be understood from FIG. 18, the resulting textual information is easy to read, the line can help the user to stay focused, and the line can be easily distinguished from the background.
However, the inventors went a step further, and also experimented with overlaying textures (perforated or unperforated). Tests have shown that overlay with certain textures can transform a pure white background (which appears shiny) into a matte background, which can be much more pleasant to read. In one experiment, an image was taken of a foggy sky, a sub-image was extracted, a horizontal and vertical gradient was removed, the average color was adjusted, the contrast between the brightest pixels and the darkest pixels was adjusted, and the sub-image was adjusted as a semi-transparent overlay tiled over the screen, which provided surprisingly good results, in particular because it seemingly removes the gleaming white background from most text documents, replacing it with a matte background, while leaving the text surprisingly easy to read.
If no semi-transparent line or cross has been added in the overlay (e.g. for underlining text to be read, or for indicating the position of a cell in a table or spreadsheet), good results can be obtained by using a relatively high transparency value (eg T in the range of about 80% to 99% for a non-perforated bitmap, or T in the range of about 60% to 99% for a perforated bitmap).
If a line or cross is added in the overlay window, it may be better to punch the bitmap as described above. Surprisingly, the texture pattern can be made very visible even with a 50% perforation, and aesthetically pleasing results can be obtained. However, tests have shown that a single texture bitmap may not be suitable for all transparency levels unless some modification of the pixel values is made. In either case (perforated or not), the contrast of the texture bitmap should be adjusted or adjusted as a function of the transparency level T or alpha transparency level α of the overlay window.
In preferred embodiments, the transparency level T of the overlay window is selectable or adjustable by the user, and the contrast level of the texture bitmap is automatically adjusted as a first or second order polynomial of the transparency level T. The value of the contrast itself is not critical (this can typically range from about 0.5 times an optimal value to about 2.0 times the optimal value), but it is important that this value is not fixed, but can be adjusted when the transparency of the overlay window is varied, otherwise the texture are too pronounced and disturbing to the reader, or are almost invisible and look almost the same as a monochrome color overlay. Of course it is also possible to
BE2018 / 5637 use multiple texture bitmaps in the overlay application, but that consumes more memory space, and can be avoided.
The inventors have found that good results, e.g. near-optimal results, are obtained when the contrast is adjusted so that the texture or pattern is barely visible, or slightly visible above. If the contrast level is chosen below this ideal value range, a white background is converted to a shiny (eg glossy plastic-like) background without the texture or pattern being noticeable. If the contrast level above this ideal value range is chosen, the pattern or texture becomes dominant and may distract the user. However, when the contrast level is chosen within the ideal value range, the pattern or texture will cause a slight variation of the background color, which is perceived or perceived as a texture or a pattern or a matte color, and this even appears to be the sharpness of the text characters below that. The ideal value range can be different for each pattern bitmap, and is preferably also user-tunable, in addition to the automatic adjustment related to the transparency value T.
FIG. 19 shows a schematic block diagram of a computer system 1900 according to an embodiment of the present invention, which can be viewed as a variant of the computer system of FIG. 10 and the computer system of FIG. 12. Most of what has been described above applies here, too, except that the overlay image of the computer system 1900 of FIG. 19 comprises at least two semi-transparent elements: i) a (non-perforated) line 1910, and ii) a perforated texture bitmap 1930. The line 1910 is movable according to movements of the input device 1903.
The perforated texture bitmap can be derived from a predefined texture bitmap (see, for example, FIG. 27) by assigning a pseudo color value corresponding to a predefined full transparency value to half of the pixel locations, which are the pixels P6, and by optionally changing the average color of the other pixels P8, optionally adjusting the contrast of the pixels P8, and optionally tiling the said bitmap to fill the area of the overlay window 2083.
FIG. 20 is a schematic representation of a Z-order of four windows or image planes 2081-2084 that can be used in the computer system of FIG. 19.
FIG. 21 shows a schematic block diagram of another computer system 2100 according to an embodiment of the present invention, which computer system includes an overlay application for providing an overlay window comprising at least one semi-transparent object in the form of a so-called perforated texture bitmap with a first multitude completely
BE2018 / 5637 transparent pixels P6 and a second plurality of opaque or semi-transparent pixels P8. This embodiment is a variant of the embodiment of FIG. 21, omitting line 1910.
FIG. 22 is a schematic representation of a Z-order of four windows or image planes 2281-2284 that can be used in the computer system of FIG. 21. As described above, the overlay application may also have a second, opaque user interface window (not shown), where a user can select an average color of the pixels P8, and / or adjust the alpha transparency value α, and / or manually can adjust a contrast of the texture bitmap, and / or select a texture bitmap from a list of texture bitmaps.
FIG. 23 shows a schematic block diagram of another computer system 2300 according to an embodiment of the present invention, which computer system includes an overlay application for providing an overlay window containing at least one semi-transparent object in the form of a so-called non-perforated texture bitmap consisting of a plurality of semi-transparent pixels P8. This embodiment is a variant of the embodiment of FIG. 23, omitting the fully transparent pixels P6.
FIG. 24 is a schematic representation of a Z-order of four windows or image planes 2481-2484 that can be used in the computer system 2300 of FIG. 23.
As can be understood from FIG. 14 and FIG. 15, provide embodiments with an overlay application with a perforated texture bitmap (illustrated in FIG. 21 and FIG. 22) for a transparency level T ranging from 0% to 100% on the one hand, and embodiments with an overlay application with a non-perforated texture bitmap (illustrated in FIG. 23 and FIG. 24) for a transparency level T in the range of 50% to 100% on the other hand, (spatial average) comparable results. These embodiments are extremely useful for converting a glossy (e.g. plastic-like) background surface into a matte (e.g. paper-like) background surface, which is more pleasant to read.
FIG. 25 shows the text fragment of FIG. 16, overlayed (covered) by an overlay image using a transparency level T of 50% (or an alpha blend value of 50%), where a minority portion of the overlay image is occupied by a blue line segment (as in FIG. 16) and a majority portion of the overlay window includes a perforated texture bitmap, the bitmap having 50% fully transparent pixels and the other 50% pixels forming a texture bitmap with an average color value of (R, G, B) = (128, 128, 128). Thus, the resulting overlayed image has an average color of approximately (224, 224, 224).
BE2018 / 5637
Comparison of FIG. 25 and FIG. 18 provides an example of the difference between an overlay with a perforated monochrome bitmap, and an overlay with a perforated texture bitmap. This overlay image can be created by the computer system 1900 of FIG. 19.
FIG. 26 shows an example of an overlay image as can be generated by the computer system of FIG. 23, using a non-perforated texture bitmap with an average color of about (61, 61, 61) and using a transparency level of T = 80%, resulting in an average background color of the overlayed image of about (216, 216, 216). Comparison of FIG. 26 and FIG. 25 demonstrates that using a perforated bitmap does not or does not significantly degrade the image quality, and therefore the readability of the textual information.
FIG. 27 shows the exemplary texture bitmap used to image the FIG. 25, before color adjustment, contrast adjustment and perforation. In this example, the texture bitmap was a grayscale image, but the invention is not limited thereto, and colored texture images can also be used.
Experiments have shown that the texture bitmap can be easily transformed into another texture bitmap of a desired color (Rnew, Gnew, Bnew) by treating each of the color components separately, for example using the following formulas or equivalent formulas:
Rnew [x, y] = minmax (0, CF x (Rorig [x, y] -Ravg) + Rnew, 255);
Gnew [x, y] = minmax (0, CF x (Gorig [x, y] -Gavg) + Gnew, 255);
Bnew [x, y] = minmax (0, CF x (Borig [x, y] -Bavg) + Bnew, 255);
[6] [7] [8] where Rnew [x, y] is the Red component of the pixel of the new texture bitmap at position (x, y); and Rorig [x, y] is the Red component of the pixel of the original texture bitmap at position (x, y); and CF is the contrast factor (CF = 1 means no contrast increase, CF <1 means decrease contrast, CF> 1 means increase contrast); Ravg is the mean Red color component,
Gavg is the average Green color component, and Bavg is the average Blue color component of the original texture bitmap; and where minmax (a, b, c) is a function that provides the result b limited to the range from a to c, thus returning the value b unless b <a in which case a is returned, or unless b> c in which case c is returned. CF is a floating point number, but the result of each formula is converted to an integer from 0 to 255 (also called byte).
BE2018 / 5637
CF can be calculated using one of the following formulas, or an equivalent formula:
CF = K + L x T; [9] or
CF = M + N x T + Q x (TxT) [10] where K, L, M, N, Q are predefined constants that may depend on the specific texture bitmap chosen, and T is the transparency level of the overlay window.
In simple terms, these formulas can be used to shift a histogram up or down the pixel values of the texture bitmap (for example, from Ravg to Rnew), and which can widen or narrow the width of the histogram characteristic.
FIG. 28 schematically illustrates how the overlay with a perforated bitmap, for example a perforated monochrome bitmap or a perforated texture bitmap, which is repeatedly shifted back and forth by a single pixel distance, can be used for time multiplexing or time averaging, in addition to spatial averaging.
FIG. 28 (at time t1) shows a pattern consisting of four pixels, two of these pixels are fully transparent pixels P6, the other two pixels are color pixels that are opaque or that will be alpha mixed. This 2x2 pixel pattern is typically repeated (tiled) over the entire overlay bitmap.
FIG. 28 (at time t2) shows a pattern similar to that of FIG. 28 (at time t1), but where other pixel locations are completely transparent.
It is interesting to note that the pattern of FIG. 28 (at t2) can be obtained by a mere shift of the bitmap of FIG. 28 (at t1) by 1 pixel up, or 1 pixel to the left, or 1 pixel to the right or 1 pixel down. So if the overlay bitmap is intentionally chosen 1 pixel wider and / or 1 pixel higher than the width W and height H of the overlay window, and if the overlay bitmap is shifted back and forth each frame by 1 pixel position, or every two frames, or each three frames, the underlying image is overlaid in a time-multiplexed manner. If the multiplex frequency is sufficiently high (e.g., at least 15 Hz or at least 20 Hz or at least 25 Hz or at least 30 Hz), time multiplexing can reduce visual artifacts caused by overlay with the perforated bitmap. If the multiplex frequency is low (for example, less than 10 Hz), pixels on edges may appear to swirl.
It is also possible to use two separate bitmaps, each of the size W x H, and to alternately mix with one bitmap, but such an implementation would require more memory and more time to create the two bitmaps, and is therefore not preferred.
BE2018 / 5637
FIG. 29 and FIG. 30 have been added for the sake of completeness to show that the principles described above still work despite further image processing outside of the overlay application itself. Experiments have shown that the screen resolution and the scaling factor or zoom factor for text and applications selected in the calculator also affect how the final image will look. Two specific examples are shown here, in FIG. 29 (a) - (c) on the one hand and FIG. 30 (a) - (c) on the other.
FIG. 29 (a) shows a screenshot of the same image as used in FIG. 18 (with a perforated monochrome bitmap overlay), but with 2 lines, displayed on a display device with a native resolution of 3840 x 2160, configured at a resolution of 2560 x 1440, with a scale factor (or zoom factor) for text and applications of 100%, as for example can be selected in the screen resolution dialog in case of the Windows 10 operating system.
FIG. 29 (b) shows the image of FIG. 29 (a), zoomed in by a factor of about 350%. Individual pixels can be distinguished in this image. The checkerboard pattern becomes visible but the text remains very legible.
FIG. 29 (c) shows a portion of the image of FIG. 29 (b), zoomed in further by a factor of about 200%, i.e. zoomed in total by a factor of about 700%. In this image, the effect of overlaying a perforated bitmap with 50% fully transparent pixels and 50% semi-transparent pixels is clearly visible. It was very surprising to the inventors that the individual pixels are not, or not significantly, noticeable in FIG. 29 (a) even if they are actually present.
FIG. 30 (a) shows the same image as FIG. 29 (a), displayed on a display device with a native resolution of 3840 x 2160, configured at a resolution of 2560 x 1440, but with a scaling factor (or zoom factor) for text and applications of 125%, as can be selected, for example in the screen resolution dialog in case of the Windows 10 operating system.
FIG. 30 (b) shows the image of FIG. 30 (a), zoomed in by a factor of about 350%. Individual pixels can be distinguished in this image, but the checkerboard pattern is less pronounced.
FIG. 30 (c) shows a portion of the image of FIG. 30 (b), zoomed in further by a factor of about 200%, i.e. zoomed in total by a factor of about 700%. In this image, the effect of overlaying a perforated bitmap with 50% fully transparent pixels and 50% semi-transparent pixels is no longer clearly visible, but instead a pattern of 5 by 5 pixels seems to have been added to the image , probably due to filtering related to zoom, performed outside the overlay application, but inside the computer device, since a screenshot can be taken.
BE2018 / 5637
Again, the individual pixels are not, or not significantly, noticeable in FIG. 30 (a), even if they are actually present. The text of FIG. 30 (a) is very legible, and the line is easily distinguishable from the background.
It seems to be a matter of taste which of the two images, that of FIG. 29 (a) or that of
FIG. 30 (a) is best. The image of FIG. 29 (a) appears somewhat sharper, that of FIG. 30 (a) appears to show some wrinkle or pattern due to the combined effect of the perforation followed by filtering, which is also aesthetically pleasing as it gives the impression of a matte background. In any case, both images provide text that is easy to read, both text in the background and text below the line, and in both cases the line is easily distinguishable from the background, and in both cases the average brightness of the white background reduced, which can lead to less eye fatigue.
Experiments with other scale factors for text and applications (in particular 150%, 175%, 225%, 250%, 350%) show results similar to those of FIG. 30 (a) to FIG. 30 (c), while scale factors of 200% or 300% appear to yield results similar to those of FIG. 29 (a) to FIG. 29 (c).
FIG. 31 shows a flow chart of a computer-implemented method according to some embodiments of the present invention. This method is performed by a computer system of FIG. 9, FIG. 11, FIG. 19, FIG. 21, FIG. 36 and FIG. 38.
An overlay window is provided in step 3101.
The overlay window can be configured as an opaque window, which means that this window can have non-transparent pixels and / or fully transparent pixels, but not semi-transparent pixels.
Alternatively, the overlay window can be configured as a semi-transparent window with an alpha transparency α in the range from 1% to 99% or from 2% to 98% or from 5% to 95%, which means that this window has completely transparent pixels and / or semi-transparent pixels, but not completely opaque pixels.
In step 3102, at least one visible object is provided in said overlay window, the at least one visible object comprising a bitmap with a first plurality of pixels P6 which are fully transparent pixels, and a second plurality of pixels P7, P8 which are opaque pixels or semi -transparent pixels (ie not completely transparent pixels); and wherein the first plurality of pixels P6 and the second plurality of pixels P7, P8 are interleaved (alternately arranged), for example arranged in a checkerboard pattern.
In step 3103, the overlay window is configured in click-through mode.
Step 3103 can be performed before step 3102.
BE2018 / 5637
In one embodiment, the overlay window is configured as an opaque window, and the second plurality of pixels P7, P8 are opaque pixels.
In another embodiment, the overlay window is configured as a semi-transparent window, and the second plurality of pixels are semi-transparent pixels.
Configured in click-through mode means that the operating system O / S is informed that input messages (e.g. caused by mouse and keyboard events) must be sent to one or more applications that are overlayed, or to components thereof, instead from to the overlay application itself. Or more correctly, if the overlay application contains, for example, two windows, one user interface window that is not configured in click-through mode and another window that is configured in click-through mode, it will send O / S mouse events to the user interface window when the mouse is over this UI window, and to the one or more underlying applications if the mouse is over the click-through window, but not above the user interface window.
FIG. 32 shows a flow chart of a more specific method 3200 according to a preferred embodiment of the present invention. This method is performed by a computer system of FIG. 9, FIG. 11, FIG. 19, FIG. 36 and FIG. 38.
In step 3201, an overlay window is provided, and configured as a semi-transparent window with an alpha transparency α in the range of 1% to 99% or from 2% to 98% or from 5% to 95% or from 10% to 90% or from 20% to 80% or from 30% to 70%.
In step 3202, at least one visible object (e.g., a horizontal line) is provided in said overlay window, which occupies only a minority portion (e.g., less than 20% or less than 10%) of the area of the overlay window. For example, if the overlay window has a width W and a height H, the first object can be, for example, a short horizontal line with a length less than W / 2 and a height less than H / 10, thus occupying an area that is smaller then WxH / 20; or may be, for example, a horizontal line extending across the entire width W of the overlay window, having a height less than H / 10, thus occupying an area smaller than WxH / 10.
In step 3203, a bitmap, or an object with a bitmap, is provided, which bitmap has a first plurality of pixels P6 that are completely transparent, and a second plurality of pixels P7, P8 with a color value to be alpha mixed with pixels of one or more underlying windows. The second plurality of pixels P7 can all be the same color (perforated monochrome bitmap), or they can be different colors. The colors can form a color gradient, or can form a texture bitmap. The first and second plurality of pixels are interleaved (alternately positioned), for example according to a fixed pattern, for example a checkerboard pattern. The bitmap can be above the visible object.
BE2018 / 5637
In step 3204, the overlay window is configured as a so-called click-through window. The operating system will send input events such as mouse clicks and mouse movements to one or more underlying windows (or to the desktop window).
In step 3205, position information X, Y is obtained from a so-called mouse pointer or mouse cursor, for example in the form of screen coordinates. For example, the mouse cursor position can be queried from the operating system, for example in the case of Windows O / S by using the GetCursorPos function. It is an advantage to use these coordinates because it allows to align or position the at least first visible object relative to the mouse cursor. Alternatively, motion or displacement information dx, dy can be obtained from the operating system in the form of raw input messages, for example, by configuring the operating system to send raw input messages, and by configuring the overlay application to send raw input messages. received and processed. This aspect is described in more detail in patent application BE2017 / 5895 filed by the same applicant on December 4, 2017, entitled METHOD AND DEVICE AND SYSTEM FOR PROVIDING DUAL MOUSE SUPPORT, which document is incorporated by reference herein in its entirety, and further herein is referred to as the co-pending double mouse application. In the event of any inconsistencies between the present application and the co-pending application, the present document prevails.
In step 3206, a position of the at least one visible object (e.g., the X and Y position of a short horizontal line, or the Y position of a large horizontal line) is adjusted in accordance with the obtained mouse cursor position X, Y or in accordance with the motion or displacement information dx, dy.
In the case of a small or large cross, the mouse cursor position is preferably at or near the intersection of the horizontal and vertical lines. In the case of a small horizontal line, the mouse cursor may be at the left end or right end, or in the center of the horizontal line, or in any other suitable position.
Step 3205 and Step 3206 are repeated, for example triggered by a timer event with a period in the range of 1 ms to 200 ms, for example in the range of 2 ms to 100 ms, for example equal to about 5 ms or about 10 ms , or about 16.7 ms (60Hz), or about 20 ms or about 25 ms, or about 30 ms, or about 40 ms, or about 50 ms, or about 100 ms, or about 150 ms. This value is not critical. The smaller this value, the faster the overlay application responds to mouse movements, but the more computing power is required, among other things, for drawing the at least one object at intermediate positions during movement of the input device. Those skilled in the art can find a suitable compromise between fast response time and a reasonable amount of computing power. The overlay application can allow the user to select from at least two predefined values.
BE2018 / 5637
FIG. 33 shows a flow chart of another method 3300 according to an embodiment of the present invention. This method is performed by the computer system of FIG. 7 and the computer system of FIG. 38.
An overlay window is provided in step 3301.
The overlay window may have a bitmap that substantially fills the entire area of the window with fully transparent pixels (as shown in FIG. 7 and in FIG. 38), or it may include a perforated bitmap (as in FIG. 10 or FIG. 12). ) or a perforated texture bitmap (as in FIG. 20 and FIG. 22) or a non-perforated texture bitmap (as in FIG. 24).
In case the window includes fully transparent pixels (as shown in FIG. 7 and FIG. 38), the overlay window 883, 3983 can be configured as opaque or semi-transparent. In case the window includes a perforated bitmap, the overlay window 883, 3983 is preferably configured as semi-transparent. Preferably, the alpha transparency of the semi-transparent window is a value α in the range from 1% to 99% or from 2% to 98% or from 5% to 95% or from 10% to 90% or from 20% to 80 % or from 30% to 70%.
In step 3302, a first visible and movable object in the overlay window is provided in the form of a horizontal line 721, 3821 occupying only a minority portion (e.g., less than 20% or less than 10%) of the area of the overlay window. The horizontal line preferably extends over the entire width of the overlay window, which preferably itself extends over the entire pixel area of the screen, or over the entire working area of the screen.
In step 3303, a second visible and movable object is provided in the overlay window that is in the form of a vertical line 722, 3822 occupying only a minority portion (e.g., less than 20% or less than 10%) of the area of the overlay window. The vertical line preferably extends over the entire height of the overlay window, which preferably extends itself over the entire pixel area of the screen, or over the entire working area of the screen.
Together, the horizontal line and the vertical line form a cross. If the window includes a bitmap, the lines are located above the said bitmap so that they are visible. In the embodiment of FIG. 7, the horizontal line 721 and the vertical line 722 are monochrome lines, for example of a red or blue or black or purple or yellow color, or any other color. In this case, the overlay window is preferably configured as semi-transparent so that text is visible below lines 721, 722. In the embodiment of FIG. 38, the horizontal line 3821 and the vertical line 3822 contain a perforated bitmap. In this case, the overlay window 3983 can be configured as semi-transparent or it can contain completely transparent pixels.
In step 3304, the overlay window is configured as a so-called click-through window, so that the operating system will send keyboard and mouse events to the underlying window (s), despite the overlay window being above it, and having a higher Z order.
BE2018 / 5637
In step 3305, position information X, Y is obtained from a so-called mouse pointer or mouse cursor, for example in the form of screen coordinates.
Alternatively, motion information related to pointer movements can be obtained from the operating system using raw input messages, in which case the processing of the raw input messages and the actual update of the position of the visual object (and ) is preferably disconnected from each other, for example, as explained in more detail in the co-pending double mouse application.
In step 3306, the position of the movable objects (e.g. the X coordinate of the vertical line, and the Y position of the vertical line) is adjusted in accordance with the obtained mouse cursor position X, Y or in accordance with the motion information.
The native mouse cursor position is preferably at or near the intersection of the horizontal and vertical lines.
Step 3305 and Step 3306 are repeated, for example using a timer interrupt with a period in the range from 1 ms to 200 ms, for example in the range from 2 ms to 100 ms, for example equal to about 5 ms or about 10 ms, or about 16.7 ms (60Hz), or about 20 ms or about 25 ms, or about 30 ms, or about 40 ms, or about 50 ms, or about 100 ms, or about 150 ms.
FIG. 34 shows a simplified high-level block diagram as a possible representation of the software components and hardware components located in a computer system, which usually cooperate in performing a method according to embodiments of the present invention, but it is explicitly pointed out that the present invention is not implementation is limited, and that other implementations can also be used. The main purpose of this block diagram is to show a possible implementation of an embodiment implemented in a Windows compatible computer.
FIG. 34 shows an overlay application 3474 and an operating system or O / S 3460 (e.g. MICROSOFT WINDOWS 10 available from Microsoft Corporation of Redmond, Wash, US), a graphics accelerator 3425, output interfaces 3480, a display 3492, an input interface 3410, an input pointing device 3401 and a bitmap generator 3420. Other components include, within the O / S 3460, input device drivers 3466 and graphics API (Application Programming Interface) 3440.
FIG. 34 is very similar to FIG. 8 of US6333753 (B1), which is incorporated by reference herein in its entirety, but especially FIG. 8 of which and the accompanying description. Since many aspects are already known, they do not need to be explained in full again.
The main similarities of the block diagram 3400 of this document and the block diagram of FIG. 8 of US6333753 (B1) are:
BE2018 / 5637 * the internal functioning of the video back-end components, the graphics API 3440, and the graphics accelerator 3425 and the output interfaces 3480 may be largely similar to those of the prior art, e.g. the way in which fully transparent pixels are processed, and how alpha mixing is performed.
The main differences between the block diagram 3400 of this document and the block diagram of FIG. 8 of US6333753 (B1) are:
* the present invention does not require a mouse with touch sensors, and therefore does not require special device drivers 3466. Classic device drivers such as a standard USB mouse driver (mouse control) may be used instead;
* the overlay application 3474 of the present invention does not use animated fade-in and fade-out. Optionally, however, time multiplexing of the perforated bitmap can be used as described above (FIG. 28), in which case a time multiplexing component 3415 is used, which can transmit new position information of the offset perforated and / or textured bitmap, for example any frame, or every two frames, or every three frames, or every four frames. Preferably, the timing is synchronized with the frame rate;
* in the present invention, the bitmap generator 3420 can generate one or more stationary and / or movable objects, such as, for example, a small horizontal line, or a small cross (e.g., which includes a small horizontal line and a small vertical line), or a large horizontal line (which, for example, extends across the entire width of the overlay window), or a large cross (which includes, for example, a large horizontal line that preferably extends across the entire width, and a large vertical line that preferably extends across extends the entire height);
* although not shown in FIG. 34, the present invention preferably also provides a user interface window, an example of which is shown in FIG. 35. The user interface window is usually implemented as a second, mainly opaque window (it can include fully transparent pixels), and is preferably located (in Zorde) between the overlay window and the mouse cursor plane. For example, this user interface window allows the user to set or select or change dimensions and / or a color of the horizontal line (pixels P3), and / or to set or select the desired transparency level T (or alpha transparency value α), and / or a color set or select monochrome pixels (P7) of the overlay bitmap, and / or set or select an average color of pixels of texture pixels (P8), etc. This user interface window is not configured in click-through mode, as it must be capable of mouse clicks and / or keyboard input, and its area is usually reduced, eg minimized during normal use of the graphic overlay application.
BE2018 / 5637
Although not shown in FIG. 34, in addition to the overlay application 3474, the computer device usually also executes one or more application programs, such as, for example, a word processor, a PDF viewer, a web browser, a spreadsheet, etc.
The input positioning device 3401, for example a mouse or a touch pad, is via cables
3403 connected to input interface 3410, for example, a USB input interface, which processes and / or forwards signals from the input device to input device drivers 3466 that are a component within the O / S 3460. The device drivers 3466 interpret the signals produced by the pointing device 3401 and generate response-appropriate events. In particular, the O / S moves a mouse pointer or mouse cursor in accordance with movements of the device 3401, and passes these events to an application (e.g. a text processor or web browser), since the overlay application 3474 is configured in click-through mode or pass-through mode.
Bitmap generator 3420 can include the following:
* a horizontal line generator 3422 for generating at least one horizontal line or beam with a predefined or customizable shape (e.g. rectangular with sharp edges or with rounded edges) and size (e.g. width and height) and position and appearance ( e.g. as a monochrome bitmap, or with a color gradient, or as a perforated bitmap);
* a vertical line generator 3424 for generating at least one vertical line or beam with a predefined or customizable shape (e.g. rectangular with sharp edges or with rounded edges) and size (e.g. width and height) and position and appearance ( e.g. as a monochrome bitmap, or with a color gradient, or as a perforated bitmap);
* a background bitmap generator 3426 for generating a relatively large bitmap that preferably occupies the entire area of the overlay window, for example, as a monochrome bitmap (similar to FIG. 6), or as a perforated bitmap with 50% fully transparent pixels P6 and 50% monochrome pixels P7 (as in FIG. 10 and FIG. 12), or as a perforated bitmap with 50% fully transparent pixels P6 and 50% texture pixels P8 (as in FIG. 20 and FIG. 22), or as a bitmap with 100% fully transparent pixels P5 (as in FIG. 8, FIG. 37 and FIG. 39), or as a bitmap with 100% texture pixels P8 (as in FIG. 24). One or more of these objects can be generated on the fly (gradually), optionally with the help of a graphical API 3440, and / or with the help of a graphical accelerator 3425.
As explained above, in the case where a semi-transparent window is used as the (first) overlay window, its transparency level T is preferably selected in the range of 30% to 75% (or the alpha blend value is preferably selected in the range from 25% to 70%), so that underlying document objects are clearly visible (although optionally darkened or textured slightly) through the overlay window.
BE2018 / 5637
In case time multiplexing is used (which is entirely optional), the time multiplexing module or time multiplexing process 3415 will issue instructions, preferably one such instruction for every N display frames (where N is preferably equal to 1 or 2 or 3 or 4), to shift the perforated bitmap back and forth over a number of pixels, preferably an odd number of pixels, eg 1 pixel up / down or 1 pixel left / right, such that a certain pixel of the underlying document object alternates overlayed (covered) by a fully transparent pixel or by a semi-transparent pixel. These instructions will be delivered, as schematically represented by line 3430 and / or line 3413, to O / S 3460, which in turn can pass these instructions to Graphics API 3440 and eventually to Graphics Accelerator 3425.
If no graphics accelerator is used, then, as symbolized by line 3455, graphics API 3440 can deliver graphics output directly to output interfaces 3480 (specifically a standard video card, not shown, therein) which, in turn, will generate appropriate video signals and providing those signals via cables 3486 to display 3492. In this case, computer system 3400 will need to be fast enough to implement the appropriate graphics capabilities in software, which would otherwise be provided by accelerator 3425.
While alpha transparency functionality is supported by a wide variety of current graphics accelerators, this functionality can be easily simulated in software, in a well-known way, by conventional 2-D (two-dimensional) or 3-D (three-dimensional) graphics APIs such as D3D (which is a 3-D graphics API, produced by Microsoft Corporation as a component of a WINDOWS operating system, where WINDOWS is a registered trademark of Microsoft Corporation), OpenGL (which is available in the art today) or GDI (historically just a 2-D low level graphics processing layer, currently produced by Microsoft Corporation and also included as a standard component of a WINDOWS operating system).
In a variant of the block diagram of FIG. 34, the overlay application does not request the position of the native cursor position of the operating system, but further comprises a processor (handler) of raw input messages configured to process raw input messages provided by the operating system, and adapted for extracting motion information dx, dy from said event messages, and for moving at least one movable object, and for optionally overriding the native cursor position of the O / S. While not relevant to understanding the present invention, this aspect is described in more detail in the co-pending double mouse application.
The bitmap generator 3420 can generate one or more image data structures, which can be one or more bitmaps, or a collection of commands in a language such as Display Postscript or Quickdraw. Regardless of the details, the image data structure must contain sufficient information
BE2018 / 5637 so that it can be rasterized (if not already a bitmap) at the display resolution or otherwise processed for display.
FIG. 35 shows an exemplary user interface window as can be used by an overlay application according to the present invention. This user interface window can be displayed as a second, opaque window, on top of the first overlay window 883, 1083, 1284, 2083, 2283, 2483, 3783, 3983, or can be displayed as a very small object (e.g. a rectangular object) with a bitmap containing a logo of the overlay application, or with a text field containing the name of the overlay application, and with a size smaller than 150 x 30 pixels, e.g. smaller than 100 x 30 pixels), or in the taskbar. The overlay application may be provided to enlarge the user interface window when the mouse moves over the bitmap containing the logo. In this way, the user interface window is easily accessible, while it does not take up much space on the screen.
FIG. 36 shows a schematic block diagram of another computer system 3600 according to an embodiment of the present invention, with a computer device 3601 having an overlay application for providing an overlay window 3783 with a movable object 3610 having a perforated bitmap. The line 3610 is movable in accordance with movements of the input device 3603. This embodiment offers the advantage that the overlay window 3783 can be opaque with most pixels being completely transparent pixels, yet the effect of a semi-transparent line 3610 can be obtained. Such an embodiment may be particularly useful in computing devices that do not have alpha mixing functionality, such as eReader devices, for example.
FIG. 37 is a schematic representation of a Z order of four windows or image planes
3781 to 3784 that can be used in the computer system 3600 of FIG. 36.
FIG. 38 shows a schematic block diagram of another computer system 3700 according to an embodiment of the present invention, with a computer device 3801 having an overlay application for providing an overlay window 3983 with a movable object in the form of a perforated cross. The cross is composed of a horizontal line 3821 and a vertical line 3822, and is movable in accordance with movements of the input device 3803.
FIG. 39 is a Z-order schematic of four windows or image planes 3981-3984 that can be used in the computer system 3800 of FIG. 38.
BE2018 / 5637
FIG. 40 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application providing an overlay window 4184 with a movable object in the form of a horizontal line 4020, preferably extending the entire width of the screen.
The overlay window 4184 is preferably configured as a semi-transparent window, preferably with an alpha transparency in the range of 5% to 95%, or from 30% to 70%, although not absolutely required, and the window 4184 may also as opaque, in which case the pixels P5 would be completely transparent pixels.
The overlay window 4184 includes a semi-transparent line with pixels P3. An important advantage of this overlay application is that the line can be used to highlight or underline corresponding text fragments in two different applications, for example a translate PDF source text positioned on the left side of screen 4004 and an MS-Word target text. contains the translated text, on the right side of screen 4004. Line 4020 helps to quickly rediscover the context when switching back and forth between the two documents, for example during proofreading, making the proof reader's cognitive load significant reduced.
Since the overlay application is configured in click-through mode, the mouse 4099 can be used to click, drag or scroll anywhere on the screen. In particular, it allows the PDF document to be moved slightly up or down (eg by dragging or scrolling) to keep the text fragments in it aligned with corresponding text fragments of the Word document.
The overlay application causes the Y coordinate of the line 4020 to change in accordance with the Y coordinate of the mouse cursor 4099. The Y coordinates can be identical or slightly offset, such that the line is slightly above or slightly below the mouse cursor appears.
FIG. 41 is a schematic representation of a Z order of five windows or image planes 4181-4185 that can be used in the computer system of FIG. 40.
The embodiment of FIG. 40 and FIG. 41 can be seen as a variant of the embodiment of FIG. 38 and FIG. 39 or as a variant of the embodiment of FIG. 7 and FIG. 8.
In a variant (not shown) of the embodiment of FIG. 40 and FIG. 41, the line 4020 contains a perforated bitmap (as in FIG. 37), or the line contains a color gradient (vertical or horizontal), or the line contains a texture bitmap.
BE2018 / 5637
FIG. 42 shows a schematic block diagram of another computer system 4200 according to an embodiment of the present invention, with an overlay application providing an overlay window 4384 that includes a vertical line 4240 defining a left area and a right area of screen 4204, and a horizontal line segment 4260 extending across the width of either the left region or the right region.
The overlay window 4384 is preferably configured as a semi-transparent window, preferably with an alpha transparency in the range of 5% to 95%, or from 30% to 70%, although not absolutely required, and the overlay window 4384 may also as opaque, in which case the pixels P5 would be completely transparent pixels.
The horizontal position of the vertical line 4240 may be predefined, e.g. substantially in the center of the overlay window 4384, or may be selectable via a user interface, but is preferably configurable by dragging the line with the mouse cursor 4299. The the latter is not evident if the overlay window 4384 is configured in click-through mode, but is possible, for example, by dynamically configuring the overlay window 4384 in click-through mode or not in click-through mode, depending on whether the mouse cursor is above vertical line 4240 or not.
The overlay application is configured to update the vertical position Y of the horizontal line segment 4260 according to the position of the mouse cursor 4299 or pointer or position device movements, e.g. mouse device 4203, but only if the mouse cursor is in the left area (ie on the left side of vertical line 4240). When the mouse is in the right region (i.e., on the right side of the vertical line 4240), the horizontal line segment 4260 maintains its position (which is also referred to herein as freezing).
An important advantage of this overlay application is that the line segment 4260 can be used to highlight or underline text fragments in a document to be translated (e.g. a PDF document located on the left side of vertical line 4240) while working ( with the keyboard 4202 and the pointing device, e.g. mouse device 4203) in a text editor located on the right side of vertical line 4240. The line segment 4260 helps to quickly regain context when switching back and forth between the two documents, for example during translation, significantly reducing the translator's cognitive load.
This overlay application can also be used by proofreaders, using the line segment 4260 to underline or highlight text fragments (by moving the mouse cursor left by line or single line of source text), and then moving the mouse cursor to the right area , and using the mouse cursor 4299 to hover over the target text, and optionally to edit or correct the latter.
BE2018 / 5637
FIG. 43 is a schematic representation of a Z order of five windows or image planes 4381-4385 that can be used in the computer system of FIG. 42.
In a variant (not shown) of the embodiment of FIG. 42 and FIG. 43, the line segment 4260 contains a perforated bitmap (as in FIG. 37), or contains a color gradient (vertical or horizontal), or contains a texture bitmap.
In a variant of the embodiment of FIG. 42 and FIG. 43, the line segment 4260 is on the right side of the vertical line 4240, and is movable if the mouse cursor or pointer 4299 is to the right of the vertical line 4240, and freezes if the mouse cursor or pointer is on the left side of the vertical line 4240 . This embodiment is considered particularly suitable for left-handed persons.
FIG. 44 shows a flow chart of a method used by the computer system 4200
FIG. 42 and FIG. 43 can be executed. FIG. 44 shows a computer-implemented method 4400 for overlaying a graphic image in a computer, the method comprising the steps of:
a) providing 4401 with an overlay window 4384, preferably as a semi-transparent window;
b) providing 4402 a first visible object in the form of a vertical line 4240;
c) providing 4403 a second visible object in the form of a horizontal line segment 4260 located either on the left or right side of vertical line 4240;
repeatedly performing the following steps:
d) obtaining 4404 position information X, Y from a mouse pointer or mouse cursor 4299;
e) testing 4405 whether the mouse cursor or pointer 4299 is on or above the vertical line 4240, and if the result of the test is positive, proceed to step f), otherwise proceed to step h);
f) if the overlay window is not already configured in non-click-through mode, configuring overlay window 4384 in non-click-through mode 4406 (which means: not in click-through mode but in a mode where the operating system keyboard and mouse events to the overlay window or objects will send them, rather than to underlying applications, if the mouse is positioned over the overlay window);
BE2018 / 5637
g) testing whether a mouse button has been pressed, and if the result of this test is true, dragging the vertical line; and go back to step e);
h) if the overlay window is not already configured in click-through mode, configuring 4408 of the overlay window 4384 in click-through mode;
i) adjusting a position of the horizontal line segment 4260 if the mouse cursor or pointer 4299 is on the same side of the vertical line 4240 as the horizontal line segment 4260, and freezing the horizontal line segment 4260 if the mouse cursor 4299 is on the opposite side of vertical line 4240, and return to step (e).
This embodiment provides a line segment 4260 that freezes when the mouse cursor on the right side of vertical line 4240 is moving or is located there, and that moves along with the mouse cursor when mouse cursor 4299 is on the left of vertical line 4240 or is moving there, or vice versa. In addition, this embodiment makes it possible to drag the vertical line 4240 in a very intuitive manner. This embodiment is particularly suitable for translators and proofreaders who do not use a Computer Aided Translation tool, or to verify a translation outside of the CAT tool.
It is noted that there was a requirement to make the position of the vertical line selectable or configurable, and since this configuration requires user interaction, the logical place to implement this functionality is in the user interface window, because that window is capable of keyboard and receive mouse events. However, the inventors came up with the idea that it would be simpler and more intuitive for the user if the vertical line could be dragged, but that was not possible because the vertical line was implemented in a window configured in click-through mode. In an effort to find a solution to enable dragging anyway, the inventors experimented with dynamically changing the click-through functionality of the overlay window, not knowing if, let alone expecting it to work at all, and whether it would work reliably to work. Surprisingly, the experiments showed that it is indeed possible to dynamically change the click-through mode of a window depending on the position of the mouse cursor, particularly depending on whether the native mouse cursor is above the area of the vertical line. At least this aspect is not trivial.
In a variant of the method for FIG. 44, the line segment 4260 is on the right side of the vertical line 4240, and is movable if the mouse cursor or pointer 4299 is to the right of the vertical line 4240, and freezes if the mouse cursor or pointer is on the left side of the vertical line 4240 . This embodiment is believed to be especially useful for left-handed translators and proofreaders and writers.
BE2018 / 5637
FIG. 45 shows a mobile device such as, for example, an eReader or a smartphone device, which comprises a touch screen, and which runs an app or application that shows textual information, for example in the form of an eBook, or a web browser. The textual information is overlayed (covered) by a line 4510 that allows a user to underline or highlight text information he or she is reading. Line 4510 can be implemented in an overlay window associated with an overlay application or as part of the operating system running on the device, or it can be implemented as part of the app or application itself, e.g. the eBook reader or web browser app.
In one embodiment, line 4510 is a semi-transparent line created in an overlay window with an alpha transparency in the range of 1% to 99% or from 2% to 98% or from 5% to 95%, preferably in the range of 30% to 70%. The line 4510 may contain a plurality of monochrome pixels (as in FIG. 8), or may contain a color gradient (not shown), or may contain a texture.
In another embodiment, line 4510 is formed by, or includes, a bitmap that includes a first plurality of fully transparent pixels P6 and a second plurality of opaque or semi-transparent pixels P7 that are interleaved (alternately positioned) to make the line appear semi-transparent, herein referred to as a perforated bitmap. In case the pixels P7 are opaque pixels, no alpha blending is required, and yet the line does not obscure the underlying textual information.
Line 4510 can be automatically movable, eg based on a timer. Preferably, the speed is configurable or adjustable via a user interface (not shown).
Line 4510 may be movable by dragging, in ways known per se in the art, based on a user making contact with the screen with his or her finger, and then moving the finger.
The line can help the reader to stay focused while reading, despite distractions.
In an embodiment where the line 4510 extends substantially across the entire width of the display of the device 4500, the reader application or the overlay application need only indicate the vertical position (Y) of the line relative to the screen according to the user's movements on the touch screen.
In an embodiment where line 4510 extends only a fraction (e.g. less than 50%) of the width of the display of device 4500, the reader application or the overlay application may be adapted to have both a horizontal position X and a vertical position. adjust position Y of the line according to movements of a user's finger on the touch screen.
BE2018 / 5637
FIG. 46 shows a schematic block diagram of another computer system 4600 according to an embodiment of the present invention, which can be viewed as a variant of FIG. 21, wherein the computer system 4600 provides a graphic overlay image 4784 in the form of a so-called perforated texture bitmap overlaid by the display device 4604.
In this embodiment, the computer device 4601 generates the image planes 4781 to 4783, and supplies corresponding image data to the display device 4604, as a conventional computer would, but instead display the said computer data on a display panel. overlays said image data with an overlay image 4784 stored in the display device and displays the result.
Display devices are well known in the art, especially standalone LCD display devices or standalone LCD monitors. Such a display device mainly comprises three parts: an application processor, a memory device that includes a frame buffer, and a display device that includes a display controller and a display panel. The application processor is responsible for drawing and generating an image frame, and for storing the generated image frame in the frame buffer in the memory device. The frame buffer is used to store data from pixels on a display screen. The display controller is primarily responsible for reading the pixel data from the frame buffer, and sequentially writing the pixel data to logic panels on the display panel that correspond to the pixels. In a process of displaying an image by the display system, the application processor continuously generates a new image frame and sends the image frame to the frame buffer.
In the context of the present invention, the application processor is adapted to provide an overlay image 4784 over said computer image, the overlay image being a perforated texture bitmap as explained above. In other words, this bitmap 4730 comprises or consists of a first plurality of pixels P6 which are fully transparent pixels, and a second plurality of pixels P8 which are semi-transparent pixels, which pixels P6 and P8 are interleaved (alternately positioned), for example arranged in a regular pattern, preferably a checkerboard pattern. The actual overlay step may include alpha blending of the computer pixels with corresponding pixels of the perforated texture bitmap.
The perforated texture bitmap can be a predetermined bitmap built into the display device, or is preferably a downloadable image, or displayed on the computer device, and captured by the display device at the request of the user, e.g. initiated by pressing a button of the display device (not shown), and then stored in a non-volatile memory of the display device for further use. However, other means for downloading the texture bitmap have also been considered, for example using a USB interface for inserting a memory stick, or using a
BE2018 / 5637 wireless connection, for example a Bluetooth connection for establishing a wireless communication with the computer device, in which case the computer device is preferably equipped with a special application for selecting and sending a texture bitmap to the display design.
Optionally, the display is further adapted for scaling and / or resizing and / or tiling and / or perforating the texture bitmap, although the perforation can also be implied by alpha mixing only a subset of the pixels, for example, one pixel for every two pixels according to the checkerboard pattern.
Preferably there are at least four or five alpha mixing levels, for example corresponding to a mixture of 0/4 = 0%, 1/4 = 25%, 2/4 = 50%, 3/4 = 75% and 4/4 = 100 %. More preferably there are at least eight or nine alpha mixing levels, e.g. corresponding to a mix of 0/8 = 0%, 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7 / 8 and 8/8 = 100%.
Optionally, the display is further adapted to adjust the transparency level and / or the color and / or the contrast of the texture bitmap, for example using some or all formulas [6] to [10], based on a predefined color and / or contrast level, which may be provided to the display device by a user, using one or more buttons on the display device, or using said special application for configuring the display device using the mentioned wireless connection, eg Bluetooth connection.
Optionally, the processor is further adapted to repeatedly adjust a position of the texture bitmap such that pixels of the underlying graphic image are overlayed by a fully transparent pixel of the first plurality of pixels P6 at a first time t1, and overlayed by a semi transparent pixel P8 of the second plurality of pixels at a second and optionally third and fourth time t2, t3, t4, for example as explained above, when discussing FIG. 28.
Rather than physically shifting the texture bitmap back and forth by a horizontal or vertical distance of, for example, one pixel for each frame, and rather than storing a perforated bitmap, the processor may be adapted to combine the step of simultaneously punching and time multiplexes, when generating the image to be stored in the frame buffer, e.g. by doing the following:
i) at even times (t2, t4) copying computer image data from the input buffer at pixel locations X, Y for which the sum of the row index X and the column index Y is odd, and by alpha mixing computer image data from the input buffer with corresponding pixel data from the texture bitmap at pixel locations X, Y for which the sum of the row index X and the column index Y is even; and
BE2018 / 5637 ii) at odd times (t3, t5) copying computer image data from the input buffer at pixel locations X, Y for which the sum of the row index X and the column index Y is even, and the alpha mixing of computer image data from the input buffer with corresponding pixel data from the texture bitmap at pixel locations X, Y for which the sum of the row index X and the column index Y is odd.
Although this solution requires a special display device, it provides several advantages: This solution can reduce eye fatigue for users, especially when the computer image contains a clear image. It is a further advantage of this solution that it can transform the appearance from a pure white image into a textured image, for example a paper-like image, which may be aesthetically pleasing to read. It is an advantage that the display device performs the alpha mixing instead of the CPU of the computer device, so the CPU load is not affected by this overlay. Finally, it is an advantage to use a perforated bitmap instead of a non-perforated bitmap as it requires less computing power since only a subset of the pixels, for example only 50% of the pixels of each frame, need to be alpha mixed, while the other pixels without alpha mixing can be passed.
FIG. 47 is a schematic representation of a Z order of three windows or image planes that can be used in the computer system of FIG. 46, and above that, a fourth semi-transparent overlay image is provided by the display device.
FIG. 48 shows a schematic block diagram of another computer system according to an embodiment of the present invention, which can be viewed as a variant of FIG. 46, wherein the computer system provides a graphic overlay image in the form of an unperforated texture bitmap, which is overlayed by the display device. Everything mentioned with respect to FIG. 46 and FIG. 47 also applies here, except that all pixels of the texture bitmap should be alpha mixed with the pixels of the computer image.
FIG. 49 is a Z-order schematic of three windows or image planes that can be used in the computer system of FIG. 48, and above, a fourth, semi-transparent overlay image provided by the display device 4804.
Referring to FIG. 50 and FIG. 51.
As already discussed in the background section, the F.Lux tool provides a semi-transparent graphical overlay application using a bitmap, in particular a yellowish monochrome bitmap, the color of which is dynamically adjusted over time.
BE2018 / 5637
While this tool can help reduce eye fatigue, the tool has several limitations, such as:
(1) the overlay does not take into account the actual underlying images provided by the underlying applications. Indeed, the inventors found that, in order to reduce eye fatigue, some images should be darkened than other, in particular, applications that provide a window with a pure white background (such as MS Paint or MS Word or MS Powerpoint) should be darkened then become applications that provide a gray background. Unfortunately, F.Lux does not take this into account. A user could edit manually, but this is cumbersome.
(2) many computer users occasionally take a screenshot (screen shot) (e.g. using the Print-Screen button on the keyboard), but when F.Lux is executed, the colors of such a screenshot are not the true colors of the original image stack associated with the underlying applications, but are colored yellowish or orange. A user who wants to take a screenshot could temporarily deactivate F.Lux to take a screenshot, and then reactivate F.Lux, but this is time consuming, but worse, when F.Lux is deactivated, he or she gets the original images that are typically much brighter and can cause eye fatigue.
The inventors of the present invention came up with the idea of addressing these problems by the computer-implemented method 5000, respectively, shown in FIG. 50, and the computer-implemented method 5100 of FIG. 51.
FIG. 50 is a flow chart of a computer-implemented method 5000 for overlaying a graphic image in a calculator, which can be viewed as a variant of the calculator 2300 of FIG. 23 and FIG. 24. The method includes the steps of:
a) providing 5001 of a semi-transparent overlay window with an alpha transparency value α, and comprising a first bitmap bmp1. The first bitmap can be a monochrome bitmap where all pixels are the same color, or it can be a perforated monochrome bitmap, or it can be a texture bitmap, or it can be a perforated texture bitmap, or another suitable bitmap.
The method further includes a loop repeating the following steps:
b) taking a screenshot 5002, thereby obtaining a second bitmap bmp2;
d) determining 5003 of at least one characteristic of the second bitmap bmp2;
e) adjusting 5004 the alpha transparency value α of the overlay window and / or adjusting one or more pixel values of the first bitmap bmp1 based on the at least one particular feature.
BE2018 / 5637
The first bitmap can be a monochrome bitmap.
The first bitmap can contain a color gradient.
In an embodiment, step d) comprises: determining an average intensity and / or a maximum intensity and / or an intensity histogram of the pixels of the second bitmap bmp2; and includes step e): adjusting the alpha transparency level α such that the average intensity or maximum intensity or intensity histogram meets a predetermined criterion.
In one embodiment, step d) comprises: determining average color component values Ravg, Gavg, Bavg and / or a color component histograms of the pixels of the second bitmap bmp2; and includes step e): adjusting one or more pixel values of the first image bmpl such that the average color component or the color component histogram meets a predetermined criterion.
Better results (in terms of less eye fatigue) can be achieved if the brightness of the room is also taken into account. In such an embodiment, step d) further comprises: determining a brightness value indicative of a brightness in a room in which the calculator performing the overlay method is located, and the adjustment of step e) further takes into account the brightness value. This brightness value can be obtained, for example, from a light sensor, or can be derived from an image taken by a webcam, or in another suitable manner.
In a specific embodiment, especially suitable for people who work with two documents or who work on two different displays, step a) further comprises: providing at least one vertical line or bar in the first bitmap bmpl for defining at least two areas comprising a first area and a second area; and step c) includes determining at least a first feature of the first area and a second feature of the second area; and comprising step e): adjusting one or more pixel values of the first area such that the first characteristic meets a first predetermined criterion, and adjusting one or more pixel values of the second area such that the second characteristic meets a second criterion.
In a variant of this method, each of the left and right areas can be further divided by a horizontally movable line or beam, leading to four different areas. Preferably, each of these areas of the first bitmap is monochrome, that is, the first area has pixels of color value (R1, Gl, Bl), the value of R1, Gl and B1 being determined based on the graphic content of the images included under the first area
BE2018 / 5637, the second area has pixels with color values (R2, G2, B2) where the values of R2, G2 and B2 are determined based on the graphic content of the images below the second area, etc. These areas can be perforated or not perforated. Alternatively, the first area contains pixels derived from a first texture bitmap, having a first average color and a first contrast ratio derived depending on at least one feature of the image (s) underlying the first area, the second area contains pixels extracted from a second texture bitmap, with a second average color and a second contrast ratio, etc. Those skilled in the art with the benefit of the present disclosure can easily devise other variants.
It is also contemplated that the screenshot may be further analyzed, for example, to detect the presence of alphanumeric characters, and their size, and / or the presence of a monochrome background surrounding these characters, and adjusting attributes of a texture overlay bitmap based thereon , for example, to create a texture bitmap with a sufficiently low contrast so that it does not negatively affect the legibility of the characters, and with a sufficiently high contrast so that it is noticeable by the user, and for example a paper-looking effect, or a matte-looking background creates.
FIG. 51 is a flow chart of a computer-implemented method 5100 for overlaying a graphic image in a calculator, which can be viewed as a variant of the calculator 2300 of FIG. 23 and FIG. 24. The method includes the steps of:
a) providing 5101 a semi-transparent overlay window with an alpha transparency value α in the range of 1% to 99%, and comprising a first bitmap bmp1. The first bitmap can be a perforated bitmap or a non-perforated monochrome bitmap, or a perforated or non-perforated texture bitmap, or a bitmap with a gradient based on only two color values.
b) taking a screenshot 5002, thereby obtaining a second bitmap bmp2;
c) calculating a third bitmap bmp3 to compensate for the effect of the graphic overlay based on the first bitmap bmp1 and the second bitmap bmp2 and the alpha transparency value α.
It is noted that, in case the overlay application not only contains a bitmap but also contains one or more movable objects such as for example a horizontal line or a vertical line, it is possible to maintain either these visible objects in the recorded image , or to compensate for these objects as well, so that these objects are virtually removed from the recorded image. It is possible to remove the movable objects or not in advance
BE2018 / 5637 are certain settings, or can be configurable by a user, for example in the user interface window.
Optionally, the method further comprises step d) of: storing 5104 the third bitmap bmp3 to a non-volatile memory (e.g., on a memory stick) or on a storage device (e.g., a hard disk or a network drive), optionally after format conversion (eg BMP to PNG) and / or after image compression (eg lossless or lossy JPEG compression).
Optionally, the method further includes step e) of: copying 5105 the third bitmap bmp3 to a clipboard of an operating system running on the computing device (for example: MS Windows 10 or a later version with the same or similar functionality). This feature is similar to the print screen button found on many keyboards, but with the added benefit of compensating for alpha blending. In this way, a user who suffers from eye fatigue can work on his computer, and can even take screenshots without having to deactivate the overlay image. This is a very big advantage for such users, both because of reduced eye fatigue and time saved.
The following set of formulas, or an equivalent set of formulas, can be used to perform the compensation:
"~ R3 [x, y] = [R2 [x, y] - R1 [x, y] * (1-T)] / T [11]
G3 [x, y] = [G2 [x, y] - G1 [x, y] * (1-T)] / T [12]
B3 [x, y] = [B2 [x, y] - B1 [x, y] * (1-T)] / T [13]
T = 1-q [14] where R1, G1, B1 are color components of a pixel at a location (x, y) of the first bitmap bmp1 that is overlayed, and where R2, G2, B2 are color components of a pixel on a location (x, y) of the second bitmap bmp2 taken as the screenshot, and wherein R3, G3, B3 are color components of a pixel at a location (x, y) of the third bitmap bmp3, and where α is the alpha transparency value , and T is a transparency value, both expressed as a floating point number in the range of 0.01 to 0.99.
In case the first bitmap is perforated, formulas [11] to [14] need only be applied in case the pixel of the first bitmap is not completely transparent.
FIG. 52 to FIG. 59 illustrate another aspect of the present invention, referred to herein as the inverted color function or the color inversion function. As described in the background section, one of the underlying problems of the present invention is that clear images (e.g. caused by a word processor or PDF document viewer or
BE2018 / 5637 a website with a white background) can cause eye fatigue, especially in people who work many hours in front of a computer screen.
While this problem has already been addressed above (e.g., by providing a perforated or non-perforated rather dark semi-transparent overlay), the inventors also came up with the idea of providing an overlay that essentially inverts the colors, in the sense that bright pixels are transformed into dark pixels, and vice versa. Indeed, a command prompt window (also known as a DOS window) usually causes much less eye fatigue than most modern windows applications, because most of the screen is black except for a few white characters.
It is noted that the Windows operating system already has an Increased Contrast mode, but there are several drawbacks with the existing implementation provided by the Operating System. For example, this feature is not easy to find for an average user, as multiple menus have to be run through, making it impractical to switch ON and OFF on a regular basis (e.g. every few minutes). Second, Windows O / S's Enhanced Contrast mode only works with a full screen, not a partial screen. That's a shame because professional users could work with multiple applications at the same time (especially on a computer with a multi-tasking operating system and with a large screen), but they may not want to invert the colors of all applications. The user may want to invert the colors of a right part of the screen where a word processor is located, but not the colors of a left part of the screen where, for example, a drawing program is located. Third, when working in a text document, the user may want to change the font color of certain characters, and / or highlight some parts of text in yellow or green background color, but this is very difficult if the colors are inverted all over the screen.
The inventors were keen to provide a color inversion function in an overlay application, but realized that the overlay application cannot access the data of the underlying image. The overlay application can take a screenshot, but that would not yield the underlying image data, but a mix of the underlying image data and the overlay window. It would be possible to temporarily disable the overlay function (e.g. by making the overlay window completely transparent), and take a screenshot of the underlying image, then provide an opaque overlay image with the inverse image and turn this image on. for a predetermined period of time, and this would work for a stationary image, but this does not work well if the underlying image changes, as is typically the case when a user is working on the computer device (e.g. when editing of a text document, reading or replying to emails, etc.). In addition, temporarily turning off the overlay would usually cause a bright flash, which is particularly annoying. For these and other reasons,
BE2018 / 5637 was thought to be impossible to implement a color inversion function in the overlay application without causing a flash.
In order to avoid the bright flash, the inventors came up with the idea of not temporarily disabling the semi-transparent overlay, and attempting to provide a color inversion based on the pixel data obtained from the screenshot. Tests were performed taking a screenshot, and depending on the pixel values of the screenshot, the overlay image was adjusted. More specifically, if the pixel of the screenshot had a value (R, G, B), the corresponding pixel in the overlay image was assigned a value of (255-R, 255-G, 255-B). Unfortunately, this technique did not provide a stable image, but the pixel values drift away.
It didn't work until the inventors came up with the idea to calculate the pixel values for the overlay image, not only based on the pixel values of the screenshot image, but also taking into account the pixel values of the (previous) overlay image . More specifically, the inventors came up with the idea to use the above formula [1] to reverse the alpha blending operation, in order to calculate or estimate the pixel values of the underlying image (referred to herein as "unmixed" pixel values). The idea was that once these unmixed pixel values are known, they can be inverted leading to the new target pixel values, and based on these target pixel values, appropriate mask pixel values (values of the overlay bitmap) could be calculated or estimated that would result in the target pixel values. It could not be predicted that this idea would work, especially when one takes into account that to override the underlying values (because a bright pixel should turn black, and a black pixel should turn bright after alpha blending), the transparency level T is fairly should be small (e.g. less than 20%), but it was unclear whether that would still allow detecting the underlying pixel data in the screenshot with sufficient accuracy, especially for image data where quantization errors are very visible.
So the inventors started to experiment. The above formula [1] is repeated here as formula [15], for convenience.
(Rm, Gm, Bm) = (R1, G1, B1) x (1-T) + (R2, G2, B2) xT [15]
For the inverse color function, (Rm, Gm, Bm) are the values obtained by the screenshot, and they are renamed here to (Rmix, Gmix, Bmix), and the values (R2, G2, B2) are the values of the underlying image and they are renamed here to (Rorig, Gorig, Borig), and the values (R1, G1, B1) are those of the overlay image and they are renamed here to (Rmask, Gmask, Bmask).
So formula [15] can be rewritten as the following system of equations:
Rmix = Rmask * (1-T) + Rorig * T
Gmix = Gmask * (1-T) + Gorig * T
Bmix = Bmask * (1-T) + Borig * T [16a] [16b] [16c]
BE2018 / 5637 where each of Rmix, Gmix, Bmix, Rmask, Gmask, Bmask, Rorig, Gorig and Borig values are in the range of [0..255], and T is a value in the range of 0.0 to 1.0, or more precisely a value in the discrete set from 0/256 to 255/256. The inventors came up with the idea to estimate the original values (or calculate the unmixed values) using the following set of formulas:
Runmix «Rorig = [Rmix - Rmask * (1-T)] / T, limited to [0..255]
Gunmix «Gorig = [Gmix - Gmask * (1-T)] / T, limited to [0..255]
Bunmix «Borig = [Bmix - Bmask * (1-T)] / T, limited to [0..255] [17a] [17b] [17c] (where« means: is approximately equal to).
Integer calculations were performed, and instead of multiplication and division, shearing operations were used to speed up the calculations. To this end, the value of T was chosen as a power of 2 divided by 256. More specifically, in preferred embodiments, T = 16/256 or T = 32/256.
The next step is to determine the target values of the inverted pixels. A simple formula for converting white pixels (value 255) to black pixels (value 0) and vice versa is given by the following set of functions:
Rtarget = (255-Runmix);
Gtarget = (255-Gunmix);
_Btarget = (255-Bunmix);
[22a] [22b] [22c] but it is expressly pointed out that the present invention is not limited to these specific functions. In fact, any set of three monotonically decreasing functions that provide values in the range of 0 to 255 (or a subset thereof), for arguments in the range of 0 to 255, is considered to provide a color inverted image. As an example, the following set of functions can be a suitable set:
Rtarget = 235 - round (0.95 * Runmix), limited to [0..255]; [23a] - Gtarget = 220 -Gunmix, limited to [0..255]; [23b]
Btarget = 255 - round (1.50 * Bunmix), limited to [0..255]; [23c]
In this particular example, all functions are linear functions with a negative slope, but the present invention is not limited thereto, and non-linear functions can also be used, provided they are monotonically decreasing for arguments in the range of [0 to 255] .
Once the target values have been determined as a function of the unmixed values, the mask values for obtaining these target values (after mixing with the semi-transparent window containing these mask pixels) can be calculated using the following formulas (in
BE2018 / 5637 analogy to formula [1], which has been rewritten as follows (after replacing Rorig with Runmix, Gorig with Gunmix and Borig with Bunmix):
Rtarget = Rmask * (1-T) + Runmix * T, limited to [0..255] [24a]
Gtarget = Gmask * (1-T) + Gunmix * T, limited to [0..255] [24b]
Btarget = Bmask * (1-T) + Bunmix * T, limited to [0..255] [24c] or resolved for the mask values:
Rmask: = (Rtarget - Runmix * T) / (1-T), limited to [0..255] [25a]
Gmask: = (Gtarget - Gunmix *) / (1-T), limited to [0..255] [25b]
Bmask: = (Btarget - Bunmix *) / (1-T), limited to [0..255] [25c]
These formulas provide excellent results, but are computationally expensive.
Looking for faster solutions, further experiments were conducted, and it was surprisingly found that excellent results are also obtained if the mask values are simply equated to the target values of Formula [22a-c], as can be expressed by the following set of equations:
Rmask: = (255-Runmix), limited to [0..255] [26a]
Gmask: = (255-Gunmix), limited to [0..255] [26b]
Bmask: = (255-Bunmix), limited to [0..255] [26c]
This was very surprising because previous tests where the mask values were equated to (255-R, 255-G, 255-B) where (R, G, B) were the values of the screenshot pixels did not give a stable image. However, setting the mask values to (255-Runmix, 255-Gunmix, 255Bunmix) where Runmix, Gunmix and Bunmix are obtained from formula [17] gives very good results.
Excellent results were also obtained for the following set of equations:
Rmask: = (256-Runmix), limited to [0..255]
Gmask: = (256-Gunmix), limited to [0..255]
Bmask: = (256-Bunmix), limited to [0..255] [27a] [27b] [27c]
However, further experiments showed that for some image content (e.g., a soft gradient image), some artifacts occurred in the form of oscillations. More specifically, a small collection of pixels alternated between two values, and also the corresponding mask values of these pixels alternated between two values. Further research showed that this is likely due to rounding errors and / or quantization effects. This problem can be solved in several ways, for example by adding pseudo-random noise, or by detecting that a particular pixel is oscillating (by comparing subsequent
BE2018 / 5637 screenshots), etc., or by quantizing the mask values, or by assigning certain mask values for certain demixing values, or combinations thereof. Those skilled in the art can easily find a solution for these specific values by performing routine experiments and solving them in a simple manner.
Referring now to the figures.
FIG. 52 (a) shows a schematic block diagram of a computer system 5200 according to an embodiment of the present invention. The computer system of FIG. 52 (a) includes a calculator 5201 and a screen or monitor, e.g., an LCD display device. The calculator 5201 may further be connected to a keyboard 5202 and a mouse 5203. The calculator 5201 includes at least one processor running a Multi-Tasking Operating System (eg, a Windows version from Microsoft Corporation, or a version of Macintosh O / S, eg Sierra or High-Sierra, or Linux or other suitable operating system). The calculator 5201 shown in FIG. 52 (a) and FIG. 52 (b) executes at least one application displaying information, eg a word processor or a spreadsheet application or an internet browser or a PDF reader, etc. The calculator 5201 further includes an overlay application for implementing an overlay method which is is able to substantially invert the colors of at least a portion of the image that is overlayed.
In the example shown in FIG. 52 (a), the overlay application is arranged for color inverting a central region of the screen 5204. The size and position of this region may be predetermined, or may be configurable by the user. The size and position of the area may be defined by a frame (frame), or by an upper horizontal line segment 5244, and a lower horizontal line segment 5246, and a left vertical line segment 5240 and a right vertical line segment 5242. This line segments are preferably movable in a similar manner as explained in FIG. 44 (by temporarily configuring the overlay window in non-click-through mode when the mouse moves over the respective line segments) so that the line segments can be dragged. The overlay application is arranged to perform a method as shown in FIG. 54.
An advantage of this overlay application is that it makes it possible to darken part of the screen to reduce eye fatigue, and it does so in a very specific way, by inverting colors. This is especially useful for converting the look and feel of applications that display textual information as fairly dark characters on a fairly bright background, such as (unfortunately most) word processing applications, email applications, web pages, PDF document viewers, etc.
FIG. 52 (b) shows the same computer system 5200, but runs a different overlay application (operating mainly in full screen mode), or runs the same overlay application, but
BE2018 / 5637 configured differently. More specifically, the overlay application running on the computer device of FIG. 52 (b) configured to generate a full screen overlay image (or full screen except for a small portion at the top or bottom, e.g., for the task bar) which inverts the contents of the entire screen. The algorithm or operations performed on the pixel data can be exactly the same as in FIG. 52 (a), but as can be understood, for example, if the image is 4.0 times larger, it usually takes about 4 times longer to calculate the image. This means that the frequency at which the mask can be updated is reduced by a factor of about 4.0.
To get an idea of the performance, experiments were performed on a (rather slow) laptop computer with a 64 bit Operating System (Windows 10), with a 2.00 GHz AMD processor and 8 GB RAM, with AMD Radeon R34 Graphics. The overlay application was configured to run inverse color mode.
* it took about 160-200 ms to perform one iteration (including taking a screenshot, and calculating and applying a new mask) to invert an area of about 1920x1020 pixels. (full screen minus the taskbar).
* it took about 90-130 ms to convert about 1/2 screen (960x1020 pixels), and * it took about 50-90 ms to convert about 1/4 screen (960x510 pixels).
So on this (rather slow) laptop computer, the full-screen image can be color-inverted (or color-converted) about 4 to 5 times per second by the overlay application, the half-screen image can be inverted about 7 times per second , and the quarter screen image can be updated about 10 times per second. While the delay of about 200 ms can be annoying for some users, other users will appreciate the reduced eye fatigue, especially when working with text documents.
Experiments were also performed on another (fairly fast) laptop computer, with a 64-bit Windows 10 Operating System, and an Intel I5 processor at 2.3 GHz, and 8 GB RAM, but with more efficient graphics support. On this computer, it only took about 55-65 ms to color-invert a full-screen image (1920x1040 pixels) in the overlay application, and it only took about 25 to 35 ms to invert a half-screen image. On this computer, the delay is no problem at all, and such performance can be considered almost real time.
FIG. 53 is a schematic representation of a so-called Z-sequence of five windows or image planes 5381, 5382, 5383, 5384, 5385 as can be used in the computer system of FIG. 52 (a) and FIG. 52 (b).
BE2018 / 5637
In the example, the first window 5381 at height Z1 is called the desktop window, and is provided by the operating system.
The second window 5382 (in this example) is provided by a text application that typically has a white background and black characters. One of the black pixels is shown in an enlarged view.
The third window 5383 at height Z3 is provided by the overlay application of the present invention. The window shown here includes two horizontal line segments 5344, 5346 and two vertical line segments 5340, 5342, and a bitmap bmp1 located in an area defined by these line segments, but the present invention is not limited thereto, for example, a single horizontal line (e.g. to implement mode 5506 of FIG. 55), or a single vertical line (e.g. to implement mode 5502 or mode 5504 of FIG. 55), or a single horizontal line segment and a single vertical line segment (e.g. to implement mode 505 or mode 5507 of FIG. 55) can also be used.
The window 5383 is configured as a semi-transparent window, preferably with a transparency value in the range of 5% to 45%, preferably with transparency level T equal to T = 16/256 = 1/16 = 6.25% or T = 32/256 = 1/8 = 12.5%. Or in other words, with an alpha transparency α equal to (1-T) in the range of 55% to 95%, preferably with an alpha transparency value α equal to α = 15/16 = 93.75% or α = 7/8 = 87.5%.
As described above, the overlay application dynamically adjusts the pixels of the bitmap bmp1 (also called the mask) so that the pixels located under this bitmap are essentially inverted, and it does so on the fly, which that is, if the underlying image changes, the mask changes automatically. Where such functionality can be fairly easily implemented in an operating system that has access to the original image data located under the overlay application, this is anything but trivial for an overlay application, in part because the overlay application cannot access the underlying statue.
Apart from the math described above, the operation of the overlay application configured to perform color inversion can be understood by way of example. For example, consider the black pixel 5301 of the second window 5382. The idea is to overwrite the contents of the black pixel by overlaying this pixel location with a bright pixel located in the bitmap bmp1 (also called the mask). Those skilled in the art will understand that although the dark pixel is still partially visible (e.g. about 6% or about 13%), the brightness of the bright pixel at the top largely overwrites the black value, because the pixels of the overlay window 5383 have a larger weighting factor (e.g. about 94% or about 87% respectively) than that of the word processing window 5382 if T is set to 1/16 or 1/8 respectively. It is noted that the principle of inverting colors by the underlying
BE2018 / 5637 to overrule color by using the opposite color using a stronger weight factor is fairly easy to understand, but it was much more difficult to understand and even believe, let alone predict, that it was possible to capture the underlying image with sufficient accuracy. detect to enable color inversion, by taking a screenshot and reverse alpha mixing, knowing that the relevant information is only about 6% or about 13% present in the image, and that it is barely visible (even if it is e.g. slightly visible when typing text in a text document, or when scrolling a text document). However, tests have shown that this is exactly the case, and the results are much better than expected not only for text or menus from typical desktop applications, but even for images, as will be shown in FIG. 56 to FIG. 59 by way of example.
The fourth window 5384 is a user interface window optionally provided by the overlay application. If present, it is preferably located above the third window 5383 and is preferably configured as a non-click-through window.
The fifth window 5383 is a cursor plane, which is preferably provided by the operating system.
FIG. 54 is a flowchart of a computer-implemented method performed by an overlay application according to an embodiment of the present invention, as can be used in the computer system of FIG. 52 (a) or FIG. 52 (b), and capable of inverting the underlying image generated by the operating system of color. The method includes the following steps:
a) providing 5401 a semi-transparent overlay window 5383 comprising a bitmap bmp1 (or mask) with a plurality of semi-transparent pixels, and having a transparency level (T) ranging from 5% to 45% (or an alpha - transparency value (α) in the range from 55% to 95%);
b) configuring the overlay window 5402 in click-through mode to allow keyboard input and mouse input to be sent to the underlying window instead of being interpreted by the overlay window;
Repeatedly perform the following steps:
c) taking a screenshot 5403, thereby obtaining a second bitmap bmp2;
d) updating the pixel values of the first bitmap bmp1 (or mask) such that the color value of the pixels underlying the overlay window are essentially inverted when they are alpha mixed by the graphic overlay.
The first bitmap bmp1 can occupy all or part of the window 5383, for example as defined by the screen edges and / or one or two
BE2018 / 5637 horizontal line segments and / or one or two vertical line segments. Preferably, the first bitmap bmp1 occupies at least 20% or at least 40% or at least 60% or at least 80% or at least 90% of the size of the overlay window 5383. Preferably at least 90% of all pixels of the first bitmap are bmp1 semi -transparent (ie not fully transparent).
FIG. 55 shows an exemplary user interface window as can be used by the application running on the computer system of FIG. 52 (a) and / or FIG. 52 (b). In the specific example shown in FIG. 56, the user can activate color inversion by checking the checkbox 5601, and the user can choose which part of the screen to be color-inverted, for example when clicking button 5502 the first bitmap bmp1 is configured to the left half of the screen, when button 5503 is clicked, the first bitmap bmp1 is configured to occupy essentially the entire screen (preferably without the windows taskbar area), and when button 5504 is clicked, the first bitmap bmp1 configured to occupy the right half of the screen.
The other six buttons (including buttons 5505, 5506, 5507) show other areas of the screen that need to be overlaid with a reduced height. After one of these predetermined bitmaps bmp1 is displayed, its size and / or position can be changed by dragging one or more of the horizontal and / or vertical line segments, such as shown in FIG. 52 (a) and FIG. 52 (b).
FIG. 56 and FIG. 57 show an exemplary image to show the surprisingly good quality of the color inversion available through the overlay application. The images show part of the well-known Paint application, containing a color-gradient image. FIG. 56 shows the original image (without color inversion), FIG. 57 shows this image after color inversion by an overlay application according to an embodiment of the present invention. The value of T is equal to 32/256 or 1/8 in this example.
FIG. 58 and FIG. 59 show another exemplary image to show the surprisingly good quality of the color inversion available through the overlay application. The images show part of an image taken from a web browser showing dummy text known as Loren ipsum. FIG. 58 shows the original image (without color inversion), FIG. 59 shows this image after color inversion by an overlay application according to an embodiment of the present invention. The value of T is equal to 32/256 or 1/8 in this example.
BE2018 / 5637
AND LAST BUT NOT LEAST,
While individual features have been illustrated in various drawings and in various embodiments of the present invention, it is contemplated that features of different embodiments may be combined, as would be obvious to those skilled in the art, upon reading this document.
Although all embodiments are shown with a desktop computer and a mouse and a keyboard, other calculators and / or other input devices can also be used. For example, the mouse device can be replaced by other devices that provide the same or similar function and control, such as a pointing device, a track ball or a joystick or a touchpad or a stylus.
Although the invention has been primarily described for the Windows operating system of Microsoft Corporation, the invention is not limited thereto, and may also be used, for example, on an Apple computer with the Macintosh OS of Apple computer, or on a smartphone, or on a eReader, or other operating system devices with a graphical user interface.
The present invention discloses several embodiments, which can be enumerated as follows:
E1) a computer-implemented method of overlay with a perforated bitmap.
This can be summarized as follows. A computer-implemented method of overlaying a graphic image in a computing device, comprising the steps of: a) providing an overlay window; b) providing at least one visible object in said overlay window, the at least one visible object comprising a bitmap with a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are opaque pixels or semi-transparent pixels ; wherein the first plurality of pixels P6 and the second plurality of pixels are interleaved (alternately positioned), for example in a checkerboard pattern; c) configuring the overlay window in click-through mode. The object is movable in accordance with mouse movements.
E2) a computer-implemented method of large cross overlay.
This can be summarized as follows. A computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing an overlay window; b) providing an elongated horizontally visible object; c) providing an elongated vertically visible object in said overlay window; d) configuring the overlay window in click-through mode; repeatedly f) obtaining position information X, Y from a mouse pointer or obtaining motion information dx, dy from the at least one pointing device; repeatedly g) adjusting a position of the first and second
BE2018 / 5637 visible object based on the obtained position information or using the motion information. The cross can be semi-transparent, and can extend over the entire height and width of the screen. Use of this overlay application as an add-on (extension) to spreadsheet applications for highlighting alphanumeric information and / or extracting information from tables.
E3) a computer-implemented method of overlaying with a texture bitmap.
E4) a computer-implemented method of overlaying a vertical line to split the screen, and a horizontal line that moves or freezes, also referred to herein as a frozen line.
This can be summarized as follows. A computer-implemented method of overlaying a graphic image, comprising: a) providing an overlay window; b) providing a vertical line; c) providing a horizontal line located on the left or right side of the vertical line; repeatedly: d) obtaining position information from a mouse cursor; e) testing whether the mouse cursor is on the vertical line, and if so, proceed to step f), otherwise proceed to step h); f) configuring the overlay window in non-click-through mode; g) testing whether a mouse button has been pressed, and if so, dragging the vertical line; and go to step d; h) configuring the overlay window in click-through mode; i) adjusting a position of the horizontal line. The vertical line splits the screen, and can be dragged. The horizontal line moves or freezes depending on which side the mouse cursor is moving.
E7) a computer-implemented method of dynamically overlaying, taking into account visual aspects (e.g., color and / or brightness) of the underlying applications, referred to herein as automatic adjustment, e.g., auto darkening.
This can be summarized as follows. A computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing a semi-transparent overlay window with an alpha transparency value α in the range of 1% to 99% and comprising a first bitmap; repeatedly performing the following steps: b) taking a screenshot, thereby obtaining a second bitmap; c) calculating at least one characteristic of the second bitmap; e) adjusting the alpha transparency value of the overlay window and / or adjusting one or more pixel values of the first bitmap based on the at least one particular feature. The overlay method may further include a vertical bar to define different areas that can be individually adjusted depending on the image content below the areas.
BE2018 / 5637
E6) allows a computer-implemented method of overlay while taking a screenshot, which screenshot is compensated for the graphic overlay, herein referred to as a compensated screenshot.
This can be summarized as follows. A computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing a semi-transparent overlay window having an alpha transparency value α in the range of 1% to 99%, and which includes a first bitmap;
b) taking a screenshot, thereby obtaining a second bitmap; c) calculating a third bitmap to compensate for the effect of the graphic overlay based on the first bitmap and the second bitmap and the alpha transparency value α; d) optionally storing the third bitmap in a non-volatile memory or storage device; e) optionally copying the third bitmap to an operating system clipboard.
E7) a computer-implemented method of overlay to substantially invert the colors of an underlying image.
This can be summarized as follows. A computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing a semi-transparent overlay window comprising a first bitmap with a plurality of semi-transparent pixels and a transparency level ranges from 5% to 45%, b) configuring the overlay window in click-through mode; Repeatedly performing the following steps: c) taking a screenshot, thereby obtaining a second bitmap; d) updating the pixel values of the first bitmap such that the color values of the pixels of the bitmap located below the overlay window are substantially inverted. A computer program product for performing this method. A computer device that includes computer-executable instructions for performing this method. A computer system that includes such a computer device.
E8) a computer device for performing one of the computer-implemented methods
E1) to E7).
E9) a computer system comprising a computer device E8).
E10) a computer program product for performing one of the computer-implemented methods E1) to E7).
E11) a portable device (e.g. eReader) with an overlay with a semi-transparent line.
This can be summarized as follows. A portable calculator comprising: a touch screen; at least one processing unit and a first memory for storing computer executable instructions; where the instructions are configured for the
BE2018 / 5637 generating a graphic image containing textual information and for displaying that graphic image on the touch screen; wherein the instructions are further configured to generate a line or an elongated object covering said textual information; wherein the line or the oblong object comprises a plurality of semi-transparent pixels with a transparency level of 5% to 95%, or wherein the line or the oblong object contains a first plurality of fully transparent pixels and a second plurality of opaque or semi-transparent pixels , which are interleaved (alternately positioned); wherein the instructions are further configured to detect a contact position on the touch screen, and to adjust a position of the line accordingly. For example, an eReader device that displays textual information overlaid by a semi-transparent line.
E12) a display device for overlaying with a perforated bitmap,
E13) a display device with a movable object that looks semi-transparent through time-multiplexed overlay.
It is further explicitly pointed out that embodiments of the present invention can be combined in any suitable manner, for example:
- embodiment E1 perforated bitmap can be combined with embodiment
Darken E5 automatically or with E6 offset screenshot;
- embodiment E2 large cross can be combined with embodiment E5 automatically darkening or with embodiment E6 compensated screenshot;
- embodiment E3 texture bitmap can be combined with embodiment E5 darkening automatically or with embodiment E6 compensated screenshot;
embodiment E1 perforated bitmap or embodiment E3 texture bitmap can be combined with embodiment E7 inverted colors, for example for different parts of the screen;
to name just a few.
It is further explicitly pointed out that embodiments of the present invention can be combined in any suitable manner with embodiments of the co-pending double mouse application.
For example, by adding to the embodiments of the present invention a second pointing device and the associated functionality for obtaining double mouse support, as described in co-pending application. Or, for example, by adding a special hub and / or a special display device and / or a special video adapter to embodiments of the present invention.
BE2018 / 5637
Some of these combined embodiments may be particularly suitable for translators, or proofreaders, or technical writers, or patent attorneys or other professional users who spend many hours in front of a computer screen and often switch back and forth between different documents, because the cognitive load is related to regain the current context after switching back and forth between documents can be significantly reduced thanks to the multiple visible objects movable by the two (or more) pointing devices, and because eye fatigue related to a bright backlight can also be reduced.

权利要求:
Claims (12)
[1]
A computer-implemented method (5400) of overlaying a graphic image (bmp_orig) in a calculator (5201), the method comprising the steps of:
a) providing (5401) a semi-transparent overlay window (5383) that includes a first bitmap (bmp_mask) with a plurality of semi-transparent pixels and that has a transparency level (T) ranging from 5% to 45%,
b) configuring (5402) the overlay window (5383) in click-through mode;
Repeatedly perform the following steps:
c) taking (5403) a screenshot, thereby obtaining a second bitmap (bmp_mix);
d) updating the pixel values of at least a portion of the first bitmap (bmp_mask) such that color values of the pixels of the graphic image (bmp_orig) located below the overlay window (5383) are essentially inverted, including updating :
i) estimating the pixel values of the underlying bitmap (bmp_orig) based on the pixel values of the second bitmap (bmp_mix) of the screenshot, and based on the pixel values of the first bitmap (bmp_mask) of the overlay window, and based on of the following set of formulas or an equivalent set of formulas:
Runmix = [Rmix - Rmask * (1-T)] / T, limited from 0 to 255 or part range thereof;
- Gunmix = [Gmix - Gmask * (1-T)] / T, limited from 0 to 255 or a part range thereof;
Bunmix = [Bmix - Bmask * (1-T)] / T, limited from 0 to 255 or a part range thereof;
where (Runmix, Gunmix, Bunmix) is the estimated Red, Green and Blue value of the pixels of the underlying bitmap, and (Rmask, Gmask, Bmask) the Red, Green, and
Blue value of the pixels of the first bitmap applied when taking the screenshot, where T is the transparency level;
ii) inverting the estimated pixel values, thereby obtaining target pixel values;
iii) adjusting the pixel values of the first bitmap (bmp_mask) of the overlay window based on at least the target pixel values.
[2]
A computer implemented method according to claim 1, wherein the transparency level (T) is selected from the group consisting of 4/256, 8/256, 16/256, 32/256 and 64/256; or wherein the alpha transparency level (α) is selected from the group consisting of 252/256, 248/256, 240/256, 224/256, and 192/256.
BE2018 / 5637
[3]
A computer-implemented method according to claim 1 or 2, wherein step ii) determining the target pixel values using three monotonically decreasing functions providing values in the range 0 to 255 or a subset thereof, for arguments in the range from 0 to 255.
[4]
A computer-implemented method (5400) according to any one of the preceding claims, wherein step iii) comprises:
iii) calculating the pixel values of the first bitmap (bmp_mask) of the overlay window as a function of only the target pixel values.
[5]
5. A computer-implemented method (5400) according to claim 4, wherein step iii) includes: iii) calculating the pixel values of the first bitmap (bmp_mask) based on the
following set of formulas, or an equivalent set of formulas:
Rmask: = (A - B * Runmix), limited to the range from 0 to 255 or part range thereof;Gmask: = (C - D * Gunmix), limited to the range from 0 to 255 or part range thereof;Bmask: = (E - F * Bunmix), limited to the range from 0 to 255 or a subrange thereof;
where (Rmask, Gmask, Bmask) are the Red, Green and Blue color component of the pixels of the first bitmap (bmp_mask), and where A, B, C, D, E, F are predetermined constants in the range of 0, 20 to 2.0.
[6]
A computer-implemented method (5400) according to claim 5, wherein A, C and E are integers in the range from 190 to 500 or in the range from 190 to 400 or in the range from 190 to 300 or in the range from 190 to 256;
and wherein each of the numbers B, D and F are values in the range of 0.20 to 2.0.
[7]
A computer implemented method (5400) according to claim 5 or 6, wherein each of the values B, D and F is selected from the group consisting of 1/2, 3/4, 5/8, 6/8, 7 / 8, 9/16, 10/16, 11/16, 12/16, 13/16, 14/16, 15/16, and 1.
[8]
A computer-implemented method (5400) according to any one of claims 1 to 3, wherein step iii) comprises:
iii) calculating the pixel values of the first bitmap (bmp_mask) of the overlay window as a function of the target pixel values and the pixel values of the second bitmap (bmp_mix) of the screenshot, or as a function of the target pixel values and estimated pixel values (Runmix , Gunmix, Bunmix) of the underlying bitmap (bmp_orig).
BE2018 / 5637
[9]
A computer-implemented method according to any preceding claim, wherein the first bitmap (bmp_mask) occupies more than 80% of the area of the overlay window (5383).
[10]
A computer-implemented method according to any one of claims 1 to 8, wherein the first bitmap (bmp_mask) is less than 95%, or less than 90%, or less than
Occupies 80%, or less than 60%, or less than 45%, or less than 40% of the area of the overlay window (5383).
[11]
A computer implemented method according to claim 10, wherein the overlay window (5383) further comprises a third bitmap (bmp3) consisting of a first plurality of fully transparent pixels and a second plurality of semi-transparent pixels interleaved.
A computer device (5201), comprising:
- at least one central processing unit (CPU), and a first memory connected to the at least one central processing unit (CPU) with computer executable instructions stored therein;
wherein the computer executable instructions comprise code snippets for performing an overlay method according to any one of claims 1 to 11.
A computer system (5200) comprising:
- a computer device (5201) according to claim 12;
at least one display device (5204) connected to an output of the computer device (5201), for displaying the graphic image mixed with the overlay image.
A computer program product for providing a graphic overlay, the computer program product containing executable instructions which when executed on at least one central processing unit (CPU) of a computer device (5201) according to claim
[12]
12, or a computer system (5200) according to claim 13, causing the computer device to perform a method according to any one of claims 1 to 11.

类似技术:

---

公开号 | 公开日 | 专利标题

---

CN105705991B|2020-06-09|Display system and method

---

CN105009057B|2020-09-08|Multi-window smart window placement using high DPI screens

---

US20210026508A1|2021-01-28|Method, device and computer program for overlaying a graphical image

---

BE1026516B1|2020-03-09|METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

---

US20150097879A1|2015-04-09|Electronic display

---

WO2019063495A2|2019-04-04|Method, device and computer program for overlaying a graphical image

---

BE1025952B1|2019-08-27|METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

---

BE1025598B1|2019-04-29|METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

---

NL2023600B1|2020-02-17|Method, device and computer program for overlaying a graphical image

---

CN104517302B|2019-07-12|The method for showing equipment and the font effects for providing the display equipment

同族专利:

---

公开号 | 公开日

---

BE1026516A1|2020-03-02|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

CA2124505C|1993-07-21|2000-01-04|William A. S. Buxton|User interface having simultaneously movable tools and cursor|

---

US6333753B1|1998-09-14|2001-12-25|Microsoft Corporation|Technique for implementing an on-demand display widget through controlled fading initiated by user contact with a touch sensitive input device|

---

法律状态:
2020-04-16| FG| Patent granted|Effective date: 20200309 |

优先权:

---

申请号 | 申请日 | 专利标题

---

EP18187882|2018-08-07|US16/650,048| US20210026508A1|2017-09-29|2018-09-24|Method, device and computer program for overlaying a graphical image|

---

PCT/EP2018/075836| WO2019063495A2|2017-09-29|2018-09-24|Method, device and computer program for overlaying a graphical image|

---

NL2023600A| NL2023600B1|2018-08-07|2019-08-01|Method, device and computer program for overlaying a graphical image|